ML Interview Questions

How Large Language Models (LLMs) handle out-of-vocabulary (OOV) words or tokens?

How do you reduce inference cost for LLMs?

Can you explain how Large Language Models (LLMs) are typically trained? What are the key components and phases involved in their training process?

Discuss the differences between encoder-only, decoder-only, and encoder-decoder transformer architectures, focusing on their specific characteristics and potential applications.

Explain the attention mechanism in transformers, focusing on self-attention and multi-head attention. Discuss their importance in the architecture and functioning of transformer models.

Explain how Retrieval-Augmented Generation (RAG) works and its advantages over traditional large language models (LLMs).

Discuss in-context learning within the framework of Large Language Models (LLMs). How does few-shot prompting facilitate model adaptation without updating model parameters? Provide examples of practical applications and challenges associated with this approach.

What are the ethical considerations when deploying large language models (LLMs), specifically focusing on issues such as bias, misinformation, and copyright concerns?

Describe the process and components of Reinforcement Learning from Human Feedback (RLHF) in the context of training large language models (LLMs). Discuss how RLHF incorporates key elements such as reward model training and proximal policy optimization (PPO). Furthermore, explore the challenges faced in aligning LLMs with human preferences using RLHF, and evaluate the limitations of this approach. What are some alternative methods being explored for improving alignment in LLMs?

Explain how Reinforcement Learning from Human Feedback (RLHF) is employed to align Large Language Models (LLMs) with human values and intentions.