LLM Theory Round: Transformers, Sampling, Fine-Tuning and RL

Problem Overview

A spoken LLM theory round, 60 minutes in one onsite report and 30 minutes as the first of two back-to-back phone-screen rounds in another. It covers a broad set of LLM topics. Topics named across reports: (1) transformer architecture, including the encoder, (2) attention, (3) causal (autoregressive) decoding, (4) sampling,…

• 6 candidate-reported follow-ups — the exact probes interviewers asked, with the trigger for each
• The rest of the problem statement — full requirements, constraints, and edge cases
• Approach and trade-offs — what passing candidates did, and the mistakes that sink people