AI Training Papers: A Reading Guide

THE paper. Every modern LLM (GPT, Llama, Claude, Gemini) is built on the architecture introduced here.

Replaces sequential (recurrent) processing with self-attention — a mechanism where every token can directly look at every other token to build context, like a conference room where everyone hears everyone at once. Adds multi-head attention (parallel attention patterns) and positional encodings (where each token sits).

Key takeaway: Self-attention replaced sequential processing with parallel processing, making large-scale language models possible.

Turned “how big should our model be?” from guesswork into science — performance follows predictable power laws in size, data, and compute.

Training hundreds of models revealed smooth, predictable curves: 10× the model → a predictable performance gain, same for data and compute. The paper argued model size matters more than training time — i.e., “go big on parameters.”

Key takeaway: Model performance is predictable from size/data/compute. This paper said go big on parameters — the next one overturned that.

Overturned the previous scaling advice: most large models were undertrained — too many parameters, not enough data.

For a fixed compute budget you should scale model size AND data together. A 70B model on 1.4T tokens beats a 280B model on 300B tokens at equal compute. Rule of thumb: ~20 tokens per parameter.

Key takeaway: Don’t just make models bigger — train them on proportionally more data. “20 tokens per parameter” became the industry guideline.

What happens when you run out of high-quality data — increasingly relevant as models exhaust the internet's text.

Repeating data helps but decays after ~4 epochs; when data is limited, a slightly larger model for fewer steps beats a smaller model for more; and quality beats quantity. Provides modified scaling laws for the data-limited regime.

Key takeaway: When data is limited, invest in quality over quantity, and don’t repeat data more than ~4×.

First major paper on splitting a model into stages across devices while keeping them all busy.

Like an assembly line: GPU 1 handles layers 1-10, GPU 2 handles 11-20, etc. The catch is the “pipeline bubble” (idle GPUs); GPipe fixes it by splitting each batch into micro-batches so GPUs pipeline the work.

Key takeaway: Pipeline parallelism splits layers across devices and uses micro-batching to keep them busy — one of the three core parallelism strategies.

Shows how to split individual layers across GPUs, complementing pipeline parallelism.

Where GPipe splits the model vertically (layers), Megatron splits it horizontally — a single layer's matrix multiplies are divided across GPUs, each handling a slice, then combined. Efficient patterns for both attention and feed-forward blocks.

Key takeaway: Tensor parallelism splits individual operations across GPUs. Combined with pipeline + data parallelism you get “3D parallelism.”

Dramatically reduces per-GPU memory, enabling much larger models without model parallelism. Foundation for FSDP.

In plain data parallelism every GPU holds a full, redundant copy of model + optimizer states + gradients. ZeRO shards these across GPUs in three stages (optimizer states → gradients → parameters), gathering on demand and trading communication for memory.

Key takeaway: ZeRO eliminates redundant memory across GPUs — each stores a shard, they share on demand.

PyTorch's production implementation of ZeRO-3 — how Meta actually trains Llama and other large models.

Each GPU holds only a shard of parameters; the full set is gathered before a layer's forward/backward pass, then released. The blog covers the practical engineering: module wrapping, communication, mixed precision, checkpointing.

Key takeaway: FSDP is the production-ready version of ZeRO — how much of the industry trains large models in PyTorch.

A recent three-layer distributed-training architecture with autotuning.

Organizes training into a resource-management layer, a training-orchestration layer, and an autotuning layer that automatically searches for good configs (batch size, parallelism degree, communication strategy) instead of hand-tuning hundreds of knobs.

Key takeaway: Autotuning distributed-training configs can improve GPU utilization without deep systems expertise per run.

Made long-context training practical — 2-4× faster attention with far less memory.

Standard attention materializes an n×n matrix in slow GPU memory (HBM). FlashAttention is IO-aware: it tiles the computation to fit fast on-chip SRAM and never writes the full matrix, dropping memory from O(n²) to O(n) with identical math.

Key takeaway: Same math, less data movement. Optimizing memory access — not arithmetic — enabled longer context windows.

A further ~2× speedup through better GPU utilization.

Reduces non-matmul work (GPUs are optimized for matrix multiplies), parallelizes across the sequence dimension, and improves work partitioning within thread blocks — reaching ~70% of peak FLOPS vs. ~35% for the original on a 2K sequence.

Key takeaway: Same idea as FlashAttention, better GPU utilization — an engineering refinement that doubled speed again.

Made fine-tuning large models accessible — ~10,000× fewer trainable parameters at comparable quality.

Weight changes during fine-tuning are low-rank, so instead of updating a full 4096×4096 matrix you train two tiny ones (4096×8 and 8×4096). Base weights stay frozen; adapters (~10MB) merge back at inference with zero overhead and hot-swap onto one base model.

Key takeaway: LoRA makes fine-tuning cheap and modular — train tiny adapter matrices instead of the whole model.

Fine-tune a 4-bit-quantized base model with LoRA adapters — fits large models on a single GPU.

Freezes the base model in 4-bit NF4 precision and trains LoRA adapters on top, with paged optimizers to survive memory spikes. Delivers near-full-precision quality at a fraction of the memory.

Key takeaway: Compress the frozen base model, train small adapters — the recipe that democratized fine-tuning.

Pushes quantization to the extreme — a ~2-bit (down to 1.15-bit) base model with LoRA adapters.

At 2 bits, naive LoRA breaks down from quantization error. LowRA optimizes fine-grained quantization (mapping, thresholds, precision assignment) with efficient CUDA kernels, staying accurate down to ~1.15 bits and cutting memory up to 50%.

Key takeaway: The frontier of the accuracy/efficiency trade-off — useful for understanding where compression limits are.

How do you add new knowledge without the model forgetting what it already knows?

Fine-tuning on new data causes “catastrophic forgetting.” The survey covers the four families of fixes: replay (mix old data in), regularization (penalize big weight changes), architecture (dedicated adapters), and distillation (old model as teacher).

Key takeaway: Updating models without losing capabilities is hard — replay, regularization, adapters, and distillation manage the trade-off.

The RL algorithm behind RLHF — you need it to understand how models are aligned.

PPO improves a policy without changing it too drastically in one step (big steps cause instability) by “clipping” updates that stray too far from the old policy. In RLHF the policy is the LLM, the action is emitting a token, and the reward comes from a human-preference reward model.

Key takeaway: PPO = a stable RL algorithm that makes small, safe updates. The engine of RLHF.

The proof-of-concept for RLHF — human preferences beat supervised learning on subjective quality.

The full RLHF loop on summarization: generate two summaries, a human picks the better, train a reward model to predict that preference, then PPO-optimize the model against it. Result beat both the supervised baseline and human reference summaries.

Key takeaway: RLHF works — and beats supervised learning where quality is subjective. This pipeline became the template.

THE alignment paper — the pipeline that turned GPT-3 into an instruction-follower (ChatGPT's predecessor).

Three steps: supervised fine-tuning on human demonstrations → a reward model trained on human rankings → PPO against that reward model. A 1.3B InstructGPT was preferred over the 175B GPT-3 base, and alignment transferred across languages.

Key takeaway: SFT → Reward Model → PPO is how raw models become helpful assistants. A small aligned model beats a big unaligned one.

The most detailed reference implementation of RLHF; introduces the helpfulness-vs-harmlessness tension.

A practical deep-dive: helpfulness and harmlessness pull against each other (too cautious refuses too much; too helpful says unsafe things); you can train separate reward models and combine them; and preference-data quality dominates outcomes.

Key takeaway: RLHF is as much about data quality and evaluation as algorithms — helpfulness and safety need careful balancing.

Reduces reliance on expensive human feedback by having the model critique itself against written principles.

Write a “constitution” (e.g. “be honest,” “don't help with illegal activity”), then have the model generate → critique its own output against the principles → revise, and use the revisions as preference data for RLHF. Cheaper, faster, and auditable.

Key takeaway: AI feedback guided by explicit principles can replace much human feedback — cheaper and more transparent.

Simplifies RLHF dramatically — no reward model, no PPO, no RL loop; just a supervised loss.

DPO shows the optimal RLHF policy can be learned directly from preference pairs with a simple supervised loss: push up preferred responses, push down rejected ones, with a KL term to stay near the base model. Simpler, more stable, comparable results.

Key takeaway: DPO removes RLHF's moving parts while matching its results — the preferred alignment method for many teams.

A retrospective on what actually works — and breaks — in RLHF.

Reward-model quality is the bottleneck (a wrong reward model gets “reward-hacked”); the KL penalty is crucial but finicky; distribution shift between reward-model data and policy outputs persists; and alignment evaluation is itself unsolved.

Key takeaway: RLHF works but is fragile — reward hacking, distribution shift, and evaluation gaps are real.

Addresses practical stability issues in RLHF training.

RLHF is sensitive to noisy preferences (labelers disagree ~30% of the time), reward-model imperfection, and distribution shift. Proposes noise-tolerant reward modeling, more conservative policy updates, and better out-of-distribution handling.

Key takeaway: Production-reliable RLHF requires explicit handling of noise, disagreement, and distribution shift.

Meta's first major open LLM with detailed training documentation — a full, transparent recipe.

Covers 2T-token pre-training, 7B/13B/70B sizes, FSDP, 27K high-quality SFT demonstrations, and RLHF with 1M+ preference annotations. Key findings: a little high-quality SFT data beats a lot of mediocre data, and rejection sampling pairs well with PPO.

Key takeaway: A practical end-to-end recipe — quality over quantity for SFT, and rejection sampling is surprisingly effective.

Meta's most detailed report — full stack from data curation to post-training.

15T+ tokens, context up to 128K, multimodal. Emphasizes data quality (dedup, filtering, synthetic data) and an iterative post-training loop: SFT → DPO → rejection sampling → repeat. Covers tool use, safety, and deployment.

Key takeaway: Data quality and iterative post-training are the biggest levers — the modern pipeline in one report.

GPT-4-class performance at a fraction of the reported training cost via MoE + engineering efficiency.

A 671B-parameter Mixture-of-Experts model that activates only ~37B per token, trained with FP8 mixed precision, fine-grained expert routing, and multi-stage post-training. Architectural innovation, not just scale, drives the cost win.

Key takeaway: MoE can deliver competitive performance at much lower cost — a reference for efficient large-model training.

The foundational MoE paper — the core ideas now in DeepSeek, Mixtral, and others.

Many “expert” networks plus a gating network that routes each input to a few (typically 1-2). Enormous total capacity, fractional active compute. Introduces noisy top-k gating and load-balancing losses to handle the hard parts.

Key takeaway: MoE = many specialists + a router. More capacity without proportional compute — the basis of modern efficient models.

MoE applied outside language — scientific computing for partial differential equations (NeurIPS 2025).

Routes different PDE types (heat, wave, fluid) to specialized experts (4 of 16 routed + 2 shared per layer), so one pre-trained model handles heterogeneous equations and fine-tunes per family. ~40% lower zero-shot error at fewer active parameters.

Key takeaway: MoE is a general architectural principle, not just for language — capacity scaling plus specialization.

Understanding training-data composition is critical — “data is the new oil” for LLMs.

825GB from 22 diverse sources (papers, code, books, web, legal, patents). The key result: source diversity beats scaling up a single source like Common Crawl. Documents construction, deduplication, and per-source filtering.

Key takeaway: Training data should be diverse across domains — composition directly shapes what a model learns.

A broad survey of the full landscape — good for filling gaps.

Connects deep-learning fundamentals (layers, training loops, loss functions), the RNN → Transformer → LLM evolution, and AutoML (architecture search, HPO, feature engineering) into one narrative. Broad rather than deep.

Key takeaway: A good “zoom out” if you want to see how the individual techniques fit together.

Covers the optimizer landscape (SGD, Adam, AdamW) that underpins all training.

From SGD to adaptive methods like Adam (per-parameter step sizes from gradient history) and AdamW (fixes Adam's weight-decay bug), plus learning-rate schedules and warmup. The optimizer/LR choice can decide whether training converges or diverges.

Key takeaway: Optimizers update the weights. Adam/AdamW is standard, but choice and tuning materially affect training success.

Introduced Multi-head Latent Attention (MLA), which slashes the KV-cache bottleneck — still used downstream.

MLA compresses the key/value matrices into a small latent space during pre-training and decompresses to compute attention, massively reducing the KV cache in large autoregressive models — a core efficiency primitive reused in later architectures.

Key takeaway: MLA attacks the KV-cache bottleneck — a foundational efficiency idea for long-context inference.

Kimi Delta Attention (KDA) — a linear-attention design that beats full attention across short, long, and RL regimes.

A layerwise hybrid pairing three KDA layers with one MLA layer. KDA extends Gated DeltaNet with fine-grained channel-wise gating, selectively “forgetting” irrelevant memory for fast decoding and ~75% less KV-cache memory at 1M context.

Key takeaway: Fine-grained gated linear attention can outperform full attention while slashing memory — a live frontier direction.

~3-bit KV cache (6× less memory, faster attention) via vector quantization — the KV-efficiency frontier.

Two stages: PolarQuant rotates each key/value vector so its coordinates look Gaussian and quantize cleanly, then a 1-bit Quantized Johnson-Lindenstrauss (QJL) transform corrects the residual error. Big memory wins at near-zero accuracy loss.

Key takeaway: Quantization + a JL error-corrector compresses the KV cache hard — part of the multi-year push on KV efficiency.

A million-token-context MoE that stays cheap — the payoff of the whole KV-efficiency line of work.

Combines Compressed Sparse Attention (CSA, ~4× KV compression + top-k block selection) with Heavily Compressed Attention (HCA). At 1M tokens, V4-Pro needs ~27% of the per-token FLOPs and ~10% of the KV cache of DeepSeek-V3.2.

Key takeaway: Hybrid compressed-attention makes million-token context practical by trading granular detail for scale.