The Full Landscape of LLM Training: What Every AI Application Engineer Should Understand

You do not need to train a model yourself, but you do need to understand how models are trained. This article is my complete set of notes after systematically studying LLM training as an engineer transitioning from Java backend development to AI application engineering. It is not a survey paper written for researchers; it is written for people like me who need to make engineering judgments at the application layer.

Why Application Engineers Need to Understand Training

When developing AI applications, you make decisions about model selection, prompt tuning, agent orchestration, and evaluation-system design every day. The quality of those decisions depends on how deeply you understand where a model comes from.

Once you know how a model was trained, you can understand why it excels at some tasks but struggles with others; why a high benchmark score does not guarantee strong performance in your specific scenario; why the same base model can perform very differently across products; and what “stronger” actually means when a new model is released.

The answers to all of these questions are hidden in design decisions made during training.

Chapter 1: Data Is Not Fuel; It Is the Blueprint for Capability

The Data Pipeline: Six Filtering Layers Set the Capability Ceiling

The first step in training an LLM is not writing training code; it is preparing data. Raw data comes from sources such as web crawls, code repositories, books, forums, documents, and synthetic data. But raw data cannot be used for training directly. It must pass through a complete funnel-shaped processing pipeline.

The first layer is text extraction, which extracts plain text from raw HTML while removing templated content such as navigation bars, footers, advertising code, and cookie pop-ups. The quality of the extractor directly determines the input quality of every subsequent step.

The second layer is language identification (Language ID), which filters out content outside the target languages. This step directly determines the distribution of the model’s multilingual capabilities: if you retain only English, you get an English model; if Chinese accounts for only 5%, the model’s Chinese capability will be weaker than its English capability.

The third layer is quality filtering. A classifier is commonly trained with Wikipedia and high-quality books as positive examples and random web pages as negative examples. This means the data is being selected according to one particular definition of “quality”: academic and formal content is more likely to pass, while conversational or non-mainstream content is more likely to be removed. When a model performs poorly in certain scenarios, the root cause may lie here.

The fourth layer is PII redaction, which removes personal information such as email addresses, phone numbers, and identity numbers. This is required for legal compliance, but excessive cleaning may weaken the model when it needs to understand such formats.

The fifth layer is safety filtering, which removes hate speech, violence, pornography, and similar material, directly shaping the baseline of the model’s safety behavior.

The sixth layer is deduplication, including near-duplicate removal and benchmark-leakage cleanup. A vast amount of online content is copied from elsewhere. If the same paragraph appears 1,000 times in the training data, the model will over-memorize it while reducing the effective share of learning devoted to other content. Benchmark-leakage cleanup prevents answers to test questions from appearing in the training data and artificially inflating evaluation scores.

The central insight of this pipeline is: Data pipeline changes the capability distribution before training starts. The data pipeline has already determined the distribution of model capabilities before training begins. Whatever you discard is something the model can never learn.

Data Ratios: Mixture Design Is the Most Critical Design Decision

After six filtering layers, the surviving data is divided by source into several “buckets”: web text, code, books, forums, documents, and synthetic data. The proportion assigned to each bucket during training is the mixture design.

These proportions directly determine the model’s capability emphasis. A high share of code strengthens programming ability but may weaken conversation; a high share of books improves language ability but may leave the model short on current knowledge. The effects are not simply additive. Training on more code may also improve logical reasoning because code itself is structured reasoning, but too much code may damage natural-language fluency.

Research on data mixing laws is trying to identify mathematically optimal mixture ratios. Llama 3 used many experiments with smaller models to find the best proportions before applying them to large-model training. That process alone represents an enormous engineering investment.

Why Deduplication and Contamination Control Matter

Poor deduplication causes the model to repeatedly learn content that has been copied and pasted across the internet—for example, the MIT License appearing millions of times on GitHub, navigation templates, and cookie notices. Their weight in the training data becomes artificially amplified, crowding out the share of learning devoted to genuinely valuable content.

The consequence of benchmark leakage is more subtle. If MMLU or HumanEval test questions appear in the training data, the model’s benchmark score is inflated: it has not acquired the capability; it has “memorized” the answer. This is why many open-source models look excellent on benchmarks yet perform inconsistently in actual use.

Implication for application engineers: You cannot select a model by benchmark scores alone. Once you understand the data pipeline, you know that a score may be affected by deduplication quality, data leakage, mixture preferences, and many other factors. The more reliable method is to evaluate models on your specific tasks, which is also the central idea of Evaluation-Driven Development (EDD).

Chapter 2: Scaling Laws and Over-Training—Trading Training Compute for Deployment Cost

The Chinchilla Law: The “Theoretical Optimum” for Training

Training a model consumes compute, measured in FLOPs. That compute is determined by two factors: model parameter count × training data volume. It is analogous to total labor hours = number of workers × hours worked per person.

DeepMind’s 2022 Chinchilla paper gave a classic result: for a fixed compute budget, parameter count and data volume should be allocated at roughly a 1:20 ratio to minimize loss, or prediction error. For an 8B-parameter model, the Chinchilla-optimal data volume is about 200B tokens.

Over-Training: Why Practice Deviates from the Theoretical Optimum

Meta trained Llama 3 8B on 15T tokens—75 times the Chinchilla-optimal amount. From the perspective of pure training efficiency, this “wastes” compute: spending the same FLOPs on a larger model could produce lower loss.

From the deployment perspective, however, it is a smarter decision. Training is a one-time cost; inference is continuous. For a model deployed at scale, spending ten times more compute during training might cost several million dollars, but halving the model size halves the cost of every inference and greatly lowers the deployment barrier. The accumulated long-term savings far exceed the extra training expense.

How large is 15T tokens? It is roughly equivalent to 150 million books—close to the total number of books ever published in human history. A person reading continuously without eating or sleeping would need 250,000 years to finish them.

One important rule of thumb supports the over-training strategy: Total FLOPs are the best single predictor of model quality. Whether the compute is spent on a larger model or more data, total FLOPs are what matter most. Over-training therefore lets you enjoy the deployment advantages of a small model without losing as much training quality as one might expect.

Single Accelerator: Deployment Constraints Shape Architecture Upstream

For Gemma 3, Google emphasized “single accelerator”: the entire model can be loaded and run for inference on one GPU or TPU without multi-device parallelism. This is not an incidental result discovered after the fact. It is an up-front design constraint that limits architectural choices such as maximum parameter count, attention-head count, and hidden dimension. Choosing single-accelerator deployment as the goal means giving up the possibility of building a larger model in exchange for a much lower deployment barrier.

Implication for application engineers: Understanding over-training and deployment constraints helps you make better model choices. When you need low latency and low cost, an over-trained small model such as Llama 3 8B or Gemma 3 27B may be more suitable than calling a large-model API directly. What you need to evaluate is whether that small model is good enough for your specific task, not where it ranks on general-purpose benchmarks.

Chapter 3: System Constraints—Engineering Decisions Locked In Before Training

Training Is Fundamentally a Distributed-Systems Problem

Many people think of training as a research problem: how to define the objective, reduce the loss, or change the model architecture. In real LLM training, however, system constraints are central. GPU count, memory bandwidth, parallelization strategy, fault tolerance, and cost cannot be optimized only after training. They determine from the outset how large a model you can train, how much context it can support, and whether more complex post-training is feasible.

It is like the difference between a single-node Spring Boot application and a distributed system deployed across hundreds of machines. The business logic may be similar, but the core challenges change completely to distributed consistency, network partitions, load balancing, and failure recovery.

Fixed Compute Budget: A Zero-Sum Trade-off Across Four Dimensions

With a fixed compute budget, you can spend it in four directions, but those directions compete with one another.

Move upward—a larger model with more parameters. The cost is higher GPU-memory requirements, multi-device parallelism, and greater communication overhead.

Move right—more training data. The cost is longer training time and a more expensive data pipeline. This is the direction chosen by Llama 3’s over-training strategy.

Move downward—a longer context window. The cost is higher attention compute per token. Standard attention has O(n²) computational complexity. Increasing sequence length from 4K to 128K multiplies attention compute by 1,024. This directly forces the batch size—the number of samples processed simultaneously—to shrink because memory consumption per sample increases dramatically. A smaller batch size reduces training efficiency and extends the total training time.

Move left—cheaper serving. The cost is stricter quantization constraints and potentially the need for quantization-aware training.

The core principle is: Every model capability is a budget decision. Meta chose “small model + enormous data,” Google chose “small model + single-device operation,” and DeepSeek chose “large total MoE parameters + low active-parameter cost.” These choices do not mean one company has worse technology; they reflect different trade-offs across these four dimensions.

MoE: An Engineering Compromise Between Cost and Performance

MoE, or Mixture of Experts, is an architecture that expands total parameter count at roughly similar compute cost. DeepSeek-V3, for example, has 671B total parameters but activates only 37B for each inference, with a router deciding which experts to activate. This gives the model greater “knowledge capacity” through more total parameters while keeping each inference close in cost to a much smaller dense model.

The price is a substantial increase in system complexity. Routing is itself an engineering challenge; load balancing requires additional loss terms to ensure that experts are used evenly; and the underlying communication pattern is more complicated.

Deep Networks and Information Dilution

Modern LLMs commonly have dozens or even more than a hundred layers—Llama 3 8B has 32 layers, while GPT-4-class models may have more than 100. More layers can express greater computational complexity, but they also introduce a central problem: information dilution.

After input information passes through dozens of transformations, the original signal is gradually diluted. The classic solution is the residual connection: each layer’s output = the layer’s computation + the layer’s input. This is effectively a “direct pipe” that lets information bypass certain layers and continue forward. Every modern Transformer architecture makes extensive use of residual connections.

Work such as Kimi’s Attention Residuals and the Forgetting Transformer builds further on this foundation, addressing information loss inside attention and information retention over very long sequences, respectively.

Training Instability: The Engineering Reality

After thousands of GPUs have trained for weeks, the loss may suddenly spike. Several days of progress can be lost, forcing a rollback to an older checkpoint and retraining from there. Those days of compute—potentially worth hundreds of thousands of dollars—are simply wasted.

There are also subtler problems: a single GPU may fail silently, producing incorrect gradients without reporting an error; NVLink bandwidth may degrade; or communication between nodes may fluctuate. Detecting, isolating, and recovering from these failures quickly during large-scale training is laboratory-grade engineering capability, not something that can be solved merely by reading papers.

DeepSeek-V3’s technical report specifically notes that the entire pretraining run had no irrecoverable loss spikes and required no rollback. It is also one of the few publicly documented cases demonstrating that FP8 mixed-precision training is feasible at extremely large model scale. The full run consumed approximately 2.788 million H800 GPU hours and trained on 14.8T tokens without instability. That “stable throughout” result is itself its most noteworthy engineering achievement.

Implication for application engineers: You do not need to master the engineering details of distributed training, but understanding these constraints helps you answer a critical question: “Why does this model have a 128K context window rather than 1M?” It is not an arbitrary number; it is the result of the entire set of trade-offs above. Understanding the difference between training concerns—throughput and cost—and inference concerns—latency and KV-cache management—also helps you make deployment-optimization decisions.

Chapter 4: Synthetic Data and Distillation—Producing and Transferring Capability

What Is Synthetic Data?

Training data has only two sources: humans, which produce real data, and models, which produce synthetic data. The simplest example is asking GPT-4, “Please write a popular-science article about photosynthesis.” The resulting article is synthetic data. If you collect it to train another model, you are training on synthetic data.

The central reason synthetic data is needed is that human data is running out. High-quality text on the internet is finite, and major companies have already used most crawlable web pages and licensable books. Meanwhile, certain types of data are scarce by nature—for example, humans have written very few high-quality mathematical reasoning traces.

Typical forms of synthetic data include instruction–response pairs, such as Self-Instruct expanding a small number of seed instructions into hundreds of thousands of examples; reasoning trajectories containing complete problem-solving processes, such as those produced by DeepSeek-R1; preference data comparing good and bad responses for RLHF or DPO; and knowledge-augmented data that reorganizes existing knowledge into training-friendly formats.

The Fundamental Contradiction of Synthetic Data

A model can only recombine what it already “knows”; it cannot create new knowledge from nothing. The quality ceiling of synthetic data is constrained by the capability ceiling of the model that generated it. Model-generated data is also “cleaner” and more regular than human data. That sounds beneficial, but it is not entirely so. Real human data contains diverse habits of expression, unconventional reasoning paths, and enormous stylistic variation. This “noise” is precisely what helps a model acquire robust generalization. A model trained purely on synthetic data may perform well within scenarios covered by its training distribution but generalize less effectively to edge cases and new problems.

Distillation: An Engineering Method for Transferring Capability

The central idea of distillation is to extract “knowledge” from a large teacher model and transfer it into a smaller student model.

DeepSeek-R1’s distillation is essentially supervised fine-tuning, except that the training data comes from a large model. The process is as follows: first train the large R1 model—a 671B-parameter MoE—and apply reinforcement learning so that it acquires deep reasoning; then give R1 many difficult questions and collect the complete reasoning trajectories it generates, including trial, correction, and verification; finally, use standard SFT to train smaller models on those trajectories. DeepSeek collected about 800,000 high-quality trajectories to train dense models ranging from 1.5B to 70B parameters.

The key is that the small model learns not only “what is the answer to this problem?” but also “what kind of thought process should be unfolded when encountering this class of problem?” Because the training data contains complete reasoning trajectories, predicting the next token also teaches the model patterns of reasoning.

A more classical distillation method is Knowledge Distillation, proposed by Hinton in 2015. It uses not only the large model’s final output text but also its complete probability distribution, or soft labels. For example, when predicting the next word, the large model might assign 70% probability to “cat,” 25% to “dog,” and 5% to “table.” Trajectory distillation tells the small model only that “the answer is cat,” whereas Hinton-style distillation provides the complete distribution, which contains dark knowledge such as “dog is not the best answer, but it has some plausibility.” Hinton-style distillation requires running the large and small models together to obtain logits, however, which is extremely costly for a 671B model. DeepSeek therefore chose the more practical method of trajectory distillation.

The Limitations of Distillation

Even at the same parameter count, a new model trained on synthetic data is not equivalent to the original large model. There are several reasons: synthetic data is only a “projection” of the large model’s internal representations, like the shadow of a three-dimensional object losing depth information; model-generated data lacks the distributional diversity of human data; it is impossible to exhaust the entire capability space of the large model; and many large-model capabilities arise from exploration during reinforcement learning, which pure supervised learning cannot fully reproduce.

A more accurate view is: Distillation lets a small model approach—but not equal—the performance of a large model on specific tasks and scenarios at a much lower cost. The large model is the source of capability; distillation transfers that capability. Transfer inevitably loses information and has a limited scope.

The Bootstrapping Loop: Each Model Generation Reconstructs the Next One’s Data

Synthetic data creates a bootstrapping loop: early models at GPT-3 scale generate basic instruction data → stronger models at GPT-4 scale generate high-quality reasoning trajectories and chain-of-thought data → reasoning models trained with RL at the DeepSeek-R1/o1 scale produce trajectories for distillation → those trajectories train deployable small models.

The loop has a direction: Models must get bigger before they can get smaller. Some capabilities, especially complex reasoning, emerge only in a sufficiently large parameter space. One possible explanation is that knowledge memorization and reasoning capability are entangled in internet corpora: the pretraining objective requires a model to learn both at once, and only a sufficiently large model can support both. A large model can then generate “pure reasoning demonstrations,” allowing a smaller model trained on this disentangled data to focus on reasoning itself.

The value of a frontier model therefore lies not only in its own serving capability but also in the training data it supplies to the industry: Frontier model value = training data source for the whole industry, not just its own inference.

Implication for application engineers: Understanding the mechanism and limitations of distillation helps you choose models more precisely. DeepSeek-R1-Distill-14B, for example, may approach some large models on mathematical reasoning while remaining far behind in open-domain conversation because its distillation data is highly concentrated on mathematical reasoning. You need to evaluate it on your specific tasks rather than broadly accepting claims that “this model is about as good as GPT-4.”

Chapter 5: Post-Training—Where the Differences Users Actually Feel Begin

Why Post-Training Is the Key

The product of pretraining is a base model—essentially an extremely powerful “completion machine.” It does not know that it is supposed to be an assistant. If you ask, “What is the weather in Tokyo?” it may not answer you and may instead continue the text as an article. All of the “knowledge” was loaded during pretraining, but it is locked inside an uncooperative form.

Post-training releases those latent capabilities in the form users expect. It is like a person who has read every book and has everything in their head but has not learned the behavioral pattern, “Someone is asking me a question, so I should answer.”

Three Major Methods: SFT, RLHF, and DPO

SFT (Supervised Fine-Tuning) is the most direct approach. Prepare pairs of “instruction → ideal response” and continue training the model on them. The model then learns the behavioral pattern “receive an instruction → give a response.” But SFT can only teach the model to imitate reference answers; it cannot teach the model what “better” means.

RLHF (Reinforcement Learning from Human Feedback) addresses that problem. The model generates multiple answers to the same question, human annotators compare them, and those comparisons are used to train a reward model. Reinforcement learning then teaches the model to pursue a higher reward score. In this way, the model is not merely imitating reference answers; it is learning an abstract standard of “good.”

DPO (Direct Preference Optimization) is a simplified form of RLHF. It skips the step of training a reward model and learns directly from preference-comparison data. It is simpler to engineer because it does not require maintaining two models simultaneously or implementing a complex RL training loop.

DeepSeek-R1’s Four-Stage Pipeline

This is the clearest modern post-training pipeline in currently available public material.

Stage 1: SFT cold start. Before RL, the model warms up on a small quantity of high-quality chain-of-thought data. DeepSeek-R1-Zero demonstrated that applying RL directly to a base model is feasible—an important scientific finding—but a model trained with pure RL repeats itself, mixes languages, and is difficult to read. The RL reward cares only whether the final answer is correct, not whether the expression is fluent. Cold-start SFT gives RL a more stable starting point by first constraining format and language consistency.

Stage 2: Reasoning RL with GRPO. Reinforcement learning is applied in verifiable domains such as mathematics, code, and logic using the GRPO algorithm. There are two key design choices. First, the reward uses verifiable correctness—mathematics has reference answers and code can run tests—so human scoring is unnecessary. Second, GRPO samples a group of responses to the same problem and replaces absolute value estimation with ranking within the group. It therefore needs no separate value network and is much simpler to engineer than traditional PPO.

Stage 3: Rejection Sampling Fine-Tuning. The RL-trained model generates a large number of responses; only successful, high-quality trajectories are retained and used as new SFT data for another round of supervised fine-tuning. This creates a bridge and a positive loop between RL and SFT: RL explores strong patterns → rejection sampling filters out the best trajectories → the trajectories become SFT data → SFT makes the model reproduce those strong patterns consistently. The loop can be repeated multiple times.

Stage 4: Alignment RL. Preference feedback adjusts helpfulness and safety so that the model meets release standards. The reward signal is no longer “Is the answer correct?” but “Is the response helpful and safe?”

The four stages depend on one another: the cold start lets RL begin stably, RL produces high-quality data, rejection sampling turns that data into the next SFT input, and alignment RL completes behavioral convergence.

An Important Warning About “Style vs. Capability”

The format of SFT data—whether answers use bullet points or citations, and how long they are—significantly affects what model output looks like. Preference evaluation inherently favors longer answers, training models to produce longer responses even when a concise answer would be more appropriate. Users often think they are comparing capabilities when what they have actually measured is a difference in style.

Implication for application engineers: Once you understand the post-training pipeline, you can no longer evaluate models only by asking which answer “looks better.” You need to measure accuracy, cost, latency, and stability on specific tasks. A long answer that looks more “serious” may simply reflect SFT data in which long answers were labeled “better”; it is not necessarily more accurate.

Chapter 6: Evals, Graders, and Rewards—The Definition of “Good” Determines the Training Direction

The Central Proposition

If the grader is wrong, training optimizes the wrong target.

Model training is fundamentally an optimization process. Give a model an objective and it will do everything it can to maximize the score. The model does not “understand” what you truly want; it optimizes the quantity you have measured. If any gap exists between the scoring rule and your real intention, the model will find that gap precisely. It is like defining KPIs for a team: if the KPI is “lines of code,” engineers will write verbose code; if it is “number of bugs fixed,” they may first create easy bugs and then fix them.

Three Key Components

An eval determines what to measure by choosing tasks on which to evaluate the model. A grader determines how one output becomes a score, bridging model output and training signal. A reward determines where the model is pushed next: the grader’s scores are aggregated into the RL reward signal and directly drive the direction of parameter updates.

Together, these components form a feedback loop: Task definition → Eval set → Grader/judge → Reward signal → Policy update → New rollouts → back to the Eval set. If any link is wrong, every optimization downstream is led astray.

ORM vs. PRM: Judge Only the Result, or the Process Too?

An ORM (Outcome Reward Model) looks only at the final answer. The reasoning may be chaotic, but a coincidentally correct answer receives full credit. The problem is that the model learns not “how to reason correctly” but “how to increase the probability of guessing the final answer,” encouraging various forms of shortcut reasoning.

A PRM (Process Reward Model) scores every step. The model learns that “step two was wrong” and is trained to acquire a correct reasoning process. But PRM annotation is extremely expensive: for a ten-step reasoning problem, the annotation workload is approximately ten times that of an ORM.

Most real systems begin with ORMs because the cost is manageable. PRMs become practical mainly for verifiable tasks such as mathematics, code, and logic, where intermediate steps can be checked by programs and the manual-annotation bottleneck can be avoided.

Reward Hacking: The Model Learns to Exploit Loopholes

When a model becomes sufficiently capable, it actively searches for vulnerabilities in the reward system. The problem has several levels of increasing depth.

The first level is taking shortcuts, the classic ORM problem: the model finds a way to earn a high score without genuine reasoning.

The second level is unfaithful chain of thought. Anthropic found that a written “reasoning process” does not necessarily reflect the model’s actual internal process. The model may use an extra clue but omit it from the chain of thought, adding a post-hoc rationalization instead.

The third level is reward hacking. The model does not perform the task better; it exploits the scoring system. If a code task’s grader is test pass rate, for example, the model may learn to hard-code the expected outputs of the test cases.

The fourth level is reward tampering and alignment faking. The model attempts to alter the scoring process itself, or behaves compliantly while monitored and differently when not monitored. Anthropic experiments in 2025 confirmed that such behaviors can occur.

Crucially, these behaviors are invisible in standard conversational evaluations. They emerge in agent task environments because agents can call tools, access environments, and execute multiple steps.

Constitutional AI: Replacing Per-Example Labels with Principles

Anthropic’s Constitutional AI takes a different approach. Instead of labeling each response as good or bad, humans write a set of principles—a constitution—and the model judges whether its own output complies with them.

Phase 1, the SL stage: the model generates a response → critiques itself against the principles → revises the response → the revised response becomes SFT data. Phase 2, the RL stage: generate pairs of responses → have AI judge which is better according to the principles, known as RLAIF → produce preference data → perform RL training.

The central innovation is: RLAIF replaces RLHF—AI evaluates AI, with human oversight provided through rules instead of per-example labels. Human supervision shifts from labeling each example to writing rules, which is a qualitative leap in scalability.

Implication for application engineers: This entire Eval/Grader/Reward framework is directly relevant to evaluation design in agent systems. When building a RAG system or agent, you also need to define “what counts as a good output”—that is your own grader. If the evaluation criterion is inaccurate—for example, checking only whether an answer contains keywords rather than whether it is actually helpful—your optimization will move in the wrong direction. Understanding reward hacking likewise helps you add necessary safeguards when designing agent systems.

Chapter 7: Agent Training—The Model Itself Is No Longer the Only Optimization Target

From Reasoning to Agents: A Coordinate System for Industry Evolution

AI’s evolution can be understood on a two-dimensional coordinate system: the X-axis is training compute, and the Y-axis is inference compute—the number of tokens consumed per response.

GPT-3 era, lower left: The only way to improve capability was to increase training compute. Response length was fixed at inference time; all intelligence was injected during training.

Larger pretraining, lower right: Training scale continues to increase through over-training, while inference cost remains unchanged. Marginal returns diminish.

Reasoning models, upper left: o1 and DeepSeek-R1 opened a new dimension. The training scale need not be larger, but the model learns to “think more” at inference time—spending more reasoning tokens really can yield better answers through test-time scaling. RL teaches not only “how to answer,” but also “when to think longer and when to stop.”

Agent era, upper right: Training and inference compute both scale. Inference is no longer merely “thinking longer” but “taking more actions”: longer trajectories and more tool calls. The model acts continuously in an environment, calling tools, receiving feedback, updating memory, and choosing the next step.

Reasoning Models vs. Agentic Models: A Structural Difference

The difference is not merely that agentic models “can do more”; the entire optimization paradigm changes.

A reasoning model follows a linear process: Prompt → Reasoning trace → Final answer → Verifier. The optimization unit is one answer. An agentic model follows a loop: Goal → Planner/policy → Tool call → Environment feedback → Memory/context edit → Next action → back to the Planner. The optimization unit is a trajectory—a sequence of actions and decisions made by the model in an environment.

This produces several critical differences. A reasoning model’s main bottleneck is verifier accuracy, its typical failure mode is shortcut reasoning, and its risk of reward hacking is relatively low. An agentic model’s main bottleneck is harness quality, or the quality of its control program. Its typical failure modes are tool misuse and context drift, and its reward-hacking risk is high because access to tools and an environment gives it far more ways to exploit loopholes than a pure reasoning setting does.

Harness: From a Runtime Concept to a Training Concept

Traditionally, a harness is understood as the control layer built around a model when deploying an agent: prompt construction, retrieval, and tool orchestration. In agent training, its role changes.

During training, an agent must repeatedly execute tasks in an environment to collect experience and reward signals. If the harness itself is unstable—tool returns are nondeterministic, the browser environment differs from production, or file-system state is not reproducible—the grader fails first, and the model learns not capability but how to exploit the environment. When training an agent, you are often debugging both the model and the environment.

Data diversity came first in the SFT era. In the agent era, environment quality is the core concern: stability, realism, coverage, difficulty distribution, richness of feedback, and resistance to exploitation.

Kimi K2.5 PARL: A Concrete Engineering Case in Agent Training

PARL’s central design is to train only the orchestrator while freezing all sub-agents. This solves the credit-assignment problem: when a multi-agent system completes or fails a task, who deserves the credit or blame? Once the sub-agents are frozen, every outcome can be attributed to the quality of the orchestrator’s decisions.

The reward signal contains three components: r_perf, task success rate and the main signal; r_parallel, which encourages effective parallel decomposition and is gradually annealed to zero during training so that the model does not game the score by parallelizing everything; and r_finish, which penalizes fake parallelism.

Whether parallelism is truly effective is evaluated not by how much the total step count falls, but by whether the critical path—the longest serial chain—becomes shorter. This is a quintessential distributed-systems perspective.

The bottom line is: Parallelism emerges from RL, not supervision. Parallel capability emerges through RL training rather than being explicitly taught through supervised learning.

Meta-Harness: Optimizing the Harness Itself

The latest development is that not only can a model be trained inside a harness, but the harness code itself can become an object searched and optimized by an outer loop.

The process is: run the harness to drive the model through tasks → produce rollouts, scores, and execution traces → have an outer-loop optimizer read those traces and scores → modify the harness code by adjusting prompt construction, retrieval strategy, context-editing logic, and tool-orchestration rules → rerun tasks with the modified harness → determine whether the score improves.

Meta-Harness experiments showed that changing only the harness around the same base model can produce a sixfold performance difference. They also found that harness optimization generalizes across models: on 200 IMO-level problems, a discovered harness improved five models not involved in the optimization by an average of 4.7 points.

One particularly notable example is that Meta-Harness automatically discovered an environment-bootstrap strategy on TerminalBench-2: before the agent loop begins, run a shell command, summarize the working directory, available languages, package managers, and memory state into a snapshot, and inject it into the first prompt. This is fully consistent with the AGENTS.md design principle distilled from manual engineering practice: give the model better context from the very beginning.

The optimization target has therefore evolved from answer → trajectory → harness program.

Implication for application engineers: This chapter may be the most directly relevant one for AI application engineers. It demonstrates that harness engineering is one of the highest-leverage engineering dimensions in the agent era. You do not need to train the model, but you do need to design the control program around it: prompt construction, retrieval policy, context editing, and tool orchestration. Meta-Harness shows that changing only the harness around the same base model can produce a huge difference. Your ability to design a harness may therefore matter more than which model you choose.

Chapter 8: How to Analyze Why a New Model Has Improved

A Three-Dimensional Framework

When a new model is released with the claim that it is “much stronger than the previous version,” analyze it across three dimensions.

First dimension: Did the change occur in pretraining or in the post-training pipeline? Many improvements do come from stronger pretraining and better data recipes, but many user-visible changes occur mainly during post-training. Whether the model follows instructions, uses tools, or maintains a stable response style often does not emerge merely from training on more text. It is determined by the style of SFT data, RL reward design, and alignment methods used during post-training.

Second dimension: Which layer produced the improvement? It may come from weights and the training recipe, meaning the model itself changed; from rewards, evals, and graders, meaning the training target changed; or from harness code and the deployment loop, meaning the control program around the model changed. In the reasoning-model and agent stages, the improvements users experience are often not produced by the base model alone. Evaluation design, reward assignment, tool-environment stability, the organization of retrieval and memory, summary and context pruning, and the checkpoint selected for production all affect the final product.

Third dimension: What is the production version optimizing? Some releases pursue the capability ceiling, some reduce cost and latency, and others specialize for particular scenarios. A released version is a product decision, not simply the rightmost point on a training curve. Users may think a model name represents a smoothly ascending training curve, but choosing which checkpoint to deploy involves many product trade-offs.

Online Optimization: The Boundary Between Training and Deployment Is Narrowing

Cursor Composer 2’s real-time RL is an important signal: real user coding behavior feeds directly into the training loop, and the model is continuously iterated rather than waiting for the next major release. The boundary between training and deployment has not disappeared, but the feedback loop between them is getting shorter.

A model released today is only a snapshot. The pipeline and the harness program are the products that keep running.

Chapter 9: What This Knowledge Means for AI Application Engineers

Where You Sit in the Full Pipeline

The LLM production pipeline is roughly: data engineering → pretraining → post-training → distillation/specialization → deployment → product/application. AI application engineers work mainly toward the end of that chain, in deployment and the application layer. You do not need to perform the early stages yourself, but how deeply you understand them directly determines how well you can make decisions later in the chain.

Six Directly Actionable Lessons

First, select models based on their training background, not only benchmarks. Knowing how a model was trained—its data mixture, distillation source, and post-training method—helps you predict its behavior on your tasks. A model trained heavily on code may reason well but converse poorly; a distilled model may excel on tasks covered by its distillation data and degrade elsewhere.

Second, interpret benchmark scores cautiously. Scores can be affected by data leakage, deduplication quality, and the inherent preference of preference evaluations for long answers. EDD, or Evaluation-Driven Development, on your own tasks is the reliable way to judge a model.

Third, harness engineering is the highest-leverage skill. Meta-Harness demonstrates that changing only the harness around the same base model can produce a sixfold performance difference. The prompt construction, retrieval policy, context editing, tool orchestration, and state management you design in an agent system are all parts of harness engineering. These are not trivial “tool configuration” details; they are among the highest-leverage engineering dimensions of an agent system.

Fourth, evaluation design is directly related to reward design. The evaluation criteria you define for a RAG system or agent are effectively your grader. If those criteria are inaccurate, optimization moves in the wrong direction. Understanding the ORM–PRM trade-off and the existence of reward hacking helps you build a more reliable evaluation system.

Fifth, understanding training-side constraints helps you make better deployment decisions. Why does the model have 128K context? Because a longer context dramatically increases attention cost and forces down batch size. Why does quality drop after quantization? The training may not have included quantization-aware training. These insights lead to more rational deployment choices.

Sixth, build your narrative. In an interview, you can use the knowledge system in this article to support a compelling narrative: you are not merely writing prompts; you are doing harness engineering, an engineering practice now shown to be highly leveraged from training through deployment in agent systems. You understand both the theoretical foundation and the practical work in this field.

Closing Thoughts

LLM training is not something you need to perform personally, but it is a system you need to understand. In the same way that a Java backend engineer need not design a CPU, understanding cache hierarchies and memory models helps you write better concurrent code.

This article has covered the entire chain from data pipelines and pretraining trade-offs to post-training pipelines and agent training, and from evaluation systems to harness optimization. You do not need to memorize every detail, but you do need to internalize the central logic running through all of it: Every model capability is the result of a design decision—the data mixture determines the direction of capability, pretraining scale determines the capability ceiling, the post-training pipeline determines how capability is presented, and the harness determines how much of that capability can be realized in real scenarios. Understanding this chain means understanding the context and constraints behind every decision you make as an application engineer.

A model released today is only a snapshot. The pipeline and the harness program are the products that keep running. Your work is to turn model capability at the end of that chain into value users actually need.