Compiling Functions From Natural Language? Compile by Training

Do We Really Need to Call a Big Model Every Time?

Picture one recurring text function. When you sort emails by priority, a message like "Signature needed by EOD" must be handled right away, while a newsletter can be pushed back. Such behavior is easy to describe in words, but writing it as rule code gets too messy. So we mostly solve it with a large language model (LLM) API.

In a development environment where calling a big model API every time is taken for granted, the formula "recurring text function = large remote LLM call" is now worth doubting. One paper targets exactly this middle ground: 'Compile by Training: Turning Natural-Language Specifications into Local Neural Functions' (arXiv:2609.04199, EMNLP 2026 System Demonstrations), by Yuntian Deng and Pengyu Nie of the University of Waterloo and Stuart Shieber of Harvard.

Replacing Compilation With Training

The core of the proposal is to "turn adaptation into a build step rather than a runtime dependency." Write down the function's behavior in natural language, and during the editing process teacher models generate examples of that spec, and those examples train a small adapter to specialize a compact interpreter. The result becomes a local neural function that runs without any teacher model.

Existing: every input x -> large remote LLM call (cost, latency, and provider dependence every time) Proposed: spec s -> compile (training) -> program p_s -> every input x -> local execution

This system stands on the same team's earlier 'Program-as-Weights' (PAW) paradigm. PAW keeps a fixed interpreter shared by many functions and specializes each function into a neural program. Existing PAW's fast compiler predicts an adapter in a single forward pass and finishes in seconds, but it has the drawback of spending the same amount of compute on every function. Compile by Training takes that fast prediction as a starting point and spends extra compute on teacher synthesis and per-function optimization to make a better program.

Let's See How It Trains

Compilation proceeds in two stages. First, teacher models synthesize task-specific example pairs from the spec. Requests are sent in a structured JSON format, and the compiler validates the returns to filter out wrong or incomplete batches, passing only the examples that pass into the training pipeline.

Gradient descent specializes a very small adapter, and here the interpreter is a single quantized Qwen3-0.6B model shared by all functions. What each function learns is only a rank-64 LoRA adapter and a runtime scaffold (a prompt template that holds the spec as structured instructions and examples). When training ends, the adapter, scaffold, original spec, and interpreter metadata are bundled into a single program artifact. It looks like a .paw file that can be stored, version-controlled, and cached, and reused like ordinary software.

Source: arXiv:2609.04199 Figure 2 — Compile by training

Image: arXiv:2609.04199 Figure 2 · Deng, Nie, Shieber · CC BY 4.0 · unmodified

Compilation is a hosted process, so the spec is sent to the PAW service and teacher API to create supervised data and training. Local SDK execution, by contrast, does not send future inputs to PAW or the teacher. After compilation, at runtime the SDK merely combines new inputs with the program's runtime prompt and adapter and processes them through the shared interpreter. Execution cost is also fixed per function.

The Price of Higher Accuracy Is Compile Time

The authors evaluated on FuzzyBench-Hard—a subset where PAW's fast compiler produced not a single exact match—and Compile by Training recorded a semantic accuracy of 83.6%. Accuracy rose from 22.4% to 83.6%, up 61.2 percentage points, but in exchange compile time is 50.9 seconds, longer than PAW's 3.5 seconds.

The secret to accuracy also lies in data scale. Where the LEM averaged 74.6% with GPT-5.4-mini as a single teacher, mixing GPT-5.4-mini and GPT-5.5 at 2:1 raised it to 85.1%. The number of training examples also rises from 82.1% at 1,440 pairs to 86.6% at 7,200 pairs, but diminishing returns are clear from 2,400 pairs onward.

The latency measurements at deployment are interesting too. As released in May 2026, the same spec compiled in 50.9 seconds on a B300, 68.2 seconds on an H200, and 99.2 seconds on an RTX GPU. In a load test submitting 4 tasks at once, it was handled with an average queue wait of 1.01 seconds, and since teacher synthesis eats most of the time, the design of overlapping synthesis and training actually leads to fast responses, the paper explains.

Real Application Cases

The paper shows real behavior with three demonstrations. It routed 30 compiled programs into a website helper called paw-helper. When a class question comes in, it classifies whether it's a link or an answer, and composes a pipeline that combines a course-page answer with Piazza search results.

In Avatar Director, you control a 3D avatar with natural language. Enter an instruction like "jump twice, then dance" in natural language, and the compiled program translates it into a DSL and shows it as an animation in the browser. Of 44 validation instructions, it produced the expected structure on 43.

There's also a service that bidirectionally translates between English and Claudish. It can translate expression patterns related to Claude Code into English and vice versa, and both directions run on small adapters.

Why Pay Attention Now

Where you've so far chosen a function between 'rules' and 'a large model call,' Compile by Training gives a third option: 'build once and use locally.' Especially for tasks with many repeated calls—classification, routing, format conversion—if you can bear the compile time, you can substantially reduce the cost, latency, and external dependence at the execution stage. A public interactive service (https://programasweights.com) and source (https://github.com/programasweights/paw-helper) are also released, so you can check it yourself.

References

• Compile by Training on arXiv
• PAW official service
• paw-helper GitHub