The Conditions for Trustworthy Synthetic Inference: A Size-Weight Frontier

Why Synthetic Data, Why Now

As LLM-generated responses increasingly appear in surveys, usability studies, and clinical trial metrics, the question of "how much can we trust the results" is becoming critical in practical inference. Simply adding more synthetic data may improve diversity and coverage, but when synthetic data is mixed into the actual collection process, confidence intervals can collapse unrealistically. This paper analyzes this issue mathematically and formalizes it as a size-weight frontier.

Background: Intersection of Synthetic Data and Statistics

The practice of augmenting real data with synthetic data is not new. Recently, LLMs have been used to generate large volumes of responses, and are actually being used in areas like survey augmentation and cold-start recommendations. The problem is that synthetic data can actually differ in distribution and statistically from real data. If we treat this as if it were actual data, estimators can appear stable while actually targeting incorrect parameters. In other words, a project that separates diversity and trust is needed.

Key Concept: The Frontier, and Two Calibration Methods

The paper decomposes synthetic data into three components: actual data count n, synthetic data count k, and the weight lambda of synthetic data. Here, weight means "when the contribution of one actual data point is defined as 1, how much does one synthetic data point count."

The authors define, for each lambda, "the maximum k at which all k maintain target coverage" as a boundary. They call this the oracle size-weight frontier. The (lambda, k) pairs below this boundary form a safe synthetic augmentation regime.

The paper acknowledges that in actual research this boundary is difficult to observe directly. So the authors propose empirical projectors to estimate the boundary. The core idea is to build a "reference confidence set that respects the synthetic augmentation regime," and they propose two approaches.

• full-sample proxy: Build the reference confidence set using the entire actual dataset, and verify consistency with the synthetic augmentation regime.
• split-sample proxy: Use part of the actual dataset for the reference confidence set, and the rest for the synthetic augmentation regime.

Both approaches construct dual projections, but the full-sample is more computationally expensive and statistically conservative, while the split-sample requires dataset splitting but enables task-conditional coverage.

Refined Calibration: Safe Below the Frontier

The selected frontier k* simultaneously provides sample-level coverage guarantees for all (lambda, k) pairs on the boundary. More precisely, the learned frontier range in the calibration ensemble provides conservative coverage guarantees for any subsequent task relative to the number of samples J, meaning the coverage stays slightly above the target.

Importantly, even if we choose the (lambda, k) that minimizes the gap after learning the frontier, there is no additional risk of over-coverage. In other words, a selection-then-check regime with the same concern is possible.

Experiment: Augmenting a European History Survey with LLM Responses

The authors experimented with the WorldValuesBench dataset. They created 235 question-item pairs, generated small actual responses from approximately 96,000 real response pools, simulated cold-start situations, and used GPT-4o and GPT-5 mini as synthetic generators.

Experimental results showed that the learned frontier range maintained target coverage. At the same time, they confirmed the effect of expanding confidence intervals beyond actual data alone. Even when reducing weight to limit the influence of individual synthetic data, the strategy of pushing synthetic information as much as possible within the allowable range of the frontier actually worked.

Why It Matters

Existing synthetic data research often vaguely states "it's good" or "it's risky." This paper brings that gray area into a structural condition. Practical experiments and cold-start UX research show that even when actual data is sufficient, adjusting the number and weight of synthetic data below the frontier can improve data efficiency while reducing inference overhead.

These results can be directly applied to the current situation where LLM synthetic responses are being used as evaluation metrics. If the weight is too high, synthetic data can cause actual data to diverge in distribution, and coverage guarantees can break; mathematically, this can be detected.

Implications and Limitations

Because frontier selection itself requires additional data, full-sample proxies can be unstable on very small actual datasets. Also, while the authors only experimented with GPT-4o and GPT-5 mini as synthetic generators, whether the same boundary holds for other model families or domains — such as images, audio, or log data — has not been verified.

WorldValuesBench is a comparative survey with relatively simple structure, so expansion to activity logs or time-series data depending on the situation is also a future task. Nevertheless, from the point of providing a minimum threshold for safely using synthetic data, the practical significance of this paper is considerable.

References

• arXiv paper: Learning a Size-Weight Frontier for Synthetic-Augmented Inference
• arXiv PDF: 2608.28576.pdf