Safety-Openness Pareto Frontier Technical Report Released

Anyuan AI released The Safety-Openness Pareto Frontier Technical Report, scoring nearly 100 frontier AI models on openness and measured safety. The chart combines an Artificial Analysis-based openness index with a platform safety score and marks the current Pareto frontier of that trade-off.

Introduction

Whether frontier AI models should be more open or more closed remains a standing debate in AI governance. Anyuan AI, using its frontier AI risk monitoring platform, scored nearly 100 monitored models and plotted the current safety-openness trade-off boundary, the Pareto frontier.

The accompanying technical report, The Safety-Openness Pareto Frontier Technical Report, is aimed at policymakers, model developers, and third-party researchers. Authors are Wang Jinge, Wang Weibing, Fang Liang, and Xie Minxi.

Arguments for openness include faster innovation and knowledge sharing, independent audit and reproduction of model behavior, transparency of training methods and data as a basis for external review, less concentration of power in a few firms, and easier safety research. Arguments for caution include the risk that, as capabilities near practical thresholds in cyber offense and biological risk, unrestricted release could enable misuse, that built-in safety guards can be removed by fine-tuning, and that closed deployment at least retains a control point.

The debate has lacked a shared metric. The platform page places both dimensions on one chart: the x-axis is an openness index, the y-axis is a safety score, and a dashed line marks the Pareto frontier. Models on that frontier have no strictly better alternative on both goals at once. Moving up and left trades safety for openness. Moving down and right trades openness for safety. Models that score best on both sit toward the upper right.

What the Chart Shows

Each point is a frontier AI model.

The x-axis is the openness index (0-100, farther right is more open). It measures weight availability, training transparency, and information disclosure. It is synthesized from the Artificial Analysis (AA) Openness Index and a platform disclosure dimension.

The y-axis is the safety score (0-100, farther up is safer). It is the platform safety score, a weighted blend of base, jailbreak, and tamper safety scores.

The gray dashed line is the Pareto frontier. In Figure 1 the frontier is joined end to end by two groups. The left end is high-safety, disclosure-complete closed models Claude Opus 4.8 and Claude Sonnet 4.5. To the right are open-weight models with mixed disclosure and openness: Qwen 3.5 397B Reasoning, GLM 5, and GLM 4.5.

Closed models cluster on the left, with generally low AA openness scores. The frontier slopes down from upper left to lower right, so openness and safety are negatively correlated along the boundary, and the region that is both open and safe is sparse. The frontier is formed by disclosure-complete closed models at one end and open-weight models at the other. Claude stands for closed and safer systems. GLM and DeepSeek stand for highly open systems.

Methodology

Openness index (x-axis)

Platform openness index = (AA raw score + disclosure raw score) / 23 × 100.

AA raw score (0-18) is the AA Openness Index (0-100) converted back to raw points: AA index × 18 / 100. Sub-ranges are model availability 0-6, method transparency 0-6, post-training data transparency 0-3, and pre-training data transparency 0-3.

For closed models that AA does not list, monitored models reachable by API receive the lowest availability band of 1 raw point and 0 on other AA items. Closed models with no API remain at 0. Open models treated as missing data are not added.

Disclosure raw score (0-5) covers five binary items from platform research.

The AA Openness Index measures how open the model itself is (weights, methods, data). It does not measure how much a company publishes around the model. The disclosure dimension adds capability evaluations, safety evaluations, model specifications, risk-management policy, and system prompts. Two closed models can share an AA score while differing sharply in documentation useful to policymakers and safety researchers.

Disclosure scoring used AI research plus human review across 128 models from 14 vendors. Each of five items is 0 or 1.

Capability evaluation and safety evaluation are separate items. Publishing benchmark scores is common (91% of models scored). Publishing safety-evaluation results is less common (38%). Model specifications count only substantive written behavior-spec documents that define behavior logic, value preferences, and safety bounds. Positive examples cited are Anthropic Constitution, OpenAI Model Spec, and Alibaba Model Specification. Technical specs already in the AA score, user policies, employee codes, and generic company AI principle pages do not count. By this rule, 24 of 128 models scored. A spec published after the model release scores 0 for that model.

System prompts count only official, explicit default system prompts. Suggested, recommended, or quick-start example prompts do not count. Risk management scores if the company has a written policy for its own models covering identification, assessment, and mitigation. Frontier-style preparedness or responsible-scaling frameworks are not required. The policy must predate the model release and must apply to the model or model family, not only to a cloud platform or other downstream product. Evidence is limited to official channels. Third-party tests, leaks, and reverse-engineered material are excluded.

Item design references the FLI AI Safety Index (summer 2026) and Stanford FMTI (December 2025). Those indices score companies on broader governance. This index scores models one by one and is not limited to a safety lens.

Safety score (y-axis)

The platform safety score (0-100) matches the platform risk-index definition. It weights base safety (average of non-red-team safety benchmarks), jailbreak safety (average of red-team safety benchmarks), and tamper safety (0 for open-source models, 100 for closed-source models) at 0.4 : 0.4 : 0.2. The loss-of-control domain has no jailbreak or tamper sub-score and uses that domain's safety-benchmark average. The score is behavioral. It measures refusal of harmful requests and related alignment tests. It does not infer capability. Models without safety-benchmark data are omitted.

How to Use the Chart

A minimum-capability filter uses the average of five domain capability scores (cyber offense, biological risk, chemical risk, harmful manipulation, loss of control), with thresholds 35, 40, 45, 50, and 55. Those bands retain about 58, 51, 36, 22, and 14 models. At capability 50 and 55, 22 and 14 models remain.

A release-date filter offers the past two years, past year, and past six months. Axis scales stay fixed when filters change so remaining points keep their positions. A vendor-colored legend can hide or show vendors. Hovering a point shows name, vendor, safety score, capability score, openness index, AA raw score (0-18), disclosure score (0-5), and the five disclosure items. A table below the chart shares filter state, defaults to openness descending, and lists disclosure items as columns.

Data Sources

AA Openness Index snapshot: 2026-09-08. AA has no public openness API. The project collected in-page flight data covering 227 AA models and mapped them to platform models, with fuzzy matching on reasoning-configuration suffixes.

Model disclosure data, last reviewed 2026-09-21, covers 15 vendor series and 128 platform models: Claude (Anthropic), GPT (OpenAI), Gemini (Google DeepMind), Grok (xAI), Qwen (Alibaba), Kimi (Moonshot), GLM (Zhipu), MiniMax, Hunyuan (Tencent), DeepSeek, Llama (Meta), MiMo (Xiaomi), Doubao (ByteDance), Baidu, and a remainder set.

The Pareto chart requires both a platform safety score and disclosure data. Sixty-two models currently meet every condition and form the Figure 1 scatter set. All 128 models enter the disclosure-score statistics.

Results

Results use data as of 2026-09-21.

Raising the capability floor from all models to 45 cuts the set from 62 to 36. The right-hand frontier endpoint moves up: GLM 4.5 (openness 52.2, safety 41.6) is replaced by GLM 5 (openness 43.5, safety 55.7). Openness falls while safety rises. The left segment stays stable. Claude Opus 4.8 has safety 85.8 and capability 62.3. Claude Sonnet 4.5 has safety 85.4 and capability 48.2. The highest safety scores are not tied to low capability.

In the full view, the frontier runs from safety near 85 at upper left to openness-high models with safety 41-42 at lower right. The past-six-months view leaves 12 new models. Safety spans about 40-86. The right-end safety score is about 51, above the full-set right endpoint. Sample size is small.

Among 128 models, 10 scored 5, 14 scored 4, 14 scored 3, 26 scored 2, 57 scored 1, and 7 scored 0. The mean is about 2.01. Most models sit at 1-2. Typical 1-point models score only on capability evaluation. Typical 2-point models add safety evaluation or risk management. A typical frontier model publishes benchmark scores, does not publish safety-evaluation results, and lacks a written behavior spec, risk-management policy, and default system prompt.

The report also includes a limitations section, Appendix A on disclosure judgment rules, Appendix B on boundary cases, and a note on collaboration.