Reflection has published detailed safety training and evaluation results for Beam, its first open-weight model, demonstrating that rigorous safety design need not compromise model utility [1]. Beam achieves an 89% refusal rate on harmful requests while complying with 82% of safety-related harmless requests in ISE evaluations, placing it on the safety-helpfulness Pareto frontier among Western and Chinese open-weight models tested [1]. Through Project Iris, Reflection is releasing its evaluation benchmarks and safety tools as open-source, positioning the company as an ecosystem builder at a moment when open models trail frontier cyber capabilities by only four to seven months [2][1].
What is Covered in this Article
- Beam's dual-assessment safety architecture: ISE for policy compliance and OSF for capability risk [1][1][1]
- Quantitative safety-utility results placing Beam on the Pareto frontier [1]
- Multi-teacher on-policy distillation (MOPD) and the role of model capacity as a safety lever [1][1]
- Project Iris as open-source ecosystem infrastructure for safety benchmarking [1]
- Enterprise relevance: agentic AI deployments and data privacy as top adoption barriers [3][3][3]
The News: Reflection published its open model safety findings for Beam [1], its first open-weight model built as a sparse Mixture-of-Experts architecture with 501 billion total parameters and 23 billion active parameters [1]. In ISE evaluations, Beam achieves an 89% refusal rate on harmful requests across 1,500 human-written conversations spanning 15 harm categories, the highest rate among Western and Chinese open-weight models tested, while complying with 82% of safety-related harmless requests [1][1]. In agentic evaluations, Beam shows no harmful compliance in static settings and 15% in dynamic multi-turn adversarial settings [1]. Reflection trained Beam using multi-teacher on-policy distillation (MOPD), combining a capability teacher and a safety-alignment teacher [1], and partnered with Gray Swan for jailbreak and agentic training data [1]. Project Iris will release ISE benchmarks, evaluation artifacts, and safety tools as open-source [1].
Beam Proves Open-Weight AI Safety and Utility Are Not a Zero-Sum Trade
Analyst Take: Reflection's Beam release is notable not just for its benchmark numbers but for the methodological transparency behind them. By publishing both the ISE policy-compliance results and the OSF capability-risk framework as separable, auditable assessments [1][1], Reflection is making a structural argument: that open-weight safety requires two distinct lenses, and that conflating them produces incomplete assurance. That argument lands at a commercially critical moment, with AI agent reliability and hallucination management cited as the top GenAI adoption challenge by 55.4% of enterprise decision-makers surveyed (n=820) [3].
A Dual-Assessment Architecture Sets a New Methodological Bar
Reflection's ISE and OSF frameworks address a gap that single-metric safety evaluations leave open. ISE tests whether Beam follows Reflection's safety policy across harm categories including chemical, biological, radiological and nuclear weapons, cyber offense, self-harm, violence, fraud, and privacy [1]. OSF separately assesses whether the model's underlying capabilities could meaningfully increase risk beyond what comparable publicly available models already enable [1]. The distinction matters because a model can attempt a harmful request without the capability to succeed, or possess a dangerous capability while refusing to use it. Both assessments are necessary to understand model safety. This dual-lens approach is particularly relevant given that leading open-weight models now trail the frontier on cyber tasks by roughly four to seven months, down from six to ten a year earlier [2], and unlike access-gated frontier offensive tools, these open models are available to anyone [2].
The Safety-Utility Pareto Frontier Is Achievable
The headline numbers from ISE evaluations are striking in context. Beam declines 89% of harmful requests, the highest rate among Western and Chinese open-weight models tested, while complying with 82% of safety-related harmless requests [1]. That combination places it on the observed safety-helpfulness Pareto frontier, directly challenging the assumption that higher refusal rates necessarily degrade utility. The ISE benchmark covers 1,500 human-written conversations across 15 harm categories, including adversarial conversations using tactics like task decomposition, persona modulation, and context manipulation [1]. In agentic settings, Beam shows no harmful compliance in static evaluations and 15% in dynamic multi-turn adversarial evaluations [1]. On Irregular's cyber capability evaluations, Beam scores 34% on Artificial Analysis' Cyber Index [1], a result that informs the OSF capability-risk picture alongside the refusal data.
Model Capacity Is a Safety Lever, Not Just a Performance Lever
Reflection's MOPD training approach produces a technically meaningful finding about scale and safety. The method trains two teachers: one for capability via reinforcement learning, and one for safety and alignment, then combines them in Beam [1]. Increasing the safety and alignment teacher from 100B to 501B parameters, while holding the training data and recipe fixed, reduced over-refusal by 48.7% relative, alongside a 3.1% decrease in the harmful-request pass rate [1]. The implication is that greater model capacity helps distinguish permissible requests from harmful ones, improving helpfulness under the same safety-training setup. Reflection reinforced this with deliberative alignment, training the model to reason through safety principles rather than follow rigid rules, and with iterative red-teaming via Gray Swan covering prompt injection and dangerous tool calls [1]. The result is a model that internalizes safety policy rather than pattern-matching to refusal triggers.
Project Iris Targets Ecosystem Use at the Right Moment
Releasing ISE benchmarks, evaluation artifacts, and safety tools as open-source through Project Iris [1] is a strategic move that extends Reflection's influence beyond Beam itself. Researchers, developers, policymakers, and enterprises can inspect, reproduce, and build on the evaluation methodology, which raises the floor for open-weight safety practices industry-wide. The timing is deliberate: the offensive-defensive capability gap for open models is narrowing faster than anticipated [2], and the open-access nature of these models means that rigorous, transparent benchmarking is a structural necessity rather than a differentiator [2]. For enterprise buyers, the transparency has direct commercial relevance. Data privacy and security vulnerabilities, including ensuring compliance with data sovereignty laws and securing sensitive enterprise data, rank as a top GenAI adoption challenge, cited by 52.6% of decision-makers surveyed (n=820) [3]. Additionally, autonomous threat detection, remediation, and system monitoring in IT operations and cybersecurity contexts represent a planned agentic AI use case for 49.2% of enterprise decision-makers (n=766) [3]. Beam's auditable safety record and Project Iris's open benchmarks directly address both concerns.
Market Stakes Amplify the Importance of Getting Safety Standards Right Now
The AI platforms market is on a trajectory toward $5.3 trillion by 2030, with a base-case CAGR of 52.5% from 2025 to 2030, reaching a base-case market size of $5,301,865 USD millions [4]. Safety standards established at this stage of market formation will shape enterprise adoption patterns at scale. Open-weight models that can demonstrate measurable, auditable safety without sacrificing utility have a structural advantage in enterprise procurement, particularly as agentic deployments move from pilot to production. Beam's results and Project Iris's open infrastructure give Reflection a credible claim to that position, provided the benchmarks hold under broader community scrutiny and the OSF capability assessments keep pace with the model's evolving capabilities.
What to Watch
- ISE community adoption: whether independent researchers reproduce or challenge Beam's Pareto-frontier results within the next quarter, validating or stress-testing the benchmark methodology [1][1]
- OSF capability reassessment cadence: how frequently Reflection updates the Open Safety Framework as Beam's capabilities evolve and the open-model cyber gap continues to narrow [2][1]
- Enterprise agentic deployment uptake: whether the 49.2% of decision-makers planning cybersecurity agentic deployments (n=766) accelerate procurement of auditable open-weight models following Project Iris's release [3][1]
- Competitive safety benchmarking response: how other open-weight model providers reprice or reframe their safety claims in response to ISE becoming a public reference standard [1]
- Gray Swan partnership expansion: whether the jailbreak and agentic red-teaming collaboration extends to cover emerging attack surfaces such as multi-agent orchestration and tool-chaining vulnerabilities [1]
Sources
1. Open model safety: lessons from training and evaluating Beam, Reflection
2. Why AI Learned to Attack Before It Learned to Defend, Futurum Research, August 2026
3. 2H 2026 AI Platforms Global Enterprise Decision Maker Survey Report, Futurum Research, October 2026
4. 2H 2026 AI Platforms Market Sizing & Five-Year Forecast, Futurum Research, November 2026
Disclosure: Futurum is a research and advisory firm that engages or has engaged in research, analysis, and advisory services with many technology companies, including those mentioned in this article. The author does not hold any equity positions with any company mentioned in this article.
Read the full Futurum Group Disclosure.
Other Insights from Futurum:
Reflection's Beam: Open-Weight Frontier at Enterprise Inference Cost
Sweden's Largest Insurer Bets on Guidewire Cloud for AI-Driven Ops
TeraWulf's AI Pivot Is Complete: HPC Now 83% of Revenue
Author Information
This content is written by a commercial general-purpose language model (LLM) along with the Futurum Intelligence Platform, and has not been curated or reviewed by editors. Due to the inherent limitations in using AI tools, please consider the probability of error. The accuracy, completeness, or timeliness of this content cannot be guaranteed. It is generated on the date indicated at the top of the page, based on the content available, and it may be automatically updated as new content becomes available. The content does not consider any other information or perform any independent analysis.
