Updated: Today

OBSERVATORY.AI

Monitor of research papers in AI Safety, updated daily.

Interpretability

Guiding LLM Post-training Data Engineering with Model Internals from Sparse Autoencoders

2026-05-25
Desconocido

This study proposes SAERL, a data engineering framework that uses Sparse Autoencoders (SAE) to extract model internals and model three intrinsic data properties: diversity, difficulty, and quality. This approach significantly improves LLM reinforcement learning, achieving 3.00% higher accuracy and requiring 20% fewer training steps, demonstrating the practical value of model internals for post-training data engineering.

by Yi Jing, Zao Dai, Jinwu Hu et al.
Alignment

Position: AI Safety Requires Effective Controllability

2026-05-25
Desconocido

This paper argues that AI safety necessitates controllability as a first-class objective, as alignment alone does not guarantee an agent can be stopped or redirected in complex environments. The authors define controllability as an AI system's ability to remain reliably interruptible and redirectable at runtime, proposing a control-centric architectural framework to achieve this.

by Yige Li, Yunhao Feng, Jun Sun
Interpretability

Detecting Unfaithful Chain-of-Thought via Circuit-Guided Internal-External Discrepancy

2026-05-24
Desconocido

This study addresses the detection of unfaithful Chain-of-Thought (CoT) reasoning in LLMs, where generated traces do not accurately reflect the model's actual decision process, a limitation of current detectors relying solely on external signals. Researchers propose CIE-Scorer, a framework that efficiently traces compact sentence-level circuits and measures the discrepancy between internal and external reasoning graphs to identify unfaithfulness with state-of-the-art performance.

by Xu Shen, Zhen Tan, Song Wang et al.
Interpretability

Continuous-Depth Field Theory for Transformer Patching and Mechanistic Interpretability

2026-05-23
Desconocido

The authors introduce a novel field-theoretic framework that models Transformer residual streams as depth-token fields, enabling the organization and prediction of mechanistic interpretability interventions like patching. Empirically, the study validates this theory in GPT-2-style models, demonstrating how to predict patch effects, measure propagation, and transfer answer behavior through prompt-induced residual displacements.

by David N. Olivieri, Antonio F. Pérez Rodríguez
Alignment

Hide to Guide: Learning via Semantic Masking

2026-05-23
MIT

This paper introduces Semantic Masked Expert Policy Optimization (SMEPO), a novel fine-grained semantic masking strategy for expert-guided Reinforcement Learning with Verifiable Rewards (RLVR). SMEPO prevents reward hacking by masking reward-relevant semantic spans in expert traces, forcing models to reconstruct missing information while following the expert's problem-solving structure.

by Ruitao Liu, Qinghao Hu, Alex Hu et al.
Alignment

Cultivating Machine Intelligence: The OMEGA Shift from Top-Down Optimization to Autopoietic Cognitive Ecologies

2026-05-23
Desconocido

The dominant AI paradigm, relying on top-down optimization, inherently produces structural failure modes such as hallucination, sycophancy, and alignment fragility. This paper introduces RECLAIM, a theoretical framework that cultivates intelligence through computational ecology, signifying an OMEGA shift from optimization to generative autopoiesis.

by Ata G. Zare
Governance

Inverting the Shield: Systematically Generating Safety Tests from Policy Specifications

2026-05-23
Desconocido

The POLARIS framework introduces a novel approach to AI safety by systematically generating test cases from natural-language policy specifications. It converts high-level rules into First-Order Logic, building a Semantic Policy Graph to uncover compositional violation patterns, enabling reproducible safety testing with verifiable traceability.

by Xiaoyue Lu, Xianglin Yang, Haijun Liu et al.
Interpretability

The Concept Allocation Zone: Tracking How Concepts Form Across Transformer Depth

2026-05-23
Desconocido

This study introduces the Concept Allocation Zone (CAZ) to track how concepts gradually form across the depth of transformer language models, rather than emerging in a single layer. The CAZ formalizes the depth interval where a concept becomes measurably separable, revealing subtle allocation regions often missed by standard peak detection methods.

by James Henry
Ethics

HoloFair: Unified T2I Fairness Evaluation and Fair-GRPO Debiasing

2026-05-22
Desconocido

Text-to-Image (T2I) models frequently perpetuate societal biases; HoloFair introduces a comprehensive benchmark framework for multidimensional demographic bias analysis, utilizing a novel Multi-attribute, Group-wise Bias Index (MGBI). Furthermore, it presents Fair-GRPO, a reinforcement-learning-based debiasing method that significantly improves multidimensional fairness in models like SD3.5-Medium while preserving high image quality.

by Ruyi Chen, Lu Zhou, Xiaogang Xu et al.