OBSERVATORY.AI
Monitor of research papers in AI Safety, updated daily.
Guiding LLM Post-training Data Engineering with Model Internals from Sparse Autoencoders
This study proposes SAERL, a data engineering framework that uses Sparse Autoencoders (SAE) to extract model internals and model three intrinsic data properties: diversity, difficulty, and quality. This approach significantly improves LLM reinforcement learning, achieving 3.00% higher accuracy and requiring 20% fewer training steps, demonstrating the practical value of model internals for post-training data engineering.
Position: AI Safety Requires Effective Controllability
This paper argues that AI safety necessitates controllability as a first-class objective, as alignment alone does not guarantee an agent can be stopped or redirected in complex environments. The authors define controllability as an AI system's ability to remain reliably interruptible and redirectable at runtime, proposing a control-centric architectural framework to achieve this.
Detecting Unfaithful Chain-of-Thought via Circuit-Guided Internal-External Discrepancy
This study addresses the detection of unfaithful Chain-of-Thought (CoT) reasoning in LLMs, where generated traces do not accurately reflect the model's actual decision process, a limitation of current detectors relying solely on external signals. Researchers propose CIE-Scorer, a framework that efficiently traces compact sentence-level circuits and measures the discrepancy between internal and external reasoning graphs to identify unfaithfulness with state-of-the-art performance.
Continuous-Depth Field Theory for Transformer Patching and Mechanistic Interpretability
The authors introduce a novel field-theoretic framework that models Transformer residual streams as depth-token fields, enabling the organization and prediction of mechanistic interpretability interventions like patching. Empirically, the study validates this theory in GPT-2-style models, demonstrating how to predict patch effects, measure propagation, and transfer answer behavior through prompt-induced residual displacements.
Hide to Guide: Learning via Semantic Masking
This paper introduces Semantic Masked Expert Policy Optimization (SMEPO), a novel fine-grained semantic masking strategy for expert-guided Reinforcement Learning with Verifiable Rewards (RLVR). SMEPO prevents reward hacking by masking reward-relevant semantic spans in expert traces, forcing models to reconstruct missing information while following the expert's problem-solving structure.
Cultivating Machine Intelligence: The OMEGA Shift from Top-Down Optimization to Autopoietic Cognitive Ecologies
The dominant AI paradigm, relying on top-down optimization, inherently produces structural failure modes such as hallucination, sycophancy, and alignment fragility. This paper introduces RECLAIM, a theoretical framework that cultivates intelligence through computational ecology, signifying an OMEGA shift from optimization to generative autopoiesis.
Inverting the Shield: Systematically Generating Safety Tests from Policy Specifications
The POLARIS framework introduces a novel approach to AI safety by systematically generating test cases from natural-language policy specifications. It converts high-level rules into First-Order Logic, building a Semantic Policy Graph to uncover compositional violation patterns, enabling reproducible safety testing with verifiable traceability.
The Concept Allocation Zone: Tracking How Concepts Form Across Transformer Depth
This study introduces the Concept Allocation Zone (CAZ) to track how concepts gradually form across the depth of transformer language models, rather than emerging in a single layer. The CAZ formalizes the depth interval where a concept becomes measurably separable, revealing subtle allocation regions often missed by standard peak detection methods.
HoloFair: Unified T2I Fairness Evaluation and Fair-GRPO Debiasing
Text-to-Image (T2I) models frequently perpetuate societal biases; HoloFair introduces a comprehensive benchmark framework for multidimensional demographic bias analysis, utilizing a novel Multi-attribute, Group-wise Bias Index (MGBI). Furthermore, it presents Fair-GRPO, a reinforcement-learning-based debiasing method that significantly improves multidimensional fairness in models like SD3.5-Medium while preserving high image quality.