Accès ouvert

MoralityGym: A Benchmark for Evaluating Hierarchical Moral Alignment in Sequential Decision-Making Agents

Article scientifique 2026 Autre

Résumé

Evaluating moral alignment in agents navigating conflicting, hierarchically structured human norms is a critical challenge at the intersection of AI safety, moral philosophy, and cognitive science. We introduce Morality Chains, a novel formalism for representing moral norms as ordered deontic constraints, and MoralityGym, a benchmark of 98 ethical-dilemma problems presented as trolley-dilemma-style Gymnasium environments. By decoupling task-solving from moral evaluation and introducing a novel morality metric, MoralityGym allows the integration of insights from psychology and philosophy into the evaluation of norm-sensitive reasoning. Baseline results with Safe RL methods reveal key limitations, underscoring the need for more principled approaches to ethical decision-making. This work provides a foundation for developing AI systems that behave more reliably, transparently, and ethically in complex real-world contexts.

Citer ce document

Rosen, S., Singh, S., Gelo, E., Robertson, H., Suder, I., Williams, V., Rosman, B., Tasse, G., James, S. (2026). MoralityGym: A Benchmark for Evaluating Hierarchical Moral Alignment in Sequential Decision-Making Agents. https://doi.org/10.65109/sakl6648

Accès au document

Texte intégral en lecture en ligne, réservé aux abonnés SPHAERO et aux membres de l'institution. Se connecter

Auteur(s)

Simon Rosen Siddarth Singh Ebenezer Gelo Helen Sarah Robertson Ibrahim Suder Victoria Williams Benjamin Rosman Geraud Nangue Tasse Steven James

Statistiques

Consultations : 4

Téléchargements : 0