Modern reinforcement learning (RL) agents come in many different families thatare incommensurable with one another. With current tools, they are difficult tocompare on performance across tasks, on adaptation to modified environments,and on energy efficiency.We present ML-school, a framework for grading RL agents from structurallyunrelated families on identical capability exams in shared environments. Everyagent is a black box behind the one interface all families share, observation→ action; the environment is the sole judge of performance; and every recordcarries cost in shared units, separated into hardware-independent and hardware-bound quantities. The first exam grades resilience: a policy is probed along fourperturbation axes and its degradation summarized as the area under a fittedrecovery curve, in both a policy-relative and an absolute form. We validatethe instrument with PPO on Pendulum-v1 across six training budgets and fourseeds, then run an Echo State Network (ESN) through the identical exam todemonstrate cross-family commensurability.The framework and the code are open source at https://github.com/CHA0STHE0RX/ML-school (Apache-2.0)
Peter Visi (Sun,) studied this question.