Most reinforcement learning benchmarks reset the world every episode. Actual operations are never reset. Skyfall AI’s MORPHEUS targets that gap. It is a persistent enterprise simulation platform for continuous reinforcement learning (CRL).
What is Morpheus?
MORPHEUS is based on the Big World Hypothesis (Javed & Sutton, 2024). It is said that the complexity of the world exceeds the expressive ability of any agent. As a result, the environment appears unsteady even under fixed dynamics.
To enforce continuous learning, MORPHEUS requires three properties: persistence, non-stationarity, and operational complexity. Persistence means that past decisions are combined into future dynamics. Nonstationarity means that a fixed policy eventually becomes suboptimal. Operational complexity means that there is no fixed optimal policy.
Each environment is a self-contained TypeScript world plugin. Export operational descriptors (ODs), simulation schedulers, seed data, and documentation. OD defines a step-by-step execution plan for a function. The agent works through a functional API, and each call triggers an OD execution.
How the platform works?
Based on its architecture, unsteadiness arises from two engines. First, the fault injection engine injects typed interruptions between OD steps. Extracted from 11 failure types including: missing_data, dependency_failureand rate_limit. It runs at four preset rates: Light (5%), Real (8%), Medium (15%), and Aggressive (30%).
Second, an asynchronous configuration shift controller modifies fault presets and requests at fixed timestamps. It runs independently of the training loop, so the shifts never coincide with gradient updates. This prevents the agent from using the update period as a proxy clock.
In addition to these engines, you will also earn rewards from three behavioral verifiers natively recorded by the platform. These are failure event signals, financial ledger status, and resource throughput. Composite rewards are a combination of them. The default weights are: w_f = 0.5 and w_l = w_p = 0.25.
# Composite reward — MORPHEUS, Appendix C (default weights).
def clip(x, lo, hi):
return max(lo, min(hi, x))
def composite_reward(tickets, actual_cost, planned_cost, units, capacity,
w_f=0.5, w_l=0.25, w_p=0.25):
r_f = -sum(t["severity"] for t in tickets) # failure event signal
r_l = clip(1 - actual_cost / planned_cost, -1, 1) # financial ledger
r_p = clip(units / capacity, 0, 1) # resource throughput
return w_f * r_f + w_l * r_l + w_p * r_pUnder the upper bound assumptions (zero failures, minimum cost, maximum throughput), the limit per configuration is equal to 0.50.
Initialize policy
Due to the large action space, it is not practical to do pure RL from the beginning. Therefore, MORPHEUS uses a two-stage pipeline. The Frontier model (Gemini 3.1 pro) uses the ReAct framework to collect trajectories. These traces fine-tune Qwen3-14B by supervised fine-tuning (SFT).
Therefore, all RL executions start from this shared SFT checkpoint. This separates continuous learning behavior from basic operational ability. All baselines use PPO as the base optimizer for post-online training.
Evaluation protocol for six indicators
Cumulative rewards alone are not enough when training is defined. Scalar summation hides performance over non-stationary regions. The research team therefore proposes six indicators instead. These are reward, adaptation speed, forgetting, recovery time, stability, and performance gaps per configuration.
Among them, the speed of adaptation is a key indicator. Count steps until the moving average reward reaches half of the upper limit. Two complementary diagnostics also track relative adaptive advantage (RAA) and plasticity via effective ranks.
Baseline results
Using this protocol, the research team tests four families of algorithms from a shared SFT checkpoint. Two tasks are defined. Task 1 is dynamic resource allocation under structured drift. Task 2 is scheduled under drift with a delay effect.
| family | mechanism | Outbound task 1 | Outbound task 2 | Inbound task 2 |
|---|---|---|---|---|
| PPO | No CL mechanism | baseline of failure | Adapt only early | Baseline reward |
| she | hindsight replay | interim compensation | best reward | Highest reward, highest rank |
| EWC | weight integration | best reward | best adaptation | the weakest reward |
| LCM | latent context model | fastest adaptation | No benefit | best adaptation |
Across these results, no single family predominates. above process-outbound Task 1, EWC leads the reward and LCM adapts fastest. In task 2, HER leads in reward, while LCM loses its advantage with delayed reward. On the other hand, the average performance gap is close to 1.0 for all methods. This indicates a large steady-state deficit rather than a small adjustment gap.
In particular, PPO and HER are typically only adapted in their initial configuration. It is then unable to adapt to later regimes even in the absence of label signals.
Usage and examples
In fact, MORPHEUS is suitable for several reader roles. For AI engineers, test whether the agent detects regime shifts without labels. For example, when demand switches from low to bursty, the policy must adapt even in the absence of a signal.
For data scientists, the delay in unit assignment is highlighted. For example, On-Time In-Full (OTIF) delivery is only observable several days after the shipping decision. For software engineers, the TypeScript plugin format allows them to exchange rewards and toggle observability without changing dynamics.
Advantages and disadvantages
Strengths:
- A persistent world with no resets is consistent with deployed enterprise systems.
- Parameterizable and reproducible regime shifts for fair inter-algorithm comparisons.
- Compensation from native behavior verifiers. No external annotations are required.
- Open source evaluation code (
Skyfall-Research/morpheus-evals).
Weaknesses:
- So far, only two of the five environments have been evaluated.
- The upper bound assumes zero failures, so it remains optimistic.
- Change is caused externally, not by complex decisions.
- Compensation weights are research variables and are not validated industry targets.
Important points
- MORPHEUS runs a persistent enterprise world that never resets, unlike ephemeral RL benchmarks.
- Five environments are included. Two things are evaluated here.
process-outboundandprocess-inbound. - The six-metric protocol scores reward, adaptation, forgetting, recovery, stability, and gap to upper limit per setting.
- All four baselines (PPO, HER, EWC, LCM) are well below the theoretical upper limit.
- No single algorithm wins. Different winners are chosen depending on reward and speed of adaptation.
Please check paper and Project page. Please feel free to follow us too Twitter Don’t forget to join us 150k+ML subreddit and subscribe our newsletter. hang on! Are you on telegram? You can now also participate by telegram.
Need to partner with us to promote your GitHub repository, Hug Face Page, product release, webinar, etc.? connect with us
Michal Sutter is a data science expert with a master’s degree in data science from the University of Padova. With a strong foundation in statistical analysis, machine learning, and data engineering, Michal excels at transforming complex datasets into actionable insights.
