Case study · ServiceNow
Evaluating Lucidic's training method on ServiceNow's enterprise workflows.
ServiceNow evaluated Lucidic's training method on 1,150 tasks drawn from real enterprise workflows, published as the EnterpriseOps-Gym environment. Lucidic trained the agent's policy in simulation on half of the tasks and evaluated it on the held-out half. The trained policy achieved a strict task success rate of 65%, compared with 51.1% for the strongest frontier model.
65%
strict task success rate on the held-out task set
+13.9 pts
relative to Claude Fable 5, the strongest frontier model (51.1%)
2.3×
the strongest published RL fine-tune of an open-weight model (28.3%)
Task success rate on held-out tasks
Reward is binary. A task is counted as successful only if every verifier over the final database state passes.
Lucidic
trained policy
Lucidic
65.0%
Claude Fable 5
max effort, Opus 4.8 fallback
Frontier model
51.1%
Gemini 3.5 Flash
high
Frontier model
50.1%
Claude Opus 5
max effort
Frontier model
47.5%
GPT-5.5
xhigh
Frontier model
46.6%
Weight-only RL fine-tune
Qwen3.5-35B-A3B, AgentMercury
Weight-only fine-tune
28.3%
Note: the 1,150 tasks were divided into a training set and a held-out set using a 50/50 split. All Lucidic results reported on this page are on the held-out set. Reference scores for the frontier models and the fine-tuned model are as published on the full task set.
Evaluation setting
ServiceNow's agents perform operational work in enterprise systems, such as updating customer records, closing IT tickets, and rescheduling meetings. Before adopting a training method, ServiceNow required evidence that the method improves performance on this class of work.
The evaluation environment is ServiceNow's EnterpriseOps-Gym. The agent completes each task through live tool calls against a sandboxed enterprise backend. Reward is binary: a task is counted as successful only if every verifier over the final database state passes, and no partial credit is awarded.
1,150
expert-curated tasks
8
enterprise domains
512
tools over 164 database tables
0
partial credit: every verifier must pass
The task set is demanding. The strongest frontier model, Claude Fable 5, achieves a success rate of 51.1%, and success rates in the HR and IT service management domains remain below 30%.
Method
The agent was treated as a policy and trained on reward in simulation. In each iteration, rollouts were sampled across the task distribution and scored with custom reward and loss terms derived from ServiceNow's verifiers, so that every failed verifier contributed a training signal.
Credit was assigned to the step at which a failing trajectory diverged from a successful one. That step was then branched to sample alternative decisions, which concentrated training on the decisions that determined task outcome.
The policy update was performed as a population search over the agent's decision procedure: which domain rules to consult before acting, which writes to verify before committing, and which prior trajectories to retrieve. Each candidate policy was evaluated on held-out tasks, and only candidates that improved held-out reward were retained.
The trained policy is packaged as a single checkpoint and loaded through the Lucidic SDK.
Results
The trained policy achieved a strict task success rate of 65% on the held-out task set. This is 13.9 percentage points above Claude Fable 5, the strongest frontier model, and 2.3 times the strongest published RL fine-tune on the same task set, which improved from 24.8% to 28.3%.
On the basis of this evaluation, ServiceNow can rely on Lucidic's training method to improve the performance of its agents.
