Use your data to improve your agents
People get better in more than one way: they build skill, and they get better instructions and tools. Agents are the same. Improving an agent means improving the model and the harness around it
Improvement
Prompts, tools, context, memory and policies turn into tunable parameters. The model weights stay frozen
Lucidic inserts training modules where your agent fails, tunes them, and tests every candidate in offline simulation.
The winning configuration comes back as a clear change to your harness. Same model, better agent
Your harness stays as it is, with every parameter still tunable. Now the model weights are on the table too, and the two are trained as one system.
Lucidic sorts your traces and sends every case to the method that learns from it best: LoRA SFT, on-policy distillation, training modules or GRPO.
Hill climbing in simulation finds the best combination. Every parameter of your agent, from the weights to the prompts, is tuned to your reward.
Data
Company and production data become datasets and learnings that keep improving your agent over time
Step 1
We connect to wherever yours live and pull them into one pipeline. No one else has this data, and it shows exactly how your customers use your agent and where it falls short.
Company documents
Past tickets
Policies
Agent traces
Step 2
Your agent produces thousands of traces, and most are routine. We group similar ones and keep what matters: unusual cases, failures, and one example of each common pattern. The result is a small dataset that shows how your agent really behaves, at a fraction of the cost.
Step 3
We build a replica of your agent's world: its tools, its databases and simulated customers. The agent runs through it step by step, and when a run goes wrong, we go back to the exact step where it slipped. From there we replay with a different choice, again and again, keeping only the paths that score higher, until it reaches the reward.
When a better model comes out, you do not start over: your datasets and learnings carry over to the new model
Case studies
The policy trained in simulation achieved a strict task success rate of 65%, compared with 51.1% for the strongest frontier model.
Read full ServiceNow case studyResults
Harvey’s LAB benchmark
Harvey builds AI for lawyers at large firms. Its Legal Agent Benchmark hands an agent a partner-style instruction and a folder of client documents: draft this agreement, mark up the counterparty’s redline, pull the economic terms from this convertible note.
The output is a review-ready Word document, graded against a rubric written by practicing lawyers. A task passes only if every criterion passes.
Sierra’s τ²-bench
Sierra builds customer-service agents for large consumer brands. Its τ²-bench puts an agent on an airline, retail, or telecom support desk. The agent has to follow the company’s written policy, look up and change records with tools, and talk a simulated customer through the fix.
A task passes only if the final account state matches what the policy says it should be.
Mercor’s APEX-Agents
Mercor’s APEX-Agents models the work of investment banking analysts, management consultants, and corporate lawyers. Each task was written by someone who did that job at a top firm, and takes a human expert over an hour.
The agent works inside a simulated office — documents, spreadsheets, email, chat, calendar — with 160+ files per case. A task passes only if every expert-written criterion is met.
Zapier’s AutomationBench
Zapier automates work across thousands of apps. Its AutomationBench drops an agent into a simulated company’s stack — Gmail, Sheets, Slack, Salesforce, Zendesk, Jira, HubSpot — and asks it to finish a Sales, Marketing, Operations, Support, Finance, or HR workflow end to end.
The environments hold misleading records on purpose. The agent is scored on objectives completed without breaking a single business rule.
HotpotQA · exact match
The standard academic benchmarks used to compare prompt optimizers: multi-hop question answering (HotpotQA), instruction following (IFBench), and privacy-preserving delegation (PUPA).
Lucidic against DSPy GEPA and the labs’ own prompt optimizers, all on the same base model.
HotpotQA · F1
The standard academic benchmarks used to compare prompt optimizers: multi-hop question answering (HotpotQA), instruction following (IFBench), and privacy-preserving delegation (PUPA).
Lucidic against DSPy GEPA and the labs’ own prompt optimizers, all on the same base model.
IFBench
The standard academic benchmarks used to compare prompt optimizers: multi-hop question answering (HotpotQA), instruction following (IFBench), and privacy-preserving delegation (PUPA).
Lucidic against DSPy GEPA and the labs’ own prompt optimizers, all on the same base model.
PAPILLON (PUPA)
The standard academic benchmarks used to compare prompt optimizers: multi-hop question answering (HotpotQA), instruction following (IFBench), and privacy-preserving delegation (PUPA).
Lucidic against DSPy GEPA and the labs’ own prompt optimizers, all on the same base model.
Research
Discover insights from our ongoing research into AI agent optimization, performance analysis, and evaluation methodologies
FAQ
Lucidic is for teams that want more reliable, higher-performing AI agents. Whether you fine-tune models or rely on off-the-shelf LLMs, Lucidic helps teams systematically optimize all parameters of the agent without manual iteration.
No. We don’t replace your agent architecture or infrastructure. We optimize and train the agents you already have, focusing on reliability, accuracy, and continuous improvement.
No, Lucidic integrates with all major LLM providers and agent frameworks. We don’t rebuild or replace your agents. Instead, we train and improve the components you already use, including prompts, tools, policies, and decision logic.
Lucidic runs structured simulations that systematically vary agent design choices across scenarios. Performance is evaluated against your chosen metrics, and learning algorithms, including evolutionary methods, reinforcement learning, and Bayesian optimization, use this signal to iteratively improve agent behavior over time.
Harness optimization turns your agent’s prompts, tools, context, memory and policies into tunable parameters and keeps the model weights frozen: same model, better agent, at low cost. Model-harness co-training also puts the model weights on the table and trains the model and the harness as one system, with LoRA SFT, on-policy distillation, training modules or GRPO. It costs more and has the highest ceiling.
Your agent traces, existing datasets, and company context. Lucidic groups similar traces and keeps what matters (unusual cases, failures, and one example of each common pattern), then builds a replica of your agent’s world, with its tools, databases and simulated customers, to improve the agent in simulation.
No. Lucidic can work with what you already have. If a training environment or reward signal isn’t fully defined, our team can help design and refine them as part of the engagement. Clearer environments and richer signals enable stronger learning, and we work with customers to progressively build those pieces over time.
Lucidic works with any LLM-powered agent, including customer support agents, coding assistants, data analysis tools, and custom enterprise agents. If it uses an LLM, we can optimize it.
You can retrain. When a better model comes out, you do not start over: your datasets and learnings carry over to the new model.
Yes. Lucidic is SOC 2 Type II and HIPAA compliant. Our Trust Center at trust.lucidic.ai has our security documentation and how to request our reports.
