Making GPT-4.1-mini as Good as GPT-5.2

The gap between frontier models and their smaller counterparts has always been significant. GPT-5.2 achieves 74.3% on τ²-bench, while GPT-4.1-mini languishes at 53%. Most teams accept this gap as the cost of using a faster, cheaper model.

We didn't.

Using Lucidic's optimization algorithms, we improved GPT-4.1-mini from 53% to 72% on τ²-bench — within striking distance of GPT-5.2's performance, at a fraction of the inference cost.

The Optimization Landscape

Comparing Optimization Approaches

We benchmarked Lucidic against the leading prompt optimization frameworks: DSPy GEPA, OpenAI's Optimizer, and Anthropic's Optimizer. The results speak for themselves:

Lucidic

72%

DSPy GEPA

53%

Baseline

53%

OpenAI Optimizer

46.5%

Anthropic Optimizer

42.5%

Lucidic achieved a 36% relative improvement over the baseline, while DSPy GEPA showed no improvement. OpenAI and Anthropic's optimizers actually degraded performance — a common failure mode when optimization algorithms overfit to training examples.

Approaching Frontier Performance

The more striking result is how close optimized GPT-4.1-mini gets to frontier model performance:

GPT-5.2 Baseline

74.3%

GPT-4.1-mini + Lucidic

72%

GPT-4.1-mini Baseline

53%

GPT-4.1-mini + Lucidic reaches 97% of GPT-5.2's baseline performance. For most production use cases, this difference is negligible — but the cost savings are substantial.

Why This Matters

The Economics of AI

GPT-5.2 costs roughly 20x more per token than GPT-4.1-mini. For high-volume applications, this difference compounds quickly. A company processing 10 million customer service interactions per month could save hundreds of thousands of dollars by using an optimized smaller model instead of a frontier model.

The Technical Insight

The key insight is that smaller models aren't fundamentally less capable — they're undertrained for specific tasks. The knowledge exists in the weights; it just needs the right prompts to surface it.

Lucidic's optimization algorithms systematically explore the prompt space, finding the specific phrasings, examples, and structures that unlock the model's latent capabilities. This isn't about making the model do something it can't do — it's about helping it do what it already knows how to do.

Methodology

We evaluated on τ²-bench, Sierra's benchmark for conversational AI agents. The benchmark pairs two LLM agents — one as a customer service agent, one simulating a customer — and measures task completion across hundreds of realistic scenarios.

All experiments used the same base model (GPT-4.1-mini) and evaluation protocol. We ran each configuration 5 times to account for stochastic variance and report the mean score.

The baseline represents GPT-4.1-mini with a standard customer service prompt. Each optimizer was given the same training budget: 100 optimization steps with access to 50 training examples.

Implications

This result challenges the assumption that you need frontier models for frontier performance. With the right optimization, smaller models can punch far above their weight class.

For teams building production AI systems, this opens up new possibilities:

- Lower latency: Smaller models respond faster

- Lower cost: 20x cost reduction per token

- Better scaling: More requests per GPU

The frontier isn't just about model size anymore. It's about how intelligently you deploy the models you have.

If you're interested in seeing what Lucidic can do for your models, get in touch.