We always tend to use the latest, best, and strongest tools. Many people update their iPhones every year without considering the cost-benefit of spending a thousand dollars.
The same logic applies to GenAI development. With model updates arriving every month, we are quick to update our environments. In doing so, we tend to forget that what we consider "obsolete" now was once the frontier.
But when it comes to scaling a product, model cost hits you in the face.
Let's put this in perspective. Running the same volume of requests on an expensive frontier model like Claude Opus versus a cheap, capable model like Claude Haiku can mean a 15x to 20x difference in API costs. If you are building a product that processes millions of interactions, that difference translates directly into the viability of your business model.
Which model do you choose? Can you get the same result quality from a cheaper model?
The answer is yes, but you have to change how you prompt.
Consider models as your employees. A frontier model is like your senior developer: you throw an open-ended, ambiguous problem at them, and they return with a solid solution. A smaller, cheaper model is like your junior developer: they are highly capable of execution, but you need to give them a detailed roadmap and tell them step-by-step what to do.
The Math: Why Cost Optimization is Not Optional
Let’s look at a real-world scenario I encountered. We needed to run a batch job scoring 400,000 images based on visual quality, composition, and lighting.
Let’s assume an average prompt footprint of 2,000 input tokens (image payload metadata + detailed criteria) and 500 output tokens per run.
- Total Input Tokens: 400,000 × 2,000 = 800 Million tokens
- Total Output Tokens: 400,000 × 500 = 200 Million tokens
Using pricing rates for a premium model versus a lightweight model:
- Claude 3.5 Sonnet / Opus (Estimated $15/M input, $75/M output):
- Input Cost: $12,000
- Output Cost: $15,000
- Total Cost: $27,000
- Claude 3.5 Haiku ($0.80/M input, $4.00/M output):
- Input Cost: $640
- Output Cost: $800
- Total Cost: $1,440
By shifting the workload to a cheaper model, we reduced the cost from $27,000 to $1,440—saving over $25,500 on a single run, while gaining a massive boost in execution speed.
But how do you ensure the cheaper model doesn't output garbage?
4 Rules to Make Cheaper Models Perform Like Frontier Models
To get Claude Haiku (or GPT-4o-mini) to match the accuracy of Sonnet or Opus, you need to structure your prompt as a deterministic roadmap. Here is the blueprint:
1. Deconstruct the Thought Flow (Chain of Thought)
With a frontier model, you can ask: "Score this image from 1 to 10." The model implicitly reasons about composition, lighting, and focus before outputting a number.
If you ask a cheap model to do that, it will hallucinate a random number based on surface-level associations. Instead, you must explicitly guide its chain of thought:
- First, evaluate the focus and sharpness of the primary subject.
- Second, evaluate the lighting exposure (overexposed, underexposed, or balanced).
- Third, evaluate the composition (rule of thirds, center-focused, or cluttered).
- Finally, synthesize these evaluations into a final score.
2. Define Strict Scoring Rubrics
Ambiguity is the enemy of smaller models. You must explicitly define what each score represents:
- 9–10 (Professional): Perfect sharp focus, balanced natural lighting, no noise, strong intentional composition.
- 7–8 (Good): Clear focus, minor lighting imbalances, low noise, acceptable composition.
- 4–6 (Fair): Slight motion blur, noticeable noise, poor composition, but subject is recognizable.
- 1–3 (Poor): Severe blur, heavy over/underexposure, unrecognizable subject.
3. Leverage "Reason-First" JSON Output
JSON output is crucial for downstream data pipelines. However, smaller models often struggle to output valid JSON if they have to calculate a value and format it simultaneously.
The secret is to structure your JSON schema so the model writes its reasoning fields before the final value. Because LLMs generate text token-by-token, writing the explanation first forces the model's weights to compute the logic before committing to a final number.
Here is the exact schema structure we used:
{
"quality_assessment": "<Step-by-step evaluation of sharpness, exposure, and composition based on rules>",
"image_id": "<echo the unique image identifier>",
"quality_score": <integer 1-10; selected strictly using the defined scoring bands and the assessment text above>
}
Note how quality_assessment comes before quality_score. This acts as an embedded Chain-of-Thought inside the structured JSON.
4. Provide Few-Shot Examples
Nothing teaches a smaller model faster than concrete examples. Include 2 or 3 examples in your prompt showing the exact inputs, the step-by-step reasoning, and the final JSON response. This aligns the model's output formatting and tone perfectly.
The Results: The Best of Both Worlds
By applying these prompt engineering techniques to Claude Haiku, we achieved:
- 96.4% Alignment with the scores generated by the expensive frontier model.
- 18.7x Cost Reduction, saving tens of thousands of dollars.
- 3x Faster Latency, speeding up our batch processing pipeline.
In GenAI product development, cost-efficiency is a core feature, not an afterthought. By treating cheaper models like junior developers—giving them detailed instructions, strict guardrails, and structured templates—you can scale your product to millions of users without scaling your API bills.
Are you currently defaulting to the most expensive model out of convenience, or have you built the roadmaps your cheaper models need to succeed?