Out-of-the-box AI models are generalists. They know a little about everything but may not know your business—your terminology, your products, your processes, your voice.
Customization makes AI speak your language. But it comes at a cost.
The question isn't whether to customize—it's when the investment pays off.
The Business Operator's decision:
| Cost Component | Typical Range | Notes |
|---|---|---|
| Embedding generation | $0.0001/1K tokens | One-time per document |
| Vector database | $0-100/month | Pinecone, Weaviate, Supabase |
| Retrieval per query | ~$0.0002 | Minimal compute |
| Augmented prompt tokens | +500-2000 tokens | Added context increases cost |
Example RAG cost calculation:
Compare to: Hiring someone to answer those questions
| Cost Component | OpenAI GPT-4o-mini | OpenAI GPT-4o |
|---|---|---|
| Training cost | $3/million tokens | $25/million tokens |
| Inference cost | 2x base model | 2x base model |
| Minimum examples | 10 (50+ recommended) | 10 (50+ recommended) |
| Training time | Minutes to hours | Hours |
Example fine-tuning cost calculation:
The hidden cost: Creating high-quality training data requires expert time. If you need 500 examples of perfect customer support responses, someone has to write them.
Most production systems use multiple approaches:
User query
↓
[RAG] Retrieve relevant knowledge from your docs
↓
[Fine-tuned model] Generate response in your brand voice
↓
[Prompt guard] Ensure output meets formatting requirements
↓
Response to user
Example: Customer Support Bot
| Approach | Build In-House | Use Managed Service |
|---|---|---|
| RAG | Pinecone + custom pipeline | LangChain, Anthropic Claude Projects |
| Fine-tuning | OpenAI API + data prep | Jasper, Copy.ai (domain-specific) |
| Full custom | Azure OpenAI + enterprise | Specialized vendors |
When to buy:
When to build:
AI customization exists on a spectrum from cheap-and-quick to expensive-and-permanent:
| Approach | Cost | Time | Flexibility | Best For |
|---|---|---|---|---|
| Prompt Engineering | $0 | Minutes | High (change anytime) | Most use cases |
| Few-Shot Examples | $0 | Hours | High | Pattern matching, formatting |
| RAG (Retrieval) | $-$$ | Days | Medium | Knowledge bases, docs |
| Fine-Tuning | $$-$$$ | Weeks | Low (retraining needed) | Style, specialized domains |
| Custom Training | $$$$ | Months | Very Low | Unique capabilities |
Key insight: Most projects should start at the top of this spectrum and move down only when necessary.
RAG = Retrieve relevant context → Augment the prompt → Generate response
Instead of training knowledge into the model, you look it up at query time.
How it works:
User asks: "What's our refund policy for premium members?"
↓
Step 1: RETRIEVE — Search your knowledge base
→ Finds: "Premium members get 30-day refunds..."
↓
Step 2: AUGMENT — Add context to prompt
→ Prompt: "Using this policy [context], answer: ..."
↓
Step 3: GENERATE — Model produces answer
→ "Premium members are entitled to full refunds within 30 days..."
RAG advantages:
RAG limitations:
Fine-tuning = Training a model on your examples to change how it behaves.
Not about what the model knows—about how it responds.
Good fine-tuning use cases:
Poor fine-tuning use cases:
Ask these questions in order:
1. Can prompt engineering solve this?
2. Is the problem knowledge or behavior?
3. What's the volume?
4. How fast do requirements change?
Don't customize when:
Red flags that you're over-engineering:
The 80/20 rule applies: 80% of AI value comes from basic prompting + good workflow design. 20% comes from advanced customization.
Don't pursue the 20% until you've captured the 80%.
CUSTOMIZATION ROI WORKSHEET
1. CURRENT STATE
Manual handling time per task: ___ minutes
Tasks per month: ___
Loaded labor cost per hour: $___
Monthly labor cost: $___
2. WITH AI (BASIC PROMPTING)
Accuracy rate (usable without edits): ___%
Time saved per usable task: ___ minutes
Monthly time saved: ___ hours
Monthly value of time saved: $___
3. WITH CUSTOMIZATION
Expected accuracy improvement: +___%
Additional monthly value: $___
4. CUSTOMIZATION COSTS
One-time setup:
- RAG infrastructure: $___
- Fine-tuning training: $___
- Human data prep time: $___
Ongoing monthly:
- Compute increase: $___
- Maintenance: $___
5. PAYBACK CALCULATION
Incremental monthly value: $___
Incremental monthly cost: $___
Net monthly benefit: $___
One-time investment: $___
Payback period: ___ months
You're the operations lead at a mid-size law firm. Attorneys waste hours drafting routine documents that follow standard templates. You want to deploy AI assistance.
Analyze these three use cases:
Use Case A: Contract summarization
Use Case B: Standard letter drafting
Use Case C: Legal research assistance
For each use case, determine:
Primary need: Knowledge, behavior, or both?
Recommended approach: Prompt engineering, RAG, fine-tuning, or hybrid?
Cost-benefit analysis:
Build vs. buy decision: In-house or vendor?
Risk assessment: What could go wrong?
You've mastered Agentic Orchestration. You can now:
Before moving to Phase 4, complete:
Lab 5: Build an AI Assistant — Create a functional AI assistant using no-code tools
Lab 5b: Multi-Agent Orchestration — Design a system where AI agents collaborate
Phase 3 Deliverable: Multi-Agent System — Build and deploy a working multi-agent system that demonstrates autonomous reasoning and collaboration
You've reached the end of this module. Review the checklist below to ensure you've understood the key concepts.