EVERYTHING YOU NEED TO KNOW ABOUT AI AGENTS: PROMPT ENGINEERING, RAG, AND FINETUNING (PART 2)

Plenty of people use ChatGPT, Gemini, or a sales AI Agent every day without ever asking: why does the AI sometimes nail exactly what you meant, sometimes stay frustratingly generic, and sometimes confidently invent facts about your own orders or company policy? The answer comes down to three foundational techniques that every mature AI Agent relies on at least one of: Prompt Engineering, RAG, and Fine-tuning. Understanding what prompt engineering is and what RAG is will help you judge whether an AI Agent is genuinely usable for a business — or just running an impressive demo on stage.

1.Recap: How Does an AI Agent “Talk” to an LLM?

In Part 1, we compared the AI industry to a giant tree. The Roots are the LLMs (Large Language Models) like GPT, Gemini, Claude, Llama, and DeepSeek — where the core process of collecting and training data happens. In reality, only around 10 major LLMs dominate the entire space. The trunk and Branches split by domain — language, image, video, audio — with fewer than 30 significant branches. The Leaves are the AI applications, and AI Agents make up the largest share — hundreds of thousands of them, growing every day, even though most rely on the same limited set of Roots and Branches behind a flashy marketing layer.

Each leaf — each AI Agent — talks to the roots in its own way, but mainly through three techniques: Prompt Engineering, RAG, and Fine-tuning. The next three sections break down each one with concrete examples.

2.What Is Prompt Engineering?

Every time you ask ChatGPT or Gemini a question, you’re really feeding it a prompt — and the craft of writing effective prompts is called prompt engineering. Not everyone knows how to write a prompt good enough to get the result they actually want; in fact, some AI Agents exist solely to optimize prompts for other users, and still make money doing it.

For example, instead of a plain prompt like “Write an article about the differences between B2B and B2C CRM,” a refined prompt could be: “Act as Nam Nguyen, author of the book ‘Winning Over CRM,’ and write an article…”, or “Imagine your readers are fresh graduates who’ve never heard of CRM, then write an article…”. At a more advanced level, a prompt can even set up multiple branching scenarios with specific conditions to produce more relevant, more token-efficient answers.

3.What Is RAG?

Prompt engineering alone can’t manually map prompts to every specific context, because real-world requests are too varied. It works well for general topics, but for specific queries — a customer complaint, looking up order details, or accessing proprietary business data — an AI Agent needs to pull in outside information. This technique is called RAG (Retrieval-Augmented Generation). External data sources can include:

• Previous chat history with a customer
• A company’s private database (e.g. customer data, orders inside a CRM)
• Structured or unstructured data from various formats

Implementing RAG well is its own art — balancing response speed, data security, and token cost.

4.What Is Fine-Tuning?

Fine-tuning is the process of retraining an AI Agent by adjusting the model’s weights using data that’s more precise or more current than the LLM’s original training data. Here’s a step-by-step comparison:

• Regular prompt: “Write a poem about CRM.”
• Refined prompt (prompt engineering): “Write a poem about CRM in Nam Nguyen’s writing style.”
• Prompt with RAG: “Write a poem about CRM in Nam Nguyen’s writing style from line 2, page 129 of ‘Winning Over CRM.'” (this requires loading the entire book as an external data source)
• Fine-tuning: if the poem the AI produces still sounds outdated or generic, you retrain the AI on all of Nam Nguyen’s 2024 CRM-related Facebook posts so it picks up the latest industry concepts.

In essence:

• RAG ensures factual accuracy (facts).
• Fine-tuning improves style and relevance (form).
• If a prompt lacks context — use RAG.
• If an answer is factually correct but lacks the right nuance — use fine-tuning.

5.When Should You Use Prompt Engineering, RAG, or Fine-Tuning?

The current AI landscape shows that an AI Agent can use prompt engineering, RAG, fine-tuning, or a combination of RAG and fine-tuning (known as RAFT). At present, however, RAG remains the most widely used technique — because most real business needs call for AI that answers correctly from constantly changing internal data, more than AI that writes in someone’s exact voice.

Applying This in Practice

Most businesses that are new to AI Agents tend to stop at prompt engineering — using one general-purpose chatbot to answer product questions, draft emails, or summarize calls. But the moment a customer asks about a specific order, their interaction history, or pricing tied to their customer segment, plain prompt engineering runs out of steam, because the AI has no access to that data.

That’s exactly when RAG becomes necessary — and when a structured customer data system, like a CRM, becomes the ideal external data source for an AI Agent to query. An AI Agent connected to a CRM can accurately answer “what did customer B buy, when, and where are they in the pipeline” — instead of giving a generic answer or making something up. Fine-tuning, by contrast, usually only becomes worthwhile once a business has accumulated enough of its own data (brand voice, industry terminology) to make retraining worth the cost.

Final Thoughts

Prompt engineering, RAG, and fine-tuning aren’t three competing choices — they’re three complementary layers of capability on the path from an AI Agent that “can talk” to one that’s actually usable inside a business. Understanding what prompt engineering is helps you give the right instructions; understanding what RAG is lets you demand that an AI Agent answer using your company’s real data; and understanding fine-tuning tells you when it’s worth investing further. In Part 3 of this series, we’ll look at how AI Agents are actually deployed inside a B2B sales process.

Frequently Asked Questions

What is prompt engineering?+

Prompt engineering is the craft of designing a prompt so the AI understands the real intent and returns a result that closely matches the need, without changing the underlying model. For example, instead of asking generically “write an article about CRM,” a refined prompt spells out the role, the audience, and the desired format.

What is RAG?+

RAG (Retrieval-Augmented Generation) is a technique that lets an AI Agent pull in outside data — such as chat history, a company’s private database, or internal documents — before generating an answer, instead of relying only on knowledge baked into the model. RAG keeps answers accurate and current against real data.

How is fine-tuning different from RAG?+

RAG injects outside data into each individual answer (ensuring factual accuracy), while fine-tuning retrains the model’s weights to change its style, terminology, or reasoning long-term (improving form). If a prompt lacks context, use RAG; if the answer is factually correct but lacks the right nuance, use fine-tuning.

Should a small business start with prompt engineering, RAG, or fine-tuning?+

Start with prompt engineering — it’s the cheapest and fastest to deploy. Move to RAG once you need the AI to answer using your company’s own data (orders, policies, customer history). Fine-tuning is only worth considering once you have a large enough dataset and a very specific need around tone or domain expertise.

Which technique is most commonly used in AI Agents today?+

Based on current analysis of AI Agents in production, RAG is the most widely used technique, because it strikes the best balance between cost, deployment speed, and accuracy — especially for businesses that need AI to answer using internal data that keeps changing.