Everything You Need to Know About AI Agents: Prompt Engineering, RAG, and Finetuning (Part 2)

To keep things clear before diving deeper into the topic of AI agents, imagine a giant tree:

• The Roots represent LLMs (Large Language Models) like GPT, Gemini (Google), Claude, Llama (Meta), and DeepSeek—where the core Machine Learning process of data collection and training happens. In reality, only about 10 major LLMs dominate the space.
• The trunk and main branches are split into different areas—some are focused on language, and others are focused on images, video, or music. Examples include DALL-E2, Imagen, Claude, and Gemini. There are likely fewer than 30 significant branches.
• The Leaves represent AI applications. Today, there are hundreds of thousands of them, with new ones appearing daily. Many AI-powered apps might sound impressive in marketing, but in the end, they all rely on the same limited set of Roots and Branches.

AI Agents are like the leaves. Each leaf communicates with the Roots in different ways, primarily through Prompt Engineering, RAG, and Finetuning.

‍
1. Prompt Engineering

Whenever we ask ChatGPT or Gemini a question, we are essentially providing a Prompt—and the art of crafting effective prompts is called Prompt Engineering. However, not everyone knows how to write effective prompts to get the desired results.Some AI Agents exist solely to optimize prompts and still make money doing so. For example, instead of a simple prompt like:🟢 "Write an article about the differences between B2B and B2C CRM,"A refined prompt could be:

• "Act as Nam Nguyen, the author of the book ‘Winning Over CRM,’ and write an article..."
• "Imagine your readers are fresh graduates who have never heard of CRM. Write an article..."
• Or a more advanced version could set up a multi-branch scenario with specific conditions to generate more relevant and cost-effective responses (by optimizing token usage).

2. RAG (Retrieval-Augmented Generation)

Prompt Engineering alone cannot map prompts to specific contexts manually because of the vast variety of requests. It works well for general topics, but when dealing with specific queries—such as customer complaints, retrieving order details, or accessing proprietary business data—AI Agents need to pull external information.This approach is called RAG (Retrieval-Augmented Generation).External data sources could include:

• Previous chat logs
• A company’s private database
• Structured or unstructured data from various formats

Implementing RAG effectively is an art—balancing speed, security, and cost efficiency in token usage.
‍

3. Finetuning

Finetuning involves retraining AI Agents by adjusting model weights using more precise or updated datasets than the original LLM training data.Here’s a simple comparison:🟢 Regular Prompt: "Write a poem about CRM."

‍Enhanced Prompt (Prompt Engineering): "Using Nam Nguyen’s writing style, write a poem about CRM."

Prompt with RAG: "Using Nam Nguyen’s writing style from line 2, page 129 of the book ‘Winning Over CRM,’ write a poem about CRM." (This requires loading the book as an external database.) 🟢 Finetuning: If the system generates a poem that is outdated or sounds generic, we train the AI by feeding it all of Nam Nguyen’s CRM-related Facebook posts from 2024 so it can learn the latest industry concepts.In essence:

• RAG ensures factual accuracy (facts).
• Finetuning refines the style and relevance (form).
• If a prompt lacks context, use RAG.
• If a response is factually correct but lacks relevance or nuance, use Finetuning.

The Current AI LandscapeAI Agents can use Prompt Engineering, RAG, Finetuning, or a mix of RAG + Finetuning (RAFT). However, at present, RAG is the most commonly used approach.