Introduction

A Large Language Model can be extraordinarily capable while knowing nothing about your organization's latest documents, having no direct access to your database, and being unable to perform even a simple external action unless an application provides the necessary connection.

This distinction is fundamental to modern Generative AI engineering.

The LLM is usually not the application. It is one component inside a larger software system.

A production Generative AI application may retrieve information from external knowledge sources, construct context dynamically, call APIs, query databases, maintain task state, select tools, execute multi-step workflows, validate intermediate results, and determine when human approval is required. The language model provides powerful reasoning and generation capabilities, but the surrounding architecture determines what information and actions are available to it.

This changes the engineering question. Instead of asking only, "Which model should I use?", we begin asking "What system should I build around the model?"

In this final AI Skills Cookbook guide, we will progress from a simple model request to retrieval-augmented generation (RAG), tool calling, structured workflows, and agentic systems. Frameworks will inevitably evolve, but the architectural ideas introduced here are much more durable.

A useful high-level model is:

User Request → Application → LLM + Context → Retrieval / Tools → Workflow → Validation → Response

Layer 1: Calling a Model Through an API

The simplest Generative AI application sends input to a model and receives generated output. Modern model APIs expose this capability programmatically, allowing an application to integrate language intelligence into a conventional software architecture.

Conceptually:

Application
    ↓
Prompt / Input
    ↓
Model API
    ↓
Generated Output
    ↓
Application

A minimal Python example is:

from openai import OpenAI

client = OpenAI()

response = client.responses.create(
    model="gpt-5.6",
    input="Explain gradient descent in three paragraphs."
)

print(response.output_text)

The exact SDK syntax will evolve, which is precisely why understanding the architectural pattern matters more than memorizing an API call. The application constructs an input, sends it to a model endpoint, receives structured output, and decides what to do next.

At this stage, the model can generate useful responses, but its knowledge is primarily bounded by its training and whatever information appears in the current context. If we ask questions about private company policies, today's inventory, a learner's course records, or newly published research, the model needs additional information.

That brings us to context engineering.

Layer 2: Prompt + Context

A prompt is more than the user's question. In an application, the model may receive several kinds of contextual information simultaneously: system-level instructions, the user's request, retrieved documents, previous conversation state, tool results, structured metadata, and formatting requirements.

Conceptually:

Model Input
│
├── Instructions
├── User Request
├── Relevant Context
├── Tool Results
└── Output Requirements

Suppose we are building a university course assistant. Instead of sending only:

When is Assignment 3 due?

the application might construct:

You are a course assistant.

Answer only from the supplied course information.
If the information is insufficient, say so.

COURSE CONTEXT:
Assignment 3: Neural Network Analysis
Due: 18 October 2026, 11:59 PM
Weight: 15%

QUESTION:
When is Assignment 3 due?

The model has not permanently learned this information. We have temporarily placed the relevant information into its context window.

This is an important distinction. Training changes model parameters. Context changes the information available to the model during a particular inference.

Many practical Generative AI systems gain enormous capability not by modifying the underlying model but by becoming much better at deciding what context the model should receive, when it should receive it, and in what form.

Why We Cannot Simply Put Everything in the Prompt

Suppose an organization has 100,000 internal documents. Even with increasingly large context windows, sending the entire document collection with every request would be inefficient and usually undesirable.

More context is not automatically better context.

Irrelevant material consumes tokens, increases latency and cost, and can make it harder for the model to focus on the evidence that actually matters. Sensitive information may also require access controls that prevent certain documents from entering particular users' contexts.

The application therefore needs a mechanism for selecting a small amount of relevant information from a much larger knowledge collection.

This is the problem that retrieval solves.

Layer 3: Embeddings and Semantic Retrieval

Traditional keyword search looks for overlapping terms. Semantic retrieval attempts to identify content with similar meaning, even when the wording differs.

One common approach represents text as numerical vectors called embeddings.

Conceptually, an embedding function maps some content $x$ into a vector:

where $d$ is the dimensionality of the embedding space.

Texts with related meanings tend to occupy nearby regions of this space. A query such as "When must I submit the neural network project?" may retrieve a passage containing "Assignment 3 deadline: 18 October 2026" even though the exact words differ.

A common similarity measure is cosine similarity:

The retrieval system embeds the query, compares that representation against stored document embeddings, and selects the most relevant chunks.

Documents → Chunking → Embeddings → Vector Index

User Question → Query Embedding → Similarity Search → Relevant Chunks

This connects directly with the ideas explored in The Science of AI. Learned representations allow semantically related information to occupy meaningful regions of a high-dimensional representation space. Generative AI engineering turns that theoretical concept into a practical retrieval mechanism.

Layer 4: Retrieval-Augmented Generation

Once relevant information has been retrieved, it can be supplied to the LLM along with the user's question. This architecture is called Retrieval-Augmented Generation, or RAG.

User Question
      ↓
Retrieve Relevant Information
      ↓
Construct Prompt + Context
      ↓
LLM
      ↓
Grounded Response

Suppose the retrieved passages are stored in context:

prompt = f"""
Answer the question using only the supplied context.
If the context does not contain enough information,
say that the answer cannot be determined.

CONTEXT:
{context}

QUESTION:
{question}
"""

The crucial word is grounded. Rather than asking the model to answer entirely from information encoded in its parameters, we ask it to generate an answer grounded in evidence retrieved from an external source.

This provides several advantages. Knowledge can be updated without retraining the model. Private organizational information can be made available selectively. Responses can potentially cite their supporting sources. The knowledge system can also be maintained independently from the foundation model.

However, RAG does not magically eliminate hallucinations. Retrieval may return the wrong passages, relevant information may be absent, documents may conflict, or the model may misinterpret correct evidence.

RAG Is Really Two Problems

It is useful to separate a RAG system into retrieval quality and generation quality.

Suppose the correct answer exists in the knowledge base but the retriever fails to return the relevant document. The language model never receives the evidence, so improving the generation prompt may accomplish very little. Conversely, the retriever may return exactly the correct passage while the model produces an inaccurate interpretation.

This is not a universal evaluation equation, but it captures an important engineering principle: end-to-end quality depends on multiple components succeeding together.

Advanced RAG systems may therefore include query rewriting, hybrid keyword-and-vector retrieval, metadata filtering, reranking, contextual compression, multiple retrieval stages, citation verification, and specialized evaluation.

Layer 5: Giving Models Tools

Retrieval gives a model access to information. Tools give it access to capabilities.

A language model should not invent current inventory. The application could instead expose a function:

def get_inventory(product_id):
    ...

The model's role is not necessarily to execute arbitrary code itself. Instead, the application can provide descriptions of permitted tools and their expected arguments. The model determines when a tool is appropriate and generates a structured request. The application validates that request, executes the actual function, and returns the result to the model.

User Request
     ↓
LLM
     ↓
Tool Request
     ↓
Application Validates Request
     ↓
Tool / API
     ↓
Result
     ↓
LLM
     ↓
Final Response

This architecture separates reasoning about an action from executing the action. The model proposes. The surrounding software controls what is actually allowed to happen.

Structured Outputs Matter

Natural language is excellent for communicating with humans but often unreliable as an interface between software components.

A structured result can be more useful:

{
  "intent": "refund_request",
  "amount": 2500,
  "currency": "INR",
  "requires_review": true
}

Modern Generative AI applications increasingly use schemas to constrain model outputs so that they can be validated and consumed reliably by other components. The LLM is not always generating the final text a user sees; sometimes it functions as a semantic transformation component inside a larger program.

Layer 6: Workflows Before Agents

Suppose an application needs to perform:

Receive Question
      ↓
Classify Request
      ↓
Retrieve Documents
      ↓
Generate Draft
      ↓
Verify Evidence
      ↓
Return Answer

This is a workflow. The sequence is largely predetermined by application logic. For many production applications, this is desirable because workflows are relatively predictable, testable, observable, and easy to constrain.

def answer_question(question):
    category = classify(question)
    documents = retrieve(question, category=category)
    draft = generate_answer(question, documents)
    verified = verify_answer(draft, documents)
    return verified

The objective should not be to maximize autonomy. It should be to use only as much autonomy as the task requires.

Layer 7: What Makes a System Agentic?

An agentic system gives the model greater responsibility for deciding what should happen next.

Goal
 ↓
Observe State
 ↓
Decide Next Action
 ↓
Use Tool
 ↓
Observe Result
 ↓
Continue or Finish?
 ↺

We can represent this abstractly as:

where $s_t$ represents the current state, $a_t$ the selected action, and $o_{t+1}$ the resulting observation.

This enables systems capable of research, coding, data analysis, and multi-stage information gathering. But greater autonomy also creates greater uncertainty. Agent engineering is therefore as much about control as capability.

State and Memory Are Not the Same Thing

Agent discussions frequently use the word memory, but several different concepts are often being combined. A system may maintain short-term conversational state, persistent user preferences, task state, or externally retrieved knowledge. These are different architectural problems.

state = {
    "question": question,
    "sources_found": [],
    "claims_verified": [],
    "open_questions": [],
    "status": "researching"
}

This is not human-like memory. It is explicit application state. Making state explicit improves observability because developers can inspect what the system believes has happened and determine why a particular action was selected.

Frameworks Are Implementations, Not the Architecture

Libraries such as LangChain and LangGraph can help developers implement retrieval systems, tool-using applications, stateful workflows, and agents. They can reduce boilerplate and provide useful abstractions.

But the framework should not become the mental model.

If you understand:

Model → Context → Retrieval → Tools → State → Workflow → Evaluation

you can implement those ideas using a framework, a cloud platform, an SDK, or ordinary Python functions.

Where LangChain and LangGraph Fit

As applications become more complex, developers repeatedly need similar capabilities: prompt construction, model invocation, document retrieval, tool schemas, state management, branching logic, retries, persistence, tracing, and evaluation.

A linear sequence might resemble:

Input → Retriever → Prompt Builder → Model → Output Parser

while graph-oriented orchestration becomes particularly useful when a workflow contains branching, loops, checkpoints, human approvals, or persistent state.

The underlying idea is broader than any particular AI framework: complex software becomes easier to reason about when control flow is explicit.

Human-in-the-Loop Architecture

Some decisions should not be delegated entirely to an AI system.

Agent Produces Proposed Action
           ↓
Policy Validation
           ↓
Human Approval Required?
       ↙          ↘
     Yes           No
      ↓             ↓
 Human Review     Execute
      ↓
 Approve / Reject

Human-in-the-loop design identifies boundaries where human judgment, authority, or accountability remains necessary. As model capabilities expand, thoughtful allocation of responsibility between humans and machines becomes an architectural requirement rather than merely an ethical aspiration.

Evaluating Generative AI Systems

Traditional machine learning evaluation often focuses on a model and a metric. Generative AI applications require evaluation across an entire system.

Retrieval
   ↓
Did we find the right evidence?

Generation
   ↓
Did the answer correctly use that evidence?

Tool Use
   ↓
Was the correct tool selected with valid arguments?

Workflow
   ↓
Did the system follow the appropriate path?

Outcome
   ↓
Did the user receive a correct and useful result?

Once an LLM becomes part of a larger system, model quality and application quality are no longer the same thing. A stronger model cannot automatically repair a poorly designed retrieval layer, insecure tool permissions, or an incoherent workflow.

Security: Treat Model Inputs as Untrusted

Connecting models to external data and tools creates new security boundaries. A retrieved document may contain instructions that were never intended for the model to follow. A malicious user may attempt to manipulate the prompt. A tool may expose more functionality than necessary. Sensitive information may accidentally enter model context or logs.

Tools should operate with least privilege. Inputs should be validated. High-impact actions should require authorization. Retrieved content should be treated as data rather than automatically trusted instructions. Secrets should remain outside prompts and source code. Logs should provide traceability without unnecessarily exposing sensitive information.

The model is part of the security boundary, not outside it.

A Complete Generative AI Architecture

                     USER
                       ↓
                APPLICATION LAYER
                       ↓
              Instructions + State
                       ↓
                ┌──────┴──────┐
                ↓             ↓
           RETRIEVAL          LLM
                ↓             ↓
        Knowledge Sources  Decision
                ↓             ↓
          Relevant Context   Tools
                └──────┬──────┘
                       ↓
                  WORKFLOW
                       ↓
              Validation / Policy
                       ↓
               Human Approval?
                       ↓
                   RESPONSE

A real system does not necessarily require every component. Architecture should follow requirements.

The most sophisticated design is not the one containing the most components. It is the one that introduces exactly the complexity necessary to solve the problem reliably.

From Model-Centric to System-Centric AI

At first:

Prompt → Model → Response

Then:

Question → Retrieval → Context → Model → Response

Then:

Question → Model → Tool → Observation → Model → Response

Finally:

Goal
 ↓
State
 ↓
Reason / Decide
 ↓
Retrieve or Act
 ↓
Observe
 ↓
Validate
 ↓
Continue / Approve / Finish

Each stage makes the surrounding system more important.

This is why Generative AI engineering increasingly resembles systems engineering. The model remains central, but reliability emerges from how the model interacts with software, information, permissions, people, and the external world.

Conclusion

This article completes a journey that began with setting up Python and Jupyter.

In the first AI Skills Cookbook guide, we constructed a reproducible computational workspace. We then learned to transform raw data using NumPy, Pandas, and visualization. Next, we built a complete classical machine learning workflow with preprocessing, pipelines, validation, and evaluation. PyTorch then allowed us to look beneath .fit() and understand tensors, neural networks, backpropagation, and optimization.

Now the direction has changed.

Rather than training increasingly large models ourselves, we have learned how to build systems around powerful pretrained models.

Retrieval gives those systems access to external knowledge. Embeddings make semantic search possible. RAG grounds generation in retrieved evidence. Tools allow models to interact with software and external services. Workflows impose structure. Agents introduce adaptive decision-making. State preserves information across steps. Human approval creates accountability boundaries. Evaluation tells us whether the complete system actually works.

That progression captures one of the defining ideas of modern AI engineering:

The foundation model provides intelligence; the application architecture determines how that intelligence becomes useful.

Key Takeaways

• A Large Language Model is usually one component of a larger Generative AI application, not the complete application itself.
• Context engineering determines what instructions, evidence, state, and tool results the model receives during inference.
• Embeddings represent semantic information numerically and enable similarity-based retrieval.
• Retrieval-Augmented Generation (RAG) combines external knowledge retrieval with model generation so responses can be grounded in relevant evidence.
• RAG quality depends separately on retrieval quality and the model's ability to interpret retrieved information correctly.
• Tool calling allows an LLM to request external capabilities while the surrounding application retains control over execution.
• Structured outputs allow models to participate reliably in software workflows rather than communicating exclusively through prose.
• Deterministic workflows are often preferable when the required sequence of operations is already known.
• Agents are useful when the system genuinely needs to choose actions dynamically based on intermediate observations.
• State, conversational history, persistent memory, and external knowledge are different architectural concepts and should be designed separately.
• Human approval should remain part of workflows where actions have significant financial, legal, safety, privacy, or organizational consequences.
• Generative AI systems require evaluation at the retrieval, generation, tool-use, workflow, and end-to-end outcome levels.
• Frameworks such as LangChain and LangGraph can implement these patterns, but architecture is the durable skill; frameworks are implementations.
• Secure Generative AI engineering requires least-privilege tools, validation, authorization, privacy controls, and careful treatment of untrusted content.
• The progression from LLM → RAG → Tools → Workflows → Agents is fundamentally a progression from model-centric AI toward system-centric AI engineering.