Module 1: Large Language Models & Enterprise NLP

Introduction

From Enterprise Text Processing to Large Language Models

Language sits at the centre of enterprise work. Organisations operate through contracts, policies, reports, emails, customer conversations, technical documentation, meeting notes, support tickets, research papers, and internal knowledge repositories. For decades, software systems could store and search this information, but they struggled to interpret meaning with the flexibility of a human reader.

Natural Language Processing, or NLP, emerged to address this challenge. Early NLP systems relied heavily on handcrafted rules, dictionaries, and statistical methods. They could identify keywords, classify short messages, or extract structured information from predictable documents, but they often failed when language became ambiguous, contextual, domain-specific, or conversational.

Modern Large Language Models represent a major shift. Instead of depending entirely on manually written linguistic rules, they learn statistical patterns from enormous collections of text. Their ability to represent words numerically, identify relationships across long passages, and generate coherent language has transformed how enterprises approach search, summarisation, document analysis, customer support, software development, and knowledge management.

However, using an LLM effectively requires more than sending a prompt to an API. Enterprise AI professionals must understand the architecture beneath the interface. They need to know how text becomes tokens, how tokens become embeddings, how attention identifies relevant relationships, how transformer layers process context, and how model size, context length, latency, security, and cost affect deployment decisions.

Enterprise Language Data
        ↓
Tokenization
        ↓
Embeddings
        ↓
Attention and Transformer Layers
        ↓
Large Language Model
        ↓
Enterprise Application

This module develops that understanding. You will move from the historical evolution of NLP to the internal structure of modern LLMs. You will also work with practical examples using Hugging Face, sentence-transformers, and OpenAI-compatible APIs. The purpose is not to train a frontier model from scratch. Instead, the aim is to understand the components well enough to make sound architectural and engineering decisions.

Why This Matters in Enterprise AI

Enterprises rarely use language models in isolation. Models are integrated into larger systems containing business rules, retrieval pipelines, databases, user permissions, monitoring, compliance controls, and human approval. A technically impressive model can still produce poor business outcomes if it is selected without considering domain requirements, deployment constraints, or governance.

For example, a legal document assistant may require long-context processing and strong source traceability. A customer-service assistant may prioritise low latency and predictable tone. A healthcare application may require private deployment, auditability, and strict human oversight. The same model is not automatically suitable for all three.

By understanding how LLMs process language, you will be better prepared to evaluate model behaviour, design prompts, interpret limitations, select appropriate architectures, and explain technical trade-offs to enterprise stakeholders.

Learning Objectives

By the end of this module, you should be able to:

• Explain how Natural Language Processing evolved from rule-based systems to transformer-based language models.
• Describe the role of tokens and tokenization in modern LLM pipelines.
• Explain how embeddings represent semantic meaning numerically.
• Describe the purpose of self-attention and why it improved language modelling.
• Explain the major components of the transformer architecture.
• Interpret the high-level architecture of a decoder-based Large Language Model.
• Explain how context windows affect model capability, performance, and cost.
• Apply advanced prompt engineering principles to enterprise use cases.
• Use Hugging Face and sentence-transformers for basic NLP and embedding tasks.
• Call an OpenAI-compatible model through a structured API workflow.
• Compare language models using capability, latency, privacy, deployment, and cost criteria.
• Identify common enterprise risks associated with language-model deployment.

How to Approach This Module

This is an advanced conceptual and practical module. You are not expected to derive every mathematical equation behind transformers, but you should understand the engineering logic of each component. When you encounter code, focus on what enters the system, what transformation occurs, what output is produced, and how that step contributes to the larger LLM pipeline.

Evolution of Natural Language Processing

Why Language Has Always Been Difficult for Computers

Human language is ambiguous, contextual, and constantly evolving. A word may have several meanings, a sentence may depend on information stated much earlier, and the intended meaning may be shaped by culture, tone, or professional context. This makes language fundamentally different from structured numerical data.

Consider the word bank. It may refer to a financial institution or the side of a river. A traditional software system cannot determine the intended meaning from the word alone. It must examine the surrounding context:

"The company deposited the funds in the bank."

"The hikers rested on the river bank."

The history of NLP can therefore be understood as a sequence of attempts to represent context more effectively.

Rule-Based NLP

Early NLP systems relied on handcrafted grammatical rules, dictionaries, and pattern matching. Engineers explicitly programmed how the system should identify names, classify phrases, or respond to particular words.

if "refund" in customer_message:
    route_to_billing_team()

Rule-based systems were transparent and predictable. They worked reasonably well in narrow environments where vocabulary and sentence structure were controlled. However, they became difficult to maintain as the number of exceptions grew.

A customer might ask for a refund without using the word refund:

"I want my money back."
"Please reverse this charge."
"I was billed incorrectly."

Capturing every variation through manual rules quickly becomes impractical.

Statistical NLP

Statistical NLP shifted the focus from manually written rules to patterns learned from data. Techniques such as n-gram models estimated the probability of a word based on the words that appeared before it.

P(word | previous words)

Statistical methods supported applications such as spam detection, speech recognition, machine translation, and text classification. They were more flexible than rule-based systems, but they still struggled with long-range context and sparse data.

Machine Learning and Feature Engineering

As Machine Learning became widely adopted, NLP systems began representing text through features such as word counts, term frequency, inverse document frequency, and manually selected linguistic properties.

A common approach was the bag-of-words representation. It counted how often words appeared in a document while ignoring word order.

Document A: "AI improves customer service"
Document B: "Customer service improves with AI"

Bag-of-words representation:
AI: 1
improves: 1
customer: 1
service: 1

This was useful for classification and search, but it treated semantically related words as unrelated symbols. The words purchase and buy, for example, would occupy separate dimensions even though they often express similar meaning.

Neural NLP and Distributed Representations

Neural networks introduced distributed representations, where words were represented as dense numerical vectors rather than isolated symbols. This enabled systems to learn semantic relationships directly from data.

Recurrent Neural Networks and Long Short-Term Memory networks improved sequence processing by carrying information from earlier words into later stages. These models supported stronger translation, summarisation, and language generation systems.

However, recurrent models processed tokens largely one after another. This limited parallel computation and made it difficult to preserve information across very long passages.

The Transformer Breakthrough

Transformers changed NLP by using attention as the primary mechanism for relating tokens. Instead of processing a sentence strictly from left to right, a transformer can examine relationships among many tokens in parallel.

Rule-Based NLP
      ↓
Statistical NLP
      ↓
Machine Learning with Engineered Features
      ↓
Neural Networks and Word Embeddings
      ↓
Transformers
      ↓
Large Language Models

This architecture made it possible to train much larger models on far more data. Modern LLMs emerged from scaling transformer-based systems and refining how they are trained, aligned, and adapted for human interaction.

Enterprise Implications

The evolution of NLP has expanded the range of enterprise problems that can be addressed. Earlier systems were suitable for narrow classification tasks. Modern LLMs can work across summarisation, document generation, semantic search, coding, question answering, and multi-step knowledge workflows.

This does not mean older approaches are obsolete. Rule-based validation remains useful when decisions must be deterministic. Statistical classifiers may be faster and cheaper for narrow, high-volume tasks. Enterprise architecture often combines several generations of NLP rather than replacing every earlier method with an LLM.

Tokens & Tokenization

How Language Enters a Large Language Model

A language model does not receive text in the same form that humans read it. Before processing, the text is converted into smaller units called tokens. Tokenization is the process of dividing text into those units and mapping each token to a numerical identifier.

Text:
"Enterprise AI transforms work."

Possible tokens:
["Enterprise", " AI", " transforms", " work", "."]

Token IDs:
[24891, 15592, 43172, 990, 13]

The exact tokens depend on the tokenizer associated with the model. Different models may split the same sentence differently, which means token counts are model-specific.

Why Models Do Not Simply Use Words

Word-level tokenization creates several difficulties. Languages contain enormous vocabularies, new words appear constantly, and many words have related forms:

analyse
analysed
analysing
analysis
analytical

Treating each form as an entirely separate token would make the vocabulary unnecessarily large. Character-level tokenization avoids unknown words, but sequences become much longer and semantic patterns become harder to learn.

Modern LLMs therefore commonly use subword tokenization. Frequent words may remain whole, while uncommon words are divided into reusable parts.

"unpredictable"

Possible subword tokens:
["un", "predict", "able"]

Subword tokenization balances vocabulary size, sequence length, and the ability to represent new terms.

Common Tokenization Approaches

Several algorithms are widely used:

• Byte Pair Encoding (BPE): repeatedly merges frequently occurring symbol pairs.
• WordPiece: selects subword units that improve the likelihood of the training data.
• Unigram Language Model: begins with a large vocabulary and removes less useful token candidates.
• Byte-Level Tokenization: represents text through byte-level units, improving coverage across languages and symbols.

The important enterprise lesson is not to memorise every algorithm. It is to understand that tokenization affects cost, latency, context usage, multilingual performance, and how reliably a model handles domain-specific terminology.

Inspecting Tokens with Hugging Face

The Hugging Face transformers library provides tokenizers associated with many pretrained models.

from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")

text = "Enterprise AI transforms knowledge work."

tokens = tokenizer.tokenize(text)
token_ids = tokenizer.encode(text)

print("Tokens:", tokens)
print("Token IDs:", token_ids)
print("Token count:", len(token_ids))

A typical output may include special tokens added by the model:

Tokens:
['enterprise', 'ai', 'transforms', 'knowledge', 'work', '.']

Token IDs:
[101, 6960, 9932, 21743, 3716, 2147, 1012, 102]

Here, 101 and 102 represent special boundary tokens used by BERT-style models. These tokens help the model understand where the input begins and ends.

Encoding and Decoding

Tokenizers usually support both encoding and decoding.

encoded = tokenizer(
    text,
    return_tensors="pt",
    padding=True,
    truncation=True
)

print(encoded["input_ids"])
print(encoded["attention_mask"])

decoded = tokenizer.decode(encoded["input_ids"][0])
print(decoded)

The input_ids contain the numerical token identifiers. The attention_mask identifies which positions contain real tokens and which positions contain padding.

Tokenization and Context Windows

Context windows are measured in tokens, not words. This matters because a 5,000-word document may require far more than 5,000 tokens. Code, tables, uncommon terminology, and some languages can produce especially high token counts.

Tokenization therefore influences:

• how much content fits into one prompt
• how much an API request costs
• how quickly the model responds
• whether important content is truncated
• how documents should be chunked for retrieval

Tokenization in Enterprise Documents

Enterprise text frequently contains product codes, legal references, technical abbreviations, account numbers, and specialised vocabulary. A general-purpose tokenizer may divide these terms into many fragments.

Domain term:
"ISO27001-compliant"

Possible tokens:
["ISO", "270", "01", "-", "compliant"]

Fragmentation is not always harmful, but it can increase sequence length and reduce efficiency. When evaluating models for highly specialised domains, engineers should inspect how representative documents are tokenized rather than relying only on benchmark scores.

Counting Tokens Before an API Call

In production systems, token counts should be estimated before sending a request. The exact method depends on the model provider, but the workflow is generally:

def prepare_request(text, tokenizer, token_limit):
    token_ids = tokenizer.encode(text)

    if len(token_ids) > token_limit:
        token_ids = token_ids[:token_limit]

    return tokenizer.decode(token_ids)

Real enterprise applications should avoid silent truncation wherever possible. Important sections may be removed without the user realising it. Better approaches include summarising, chunking, retrieving only relevant passages, or asking the user to narrow the request.

Special Tokens and Chat Templates

Chat-oriented models often use special tokens to distinguish system instructions, user messages, assistant responses, and tool outputs. The model's tokenizer may include a chat template that formats these roles correctly.

messages = [
    {
        "role": "system",
        "content": "You are an enterprise policy assistant."
    },
    {
        "role": "user",
        "content": "Summarise the remote-work policy."
    }
]

formatted = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

print(formatted)

Incorrect message formatting can reduce response quality, even when the underlying prompt text appears reasonable. Enterprise systems should therefore use the chat template designed for the selected model whenever one is available.

Engineering Implications

Tokenization is not merely a preprocessing detail. It directly affects architecture and operating cost. Engineers should monitor average input length, output length, truncation rates, and token consumption across real user workflows.

These measurements help answer practical questions:

• Should long documents be summarised before processing?
• Should retrieval be used instead of sending the entire knowledge base?
• Is the selected model economical for high-volume workloads?
• Does the tokenizer handle multilingual and domain-specific content efficiently?

In the next section, you will move from discrete token identifiers to embeddings, the dense numerical representations that allow models to capture similarity and semantic relationships.

Embeddings

Representing Meaning as Numbers

Tokens provide a numerical identifier for each unit of text, but token IDs do not contain semantic meaning. The number assigned to a token is simply an index in the tokenizer vocabulary. A model therefore requires a richer representation before it can compare words, phrases, or documents meaningfully.

An embedding is a dense numerical vector that represents important characteristics of a token, sentence, paragraph, or document. Texts with related meanings tend to receive vectors that are located near one another in the embedding space.

"purchase a product"
"buy an item"
"place an order"

Although these phrases use different words, their embeddings should be more similar to one another than to an unrelated phrase such as:

"schedule an annual leave request"

This ability to represent semantic similarity is one of the foundations of modern NLP. It enables search systems to retrieve relevant information even when the user does not use exactly the same wording as the source document.

From Sparse Vectors to Dense Embeddings

Earlier NLP systems frequently represented text using sparse vectors such as one-hot encoding or bag-of-words. In a vocabulary containing 50,000 terms, a word might be represented by a 50,000-dimensional vector containing one value of 1 and 49,999 values of 0.

Vocabulary:
["account", "invoice", "policy", "refund"]

One-hot representation:

"account" → [1, 0, 0, 0]
"invoice" → [0, 1, 0, 0]
"policy"  → [0, 0, 1, 0]
"refund"  → [0, 0, 0, 1]

These vectors identify words, but they do not express relationships between them. Dense embeddings solve this problem by representing each item using a smaller number of continuous values.

"invoice" → [0.42, -0.18, 0.73, ...]
"billing" → [0.39, -0.12, 0.69, ...]

The individual dimensions are not usually interpreted manually. Meaning emerges from the position of the entire vector relative to other vectors.

Static and Contextual Embeddings

Earlier embedding methods such as Word2Vec and GloVe produced one fixed vector for each word. This is called a static embedding.

A fixed representation creates difficulty when a word has several meanings. Consider:

"The client visited the bank."

"The boat reached the river bank."

A static embedding assigns the same vector to bank in both sentences. Transformer-based models instead create contextual embeddings. The representation of each token depends on the surrounding text, allowing the model to distinguish the financial meaning from the geographical meaning.

Sentence and Document Embeddings

Enterprise applications often need one vector for a complete sentence, paragraph, or document rather than a separate vector for every token. Sentence embedding models are trained to place semantically related passages close together.

This supports applications such as:

• semantic enterprise search
• document clustering
• duplicate detection
• recommendation systems
• retrieval-augmented generation
• similarity-based classification
• knowledge-base organisation

Creating Embeddings with sentence-transformers

The sentence-transformers library provides pretrained models designed specifically for sentence and document embeddings.

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("all-MiniLM-L6-v2")

sentences = [
    "The employee requested annual leave.",
    "The staff member submitted a vacation request.",
    "The customer reported a damaged product."
]

embeddings = model.encode(
    sentences,
    normalize_embeddings=True
)

print(embeddings.shape)

The output shape might be:

(3, 384)

This means that three sentences were converted into three vectors, each containing 384 numerical dimensions.

Measuring Similarity

A common way to compare embeddings is cosine similarity. It measures the angle between two vectors rather than their absolute magnitude.

from sentence_transformers.util import cos_sim

similarity_matrix = cos_sim(embeddings, embeddings)

print(similarity_matrix)

The first two sentences should receive a higher similarity score because they express nearly the same idea. The third sentence should be less similar.

A Simple Semantic Search Example

The same principle can be used to search an enterprise knowledge base.

from sentence_transformers import SentenceTransformer
from sentence_transformers.util import cos_sim

model = SentenceTransformer("all-MiniLM-L6-v2")

documents = [
    "Employees must submit annual leave requests through the HR portal.",
    "Invoices above ₹100,000 require finance director approval.",
    "Customers can return damaged items within thirty days."
]

document_embeddings = model.encode(
    documents,
    normalize_embeddings=True
)

query = "How do staff apply for vacation?"

query_embedding = model.encode(
    [query],
    normalize_embeddings=True
)

scores = cos_sim(query_embedding, document_embeddings)[0]

best_index = int(scores.argmax())

print("Best match:", documents[best_index])
print("Similarity:", float(scores[best_index]))

The system retrieves the annual-leave policy even though the query uses the word vacation instead of annual leave. This is the essential difference between semantic search and exact keyword matching.

Embeddings in RAG Systems

Retrieval-Augmented Generation systems divide documents into chunks, embed each chunk, and store the vectors in a vector database. When a user asks a question, the system embeds the question and retrieves the nearest document chunks.

Documents
    ↓
Chunking
    ↓
Embeddings
    ↓
Vector Database

User Question
    ↓
Query Embedding
    ↓
Similarity Search
    ↓
Relevant Chunks
    ↓
Large Language Model

The quality of retrieval depends on several factors:

• the embedding model
• the domain of the source documents
• chunk size and overlap
• metadata quality
• similarity metric
• the number of results retrieved

Enterprise Considerations

Embedding models should be evaluated using representative organisational data. A model that performs well on general English sentences may perform poorly on legal clauses, medical terminology, product codes, or multilingual support content.

Engineers should also consider privacy. Sending confidential documents to an external embedding API may be unacceptable in regulated environments. In such cases, organisations may prefer locally hosted embedding models or private cloud deployments.

Embeddings should also be refreshed when source information changes. Updating the original document without regenerating its embedding can leave the retrieval index out of date.

Attention Mechanism

Learning Which Parts of the Input Matter

Embeddings represent tokens numerically, but a language model must still determine which tokens are relevant to one another. The attention mechanism allows the model to assign different levels of importance to different positions in the input.

Consider the sentence:

"The manager approved the proposal because it satisfied the budget requirements."

To interpret the word it, the model must connect it with proposal, not with manager or budget requirements. Attention helps the model identify that relationship.

Why Earlier Sequence Models Struggled

Recurrent Neural Networks process a sequence one token at a time. Information from earlier tokens must be carried forward through the hidden state. As sequences become longer, important information may weaken or disappear.

Token 1 → Token 2 → Token 3 → ... → Token 500

Transformers use attention so that a token can examine other relevant tokens directly, regardless of their distance in the sequence.

Token 500
   ↙   ↓   ↘
Token 20  Token 173  Token 421

This direct relationship modelling improves long-range context and enables efficient parallel training.

Queries, Keys, and Values

Self-attention creates three representations for every token:

• Query: what information is this token looking for?
• Key: what information does this token contain?
• Value: what information should be passed forward if this token is relevant?

The query of one token is compared with the keys of other tokens. Stronger matches receive larger attention weights. Those weights determine how much of each value contributes to the new representation.

Token Embedding
      ↓
Query   Key   Value
   \     |     /
    Attention Scores
          ↓
Weighted Combination
          ↓
Context-Aware Token Representation

Scaled Dot-Product Attention

At a high level, attention follows this sequence:

1. Compare the query with all keys.
2. Scale the scores to keep training stable.
3. Apply softmax to convert scores into weights.
4. Use the weights to combine the value vectors.

Attention(Q, K, V) =
softmax((QKᵀ) / √d) V

You do not need to derive this equation in detail, but you should understand its role. The result is a new vector for each token that contains information gathered from other relevant tokens.

Self-Attention

In self-attention, the queries, keys, and values all come from the same sequence. Every token can therefore attend to other tokens within the input.

In the sentence:

"The contract that the legal team reviewed was approved."

The representation of contract may attend to reviewed and approved, while legal team may attend strongly to reviewed.

Multi-Head Attention

One attention calculation may focus on one type of relationship. Transformers therefore use multi-head attention, where several attention heads learn different patterns in parallel.

One head may focus on grammatical relationships, another on entity references, and another on broader semantic relationships.

Input Representations
       ↓
Head 1: grammatical relationships
Head 2: entity relationships
Head 3: semantic relationships
Head 4: positional relationships
       ↓
Concatenate and Transform
       ↓
Combined Representation

The interpretations of individual heads are not always this clean in practice, but the architecture allows the model to capture several relationship patterns simultaneously.

Causal Attention in Generative Models

Decoder-based language models generate text one token at a time. During generation, a token must not see future tokens that have not yet been produced. A causal mask prevents attention from looking forward.

Input:
"Enterprise AI can"

Allowed attention:
"Enterprise" → sees "Enterprise"
"AI"         → sees "Enterprise AI"
"can"        → sees "Enterprise AI can"

Future tokens remain hidden.

The model predicts the next token using only the tokens already available.

Attention Masks

Attention masks are also used to distinguish meaningful tokens from padding. When several sequences of different lengths are processed together, shorter sequences are padded so that the batch has a consistent shape.

Sequence A: [token, token, token, token]
Sequence B: [token, token, PAD,   PAD]

Attention mask:
A: [1, 1, 1, 1]
B: [1, 1, 0, 0]

The mask prevents the model from treating padding tokens as meaningful content.

Why Attention Matters for Enterprise Applications

Attention enables models to connect information across contracts, reports, conversations, and code. It supports tasks such as:

• resolving references across long passages
• identifying relevant clauses in documents
• maintaining conversational context
• summarising multi-section reports
• relating questions to retrieved evidence
• generating code that depends on earlier definitions

However, attention does not guarantee perfect understanding. Models may still focus on irrelevant information, overlook important details, or become distracted when prompts contain excessive context.

Computational Cost

Standard self-attention compares every token with every other token. As sequence length increases, the number of comparisons grows rapidly. This is one reason long-context processing can be computationally expensive.

More Tokens
    ↓
More Attention Comparisons
    ↓
More Memory and Computation
    ↓
Higher Latency and Cost

Modern architectures use several techniques to improve long-context efficiency, but enterprise teams should still avoid sending unnecessary information to a model. Retrieval, filtering, summarisation, and structured context management remain important architectural practices.

Attention provides the central mechanism that allows transformer models to build context-aware representations. In the next part, you will examine how attention is combined with feed-forward layers, residual connections, normalization, and positional information to form the complete transformer architecture.

Transformers

From Attention to a Complete Architecture

Attention explains how a model identifies relationships among tokens, but attention alone does not form a complete language model. A transformer combines attention with positional information, feed-forward networks, residual connections, and layer normalization. These components are arranged in repeated blocks, allowing the model to refine its representation of the input at every layer.

Token Embeddings + Positional Information
                 ↓
        Multi-Head Attention
                 ↓
      Add & Layer Normalization
                 ↓
        Feed-Forward Network
                 ↓
      Add & Layer Normalization
                 ↓
        Contextual Representations

Encoder and Decoder Architectures

The original transformer contains an encoder and a decoder. The encoder reads an input sequence and creates context-aware representations. The decoder generates an output sequence while attending to both earlier generated tokens and the encoder output. Translation systems commonly use this encoder-decoder design.

Modern NLP models often use only part of the architecture. Encoder-only models such as BERT are effective for classification, extraction, and semantic understanding. Decoder-only models generate text by repeatedly predicting the next token. Most contemporary conversational LLMs use this decoder-only pattern.

Positional Information

Because attention processes many tokens in parallel, the model needs an additional signal indicating token order. Positional encodings or learned positional representations are added to token embeddings. Without this information, the model could not reliably distinguish between sentences such as “The manager approved the analyst” and “The analyst approved the manager.”

Multi-Head Attention and Feed-Forward Networks

Multi-head attention allows the model to examine several relationship patterns simultaneously. Its output then passes through a feed-forward network applied independently to each token position. Attention exchanges information across tokens; the feed-forward network transforms the resulting representation within each position. Repeating these operations across many layers enables increasingly abstract language representations.

Residual Connections and Layer Normalization

Residual connections add a block's input back to its output, helping information and gradients move through deep networks. Layer normalization stabilizes activations during training. Together, these techniques make it practical to train transformers containing many stacked layers.

Why Transformers Replaced Recurrent Models

Recurrent models process sequences largely one step at a time. Transformers process many positions in parallel, making training more efficient on modern hardware. Attention also creates direct relationships between distant tokens instead of forcing information through a long chain of recurrent states. These advantages made transformers easier to scale across larger datasets, model sizes, and enterprise workloads.

Running a Transformer with Hugging Face

The following complete example loads a pretrained transformer pipeline, analyses an enterprise support message, and prints the predicted sentiment.

from transformers import pipeline


def analyse_message(text: str) -> None:
    classifier = pipeline(
        task="sentiment-analysis",
        model="distilbert-base-uncased-finetuned-sst-2-english"
    )

    result = classifier(text)[0]

    print("Message:", text)
    print("Label:", result["label"])
    print("Confidence:", round(result["score"], 4))


if __name__ == "__main__":
    analyse_message(
        "The new support portal resolved my issue much faster than expected."
    )

The pipeline handles tokenization, tensor preparation, model inference, and output decoding. In production, engineers should load the model once when the service starts rather than recreating it for every request.

Transformers provide the architectural foundation for modern LLMs. The next section will examine how decoder-only transformer blocks are scaled and trained to produce a complete Large Language Model.

Large Language Model Architecture

From Transformers to Large Language Models

A transformer becomes a Large Language Model when the architecture is scaled across many layers, trained on very large text collections, and optimised to predict language patterns across a broad range of domains. The underlying mechanism remains conceptually simple: the model receives a sequence of tokens and estimates which token is most likely to come next. What makes an LLM powerful is the scale and depth at which this process is learned.

Modern language models contain repeated transformer blocks. Each block refines the representation of every token by combining information from attention, feed-forward processing, residual connections, and normalisation. Earlier layers may capture local or syntactic relationships, while deeper layers can represent more abstract patterns involving meaning, intent, structure, and task context.

Input Text
    ↓
Tokenization
    ↓
Token Embeddings + Positional Information
    ↓
Repeated Transformer Blocks
    ↓
Final Hidden Representations
    ↓
Vocabulary Logits
    ↓
Next-Token Probability Distribution
    ↓
Selected Output Token

This sequence is repeated during generation. After the model selects one token, that token is added to the existing sequence and the model predicts again. A paragraph that appears to have been written as a complete thought is therefore produced one token at a time.

Decoder-Only LLMs

Many generative LLMs use a decoder-only transformer. Unlike encoder models, which can examine all tokens in an input simultaneously, decoder-only models use causal attention. Each position may attend only to earlier positions in the sequence.

Input:
"The quarterly report indicates"

Allowed attention:
"The"        → "The"
"quarterly"  → "The quarterly"
"report"     → "The quarterly report"
"indicates"  → "The quarterly report indicates"

This restriction prevents the model from seeing future tokens during training. It must learn to predict each next token from the text that precedes it. The same architecture can later generate emails, reports, code, summaries, explanations, and structured outputs because all of these tasks can be represented as continued token prediction.

From Token Embeddings to Logits

When text enters the model, every token ID is mapped to an embedding vector. Positional information is added so that the model can distinguish token order. The resulting vectors pass through the transformer layers, where attention repeatedly updates each token representation using the available context.

At the final layer, the hidden representation for the current position is projected into a vector containing one score for every token in the model vocabulary. These raw scores are called logits.

Vocabulary candidates:

"growth"     → logit 8.4
"risk"       → logit 6.9
"decline"    → logit 5.7
"office"     → logit 0.8

A softmax operation converts the logits into a probability distribution. The decoding strategy then determines how the output token is selected. Greedy decoding chooses the most probable token, while sampling methods can introduce controlled variation. Parameters such as temperature and top-p affect this selection process, but they do not change the model's learned knowledge.

Pre-training: Learning General Language Patterns

During pre-training, a decoder-only model processes enormous collections of text and repeatedly predicts the next token. If the training sequence is:

"Employees must submit expense claims within thirty days."

the model may receive:

Input:  "Employees must submit expense claims within"
Target: "thirty"

Prediction errors are measured through a loss function. Optimisation algorithms adjust billions of model parameters so that likely continuations receive higher probabilities in similar contexts. Across vast numbers of examples, the model learns grammar, factual associations, writing patterns, programming syntax, document structures, and many relationships encoded in the training data.

Pre-training does not create a searchable internal database containing exact documents. Knowledge is distributed across model parameters as learned statistical relationships. This is why an LLM may generate a plausible statement without being able to identify its source, and why retrieval is still required when an enterprise answer must be grounded in current or authoritative evidence.

Scaling Laws

Language-model performance has often improved when developers increase model parameters, training data, and computational resources together. These relationships are commonly discussed as scaling laws. Larger models can learn more complex patterns and frequently demonstrate stronger generalisation across tasks that were not explicitly programmed.

Scaling, however, produces trade-offs. A larger model may improve reasoning or generation quality, but it usually requires more memory, greater inference cost, specialised hardware, and longer response times. Enterprise engineering therefore focuses on the best model for a workload rather than automatically selecting the largest available model.

Fine-Tuning

A pretrained model is general-purpose. Fine-tuning continues training on a smaller, task-specific or domain-specific dataset. The objective is to adapt model behaviour without repeating the full pre-training process.

Fine-tuning may help a model learn:

• a specialised document format
• domain terminology
• a required tone or response style
• a classification or extraction task
• structured output conventions
• organisation-specific interaction patterns

Parameter-efficient methods can update a relatively small set of additional parameters rather than modifying the entire model. This may reduce training cost and make it easier to maintain several specialised variants. Fine-tuning is not, however, the best method for injecting frequently changing facts. Retrieval-Augmented Generation is generally more suitable when source information must remain current, traceable, and easy to update.

Instruction Tuning

A base model predicts likely text continuations, but it may not reliably follow user requests. Instruction tuning trains the model on examples containing an instruction and an appropriate response.

Instruction:
"Summarise the following policy in three bullet points."

Expected response:
- ...
- ...
- ...

Through many examples, the model learns that prompts are not merely text to continue; they often represent tasks to complete. Instruction tuning improves usefulness across question answering, summarisation, extraction, transformation, and conversational assistance.

RLHF and Preference Alignment

Reinforcement Learning from Human Feedback, or RLHF, is one approach used to align model responses with human preferences. Human evaluators compare candidate outputs, and those preferences are used to train a reward model or preference signal. The language model is then optimised to produce responses that are more helpful, relevant, and appropriate.

Modern alignment pipelines may also use other preference-optimisation techniques, synthetic feedback, safety training, and policy-based evaluation. The central idea remains the same: pre-training teaches broad language capability, while post-training shapes how that capability is expressed.

Pre-training
    ↓
General Language Capability
    ↓
Instruction Tuning
    ↓
Task-Following Behaviour
    ↓
Preference and Safety Alignment
    ↓
Assistant-Style Model

Alignment reduces many undesirable behaviours, but it does not guarantee factual accuracy, fairness, security, or compliance. Enterprise applications must still add evaluation, access controls, monitoring, and human oversight around the model.

The Inference Pipeline

Inference is the process of using a trained model to generate an output. A production request usually involves more than the neural network alone.

User Request
    ↓
Authentication and Access Control
    ↓
Prompt Construction
    ↓
Optional Retrieval or Tool Results
    ↓
Tokenization
    ↓
Model Inference
    ↓
Decoding
    ↓
Output Validation and Safety Checks
    ↓
Application Response
    ↓
Logging and Monitoring

The prompt may include system instructions, conversation history, retrieved documents, user content, and output requirements. The model provider or serving infrastructure tokenizes the request, executes the transformer layers, and decodes the generated tokens. The application may then validate structure, detect sensitive content, attach citations, apply business rules, or request human approval.

Calling an OpenAI-Compatible API

Many hosted and self-managed model platforms expose APIs that follow an OpenAI-compatible interface. This allows an application to keep a broadly similar client workflow while changing the model, provider endpoint, or deployment environment.

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["LLM_API_KEY"],
    base_url=os.environ.get(
        "LLM_BASE_URL",
        "https://api.openai.com/v1"
    )
)

response = client.responses.create(
    model=os.environ["LLM_MODEL"],
    input=[
        {
            "role": "system",
            "content": (
                "You are an enterprise policy assistant. "
                "Answer clearly and do not invent missing facts."
            )
        },
        {
            "role": "user",
            "content": (
                "Explain why confidential documents should not "
                "be sent to an unapproved external model."
            )
        }
    ]
)

print(response.output_text)

The API key, model identifier, and base URL are stored in environment variables rather than embedded directly in the source code. This makes the example easier to configure across development, testing, private-cloud, and production environments.

An OpenAI-compatible interface reduces integration effort, but compatibility should be verified rather than assumed. Providers may differ in supported parameters, tool calling, structured outputs, streaming behaviour, context limits, authentication, and response metadata.

Enterprise Deployment Considerations

Enterprise deployment begins with data classification. Teams must determine whether prompts may contain public, internal, confidential, regulated, or personally identifiable information. The permitted model endpoint, retention settings, geographic region, and logging policy should follow from that classification.

Important architectural considerations include:

• Privacy: determine what data may leave the organisational boundary.
• Security: protect credentials, endpoints, prompts, tools, and retrieved information.
• Access control: ensure users can retrieve only information they are authorised to view.
• Reliability: implement timeouts, retries, fallbacks, and graceful failure handling.
• Observability: monitor latency, token usage, error rates, model versions, and output quality.
• Evaluation: test with representative tasks before and after every model or prompt change.
• Cost management: control context length, output length, model routing, and request volume.
• Governance: record approved use cases, responsibilities, risks, and review procedures.

Model outputs should never be treated as automatically trustworthy merely because they are fluent. High-impact workflows may require source grounding, deterministic validation, confidence thresholds, or human approval. Organisations should also maintain versioned prompts and evaluation datasets so that behavioural changes can be measured when a model is updated.

The LLM is therefore only one component of an enterprise AI system. Its architecture explains how language is generated, but production quality depends on the controls, data pipelines, application logic, and governance surrounding it. The next part examines context windows, advanced prompt engineering, and the practical criteria used to select a model for an enterprise workload.

Context Windows

What a Context Window Represents

A Large Language Model does not retain an unlimited working memory. Instead, it processes a bounded sequence of tokens called the context window. This window may contain system instructions, user messages, conversation history, retrieved documents, tool outputs, and the model's own generated response.

Context length is measured in tokens rather than words. A token may represent a complete word, part of a word, punctuation, or a formatting element. As a result, the same number of words can produce different token counts depending on language, vocabulary, code, tables, and document structure.

Context Window
├── System instructions
├── User request
├── Conversation history
├── Retrieved evidence
├── Tool results
└── Space reserved for the response

The input and generated output usually share the same overall limit. If too much space is consumed by documents and conversation history, fewer tokens remain for the answer. Applications must therefore manage context as a limited computational resource.

Long Context Does Not Guarantee Better Results

A larger context window allows an application to provide more information, but additional text does not automatically improve accuracy. Important instructions can become harder for the model to prioritise when they are surrounded by repetitive, weakly relevant, or conflicting material. Long prompts also increase latency and inference cost.

Enterprise applications should therefore select context deliberately. Instead of sending an entire policy repository, a retrieval system can identify the small number of passages most relevant to the user's question. The prompt can then include those passages together with source metadata and clear instructions about how they should be used.

Practical Context Management

Common context-management techniques include:

• removing irrelevant conversation turns
• summarising older exchanges
• retrieving only the most relevant document sections
• deduplicating repeated information
• placing critical instructions in a stable system message
• reserving enough space for the expected output
• recording source identifiers separately from the model response

Context design is therefore not merely a prompting issue. It is an architectural decision involving retrieval, permissions, token budgeting, latency, cost, and traceability.

Prompt Engineering

Designing Instructions for Reliable Behaviour

Prompt engineering is the practice of structuring instructions and supporting information so that a language model can perform a task consistently. A strong enterprise prompt usually specifies the role of the model, the task to complete, the evidence it may use, the expected output format, and the constraints it must follow.

Role
    ↓
Task
    ↓
Relevant Context
    ↓
Rules and Constraints
    ↓
Required Output Format
    ↓
Examples, when useful

Vague instructions often produce vague outputs. For example, asking a model to “analyse this report” leaves many questions unanswered. Should it summarise, identify risks, calculate metrics, compare periods, or recommend actions? A more precise prompt defines the expected analysis and the structure of the response.

A Structured Enterprise Prompt

You are an internal procurement analyst.

Task:
Review the supplied vendor assessment and identify the three
highest-priority operational risks.

Evidence rules:
- Use only the supplied assessment.
- Do not invent missing facts.
- State "Not provided" when evidence is absent.

Output:
Return a table with:
1. Risk
2. Supporting evidence
3. Business impact
4. Recommended next action

This prompt reduces ambiguity by separating role, task, evidence rules, and output requirements. It also creates a response that can be reviewed more easily by a human or validated by downstream software.

Zero-Shot, Few-Shot, and Structured Prompting

In zero-shot prompting, the model receives instructions without examples. This is often sufficient for familiar tasks. In few-shot prompting, one or more examples demonstrate the desired pattern. Examples are especially useful when the task involves specialised labels, unusual formatting, or organisation-specific conventions.

Structured prompting asks the model to produce predictable fields, such as JSON, tables, or predefined categories. This can improve integration with business systems, but the output must still be validated before it is stored or used in automated decisions.

Prompt Injection and Untrusted Content

Enterprise prompts may include content retrieved from websites, uploaded files, databases, or emails. This content should be treated as untrusted data rather than authoritative instructions. A malicious document may contain text such as “ignore previous instructions” or request access to confidential information.

Applications should isolate instructions from retrieved content, restrict tool permissions, validate model actions, and require confirmation for high-impact operations. Prompt engineering can reduce risk, but it cannot replace access control and application-level security.

Prompt Evaluation and Versioning

Prompts should be treated as versioned application assets. Teams should test them against representative examples, edge cases, adversarial inputs, and failure scenarios. Evaluation should measure not only fluency but also factual grounding, completeness, format compliance, safety, latency, and cost.

When prompts or models change, the same evaluation set should be rerun. This allows teams to identify regressions before deployment rather than relying on a few successful demonstrations.

Model Selection

Selecting for the Workload

Model selection should begin with the business task rather than the popularity of a model. Different workloads require different combinations of reasoning ability, speed, context length, structured output reliability, multimodal support, privacy, deployment flexibility, and cost.

A large general-purpose model may be appropriate for complex analysis, while a smaller model may be better for classification, extraction, routing, or repetitive high-volume tasks. Some systems use several models together, routing each request to the least expensive model that can meet the required quality threshold.

Key Selection Criteria

• Task quality: performance on representative business examples
• Latency: time to first token and total response time
• Cost: input, output, hosting, and operational expenses
• Context capacity: ability to process the required working set
• Reliability: format compliance, consistency, and failure behaviour
• Privacy: data handling, retention, location, and training policies
• Security: authentication, isolation, auditability, and tool controls
• Deployment: hosted API, private cloud, on-premises, or edge environment
• Governance: documentation, model versioning, monitoring, and vendor transparency

Hosted, Private, and Self-Managed Models

Hosted APIs reduce infrastructure effort and can provide rapid access to advanced models. Private or dedicated deployments may offer stronger isolation and configuration control. Self-managed open models provide the greatest control over hosting and customisation, but require expertise in hardware, serving, scaling, patching, evaluation, and security.

The correct choice depends on organisational constraints. A team should not assume that self-hosting is automatically cheaper or safer. Operational mistakes, weak access controls, unpatched dependencies, and poor monitoring can create substantial risk.

A Simple Model Evaluation Pattern

1. Define the task and acceptable quality threshold.
2. Build a representative evaluation dataset.
3. Test several candidate models with the same prompts.
4. Measure quality, latency, cost, and failure rate.
5. Review privacy, security, and deployment constraints.
6. Select the smallest model that satisfies the requirements.
7. Monitor performance after deployment.

Model Routing and Fallbacks

Mature systems may use a routing layer. Straightforward requests can be handled by a small model, while complex or high-risk tasks are escalated to a stronger model or a human reviewer. Fallback models can improve resilience when a provider is unavailable, but fallback behaviour must be tested because models may interpret prompts differently.

Model selection is therefore an ongoing engineering process rather than a one-time procurement decision. Model versions, prices, capabilities, and organisational requirements change. Continuous evaluation ensures that the chosen architecture remains appropriate.

Together, context management, prompt design, and model selection determine how effectively an LLM can be used in an enterprise application. The final part of this module will consolidate these ideas through a summary, knowledge check, and navigation to the next stage of the programme.

Module Summary

This module introduced the foundations of modern enterprise language AI, beginning with the development of Natural Language Processing and progressing through the architecture and operation of Large Language Models. The central idea is that an LLM is not a rule-based system or a conventional searchable database. It is a probabilistic model that processes tokens, represents them as vectors, transforms those representations through repeated neural-network layers, and predicts likely continuations.

Tokenization converts text into units that a model can process. Embeddings then map those tokens into numerical vectors that capture useful relationships. Attention allows the model to determine which parts of the available sequence matter for each token. Transformers combine attention with feed-forward networks, residual connections, positional information, and layer normalization, enabling efficient parallel training and stronger handling of long-range relationships than traditional recurrent architectures.

Many generative LLMs use a decoder-only transformer with causal attention. During pre-training, the model learns by predicting the next token across very large text collections. Instruction tuning and preference-alignment methods subsequently shape the model into a more useful assistant. Fine-tuning can adapt behaviour for a particular task or domain, but frequently changing enterprise knowledge is usually better supplied through retrieval than encoded permanently into model parameters.

At inference time, token embeddings pass through the transformer layers and are converted into logits over the vocabulary. A decoding strategy selects the next token, which is added to the sequence before the cycle repeats. The apparent fluency of the output does not guarantee that it is factual, authorised, safe, or appropriate for business use. Enterprise applications must therefore surround the model with authentication, retrieval, validation, monitoring, logging, governance, and human oversight.

Context windows limit how much information the model can process in a single request. More context is not always better: irrelevant or conflicting information may reduce quality while increasing latency and cost. Effective systems retrieve and organise only the most relevant information, protect access to sensitive data, and reserve enough space for the required output.

Prompt engineering provides a structured way to define the model's role, task, evidence, constraints, and output format. Prompts should be treated as versioned application assets and tested against representative examples and failure cases. Retrieved documents and external content must remain untrusted data, because they may contain instructions designed to manipulate the model.

Finally, model selection should be based on the workload. Quality, latency, context capacity, privacy, security, deployment options, reliability, and cost all matter. The strongest general model is not always the best operational choice. Mature systems may route simple requests to smaller models and reserve more capable models or human review for difficult and high-risk cases.

Knowledge Check

Use the following questions to review the main ideas introduced in this module. Try to answer each question before expanding the explanation.