Skip to content
Human Capital Sense

Learning Path

Generative AI

A structured learning path through Generative AI — foundations to enterprise patterns — in short, practical concept pages. 90 concepts across 14 learning stages — follow the path in order, or jump to what you need.

Stage 01

Foundations

Treat the LLM as a reasoning engine with frozen parametric memory — not a database of facts.

7 concepts

  1. 01

    What Is Generative AI

    AI systems that create new text, code, images, or audio by learning statistical patterns from large datasets instead of executing only hand-written rules.

  2. 02

    LLM as Reasoning Engine

    Treat the model as a general-purpose inference layer that transforms inputs into structured decisions and language—not as a database of organizational truth.

  3. 03

    Parametric Memory Limits

    Knowledge compressed into model weights is static, approximate, and cannot be updated per request—unlike retrieval or tool-backed non-parametric memory.

  4. 04

    Prompting

    Structuring instructions, context, and examples so the model produces useful, controllable outputs for a specific task.

  5. 05

    System Prompts

    Persistent instructions that define role, rules, and tone across turns, separate from the user's immediate request.

  6. 06

    Zero-Shot and Few-Shot Learning

    Zero-shot asks the model to perform a task with instructions alone; few-shot adds inline examples that demonstrate the desired input-output pattern.

  7. 07

    Chain of Thought

    Prompting the model to expose intermediate reasoning steps before the final answer, improving accuracy on multi-step problems.

Stage 02

Models & Tokens

Tokens, cost and model choice — optimize for semantic density, not maximum context size.

8 concepts

  1. 08

    Tokenization

    The process of splitting text into subword units (tokens) that the model reads, generates, and bills against.

  2. 09

    Tokens and Cost

    Provider pricing and latency scale with tokens processed—input, output, and often cached-input discounts.

  3. 10

    Semantic Density

    How much meaning you pack per token—dense prompts convey constraints and evidence compactly without wasting context window.

  4. 11

    Model Families

    Lineages of models sharing architecture, tokenizer, and training recipe—GPT, Claude, Gemini, Llama, Mistral, and domain-specific variants.

  5. 12

    Open vs Closed Models

    Closed models are proprietary APIs; open-weight models can be self-hosted, fine-tuned, and inspected—each with different ops and compliance tradeoffs.

  6. 13

    Temperature and Sampling

    Sampling controls how randomly the model picks next tokens—low temperature for deterministic tasks, higher for creative variation.

  7. 14

    Structured Output

    Constraining model responses to machine-parseable schemas—JSON, enums, or tool-call payloads—so downstream code can act reliably.

  8. 15

    Fine-Tuning

    Continued training on curated data to adapt model behaviour—tone, format, domain language—without replacing base reasoning capabilities.

Stage 03

Transformers & Attention

Enough architecture to understand limits: attention, context cost and why recall degrades.

6 concepts

  1. 16

    Transformers

    Neural architecture using self-attention to relate all tokens in a sequence—foundation of modern LLMs and many multimodal models.

  2. 17

    Attention Mechanism

    Weighted lookup that lets each token focus on the most relevant other tokens when building its representation.

  3. 18

    Encoder-Decoder Architecture

    Two-stack design: encoder builds bidirectional representations; decoder generates output autoregressively—common in translation and some RAG rerankers.

  4. 19

    Pretraining vs Inference

    Pretraining learns general language patterns offline at massive scale; inference applies those weights to new prompts at serving time.

  5. 20

    Scaling Laws

    Empirical relationships showing predictable quality gains as models, data, and compute grow—guiding frontier vs efficient model choices.

  6. 21

    Lost in the Middle

    Models often under-use information placed in the middle of long contexts, overweighting beginnings and endings.

Stage 04

Context Engineering & Caching

Assemble what the model sees at inference — caching, static prefixes and avoiding silent cache misses.

7 concepts

  1. 22

    Context Window

    Maximum tokens the model can attend to in one forward pass—system, history, retrieval, tools, and completion combined.

  2. 23

    Context Engineering

    Deliberate curation of what enters the prompt—ordering, compression, retrieval, and tool results—to maximize answer quality per token.

  3. 24

    Long Context

    Models and techniques supporting hundred-thousand to million-token inputs—whole codebases, corpora, or transcripts in one shot.

  4. 25

    Prompt Caching

    Reusing computed prefix states across requests with identical early prompt segments to cut latency and input-token cost.

  5. 26

    Cache Miss Patterns

    Common production habits that break prefix reuse—mutable system prompts, non-deterministic tool JSON, and sliding-window history.

  6. 27

    Memory Patterns

    Architectures for what the system remembers across sessions—working context, summaries, vector recall, and external stores.

  7. 28

    Chat History Management

    Strategies to trim, summarize, and structure multi-turn conversations so quality stays high without blowing token budgets.

Stage 05

Programmatic Prompting

Stop hand-tuning adjectives — declare signatures, compile programs and optimize with metrics (DSPy and beyond).

6 concepts

  1. 29

    DSPy

    Framework treating LLM pipelines as optimizable programs—signatures, modules, and teleprompters search for better prompts and weights.

  2. 30

    DSPy Signatures

    Typed input-output contracts in DSPy that specify fields, docstrings, and constraints for each module step.

  3. 31

    Teleprompters

    DSPy optimizers that search over prompts, few-shot sets, or module parameters to maximize a training metric.

  4. 32

    GEPA

    Genetic-Pareto prompt optimizer in DSPy that evolves prompt candidates under multi-objective metrics.

  5. 33

    MiPROv2

    DSPy teleprompter using model-informed proposal and search to refine instructions and few-shot sets efficiently.

  6. 34

    Programmatic Prompting

    Building prompts through code—templates, optimizers, and typed modules—instead of one-off strings in notebooks.

Stage 06

Embeddings & Representation

Turn meaning into vectors so systems can compare, search and cluster by similarity.

8 concepts

  1. 35

    Embeddings

    Dense vector representations of text (or other modalities) where semantic similarity approximates geometric closeness.

  2. 36

    Vector Similarity

    Scoring how close two embeddings are—usually cosine similarity or dot product—to rank candidates for retrieval.

  3. 37

    Vector Databases

    Storage engines optimized for approximate nearest-neighbor search over millions of embeddings with metadata filters.

  4. 38

    Chunking

    Splitting documents into retrieval-sized pieces before embedding—balance context completeness against search precision.

  5. 39

    Semantic Chunking

    Splitting text at natural topic boundaries detected by embedding similarity shifts between sentences or paragraphs.

  6. 40

    Late Chunking

    Embed the full document (or large span) first, then derive chunk vectors from internal model states—preserving global context in each piece.

  7. 41

    Structure-Aware Chunking

    Splitting along document structure—headings, tables, slides, code blocks—so chunks respect logical units and metadata.

  8. 42

    Multimodal Embeddings

    Joint vector spaces for text, images, audio, or video—enabling cross-modal search and retrieval.

Stage 07

Retrieval & Ranking

Hybrid search, reranking and query transformation — broad recall first, then precision.

7 concepts

  1. 43

    Semantic Search

    Finding documents by meaning similarity between query and corpus embeddings rather than exact keyword match.

  2. 44

    Hybrid Search

    Combining dense vector retrieval with sparse lexical methods (BM25) for robust recall across paraphrase and keyword queries.

  3. 45

    Reranking

    Second-stage model that scores query-passage pairs with richer interaction than bi-encoder retrieval alone.

  4. 46

    ColBERT

    Late-interaction retrieval model keeping token-level embeddings for efficient fine-grained matching between query and document.

  5. 47

    Query Transformation

    Rewriting user queries—expansion, decomposition, or step-back—for better retrieval against the index.

  6. 48

    HyDE

    Hypothetical Document Embeddings—generate a fake answer passage, embed it, and retrieve real documents similar to that hypothesis.

  7. 49

    Indexing Strategies

    How and when you chunk, embed, and refresh corpora—batch, incremental, multi-version, and metadata-rich pipelines.

Stage 08

Production RAG

From naive retrieve-and-stuff to modular pipelines: chunking, GraphRAG and grounded generation.

9 concepts

  1. 50

    Retrieval-Augmented Generation

    Retrieve relevant external documents at query time, inject them into the prompt, then generate an answer grounded in that evidence.

  2. 51

    RAG Architecture

    End-to-end components—ingestion, indexing, retrieval, reranking, generation, citation, and feedback loops—for grounded QA.

  3. 52

    Citation and Grounding

    Requiring answers to quote or link retrieved evidence—and refusing when support is insufficient.

  4. 53

    RAG Evaluation

    Metrics and datasets for retrieval quality and generation faithfulness—Precision@K, Recall@K, faithfulness, answer relevance.

  5. 55

    Advanced RAG

    Patterns beyond naive retrieve-once—multi-query, rerank, compress, route, and agentic retrieval loops.

  6. 56

    REFRAG

    Retrieval compression pattern that keeps many candidate chunks as compact embeddings and expands only the ones the decoder needs back into tokens.

  7. 57

    GraphRAG

    Combining knowledge graphs or community summaries with vector retrieval for global and relational questions over corpora.

  8. 58

    Knowledge Bases

    Curated corpora—wikis, tickets, PDFs, APIs—governed for ingestion, access control, and freshness as RAG source of truth.

  9. 59

    Naive vs Production RAG

    Naive RAG embeds docs and calls the LLM once; production RAG adds hybrid search, reranking, evals, guardrails, and ops.

Stage 09

Agents & Orchestration

Stateful multi-agent systems — graphs, crews and conversational loops with explicit memory tiers.

8 concepts

  1. 60

    AI Agents

    LLM-driven systems that plan, use tools, and iterate toward goals—not just single-shot text completion.

  2. 61

    Planning and Reasoning

    Decomposing goals into steps, choosing tools, and revising plans when observations contradict assumptions.

  3. 62

    Human in the Loop

    Checkpointing agent actions for human approval, correction, or escalation before irreversible side effects.

  4. 63

    Multi-Agent Systems

    Multiple specialized agents coordinating—researcher, coder, reviewer—via shared state or message passing.

  5. 64

    LangGraph

    Graph-based agent orchestration modeling workflows as state machines with nodes, edges, and checkpointed state.

  6. 65

    CrewAI

    Multi-agent framework organizing agents as crews with roles, goals, and delegated tasks—emphasizing collaborative role-play.

  7. 66

    AutoGen

    Microsoft framework for conversational multi-agent interaction—agents message each other until termination conditions.

  8. 67

    Agent Memory Tiers

    Layered memory—working buffer, episodic summaries, long-term vector store—for agents across sessions and tasks.

Stage 10

Tools & MCP

Connect agents to the world safely — tool calling and the Model Context Protocol as a universal interface.

3 concepts

  1. 68

    Tool Calling

    Pattern where models emit structured calls to external tools—APIs, databases, code—instead of only natural language.

  2. 69

    Function Calling

    Vendor API pattern where models return named functions with arguments matching predefined schemas for runtime execution.

  3. 70

    Model Context Protocol (MCP)

    Open protocol connecting AI hosts to external MCP servers exposing tools and resources through a standard client-server contract.

Stage 11

Evaluation & Quality

Probabilistic evaluation: faithfulness, retrieval metrics and LLM-as-a-judge methods like G-Eval.

7 concepts

  1. 54

    RAGAS

    Reference-free RAG evaluation suite measuring faithfulness, answer relevance, context precision, and context recall with LLM-assisted scoring.

  2. 71

    LLM Evaluation

    Systematic measurement of quality, safety, and cost across prompts, models, and pipelines—not vibe checks alone.

  3. 72

    Hallucination

    Model outputs that sound plausible but are factually unsupported or contradict provided evidence.

  4. 73

    LLM as Judge

    Using a strong model to score another model's outputs against rubrics—relevance, safety, coherence.

  5. 74

    G-Eval

    Evaluation framework using LLMs with chain-of-thought rubrics to score outputs on dimensions like coherence and groundedness.

  6. 75

    Observability for LLM Apps

    Tracing prompts, retrievals, tool calls, latencies, token costs, and scores across production requests.

  7. 76

    Faithfulness and Relevance

    Faithfulness measures whether answers are supported by context; relevance measures whether they address the question.

Stage 12

Guardrails & Safety

Runtime controls — prompt injection defense, content safety models and programmable rails.

5 concepts

  1. 77

    Guardrails

    Policy layers—input filters, output validators, tool allowlists—that constrain model behaviour before and after generation.

  2. 78

    Prompt Injection

    Attacks embedding instructions in untrusted content—emails, web pages—to hijack agent behaviour.

  3. 79

    Red Teaming

    Adversarial testing to discover jailbreaks, data leaks, and unsafe tool use before attackers do.

  4. 80

    Llama Guard

    Safety classifier models (Llama Guard family) scoring inputs and outputs against policy categories for allow/block decisions.

  5. 81

    NeMo Guardrails

    NVIDIA NeMo Guardrails-style programmable rails: declarative Colang policies, dialog boundaries, and tool constraints around LLM calls.

Stage 13

Multimodal

Beyond text — images, audio and documents as first-class inputs and outputs.

3 concepts

  1. 82

    Multimodal Models

    Models accepting and generating multiple modalities—text, images, audio—in unified or paired architectures.

  2. 83

    Speech to Text

    Automatic transcription of audio into text for downstream LLM summarization, search, and agent tools.

  3. 84

    Summarization Patterns

    Map-reduce, hierarchical, and extractive-abstractive blends for long content—docs, calls, threads.

Stage 14

Enterprise Patterns & Governance

Ship reliably: architecture patterns, privacy, cost control and accountable AI use.

6 concepts

  1. 85

    Enterprise AI Patterns

    Reference architectures for secure, multi-tenant GenAI—VPC deployment, SSO, audit logs, staged rollouts.

  2. 86

    Privacy and Data Handling

    Policies for PII redaction, data residency, retention, and customer consent when sending text to models.

  3. 87

    Model Routing

    Sending requests to different models by task complexity, cost tier, latency SLO, or data sensitivity.

  4. 88

    Cost Optimization

    Controlling spend via caching, routing, batching, compression, and step budgets without destroying quality.

  5. 89

    AI Governance

    Policies, roles, and review boards governing model selection, data use, eval evidence, and incident response.

  6. 90

    Responsible AI

    Principles and practices for fairness, transparency, safety, and accountability in generative systems.