AI Technology Landscape: A Six-Layer Map from Data and Models to Agents, MCP, and Governance

The hardest part of learning AI is often not a lack of concepts. It is that the concepts belong to different layers. Transformer is a model architecture. RAG is a runtime evidence pattern. MCP is a connection protocol. Dify is an application platform. vLLM is an inference engine. Put them in one “top AI technologies” list and the questions become confused: can RAG replace fine-tuning, can MCP replace an agent, and is a vector database a model?

This guide does not try to list every term. It provides a dependency map. Start with the problem and data, then place models, context and tools, applications, operations, and governance around it. With that structure, a new framework or product can be classified without rebuilding your understanding from scratch.

The Six-Layer Map

An AI system can be compressed into six layers from user need to production responsibility:

Layer 6  Governance: ownership, risk, privacy, security, compliance, human control
Layer 5  Operations: inference serving, deployment, monitoring, cost, versions, rollback
Layer 4  Applications: assistants, search, workflows, agents, industry products
Layer 3  Context and tools: prompts, RAG, memory, function calls, MCP
Layer 2  Models: classical ML, deep learning, foundation models, training, adaptation
Layer 1  Problem, data, evaluation: objectives, samples, labels, metrics, test sets

The arrows are not one-way. Production failures change the evaluation set. Cost constrains model selection. Permission requirements constrain tools. Evaluation and governance also span all six layers rather than appearing only at the end.

Layer	Core question	Main artifacts	Common mistake
Problem, data, evaluation	What outcome counts as success	Datasets, test sets, metrics, acceptance rules	Selecting a model before defining the problem
Models	How does the system learn or generate	Parameters, tokenizers, embeddings, classifiers	Treating a model name as complete product capability
Context and tools	What evidence, state, and actions are needed at runtime	Prompts, retrieval results, tool schemas, protocol connections	Assuming a connected tool is automatically reliable
Applications	How does a user complete a task	Interfaces, APIs, workflows, agent state	Using multiple agents instead of designing a process
Operations	How is the system delivered reliably and economically	Endpoints, caches, monitoring, versions, rollback	Testing a demo but not failures or concurrency
Governance	Who may do what, and who owns failures	Permissions, approvals, audits, retention, exit plans	Adding safety just before launch

Layer 1: Problem, Data, and Evaluation

Classical machine learning normally begins with samples, targets, preprocessing, and evaluation. The scikit-learn getting-started workflow composes preprocessors and estimators in a Pipeline, then uses held-out data or cross-validation to test generalization. Generative AI should follow the same discipline: define the task and test set before comparing models or applications.

An enterprise knowledge assistant should answer at least these questions:

• Which real questions will users ask, and which questions should be refused?
• Which documents and permissions contain the authoritative answer?
• How will citation correctness, answer correctness, and task completion be scored separately?
• What happens when no answer exists, access is denied, or an upstream service fails?
• What are the limits for latency, cost per task, and human escalation?

“The answer sounds natural” is not an evaluation method. Classification can use accuracy, recall, and related metrics. Generative tasks usually combine deterministic checks, model-assisted review, and human sampling. The test set must include failure paths, not only showcase prompts.

Layer 2: Models and Training

Machine learning is the broad set. Deep learning is a family of methods based on multilayer neural networks. Generative AI emphasizes producing text, images, audio, video, or other content. These are not three product generations that replace one another by date.

Classical Machine Learning

Linear models, trees, clustering, dimensionality reduction, and ensembles remain useful for structured data, interpretable baselines, and modestly sized tasks. Not every prediction problem needs a neural network, and even fewer require a large language model. With limited data and a clear target, a simple model evaluated correctly can be easier to explain and operate.

Deep Learning and Foundation Models

Convolutional networks exploit local spatial structure. Recurrent networks have been widely used for sequences. Transformers use attention and parallel computation and now underpin many language and multimodal systems. This is not a straight line in which older architectures disappear; data structure, latency, hardware, and product constraints still drive choices.

Foundation models learn transferable capabilities through large-scale pretraining, then adapt through instruction tuning, preference optimization, fine-tuning, or runtime context. PyTorch’s basic workflow separates data, models, automatic differentiation, optimization, and saving and loading. A model is therefore more than its network diagram; it includes data processing, training state, and a reproducible environment.

Capability Domains Are a Horizontal Axis

Language, vision, speech, recommendation, control, and scientific computing are not separate upper layers. They are horizontal capability domains that cut across data, models, applications, and governance. A voice assistant, for example, needs audio data, ASR and TTS models, a conversational application, real-time inference, and recording consent.

Layer 3: Prompts, RAG, Memory, Tools, and MCP

This layer does not retrain the base model. It supplies task instructions, evidence, state, and external actions when a request runs.

Prompting, RAG, and Fine-Tuning

Method	What changes	Good fit	Poor fit
Prompt and in-context examples	Instructions and examples in one request	Output format, role, current task	Large bodies of frequently changing knowledge
RAG	External evidence retrieved at runtime	Private documents, current knowledge, citations	Automatically eliminating bad retrieval or reasoning
Fine-tuning	Model parameters updated through training	Stable behavior, style, domain patterns, task capability	A frequently changing fact database

RAG does not “solve hallucination.” It gives a model access to external evidence, but retrieval can miss the correct document and the model can misuse a retrieved passage. Reliable systems evaluate parsing, retrieval, citation, answers, and refusals separately. For enterprise selection, continue with the enterprise RAG comparison.

Memory and State

Conversation history, user preferences, task progress, and business records are different kinds of memory. Chat content may be summarized or trimmed. Business records require an authoritative system such as a database. Preferences need correction, deletion, and tenant isolation. Putting everything into a vector store does not create dependable long-term memory.

Tool Calls and MCP

Function calling lets a model produce tool parameters that conform to a schema; the application still validates and executes them. MCP standardizes how AI applications connect to data sources, tools, and workflows. Official MCP documentation presents it as a common connection standard, not as automatic authentication, least privilege, parameter validation, auditing, or human approval.

To build a protocol-level example, continue with the MCP beginner guide.

Layer 4: Applications, Workflows, and Agents

A model has inputs and outputs. A product also needs an interface, business state, permissions, failure recovery, and delivery channels.

Deterministic Workflows

When the steps are known, use ordinary code or a workflow first: read a form, query a database, generate a draft, request approval, and send the result. Inputs and outputs stay explicit, failures are easier to retry, and cost is easier to budget.

Agents

An agent lets a model decide the next step, select tools, and continue from observations inside a bounded environment. It fits tasks whose path cannot be fully specified in advance. An agent is not an unlimited loop, and more tools do not guarantee better results. Set limits for steps, time, money, and permissions. Enforce approval for high-risk writes outside the model.

Multiple agents are justified when roles genuinely have different information, tools, objectives, or permissions. Several copies of the same model discussing the same evidence often add cost rather than independent scrutiny. Compare orchestration choices in the agent framework guide, or follow a structured curriculum in the Hello-Agents study guide.

Products Are Not Models

ChatGPT, Claude, and Kimi are user-facing products. They can combine several models with search, file handling, code execution, memory, and account policies. A base-model benchmark cannot determine whether the complete product fits a team workflow.

Layer 5: Inference, Deployment, and Operations

After training produces parameters, the model still has to become a service. Inference computes output from input. Serving adds concurrency, batching, caching, routing, rate limits, and observability.

Common operating models include:

• A provider API, which reduces infrastructure work but introduces region, pricing, quota, and data-policy dependencies.
• A self-hosted open model, which increases deployment control while transferring hardware, upgrade, security, and availability work to the team.
• An on-device model, which reduces some network dependency but faces memory, power, and capability limits.
• Hybrid routing, which selects models according to task risk, cost, and latency.

Quantization, distillation, caching, and batching can reduce cost, but they also change quality, latency, or engineering complexity. Avoid quoting a speedup without hardware, sequence length, concurrency, and quality conditions. English readers can use the Ollama profile as a starting point for local model runtime concepts.

Production operations should record request versions, models and settings, retrieval and tool events, latency, token or compute cost, error types, and human escalation. Model changes require regression tests and a rollback path rather than a direct endpoint swap.

Layer 6: Governance, Security, and Accountability

Governance is not a sign-off document added before launch. NIST’s AI Risk Management Framework places trustworthiness considerations throughout the design, development, use, and evaluation of AI products, services, and systems. A project needs executable controls:

1. Assign owners: identify responsibility for models, data, prompts, tools, business rules, and release decisions.
2. Minimize permissions: prefer read-only access; scope writes by resource and action; confirm deletion, payment, publishing, and outbound communication.
3. Protect data: limit sensitive data in models, logs, and evaluation sets; define retention, deletion, and cross-border handling.
4. Preserve evidence: record versions, sources, tool parameters, approvals, and outcomes while redacting secrets and personal information.
5. Measure continuously: monitor quality drift, bias, attacks, cost, and failure modes, then update the test set.
6. Prepare an exit: export data, replace models, revoke credentials, and fall back to human or deterministic processes.

Generative AI risk is broader than an incorrect answer. Prompt injection can use retrieved content to influence tool calls. Excessive automation can bypass judgment. Vendor changes can also break cost and availability assumptions. Governance must constrain the other five layers.

Learning Paths by Role

Product, Operations, and Business Owners

Start with Layers 1, 4, and 6. Select a real process and document current time, error cost, inputs, outputs, and human accountability. Run a read-only pilot with an existing product. The goal is not to learn training equations; it is to decide where AI creates measurable value and where human control remains mandatory.

AI Application Developers

Move through Layers 1, 3, 4, and 5. Start with a fixed evaluation set and one model call. Add structured output, RAG, and read-only tools. Then add state, monitoring, and rollback. Complete a deterministic workflow before an agent. Understand APIs, JSON Schema, OAuth, and least privilege before MCP.

Model and Algorithm Engineers

Focus on Layers 1, 2, and 5. Learn Python, NumPy, probability, statistics, and classical machine learning before PyTorch, Transformers, pretraining, fine-tuning, evaluation, and inference optimization. The Happy-LLM study guide provides a structured path from Transformers and training to RAG and agents.

Architecture, Security, and Governance Teams

Work backward from Layer 6. Threat-model data flows, identities, model endpoints, retrieval sources, and tool calls. Define vendor review, log redaction, red-team tests, human approval, incident response, and exit plans. The goal is not to memorize model names; it is to keep technical change inside organizational controls.

Validate the Map with One Small Project

Use “answer questions from internal documents” as the exercise, but work only with public or synthetic material:

1. Prepare 20 answerable questions, five unanswerable questions, and two permission roles.
2. Ask one model directly and record correctness, refusal, latency, and cost as a baseline.
3. Add document parsing and retrieval, requiring a source for each conclusion.
4. Add one read-only tool, such as a synthetic order lookup, with schema validation.
5. Control retrieval, answering, and escalation with a deterministic workflow before adding an agent.
6. Version the model, prompt, index, and code; simulate timeout, empty retrieval, and unauthorized access.
7. Document retention, owners, launch thresholds, and rollback.

This project crosses all six layers without training a foundation model. It exposes more real engineering decisions than copying a visually impressive multi-agent demo.

FAQ

Should a beginner start with foundation models or machine learning?

If your goal is to use or integrate existing AI products, start with model inputs and outputs, evaluation, and the application layer, then add machine-learning foundations. If you want to train or optimize models, begin with Python, data work, probability, statistics, and classical machine learning.

How are RAG, fine-tuning, and prompting different?

A prompt constrains one request, RAG retrieves external evidence at runtime, and fine-tuning updates model parameters through training. They solve different problems and can be combined, but they are not simple substitutes.

What is the difference between an agent and a workflow?

Developers define most steps and branches in a workflow. An agent lets a model select the next step and tools within bounded controls. Prefer workflows for stable processes and add an agent only when the path genuinely needs runtime adaptation.

Is MCP an agent framework?

No. MCP is a protocol for connecting AI applications to external data, tools, and workflows. It standardizes connections but does not provide planning, authentication, least privilege, parameter validation, or human approval for the application.

Is this map enough to build an AI product?

The map establishes concept boundaries and a learning order; it does not replace practice. Choose one role-based path and complete a small project with real evaluation, rollback, and permission boundaries.

References

• scikit-learn, Getting Started, covering estimators, preprocessing, pipelines, cross-validation, and model selection, accessed July 23, 2026.
• PyTorch, Learn the Basics, covering the workflow from data and models through automatic differentiation, optimization, saving, and loading, accessed July 23, 2026.
• Model Context Protocol, What is MCP?, defining MCP’s role in connecting AI applications to data sources, tools, and workflows, accessed July 23, 2026.
• NIST, AI Risk Management Framework, covering trustworthiness and risk management across the AI lifecycle, accessed July 23, 2026.

Bottom Line

Do not begin learning AI with a list of names. Define the problem, data, and evaluation before the model. Add context, RAG, tools, or MCP when the application needs external evidence and actions. Once those capabilities become a workflow or agent, the team still owns operations and governance.

For every new term, ask three questions: which layer is it in, which layers does it depend on, and which layer handles its failure? Those questions age better than a list of the latest tools.