The AI Stack, Layer by Layer
A reference map of the modern AI stack, from data sources through the knowledge layer to models, orchestration, and interfaces.
Every team draws a different diagram of their AI system, and most leave out the layer that is causing the problems. This guide gives you one reference architecture, explains what each layer is responsible for, and shows how the common patterns of retrieval, agents, and fine-tuning map onto it. Use it to make decisions, review designs, and explain the system to people who do not build it.
What you will be able to do
- Draw your own system on a layered reference architecture.
- Name the responsibility of each layer and the interfaces between them.
- Place retrieval, agents, evaluation, and fine-tuning correctly in the diagram.
- Identify which layers you should build, buy, or defer.
Inside the guide
- 01
Why one diagram
The cost of every team drawing a different system.
- 02
The data layer
Sources, pipelines, documents, and the metadata that makes them usable.
- 03
The knowledge layer
Ontologies, graphs, indexes, and where meaning is defined.
- 04
The model layer
Hosted and open models, embeddings, fine-tuning, and routing.
- 05
The orchestration layer
Prompts, tools, agents, and control flow.
- 06
Interfaces and evaluation
APIs, user experience, and the measurement loop that closes the system.
From the guide
The knowledge layer is the one most teams skip. They connect documents directly to a model, see reasonable answers in a demo, and discover six months later that nobody can say what the system believes a 'contract' or an 'account' is. The layer was always there. It was just implicit, and implicit knowledge cannot be fixed.
Who it is for
- Technical leads designing their first production AI system.
- Engineers who have built pieces of the stack and want the full picture.
- Product managers and executives who need to review architecture decisions.