There's a limitation baked into every AI model that's easy to overlook: it only knows what it was trained on. Everything in its training data is available to it. Everything outside that data is invisible. Your company's internal documents, last week's news, the specific contract you're asking about — none of it exists from the model's perspective, because none of it was in the training set.
This is fine for general questions. It's a real problem for anything specific.
Retrieval-augmented generation (RAG) is the practical solution. Instead of asking the model to answer from memory alone, a RAG system first searches a knowledge base for content relevant to the question, then passes that content to the model as context. The model answers using both what it knows and what it just found. The result is a system that can work with your documents, your data, and current information, without requiring you to retrain the model every time something changes.
The search step is where most of the interesting engineering happens. Semantic search finds content based on meaning rather than exact keywords, using semantic similarity to match a query to relevant passages even when they don't share the same words. Vector search is the mechanism underneath: queries and documents are both converted to embeddings, and the search finds the embeddings that are closest in mathematical space. Hybrid search combines semantic search with traditional keyword matching, getting the benefits of both.
The retrieved content has to live somewhere. A vector database stores embeddings at scale and makes fast similarity search possible. A vector store is a lighter-weight version of the same idea, often used in smaller applications or as a component within a larger system.
RAG is one of the most practically useful ideas in applied AI right now. It's the reason AI tools can work with your specific knowledge base, answer questions about your products, and stay current without constant retraining. The sections ahead get into how it actually works at each step.
