Vector Databases Explained: Search by Meaning, Not Keywords

Vector databases explained in one sentence: a vector database stores lists of numbers that represent the meaning of text, images or other data, and finds the items whose numbers are closest to the numbers of your question. That makes it possible to search by meaning instead of by exact words, so a query for “how do I cancel my plan” can find a help article titled “Ending your subscription”. Vector databases sit behind many assistants that answer questions from your own documents, and behind recommendation and similar-image features.

This guide explains what the numbers are, how nearest-neighbour search works, what the indexes trade away, how a typical document search is built and, most usefully, when you do not need a vector database at all.

From text to vectors

A vector is simply an ordered list of numbers, such as 0.12, -0.48, 0.91 and so on, usually several hundred or a few thousand long. An embedding model produces one for each piece of text. Texts with similar meaning get vectors that point in similar directions, and texts about unrelated topics end up far apart. Our guide to AI embeddings describes this step in detail, and the same idea works for images and audio when the model supports them.

You can picture each vector as a point on a map with hundreds of dimensions. You cannot draw it, but the geometry still works: close points mean similar meaning. The embedding model matters a great deal. A model trained on general English text will do poorly on legal Latin or on source code, and a model trained for one language may handle another language badly. Whatever you store is only as good as the model that created the numbers, and you must use the same model for the stored items and for the queries.

How similarity is measured

To find what is “close”, the database needs a distance or a similarity score. The common choices are cosine similarity, which compares the angle between two vectors, the dot product, which also takes length into account, and Euclidean distance, which measures the straight-line gap. For most text embeddings the vectors are normalised to the same length, and the three measures then give the same ranking. Your embedding model’s documentation tells you which one to use, and you should follow it, because the wrong measure silently lowers quality.

A similarity score is not a probability and it has no universal meaning. A score of 0.8 may be excellent for one model and mediocre for another. Do not set thresholds from a blog post. Test with your own questions and a handful of known good answers, and choose a cut-off that works for your data.

Nearest-neighbour search and its shortcuts

The core operation is nearest-neighbour search: given a query vector, return the k stored vectors closest to it. The simplest version compares the query with every stored vector. That is exact, and it is fine for thousands or even a few hundred thousand items. For millions, it becomes too slow, so databases use approximate nearest-neighbour search. They build an index that lets them look at a small part of the data and return results that are very likely, but not guaranteed, to be the true closest ones. The overview of nearest-neighbour search lists the families of methods.

Two index styles are widely used. Graph indexes link each vector to a handful of neighbours and walk the graph from the query towards the closest points. They are fast and accurate, but they use a lot of memory. Cluster-based indexes divide the space into regions and search only the nearest regions. They use less memory and can be tuned, but they miss more items. Compression methods, such as storing shorter approximations of each vector, cut memory further at a small cost in accuracy. The key idea is a three-way trade between speed, memory and recall, where recall means how many of the true best matches the search finds.

Approach	Accuracy	Speed at scale	Memory	Best for
Brute force (compare with everything)	Exact	Slow beyond a few hundred thousand items	Low	Small collections, tests, prototypes
Graph index	High, approximate	Fast	High	Low-latency search on large data
Cluster index	Medium to high, tunable	Fast	Medium	Very large collections with a memory limit
Compressed vectors	Lower	Fast	Low	Huge collections on modest hardware

What a vector database adds

A vector database, as the general overview of vector databases notes, is more than an index. It stores the vector together with an identifier and metadata, such as the document title, the language, the date and the access rights. It lets you combine similarity search with filters, for example “closest chunks, but only from documents this user may read, written after January”. It handles inserts, updates and deletes, keeps data on disk, replicates it and scales across machines. Those features matter when the data changes and when several people use it.

Filtering is harder than it sounds, because the best matches may be excluded by the filter and the search must still return enough results. Good systems filter during the search, not afterwards. Ask how a product handles it before you commit to one. Access control deserves particular care: if a search index mixes documents that different people may see, a mistake leaks private information, so enforce permissions in the query and test them.

Vector databases explained through a document search

The most common use is answering questions from your own documents, the pattern known as retrieval-augmented generation, described in our guide to RAG AI. The steps are always similar. First you split each document into chunks, often a few hundred words each, with a small overlap so ideas are not cut in half. Then you create an embedding for each chunk and store it with its text and metadata. When a user asks a question, you embed the question, retrieve the closest chunks, and give them to a language model together with the question. The model writes an answer based on those chunks and, ideally, cites them.

Quality depends on every step. If chunks are too large, the retrieved text contains noise. If they are too small, they lose context. If the embedding model does not understand your domain, the right chunk never appears. If the model receives the wrong chunk, it answers confidently from it, and the result is a polished mistake. Keep in mind the limited size of the prompt, covered in AI context windows, and test the whole chain with real questions, not only the database.

Keyword search, vector search and hybrid search

Vector search is not always better than keyword search. It handles paraphrases, synonyms and questions in natural language. It struggles with exact identifiers, product codes, names and rare terms, which keyword search handles very well. A search for “error 0x80070057” or for a surname should match the exact string. Many production systems therefore combine both: keyword search and vector search each return candidates, and a ranking step merges them. This is called hybrid search, and it often beats either method alone.

Add a re-ranking step when quality matters. A second model reads the top twenty or fifty results together with the question and orders them more carefully. It costs extra time, but it improves the top few results, which are the ones that matter. Measure everything with a small test set of real questions and the answers you expect, and track recall at k, meaning how often the right chunk appears in the top k results.

When you do not need a vector database

Many projects add a vector database too early. If you have a few thousand documents, you can store the vectors in a file or an ordinary table and compare them directly. If your data already lives in a relational database, an extension that adds a vector type and an index may be enough and keeps everything in one place, with one backup and one set of permissions. If your search is mainly about exact words, filters and dates, a normal search engine does the job. The right question is not “which vector database?” but “what is the smallest system that finds the right answer for my users?”

A dedicated vector database makes sense when you have millions of vectors, a high query rate, strict latency targets, frequent updates and a team that can run it. Even then, plan for the cost: embedding all your data, storing the vectors, re-embedding when you change the model and keeping the index in sync with the source. Re-embedding is a hidden project. If you switch models, every stored vector must be recreated.

Privacy, deletion and security

Embeddings are derived from your data, so they carry information about it. Research has shown that, in some cases, text can be partly reconstructed from its embedding, so treat vectors as sensitive as the source. When a person asks you to delete their data, delete the chunks and the vectors, and the copies in backups according to your policy. Keep metadata for access control with each vector and check it on every query.

There is also a security angle. If retrieved text contains instructions, a model may follow them, which is the problem described in prompt injection. Treat retrieved content as untrusted data. Our guide to AI hallucinations explains why an answer built from the wrong chunk still reads well, and why citations and spot checks are essential.

How Ask Mio fits in

You do not need a vector database to use Ask Mio. Mio is one assistant with five modes, and you never pick a model: Mio routes each request to a suitable one. On paid plans you can upload PDF, Word, text, code or images and ask questions about them, and Research mode searches the web and cites its sources, so you can work with your documents without building any infrastructure. Our guide to AI document analysis explains how to ask good questions of a file.

If you do build a search system yourself, Mio is useful as a thinking partner. Chat and Write on the free plan can explain a concept, compare options and draft a test plan, and Coding adds Code mode and a sandbox for small scripts. Memory can keep your chosen embedding model, chunk sizes and test questions in one project. Chats and files stay yours, can be exported or deleted and are not used to train models. Always verify technical details against the documentation of the tools you choose.

Frequently Asked Questions

What is a vector database in simple terms?

It is a database built to store lists of numbers that represent meaning and to find the stored items closest to a query’s numbers. Instead of matching exact words, it matches similar meaning, so a question can find a relevant passage that uses different words. It usually also stores text and metadata alongside each vector.

Do I need a vector database for a small project?

Usually not. For a few thousand documents you can store vectors in a file or a normal table and compare them directly, which is simple and exact. A dedicated system pays off with millions of items, high traffic, frequent updates and strict speed targets. Start simple and measure before you add infrastructure.

What is the difference between vector search and keyword search?

Keyword search matches the words you typed, so it is excellent for names, codes and exact phrases. Vector search matches meaning, so it handles paraphrases and natural questions. Each fails where the other works, which is why many systems combine them in hybrid search and then re-rank the results.

Why does approximate search sometimes miss the best result?

Approximate indexes look at only part of the data to stay fast, so they may skip a true closest match. You can raise accuracy by searching more candidates, at the cost of speed and memory. Measure recall on your own test questions and choose a setting that is accurate enough for your users.

Which Ask Mio plan should I use to ask questions about my documents?

The free plan has Chat and Write only. Chat at the next level adds Research with web search, file uploads for PDF, Word and images, and memory. Coding adds Code mode and a sandbox. Check the live pricing page for current points and file size limits before choosing.

The Bottom Line

Vector databases explained simply are search engines for meaning: they store embeddings and return the closest ones quickly, with filters and updates on top. Use them when your collection is large and busy, start with something simpler when it is small, combine them with keyword search and measure quality with real questions. You can try document questions on the free plan or compare options on the Ask Mio pricing page.