A vector database stores numerical representations of content and retrieves candidates whose vectors are close to a query vector. In a RAG application, that helps select material for a model to read. A similarity score measures the chosen mathematical relationship; it does not establish factual accuracy, freshness, or permission to use the underlying content.
What goes into a vector database?
An embedding model converts a document, image, or other input into a vector. The database stores that vector together with an identifier and useful metadata, such as the source URL, document version, and access rules. Keep a way to recover the original text: a vector alone is not the document or its evidence.
At query time, the application produces a query vector in the same compatible embedding space. The retrieval system compares that vector with stored vectors and returns candidates. Different models and distance measures can produce different rankings. Microsoft provides a useful overview of vector database components.
A small example: similarity is a ranking, not an answer
The following vectors are hand-chosen teaching inputs, not embeddings generated from real documents. Run this example with Python 3. It calculates cosine similarity so that the distinction between a mathematical score and a verified answer is visible.
from math import sqrt
def cosine(left, right):
dot = sum(a * b for a, b in zip(left, right))
denominator = sqrt(sum(a * a for a in left)) * sqrt(sum(b * b for b in right))
if denominator == 0:
raise ValueError("A zero vector has no cosine direction")
return dot / denominator
query = [1.0, 0.0]
documents = {
"Document A": [1.0, 0.0],
"Document B": [0.0, 1.0],
"Document C": [-1.0, 0.0],
}
ranked = sorted(
((name, cosine(query, vector)) for name, vector in documents.items()),
key=lambda item: item[1],
reverse=True,
)
for name, score in ranked:
print(f"{name}: {score:.2f}")The output is:
Document A: 1.00
Document B: 0.00
Document C: -1.00Document A ranks first because its vector points in the same direction as the query. Nothing in this calculation reads a source, checks a date, or examines an access policy. A real application must attach those checks to the retrieved candidates.
Exact and approximate indexes answer different engineering questions
An index determines how candidate vectors are found. An exact, exhaustive search evaluates the stored vectors against the chosen metric. An approximate index aims to reduce retrieval work, with results that depend on its configuration and the data. The official Faiss index documentation distinguishes exhaustive Flat search from non-exhaustive options such as HNSW and IVF.
| Choice | What it gives you | What to verify |
|---|---|---|
| Exact search | A reference result for the chosen vectors and metric | Whether that metric retrieves useful material for actual questions |
| Approximate search | A way to reduce the work of finding candidates | Recall against the reference, latency, memory, and behavior on your own data |
| Metadata or access filtering | A way to restrict eligible candidates | That the restriction is enforced by the application and cannot be bypassed by retrieval |
Do not turn an index recall measurement into an answer-accuracy claim. Finding a nearest neighbor under a metric and finding a trustworthy answer are separate evaluations. This is why a generic speed or accuracy percentage is not enough to choose an index.
Where the database sits in a RAG workflow
- Prepare sources. Collect material you can use, retain source identifiers, and record versions or retrieval times.
- Extract and split content. Keep meaningful passages and preserve their connection to the source. The SearchCans Reader API can prepare accessible web pages as Markdown; it is a content extraction component, rather than your embedding model or vector database.
- Embed and index. Create vectors with the selected model and store them with their source metadata.
- Retrieve eligible candidates. Apply the query and required access constraints, then inspect the passages returned.
- Generate and verify. Give the model relevant passages, preserve citations, and handle insufficient or conflicting evidence explicitly.
For a broader system view, see RAG architecture and evaluation. For the separate task of combining lexical and vector retrieval, see hybrid search for RAG. Those implementation tasks should not be confused with this page’s basic explanation.
A practical decision checklist
- Can each retrieved passage be traced back to a source and version?
- Have you selected an embedding model and a compatible similarity metric?
- Do exact-match identifiers require lexical retrieval or filters?
- Have you tested relevant questions, including cases where the answer is absent?
- Are access restrictions, updates, and deletion handled outside the similarity score?
- Does the application distinguish retrieval quality from the accuracy of its final answer?
Start with a retrieval baseline that you can inspect. Change the index or add other retrieval methods when your own evaluation identifies a specific limitation, and retain the same test questions so the comparison remains meaningful.
Frequently asked questions
Q: Does a vector database give an AI a permanent memory?
A: It can store material that an application retrieves for a model, but the application must maintain that material, its versions, and its access rules. Retrieval does not change the model’s training or guarantee that the model will use every stored fact correctly.
Q: Is a high similarity score proof that a passage is correct?
A: No. It is a result of the selected vectors and metric. Verify the source, its date, and the claim itself before treating a retrieved passage as evidence.
Q: Does SearchCans provide the vector database in this workflow?
A: This workflow uses SearchCans for web discovery or content extraction. You select and operate the embedding model, storage, retrieval rules, and evaluation separately.
