I Built a Search Engine From Scratch Using Linear Algebra

Search feels effortless. Type a few words, press Enter, and a ranked list appears almost immediately.

That convenience made me curious about the machinery underneath it. How does a computer decide that one document deserves to appear above another?

I wanted to build the ranking mechanism myself. No Elasticsearch, embedding API, or LLM shortcut. The constraint was deliberately simple: start with text, use linear algebra, and see how far it goes.

What came out of that experiment was a compact search engine built around TF-IDF, cosine similarity, matrix multiplication, truncated SVD, and a few information-retrieval metrics. More interestingly, every piece was motivated by a limitation in the previous version.

The question: how can a computer turn a query into a ranked list of useful documents using only the geometry of vectors?

Start with a small corpus

I began with four tiny documents:

Machine learning predicts stock prices.
Deep learning is used for image recognition.
Stock market analysis using machine learning.
Football teams use statistics for match predictions.

For the query machine learning stock market, a human can make a reasonable call immediately. Document 3 should lead, Document 1 should be close behind, and the others should fall away.

The computer has no intuition. At first, it only has strings. So the first challenge was turning language into something I could compare mathematically.

Attempt one: count words

I collected every unique term into a vocabulary:

V = [\text{machine}, \text{learning}, \text{stock}, \text{market}, \text{deep}, \text{image}]

Each document became a vector of word counts. In this toy corpus, the first and third documents can be written as:

d_1 = [1, 1, 1, 0, 0, 0]

d_3 = [1, 1, 1, 1, 0, 0]

The query uses the same coordinate system:

q = [1, 1, 1, 1, 0, 0]

At that point, the problem changed shape. Text was now geometry. Instead of asking which sentence sounded most similar, I could ask which document vector was closest to the query.

From text to a vector

RAW TEXT
"machine learning stock market"
             │
             ▼
      tokenize + clean
             │
             ▼
      map to vocabulary
             │
             ▼
   [ 1, 1, 1, 1, 0, 0 ]
             │
             ▼
        VECTOR SPACE

This baseline worked, but it had a major blind spot: it treated every word as equally important.

TF-IDF: make rare words count more

In a large collection, words such as model, data, learning, and system show up almost everywhere. They matter, but they rarely distinguish one document from another. A word such as Kubernetes, melanoma, or liquidity can reveal much more about a document's topic.

TF-IDF captures both ideas.

Term frequency measures a word's importance inside a document:

TF(t, d) = \frac{f_{t,d}}{\sum_k f_{k,d}}

Inverse document frequency measures how rare that word is across the corpus:

IDF(t) = \log\left(\frac{N}{df(t)}\right)

Here, $N$ is the number of documents and $df(t)$ is the number of documents containing the term $t$ . Multiply them together:

TF\text{-}IDF(t, d) = TF(t, d) \times IDF(t)

The formula becomes much less abstract once it is implemented. TF asks, “Does this word matter in this document?” IDF asks, “Does this word tell me anything unusual about the corpus?” Together, they produce a useful weight.

                  TERM
                   │
        ┌──────────┴──────────┐
        ▼                     ▼
 frequent in this       frequent across
    document?             the corpus?
        │                     │
        ▼                     ▼
       TF                    IDF
        └──────────┬──────────┘
                   ▼
              TF-IDF weight

The corpus becomes a matrix

With $m$ documents and $n$ vocabulary terms, the whole collection becomes a document-term matrix:

X \in \mathbb{R}^{m \times n}

Every row is a document. Every column is a vocabulary term. After applying TF-IDF, a small example looks like this:

X = \begin{bmatrix} 0.42 & 0.31 & 0.56 & 0.00 \\ 0.00 & 0.28 & 0.00 & 0.00 \\ 0.35 & 0.26 & 0.47 & 0.61 \\ 0.00 & 0.00 & 0.00 & 0.00 \end{bmatrix}

The query is transformed into the same space:

q = [0.37, 0.29, 0.52, 0.58, \ldots]

This representation matters because it keeps the comparison fair. Query and document coordinates mean the same thing, so a mathematical similarity score now has a real interpretation.

Why cosine similarity wins

My first instinct was Euclidean distance. It seems natural: vectors that are close together should represent similar documents.

Text makes that misleading. Consider:

A = [1, 1], \qquad B = [10, 10]

Their Euclidean distance is large, even though they point in exactly the same direction. One vector is simply longer. That is common in text because a 2,000-word article naturally has larger counts than a 200-word note.

For ranking, direction is often more important than magnitude. Cosine similarity measures the angle between two vectors:

\cos(\theta) = \frac{q \cdot d}{\|q\|\|d\|}

For non-negative TF-IDF vectors, a score close to 1 means the vectors point in a similar direction; a score close to 0 means there is little overlap.

                        query q
                           ↗
                         ↗ θ
                       ↗
origin ●────────────────────────► document A
 
origin ●──────────────► document B
         \
          \
           ▼ document C
 
smaller angle  →  higher cosine similarity  →  better rank

The useful shortcut: one matrix multiplication

Computing cosine similarity document by document works, but the matrix view makes it cleaner. First, normalize each document and the query:

\hat{d_i} = \frac{d_i}{\|d_i\|}, \qquad \hat{q} = \frac{q}{\|q\|}

Stack the normalized document vectors into $\hat{X}$ . Then the entire search operation is:

\boxed{s = \hat{X}\hat{q}}

The result is a score vector. Each element is the cosine similarity between the query and one document, ready to sort from highest to lowest.

USER QUERY
"stock market ML"
       │
       ▼
tokenize + TF-IDF transform
       │
       ▼
normalized query vector q̂
       │
       ├──────────────► normalized document matrix X̂
       │                              │
       └──────────────►       X̂ × q̂
                                      │
                                      ▼
                         [ 0.71, 0.24, 0.93, 0.05 ]
                                      │
                                      ▼
                              D3 → D1 → D2 → D4

That compact multiplication is the core of the first working search engine.

Where lexical search stops

The next failure was more subtle. A query such as AI for stock prediction should have some relationship with a document about machine learning for financial forecasting.

People see the connection. A bag-of-words model does not. In a TF-IDF matrix, AI, machine learning, prediction, and forecasting occupy different columns. Without exact or near-exact vocabulary overlap, the model has little to work with.

That is the boundary of lexical search: it matches words very well, but it does not understand concepts.

I could have jumped directly to a modern embedding model, but I wanted to see whether the same linear algebra toolkit could push the baseline a little further.

SVD and the beginning of latent structure

Singular Value Decomposition factorizes the document-term matrix into three parts:

X = U\Sigma V^T

$U$ captures document-side patterns, $\Sigma$ stores the importance of those patterns, and $V^T$ captures term-side patterns.

The practical move is to keep only the largest $k$ singular values:

X_k = U_k\Sigma_kV_k^T, \qquad k \ll n

This is truncated SVD. In text retrieval, it is the foundation of Latent Semantic Analysis. Rather than compare documents only through individual vocabulary dimensions, the engine can compare them through a smaller number of shared latent patterns.

TF-IDF MATRIX X                         LOWER-DIMENSIONAL SPACE
many word dimensions                    latent concepts
 
D1  █ ░ █ ░ ░ █ ░ █                    concept 1  ▲       ● D1
D2  ░ █ ░ ░ █ ░ █ ░                              │    ● D3
D3  █ █ █ ░ ░ █ ░ █                   ───────────┼────────────► concept 2
D4  ░ ░ ░ █ █ ░ █ ░                              │ ● D2
                 │
                 │  truncated SVD
                 ▼
            X = U Σ Vᵀ

The reason this is more than a compression trick is the Eckart-Young theorem. Truncated SVD gives the best rank- $k$ approximation under the Frobenius norm:

X_k = \underset{\operatorname{rank}(B) \leq k}{\arg\min}\ \|X - B\|_F

It is not simply dropping columns. It is finding the lower-dimensional representation that retains as much useful structure as possible according to a precise objective.

Evaluating the engine instead of trusting my intuition

“The results look good” is not a reliable evaluation method. I created a small set of queries, manually marked relevant documents, and compared three versions of the system:

Bag of words plus cosine similarity
TF-IDF plus cosine similarity
TF-IDF plus truncated SVD plus cosine similarity

I used three common metrics.

Precision@K measures how many of the first $K$ results are relevant:

Precision@K = \frac{\text{relevant documents in the top K}}{K}

Recall@K measures how many of all relevant documents were found in those top results:

Recall@K = \frac{\text{relevant documents in the top K}}{\text{total relevant documents}}

Mean Reciprocal Rank rewards systems that place the first relevant result near the top:

MRR = \frac{1}{|Q|}\sum_{i=1}^{|Q|}\frac{1}{rank_i}

The experiment made the trade-offs visible. TF-IDF was a strong and efficient baseline for exact terminology. SVD helped when related documents used different vocabulary, but it did not create the contextual understanding of a modern embedding model.

The architecture I ended up with

DOCUMENTS
    │
    ▼
text processing
    │
    ▼
TF-IDF matrix X
    │
    ├────────────────────── lexical search
    │
    └──── truncated SVD ─── latent vector space
                                  │
                                  ▼
                         cosine similarity
                                  │
                                  ▼
                              ranked results

The interface is still just a search box. Underneath it, the query passes through a small mathematical pipeline that turns text into weights, weights into vectors, and vectors into an ordered list.

What this engine cannot do

This system has clear limits. TF-IDF depends heavily on vocabulary overlap, and SVD only captures broad co-occurrence patterns.

It cannot reliably distinguish “Apple released a new processor” from “I ate an apple.” It also struggles with composition and negation: “the movie was not good” contains the word good, but the sentence is negative.

Those gaps explain the shift toward learned representations. Modern embedding models map queries and documents into dense vectors whose coordinates are learned from language data rather than assigned directly to words. Once those embeddings exist, though, the geometry is familiar:

\frac{q \cdot d}{\|q\|\|d\|}

Cosine similarity is still there. The representations have become much richer, but the underlying idea remains recognisable.

What I took away

Before building this, TF-IDF, cosine similarity, and SVD felt like separate topics from different classes. Implementing a search engine connected them.

Documents became vectors. The corpus became a matrix. Relevance became an angle. Ranking became matrix multiplication. Dimensionality reduction became an optimization problem.

The most useful lesson was not a single formula. It was learning that a problem that appears linguistic can be translated into geometry. Once I saw that translation, the foundations of modern search felt far less mysterious.

Sometimes the fastest route to understanding a sophisticated system is to strip away the advanced tooling and rebuild the simplest version yourself.