← Back to Blog Feed
Lab
2026-08-25
11 min read

Vector Embeddings and Semantic Search: Building RAG Pipelines from Scratch

Vector Embeddings and Semantic Search: Building RAG Pipelines from Scratch

Vector Embeddings and Semantic Search: Building RAG Pipelines from Scratch

Traditional databases search for exact string matches using SQL queries like WHERE title LIKE '%payment%'. But what happens if a user searches for "how do I send money to my supplier?"

Keyword search fails because the word "payment" is never explicitly stated.

To bridge this gap, modern AI systems use Vector Embeddings and Retrieval-Augmented Generation (RAG). At Kone AI, we teach engineers how to build semantic retrieval engines from first principles.

1. What is an Embedding?

An embedding model (like text-embedding-3-small) maps arbitrary text into a dense vector of numbers in high-dimensional space (e.g. 1,536 dimensions).

In this geometric space, texts with similar meanings sit close to each other:

The Math: Cosine Similarity

To measure how relevant two documents are, we calculate the cosine of the angle between their vectors A\mathbf{A} and B\mathbf{B}:

Cosine Similarity=A⋅B∥A∥∥B∥=∑i=1nAiBi∑i=1nAi2∑i=1nBi2\text{Cosine Similarity} = \frac{\mathbf{A} \cdot \mathbf{B}}{\|\mathbf{A}\| \|\mathbf{B}\|} = \frac{\sum_{i=1}^{n} A_i B_i}{\sqrt{\sum_{i=1}^{n} A_i^2} \sqrt{\sum_{i=1}^{n} B_i^2}}

2. The 4-Stage RAG Pipeline

ARCHITECTURE SCHEMATIC
flowchart TD subgraph Ingestion["1. Document Ingestion Pipeline (Offline Indexing)"] Docs["Raw Corporate Documents (PDF / Markdown / Code)"] Chunk["Recursive Chunking (512 tokens + 10% overlap)"] EmbedDense["Dense Embedding Model (text-embedding-3)"] VecDB[("Vector Store (pgvector / HNSW Index)")] Docs --> Chunk --> EmbedDense -->|1536-dim Float Vectors| VecDB end subgraph Retrieval["2. Real-Time Query & Retrieval Pipeline (Online)"] Query["User Query Prompt"] EmbedQuery["Query Vector Embedding"] SimSearch{"Cosine / Dot-Product Distance Match"} TopK["Top-K Relevant Context Chunks"] Query --> EmbedQuery --> SimSearch VecDB -.->|HNSW Approximate Nearest Neighbors| SimSearch SimSearch --> TopK end subgraph Generation["3. Context-Augmented Generation"] PromptAssembler["System Prompt + Injected Context Chunks"] LLM["Foundation LLM (Claude / Gemini / GPT)"] Output["Accurate, Grounded Hallucination-Free Answer"] TopK --> PromptAssembler Query -.-> PromptAssembler PromptAssembler --> LLM --> Output end style Ingestion fill:#080d1a,stroke:#334155,stroke-width:1px,color:#f8fafc style Retrieval fill:#080d1a,stroke:#334155,stroke-width:1px,color:#f8fafc style Generation fill:#080d1a,stroke:#334155,stroke-width:1px,color:#f8fafc style Docs fill:#0f172a,stroke:#334155,stroke-width:1px,color:#f8fafc style Chunk fill:#0f172a,stroke:#334155,stroke-width:1px,color:#f8fafc style EmbedDense fill:#0f172a,stroke:#00d4ff,stroke-width:1px,color:#f8fafc style VecDB fill:#0f172a,stroke:#00d4ff,stroke-width:2px,color:#f8fafc style Query fill:#0f172a,stroke:#334155,stroke-width:1px,color:#f8fafc style EmbedQuery fill:#0f172a,stroke:#38bdf8,stroke-width:1px,color:#f8fafc style SimSearch fill:#1e293b,stroke:#f59e0b,stroke-width:2px,color:#f8fafc style TopK fill:#0f172a,stroke:#10b981,stroke-width:1px,color:#f8fafc style PromptAssembler fill:#0f172a,stroke:#334155,stroke-width:1px,color:#f8fafc style LLM fill:#0f172a,stroke:#10b981,stroke-width:2px,color:#f8fafc style Output fill:#0f172a,stroke:#10b981,stroke-width:2px,color:#f8fafc

3. Building a Pure TypeScript Similarity Search

Here is how you compute vector similarity in TypeScript without external dependencies:

export function cosineSimilarity(vecA: number[], vecB: number[]): number {
  if (vecA.length !== vecB.length) {
    throw new Error('Vector dimensions must match');
  }

  let dotProduct = 0;
  let normA = 0;
  let normB = 0;

  for (let i = 0; i < vecA.length; i++) {
    dotProduct += vecA[i] * vecB[i];
    normA += vecA[i] * vecA[i];
    normB += vecB[i] * vecB[i];
  }

  const denominator = Math.sqrt(normA) * Math.sqrt(normB);
  if (denominator === 0) return 0;

  return dotProduct / denominator;
}

Dive deep into PyTorch, RAG architectures, and vector search in our Neural Architectures & Vector Search Track.

Register at Kone School

Cohort positions are open. Build physical robotics firmware, structured web code, and master AI pathways through hands-on project systems.

Join Cohort (WhatsApp)