RAG + 3D Chat

Problem

Generic chatbots hallucinate when asked about private documentation. Users need answers grounded in their own files, with citations. When responses contain structural or spatial data — floor plans, graph layouts, 3D coordinates — plain text is insufficient; the user needs an interactive view.

What this demonstrates

• Full RAG pipeline: embed → store → retrieve → rerank → generate, with Supabase pgvector as the vector store
• Cross-encoder reranking step that improves retrieval precision beyond cosine similarity alone
• Conditional Three.js rendering — LLM response is parsed for spatial data markers and rendered in a 3D viewer when present

Architecture

Tech stack

• OpenAI Embeddings: text-embedding-3-small at 1536 dimensions; good cost/quality trade-off for document retrieval.
• Supabase pgvector: Cosine similarity search via a custom match_documents RPC; no separate vector DB required.
• Thesys C1: Structured UI generation from LLM output, used for the chat response rendering layer.
• Three.js: 3D scene rendered conditionally when the LLM response contains spatial data markers.

Code

// Retrieval + reranking pipeline
async function retrieve(query: string, topK = 5) {
  const embedding = await openai.embeddings.create({
    model: 'text-embedding-3-small',
    input: query,
  });

  const { data } = await supabase.rpc('match_documents', {
    query_embedding: embedding.data[0].embedding,
    match_threshold: 0.7,
    match_count: topK,
  });

  return data.map((doc) => doc.content);
}

Results

• Reference implementation — live demo uses mock documents; full document upload available in authenticated dashboard
• Retrieval with reranking achieves higher precision@3 than cosine-only on test queries against the portfolio codebase
• Rate-limited to 10 queries/hour (unauthenticated) via Upstash sliding window

Screenshots

GitHub

The retrieval pipeline, Supabase vector schema, and chat API route are all in this repo.