The problem

The client's laptop catalog was already large and growing fast, and keyword search couldn't keep up. A query like "something lightweight for heavy video editing under 1.5kg" has no keywords worth matching, it's a bundle of constraints (weight, GPU class, use case) wrapped in a sentence, and the existing engine only understood tokens. Engagement and conversions both took the hit.

The approach

Instead of embedding the raw query and hoping cosine similarity sorts it out, the pipeline extracts structured intent first: Gemini Flash turns "something for Premiere but I travel a lot" into actual filters, weight ceiling, GPU tier, use case, before anything touches the database. Semantic search with e5-base-v2 embeddings in pgvector handles the fuzzy matching on top of those filters, and the response streams back over SSE so the wait never feels like a wait.

What made it fast

The obvious guess, that the LLM call was the bottleneck, was wrong. The real cost was re-embedding every query from scratch and an untuned pgvector index sitting on 50k+ products. Caching the embedding layer and adding an ivfflat index fixed both, taking the pipeline from roughly 1.1 seconds to under 200ms end to end, with Redis absorbing most of the remaining round-trips to Postgres for catalog data that barely changes. The full breakdown, and what didn't work on the first try, is in the writeup.

Stack

FastAPI · PostgreSQL + pgvector · Redis · GPT-4o · Gemini Flash · Sentence Transformers · Docker · Python

Result

Sub-200ms end to end on real traffic, holding 99.5% uptime in production. More to the point, search stopped being a list of tokens to match and started working like advice: someone describes what they need, and the engine reasons about it and says why.