Embeddings for a catalogue: 111,985 vectors to maintain

A catalogue gets an embedding index when search has to understand the request instead of copying a word: 111,985 components become 111,985 vectors to keep aligned.

What gets embedded

• Name, category and tags: that is the text people use to describe what they want.
• Truncated description: the first characters are enough; the rest dilutes the vector.
• Code excluded: 177,656 bytes of snippets add no meaning to search.

import OpenAI from 'openai';

const openai = new OpenAI();

async function embedBatch(items: { id: string; text: string }[]) {
  const res = await openai.embeddings.create({
    model: 'text-embedding-3-small',
    input: items.map((i) => i.text.slice(0, 2000)),
    encoding_format: 'float',
  });

  return res.data.map((d, i) => ({ id: items[i].id, embedding: d.embedding }));
}

Do the maths first

The cost of one pass is input tokens divided by a million, times $0.02. The cost that surprises people is the second one: changing model means regenerating every vector, so store the dimension next to the data and read the new price before migrating.

curl -s "https://ui.eneawork.it/api/stats" | node -e "let s='';process.stdin.on('data',d=>s+=d).on('end',()=>{const j=JSON.parse(s);console.log(j.components*1536*4,'bytes')})"

The command reads the catalogue and multiplies by the declared dimension: rerun it when the catalogue grows instead of trusting the number written here.

Update, do not rebuild

A catalogue changes every day: embed new rows, delete removed ones and regenerate only what changed. Rebuild the index when recall drops, not on every insert.

The limitation

656 MiB is only the start: the HNSW index adds in-memory structures and has to be sized. And we have no quality measurement on this catalogue, because it would need a query set with expected answers that we have not published.

Where to continue

The search choice is in semantic search on a site, the database in SQLite or Postgres, catalogue structure in what a registry is. The records: 111,985 components, 1,556 blocks, Aceternity UI.