The decision
The team wants a vector database service for its first retrieval feature, which means another server, another contract and another on-call rota, so do you actually need one? This one is about what retrieval quality really depends on, and what a bad day with the index costs you.
The problem it removes
Retrieval projects often start with an infrastructure purchase before anyone has measured whether search is even the bottleneck, and the service then shapes the architecture, adds cost and operations, and makes the index hard to inspect or roll back. So when an ingestion job goes wrong, nobody can say what the assistant saw yesterday.
What I ran
I ran it in production first: Mimir, the team knowledge base I built for my lab, indexes our meetings, mail, chat, tickets and code every night on our own server.
On a test of 250 job postings and 40 questions, the biggest gain came before any database choice: the company and title on top of each chunk took the small model from 11 to 21 right first answers, where a model fifteen times larger managed 10 without it. With that line, keyword search alone matched the small model at 32 of 40 in the top five.
Hybrid search with the larger model put 39 of 40 in the top five.
Then I broke it on purpose: re-indexing with plain fixed-size chunks and no title line dropped right first answers from 21 to 9, while the previous version stayed queryable, and one call restored it in a third of a second.
No server, 2 to 6 milliseconds per query, and 30 MB on disk with every version kept.
Where it sits in the stack
Inside the application, as a library that reads and writes a folder of files on local disk or object storage. It keeps the chunks, their vectors and any metadata together and serves vector, keyword and hybrid queries from the same table, and every write makes a new version, so the index has a history.
The first 30 days
Week 1. Put the retrieval index in a folder next to the application, have the team write 40 questions real users ask, each with the document that answers it, and measure keyword, vector and hybrid on them, with and without the document's title on each chunk, before you pick any model.
Weeks 2 to 4. Tag the version behind every release, keep old versions for as long as your audits need, and rehearse a rollback, then move the folder to object storage once more than one service reads it.
Skills. One engineer comfortable with Python and Arrow data, and no database administrator.
Running cost. Storage for the files, and the compute is your application's.
Working at day 30 means retrieval quality has a number, you know the index version behind every answer, and rolling back takes one command.
Risk
Churn. A hundred releases in six months, currently a run of betas, and the keyword-index call most tutorials still use (create_fts_index) has been deprecated since version 0.25, so pin the version and read the changelog before upgrading.
Backlog. 689 open issues, the most of any repo in this series, so search the tracker for your storage backend before committing to it.
What I did not test. Several services writing at once, object storage, and whether a restore brings the keyword index back without a rebuild, so test all three before production.
Vendor. The company behind it sells a managed cloud and an enterprise edition, but the embedded library and the file format are open source, so the data stays yours.
The organisational risk is an index nobody owns, so name who approves a re-index and who can roll one back.
Build, adopt or buy
Build. Arrays in memory and a cosine similarity work fine for a demo, but versions, keyword search, filtering and files that outlive the process take months.
Buy. Managed vector databases scale writes, replicas and access control without your team running anything, which is worth it at high write volume or with many tenants.
Adopt the embedded library first, measure retrieval on your own questions, and let the numbers decide whether you ever need a service.
Verdict
I'd adopt it. It needs no server, since the database is a folder of files, it kept every version of the index, and let me measure keyword against vector against hybrid in an afternoon, and the keyword result is the one to take to the budget meeting, because it says measure before you buy. Pin versions, test concurrent writers before you scale out, and make rollback part of the release checklist.
Engineering notes
uv pip install lancedb: 0.39.0, 147 MB plus pyarrow 125 MB. No server:db = lancedb.connect("path/or/s3-uri").- Create and index:
tbl = db.create_table(name, data=rows)withvectorandtextfields, keyword index withcreate_indexandFTS()(create_fts_indexworks but is deprecated since 0.25). - Queries:
tbl.search(vec),tbl.search(text, query_type="fts"),tbl.search(query_type="hybrid").vector(vec).text(text), each with.limit(5).to_list(). - Versions:
tbl.version,tbl.list_versions(),tbl.checkout(n)to read an old version,tbl.checkout_latest(),tbl.restore(n). I rebuilt the keyword index after restoring. - Hybrid with the larger model: 39 of 40 in the top five, but right first slightly less often than vector alone.
- Context line:
f"{company} | {title}\n{chunk}"before embedding and indexing; store the unprefixed text for display if you prefer. - Reproducibility: vector results matched my earlier in-memory runs of the same test exactly (right first / top five: small model 11/18, small with context 21/32, large 10/16, large with context 27/36).