Postgres extension complements pgvector for performance and scale

[ricw]

I’ve been using this since early this year and it’s been great. It was what convinced me to just stick to Postgres rather than using a dedicated vector db.

Only working with 100m or so vectors, but for that it does the job.

[pqdbr]

Are you using a dedicated pg instance for vector or you keep all your data in a single pg instance (vector and non-vector)?

[esafak]

What kind of performance do you observe with what setup?

[aunty_helen]

Related discussion for pgvector perf: https://news.ycombinator.com/item?id=45798479

[tacoooooooo]

the main issue with pgvectorscale is that it's not available in RDS :(

[whakim]

Worth noting that the filtering implementation is quite restrictive if you want to avoid post-filtering: filters must be expressible as discrete smallints (ruling out continuous variables like timestamps or high cardinality filters like ids); filters must always be denormalized onto the table you're indexing (no filtering on attributes of parent documents, for example); and filters must be declared at index creation time (lots of time spent on expensive index builds if you want to add filters). Personally I would consider these caveats pretty big deal-breakers if the intent is scale and you do a lot of filtering.

[jascha_eng]

Combined with our other search extension for full text search these two extensions make postgres a really capable hybrid search engine: https://github.com/timescale/pg_textsearch

[ldng]

I'm not how you'd combine the two; care to give us a quick outline ?

[isoprophlex]

The linked blogpost is an interesting read, too, comparing well-tuned pgvector to pinecone:

https://www.tigerdata.com/blog/pgvector-vs-pinecone

[dmarwicke]

does this actually fix metadata filtering during vector search? that's the thing that kills performance in pgvector. weaviate had the same problem, ended up using qdrant instead

[mmmeff]

This is still unsupported in RDS, right?

[jascha_eng]

We have a lot of happy customers that moved from rds to tiger cloud if you think pgvectorscale is interesting to you and you don't want to self host pg.

But yes big cloud providers move slow in adopting extensions.

[tacoooooooo]

correct afaik :(

https://github.com/timescale/pgvectorscale/issues/113