top | item 31755242

(no title)

andrewbarba | 3 years ago

Are there plans to release an HTTP API to make it easier to use with services like Fastly Compute@Edge and Cloudflare Workers? And if so would the API be global or region specific?

One thing I haven't seen with "serverless" databases is an easy way to dictate where data is stored. Mongo has a pretty clever mechanism in their global clusters to specify which regions/nodes/tags a document is stored in, and then you simply specify you want to connect to nearest cluster. Assuming your compute is only dealing with documents in the same region as the incoming request, this ends up working really well where you have a single, multi-region db, but in practice reads/writes go to the nearest node if the data model is planned accordingly.

A real world example of how I am using this in Mongo today: I run automated trading software that is deployed to many AWS regions, in order to trade as close to the respective exchange as possible. I tag each order, trade, position, etc. with the exchange region that it belongs to and I get really fast reads and writes because those documents are going to the closest node in same AWS region. The big win here is this is a single cluster, so my admin dashboard can still easily just connect to one cluster and query across all of these regions without changing any application code. Of course these admin/analytics queries are slower but absolutely worth the trade off.

discuss

order

nikita|3 years ago

Absolutely! We are working on it right now and call this “regions”. We already have a proxy - you will notice that the connection string is project_name.cloud.neon.tech.

We are working on deploying the proxy globally and routing read traffic to the nearest region.

We also have some multi-master designs in collaboration with Dan Abadi. But this will take a second to build.

wharfjumper|3 years ago

So if we wanted to put PostgREST[1] in front of this would something like this work:

db-uri = "postgres://authenticator:mysecretpassword@project_name.cloud.neon.tech:5433/postgres" db-schema = "api" db-anon-role = "web_anon"

[1]https://postgrest.org/en/v8.0/tutorials/tut0.html

tpetry|3 years ago

I‘ve seen an clever approach to do location-specific data at rest. Partitioning and postgres_fdw was combined to store specific data on specific clusters in some regions. And on top of it partioning was used to again get the global view. Really nice.