zombodb's comments

Huge thanks to TCDI (www.tcdi.com) for continuing to fund development.

If there's anything the pgrx team can do to help, just let us know.

Those are the internals we currently expose as unsafe “sys” bindings.

As we/contributors identify more that are desired we add them.

pgrx’ focus is on providing safe wrappers and general interfaces to the Postgres internals, which is the bulk of our work and is what will take many years.

As unsafe bindings go, we could just expose everything, and likely eventually will. There’s just some practical management concerns around doing that without a better namespace organization —- something we’ve been working.

The Postgres sources are not small. They are very complex, inconsistent in places, and often follow patterns that are specific to Postgres and not easy to generalize.

If you’ve never built an extension with pgrx, give it a shot one afternoon. It’s very exciting to see your own code running in your database.

We have an ambitious goal with pgrx and it’s going to take many years and countless hours of developer effort to get there.

It can, however, serve as a way for newcomers to gain experience with Postgres internals.

Regarding the TableAM specifically, when we are able to create a safe Rust wrapper around it, that wrapper will be documented.

ZDB is still alive and well but I have a real job now too.

ZomboDB only requires that ES have a view of its index that's consistent with the active Postgres transaction snapshot. ZDB handles this by ensuring that the ES index is fully refreshed after writes.

This doesn't necessarily make ZDB great for high-update loads, but that's not ZDB's target usage.

> They also claim that transactions will abort if ElasticSearch runs into network trouble...

I had to search my own repo to see where I make this claim. I don't. I do note that network failures between PG & ES will cause the active Postgres xact to abort. On top of that, any error that ES is capable of reporting back to the client will cause the PG xact to abort -- ensuring consistency between the two.

Because the ES index is properly refreshed as it relates to the active Postgres transaction, all of ES' aggregate search functions are capable of providing proper MVCC-correct results, using the parallelism provided by the ES cluster.

I don't have the time to detail everything that ZDB does to project Postgres xact snapshots on top of ES, but the above two points are the easy ones to solve.

!boom!

That said, it'd be easy to prototype with pgx, assuming that compression library exists in the Rust ecosystem too?

I see what you're saying about no_std tho. Could be doable.

pgx does a lot of that too, but for only pg10/11/12. And then it post-processes the bindings and builds a "common.rs" for symbols that are identical across versions.

That way you can target all three versions at once, and then you can use Rust's #[cfg(feature=)] directive to opt into support for specific versions.

pgx has also done a little bit of work to mask differences between the three versions so they appear as part of "common.rs" anyways.

I wanted a framework that let me know, at development time, if I was trying to do a thing that isn't supported (or the same) across various PG versions.

    test=# SELECT count(*) FROM (SELECT generate_series(1, 10000000)) s;
      count   
    ----------
     10000000
    (1 row)

    Time: 399.630 ms
    test=# SELECT count(*) FROM (SELECT srf.generate_series(1, 10000000)) s;
      count   
    ----------
     10000000
    (1 row)

    Time: 478.194 ms

edit: PS, thanks for the idea of putting what I assume is the backend pid in psql's prompt. I need to go figure out how to do that right now!

edit edit: hmm, I guess that's not the PID.