[edublancas]

Papermill is great but has quite some limitations because it spins up a new process to run the notebook:

- You cannot extract live variables (needed for testing)

- Cannot use pdb for debugging

- Cannot profile memory usage

You can do all of that with ploomber-engine (https://github.com/ploomber/ploomber-engine).

Disclaimer: I'm the author of this package

[ziddoap]

Not disclosed in this comment is that edublancas is

>Ploomber (YC W22) co-founder.

[Kalanos]

who is a great technologist with a lot of hands on experience. if it made sense to leverage papermill, he would have done so and focused on something else.

[throwpoaster]

iirc, a few years back I was able to do all of these things with the Papermill IPython runtime.

Papermill is great, but yes: lots of room to hack on it and make it better.

[edublancas]

has papermill deprecated the ipython runtime? I used papermill extensively in the past and I never saw that in their docs.