Dynamically patch a Python function's source code at runtime

[breuleux]

Yeah, I've been using this trick to implement a hot reload library, to only update the specific functions that are changed without re-executing modules: https://github.com/breuleux/jurigged

I also use it in a multiple dispatch library (https://github.com/breuleux/ovld) to replace the entry point by specialized dispatch code in order to cut some overhead.

It's fun.

Also, why is every damn post these days somehow framed in an AI context? It's exhausting.

[diggan]

> Also, why is every damn post these days somehow framed in an AI context? It's exhausting.

Every 5/10-year segment of my life has somehow had one or two "This is the future"-hypes running concurrently with my life. Previously it was X, now it's Y. And most of the times, everything else is somehow connected to this currently hyped subject, no matter if it's related or not.

The only thing I've found to be helpful is thinking about and changing my perspective and framing about it. I read some article like this which is just tangentially related to AI, but the meat is about something else. So mentally I just ignore the other parts, and frame it in some other way in my head.

Suddenly people can write their articles with attachments to the hyped subject but I don't mind, I'm reading it for other purposes and get other takeaways that are still helpful. A tiny Jedi mind-trick for avoiding that exhaustion :)

[sczi]

Oh that is really interesting, I was just aware of IPython's autoreload extension, I hadn't found your library. I'm also working on hot reload for python as I'm working on a development environment for python that aims to give it a development experience closer to lisp: https://codeberg.org/sczi/swanky-python/

Some minor details. You currently aren't updating functions if their freevars have changed, you can actually do that by using c-api to update __closure__ which is a readonly attribute from python:

  ctypes.pythonapi.PyFunction_SetClosure.argtypes = [ctypes.py_object, ctypes.py_object]
  ctypes.pythonapi.PyFunction_SetClosure(old, new.__closure__)

Also I think you should update __annotations__, __type_params__, __doc__, and __dict__ attributes for the function.

Rather than using gc.get_referrers I just maintain a set for each function containing all the old versions (using weakref so they go away if that old version isn't still referenced by anything). Then when a function updates I don't need to find all references, all references will be to some old version of the function so I just update that set of old functions, and all references will be using the new code. I took this from IPython autoreload. I think it is both more efficient than gc.get_referrers, and more complete as it solves the issue of references "decorated or stashed in some data structure that Jurigged does not understand". The code for that is here: https://codeberg.org/sczi/swanky-python/src/commit/365702a6c...

hot reload for python is quite tricky to fully get right, I'm still missing plenty parts that I know about and plan on implementing, and surely plenty more that I don't even know. If you or anyone else that's worked on hot reload in python wants to talk about it, I'm happy to, just reach out, my email is visible on codeberg if you're signed in.

[modeless]

Jurigged is awesome. It works really well and saves me tons of time. Thank you for making it!

I do wish there were callbacks I could subscribe to that would notify me whenever my file changed or whenever any code changed, so I could re-run some init.

My other feature request would be a way to replace function implementations even when they are currently on the stack, as some other hot reload implementations can. But I certainly understand why this would be difficult.

[gjvc]

I use jurigged in conjunction with cmd2 to make command line driven tools. The combination works well because I can issue a command, watch the output, make a change, hit up-return and see the change just like that.

Thank you a bazillion for making it. It works quietly in the background without fuss, and I'm grateful for it every time I use it.

[Esophagus4]

> why is every damn post these days somehow framed in an AI context? It's exhausting.

It’s even in the real world now - most of my conversations with people in tech end up at AI eventually.

It kind of reminds me of the 2010s when non-tech people would ask me about crypto at social events.

[frou_dh]

In some respects that's even nicer to use than a typical editor-integrated live-coding REPL, because one doesn't have to think about what code needs (re)sending from the source to the REPL. Just save the file and it'll be figured out which parts meaningfully changed.

[johnfn]

Jurigged is really cool - thanks for the tool!

[zipy124]

jurigged is great, love using it for quick GUI prototyping with imgui!

[aeonik]

And wouldn't it be nice if that Python code, instead of a string, was just more python? Then you could use your existing Python code to append, or transform sections of your code!

That's what Lisp is!

Once you see how cool that is, then you can begin to appreciate why Lisp was the defacto standard for AI programing all the way back in the 1960s!

[almostgotcaught]

https://docs.python.org/3/library/ast.html

[zzzeek]

I really couldn't follow the use case. it looked like, they have a chain of method calls with some kind of `mark_circle().encode().properties()`, OK, so, if you want to make those methods do something different, you reach into OOP wisdom from the 1980s and write an appropriate "impl" used by whatever `alt.Chart()` is.

Someone explain to me, an old, aging programmer old enough to know UML, why this isn't some we presume very young person who has no idea how to write OOP coming up with some horrible convoluted way to do something routine?

[porridgeraisin]

> aging ... UML

Oh don't worry... they still cram that down our throats in CS undergrad in one of the courses.... Forget which one. I did my UG from 2020-24

;)

[ath3nd]

There is a reason similar approaches are called 'monkey patching'.

Just cause you can do something doesn't mean you should. I send thoughts and prayers for the people debugging programs where this is in place.

[miohtama]

Also check the history of monkey patching

https://en.m.wikipedia.org/wiki/Monkey_patch

[tovej]

Ah, self-modifying code, the more things change the more they stay the same.

Wasn't SMC one of the LISP-associated AI fields a few decades ago? iirc it's been mostly abandoned due to security issues, but some of it survives in dynamic compilation.

[thrown-0825]

this will be the next big breakthrough for agents lmao

[rob_c]

Aka I just learned how to modify python code whilst it's running.

Next step, here's how to load modules, resolve a dependency. Handle capabilities and dynamically inject more functionality 'live'.

Patching running machine code in memory for compiled objects is the same but you just need to work around the abstraction that is introduced by languages trying to make the whole stack human parseable.

[zekrioca]

Other than Lisp, is it possible to do this in other languages?

[kbolino]

Any language running on the JVM (Java, Kotlin, etc.) and CLR (C#, VB.NET, etc.). As long as you don't compile directly to native code and you aren't running in a locked-down environment where codegen is disabled, you can generate code at runtime and execute it alongside existing code.

For example, there's java.lang.reflect.Proxy [1] and System.Reflection.Emit [2].

[1]: https://docs.oracle.com/javase/8/docs/technotes/guides/refle...

[2]: https://learn.microsoft.com/en-us/dotnet/api/system.reflecti...

[OskarS]

Several! You're correct that Lisps is the most famous, and there's also languages like Erlang that have this as a core functionality. But it's also used in things like game engines for C/C++. You do have your "updateAndRenderFrame()" function in a dynamic library, having it take a pointer to the full game state as an argument. When you want to reload, you recompile your dynamic library, and the main loop can swap out the implementation, and the game keeps running with the new code. I don't see a reason why you couldn't do this in Rust, though I imagine it's trickier.

[woadwarrior01]

Greenspun's tenth rule, restated for Python instead of C/Fortran.

https://en.wikipedia.org/wiki/Greenspun%27s_tenth_rule

[mkoubaa]

Monkey patching python modules is waaaaay more straightforward than this.

setattr(mod, name, new_func)

[lavishlatern]

I'm not sure what the point of this blogpost was. As far as I can tell, the author discovered eval() but is making it more complicated for no reason? There also isn't any actual patching going on.

[duttish]

I used live patching of the function byte code to enforce type safety in python as an experiment. It was quite fun, took about a weekend or so :) not something for production though, due to the performance hit.

[tpoacher]

Neat trick indeed, but I have not yet found a compelling use-case for this that can't be done more sensibly using lambdas.

I once inherited a nice library which made heavy use of such compile/exec pairs. The code looked very convoluted and was very bug prone. I replaced all such pairs with lambda wrappers. This was much more readable: my colleagues who were also working on this code were like "oh is that what that was meant to be doing" after this change.

[pontifier]

I did this by accident in bash scripts many years ago when I was just getting into linux. I'd be running a script, and editing the script at the same time. It caused some REALLY weird issues before I figured out what was happening. For instance I'd change the text somewhere and it would change in the running program, or the program would get into states it should never be in. I didn't use it constructively, I just avoided editing running programs after that.

[ianbicking]

The use case is not that obvious to me, but I _think_ it's trying to give the LLM more context to write code by dipping into these code objects. Perhaps telling the LLM what modules are already imported or otherwise available?

Creating a debugger for an LLM (not a human) is something I haven't really seen, but seems super useful...?

[j-h-k]

One of the problem with dynamically loaded code like this is when you raise exceptions in those functions - the traceback will then end with something like `File "<magic>", line 8, in something`, which will at least annoy you when debugging.

[dfee]

Different, but you may find interest in changing function signatures, too: https://github.com/dfee/forge

[m3047]

I've contemplated patching Python's asyncio package so that the list of funcs isn't a weakref... but it's not that hard! This is way harder than it needs to be.

[edem]

wait a sec ... i thought you could monkey patch python code. you can only do it by using this technique?

[eidorb]

Another use case is applying small patches to a large function or method. Overriding would mean copy pasting largely similar code. It’s a bit ugly.

https://github.com/eidorb/ubank/blob/master/soft_webauthn_pa...