(no title)
cchianel | 1 year ago
I am unfortunately very familiar with CPython's bytecode [1], and do not recommend trying to do anything with it:
- It is extremely unstable; the same opcodes might do different things between versions. For instance: for the opcode `JUMP_IF_TRUE_OR_POP`, in Python 3.10 and below, its argument is an absolute address. In Python 3.11 and above, it is a relative address.
- It does not have synthetic (compiler-introduced) variables; this means the iterator of a loop must be kept on the stack, and as a result, exception handling must keep some items (namely, the iterators) on the stack.
- It is poorly documented (although it is getting better nowadays); there where numerous times where the stack layout an opcode expected changed between versions but were not yet included in the dis documentation.
One notable difference between CPython and other bytecode interpreters is a lack of multiple compilers. For instance, in Java, there are typically three compilers:
- `javac`, which produce class files that are interpreted by a basic, slow virtual machine.
- C1, which generates instrumented machine code to collect profiling data that C2 will use (still relatively slow).
- C2, which generates optimized machine code from the profiling data of C1 (significantly faster than C1).
CPython shoves the job of all three into its single interpreter. For instance, its CACHE instruction serves the purpose of recording the types on the stack for the subsequent instruction, which the specializing adaptive interpreter [2] uses to specialize an opcode (for instance, BINARY_OP to BINARY_OP_INPLACE_ADD_UNICODE).
All of the above makes it extremely hard to develop against CPython's bytecode, and is probably why there are few libraries for reading or transforming CPython's bytecode. Although most modifications can be done by either modifying the class object directly, wrapping the class/function, or extending the class, some cannot. For example:
- Change calls of function `foo` to function `bar`.
- Compute the automatic differentiation of a function [3].
- Translate python code blocks to another language for interoperability (for instance, to Java or SQL).
The way most library authors do it instead is by reading the AST, which is mostly stable.
[1] https://github.com/TimefoldAI/timefold-solver/tree/main/pyth..., a Python bytecode to Java bytecode translator, written to avoid excessive overhead caused by FFI calls (the new foreign Java API largely removes the overhead).
No comments yet.