top | item 11843655

Creating an embeddable Python distribution on OS X

38 points| jventura | 9 years ago |joaoventura.net | reply

14 comments

order
[+] douglasheriot|9 years ago|reply
Picking files out of homebrew for distribution is generally a bad idea. You’ve fixed the linking issues, but not -mmacosx-version-min.

If you run $ otool -l libpython3.5.dylib and look for LC_VERSION_MIN_MACOSX – you’ll see it’s compiled only for your current OS.

So, if you do this on OS X 10.11, your users will have to have OS X 10.11. It may appear to work on older versions of OS X, until you hit something that doesn’t. For example, when I tried using a homebrew library, it was compiled using newer SSE instructions that weren’t supported on older processors still supported by older OS X versions. So when testing on an old Mac, it crashed with bad instruction at a somewhat random point in execution.

[+] jventura|9 years ago|reply
Hum, nice catch! Maybe there is a way to change that LC_VERSION_MIN_MACOSX variable, although the compilation generated instructions may be a dead end.

I see two possible solutions, either try to work out from the Python provided binaries (which support OS X 10.6) or try to compile it from source in a way that is backwards compatible, Maybe cpython's makefile has some support for that?!..

[+] zZorgz|9 years ago|reply
Interesting read. I had embedded Python in one of my apps and I wish I had known about being able to zip the distribution. To save space, I compiled all the .py files into .pyc bytecode files, and tediously removed a bunch of modules my app didn't need (tkinter, antigravity, ...), along with setting PYTHONDONTWRITEBYTECODE. You can probably still do this, of course, in addition to zipping it all up.

I had compiled python myself and used static linking instead of dealing with install_name_tool, although I do wonder if I should have built a dynamic library instead. Anyway, compiling from source (or adjusting the homebrew formula?) gives you control and being mindful of eg: LC_VERSION_MIN_MACOSX as already mentioned here.

For my specific use case, I also wanted to use Apple's GCD and have Python do all of its work on a secondary thread. I had to make sure the interpreter would only stay on a single thread (whereas GCD normally doesn't make such a guarantee).

[+] srean|9 years ago|reply
It is extremely likely that that the author did not have option to look at alternatives to Python, but some other scripting languages, for example, Lua, Tcl and Guile are easier to embed because they were designed with that use in mind from the beginning.

Among other things, Tcl, Lua (not sure of latest Guile) encapsulate their interpreter thread well, so one can run independent interpreters in multiple threads that can communicate with each other by message passing but without the need for any serialization/deserialization.

[+] rpcope1|9 years ago|reply
If nothing else, I would imagine using Lua or Tcl instead of Python would make the binary size significantly smaller yeah? I seem to have gotten the impression that embedding the CPython runtime, even stripped down, could potentially grow binaries significantly, especially if statically linked.
[+] zZorgz|9 years ago|reply
I prefer to embed a language that I enjoy programming in. I don't know about Tcl, but I enjoy using Python more than Lua. Python isn't that unusual of a choice for embedding either; eg: take a look at LLDB or Sublime Text.

Threading and reference counting may be some ugly aspects of embedding CPython sure, although I don't think manipulating the stack in Lua is that praiseworthy either.