The idea that you can easily type a command and be quickly dumped into a development environment for some software your system runs is amazing. I think it’s something that actually brings people closer towards some of the ideals of GNU or Open Source and better lets you take advantage of everything being open source. (Aside: I think this is also great about Emacs; you can easily jump to the source of most things and there’s usually a way to hack or modify or inspect or debug them immediately).
One thing I couldn’t work out how to do in guix was to go the whole way through. Something like:
1. Install / use some program
2. Find some bug or potential improvement
3. Run the command to be dumped into a dev environment for that
4. Fix or investigate the bug; or implement the improvement
5. (The step I don’t know how to do) Use your fixed/improved version in your operating system
6. (Also don’t know how to leverage guix for this) Share your improvements with others
It feels like 5 is somewhat at odds with how guix is meant to work as step 4 is so mutation-heavy, but it also feels like something that an OS which wants to be as gnu as possible should really want to support. And gui should be able to make it safer too by giving you rollbacks. Maybe there’s an easy way to do it and I just don’t know it?
(note: usually packages are installed as regular user into your user profile)
See also section "Package Transformation Options" in "info guix".
For step 6, send a mail with the patch to the package definition (in an existing scm file in https://git.savannah.gnu.org/git/guix.git ) to the guix-devel mailing list. But whatever you did manually to get yourfixedversion_src.tar.gz you now should automate by programming the patching process in Scheme instead--editing a guix checkout (usually just to add: (add-after 'unpack 'patch-problematic-stuff-4711 (lambda _ (substitute* "somefile" (("regex1") "replacement"))))). For more complicated fixes, add a patch to the guix checkout and make Guix use it in Scheme (and contribute it to the actual foo upstream project--which we usually do anyway).
See section "Contributing" in "info guix".
Further automation welcome-- and probably not that difficult to add. Just keep in mind that each package build is in its own container (it's kinda like Docker would be)--so no mixing text editor and building and weird user-defined pauses (for example there is no text editor in the build container).
> The idea that you can easily type a command and be quickly dumped into a development environment for some software your system runs is amazing. I think it’s something that actually brings people closer towards some of the ideals of GNU
This is exactly right, and I think it's one of the coolest things about Guix. Guix System (the full OS based on Guix) seems almost like the GNU that was dreamt of before Linux. It even has an option built on the Hurd kernel, and it uses an init system and initrd images based on Guile Scheme, which is both the language used for Guix itself and as the extension language for GNU projects.
Guix definitely advances the whole
> Learn one simple, high-level language and you can hack on every aspect of your system
idea further than we've seen on any modern/contemporary system.
> Aside: I think this is also great about Emacs; you can easily jump to the source of most things and there’s usually a way to hack or modify or inspect or debug them immediately
I think Emacs is definitely an inspiration, or a model of the virtues intended.
The thing that really bums me out about Guix is that Emacs debugging tools for Guile aren't anywhere near as good as you get with Edebug for Emacs Lisp or similar functionality for Common Lisp, Racket, or Clojure. With any of those languages you set breakpoints, display results, evaluate expressions, etc all with a very nice interface.
With guile, you get none of that, there's seemingly no debugging tooling at all. The best Emacs integration you get is Geiser which is not a GNU project and doesn't offer the debugging functionality mentioned above.
This is very surprising considering that Emacs and Guix are under one roof, and Guix is arguably GNU's flagship new software. Why do the development tools suck?
Guile is severely lacking in manpower, people on the mailing list have talked about possibly improving the situation; but it's difficult to get any major movement going.
I use nix but, as a user of Common Lisp, I’d much rather use guix: the main blocker for me is that nix supports macOS and guix refuses to for ideological reasons.
No, Guix does not refuse to support macos and not for ideological reasons either. Where do you get your info?
There is no way to build the package graph on macos. There is no glibc port for macos, and there's no free toolchain we could legally use.
There is little value in building Guix when we are forced to use a different proprietary toolchain, a different C library, and throw away the from-source bootstrap in exchange for Gigabytes of proprietary blobs. You might as well use Guix System in a virtual machine on macos.
> This makes me wonder if Guix is ready for the real world.
ML and GPUs are a smallish subset of the real world, and a subset that's particularly plagued by proprietary software (especially the GPU side); I don't think it's a good litmus test for "is guix useful for real work".
I think Guix is more than ready for providing development environments, even for proprietary apps on open source technology stacks (but Nix will probably be easier to pitch). As an operating system much less so.
ld: /tmp/guix-build-python-pytorch-1.12.0.drv-0/source/build/lib/libtorch_cpu.so: undefined reference to `glslang::TShader::setNanMinMaxClamp(bool)'
ld: /tmp/guix-build-python-pytorch-1.12.0.drv-0/source/build/lib/libtorch_cpu.so: undefined reference to `spvtools::CreateCompactIdsPass()'
[...]
The reason is because pytorch 1.12.0 depends on shaderc-2020.4/lib/libshaderc_combined.a--and static libraries do not have automatic dependency resolution, so spirv, glsl etc is never linked.
If Guix uses Nix under the hood, is it "just" Nix with an S-expression syntax (admittedly much better than Nix syntax) and fewer packages? If so, is there a really compelling reason to prefer Guix over NixOS? The only substantive difference I'm aware of is the use of GNU Shepherd instead of systemd.
Probably because I’m not a lisp person but the syntax is just ugly and not intuitive. It would look much better as a yaml file and I don’t even really like yaml either.
I felt the same but after reading this[1] and following along with the REPL it finally started to click.
There are actually much less rules than "normal" programming languages. I had started to write my own basic functions that would be in a "standard library" (trim(), trunc(), remove() etc) and got the hang of it after a couple hours.
The only thing I don't like about the language for "serious" work on massive projects is missing static types (may make it less readable though..), and the tooling (most people lean on Emacs) could be better.
So I'm guessing there's some additional conventions for function naming, besides appending ? to the name for boolean return and ! for functions that mutate.
How familiar are you with Lisps? I heard somewhere that it's easier for a complete beginner to start learning programming with lisps than with traditional languages. I started with more mainstream languages, but my experience tends to agree with that observation. The small set of various lisp forms and their semantics are easy to learn and reason about, compared to synatx and semantics of more traditional languages. Same when s-expressions are used for representing data. The syntax closely matches the logical structure of a program or data.
Keeping this in mind, I find the Guile scheme syntax used by Guix quite intuitive and pleasant. It reduces the cognitive overhead to configure the system, at least for me.
I am not a lisp person, but it is just a configuration file with key value pairs and lists, mostly copy pasted from the documentation or other ppl configurations. A file with all configs in the same place in an unified way
Also each configuration file in Linux has already has their differences and quirks
I don't understand how the Guix documentation can refer to #:uninterned #:symbols as "keywords"; was that written with a straight face? Yikes. Common Lisp's single : sigil is perfect for keywords; two characters is excessive.
Why does gnu-build-system use %standard-phases (a symbol with %)? You'd think it could just use standard-phases. What is the big namespace concern. The % prefix is a Lisp convention for implementation internal symbols, not to be used by user code. Kind of like double underscore in C.
It could simply be the mplicit default. If most rules use %standard-phases, it's silly to be repeating it all over the place.
modify-phases could have a syntax like:
(modify-phases {phases-object | clause }*)
If an argument is a phases-object, then it's taken as the current object on which the remaining clauses operate. Otherwise it's a clause like (add-after ...). The initial current object is understood to be %standard-phases. Now you just do:
either add-after performs the transformation to lambda + invoke, or else the object stays as a list, and is later treated as a command to execute.
A macro like (cmdf "sh bootstrap") could do the parsing and generate the lambda + invoke. The "f" reminds you that it doesn't run the command but returns a function.
It's written in Guile, which is a variant of Scheme. #: is used as the keyword indicator, not an uninterned symbol like in CL.
> Why does gnu-build-system use %standard-phases (a symbol with %)
AFAIK %symbol indicates a constant in Scheme.
It's unfortunate that it isn't written in Common Lisp. CLOS would've been immensely useful there instead of the ad-hoc object system via Scheme records.
AFAIU, asdf is for providing particular versions of tools (e.g. python, etc.). In the original post, guix is also used to provide library dependencies.
Guix is the free software oriented, lisp flavored cousin of Nix. -- I've used Nix. It's got enough rough edges that writing nix code is pretty much only accessible to enthusiasts.. though, those who can get past the rough edges do recommend it.
The added complexity of guix/nix allow for having an OS distribution where the whole OS config is declared/reproducible.
[+] [-] dan-robertson|3 years ago|reply
One thing I couldn’t work out how to do in guix was to go the whole way through. Something like:
1. Install / use some program
2. Find some bug or potential improvement
3. Run the command to be dumped into a dev environment for that
4. Fix or investigate the bug; or implement the improvement
5. (The step I don’t know how to do) Use your fixed/improved version in your operating system
6. (Also don’t know how to leverage guix for this) Share your improvements with others
It feels like 5 is somewhat at odds with how guix is meant to work as step 4 is so mutation-heavy, but it also feels like something that an OS which wants to be as gnu as possible should really want to support. And gui should be able to make it safer too by giving you rollbacks. Maybe there’s an easy way to do it and I just don’t know it?
[+] [-] dannymi|3 years ago|reply
guix install foo --with-source=foo=yourfixedversion_src.tar.gz
Or:
guix install foo --with-patch=foo=yourpatch.patch
(note: usually packages are installed as regular user into your user profile)
See also section "Package Transformation Options" in "info guix".
For step 6, send a mail with the patch to the package definition (in an existing scm file in https://git.savannah.gnu.org/git/guix.git ) to the guix-devel mailing list. But whatever you did manually to get yourfixedversion_src.tar.gz you now should automate by programming the patching process in Scheme instead--editing a guix checkout (usually just to add: (add-after 'unpack 'patch-problematic-stuff-4711 (lambda _ (substitute* "somefile" (("regex1") "replacement"))))). For more complicated fixes, add a patch to the guix checkout and make Guix use it in Scheme (and contribute it to the actual foo upstream project--which we usually do anyway).
See section "Contributing" in "info guix".
Further automation welcome-- and probably not that difficult to add. Just keep in mind that each package build is in its own container (it's kinda like Docker would be)--so no mixing text editor and building and weird user-defined pauses (for example there is no text editor in the build container).
[+] [-] pxc|3 years ago|reply
This is exactly right, and I think it's one of the coolest things about Guix. Guix System (the full OS based on Guix) seems almost like the GNU that was dreamt of before Linux. It even has an option built on the Hurd kernel, and it uses an init system and initrd images based on Guile Scheme, which is both the language used for Guix itself and as the extension language for GNU projects.
Guix definitely advances the whole
> Learn one simple, high-level language and you can hack on every aspect of your system
idea further than we've seen on any modern/contemporary system.
> Aside: I think this is also great about Emacs; you can easily jump to the source of most things and there’s usually a way to hack or modify or inspect or debug them immediately
I think Emacs is definitely an inspiration, or a model of the virtues intended.
[+] [-] BaculumMeumEst|3 years ago|reply
With guile, you get none of that, there's seemingly no debugging tooling at all. The best Emacs integration you get is Geiser which is not a GNU project and doesn't offer the debugging functionality mentioned above.
This is very surprising considering that Emacs and Guix are under one roof, and Guix is arguably GNU's flagship new software. Why do the development tools suck?
[+] [-] tmtvl|3 years ago|reply
[+] [-] guilhas|3 years ago|reply
The documentation is good
It still has problems here and there, and packages missing. Probably due to the lack of wide adoption for being new and different
But packages that are missing, most are easy to create or find online, to use in your private repository, or submit to main repository
There was less difficulties than expected. Things that didn't work, or I didn't wanted to waste time fixing
[+] [-] fiddlerwoaroof|3 years ago|reply
[+] [-] rekado|3 years ago|reply
There is no way to build the package graph on macos. There is no glibc port for macos, and there's no free toolchain we could legally use.
There is little value in building Guix when we are forced to use a different proprietary toolchain, a different C library, and throw away the from-source bootstrap in exchange for Gigabytes of proprietary blobs. You might as well use Guix System in a virtual machine on macos.
[+] [-] amelius|3 years ago|reply
https://guix.gnu.org/en/packages/python-pytorch-1.12.0/
This makes me wonder if Guix is ready for the real world.
[+] [-] yjftsjthsd-h|3 years ago|reply
ML and GPUs are a smallish subset of the real world, and a subset that's particularly plagued by proprietary software (especially the GPU side); I don't think it's a good litmus test for "is guix useful for real work".
[+] [-] grnmamba|3 years ago|reply
I think Guix is more than ready for providing development environments, even for proprietary apps on open source technology stacks (but Nix will probably be easier to pitch). As an operating system much less so.
[+] [-] pxc|3 years ago|reply
[+] [-] dannymi|3 years ago|reply
If you do know the build options to make this work, Guix will gladly consider your patch.
The in-progress diff would be,
Error messages: : && /gnu/store/069aq2v993kpc41yabp5b6vm4wb9jkhg-gcc-10.3.0/bin/c++ -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DUSE_QNNPACK -DUSE_PYTORCH_QN NPACK -DUSE_XNNPACK -DUSE_VULKAN -DUSE_VULKAN_API -DUSE_VULKAN_SHADERC_RUNTIME -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing -field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno -error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wn o-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow -DHAVE_AVX512_CPU_DEFINITION -DHAVE_AVX2_CPU_DEFINITION -O3 -DNDEBUG -DNDEBUG -rdynamic - pthread caffe2/CMakeFiles/scalar_test.dir/__/aten/src/ATen/test/scalar_test.cpp.o -o bin/scalar_test -Wl,-rpath,/tmp/guix-build-python-pytorch-1.12.0.drv-0/source/build/lib: -lgtest_main -lgtest /gnu/store/0 94bbaq6glba86h1d4cj16xhdi6fk2jl-gcc-10.3.0-lib/lib/libgomp.so /gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib/libpthread.so -Wl,--no-as-needed,"/tmp/guix-build-python-pytorch-1.12.0.drv-0/source/bui ld/lib/libtorch.so" -Wl,--as-needed -Wl,--no-as-needed,"/tmp/guix-build-python-pytorch-1.12.0.drv-0/source/build/lib/libtorch_cpu.so" -Wl,--as-needed /gnu/store/9pyydl5w9xnz1qm56sxn1zh4qny6fkxz-protobuf-3.17.3 /lib/libprotobuf.so lib/libc10.so -pthread && : The reason is because pytorch 1.12.0 depends on shaderc-2020.4/lib/libshaderc_combined.a--and static libraries do not have automatic dependency resolution, so spirv, glsl etc is never linked.[+] [-] zetalyrae|3 years ago|reply
[+] [-] unknown|3 years ago|reply
[deleted]
[+] [-] ed25519FUUU|3 years ago|reply
[+] [-] llanowarelves|3 years ago|reply
There are actually much less rules than "normal" programming languages. I had started to write my own basic functions that would be in a "standard library" (trim(), trunc(), remove() etc) and got the hang of it after a couple hours.
The only thing I don't like about the language for "serious" work on massive projects is missing static types (may make it less readable though..), and the tooling (most people lean on Emacs) could be better.
So I'm guessing there's some additional conventions for function naming, besides appending ? to the name for boolean return and ! for functions that mutate.
[1]https://spritely.institute/static/papers/scheme-primer.html
[+] [-] 0x457|3 years ago|reply
It can't be a yaml file because it's essentially a program, it's a lot easier to write such things as a program.
[+] [-] sreevisakh|3 years ago|reply
Keeping this in mind, I find the Guile scheme syntax used by Guix quite intuitive and pleasant. It reduces the cognitive overhead to configure the system, at least for me.
[+] [-] guilhas|3 years ago|reply
Also each configuration file in Linux has already has their differences and quirks
[+] [-] db48x|3 years ago|reply
[+] [-] eointierney|3 years ago|reply
Tongue in cheek :)
[+] [-] kazinator|3 years ago|reply
Why does gnu-build-system use %standard-phases (a symbol with %)? You'd think it could just use standard-phases. What is the big namespace concern. The % prefix is a Lisp convention for implementation internal symbols, not to be used by user code. Kind of like double underscore in C.
It could simply be the mplicit default. If most rules use %standard-phases, it's silly to be repeating it all over the place.
modify-phases could have a syntax like:
If an argument is a phases-object, then it's taken as the current object on which the remaining clauses operate. Otherwise it's a clause like (add-after ...). The initial current object is understood to be %standard-phases. Now you just do: One piece of noise gone; N more to go.About the lambda, a simple idea would be that if the registered function is a list object (pair), then it's treated as list of exec arguments:
either add-after performs the transformation to lambda + invoke, or else the object stays as a list, and is later treated as a command to execute.A macro like (cmdf "sh bootstrap") could do the parsing and generate the lambda + invoke. The "f" reminds you that it doesn't run the command but returns a function.
[+] [-] yyyk2|3 years ago|reply
> Why does gnu-build-system use %standard-phases (a symbol with %)
AFAIK %symbol indicates a constant in Scheme.
It's unfortunate that it isn't written in Common Lisp. CLOS would've been immensely useful there instead of the ad-hoc object system via Scheme records.
[+] [-] pvinis|3 years ago|reply
[+] [-] rgoulter|3 years ago|reply
AFAIU, asdf is for providing particular versions of tools (e.g. python, etc.). In the original post, guix is also used to provide library dependencies.
Guix is the free software oriented, lisp flavored cousin of Nix. -- I've used Nix. It's got enough rough edges that writing nix code is pretty much only accessible to enthusiasts.. though, those who can get past the rough edges do recommend it.
The added complexity of guix/nix allow for having an OS distribution where the whole OS config is declared/reproducible.
[+] [-] throwway9028232|3 years ago|reply