Sebastian's personal website

Adding AppImage support to the guix package manager

Written by Sebastian Dümcke on
Tags:

Whilst participating in the 2023 Lisp Game Jam I was very frustrated about the (non-existent) options to distribute my game written in Chez scheme. At that time, I simply shipped the source code and hoped the other participants knew how to run chez programs. There had to be a better way! I remembered that AppImage is a single file solution for distributing linux applications. However one needs to ensure that all dependencies which are not typically part of a linux system need to be included in the AppImage file. Now tracking dependencies by hand is cumbersome. Luckily, I already use a package manager that excels at this: guix. This post will detail on how to combine both.

Guix already allows bundling software package definitions into different formats: containers, rpm, deb files and many more. I quickly realised that it would be sufficient to extend this framework. Secondly, by carefully reading the AppImage documentation about manual packaging I realised, that an AppImage is nothing more than a runtime and a SquashFS file system combined. The runtime mounts the SquashFS via FUSE to a temporary directory and executes the entry point.

The work then consisted of understanding where to hack into the guix pack command to add a new package format, and taking inspiration from the other code, how it generated the closure of all package dependencies. Guix already has a function to generate SquashFS files (told you it is a great package manager), so that part was relatively straightforward. I wrote a concatenation function in scheme that would write the runtime file and SquashFS “payload” into a single file with AppImage extension. According to the AppImage specification, one needs to create an AppDir folder with an AppRun entry point (can be a symlink to an executable), a .desktop file, and a logo in PNG format. Again, guix was very helpful, as it already has a function to generate .desktop files. I will comment on the logo later.

The thing I struggled the most with was packaging the AppImage runtime. Indeed, I wrote a first proof-of-concept, where I used the binary provided by AppImage. This showed that the approach works. However, guix’ focus is on reproducibility, and all packages need to be built from source, no executables allowed! So I needed to build the runtime myself. And I failed. Weird build system, lack of knowledge on how to compile and link C code from side meant I hit an error every time I tried.

This was around March 2024. At that time, I had subscribed to the guix-devel mailing list in order to keep up with development and perhaps ask for help. A message came through, where an AppImage backend for guix pack was discussed as a Google Summer of Code project. I let the mailing list know, that I already had a prototype and was close to completion. I felt it would have been a duplication of efforts, as I had a working prototype already. A few month later, Noé revived the thread asking for my progress and if he could help. I told him where I was stuck and we started collaborating. He had the skills to compile the AppImage runtime from source and package it up for guix. We also brainstormed on some last issues and he helped with testing across different systems. He also spearheaded the patch submission, for which I was very grateful. Guix uses a mailing list based submission system, which I am unfamiliar with. I hoped to learn this during this submission, but as I had limited time, Noé’s experience helped enormously to get it submitted in a timely fashion. He also took care of most of the review process. Finally, end of November that patch was accepted and merged into guix mainline.

The review process went quite smooth and in a timely manner, which is not always the case. I attribute this to the fact that the feature was welcomed and the changes were regarding guix proper and not an addition or change in packages.

Apart from the runtime packaging, I have to say, that the development and particularly debugging experience when hacking on guix is rough. I found the errors difficult to interpret, particularly syntax errors where the error message did not always mention the correct line in source. I settled on an approach where I would insert a known error into the code, then build the AppImage with guix build -K that would keep the build directory. Inside it I could then inspect everything and try to estimate what the cause of errors could be. Sometimes I would also insert scheme code to write messages to a file in the build directory which I would then read out to understand in which code path I was. A lot of hackery.

However, the feature now works as intended and seems stable. There are a few things to notice:

  1. contrary to the recommendations of the AppImage documentation, our approach packages every single dependency. While usually AppImage files use e.g. the glibc of the host system, the files generated by guix pack are 100% portable. This comes at a price: the files are much larger. Even with the best compression a simple hello world generates an AppImage of double digit megabytes.
  2. We are not compliant with the specification relating to the logo. We do not include a logo and also no way of setting it. As it turns out, the requirement for a logo is not functional, meaning it is not required for the AppImage to run. Also it is unclear to me where this would be used. Since I run all AppImage files from the terminal, I never got to see the logo. I imagine that some desktop system integration would make use of this. We discussed this together with Noé, however could not find a good solution. What would be a default logo if the user does not specify one? If we make it mandatory, how would one pack packages that do not have a logo? Should the logo be a path inside the package or a file system path? We finally decided to deal with this when somebody raises an issue (hopefully with a good use case).
  3. You need to write a package definition for your software if you want to pack it into an AppImage. It is not hard, and can also be done without knowing scheme, as guix comes with extensive documentation.

Finally, I got to test the whole thing myself, during the Fall Lisp Game Jam 2024. I used the code to package my game as an AppImage. It weighs in at 172MB (for a 3.3MB source code). This time around, no reports of technical difficulties getting the game to run. Only reports on issues with game mechanics, which are fair game. From my perspective a huge improvement.

I am very pleased to have been able to contribute to guix. I also learned a lot about its internals (I read a lot of guix source code during the development). Last I want to thank Noé in particular and the whole guix community for their support.