At Clever Cloud, we love giving new technologies a try. Since its inception, rust has always been quite appealing and we always wanted to find projects for which rust would be an adequate solution.
Now that rust is mature enough (to our taste), we started giving it a go, and we needed a way to easily package rust software for our distribution of choice: Exherbo.
Rust package manager: cargo
Amongst the tooling shipped with rust is cargo. Cargo allows you to simply build, test and install a package and its dependencies. It's actually very nice to use when developing.
If we were to install software with cargo, there are two solutions:
- Either we want to install something like a private or local projects and we just
have to go to the sources directory, where theCargo.tomlfile is, and run
cargo install - Or we want to install something publicly available, on crates.io
for example, we can just runcargo install crate_name,crate_namebeing the actual
crate (cargo package) name. We could also pass it a git URL as an alternative.
Regarding production environment and system-wide utilities, we want everything to be managed by the distribution package manager, paludis, so we had to hook up those two and try to make them work together as well as possible.
How packaging rust is different from packaging other software
Currently, rust doesn't support installing libraries system-wide as they're not confident enough in the stability of their ABI yet. What that concretely means is that it's not yet possible to install any rust library, and then use it to build dependent stuff. You have to rebuild all the dependencies each time you build a rust binary.
That particular point is the most noticeable difference on the packaging point of view. Instead of building and installing the dependencies, then the dependents, you only install the final product with everything built in.
What a "traditional" install process looks like in a source-based distribution
For traditional packages, the build and install process is composed of several sequential steps, that we can summarize like this, in that order:
Fetch
This is the step where we download everything, usually the software sources.
This is the only step for which network sandboxing is disabled.
Unpack
Here we unpack the sources in a sandboxed environment. Network sandboxing is on as well as file system sandboxing for this step and all the steps onwards.
Prepare
This is an optional phase, not needed by all packages. It's the moment when we generate build-system files if needed and apply patches to the software sources.
Configure
We decide there which features we want to enable or not for the build.
Build
Where the actual build happens.
Install
We install the resulting binaries/files in a sandboxed location replicating the / hierarchy.
Merge
Once everything gets validated in terms of file system access, everything gets installed to the root file system.
What a rust install process looks like
A rust install pretty much ressembles a traditional one, with one huge exception.
As I said before, cargo install supports either reading a local Cargo.toml, or being passed a crate name or a git URL.
On the other hand, cargo fetch, which is the command that downloads all the dependencies doesn't have these options, it only supports reading a local Cargo.toml and fetching the dependencies listed in it. If libraries were installable system-wide, we wouldn't need this step at all, but since they're not, we have to do this step after unpacking the sources, to be able to access the Cargo.toml.
This has serious implications, it means that we extend the unpack phase with three additional actions:
- disable the network sandboxing, which is really sad and we should never do that
cargo fecth- re-enable the network sandboxing
Of course, we don't have to download the dependencies at each reinstall, cargo supports a CARGO_HOME environment variable allowing us to store the downloads in a shared location with all other distfiles.
We'll eventually be able to install libraries system-wide, but I don't see that happening any time soon. The better workaround I could come up with to clean up this hack is to make cargo fetch support being given a crate name or a URL, matching its companion cargo install. Here is the corresponding github issue but I'm not sure whether it will be implemented nor when.
Actual implementation and examples
The implementation of my exlib (which is to exherbo packages what libraries are to software) can be found here.
cargo-watch is a good example of what a rust exherbo package using this exlib looks like. You can see it's fairly trivial and everything is automatic.