I’m forwarding this from the main Chaotic-AUR news channel for general visibility!
(if you followed our development processes - yes, the garuda
repo has been running on this for a longer while. Adopting it to Chaotic-AURs needs took a little longer however
Today is a special day because it marks the 6th birthday of this project, as well as the day when the next big evolution of our build tools finally goes live.
While we already highlighted the most prominent changes between those three years in our previous post, it is a pleasure to be officially announcing our new build system, also called infra 4.0 This is the end of a journey that took more than half a year to finally be filling close to all Chaotic-AUR build requirements. While this is primarily a solution written for Chaotic-AUR, it is still a generic solution everyone wanting to compile PKGBUILDs on smaller or larger scale can adopt.
Why would we need a new build system if the old one was working well? The question is easy: the pre-assumptions the previous infra had, like inhomogeneous builder landscapes lead to some drawbacks (while serving a really good job for many years nevertheless!). One of them is that we continually need to run routines to check packages for new updates, another one being unable to control the allocation of builds to individual builders.
This time, we were able to work with different pre-assumptions, which allowed us to optimize the entire process to a great extent. The fact that we’ve been running the entire load of Chaotic-AUR in our infra 4.0 test environment on our main builder besides the regular builds is a great example of how efficient it has become.
Since the entire process consists of GitLab CI pipelines and a central instance for managing build processes after they have been scheduled by the pipelines, it is also much easier for users to figure out when packages were updated, as well as being able to inspect logs for multiple revisions of builds. This also means that our well-known packages repository is now mainly operated on GitLab CI due to making use of their CI tools. To preserve issues and comment histories, the “old” repository will be kept as a read-only mirror of the main repository, and also kept open for issues and bug reports.
There are many other changes and new features, like:
- Builds getting scheduled on demand (read: whenever an AUR package changes and CI pipelines pull in the updated files, or someone pushes a change to a locally maintained package) rather than with every routine run
- Live logs can be opened either via the pipeline’s external stages, as well as being presented on our Chaotic-AUR. If you always wanted to watch your Ungoogled Chromium build logs in real-time - now it is possible
- Build queues are visible to the public, also on the status page
- Support of build for multiple repos at once
- Saving/Restoring build queue state on builder/manager shutdown
- Dependency trees
- Automated lib bumps (which are not enabled for most packages, as adding deps is a manual process. The plan is to enable this for more packages in the following months)
- No more split repos for packages and interference - the repo now consists of folders, one for each pkgbase. Inside it, you may find a .CI folder with interferes and further builds configurations
- Being able to maintain a PKGBUILD on Chaotic-AURs repo, and pushing it back to AUR automatically, whenever a new change gets pushed. This opens a door for AUR packages to be community-maintained via PRs (looking at you, proton-ge-custom!)
- The state present in the packages repo fully reflects the state of the repository. Packages not present will automatically be removed
- Being able to review non-trivial package updates (which is opt-in and turned off for Chaotic-AUR. Might explore the idea of a reviewed/trusted subset of packages in the future)
For everyone wanting to take a closer look inside, dedicated documentation is available here.
At this point, I want to especially thank two people, who played a major role in making this possible.
- TNE, whose ideas and visions ensured all of it became as efficient as it turned out.
- xiota, who did close to all of the heavy lifting regarding porting our current set of packages to v4 and provided feedback on build processes and existing issues
And of course, a heartfelt thank you goes out to the rest of the team and our community.
The submissions for our new memorial have closed and the result is available here. Thanks to everyone who contributed to it
We hope you guys enjoy the new stuff announced today. On to many more years!