Building Packages on Arch Linux (Including the AUR)
On Arch Linux, the official repositories are: core, extra and community. These packages are already compiled, and they are installed through pacman. For the most part, general users can ignore that these 3 official repositories are separate. Core contains the most critical packages, such as the kernel, boot process, networking, package management, openssh and so on. It also has stricter requirements of more thorough testing before new versions are released. Extra contains other popular packages that aren't as critical, such as an X server, window managers or web browsers. Community contains less popular packages. Only Trusted Users (about 60 active users who were voted upon by other Trusted Users) have access to make changes to the official repositories.
In 2019, there are about 11,000 packages in the official repositories, at https://www.archlinux.org/packages. But, there are many other programs available on Linux. So, the AUR (Arch Linux User Repository) exists so any Arch user can add a new program and become its maintainer, or adopt an "orphaned" package without a current maintainer. There are about 55,000 packages in the AUR, at https://aur.archlinux.org/.
There are 3 critical differences with the AUR:
- Again, these packages can be produced by any user, even a brand new one.
- The AUR only houses a
PKGBUILD, a shell script to automatically make the package, not compiled binaries. (Sometimes it also contains small text patches, or install/upgrade/uninstall shell scripts). This has done a tremendous job allowing any user to contribute, while mitigating the chance of someone being able to distribute malicious code. The Arch community is still quite helpful regarding problems with AUR packages, but it is noted that the use of them is at your own risk. Because all it provides is aPKGBUILD, it's ultimately your responsibility to review aPKGBUILDyou are going to use. (Granted, many users don't do this and just rely on others to keep watch.) - Because
pacmandoesn't directly interact with the AUR, it is your responsibility to update AUR packages. When you periodically upgrade your entire system throughpacman, it will not automatically download updates to AURPKGBUILDfiles, compile them, and install them for you.
Although this article focuses on building packages from the AUR, the same techniques can be used to build packages from the official repositories yourself.
PKGBUILD
Compared to a .spec file that many other distributions use, a PKGBUILD is a short and simple shell script. Although some packages are more complex, they can simply be similar to the following:
This doc refers to:
PKGNAME: The name of a packagePKGVER: The version of a package (almost always matching upstream's version number)PKGREL: The Arch "version" of thePKGBUILDfor a specificPKGVER(normally 1, but incremented if changes need to be made to aPKGBUILDbetween upstream releases)ARCH: The architectures the package can be built upon (somewhat legacy, as the Arch Linux official repositories only support "x86_64" (64-bit CPUs), but AUR packages can still support "i686" (32-bit CPUs) or "any" to designate architecture is irrelevant)PKGBUILD/ETC: Any files actually in the AUR repository; thePKGBUILD, and any other small text patches, or install/upgrade/uninstall shell scripts. Does not include upstream files in thesourcearray.
Although the AUR has proven to be extremely trustworthy, it's a good idea to look at a PKGBUILD/ETC to make sure it's getting the source from a place you're willing to trust; (for example, an official upstream location, which can be from github - but not just some random person's github repository who is unrelated to the upstream package); and that the PKGBUILD/ETC doesn't contain any suspect code.
Obtaining PKGBUILD/ETC
From the AUR
If the official repositories don't contain a package you're looking to install, search for it at https://aur.archlinux.org/. Hopefully, you will find that what you are looking for exists, is up to date and maintained.
The best way to obtain the PKGBUILD/ETC from the AUR is to clone it via git.
Install git, if it is not already:
Use the "Git Clone URL" shown on the AUR website for that package:
Enter the directory and look at its contents. (Everything listed here, except for . .. .git is the PKGBUILD/ETC):
If you examine PKGBUILD, you will hopefully see it uses the official upstream source code, and performs typical steps to build a package, so it seems trustworthy. The .SRCINFO just contains the information shown on the website about the package, so isn't worrisome. If there are any other files here, they aren't (directly) provided by upstream, so the files and how they are used in the PKGBUILD should be examined, to make sure they don't contain anything suspect.
From Official Repositories
Although required much less often, you can build a package already in the official repositories, to include a new patch, build a newer version, etc.
Obtain PKGBUILD/ETC from the core and extra repositories:
From the community repository:
Upgrading PKGBUILD/ETC
If an upgraded PKGBUILD/ETC is released, you can come back into this directory made using git clone, and update them:
Then, recompile and upgrade the package using the method of your choice, below.
Compiling
There are many ways to compile packages. Ultimately, everything uses makepkg. There are 2 officially supported ways:
- To directly use
makepkg, see https://docs.vultr.com/using-makepkg-on-arch-linux. - To indirectly use
makepkgin a cleanchroot, see https://docs.vultr.com/using-devtools-on-arch-linux.
There are many AUR helper programs, (like the makepkg wrapper), that aren't officially supported by Arch, such as aurutils, yay, and the recently discontinued aurman and yaourt. Even if you use one of these other helper programs, It is strongly recommended to become familiar with the officially supported ways to be more effective when something goes wrong.
The rest of this doc will use YOUR BUILDER to mean whichever method you choose.
Local Repository
You can setup a local repository to be a central location for all packages you build.
Place the local repository wherever you'd like:
Run YOUR BUILDER without any automatic installation options, and copy the package into your local repository.
Add the new package to the repository index:
To remove a package from the repository's index and the package file itself:
If you need to replace an existing package file, you need to separately remove the one being replaced, then add the new one. You cannot simply copy the new file over the old one.
Configure pacman to use your local repository, by editing /etc/pacman.conf, and add the following at the end:
You need to have pacman refresh its knowledge of repository, (including your local one), databases; to see packages you've added to it:
You can then install the package, no differently than if it was in an official repository:
Note if the package is merely a dependency of another package you are going to install, you don't need to install it directly. When you install this other package, pacman will automatically find and install the dependency packages in your local repository.
Compile Faster
By default, YOUR BUILDER compiles using a single thread. On multi CPU systems, you can allow using multiple threads where possible. The build system will compile parts of the source code in parallel when it can. Sometimes parts of code require other parts it interacts with to already be compiled, so you won't always see as many threads being used as are allowed. Edit /etc/makepkg.conf.
To allow using as many threads as you have virtual cores, add the following:
Note: This will run the command nproc every time, so it will always use the current number of cores, in case you upgrade your Vultr server
To allow using multiple virtual cores, but not all of them, such as to reduce impact to overall system performance, add a specific number. For example, if you have 24 cores, you could allow 21 to be used:
Specifying more threads than the number of virtual cores you have will decrease performance.
It's fairly rare, but some packages' build systems have problems with parallel compilation, from not properly defining dependencies between parts of code. Typically, those packages' PKGBUILD files will handle this for you by invoking make -j1, which overrides the default you set. If it needs this and it's missing, report it to the Arch package maintainer.
PGP Signature Error
A PKGBUILD source array can contain .asc or .sig files. They are often included using bash brace expansion, so can be easy to miss:
If either of these formats of signature files are included in the source array, YOUR BUILDER automatically attempts verifying the upstream source archive's signature. The signature's PGP key must be in the user's keyring; otherwise, it will abort with the error:
It's important to understand a GPG key can be shown several ways. Its fingerprint is 40 hexadecimal characters, and is what you should always use. A long key ID is the last 16 digits, and a short key ID is the last 8 digits. Although shorter is convenient, it allows duplicates which voids the entire reasoning behind verifying signatures. Worse, attackers have been known to generate fake keys that match lesser length keys for high profile developers.
Obtain and Verify PGP Key Fingerprint
If you haven't tried building the package already, download the sources which will include the signature file: (If you tried building, it will already be there)
To obtain the full fingerprint:
Ideally, you should verify this fingerprint from upstream. To be secure, upstream should give its maintainers' keys somewhere on its website or in the source. Merely searching for the key on a key server isn't really doing anything. An attacker can easily submit a fake key, because key servers do not verify authenticity. Keys can be signed by other keys, so if you already have a key you trust, you should be fairly safe trusting any keys they've signed.
That can be quite a bit of work, especially when upstream doesn't publish their fingerprint or place it somewhere easy to find. The PKGBUILD will contain a validpgpkeys array, which were added by the Arch maintainer. If the package is an official repository, that means a Trusted User placed it there, and you should be fairly safe to just trust anything listed in the array. If the package is in the AUR, remember it just means that another Arch user placed it there. If you're concerned about trusting it, you can always look into the user to see what they've done in the past with Arch.
Add PGP Key to Your Keyring
To add the fingerprint to your keyring:
You can now run YOUR BUILDER, and it will trust the fingerprint.
AUR Developmental Packages
AUR packages with names ending -git, -svn, -bzr or -hg are developmental versions, which use upstream's latest version control system commit instead of upstream's latest release. For example, a -git package would use upstream's latest commit in the master branch (or their equivalent branch.) This is great for running upstream bug fixes and new features that haven't been released yet, and when working with upstream on a bug you're reporting including if you need to verify for them it's not a bug that's been fixed by a commit not yet in a release. These packages should be considered potentially unstable. That said, unfortunately, sometimes there's no alternative because some upstream maintainers never tag releases or go excessively long between tagging releases, and expect everyone to use their most recent commit. Depending on the package, you might be the first person to try running that commit. Depending on the upstream developers, their latest commit might not even compile, or it may have new bugs not in the latest release.
It's important to understand a common mistake. Don't flag an AUR developmental package as out of date simply because it shows an old version number! Developmental package PKGBUILD files contain an additional function pkgver(), which is used to automatically parse an updated PKGVER from upstream's source code. A common format for a -git package is <TYPICAL-VERSION-NUMBER>.r<COMMITS-SINCE-LAST-RELEASE>.<GIT-COMMIT>-<PKGREL>. A package might be listed in the AUR as 5.0.0.r102.8d7b42ac21-1, because that is what its PKGBUILD contains. But, when you create a package, YOUR BUILDER will automatically update PKGVER to reflect the newly downloaded source code. In fact, if many new versions have been released, but nothing has changed in the build process, such PKGBUILD listing an old version could wind up building something much newer, such as 9.1.2.r53.2c9a41b723-1. For these packages, the version listed on the website is simply the latest version at the time the AUR maintainer last had to update the PKGBUILD.
AUR maintainers are NOT supposed to just update the PKGVER to reflect new versions. They are only supposed to do so when newer upstream commits actually require something else in the PKGBUILD to change.
Only flag a developmental AUR package out of date if you know something is actually wrong. Meaning, you have actually tried using it and it fails compiling or parsing a properly formatted new PKGVER. Sometimes things happen that force the AUR maintainer to update the PKGBUILD, like upstream dependencies change, configure options change, new GCC versions pick up errors in the source code that previous ones didn't, upstream repository locations change or upstream developers will change where their typical version is within the source code breaking the PKGVER parsing function. Understand that even if it fails compiling or working, this could either mean the AUR maintainer needs to make changes to their build process, or it could be an upstream issue with their source code that the AUR maintainer has no responsibility for.
Outdated Packages
Be sure to read the "AUR Developmental Packages" section above, before reporting a package as being out of date!
If upstream has released a newer version for a non-developmental package than in the PKGBUILD, you can click "Flag package out-of-date" and type a message to the maintainer. Use https://packages.archlinux.org for official repository packages, and https://aur.archlinux.org for AUR packages. A helpful message would be the new version number, and perhaps a link to the release announcement or the source code. The flagging feature automatically emails your message to the maintainer.
On an AUR package, if there's been no response after 2 weeks, you can click "Submit Request" with type "Orphan", if you'd like to ask a Trusted User to remove the current maintainer, and make the package orphaned, if the maintainer doesn't respond to the orphan request. Generally, people only file orphan requests if they're able and willing to take over the package, and preferably only if they already have a working current PKGBUILD.
In the meantime, you can often update an outdated package yourself. Often you only need to change a PKGBUILD by updating the PKGVER to the new version number, and integrity sums be updated. A program's updpkgsums exists in package pacman-contrib, which automatically calculates the sums and updates them in the PKGBUILD for you. It's worth checking upstream's release notes, to see if they mention that anything needs to change during the installation process of the new version. Sometimes upstream changes require more changes or overhauls to PKGBUILD/ETC. Often the source array embeds PKGVER in it, so often doesn't even need updating.
0 Comments
Be the first to comment and share your perspective with the community.