The delusions of debian
Published on 2022-03-30. Modified on 2022-11-05.
Debian was my favorite Linux distribution for servers where Linux where required to run for many years, and I think I have been running it in production since about 1998 up until the time about when the systemd conflict arose. Since that time I have noticed a rapid decline in many areas of Debian. This has continued up until today and Debian is no longer what Debian used to be.
Let me begin by saying that this article is not about systemd, I am simply using the conflict about systemd as a reference point in time.
Because of the problems with systemd, the Debian community was split in two, where a minority of former members and contributors decided to fork Debian into Devuan. Debian was, and still is, a very big project and the founders of Devuan struggled for years because forking Debian was no easy task. However, they where not the only ones struggling. The internal strive made several leading members simply quit Debian (without joining Devuan), and many people felt understandably "betrayed" by how the systemd conflict was handled.
Despite the split, the Debian project steadily kept adding new third party software to its repositories. As of writing Debian Bullseye has 96.754 packages and 168.670 packages in unstable, a number that just keeps growing. That is a massive amount of software packages!
One of the primary reasons to deploy Debian on production servers in the past was because of the way Debian maintained both the kernel, the userland, and third party packages in the stable release. On the project website Debian still "boasts" of the same procedure.
- Debian offers security support for its stable releases. Many other distributions and security researchers rely on Debian's security tracker.
- Debian's free of charge Long Term Support (LTS) version extends the lifetime of all Debian stable releases to at least 5 years. Additionally, the commercial Extended LTS initiative supports a limited set of packages for more than 5 years.
On the Debian Security Information it is further stated that:
Debian takes security very seriously. We handle all security problems brought to our attention and ensure that they are corrected within a reasonable timeframe. Many advisories are coordinated with other free software vendors and are published the same day a vulnerability is made public and we also have a Security Audit team that reviews the archive looking for new or unfixed security bugs.
Furthermore, on the Debian Security Audit Project website it is stated that:
The Debian Security Audit Project is a project which is focused upon auditing Debian packages for security issues.
In the short time it has been running it has been responsible for several Debian Security Advisories proving that this auditing process really works to improve Debian security. It is hoped more advisories will result from future work.
By taking a proactive stance in auditing code we can help to ensure that Debian continues its long history of taking security seriously.
The aim of the project is to audit as many of the packages within the Debian stable release as possible for potential flaws. Important packages which are contained in the unstable distribution may also be examined for flaws, decreasing the likelihood of insecure packages entering the stable release in the first place.
In the past, before the systemd conflict, Debian was famous in the Linux world for all of the above, and it was one of the most widely deployed Linux distributions on servers. But, as the website also rightly states further down:
Due to the sheer size of the current Debian release it is infeasible for a small team to be able to audit all the packages, so there is a system of prioritizing packages which are more security sensitive.
Debian is perhaps still the biggest Linux distribution in the world and it is still an amazing project (even though I TRULY wish that the project had rejected systemd and replaced their default GNOME desktop with something else). However, the fact of the matter is that Debian has long been experiencing a decline in the amount of people willing to participate in the project. This is further made worse by the fact that the current leadership is more worried about Debian having a black lives matter sticker somewhere on the project website than he is with the fact that tons of Debian packages are abandoned and many important packages get security updates long after the bugs have been fixed upstream.
Before that, Jonathan Carter, the former leader had these ridicules points in focus for the future of Debian.
I have been going through a number of packages during the weekend and found that the sad state of web browser support currently within Debian also extends to many of the packages.
More than that, the Debian project is absolutely delusional about its long term support!
At the time when Debian 11 was about to be released, PHP 8.0 was 10 months old yet Debian's 11 was released with PHP 7.4. That makes PHP 7.4 the standard in Debian stable for at least 3 years after its release. But PHP 7.4 only got upstream support until 28 Nov 2021 and security support is permanently ended at 28 Nov 2022, which is seven month from now. This means that unless Debian has some really good C developers, no one can provide any security fixes for PHP 7.4. Not only that, no one outside of Debian will be monitoring problems with PHP 7.4 because everyone else will long since have upgraded to PHP 8.
The delusion then continues down the stream where ordinary Debian users are left with the impression that because Debian promises long term support, there is no problem running with these outdated packages on their servers.
Maybe the problem is that a lot of myths surround many of the popular Linux distributions. When you go to e.g. Reddit, it is filled with misinformation parroted by ignorant users. But perhaps I shouldn't call it "myths", but rather misinformation because what is stated on some of these project websites, and what goes on in the real world, are often two very different things.
What I don't understand is why so many Linux distributions aren't open and clear about the problem with package maintenance!
Stop saying that you focus on security. Stop saying that you provide long term support. Tell everyone plain and clear that your so-called long term support is naturally conditional upon what upstream does, and that if a package version is abandoned by upstream, or orphaned by the package maintainer, then all bets are off.
If you're a regular BSD or Linux user reading this, you need to understand that if upstream abandons a package version and not longer provides security updates or bug fixes for the package, then likewise the package maintainer on your favorite OS cannot do anything, the LTS depends on upstream.
You also need to understand that regular and timely updates depends 100% on the package maintainer(s) time and work. A single maintainer may maintain as many as 500 packages. If you want to understand how well upstream packages are tracked in your OS, take a look at the packages that are important to you and compare the release dates upstream with the release dates of the package on your OS.
Don't blindly trust the promises made on the website of your favorite OS. Also, don't simply trust what other users on Reddit and elsewhere says, that "this is OS is oh so secure", and "project X is more secure that project Y", and bla bla bla. Many people simply have no idea what they are talking about.
This is the very basic of what you need to do if security is important to you:
- Keep a list of all the important software you install, including the version of the kernel.
- Make sure you monitor bug reports and security reports filed upstream. It's not enough to simply watch some popular CVE website, these websites are not always timely.
- Determine if and how a problem affects you. A buffer overflow problem in your favorite text editor is not a problem if you are the only user with access to the machine. An exploitable bug in something like NGINX, if you run an Internet facing web server, is a serious issue.
- Keep a careful eye on your operating system, whether they have noticed the bug report, whether they have reacted to it. You can help here by letting the ports/package maintainer know about the issue, but this is no guaranty that the package will be updated in a timely manner. If the package maintainer is ill, out of town, or simply doesn't have the time, you're on your own!
- Plan ahead on how to deal with problems. Ideally, use an OS that makes it easy to build and compile your own packages directly from upstream. This is relatively easy on all the BSDs because of the ports system. It is also relatively easy on the Linux distributions with similar systems to the ports system, such as Gentoo, Arch Linux, Void Linux, and others. The ports system is one the greatest assets for users who want flexibility and control over their software. On FreeBSD you can even use Poudriere on a build machine.