Philantrop's Blog

Disclaimer: The entire migration procedure as described below is completely optional. If you don’t need or want multilib/multibuild/multiple C targets support, you can ignore this email.
We won’t force anyone to use it. I’m only going to use it on my desktop myself. :-)

We’ve now merged all multilib branches into master (this will go down into history books as “The Great Merge”!).

Thus, we don’t have any separate branches in our official and dev repositories anymore and if you still have a multilib branch  in your repository, please merge it, too.

Now is a good time to write an updated multilib migration guide so, without further ado, here we go:

• Switch to the multibuild profile in arbor.conf (${location}/profiles/amd64/multilib)
• Ensure that CHOST is not set in bashrc, and note that CFLAGS will apply to all C targets. You can use MULTIBUILD_C_{32,64}_USER_C{,XX}FLAGS for individual C targets.
• Re-install/update sys-apps/skeleton-filesystem-layout
• Install sys-libs/glibc[bootstrap], note that this should pull in sys-devel/bootstrap-gcc, too. (This is going to take a long time.)
• Install sys-devel/gcc[-openmp] (This is potentially going to take a long time, especially if you have the java option set.)
• Install sys-libs/glibc[-bootstrap], you can let Paludis purge sys-devel/bootstrap-gcc while doing so.
• Adjust your options to include the C targets you want and re-install any packages accordingly. You might have to re-install lots of package dependencies to fulfill a newly selected target’s requirements. (I simply did a cave resolve -e world)

Best regards, Wulf

We’ve come a long way with Exherbo since we started:

What really annoys me lately is the unwillingness of my fellow Exherbo core developers to act on user-contributed patches. While it’s certainly true that some patches need extensive testing and, thus, require a lot of time, there are usually a lot of trivial patches or patches that can be tested with moderate effort.

Let’s look at an example:

From 2a11f609e8be4f61a545112f1fceda6f8c40bfeb Mon Sep 17 00:00:00 2001

From: Johannes Nixdorf <mixi@user-helfen-usern.de>

Date: Thu, 24 Mar 2011 16:26:59 +0100

Subject: [PATCH] revive agg in ::mixi

—

drizzt.graveyard | 7 ——-

1 files changed, 0 insertions(+), 7 deletions(-)

diff –git a/drizzt.graveyard b/drizzt.graveyard

index 98979b9..92ed79a 100644

— a/drizzt.graveyard

+++ b/drizzt.graveyard

@@ -287,10 +287,3 @@ www-servers/

homepage = http://nginx.net

removed-by = Wulf C. Krueger <philantrop@exherbo.org>

removed-from = drizzt

-x11-libs/

- agg/

- :0 2.5

- description = A High Quality Rendering Engine for C++

- homepage = http://antigrain.com/

- removed-by = Wulf C. Krueger <philantrop@exherbo.org>

- removed-from = drizzt

–

1.7.4.1

This patch was for ::graveyard. It simply removes a single entry for a package that found a new home since I removed it. Great!

Anyone with push access could have verified and pushed it with minimal effort and, yet, nobody did for about two days and I’m pretty sure it would still be rotting on zebrapig if I hadn’t done so today.

Unfortunately, it’s by far not the only example. At the time of writing, zebrapig keeps 16 patches. A week ago, it were over 30. Nobody gave a shit.

There are patches that have been on zebrapig for months now (Ruby comes to mind!) and help with them is rejected because some people have worked on them and instead of submitting improved patches, they keep them to themselves, effectively stopping anyone from doing anything. And hardly anyone gives a shit.

Now, why is that situation so bad:

1. Patches are submitted to improve Exherbo. You’re impeding improvement by ignoring them.
2. Patches are submitted by our contributors/non-core developers. If you ignore their patches, they will become frustrated and stop submitting patches but create their own repositories, duplicating our work and making things harder for all of us. Or they will just go away. The latter is the worst possible outcome for Exherbo as a whole because with hardly more than a handful of active core developers, we will fail if we drive contributors away.
3. You’re putting more workload on the even fewer people with push access that actually work on patches. This takes away from our time working on more interesting stuff and, at least in my case, really alienates people from Exherbo.

Effectively, I’m handling most of the contributed patches myself. Either by reviewing, discussing and pushing them myself or by asking people to work on them (and usually getting stupid answers…).

Another thing that annoys me greatly are the answers I’m getting from people I’m asking to review patches; here are some examples (in no particular order) and what I think about them (which is usually not what I say):

• "I prefer playing SC2." / "I’m watching Star Trek!" – Well, yes, fine but I’m sure you could really dedicate at least one or two hours per week to work on patches.

• "Exherbo is just something nice to play with. I don’t really want to really work on it or be professional about it." – (I really hate that one.) If you take up responsibility for something, even as a hobby or so, then live up to it or leave it be. Get used to behaving professionally now while you can afford it not to be. Once you get a job, people won’t be so forgiving.
• "I don’t really care about that package anymore." – Well, I don’t either but there’s a patch so other people obviously do care. If you really want to get rid of a package, mail the exherbo-dev mailinglist and tell people. If there’s no taker, use Dexter to bury the package if need be.
• "I prefer vodka over patches." – …
• "I’m working! Not a student anymore." / "I’m so tired!" – So am I. And I’m married, have three kids and two rabbits. I’m working hard myself so you have my sympathy but, really, are one or two hours per week too much to ask? I usually invest much more time myself.
• "I’ve been busy slacking for a year!" (or longer) – Then please make it official and resign as a core developer. At least I’m going to know whom not to ask and it’ll become more obvious we’re under-staffed.
• "I said I’d do it but now I can’t be arsed." – Great, you could at least have told me right from the start you’re not going to do anything. I could have acted accordingly then.

And sometimes, the very same people who just refused to even look at a patch will ask me to review their patches seconds later…

All of this is really annoying and it’s the most likely reason for me to quit working on Exherbo and switching all my machines to another Linux distribution at some point if we don’t find a solution.

This comes up all too often, so here’s a HowTo for systemd on Exherbo:

• You have to run Linux kernel >=2.6.36-rc1. The new kernel is only needed at runtime, not for building systemd.
• Kernel options for systemd: In your kernel config, enable autofs4, devtmpfs and cgroups. Do not enable autofs3. Here’s what I’m using (I enable more kernel options than strictly necessary, though.):

CONFIG_DEVTMPFS=y (Strictly required!)

CONFIG_DEVTMPFS_MOUNT=y (unless you're using an initramfs that's mounting it for you, e. g. one created by Dracut)

# CONFIG_AUTOFS_FS is not set (Strictly required!)

CONFIG_AUTOFS4_FS=y (Strictly required!)

CONFIG_CGROUPS=y (Strictly required!)

# CONFIG_CGROUP_DEBUG is not set

CONFIG_CGROUP_NS=y

CONFIG_CGROUP_FREEZER=y

CONFIG_CGROUP_DEVICE=y

CONFIG_CGROUP_CPUACCT=y

# CONFIG_CGROUP_MEM_RES_CTLR is not set

CONFIG_CGROUP_SCHED=y

CONFIG_BLK_CGROUP=y

# CONFIG_DEBUG_BLK_CGROUP is not set

CONFIG_FANOTIFY=y (only used for readahead stuff which is not enabled by default.)

CONFIG_FANOTIFY_ACCESS_PERMISSIONS=y (only used for readahead stuff which is not enabled by default.)

• Set the “systemd” option globally in /etc/paludis/options.conf: */* systemd
• Install systemd: cave resolve -x sys-apps/systemd (Read what cave tells you. If in doubt, read Paludis’ documentation.)
• Reinstall every package with the new option set: cave resolve world -cx
• Switch to systemd as your init system: eclectic init set systemd
• Set the desired hostname in /etc/hostname.
• Optional: Edit /etc/vconsole.conf to your liking.
• Optional: Edit /etc/machine-info to your liking.
• Read Lennart’s blog post about the other configuration files.
• Install a Linux kernel >=2.6.36-rc1. (see above for kernel options, etc.)
• Temporary measure: Set the pass field in /etc/fstab to 0 for your partitions, e. g. /dev/sda2 / ext4 noatime 0 0
• Reboot.

After that reboot, you’ll be in a console with a minimal set of services started, hopefully ready to log in. Log in as root (the keyboard layout is set to US by default!). Then you can enable whatever services (found in /lib/systemd/system) you like, suggested ones are:

• dhcpcd.service or NetworkManager.service
• sshd.socket (it doesn’t start? Missing host keys? man sshd or http://tinyurl.com/24jwxjd)

As an extremely simple and limited alternative to NetworkManager.service, there’s network.service and network.conf which get installed if you set the “simple-net” option for systemd. network.service only allows for static network setups with IPv4.

Alternatively, you can use dhcpcd.service.

If I were you, I’d not enable your display manager’s service (either kdm.service, gdm.service, xdm.service or slim.service) until your basic system has at least booted properly once and you can reach your system using ssh because in case things go wrong, it’s easier not to have to wrestle with a GUI.

To actually enable a service, run “systemctl enable <foo.service>”. More details can be found in systemd’s man page.

If you need help, it’s available in #exherbo, as usual, but if you didn’t read this before asking, grumpy me will bite your head off unless you prove you read this by mentioning “Homunculus“. :-)

FAQ section:

• “How/where do you specify extra modules to be loaded?” – You put the module name into /etc/modules-load.d/foo.conf and it will get loaded. Unless udev has already loaded it for you. Check that first.
• “My hostname is set to something funny, e. g. ’08′!” – If you’re using NetworkManager, you need to set your hostname in /etc/NetworkManager/NetworkManager.conf, too.
• “I’m getting messages about failing services, e. g. dev-hugepages.mount or sys-kernel-debug.automount. What’s up with that?” – You can either enable the corresponding kernel options, delete the symlink (e. g. /lib/systemd/system/basic.target.wants/sys-kernel-debug.automount) or just ignore those messages. They’re harmless.
• “When sshd.socket is enabled, every closed ssh connection leaves a failed service around, e. g. sshd@192…:55140.service.” – Harmless as well. There are no ressources used by those so ignore them. (This should be fixed anyway.)
• “Where can I learn more about the usual administration tasks? – Read Lennart’s series of blog posts about systemd for administrators: Part 1, Part 2, Part 3, Part 4, Part 5, Part 6, Part 7, Part 8
• “How do I debug problems with systemd?” – Read this page http://fedoraproject.org/wiki/How_to_debug_Systemd_problems
• “I’m completely lost. What do I do?” – Please remember there’s always a friend around. It’s called “man”. ;-)

There were quite a few question about systemd and its kernel requirements so here are my own notes (I probably enable more kernel options than necessary):

• You have to run at least Linux kernel 2.6.36-rc1 (I’m using 2.6.36-rc3; latest NVidia-Drivers work fine and there are patches for the VMWare modules available.).
• In your kernel config, enable autofs4, devtmpfs and cgroups.
• Do not set autofs3.

My own kernel options (with respect to systemd):

# CONFIG_AUTOFS_FS is not set

CONFIG_AUTOFS4_FS=y

CONFIG_DEVTMPFS=y

# CONFIG_DEVTMPFS_MOUNT is not set

CONFIG_CGROUPS=y

# CONFIG_CGROUP_DEBUG is not set

CONFIG_CGROUP_NS=y

CONFIG_CGROUP_FREEZER=y

CONFIG_CGROUP_DEVICE=y

CONFIG_CGROUP_CPUACCT=y

# CONFIG_CGROUP_MEM_RES_CTLR is not set

CONFIG_CGROUP_SCHED=y

CONFIG_BLK_CGROUP=y

# CONFIG_DEBUG_BLK_CGROUP is not set

CONFIG_FANOTIFY=y

CONFIG_FANOTIFY_ACCESS_PERMISSIONS=y

It has been quite a while since I last wrote something about my work on systemd in Exherbo, so here’s an update:

What has been accomplished so far:

• The Exherbo patches are done. Do NOT try to submit them upstream yet, though. I’ll take care of that when the time is ripe.
• Lots of services are done.
• You can boot and run most systems using systemd now.
• I’ve built new amd64 and x86 stages without any init system so you can start out without the baselayout-1/sysvinit cruft.
• The installation guide has been updated.

Every systemd service is implemented natively and we’re not using anything from baselayout-1 or sysvinit anymore. Instead, all the important stuff has been moved to skeleton-filesystem-layout. systemd’s dependencies have been updated accordingly.

Thus, for people using systemd, baselayout-1 and sysvinit are now obsolete. YAY!

What still needs to be done:

• Improve existing service definitions for systemd.
• Create socket definitions for several of the existing service definitions. (And new ones, of course.)
• Create systemd service files for missing services.

Rules for new service files:

• Please make sure they’re implemented natively. I won’t accept non-native service files unless you can convince me there’s definitely no other solution.
• If you convinced me, scripts go to /$(get_libdir)/systemd.
• EnvironmentFiles (configuration) go to /etc/conf.d and end in .conf. We do NOT create a configuration file for every single service but create (grep-able) logical units, e. g. now-obsolete font@.service and keymap@.service used to use console.conf).
• You can reference environment variables from configuration files in service. If you have to quote the values in the configuration file, you need to use $FOO; if you don’t quote them (preferred), you use ${FOO}. This is probably a bug (and known to upstream) but for now that’s how it is.
• Services and their (potentially) accompanying files must not collide with baselayout-1.

Requirements:

• You have to run Linux kernel >=2.6.36-rc1 (I’m using 2.6.36-rc6; latest NVidia-Drivers work fine and there are patches for the VMWare modules available.).

How to get started with systemd:

Read this.

Conclusion:

Since both systemd and its exheres have now reached an acceptable degree of stability, I don’t intend on breaking things anymore as I’ve done over the last months from time to time.

In fact, systemd is so usable these days, I’m writing this on a systemd-initialised system! This means as well that I can live without baselayout-1 and sysvinit. YAY! :-)

As you may have noticed, I’ve recently added systemd to ::philantrop for use in Exherbo. I’m writing this to

• warn you that I will break systemd (and consequently your boot process) until further notice recklessly, repeatedly and without prior warning to anyone
• make clear what I intend to do with systemd in Exherbo
• make a plan for myself.

What I want to do first is get a feeling for systemd and see if it might have the potential to replace baselayout-1 (bl-1) and, at least for myself, be used instead of the init-system-that-is-not-to-be. ;)

As I really want to replace bl-1, I’m not going to go the Gentoo way of simply adding a handful of pseudo-units that essentially just call the openrc init scripts. If you want that, you’re on your own and I won’t accept patches that do that. Instead, I’m aiming for:

• a full native set of systemd units not tainted by anything else
• a minimal set of non-native configuration files (what we currently have in /etc/conf.d; I don’t think it will be possible to avoid them completely but I will if I can)
• staying as near to upstream as possible and I’ll try to submit my patches upstream even though the DISTRO_PORTING instructions don’t exactly make inclusion seem likely
• the units included in the systemd package will at most get you to some kind of login (either graphical or console)
• all additional units for services like sshd should eventually be added to their respective packages, possibly using a “systemd” option.

The process to make systemd really usable in Exherbo will be a slow one. One that I expect to take till autumn this year because:

• I’m really busy at work,
• this summer seems to become a damn hot one again and I spend quite some time after work in our pool,
• I’ll be on holidays in France for most of July.

The steps I intend to take:

• finalise the Exherbo patches for systemd (90% done, ETA: Mid June)
• create and enhance a basic set of units for booting (5% done)
• create units for other services

How you can help:

• Remind me of the stuff I’ve forgotten due to a power outage here. :-)
• Make yourself acquainted with systemd units.
• Once I’ve pushed the Exherbo patches, test systemd and
• submit patches for the missing units. :-)
• Leave comments here or on the dev mailinglist so that I can consider your input, comments and concerns.

Exherbo has developed nicely since we went public about it two years ago and even though we haven’t yet accomplished all of our major goals yet, I think we’re doing well. In this post, I’d like to comment on a few things that we’ve accomplished, some we’re still working on and some which we yet have to deal with. This is, of course, just my personal point of view and not necessarily shared by any other Exherbo developer.

I’m writing this in the hope that some of you might want to help with the open issues and to remind myself of what we did and what we have yet to do. :-)

Distributed development

I really like how we do distributed development: Our contributors just add their git format-patch to our patch-tracking bot hacchi, we’ll review the patch and if it meets our quality standards, we can just apply it locally and push it. If the patch needs improvements, we point those out to the submitter. Like that, we’re not only getting quality exheres but we help our contributors to learn good practices (which often can be applied to non-Exherbo shell scripting as well). Not only do Exherbo core devs review such patches but even other contributors do regularly. Our peer-reviewing practices are certainly one of our strong points.

There’s one issue with this, though: Peer-reviews take time and there are usually only a handful of Exherbo core devs who take care of such patches. This we certainly could improve at. I’d love to see all of us with push access to the official repositories to regularly check the patches on hacchi and work on them. This is what I believe to be part of our responsibility to our contributors.

Multi-lib / multibuild / multi-ABI

We don’t yet have a fully integrated mutlibuild system. We’re working on a system for 64-bit/32-bit multibuild but it’s in its own branches and it is currently being revamped by Saleem “compnerd” Abdulrasool. This could benefit from getting more attention. Asking those of us to work on multibuild for obscure arches like MIPS is fine but won’t get anything done. If you care, please help us actively with it. There’s #exherbo-multibuild, btw.

RepositoryRepository

Adding new repositories is not as easy as it could/should be. What some of us envisioned is the ability to query, install, uninstall and upgrade repositories just like any exheres. As far as I know, nobody is working on it at the moment – maybe this your chance for eternal glory? ;-)

Init system / Genesis

I’ve been looking into many init systems over the years and specifically again lately. This was mostly due to the fact that Genesis, Exherbo’s own init system, was very much delayed. Let’s face it – most init systems that were/are not SysV-compatible have failed (some don’t know that yet, though ;-) ) and, to be honest, I wouldn’t bet on Genesis to succeed. Thus, while we have baselayout-1 (among other options) at the moment, I’ve been looking into Upstart again. I became curious about it again when I saw that several major distributions are moving to use it and so I set up a virtual machine, switched it to using Upstart and experimented a bit with it.

Things are looking good and so I’m probably going to create native Upstart jobs soon.

Writing a Developer’s Manual

I still want to create a Developer’s Manual as well. I like Gentoo’s DevManual and I might use that as a starting point. Or maybe not. Not sure yet but if anyone has a good idea about it…

For more great ideas to work on, see http://dev.exherbo.org/~kloeri/gsoc2010/ideas.html.

In an unexpected move, the Gentoo trustees have contacted us, Exherbo, recently with a suggestion. Disappointed with the intense internal quarreling among Gentoo developers (which Gentoo has been famous for anyway), the general inability of the current Gentoo council to provide the necessary vision and strategic governance for Gentoo as a whole as well as the lack of implementation of newer EAPIs in Portage, the trustees deemed it necessary to step in.

Gentoo Foundation, Inc.’s president, Roy “NeddySeagoon” Bamford being a regular in our IRC channel #exherbo on Freenode discussed the current state of Gentoo with his most trusted advisors, Denis “Calchan” Dupeyron (already an active Exherbo contributor) and Thomas “tanderson” Anderson (Exherbo dev). All three of them had first-hand experience with Exherbo and the way we work – distributedly, with strong peer-review and almost every single user contributing to Exherbo.

This effectiveness, the overall quality of Exherbo as well as the fast-paced development of both our exheres-0 format, the amazing speed of Paludis development (Paludis being Exherbo’s package manager) and, last but not least, Ciaran McCreesh’s latest information about the move to cave and egress, a Portage-UI-compatible Paludis client, convinced them to boldly go on with their plan to strengthen Gentoo: To join forces with Exherbo.

Of course, the transition of both resources and developers will be a slow process. While the merger with Gentoo is to become effective as of today, April 1st 2010, the announcement on the Gentoo website might lag a bit behind as Gentoo’s PR project is unfortunately somewhat overworked. It should be noted as well that not all Gentoo developers are likely to make the transition. Some have, let’s say, rather strong feelings about Exherbo, some feel they’re not up to a fast-growing, fast-paced modern Linux distribution and some might simply not notice before next year.

I’m sure there will be some friction yet to overcome yet but, all in all, I think Gentoo does the best thing possible – moving in with us, under the ultimate supervision of Bryan “kloeri” Østergaard (who once served Gentoo as head of Developer Relations and, thus, is well-suited for the job) as the “benevolent dictator” Daniel Robbins once was, will make good old Gentoo find back its way to former glory albeit under a new name – but, as the saying goes, what’s in a name? (Speaking of which: The name “Gentoo” is a trademark of Gentoo Foundation, Inc. until that is changed, too, and this site is not part of the Gentoo project and is not directed or managed by Gentoo Foundation, Inc.)