Friday 25th December 2009

A friend of mine is interested in trying out the Automatic Rule Based Time-Tracker arbtt which I programmed. Unfortunately, he is using Windows and up to now, arbtt only worked on Linux. But as I wanted to check out Haskell’s cross-platform abilities for a while, this was a good opportunity to do so. I don’t have Windows installed myself (and did not plan to do so), so I did all this under WINE, the Windows compatibility layer, which works very well: It takes only a few minutes to install the Haskell Platform for Windows and then I was able to run wine ghc --make and cabal install.

I played around with some simple programs and was surprised by these timings:

$ rm *.o *.hi; ghc --make fourfours.hs ; time ./fourfours > /dev/null
[1 of 1] Compiling Main             ( fourfours.hs, fourfours.o )
Linking fourfours ...

real	0m1.909s
user	0m1.692s
sys	0m0.208s
$ rm *.o *.hi; wine ghc --make fourfours.hs ; time wine ./fourfours.exe > /dev/null
[1 of 1] Compiling Main             ( fourfours.hs, fourfours.o )
Linking fourfours.exe ...

real	0m1.631s
user	0m1.376s
sys	0m0.092s

So it is faster to run a compiled Haskell program on top of a compatibility layer than directly on Linux! The world is in order again, though, if optimization is enabled:

$ rm *.o *.hi; ghc -O --make fourfours.hs ; time ./fourfours > /dev/null
[1 of 1] Compiling Main             ( fourfours.hs, fourfours.o )
Linking fourfours ...

real	0m0.981s
user	0m0.876s
sys	0m0.108s
$ rm *.o *.hi; wine ghc -O --make fourfours.hs ; time wine ./fourfours.exe > /dev/null
[1 of 1] Compiling Main             ( fourfours.hs, fourfours.o )
Linking fourfours.exe ...

real	0m1.270s
user	0m1.036s
sys	0m0.072s

Funny. Anyways, I wanted to port arbtt. The only platform-dependent part is the capture module that gathers the list of open Windows. The Win32 package that comes with the Haskell Platform did not cover all the functions needed to do so, but creating additional function bindings is really easy with Haskell, as can be seen in the Graphics.Win32.Window.Extra module. I also replaced the locking code that prevents two instances of arbtt-capture to run at the same time by equivalent code using Windows mutexes (module System.Win32.Mutex). With these small changes and some CPP conditionals to make the code compile for either platform, the porting was done! Even accessing the files in ~/.arbtt works correctly on Windows, where it will look in the Application Data folder, without changing the code, thanks to System.Directory.getAppUserDataDirectory.

But Windows users won’t like compiling software on their own. They won’t even like installing software by copying various files to certain directories. Therefore, I also had to create a Windows Installer. I picked Inno Setup, because it’s Free Software and seems to be simpler than NSIS. The installer puts the compiled .exe files, the example categorize.cfg and the HTML documentation in the right spot, adds icons to the Start Menu (“Edit categorize.cfg”, which fires up wordpad, a link to the documentation and the uninstaller), puts arbtt-capture in the Autorun folder, puts the path to arbtt-stats in the PATH variable and starts arbtt-capture at the end (the last three points being optional). Of course it undoes all this when removing the program again. I integrated the call to the Inno Setup installer into the usual ./Setup build process of Haskell packages. Some more details of how to create the Windows installer are mentioned in the README file.

Now all this does not magically add a graphical user interface to arbtt, so users will still have to work with arbtt-stats on the command line – even on Windows. If this is not a problem for you then you can fetch the latest installer from the arbtt homepage. And if you happen to become a serious user of arbtt on Windows and want to help maintaining the Windows port, I’ll gladly share some responsibilities.

I’m very satisfied with the process and the result and I’m happy to know that I can offer some of my programs also to Windows users in the future.It is also a big plus for Haskell – with Python, shipping a program for Windows users is likely more difficult. The next step will be providing gtk-based graphical Haskell applications for Windows, including a nice installer that ideally includes all dependencies (gtk etc.).

Red Hat has announced financial results for its fiscal year 2010 third quarter ended November 30, 2009. "Total revenue for the quarter was $194.3 million, an increase of 18% from the year ago quarter. Subscription revenue for the quarter was $164.4 million, up 21% year-over-year."

Fedora has updated kernel (F12, F11: denial of service), phpldapadmin (F11: local file inclusion).

To be found in Vladimir’s “multiple terminal” branch, GRUB 2 displays in multiple terminals simultaneously, with a separate menu viewer for each one, resulting in menus with different metrics rendering the same content!

In this screenshot, GRUB running in QEMU with a serial terminal attached to it in a separate window:

Expect this to land in GRUB Experimental branch soon.

Merry Christmas!

Patch forthcoming, but I guess most people knows how long the road from screenshot → cleaned-up, full-functioning patch → patch that does what upstream wants it to do → Debian unstable → Debian testing → Debian stable is.

Somehow, I'm still lacking the optimal media player application. Many popular ones are totally overloaded (e.g. amarok). Others like totem seems to be just a minimalistic frontend for a particular backend.

My current choice:

• Single-shot playback: to view a random song or video I usually open them with Totem (the GNOME default) and that works okay
• Library: I use MPD as player because it just seems to be rock stable. As UI I currently use Sonata, but I don't use it for much more than choosing a song from the currentl playlist.
• Editing: ExFalso seems to have the best ID3v4 support, in particular it also allows multiple genre fields. (Note that Vorbis even suggests you should use multiple artist fields instead of the common "Arist A & Artist B" way of filling the fields)

However, there is one thing I'm really not satisfied with: when putting together a CD compilation for friends (say, as Christmas present), they are quite useless. A key issue here is the total playlist length. Guess what, I want to make sure it fits on a single CD. So I really need to know the total playlist length. Why do so many media players (e.g. totem, alsa-player-gtk, xfmedia4, vlc, mplayer, ...) not show you the total playlist length? They did read all the files to get artist and title. Many even have the individual song lengths, just not the total sum.

In the past I've been using old XMMS1 to check for the total length, or a CD burning application like K3B by repeatedly importing my current folder.

Right now, I'm using Quod Libet (since I like the tag-editing component exfalso a lot) to arrange the playlist. It also gives me the total length, albeit I belive I've had incorrect song lengths in it before (broken VBR files?), and it's not perfect, too: being database-driven it has really long startup times for occasional users (because of updating the database) and is much more heavyweight. I also believe I've lost some playlists because I had moved my files around once ... so I'm a bit sceptical.

Anyway, there are still hundreds of media players I havn't looked at. Don't bother me to send me an email about one I havn't mentioned!

But if you are developing a media player, please consider the use case of putting together a music CD for your friends. In particular, for users that do not use your player all day.

Being stuck at home because of my stepfather's health condition becoming worse and therefore family meeting plans being changed, I just made the Debian project a small Christmas gift by fixing an RC bug on December 25th for the first time in my Debian life.

Apparently, after TeXLive packages changes, this one will not be the only one discovered by Lucas automated archive rebuilds.

Anyway, Merry Christmas to those of you who celebrate it.

When being told about the differences between Puppet and Chef by Puppet fans, a few common talking points emerged. They included the terms “DSL”, “declarative”, and “shell script”.

“Shell script” was probably the most entertaining meme, and referred to how, without dependencies, Chef was just like writing shell scripts in Ruby.

In actuality, a Chef cookbook appears to be roughly 17 files in 7 different directories. Most of them end in .rb, but there is also a metadata.json to keep the metadata.rb company, and perhaps an .rdoc if you like that sort of thing. I have no idea who writes shell scripts this way. I am also stupefied that people say this is easier than Puppet, because I doubt I would want to perform “system administration” by writing one of these things.

So when I started writing my own thing, I contemplated the dependency issue, and for a few minutes decided to be lazy and leave them out entirely. Once it dawned on me, however, that I was then writing shell scripts in Lua, I became ashamed.

If I were going to go that route, why not just set up a git repository with a bunch of shell scripts, and set up cronjobs on each of the clients to do a git pull and run these shell scripts as root? After all, nothing beats shell scripting for shell scripting, not Perl, not Python, not Ruby, and least of all Lua.

So I wrote a dependency resolver. It was easier than I had anticipated and I have vague plans to make it smarter than Puppet's. Then I'm justified in sticking to my Lua “DSL”.

Now Steve has a list of requirements that seem to be relatively mild. I think he could easily get away with a VCS-shellscript-cron setup. The VCS pull can be tunneled over ssh or stunnel or anything else which will provide authentication and encryption, so there is no need for a fancy custom server with its own certificate authority. In short, all the tools are there and you just need glue.

I will continue to pretend that I'll need more complex logic and power that only Lua can give me, so I'll end up doing VCS-Lua-cron until I change my mind.

Ever since Linux kernel 2.6.31 was released, my AMD Geode LX800-based FIC ION 603 can no longer boot; I get a fatal kernel crash near the end of the initramfs loading phase. At the crux of the issue seems to be a recently added kernel feature for caching filesystem ACL. I don't even use the ACL feature on my hosts, but there you go; it still crashes the kernel. Sigh.

Having filed a bug on Launchpad and reported the issue upstream failed to produce a fix - despite the sustained involvement of two members of the Ubuntu kernel team members to help me narrow down the cause of this fatal kernel crash.

Mentioning the issue to OLPC kernel developer Andres Salomon, he commented how he finds the new ACL caching code in the ext2/ext3/ext4 filesystem drivers downright creepy, but added that he currently lacks the time to further investigate the issue.

I'm thus wondering if anyone else is experiencing this issue and might possibly have a fix to propose?

Alternately, help towards debugging and fixing this issue is extremely welcome, as this bug will seriously affect users upgrading from older Debian and Ubuntu releases based on pre-2.6.31 kernels, when Debian/Squeeze and Ubuntu/Lucid are published in spring.

PS: yes, the issue still exists in kernel 2.6.32 also.

It was interesting to see Clint Adams describe love and dissatification with configuration management.

At work I've got control of 150(ish) machines which are managed via CFEngine. These machines are exclusively running Debian Lenny. In addition to these hosts we also have several machines running Solaris, OpenBSD, and various Ubuntu releases for different purposes.

Unfortunately I made a mistake when I setup the CFEngine infrastructure and when writing all the policies, files, etc, I essentially said "OK CFEngine controlled? Then it is Debian". (This has been slowly changing over time, but not very quickly.)

But in short this means that the machines running *BSD, Solaris, and non-Debian distributions haven't been managed as well via CFEngine as the rest, even though technically they could have been.

A while back I decided that it was time to deal with this situation. Looking around the various options it seemed Puppet was the way of the future and using that we could rewrite/port our policies and make sure they were both cleanly organised and made no assumptions.

So I setup a puppetmaster machine, then I installed the client on a range of client machines (openbsd, debian lenny, ubuntu, solaris) so that I could convince myself my approach was valid, and that the tool itself could do everything I wanted it to do.

Unfortunately using puppet soon became painful. It has primitives for doing various things such as maintaining local users, working with cronjobs, and similar. Unfortunately not all primitives work upon all platforms, which kinda makes me think "what's the point?". For example the puppet client running upon FreeBSD will let you add a local user, setup a ~/.ssh/authorized_keys file but will not let you setup a password. (Which means you can add users who can login, but then cannot use sudo. Subpar)

At this point I've taken a step back. As I think I've mentioned before I don't actually do too much with CFEngine. Just a few jobs:

• Fetch a file from the master machine and copy into the local filesystem. (Making no changes.)
• Fetch a file from the master machine, move it to the local system after applying a simple edit. (e.g "s/##HOSTNAME##/`hostname`/g")
• Install a package.
• Purge a package.
• Setup local user accounts, with ~/.ssh handled properly.
• Apply one-line sed-style edits to files. (e.g. "s/ENABLED=no/ENABLED=yes/" /etc/default/foo)

(i.e. I don't use cron facilities, I add files to cron directories. Similarly I don't use process monitoring, instead I install the monit package and drop /etc/monit/monitrc into place.)

There is a pretty big decision to make in the future with the alternatives being:

• Look at Chef.
• Stick with CFEngine but start again with a better layout, with more care and attention to portability things.
• Replace the whole mess with in-house-fu.

If we ignore the handling of local users, and sudo setup, then the tasks that remain are almost trivial. Creating a simple parser for a "toy-language" which can let you define copies, edits, and package operations would be an afternoons work. Then add some openssl key authentication and you've got a cfengine-lite.

For the moment I'm punting the decision but I'm 90% certain that the choice is CFEngine vs. Chef vs. In-House-Fu - and that puppet is no longer under consideration.

Anyway despite having taken months to arrive at this point I'm going to continue to punt. Instead my plan is to move toward using LDAP for all user management, login stuff, and sudo management. That will be useful in its own right, and it will coincidentally mean that whatever management system we do end up using will have on less task to deal with. (Which can only be a good thing.)

ObFilm: Terminator II