Hi there! My name is Aki Saarinen.

This post is about combining functional programming, finite-state machines and the actor model of computation using Akka FSM.

Finite-state machine, or FSM for short, is a mathematical model for structuring a computer program. Many programmers have come across them, either in a CS course in university or in practical applications. FSM is a useful tool, because it forces the programmer to lay out the states and flow between them explicitly. I often use FSMs as a mental model for reasoning about programs, even if the program would not be explicitly structures as one.

Actor model of computation, extensively used in e.g. Erlang, is another interesting concept: application is modelled as a group of independent actors, who only communicate with each other using asynchronous message passing. Actors are an old concept, first presented in 1973, but are becoming increasingly popular as a way of handling concurrency and parallelism as the number of cores increase. There is a great talk about actors on Channel 9, involving Carl Hewitt, the inventor of actors, and Eric Meijer from Microsoft Research. I recommend watching it, regardless of your level of experience with the actor model.

Regarding the third component in this post, functional programming, Michael Feathers recently wrote about hybridizing object-oriented and functional design:

Object-orientation is better for the higher levels of a system, and functional programming is better for the lower levels.

Michael throws in the usual disclaimers about this not applying to every case, to which I agree, but I still think there is something to it. I love functional programming: filtering, mapping and folding lists allow me to write very concise code, fast. However, pure code with no side-effects is ultimately of no use on itself: someone somewhere must see the results. How to orchestrate the side-effects and build the larger application around the functional pieces is a very interesting question, one which does not have a clear-cut answer.

In the spirit of Michael’s post, but instead of using traditional objects on the higher level, could we structure the application as a set of functional finite-state machine actors? Independent entities, whose behaviour is well specified in terms of states and state transitions. Entities which communicate asynchronously using actor message passing. Entities whose state transitions are written as purely functional pieces of code. This seems like an intriguing concept.

The idea itself is nothing new. The Erlang design principles, for example, describe how to implement finite-state machines using Erlang actors. I have, however, rarely seen this as the base for application design.

A few weeks ago I was excited to find Akka FSM, a finite-state machine implementation library for Akka, an actor framework for Scala.

Scala is well suited for functional programming, and Akka FSM does hence provide the possibility to combine the aforementioned three pieces into a single clean package: implementing the application as a set of actors which are internally functional finite-state machines. I have been writing code using Akka FSM for a few weeks now, and it seems like a natural way of structuring programs. So far I am very pleased.

The official documentation contains plenty of examples, so I won’t replicate them here. Check the docs and give it a go.

During the Easter holidays I updated the technology for this site.

My goal was to find a simple and elegant solution for hosting two things: my static home page and this blog. Until yesterday, the blog was hosted at Posterous. Posterous was recently acquired by Twitter for their talent, which means the service is likely no longer actively developed. On a related note, there was good discussion in Hacker News about talent acquisitions.

In addition to the fact that Posterous platform is dying, I was never really happy with writing the blog posts in a WYSIWYGish web interface. Writing in Vim with basic Markdown syntax is an attractive alternative.

Here’s what I came up with. A pretty standard stack these days, I guess. I have been reading about almost all of these tools for a while and it was nice to see how well everything worked. Not always the case with hyped tools.

Everything is based on the static site generator, Jekyll. Installation is straightforward using RubyGems, just install RubyGems in your preferred way and run gem install jekyll. Then all you need to do is run jekyll and the whole site is compiled to static HTML, ready to be hosted where ever you want. I am using Kapsi for hosting, but basically any provider, like Github pages or Heroku would do. Nice thing about this setup is that it requires nothing except a basic HTTP server from the hosting provider.

Jekyll contains support for Markdown based page rendering and Liquid templates out-of-the-box. There are at least two major projects that try to bring one-click blog hosting around Jekyll: Octopress and Jekyll-Bootstrap. I decided against using these, for two reasons. First, I wanted to keep things as simple as possible and second, I wanted to learn in the process.

To make the site look like it does, I used two tools: Compass and Twitter Bootstrap.

Compass is a CSS Authoring Framework, as they call it. In practice, it lets me write style sheets in Sass. Sass an excension of CSS3 which compass then compiles to standard CSS3. End result is cleaner stylesheet syntax and less duplication than with traditional CSS files. Being also a rubygem, installation is again simple: gem install compass.

Basic layout is achieved using Twitter Bootstrap styles, with some custom CSS thrown in. In case you are not already familiar with Bootstrap, I suggest you take a look.

Sources are hosted in a git repository and deployed to production using a simple shell script. Previewing posts and changes locally is easy:

compass watch
jekyll --server --auto

This results into compass watching for changes in Sass files and compiling them to CSS as soon as the file is modified. Jekyll starts a standalone HTTP server in port 4000 and lets me preview the latest changes by accessing http://localhost:4000/. The --auto flag automatically re-compiles the site as soon as there are any changes.

The feedback loop for testing is extremely fast, something I value a lot. Writing and publishing this blog post is now all sunshine and happiness.

I wrote an article about C++ development with Qt, together with my colleagues Sami Rosendahl and Tuomas Järvensivu at Reaktor. Check it out at Reaktor’s site.

It’s interesting to see what happens to Qt in the future. It’s clearly one of the best C++ libraries I’ve used and I’m sincerely hoping they’ll stay on the right path.

The new design of Google Reader has been debated a lot. I was also frustrated by unnecessary large margins and useless bars, especially when reading from my 13” laptop screen.

I made a simple Chrome extension to fix things up a little. You can install CleanReader from the Chrome Web Store for free. Sources are available at GitHub.

Yesterday I wanted to add a new post to the social bookmarking service, trunk.ly, by emailing a link to my username+magicnumber@post.trunk.ly address. I was typing the address out of my memory and unfortunately sent my link to @trunk.ly instead of the correct address at @post.trunk.ly.

In many services a mistake like this could result in a silent failure and a lost email. Or some technical “Deliveration failed: error 0xF3F3”. Afterall, I did email to a stupid address, right?

Trunk.ly did something different. I received a personalized reply, with trunk.ly co-founder Tim Bull as the sender:

From: Tim Bull <tim@trunk.ly>
To: Aki Saarinen <..>
Subject: Re: LiveReload

Hi Aki,

Just FYI - the links need to mailed to post.trunk.ly NOT trunk.ly :-)

Cheers,
Tim Bull

How amazing is that! It made me feel that they really want to take good care of me and my links. In a technical sense things like this are not hard to implement, it’s just that we often don’t even think about it.

Ricardo Vice Santos gave an interesting presentation about the Spotify architecture yesterday at Reaktor Dev Day.

What struck me as a clever yet rarely used idea was his description of their usage of DNS for internal service resolution and configuration in the Spotify backend.

Connections from clients come through an access point, a relatively thin proxy to various backend services. Services provide authentication, decryption keys, track information and of course the actual music data, among other things. How did they configure the whole network of services? They used DNS. It’s pretty easy to store configuration parameters as TXT records in DNS alongside other records. Access point queries the internal DNS server and then decides where to connect the incoming request.

Setting up a DNS server is really easy and DNS servers are one of the most battle tested pieces of code. When was the last time you had to worry about DNS or heard about DNS server failure? You almost never do, because it just works.

Just finished an article about building a portable boombox with no previous experience with DIY audio systems. Check it out!

Oprofiler is a system-wide profiling tool for Linux that utilizes hardware performance counters in the CPU to provide a wide range of interesting statistics about program execution. Oprofiler is not enabled in the standard Android distribution, but with some modifications, it can be used under Android also.

I was able to run oprofiler under Nexus One and Android 2.2, with one limitation. I could only profile once after a restart, further attempts to start the profiler would not succeed in collecting any data. This limits the usability of oprofiler, but it was enough for my immediate profiling needs. If you manage to fix this issue, let me know.

Compile kernel with support for profiling

Oprofiler needs support in the Linux kernel, and the prebuilt kernel for Nexus One doesn’t include this support. You need to compile your own version of the Android kernel for your device, with profiling support enabled.

You can check what kernel version your phone is currently running from Settings->About Phone. I suggest you try to compile the same version of the kernel that the phone is running as default. A good article about building the kernel and platform for Nexus One is available here. I used kernel version 2.6.32 for my phone with Android Froyo 2.2 (FRG83).

You should use standard kernel configuration tools (“make menuconfig”) to enable the support, but you can compare results with this patch to see you’re enabling the right things (CONFIG_OPROFILE, etc).

In order to keep the kernel small enough (there’s a limitation what can be fit into the phone), I additionally needed to disable ext3 and ext2 support.

Interrupt bug in 2.6.32 kernel for Nexus One

Oprofiler module in 2.6.32 kernel is missing an interrupt for Nexus One CPU. The bug is discussed in the android-platform mailing list and original patch are available here. I also have a mirror of the patch, just in case. Apply the patch, include the kernel with your custom Android 2.2 distribution and start profiling.

I recently needed to downgrade my radio firmware of an european Google Nexus One for research purposes. To my surprise, HTC or Google didn’t provide official packages of the older radio firmware. If you want to get an older radio firmware, you’re on your own. Only way of getting them is to dig deep into Android developer forums and download a radio image from unofficial sources.

Because flashing the radio firmware is one of those things that has a serious risk of bricking the phone, it’s not too fun to try with shady images and see what happens. If the image is incompatible or damaged, you’ll end up with a brick.

I can’t distribute any radio images, because their license doesn’t allow redistribution. But I can publish the checksums of the images I successfully used, so you can verify the files you’ve got to see if it’s the same that I’ve been using. I hope this can be another data point you can use to decide whether a radio image is reliable or not.

Note that I don’t have any visibility what’s inside these, they’re just a result of a few days worth of forum searches and experiments. They could even contain malware. What I can tell is that they did not brick my phone and the radio was still working after these were in. The checksums are of the radio.img file only.

I have a newer Nexus One model (the one with SLCD), microp firmware version 0c15, and hboot version 0.35.0017.

Radio 5.12.00.08

Tested to work with Android 2.2.1 Froyo (FRG83).

• MD5: 263fb298d747f9e5632b373c69ce7893
• SHA1: 2ad521b954178f0962d25c13ba45014df7d2c455

Radio 5.08.00.04

Tested to work with Android 2.2.1 Froyo (FRG83).

• MD5: dee19eddd42cd0166398bcab37663f62
• SHA1: 802656e261433400d3a56a978b0350b180bc8884

Radio 4.04.00.03_2

Tested to work with Android 2.1 Eclair (ERE36B).

• MD5: 310d85c4998163818f7dcdef730c2a12
• SHA1: 1bc692631d33f8b885a5152d602cb3f2e812250d

I wanted to downgrade my Google Nexus One momentarily back to stock Android 2.1 (called Eclair), in order to build my own version of the TaintDroid system. I first needed the stock Eclair to fetch the proprietary 2.1 compatible libraries from the phone. These libraries are not allowed to be distributed by anyone but the vendors, which means they are only embedded into the stock images. I had Android 2.2 installed to the phone, but the libraries there were incompatible with 2.1.

I wrote this post because there’s quite a lot confusing and contradictory information about downgrading, scattered around various forums. Hopefully someone can learn from my experiences. I’m using a phone with unlocked bootloader and most of the tricks only work after unlocking, so if you want to keep your phone locked, this is not the way for you.

Using these steps I managed to boot up stock 2.1 Eclair in Nexus One, even though admittedly I not in a 100% working shape. It kept throwing some exceptions into my face, but it was in a good enough condition to use adb to download the libraries. After that I successfully compiled and installed the modified 2.1 in TaintDroid, which worked just fine. Probably the stock installation could’ve fixed too, but I didn’t invest my time in finding what was wrong with it.

SLCD or AMOLED?

First piece of the puzzle when downgrading is that there are two kinds of Nexus Ones. An older generation with an AMOLED display and a newer one with an SLCD display. You can boot up your phone to the bootloader and check microp version to see which one you have: 0b15 is AMOLED, 0c15 is SLCD.

I have the newer one, which becomes a problem: the kernel in stock Android 2.1 does not support SLCD, so you need to compile a kernel by yourself, as described later on in the post. Android 2.1 can be booted up with SLCD, you just have to have support for it in the kernel. I have no experience with the AMOLED Nexus One, so I’m not sure about that, but I believe it will work out-of-the-box.

Hboot, microp and radio firmware

I had a recent version of the bootloader firmware, also called hboot. My bootloader is in version 0.35.0017. Some sources claim the newer bootloader wouldn’t work with Eclair. I had no problems with the newer bootloader, so I have proof that it does work just fine. Nexus one with SLCD also has newer version of the microp firmware (0c15) than originally used for by Eclair (0b15), and it also works just fine. No need to worry about those.

A possible troublemaker could be the radio driver. I had some problems along the way, which led me to downgrade my radio firmware back to 4.04.00.03_2. When I was all finished, though, I also successfully booted up Eclair with radios 5.08.00.04 and 5.12.00.08, so I believe they should all work and my problems were cause by other factors at the time.

If you run into trouble you can’t figure out, you can try downgrading the radio, some checksums of working radio images available in my other post.

Getting a kernel with SLCD support

In order to boot Android 2.1 Eclair with the SLCD, you need to have a new enough kernel. I used 2.6.32, newer ones might do as well.

Simplest way to get a working kernel is to use the boot image from Android 2.2 (Froyo) distribution. You can get one for Nexus one from developer.htc.com. I used Nexus One FRG83 system image. Unzip the zip, and you get a boot.img loaded with 2.6.32 kernel, which you can eventually flash into the device with ‘fastboot flash boot boot.img’.

If you want, you can also compile your own 2.6.32 kernel. This will become handy (and even a necessity) later on, if you want to customize your platform at all, but it’s not strictly necessary at this point. If you don’t want to compile your own kernel, you can skip the next section.

Compiling your own 2.6.32 kernel

A good writeup about how to compile 2.6.32 kernel for Android is available at here. First part of the post is about compiling Android 2.2 distribution, but take a look at the kernel part. After compiling the kernel, you can use ‘fastboot boot /path/to/my/kernel/zImage’ to boot up Android with your custom-built kernel instead of the one flashed to the device. This is a handy way of testing a new kernel, without having to flash it.

I compiled my kernel while still having Android 2.2.1 installed on the phone. This was convenient, as the phone already had a 2.6.32 kernel, so I could use that as a baseline for my kernel configuration. That way I could also modify one thing at once, so if the system didn’t boot, I knew the reason was my kernel, and the kernel only. Only after having a fully functioning 2.6.32 kernel image did I proceed with installing Eclair.

Downloading and flashing stock Eclair image

You can download the stock image for Android 2.1 from developer.htc.com, I used Nexus One (ERE36B) Official OS Image.

In order to flash the zip with ‘fastboot update NexusOne_ERE36B_TMOUS.zip’, you need to unzip the package, modify android-info.txt so that you allow the newer bootloader and microp versions (and radio if you didn’t downgrade):

require version-bootloader=0.33.2012|0.33.0012|0.35.0017
require version-microp=0b15|0c15
require version-baseband=4.04.00.03_2|5.12.00.08

…and then zip it back in.

This will just tell fastboot that you want to flash the images even though you have a newer bootloader, microp and radio versions. I’m not that experienced on this subject, so I’m not 100% sure if there’s a way this can screw things (flashing always has a hint of danger to it), but I had no problems myself. I believe this to be safe, if the flashing is done properly, but be warned though, I’m not going to take the blame if you brick your phone.

Booting Eclair with custom kernel

After flashing the stock Eclair, you can try booting up the phone, but with an SLCD you’ll end up with a black screen. My phone also seemed to jam up so I had to take out the battery before being able to reset it.

What you need now, is the 2.6.32 kernel image. If you want to use the one from stock Froyo, just issue ‘fastboot flash boot boot.img’ for the boot-image coming with froyo and you should be good to go.

If you compiled it yourself, simplest way is to use ‘fastboot boot myKernelImage’, so you don’t have to craft a boot.img to be flashed. If you want to be able to boot the 2.1 without some assistance from a computer and the fastboot utility, you’ll need to do that eventually, too. I didn’t because I only used the stock 2.1 for a while in order to fetch the proprietary drivers.

Fixing network drivers

Since the bundled bcm4329 network device driver in the system image is compiled to work with the exact kernel version thats bundled with the distribution, it won’t work when booting up with a different kernel using ‘fastboot boot’. To make networking work, you need to patch the bcm4329.ko from your custom kernel to the system partition of the phone, where it’s loaded from in the boot procedure. This might apply to other drivers too, however I was only experiencing problems with networking. If you only want to fetch the proprietary drivers, you won’t even need the networking, anyway.

A working Android 2.1 in Nexus One

If everything went well, you should now have booted up stock Android 2.1 in your Nexus One. Happy hacking, whatever it is you wish to do with it!