Live status display for Factor's build farm, and other improvements

Lately I've made a few additional improvements to the build infrastructure. First, build reports sent to the Factor-builds mailing list are now formatted as HTML, which makes them a bit more readable. Second, there is a Twitter feed of binary uploads: @FactorBuilds. Finally, we now have a live status display for build machines. Previously if I wanted to know what a build machine was doing, I'd have to log in with ssh and poke around -- total waste of time. Now, just clicking on the OS/CPU combination on factorcode.org takes you to a status page. For example, here is the status page for Mac OS X/x86. At the time of this blog post, I see that this machine is running tests for a certain GIT revision. Each GIT revision is a link to the GitHub mirror of the Factor repository so you can see exactly what's going on. Finally, the latest build report can be viewed too; this has the build failures, if any, as well as benchmark timings.

Our continuous build system is called mason and adds up to a total of 1196 lines of code. Eduardo Cavazos wrote the first iteration, called builder. I forked it and added additional features.

It's been over a year since we started using continuous integration in the Factor project, and I can say its been an overwhelming success. The first iteration of the build system would load all libraries and run all unit tests, and send an email with the results. If everything succeeded, it would upload a binary package. Over the last year, more than 5000 binary packages were uploaded. Over time, we added more checks to the build farm. It now runs help lint checks which ensure that examples in documentation work and that there are no broken links, and checks for compiler errors.

I think the code quality has definitely gone up over the last year; having a dozen machines running tests all the time does a good job of triggering obscure bugs, and the automatic upload of binaries when tests pass saves us from the hassle of making manual releases.

I think pretty soon, we're going to start having releases again, in addition to continuous builds. I intend on making the release process semi-automatic; when I decide to do a release, I want to send some type of command to all the build machines to have them build a given GIT ID and upload the binaries to a special directory. The goal is to have regular releases until 1.0, starting from a few weeks from now. I'm not going to commit to a schedule for 1.0 yet, but having regular releases and published change logs is the first step of the process.