Sunday 22 July 2012

Explaining the Speed of Selenium Updates

With the release of Selenium 2, we moved from incredibly slow updates and releases to far more regular ones. We were aiming for weekly releases, but have settled into something that averages between two and three weeks. That's a pretty fast update schedule. Why do we do it?

The first reason is that there are a lot of checkins every week on the project. Last week, which admittedly was pretty busy, saw 94 separate revisions in the space of the 7 days between the 14th and the 21st of July 2012. The previous week was quieter. There were 84 checkins. Each of these checkins represents either a refinement of existing functionality, a bug fix or (increasingly rarely) new features. We'd like to get those features and fixes in front of the users as quickly as possible.

The second reason is that we need to continue working with the latest and greatest versions of browsers that are out there. Although Internet Explorer is on a comparatively leisurely yearly release cycle, Chrome and Firefox release a new major version once every six weeks. Six weeks! Each of these releases tends to call for a new selenium release too.

One of the nice side-effects of the design of the webdriver APIs is that we can, in theory, decouple the release cycle of the individual browser drivers form the release cycle of the client libraries. Since the Chrome team maintain the chromedriver, this has been the case for a long time with chrome (hurrah!). Opera were the ones that blazed this particular trail, and the opera driver releases are independent too. With the next release of Selenium, this will also be true for all client languages for IE as well.

Which leaves Firefox.

If we were happy with only using synthesized events, we could relax a little with the selenium releases. The problem is, we're not happy with only using synthesized events, and neither are our users. Right now, the OS-level events are fired using a binary component built by the selenium project. The way that these components work means that we need to do a new release for every major number release of firefox. Back in the day, this meant "almost never". Today, that means "once every six weeks". It's tiring for us. I'm sure it's tiring for Mozilla. It's definitely tiring for our users.

We're working with Mozilla to try and get a mechanism for injecting events at the OS level built into Firefox itself. Once this mechanism is in place, we should be able to slow our release schedule a little (though we'd still like regular releases in order to get bug fixes in front of you)

The other thing that Mozilla are working on is the Marionette project. This is an implementation of the webdriver APIs embedded within Firefox (and mobile, and Firefox OS) and maintained by Mozilla themselves. Once this in place, the selenium project will be able to completely decouple releases of the client libraries from the browser drivers.

This decoupling should be a Good Thing for users. For a start, it'll be possible to do far more focused updates. For example, let's imagine that we discover and fix a bug that only affects IE users. We could release a new IE driver server as soon as the fix is in and users can grab that, without also needing to grab a new version of the client libraries (or the Firefox driver....)

Oh well. We'll see.