Many Months in Selenium: to November

Well, it's been a long time since I sat down and wrote a post about the adventures in Selenium-land. Time for an update!

Since I last wrote, the work of updating the JSON handling code in the java tree has been completed, and it appears to be stable. However, it would be a terrible waste of time if that was all that we had done, and fortunately it's not :)

The big thing is that we've now finished the 3.x release cycle, and we're getting ready for 4.0. Someone foolishly let me pick version numbers, so the last few releases of 3 tended to get ever closer to π. Judging by some of the bug reports, the initial jump, from 3.14 to 3.141 appears to have confused some folk, but now that we've reached 3.141.59 I think the point has been made (and, just maybe, the joke is wearing thin)

The 4.0 release is going to be a lot of fun. My main focus has been the new Selenium Grid, which features a more modern design, for use with things such as AWS and Kubernetes (and docker compose). Of course, maintaining a simple user experience has been high on our list of goals, so people used to the existing approach of "hub" and "node" will continue to be able to run the system like that. The biggest change is that under-the-covers, the new standalone server and the new Grid are exactly the same software, which is a huge change from the current approach where we have two not-terribly-well-integrated codebases in the same binary.

Another big change is that we're exploring the move from using Buck to build much of Selenium to Bazel. This hasn't been something I've been keen on doing, since I used to be the tech lead on Buck, and I think it has a number of useful properties. Despite this, Bazel has a comparatively huge amount of community support, and that means that people wanting to hack on the project have a smaller learning curve to climb.

A Month in Selenium: March

The month from February to March has been a fun one. At the beginning of March, I attended SauceCon, and gave a keynote on "Lessons From a Decade in Selenium". While the original talk had focused on milestones such as when we first started shipping code, or when we switched to Git, or when someone joined the project, as I sat in the airport waiting to fly, I realised that this was an incredibly dull talk; surely the point of keynote is to give people something to think about and consider?

This explains why I was busy rewriting the entire thing at 12km above the ground in a metal tube zipping along at a smidge over 900km/h.

In the end, I spoke about what makes working on Selenium so rewarding, focusing on the themes of "Joy", "Serendipity", "Thankfulness", "Community", "Growth", and "Striving". I've yet to see the official feedback, but I believe that the talk was well received, as people kept returning to the main themes throughout the conference.

SauceCon itself was a lot of fun. We were lucky to have some of the Selenium committers (old and new), and also supporters of the project from companies such as Sauce Labs itself, and Applitools (who are providing almost all the effort going into the new Selenium IDE) In addition, the Appium developers were well represented too. It was great to be surrounded by so many people who have spent so much time pouring energy into Open Source Software, and to catch up with some of my favourite people. There's a lovely photo of Jonathan Lipps and myself in matching bowling shirts, which I'm happy to see he tweeted.

Since we had so many people in the same place, we decided to release Selenium 3.10. The main highlights of this release were behind the scenes for most users, as we focused on the continued clean up of the internal of Grid, and the continued use of our own abstractions to handle HTTP and JSON. Having said this, there were user-supplied patches, notably moving us from Selenium's own "Duration" class to the one that ships with the JRE. Deleting code is a lot of fun.

One reason for shipping 3.10 was to lay the groundwork for a terrible dad-joke: releasing Selenium 3.11 on the 11th March (3/11 in US date format). Jim Evans and I had noticed that 3.11 was also one of the most famous of the Windows releases, so we decided to lean into the joke, and shipped "Selenium for Workgroups" as well in March. The Selenium server even reports this to users. 3.12 won't have this feature.

In a bid to help our Windows developers ship the Selenium jars, I merged a ton of upstream changes to our fork of Buck, and then spent some time attempting to resolve the issue where zips created on Windows create unreadable directories when unpacked. My fix doesn't resolve the issue, so I filed an issue with the upstream Buck project in the hope that they'd fix it for me. If I get some free time, I'll try this as well.

I've also been working on replacing GSON within our tree (though on my local machine). By the end of the month of work, I had a forked version of Selenium that didn't use GSON at all for outputting JSON. Sadly, I was a little over-ambitious when attempting to finish the work by also deserialising from JSON to proper types. It turns out that there's a bunch of code in Grid that relies on the current semantics of GSON to function. Stepping back, it looks like most of this is because GSON isn't aware of our own types, and it should be relatively easy to replace some of this. At least I know I should be working on next….

Well, that, and a new way of starting sessions that allow users to properly make use of all the features that the W3C New Session command offer.

A Month in Selenium: February

January was a quiet month for Selenium hacking, but it laid the groundwork for February's efforts. These largely centred around code cleanup in the Grid server, and migrating the project to make better use of our own abstractions over JSON and HTTP.

Why do we have our own abstractions for these incredibly common tasks? There are two main reasons. The first is that we'd like freedom to be able to choose our underlying implementation for these things, without needing to extensively rework our own first-party code. The second is that third party libraries offer generalised APIs that need to meet the needs of all users, whereas we have very specific needs met by these APIs and may need to work around some of the sharp edges (for example, in the java code, lots of classes that need to be serialised to JSON have a toJson method that GSON knows absolutely nothing about). This is typically done by writing adapters.

We started using the Apache HttpClient by default as it's the HTTP library used by HtmlUnit, which we used to ship as part of the core Selenium distribution. In keeping with the other drivers out there, the HtmlUnit team now work on the HtmlUnitDriver, so it's no longer kept in the main project source repo. The interesting thing is that since we made the choice a long, long time ago to use the HttpClient, the HTTP standard has moved forward. HTTP/2 is now a thing. HTTP/2 support is coming as part of HttpClient 5. In order to take advantage of the new options and capabilities, we'd have to rework our existing abstractions anyway, so why not take a look around for something else to use? Better yet, if we use an HTTP library that isn't a dependency of one of our dependencies, we're less likely to end up with clashing versions.

One of the reasons that Java has a terrible reputation for start up speed is because people have massively bloated classpaths. As it stands, the Selenium standalone server weighs in at a portly 24MB. The Apache HttpClient weighs in at about 1.4MB of this total, before we do the update. After the update, the beta of 5 is a touch under 1MB. In comparison, OkHttp (which already supports HTTP/2) with its dependencies is approximately 500kb. In other words, OkHttp is smaller, already supports HTTP/2, and isn't a dependency of our dependency.

So, we switched the project to use OkHttp instead of the Apache HttpClient.

Within the client code, making this change was relatively trivial. The problem is that the server-side code had leaked Apache's APIs into the code. Before we can replace the Apache HttpClient, we need to first of all replace all those usages. That's made somewhat harder by the fact that it's exposed as part of the public APIs of various classes that other libraries extend.

Fortunately, we have a process for deprecating and deleting APIs. First of all, we mark the methods to be deleted as "deprecated" for at least one release. And then we delete them. Of course, if you're going to deprecate a method, you really should provide an alternative and migrate as many uses as can be found to use the replacements. A bulk of my work this month was spent making these changes.

Of course, we needed to do a release, so we lined up 3.9 to start the process. In order to do the release, we needed to actually build it. There had been reports of some issues building the release artefacts on Windows. To resolve this, I had to update our fork of Buck to pull in the latest changes from Facebook, and then to try and work around those issues. Naturally, the Buck developers aren't aware of our fork, so merging in their changes was a somewhat time-consuming affair. Once that I was done, I wrote what I thought was a fix and pushed a new version of our fork of Buck.

I didn't work. Oh well.

The final step in doing a release is trying to get our CI builds green. These take an incredible amount of time to run, and I wondered whether we could speed them up. Travis has support for caching, so it would be nice to use that. My attempts to use caching were foiled because the cache takes into account environment variables, which we use to separate our builds. There's a bug open in the Travis tracker to allow us to name builds, which would have allowed us to work around this, but it's still open. Ho hum. As a work around, I wrote a simple wrapper around Buck that we can call within our CI servers. This makes better use of Buck's ability to parallelise work automatically, and this has helped bring our build times down. Hurrah!

Two Months in Selenium - November and December

You may have noticed a distinct lack of an update last month. It's because I was focused on client work, Christmas, and the New Year, and took some time away from the keyboard. But I'm back now!

The W3C WebDriver spec is now at the stage where we need to demonstrate multiple compatible implementations. Realistically, this means that we need two passes for each test in our test suite. The browser vendors are working hard to get things working, and progress is being made. There's not been a huge amount for me to do here, so this is more of a waiting game than anything else from my perspective. Having said that, I'm on the hook for some sections in Level 2, so I should really sit down and write those (and the matching tests)

The main thing I've been focused on has been the Selenium Grid. There are a couple of things that we really need to solve with Grid. The first is that the code is complex and hard to deal with. When we originally released it, it took a huge amount of work to review the code for thread-safety and to debug many of the issues. That code has not become easier to reason about, which makes it harder to foster Open Source contributions.

Of course, that'd be fine if we didn't care about making any changes, but we do. When Grid came into being, it was normal to have a physical server for each node in the grid. If you were lucky, you might have a massive server with VMWare running on it, which you'd cycle virtual machines on to keep the Grid healthy. The world has changed. Docker is now A Thing, and there are multiple "Selenium as a Service" (SaaS) cloud providers.

There are some projects out there that implement some of the functionality of Grid. For example, selenoid makes use of Docker, but it doesn't use the W3C dialect of the webdriver protocol, which means it doesn't do protocol conversion, and it doesn't natively support cloud providers. Zalenium builds on top of Grid, and provides support for Docker and SaaS, but they've had to work within the existing architecture, and there are obvious rough edges.

Finally, we've wanted the selenium server to be a "Grid of one". If you go into the code of the server, you'll see that there are two fairly separate trees that live side-by-side. When you start the server, it picks one and then goes with it. It'd made things like supporting the W3C protocol harder than it should be, and it's not an elegant way to run things.

As a solution to this, it seems obvious that there should only be one code path. The problem is that the standalone server is too simplistic about how it assigns work, and (as discussed) the Grid code is too complex. Over the past few releases, I've been landing code to help resolve this:

• The pass through mode: this makes the server proxy requests without doing an parsing or changes unless necessary.
• The ActiveSession abstraction has been added. This makes adding new types of provider (SaaS, Docker) far easier to write.
• A "new session pipeline" has been added, and this is being used to handle things like multiple versions of the webdriver protocol.

The most recent thing I've been working on has been a new scheduler. This will be rolled into the new session pipeline, and is composed of a number of pieces:

• The "Scheduler": this is responsible for queuing new session requests, handling retries, and waiting until nodes become available. This class is thread-safe and designed to be the main entry point.
• The "Distributor", which is solely responsible for ranking and ordering available hosts, and the sessions that can run on them.
• The "Host" abstraction, which represents a physical place where sessions can be run. Each of these has a number of....
• The "Session Factory", which is responsible for creating a new session.

The scheduler will sit within the new session pipeline. For the standalone server, we just add a single host. For the grid, we can add an arbitrary number of hosts (and therefore session factories)

As well as the new scheduler, we're preparing the 3.9 release. It should be out next week, if everything goes according to plan. :)

A Month in Selenium - October

Another month, another update, you lucky people.  The highlights:

W3C TPAC.
I attended the W3C's TPAC meeting in California. This is the main get-together for many of the "working groups" that are working on standards as part of the W3C. It's also where the Browser Tools and Testing Working Group met to discuss progress on the WebDriver spec.

Good news! Once we clean up the implementation report, we're ready to move to "Proposed Recommendation", which is the last step before becoming a standard (or "Recommendation" in W3C parlance).

More good news! The "Level 2" version of WebDriver will have a new logging infrastructure added. This will make it easier for you (yes, you!) to figure out where failures have occurred. Better insight should lead to more stable tests.

Even more good news! Some of the folks from Sauce Labs attended the face-to-face meeting. They help bring an additional perspective to the design and use cases of the protocol. Until now, the group has been mostly composed of browser implementors and people from the Selenium project. The more people involved with the spec, the better it's going to be.

The minutes for this face-to-face session are available, as are the minutes for the other face-to-face sessions.

Hacking on Selenium
Last month, we were closing in on the Selenium 3.7 release. This month we shipped 3.7.0 and then, because of a small oversight where we missed a jar file in the downloadable artefacts, 3.7.1. There are some nice changes in there. As mentioned last month, one of the areas of focus has been improving how we handle the New Session command when dealing with a local end that might speak both the W3C and JSON Wire Protocol dialects of the webdriver protocol. One of the things that the spec says we're meant to do is pass though additional top-level fields in the new session payload. 3.7.1 now does this (hurrah!)

One of the nice things from the work in 3.7 is that we've laid the groundwork for a clean up of the Selenium Grid code. As part of that, we restored a behaviour where a Grid Node, configured with a path for the Firefox or Chrome binary would have this path injected into any capabilities when starting a session. Making the nodes even more configurable is something that's on the road map for a later release.

More next month!

A Month in Selenium - September

I realise that this blog has been pretty quiet. Part of the reason for that is that I'm terrible at sitting down and just writing. What I really need is an incentive. That incentive arrived this month in the form of the Selenium Fellowship, which takes the form of a stipend to fund work hacking on Selenium. Part of the agreement is a monthly blog post. So, you all have the Software Freedom Conservancy to thank :)

So, what contributions have I been making to the Selenium project this month?

There are two major highlights. The first of these is Selenium Conf, which was in Berlin. I gave the State of the Union keynote (so called because the first one was an update of how the merger of the Selenium and WebDriver projects was going) Over the past few Selenium Conferences, the theme has slowly been building that Open Source Software depends on people to move it forward. This time, the message was far starker, as I counted the number of people who contribute to key parts of the project --- for some pieces, we depend on one person alone. I also covered the various moving pieces in the project, using Kent Beck's "3X" model as a framework to hold the talk together.

As well as being part of the show at SeConf, I also had the pleasure of helping out Jim Evans in the "Fix a Bug, Become a Committer" workshop. He did a great job explaining how the pieces fit together, and by the end of the workshop, we had everyone building Selenium and running tests in their IDEs of choice (provided that choice wasn't "Eclipse"), which is a testament to the hard work he'd put into preparing the session. It did highlight that the "getting started" docs probably need a bit of a polish to become usable. I was also invited to do a Q&A with the folks in the "Selenium Grid" workshop, where I broke from theme to talk about the role of QA in a team. Thanks for being patient, everyone!

In terms of code, as I write this, I've landed 57 commits since September 17th. Part of this was to help shape the 3.6 release. For Java, the theme of this release was the slow deprecation of the amorphous blob of data that is "DesiredCapabilities" to the more strongly-typed "*Options" classes (eg. FirefoxOptions, ChromeOptions, etc). The idea behind the original WebDriver APIs was to lead people in the right direction: if they could hit the "autocomplete" keyboard combination in their IDE of choice, then they'd be able to figure out what to do next. The strong typing is a continuation of this concept, and is something that all the main contributors are fans of.

One implementation detail we made in the Java tree is that each of the Options classes are also Capabilities. I made this choice for two reasons. The first is philosophical. We don't know ahead of time what new features will land in browsers (headless running for Chrome and Firefox are examples), so we'll always need an "escape hatch", to allow people to set additional settings and capabilities we're not aware of. The second is pragmatic. The internals of Selenium's java code is set up to deal with Capabilities, and people extending the framework have been dealing with them as an implicit contract of the code.

In the wild, there are two major, and one very minor, "dialects" of the JSON-based protocol spoken by the various implementations. The first is the original "JSON Wire Protocol", and the second is the version of that protocol that has been standardised as part of the W3C "WebDriver" specification. We took pains when standardising to make sure that a JSON Wire Protocol response is almost always a valid W3C response (technical note: because all values are returned as a JSON Object with a "value" entry, which contains the return value), but there are two areas where the dialects diverge wildly.

One area is around the "Advanced User Interactions" APIs. The end point offered by the W3C spec is significantly more flexible and nifty than the original version in the Selenium project, but it is also a lot more complex to implement.

The other area is around "New Session", which is command used to create a new Selenium session. The JSON Wire Protocol demands that the user place the set of features that they're interested in using into a "desiredCapabilities" JSON blob. This was originally designed as part of a "resource acquisition is initialisation" pattern --- you'd load up the blob with everything you might want (a chrome profile, an equivalent firefox profile, the proxy you'd like to use) mashing together items that theoretically only belonged to one browser into a single unit. The remote end was then to do a "best effort" attempt to meet those requirements, and then report back what it had provided. The local end (the driver code) was then to test whether or not the returned driver was suitable for whatever it was that users wanted to do. Which is why they were called desired capabilities --- you made a wish, and then could look to see if it came true. If nothing matched, it was legit for a selenium implementation to just start up any driver and give you that.

The W3C protocol is a lot more structured. It provides for an ordered series of matches that can be made, with capabilities that must be present in all cases. For our example above, the proxy would be used for any driver, and then there'd be an ordered set of possible matches for chrome and then firefox (or vice versa). Each driver provider gets a chance to fulfill that request, and if it can, then we use that driver. If nothing matches, then we fail to initialise the session and return an exception to the users.

The more structured data used by the W3C New Session command is sent in a different key in the JSON blob, and this is by design. In theory, it's possible to map a JSON Wire Protocol "New Session" payload to the W3C one, and to map the W3C structure to something close to the JSON Wire Protocol payload. Sadly, this process is complex and error prone, and there are language bindings that have been released that get this wrong to one degree or another (and, indeed, some that don't even make the effort) All this means that the Selenium Server has to try and discern the user's intent from the blob of data sent across the wire. Getting this right, and flexible, has been the focus of the forthcoming 3.7 release.  It's fiddly work, but it'll be worth it in the end.

Another common problem we see is that some servers out there speak the W3C protocol natively (eg. IEDriverServer, geckodriver, the Selenium Server) and others don't yet (eg. safaridriver, chromedriver, and services such as Sauce Labs). A big part of the 3.5 release was the "pass through" mode, which means that if the Selenium Server detects that both ends speak the same "dialect" of the wire protocol, it'll just shuttle data backwards and forwards. However, if it detects that the two ends don't speak the same protocol, it'll do "protocol conversion", mapping JSON Wire Protocol calls to and from W3C ones. This has been made easier by the fact that the W3C spec is congruent with the JSON Wire Protocol -- the two have identical end points for many commands.

But not all commands. The main ones that have been causing grief have been the advanced user interaction commands, particularly when a local end speaks the JSON Wire Protocol, and the remote end speaks the W3C one. Just such this situation arises for users of some cloud-based Selenium servers, and its been a constant source of questions from users. To help address this, I've landed some code that does emulation of the common JSON Wire Protocol advanced user interaction commands (things like "moveTo"). Hopefully this will address the majority of headaches that people are experiencing using this new functionality.

Let's see what the next month brings. Hopefully, we'll ship 3.7 :)

On Managing Time and Direct Emails About Selenium

I tend to get a few requests a day asking for help with a problem with Selenium. I almost never reply to these. It's not (just!) because I'm an evil minded, grumpy so-and-so, but because it's not a great use of anyone's time.

I am just one person. I have a full time job, and family and friends that I like to spend time with. I work on Selenium as a volunteer, and that means fitting it in where I can. Fortunately, work are understanding about this, and support my role in the project, which means I do far more than if I only had the occasional evening free. Still, it does mean that I prioritise my time spent on the project. This is how I do so:

1. Writing code. That comes first.
2. Spend time on the IRC channel. I often just lurk here, but it's a handy way to talk to the core development team and keep an eye on what's going on.
3. Answer emails to the selenium-developers group. This is where we run the project and hold design discussions. It's not a good place to ask for help, unless that help is about implementing Selenium itself.
4. I scan the webdriver and selenium-users lists, answering questions where I can and provided I have time.

Now, the nice thing with this ordering is that the further down the list you go, the more people there are who are able to help you with your issue. Put another way: asking for help in the user lists means that you're far more likely to get the help you want. It also means that if someone else runs into the same problem, Google can come to their rescue. A private email doesn't have that benefit.

I know that may be frustrating for you. I know that it seems to make sense to contact prominent people on the project directly. I understand your particular issue is urgent and important to you. I really do, and that's why I don't answer your emails.

The UNIX Philosophy, WebDriver and HTTP Status Codes

The UNIX philosophy can be described in many ways (and the Wikipedia page has plenty), but I've always admired its practical application in the wealth of shell commands available to a user. Rather than having a single command that Does Everything, the UNIX shell is a place of small commands focused on doing one thing well, yet which are easy to link together.

For example, I recently needed to compare the contents of two JAR files and remove class files that were duplicated from one of those jars. I ended up generating the list of shared files via:

comm -12 <(jar tf first.jar | sort | uniq) <(jar tf second.jar | sort | uniq) | grep -v META

I doubt very much whether the authors of any of those tools thought that this is what I'd be doing, yet because the tools are carefully focused and are easy to chain together, this is a trivial thing to do.

How, you may ask, does this apply to Selenium? And specifically issue 141? For those of you who can't be bothered to read the incredibly long list of comments on that issue (now at over 100), this is the one about being able to get HTTP status codes from the WebDriver API. The comments are split between those saying that this functionality doesn't belong in the API, and those who (occasionally very vociferously) claim that it does. 

From a philosophical perspective, the WebDriver API is attempting to model a user interacting with their browser. We attempt to limit the APIs we offer to just those that meet this need, only allowing ourselves to extend it to those very clear cases where the browser is the Source of Truth about a particular thing (such as with cookies), or where there's no rational way to cleanly offer a facility (such as executing Javascript --- incidentally, something that I spent a lot of time keeping out of the API)

HTTP status codes don't fall into either category. The browser isn't the the source of truth about these codes, as that's the originating web server. The user may not be aware of them either; a 404 from a .js file? That'd most likely go unnoticed. A 500 from even the main page? That may be returned as a 200 by some app servers in certain configurations. 

So that leaves our users out to dry, right? Well, it would if it wasn't for the UNIX Philosophy. You see, it's ridiculously simple to hook up a proxy that will capture this information for the user if you can't obtain the information by instrumenting the server. You can do it like this:

// Explain where your proxy lives
Proxy proxy = new Proxy();
proxy.setHttpProxy("your_proxy:8080");

// Now tell the webdriver instance about it
DesiredCapabilities caps = new DesiredCapabilities();
caps.setCapability(CapabilityType.PROXY, proxy);

WebDriver driver = new RemoteWebDriver(caps);

That's 5 lines of code in enormously verbose Java. 

Separating the concerns of "browser automation" from "logging network" traffic allows the Selenium developers (most of whom are not paid to work on Selenium) to focus on the problem of driving the browser. It means that they're not working on writing their own HTTP proxy, which is a sufficiently taxing tax that there are many projects out there working to write something solid and stable.

Great options for users looking for a powerful and capable proxy include Fiddler and Charles. Another option is the BrowserMob Proxy, which started as being a fork from the original Selenium RC codebase (Oh! The irony!) but has since matured and grown. This is amazingly simple to integrate with a WebDriver instance, as shown in their docs. For brevity, the integration can be done like so:

ProxyServer server = new ProxyServer(4444);
server.start();

// get the Selenium proxy object
Proxy proxy = server.seleniumProxy();

Following the UNIX approach, we make it easy to use a proxy with the WebDriver API. That means that we're not implementing an API for getting HTTP status codes in the Selenium project not only because it's out of scope, but there are already people doing a great job of offering that capability elsewhere. 

The Future of Mobile Testing

Six or so years ago, Jason Huggins and I were talking about the next generation of web testing tools. This wasn't a conversation about the as-then unreleased Selenium 1.0 or even of WebDriver, which was a new and shiny thing I was working on at ThoughtWorks. This was about the next generation of testing tools.

The fact that we can do automated testing on the Web is a happy accident. When MS and Netscape put Javascript into browsers and standardised the DOM they didn't do so with an eye to making it easy to write tests. They wanted new and whizzy features added that only worked in their browser in a fight to win the browser wars. Each browser implemented the features of the other and then added more in a bid to gain the edge. The fact that it was possible to build something like Selenium on top of this work was never meant to happen.

That wasn't the conversation that Jason and I were having. We were talking about what the next generation of testing tools would use; the ones that would make selenium and webdriver totally redundant. It was obvious to both of us that accessibility APIs would be The Way Forward. After all, users with one form of disability or another make up a small, but important, percentage of web users. Their equal access to information and applications are enshrined in laws. Not only is making an app accessible a groovy and lovely thing to do, it's also often a legal requirement.

The next generation of tools, we reasoned, would build upon this accidental automation infrastructure in the same way that we used the DOM and JS: to provide an API that can be used to drive and query an application from outside that application.

Microsoft lead the way, when .Net 3.0 contained an API called UI Automation. I was working on a project with Mike Two, who hacked together a proof of concept against the desktop app we were working on before flying to India to be closer to the dev team there. Some time later, White appeared, which took the concepts and followed through. Brilliant stuff. 

Then it went quiet.

Until, that is, the mobile revolution started. For an amazing number of users, their primary contact with the Web will be a mobile device, probably either Android or iOS. The problem is that neither of these platforms have "making it easy to write an automated end to end test" baked in as a concept. Increasingly, however, they do have the keys to accidental testability provided: their accessibility frameworks, and these are often called something useful like "UI Automation" 

All this means that the next generation of tools are coming.

There is, however, a missing piece. We have the WebDriver APIs for testing web-based content, and for testing native content we have the accessibility APIs (which can be wrapped to look webdriver-ish if desired)(and I think it is desired)(but I'm biased) But how do these two gel? How do we test a "hybrid" app, composed of both native and web-based content? In my view, this gap can best be bridged by augmenting the accessibility API to allow a webdriver instance to be returned from any WebViews that are found via the accessibility APIs, and by allowing returned WebElement instances to also implement the equivalent of UIAElement, so that it can be the target of OS-level simulated user inputs.

There. Problem solved.

What My Leaving Google Means for Selenium

TL;DR: nothing.

Longer version:
One of the questions I've received most since announcing that I was leaving Google was "what does this mean for the Selenium project?"

My answer is "not much" The project itself is Open Source, and more than half the commits are coming from developers outside of Google. These developers range from people working at other browser vendors (notably Mozilla and Opera) to people who are just interested in the project and write amazing code. Those people aren't going away. The OSS project has already demonstrated that if I'm not around things still get done (though the releases slow waaaay down :), but I'll still be reading every commit and still contributing where I can. Better still, I'm just one person, and the project is vibrant and humming with activity.

What about the browser vendors? One obvious impact might be that the ChromeDriver stops moving forward. That's deeply unlikely to happen: the chrome driver is maintained as part of the chromium OSS project by members of the Chrome team itself, rather than the team I was TL of (Browser Infrastructure at the last name change) This is also true of the OperaDriver, which is maintained by Opera Software, and will be true of the FirefoxDriver once the Marionette project is available on release builds. The trend we've been encouraging is that browser vendors should be responsible for their drivers --- given the complexity of the task, this is the best way to ensure that what our users want is what our users can actually do --- and the browser vendors are rising to the challenge.

Of course, this does beg the question of what my team at Google does. The answer is "a heck of a lot". Google has made a massive investment in browser automation. A lot of that investment is visible in the contributions to the Selenium project (where most of the team have earned the commit bit), but it can also be seen in other projects such as Wicked Good XPath, Web Puppeteer and the Browser Automation Atoms. The team also works on integrating these APIs with Google's infrastructure, and providing support and guidance to teams, and they're constantly striving to make writing web tests so stable and easy that even a software engineer can write them :) I may have left that team, but that investment continues unabated.

Sotto voce: if you'd like to join that team, I can pass on your CV....

Which leaves my involvement with the W3C spec. It's true, over the next couple of months I intend to spend a lot of time with my family, but I'm also planning on spending time working on the spec. Once I join Facebook, I fully intend to continue co-editing it. That work is going to continue.

So overall, my take is that my leaving Google isn't going to have an appreciable effect on the OSS project or the spec.

Which is nice.