You may have noticed a distinct lack of an update last month. It's because I was focused on client work, Christmas, and the New Year, and took some time away from the keyboard. But I'm back now!
The W3C WebDriver spec is now at the stage where we need to demonstrate multiple compatible implementations. Realistically, this means that we need two passes for each test in our test suite. The browser vendors are working hard to get things working, and progress is being made. There's not been a huge amount for me to do here, so this is more of a waiting game than anything else from my perspective. Having said that, I'm on the hook for some sections in Level 2, so I should really sit down and write those (and the matching tests)
The main thing I've been focused on has been the Selenium Grid. There are a couple of things that we really need to solve with Grid. The first is that the code is complex and hard to deal with. When we originally released it, it took a huge amount of work to review the code for thread-safety and to debug many of the issues. That code has not become easier to reason about, which makes it harder to foster Open Source contributions.
Of course, that'd be fine if we didn't care about making any changes, but we do. When Grid came into being, it was normal to have a physical server for each node in the grid. If you were lucky, you might have a massive server with VMWare running on it, which you'd cycle virtual machines on to keep the Grid healthy. The world has changed. Docker is now A Thing, and there are multiple "Selenium as a Service" (SaaS) cloud providers.
There are some projects out there that implement some of the functionality of Grid. For example, selenoid makes use of Docker, but it doesn't use the W3C dialect of the webdriver protocol, which means it doesn't do protocol conversion, and it doesn't natively support cloud providers. Zalenium builds on top of Grid, and provides support for Docker and SaaS, but they've had to work within the existing architecture, and there are obvious rough edges.
Finally, we've wanted the selenium server to be a "Grid of one". If you go into the code of the server, you'll see that there are two fairly separate trees that live side-by-side. When you start the server, it picks one and then goes with it. It'd made things like supporting the W3C protocol harder than it should be, and it's not an elegant way to run things.
As a solution to this, it seems obvious that there should only be one code path. The problem is that the standalone server is too simplistic about how it assigns work, and (as discussed) the Grid code is too complex. Over the past few releases, I've been landing code to help resolve this:
The W3C WebDriver spec is now at the stage where we need to demonstrate multiple compatible implementations. Realistically, this means that we need two passes for each test in our test suite. The browser vendors are working hard to get things working, and progress is being made. There's not been a huge amount for me to do here, so this is more of a waiting game than anything else from my perspective. Having said that, I'm on the hook for some sections in Level 2, so I should really sit down and write those (and the matching tests)
The main thing I've been focused on has been the Selenium Grid. There are a couple of things that we really need to solve with Grid. The first is that the code is complex and hard to deal with. When we originally released it, it took a huge amount of work to review the code for thread-safety and to debug many of the issues. That code has not become easier to reason about, which makes it harder to foster Open Source contributions.
Of course, that'd be fine if we didn't care about making any changes, but we do. When Grid came into being, it was normal to have a physical server for each node in the grid. If you were lucky, you might have a massive server with VMWare running on it, which you'd cycle virtual machines on to keep the Grid healthy. The world has changed. Docker is now A Thing, and there are multiple "Selenium as a Service" (SaaS) cloud providers.
There are some projects out there that implement some of the functionality of Grid. For example, selenoid makes use of Docker, but it doesn't use the W3C dialect of the webdriver protocol, which means it doesn't do protocol conversion, and it doesn't natively support cloud providers. Zalenium builds on top of Grid, and provides support for Docker and SaaS, but they've had to work within the existing architecture, and there are obvious rough edges.
Finally, we've wanted the selenium server to be a "Grid of one". If you go into the code of the server, you'll see that there are two fairly separate trees that live side-by-side. When you start the server, it picks one and then goes with it. It'd made things like supporting the W3C protocol harder than it should be, and it's not an elegant way to run things.
As a solution to this, it seems obvious that there should only be one code path. The problem is that the standalone server is too simplistic about how it assigns work, and (as discussed) the Grid code is too complex. Over the past few releases, I've been landing code to help resolve this:
- The pass through mode: this makes the server proxy requests without doing an parsing or changes unless necessary.
- The ActiveSession abstraction has been added. This makes adding new types of provider (SaaS, Docker) far easier to write.
- A "new session pipeline" has been added, and this is being used to handle things like multiple versions of the webdriver protocol.
The most recent thing I've been working on has been a new scheduler. This will be rolled into the new session pipeline, and is composed of a number of pieces:
- The "Scheduler": this is responsible for queuing new session requests, handling retries, and waiting until nodes become available. This class is thread-safe and designed to be the main entry point.
- The "Distributor", which is solely responsible for ranking and ordering available hosts, and the sessions that can run on them.
- The "Host" abstraction, which represents a physical place where sessions can be run. Each of these has a number of....
- The "Session Factory", which is responsible for creating a new session.
The scheduler will sit within the new session pipeline. For the standalone server, we just add a single host. For the grid, we can add an arbitrary number of hosts (and therefore session factories)
As well as the new scheduler, we're preparing the 3.9 release. It should be out next week, if everything goes according to plan. :)
