How Outsourced Testing can help Your Company

outsourced testing

I was having an issue with my remote web driver not being able to create a new remote session. It would just throw a SessionNotCreated exception.

I started off just googling for a solution like I do for most issues I run into, only to find out a lot of people get this error for a wide range of issues, I was not able to really find anything that related directly to what looked to be a solution to my issue, so that is what leads me to write this post, in hope to help anyone else that run into this issue when they get to upgrading to Selenium 3.

Long story short, the lowest version of firefox that the selenium 3 remote web driver supports is v48 so in your DesiredCapabilities you need to set your version to 48+.

DesiredCapabilities capability = DesiredCapabilities.firefox();
capability.setCapability(“platform”,"firefox");
capability.setCapability(“version”,"48");

theBrowser = new RemoteWebDriver(new URL(“your URL”), capability);

Yeah there are a lot of other reasons you can get this exception but I only got this issue just after upgrading to Selenium 3 and if you also have this issus right after your upgrade this might help you.

Another thing that I had to add that I have not found a solution to yet is the fact that I was getting an error about firefox's plugin container when I was calling the the .quit() on the browser. I tried a few things I found online but killing the task via taskkill ended up working for now.

Runtime.getRuntime().exec("taskkill /F /IM plugin-container.exe");

Share Open Browser with selenium webDriver

Almost everything you read online about web automation in general is always going to tell you that the recommended best practice for automation testing is to start each test in a new browser so one test does not rely on another. On paper that seems to make since, in practice, not so much.

Yes for most cases having a new browser is best but in same cases it's just not realistic.

Example, so I have been working on automation of this web application using selenium webDriver with TestNG, the test flow is somewhat simple:
  • 1. Login
  • 2. Select an option from a drop down
  • 3. Select search from menu options
  • 4. Enter criteria for search
  • 5. Click to open one of the results returned
  • 6. Perform action XYZ on this page.

Now in step 6 this is where the reason to reuse a browser starts to make sense because their are about 100 or so test cases for just this single page, all of the other test cases have the same steps 1-5, step 6 is the only difference.

Just to sum up the test cases for this page lest just say there are about 25 features on this page and each have the ability to view, edit, remove and add line items to each feature, so yes some test cases fully rely on the other one to work to even move on, I mean no reason to remove a line item if adding it did not work in the first place, and so forth.

The time it takes for the user to complete steps 1-5 is around a full minutes, where step 6 for the most part only takes about 5 seconds on average, so if I was not to share the browser for any of the 100 test would take around 2 hours to run but sharing the browser's the test only takes about 10 min, that's a drastic savings in time, yeah the 2 hour run could be ran in parallel across more than one machine using the selenium grid but that just introduces other potential problems and really you never going to get it down to the 10 minutes it takes to run the other way, you might be able to get it to run in 30 min a best.

Then on top of that there are 20 other sets of this test case and these 20 select different options in step 2 that lead them to another search but have the same about 100 test cases to run on step 6, so this is the part that I would run in parallel with the remote web driver, now the time savings becomes extravagant, even if we were able to get the test run down to 30 min without sharing the browser the total test time for these test cases would be around 10 hours of run time, but sharing the browser's and running the 20 test cases in parallel (assuming you can run only 10 at a time in each case) were only looking at about 20 min to complete all 2000 test cases.

When this is only a small fraction of the total test cases for the application this time saving allows a lot better response time in results, because there are about 5 times this amount of test cases that get ran, when we were not sharing the browser for these test cases it would take 2 days to really get results back on if a build was even good to continue testing, but now we have our answer in about 6 hours, and running these test overnight with saucelabs and Jenkins, let's us know each morning if the build from yesterday is OK to continue with, where before we would have to wait till the following day to have all the test completed, and sometimes we were not able to even run all the test cases on a final build due to the release timeline.

In an upcoming article I will write about the browser part of the selenium framework I developed so help describe the approach I took to setting up test cases to share a browser.