My Test Automation Journey. Part 2: WebDriver

Continue from Part 1.

5. Selenium WebDriver (2012–Present)

With Selenium WebDriver becoming more reliable with IE (and Chrome dominating the browser market), I deprecated RWebSpec to use raw Selenium WebDriver + RSpec. I was quite pleased after converting all existing RWebSpec tests to raw Selenium. The scripts are actually more intuitive.

Over the last 8 years, I used raw Selenium + RSpec to test my own apps and help others.

Sample test (Selenium WebDriver + RSpec):

Tool:

• Renamed iTest2 to TestWise, and added support for Selenium WebDriver

Realization

• Test syntax framework shall be in 100% scripting language like RSpec, while Cucumber is not.
• Presenting at international software testing conferences, in my opinion, is not effective in terms of influence. I met a number of attendants who liked my presentation, but few of them dared to try test automation at work (lack of confidence). So I decided to spend more time on writing books that come with good exercises, creating video demos, etc. rather than speaking at conferences.
• I worked on many projects as a contractor/consultant. I didn’t care about job titles, just wanted to work on different projects for testing challenges. After a few years, I was able to answer and solve other people’s testing problems quickly, usually in under a minute.
• While test automation was being adopted by more projects, every single project I visited failed to adopt the automation properly. Some large companies failed a few attempts.
• With my increased efficiency in test automation, my productivity as a programmer also grows significantly. Since 2012, I have created ClinicWise, SiteWise, TestWise v4, SupportWise, BuildWise, and WhenWise apps, in my spare time.

Other testing tools (tried/reviewed):

• UFT, Coded UI Test, Ranorex, Sahi Pro, Cypress. (none of them was good)

Speaker at software testing conferences:

• StarWest 2013, Los Angeles, USA

Publications:

• Selenium WebDriver Recipes in Ruby (2013)
• Selenium WebDriver Recipes in Java (2014)
• Learn Ruby Programming by Examples (2014)
• Selenium WebDriver Recipes in Python (2015)
• Selenium WebDriver Recipes in C# (2015)
• Selenium WebDriver Recipes in Node.js (2016)
• Practical Continuous Testing — make Agile/DevOps real (2021)

6. API/WebServices Testing with Ruby

While I focused mostly on web test automation, quite often, I was required to perform API/WebServices testing. As a veteran programmer, writing API test scripts is quite easy for me. However, after I worked on web test automation with many manual testers, I learned to look beyond the correct API tests: easy-to-understand (for non-programmers) and easy-to-maintain.

A common mistake with API Testing is to use tools such as SoapUI/Postman. Every project I knew of that uses these kinds of tools failed for the reasons below:

• Not flexible (fact: programming provides the ultimate flexibility)
• Hard to maintain
• Inefficient. I know some might find it hard to believe that using a good scripting language such as Ruby can be far more efficient than GUI tools.
• Difficult (or impossible) to run the tests in a CT server. Many projects did not even realize how to manage executions of a large suite. Therefore, tests went out of date quickly. As time went on, it was getting harder and harder to maintain. Eventually, there was only one outcome: the tools were dumped.

My approach with API (Soap/Rest/Microservices/GraphQL) has always been the same (for over 12 years): raw Ruby with RSpec and execute them in BuildWise.

At one project, I completed an assigned 2-month’s API testing (SoapUI) workload under one day (including initial start time), using raw Ruby.

With quick training, most manual testers can work on the scripts following my examples. For more, please read my book (listed below).

Other testing tools (tried/reviewed):

• SoapUI, ReadyAPI, Postman (none of them was good)

Publications:

• API Testing Recipes in Ruby (2016)

7. Appium with WinAppDriver for testing Win apps (2019-present)

I created a small framework RFormSpec on top of AutoIT3 for automating windows native apps. AutoIT3 has some limitations, it worked well for some scenarios. I developed a dozen of RFormSpec tests for TestWise, my desktop app. However, it wasn’t easy.

In 2019, I saw a recommendation from Microsoft: “Selenium for web apps, Appium for desktop apps”.

After quick work on proof of concept, I liked it. Therefore, I switched the automated tests of TestWise (my desktop app) to Appium + Desktop. To see a test execution of Appium+WinAppDriver, check out this article “Demo of Automated Testing for Windows Desktop App: TestWise tests TestWise”.

As of 2021–03–26, the regression suite of TestWise has 276 automated tests running in the BuildWise CT server. This reaches Level 3 of AgileWay Continuous Testing Grading.

This CT process enables me to release TestWise daily if required.

Sample test (Appium + RSpec):

8. Continuous Performance/Load testing using browser-based Selenium testa to run in BuildWise CT server (2020-present)

Load testing is moving from Protocol-Simulation to Browser-based as the modern web apps are dynamic and used AJAX. Since last year, I have combined my past experience of developing LoadWise with Web Test Automation and Continuous Testing and come up with a new way to conduct load testing.

1. An Engineer develops a load test script as a functional UI test first, in raw Selenium WebDriver +RSpec. The engineer can run the testing tool such as TestWise, and view (or from load testing perspective, preview) test execution to ensure correctness.
2. Add benchmark (timings) of operations
3. Make the test script load-testing aware of virtual users
4. Run a set of load test scripts in the BuildWise CT server which will distribute tests to run them in multiple build agents in parallel.
5. The test reports (and history) are viewable on BuildWise’s web interface.

Below is a report of the ‘User Login’ load test with 5 virtual users on BuildWise.

Currently, I am working on a new book: “Practical Performance and Load Testing”.

--

This article was originally published on my Medium blog on 2021-03-29.