My Test Automation Journey. Part 1: Pre-WebDriver

This article is a reflection of my 18-year journey in test automation since 2005. Prior to that, I had worked mostly as a Java Programmer (contractor) for 8 years.

• (2005–2006) JWebUnit (HTMLUnit) using Intellij IDEA
• (2006–2011) Watir with RSpec
• (2011–2012) RWebSpec on top of Watir and Selenium WebDriver
• (2011–2013) Load Testing with LoadWise

In 2010, I switched to daywork from programming to test automation and Continuous Testing. My love for programming was not changed, just do it differently: coding for my own products in my spare time. Thanks to my CT process, my productivity in software development is an order of magnitude better. Please read “Reflections of Software I Created over the Last 14 Years in My Spare Time”.

• (2012-present) raw Selenium WebDriver with RSpec
• (2012-present) API/WebServices Testing with Ruby
• (2019-present) Appium with WinAppDriver for testing Win desktop apps
• (2020-present) Performance/Load testing using browser-based Selenium test to run in BuildWise CT server

1. JWebUnit on top of HTMLUnit (2005–2006)

At that time, though most IT people heard of automated functional testing, it was considered a luxury as the testing tools such as Mercury Interactive’s QTP (yes, later acquired by HP) were super expensive.

In 2005, I was extremely fortunate to have a chance to pair program with a world-class programmer (Martin Fowler often mentioned him) for 6 weeks. He showed me TDD (which I was aware of) and wrote automated functional tests in Java, which was a new concept to me. I liked it instantly.

The framework we used was JWebUnit, a syntax layer on top of HtmlUnit.

Realization:

1. Automated functional testing greatly enhances programmers’ productivity
2. Automated functional testing increases job satisfaction and is fun
3. Automated functional testing helps software design skills because I need to think more from different perspectives

Sample test (JWebUnit):

2. Watir (2006–2011)

Once I ran a JWebUnit test in front of a manual tester. I pointed to the green tick in IntelliJ IDEA and said: “See, it passed”. This poor tester was confused, “How?”. I scrolled down and showed him the test output. He was half-convinced, “I want to see the test execution”. Hence, I made one of the biggest realizations of my IT career: “Seeing is believing applies to automated test execution”.

Fortunately, I found Watir and did exactly what I needed to do. Furthermore, after forcing myself to learn Ruby, I fell in love with this beautiful language quickly.

After selecting Watir as the automation framework, I naturally looked for a JUnit equivalent in Ruby as a veteran Java programmer, Test::Unit . It worked. Shortly after, I found RSpec, a BDD framework with more human-readable syntax, while still 100% Ruby (this is very different from Gherkin frameworks). I liked RSpec. In fact, I have been using RSpec mostly ever since.

Realization:

1. Automated test execution needs to be visible
2. Automated test scripts shall be in a scripting language
3. Need to think outside of the programmer’s perspective
4. Java IDE such as IntelliJ IDEA is too complex for non-programmers to use
5. Ruby is a great scripting language for writing automated tests.
6. Need to run all tests in CI server with auto-retry features (will be covered in ‘My Continuous Testing Journey’)

Sample test (Watir):

Tool:

• I created iTest2, a testing IDE that supports developing/debugging Watir tests. One of its unique features is “Functional Test Refactoring”.

Publications:

• “Refactoring Automated Functional Test Scripts with iTest2” Article on InfoQ (2009–07–22). This was my first published article, which greatly boosted my confidence in writing.
• Practical Web Test Automation (2009). My first book, and my favourite one, too. I spent countless hours on it with many revisions. To me, writing English was much harder than coding and writing test scripts.

Acknowledgment:

• iTest2 was demonstrated at AA-FTT Meeting at Agile 2009, with good feedback.
• iTest2 was listed as the first testing tool in the presentation “Trends in Agile Testing” (2009) by Lisa Crispin, co-author of the Agile Testing book.
• iTest2 was listed in Ward Cunningham (co-author of Agile Manifesto)’s Functional Testing Tools list.

Speaker at software testing conferences:

• ANZTB 2010, Melbourne
• Tianjin Software Testing Conference (TIST) 2010, China
• Tianjin Software Testing Conference (TIST) 2011, China

3. RWebSpec on Watir and Selenium WebDriver (2011–2012)

Selenium WebDriver, a.k.a. Selenium 2, was released in 2011. This was big news for me. While I have been using Watir testing IE6 fine at that time, I would like to use Mozilla Firefox which was my preferred browser for its speed. With Selenium 2, finally, I could run automated tests in Firefox.

Selenium 2 comes with Firefox support, and there was no separate driver required then. Selenium with IEDriver was unreliable though. Also, the idea of cross-browser testing became popular. I created RWebSpec which will use Watir for IE and Selenium 2 for Firefox, in the same test script.

Sample test (RWebSpec):

Other testing tools tried/reviewed:

• Selenium WebDriver in Java, using Intellij IDEA

Publications:

• Watir Recipes (2012)

4. Load Testing with LoadWise (2011–2013)

I have used JMeter for Load Testing before. It worked but was quite complicated. Therefore, I created my own performance/load testing tool: LoadWise. It did attract some commercial interest.

However, LoadWise, using the traditional HTTP-request simulations approach, does not support AJAX and more web apps were adopting AJAX. I deprecated LoadWise in 2013.

--

Soon I found one right solution and have been using it ever since, see Part 2.

--

This article was originally published on my Medium blog on 2021-03-29.