Benefits of E2E Test Automation & Continuous Testing (Part 4: to Developers)

E2E Test Automation & Continuous Testing (enables software teams to push software updates to production daily, not fake CI/CD talks. Check out “Continuous Integration at Facebook” and AgileWay Continuous Testing Grading) is the heart of the software development process. It benefits all stakeholders from a software project.

• Executives
• Managers
• Business Analysts
• Developers
• Testers (automated & manual)
• Customers

Developers spend 50+% of their time doing manual testing, anyway

You might have heard about the joke above before, of course, it went too far. But at least give you a perspective that programmers do spend a lot of time fixing wrong code. As we know, testing is a part of bug fixing.

If I tell you that programmers who work in an agile team, typically spend over 50% of their time on functional testing applications, you will probably be surprised. As a matter of fact, as a professional programmer for over 20 years, I would say 50% is a conservative figure. I have to admit that I only realized this after I mastered real test automation, which helped me to see this fact plainly.

Let’s examine how a programmer implements a new software feature (e.g. user story) for a web app.

1. Design
2. Write/debug code
3. Unit testing (only good programmers can really do this well)
4. Deploy to the local server instance
5. Verify the functions: open browser, log in a user, …, etc.
6. If issues are detected, go back to step 2.

Step 1 (Design) is usually quite light in agile teams. Among the four repetitive steps (2–5): Step 3 & 5 and a part of Step 2 are testing activities, while Step 4 (Deployment to local server) shall be lightning-quick. Therefore, the time spent on it is neglectable.

In the context of web applications, a lot of work involves tweaking CSS and JavaScript, which requires repeated verifications.

Now consider these:

• Quite commonly, it will take a few tries for a programmer to believe that he has got it right. For every try, he is actually conducting manual UI testing.
• Programmer demonstrates to a business analyst (or customer) who will usually find some issues or provide feedback. Back to Step 2, more manual testing will be carried out.
• Another fellow programmer checked in some new code; A customer reported some regression issues. More manual testing will be carried out.
• Requirement changes, as we know, are very common. When a feature is available on the test server, integration server, or especially the UAT server, requests for amendment can come at any stage. When that happens, programmers can get quite frustrated because they have most likely already started working on another user story. Now they need to search the memory for things such as “What username should I use?” when testing the application manually.

There are more scenarios like these. Every showcase, release preparation, and bug fix may involve a certain degree of manual demonstration/check as well.

Along with the development, there are more features that will be added. This means, on the first sprint/iteration, a programmer’s manual test of a user story is usually quick, such as user login. The later tests will be longer and more boring as they often need to go through more steps.

Some programmers might disagree: “I am not a tester, I am NOT performing against written-down test cases, only to check my work.” Have you heard of ‘exploratory testing’ which is opposite to ‘scripted testing’? The fact that you navigate the app to a certain state, choose login credentials such as ‘manager01’ and perform some checks, is that you are doing functional tests (= test steps + test data + checks). Dear programmers, unless you have mastered test automation (which is rare), you do a lot of manual tests every day.

Programmers, don’t feel bad about it. At some top IT companies such as Google, a programmer with testing skills is well respected.

Now I hope you have realized that programmers do spend a lot of time on manual testing (except the top ones who have mastered test automation). If still in doubt, you can read books written by software legends such as Brian Kernighan, Kent Beck, Steve McConnell, Dave Thomas, Gerald Weinberg, and Martin Fowler. While you read, please pay special attention to the testing part. There you will see the testing gene. Then it will be easier for you to see what benefits Continuous Testing can provide to programmers.

Benefits of Test Automation for Developers

1. Much quicker bug-fix cycle

Many programmers, like me, didn’t like to read defect reports in DFS (such as Quality Center or Jira). The reason is simple: the description, the steps, and the test data are often incorrect.

With automated test failures (particularly run in a CT server), developers can view the steps and error screenshots quickly (in a matter of seconds). The programmer can git pull the test and run it on his local machine. Quick and Objective! Moreover, after the programmer made the fix and reran the same failed test to see it pass, the feeling is irreplaceable!

2. Take pride in the work

As a software engineer, I would like to know whether our software is in a near-perfect working state when walking out of the office at the end of the day. Surely, 100+ outstanding defects will take that pride away. Trust me, team morale matters. Have you heard of the “Broken windows theory”?

3. Provide developers with different perspectives to design software

Designing/reviewing automated tests gives programmers a different perspective, which will help them to understand more about the software they are developing.

4. Much more productive

As explained earlier, a developer spends significant time manually testing the app. If possible, executing an automated test will be much faster to navigate the application to the desired state. For example, to work on a story card: “Manager can approve an insurance claim”, I need to create an insurance claim first.

5. Dare to refactor and try new technologies

Tech debts are common in software development. It is not that developers are not aware of them, they lack the confidence to enhance/fix the problems. Naturally, a programmer would think “If I touched it, I might introduce new regression issues. Better be safe, another day another dollar”.

A mature/agile software team should have a safety net so that the staff has a certain freedom to try out new ideas. This is common in other engineering industries. In software engineering, it is test automation (both unit testing and end-to-end testing) that provides this safety net.

I once consulted for a Ruby on Rails project. One controller class has over 7000 lines. This class is the main reason that the server is very slow (though the business paid the expensive Linode plan: $960/month). The developers have been patching the class, which was leading to a monster.

6. Test Data Generation

Test data generation with automation scripts benefits all team members, including programmers.

7. Test Automation is far more challenging and fun!

For most programmers who work on enterprise app development, it could actually be quite mechanical and boring with tasks such as:

• Collecting user data
• Processing user input
• Querying or updating database
• Rendering response on UI

Furthermore, fake Agile coaches will bother you with mostly useless Agile ceremonies (Kent Beck, the father of Agile, call it ‘waste’); and a lot of pointless Update notifications bombard you on MS Teams/Confluence/Jira.

I have been loving to program since High-School (won a prize at the state’s programming competition). My first 4 years of work at a research center were challenging and satisfactory (there I started to develop a good habit of reading books regularly). However, when I started working on EJB (as a Java Contractor), it was very boring. The job after it was Microsoft BizTalk, which was even worse. Software development, in my view at that time, was being done by mindless zombies. Frankly, it was excusable for me to do it ‘another day another dollar’ as I was the sole provider for the family.

Thankfully in 2005, Test Automation (and later Continuous Testing) re-enlightened my IT career. It has been far more challenging and fun since then!

“testing is harder than developing, … if you want to have good testing you need to put your best people in testing.”- Jerry Weinberg

Note, just after a few months of actively practising test automation (at work, nights, and weekends), my programming efficiency excelled quickly. I completed more user stories than all other contractor programmers (~4) combined, with higher quality too. Upon the team’s request, I gave a presentation on Agile/Testing practices for developers, which was well received. A programmer later joined Microsoft and Google in the USA, and he thanked me for my mentoring.

In 2010, I retired as a contract programmer and worked full-time as a Test Automation and Continuous Testing coach. This does not mean I don’t code anymore, far from that. I still love programming. I just program at a different time (nights or weekends) for my own software (see below).

8. Master “Design Simple”

As a programmer, if you think “Design Simple” is easy, don’t speak out loud, as this will reveal that you are not even at a mediocre level yet. Go read some books and do some research online. Excellent programmers would know that a “Simple and effective design” is extremely hard to achieve.

“Do the simplest thing that could possibly work”

- Kent Beck

In 2005, I was fortunate to practise pair programming with a world-class programmer for 6 weeks. I was astonished at how fast and how elegant the design he could come up with. I asked him: “How did you do it? How can I learn?”. His answer: “To design simple. It is even hard for me. To master it, you need to do a solid 2-years test automation”. I followed his advice (after witnessing the unbelievable, it is hard to find a reason not to do it). Hence, this is the main reason that I could develop a list of my own highly-claimed software (see below) in my spare time since 2007.

My Experience

Compared to myself, since I started to practise test automation and continuous testing in 2005, my productivity (as a programmer producing software) is at least an order of magnitude better. Before 2007, I never imagined I would be able to develop my own software. Over the last 12 years, I have created

• TestWise (functional testing IDE)
• BuildWise (continuous testing server and agent)
• ClinicWise (web-based clinic management system)
• SiteWise (content management system)
• WhenWise (services/resources booking app)
• SupportWise (our internal support system)

All development and support of the above were done in my spare time while I worked as an independent software testing and CI consultant. These software are well received by our customers. TestWise was a finalist of the 2010 International Ruby Award, and BuildWise won the 2nd prize of the Ruby Award in 2018.

My secret to successful software development: test automation and continuous testing. Of course, there is more to it. But all of them are based on test automation.

--

The original article was published on my Medium Blog on 2021-05-18.