Story: “What is the Most Challenging in E2E Test Automation?”, Part 1

This is one of the Stories series.

The Story

In 2016, I worked in a large tech company (over 500 IT staff, considered a large one in my city) as a test automation engineer (contractor). One day, the newly-joined testing director, S, who is responsible for the overall testing process in the company, invited all software testers (~70) to a meeting.

In the meeting, after the usual introduction (his and then self-intro from everyone), S asked what test automation technologies were used in different teams. There were many answers: Micro Focus QTP, Selenium WebDriver (Java, Ruby, C#, JavaScript, Python), SoapUI, Watir, PhanomJS, Ranorex, WebDriverIO, ProtractorJS, and self-created ‘test automation framework’s.

S and everyone (including me) were shocked by the long list. Apparently, there was a mess and lacked no planning and direction. Also, from the testers’ expression, it seemed test automation attempts had failed in all teams, except mine. (Check out the Definition of End-to-End Test Automation Success)

S wanted to soften the atmosphere (I think). He turned and wrote this question on the whiteboard, “What is the Most challenging in E2E Test Automation?”

S encouraged everyone to come up and write one on the whiteboard. So, many did.

• Lack of Training
• Not enough time
• The test framework/tool is not reliable
• The application does not provide IDs for every element
• Test Data Issues
• Not run well in CI servers, such as Jenkins
• Git branch policies
• …

Soon, the whiteboard was almost full, and at least 20 of them were there.

Before the meeting, I told my mentee (who recently turned from a manual tester to an automated tester) that we would be silent if possible. The reason: my recent rescue of a failed test automation had already attracted some attention. (For more, check out Case Study: Rescue Unreliable 20 hours of Automated Regression Testing in Jenkins ⇒ 6-Minute Highly-Reliable in BuildWise CT Server).

From my experience, nothing can be achieved in this kind of big gathering. I did not want to get involved in yet another test automation steering committee, which was usually just useless talking and debating. I prefer to remain low-profile and do hands-on real test automation.

The whiteboard was almost full. S asked, “Any more?”, and looked at me. Apparently, he has heard about me (I authored a few test automation books) or my work. I could no longer hide, so I got up and wrote “Test Maintenance” at the bottom right of the whiteboard.

Then, S asked everyone to vote on the issue by adding a tick on the issue they concurred with.

The №1 (with the most ticks) issue: “Test Maintenance” (by me).

This is an interesting finding, for most testers in the room. Please note that this was the last written item on the whiteboard. Of course, during the whole event, there were conversations among all the testers. But all (except my mentee and me) focused on the less-important issues. Please note, my mentee, who just learned test automation for a few weeks, realized the importantance of "test maintenance" because that's the majority of her work already.

By the way, the №2 issue was“Lack of Training”, I will write a separate article on this.

S sensed the awkwardness and quickly concluded the meeting by saying, “This is a good meeting. We will take the feedback into action in the coming months.” Of course, there were no follow-on actions (until I left the company a few months later).

---

In Part 2, I will explain the reasons behind this finding, also, there was a follow-up story.

The original article was published on my Medium blog, 2023-01-23