Story: “What is the Most Challenging in E2E Test Automation?”, Part 2

Continue from Part 1.

This is an Important Realization but often Neglected

“Test Maintenance” is the primary effort of test automation, and shall be the base for any decisions. Sadly, this is often neglected in practice, like in this story. For example, many software companies over-emphasized test creation when choosing a test automation framework/tool. Many managers and tech leads fell in the sales pitches of bad and expensive test automation tools, such as QTP and Ranorex, claiming “How easy to create automated tests using record-n-playback or assistance of its Object Identification GUI Utility”. History proved them wrong. The code-based (and free) approach, such as Selenium WebDriver and Playwright, has dominated test automation since 2011.

Even within the code-based camp, focusing on the presumed readability of test steps is wrong, too. While readability and maintainability might come hand-by-hand, there are still differences. For example, Gherkin tests are hard to maintain due to their interpretation tier. Maintainability shall be №1. For more, check out Maintainable Automated Test Design. Bad examples are:

• Protractor.JS (deprecated)

adding simpler syntax on top of Selenium, but losing its consistency and intuitiveness.

• Cucumber and other Gherkin-styled test syntax frameworks

check out Why Gherkin (Cucumber, SpecFlow,…) Always Failed with UI Test Automation?

Also, since ChatGPT’s release in December 2022, there has been a hype of ChatGPT for test automation. I reviewed ChatGPT (see my article, ChatGPT is Useless for Real Test Automation). It sucks for test automation, not enable to able to create working simple login tests. Anyway, AI Testing lovers might debate, “ChatGPT will get better, with training”. Yes, maybe. Please think of the “Test Maintenance” perspective. Just say, ChatGPT created a reasonably good suite of automated tests (after a long chat with you). Can you maintain them? The fact is, your application changes frequently.

The correct approach is to embrace Maintainable Automated Test Design and Functional Test Refactoring.

When my daughter started her first intern job in a large telecom company, I told her, "You are only Level 2 (of AgileWay Continuous Testing Grading), not considered good in test automation, but you would be the best test automation engineer there”. She, a fresh graduate of Uni, did not believe it. It turned out to be true (based on the feedback, check out this article, An IT Graduate’s Frustration with a Fake ‘Senior Test Automation Engineer’). The reason: I have trained her to do test automation from the maintenance perspective and practice running a whole automated test suite daily in a Continuous Testing server.

In my one-day web test automation with selenium training, I devote ~50% to test maintenance (check out the training info pack). Yes, the first training for total beginners. By the way, learning (not mastering) the core Selenium WebDriver only takes hours, because Selenium is the easiest-to-learn test automation framework.

The Story of S Continues (in a few year's time)

Several years later, S (working in a different organization, still as a manager but seems-lower position) contacted me for help via LinkedIn. He inherited a bunch of fragile Selenium Java tests (running in Jenkins). The “Test Maintenance” issue again.

I provided candid advice, and S agreed with me. However, it seemed that, again, took no real actions, such as:

• stabilize automated tests
• refactor test scripts to be maintainable.
• Get tests running reliably in Jenkins, or switch to a proper CT server.

S knew I could help him turn things around in a matter of days by engaging me as a hands-on test automation consultant and coach. But he did not do that, instead chatting with me occasionally. From what I could tell, there was no progress made. Agreeing with my advice, besides seeking comfort, he did nothing. Why did he not engage in my coaching service? I don’t know and don’t care. One thing is for sure is not about money. S knows how quickly I could get things done, and my rate (publicly listed on WhenWise) is reasonable. There shall not be any problems for his position in a quite wealthy organization.

The conversation between S and me often ended with specific scenarios, such as below:

Without seeing the AUT (application under test) and hands-on scripting (& CT), my help would be quite limited. After all, it is the automated test scripts, high-quality ones, that make test automation work.

---

The original article was published on my Medium blog, 2023-01-23