Deconfuse E2E Test Automation for IT Executives

This is an abridged version of the article on my Medium blog (2024-09-12)

Let’s start with a short video of Steve Jobs, the best IT Executive ever.

“as where the enterprise market, it’s not so simple. The people that use the products don’t decide for themselves, and the people that make those decisions sometimes are confused. (audience laughed and clapped)” — Steve Jobs

Being confused about E2E test automation and Continuous Testing is OK, but it does not have to make wrong decisions. In fact, if following common sense, making the right calls is almost a certainty. This article explains, for IT executives and senior IT managers.

1. Your Technical Advisors are confused about E2E Test Automation, too.

The Fact: To my knowledge, there is no university offering a proper E2E Testing course, yet it is a key activity in modern software development.

Therefore, your technical advisors (in various titles, such as Chief/principal software engineers, Technical architects, Technical Managers, ….), don’t really know E2E Test Automation. In fact, they never witnessed a single successful implementation.

The audience of this article is IT executives. I will use simple terms that you would understand or use examples. What a successful E2E Test Automation means at Facebook:

“Facebook is released twice a day, and keeping up this pace is at the heart of our culture. With this release pace, automated testing with Selenium is crucial to making sure everything works before being released.” — DAMIEN SERENI, Engineering Director at Facebook, at Selenium 2013 conference.

And Linkedin:

Don't be amazed by the above. It shall have been the case for every Agile project. Here is a showcase of mine, a Micro-ISV.

In other words, if your company don’t achieve “weekly or fortnightly production deployment”, enabled by automated End-to-End (via UI) regression testing, the so-called “Agile”, “Scrum” or “DevOps” is completely fake.

2. Your Technical Advisors don’t want to admit that they knew nothing about E2E Test Automation.

Those technical advisors, in theory, understand the importance of E2E Test automation.

However, most of these senior technical advisors know their limits (in E2E test automation and Agile) but are unwilling to admit that. (For obvious reasons, in particular, they thought testing was easy. Very Wrong! The fact, if it was easy, all your teams would be doing automated UI regression testing already.)

3. Use your common sense to avoid bad advice

It is actually quite easy to tell the wrong E2E Test Automation advice, just using your common sense.

“E2E Test Automation is too much effort, we should avoid them” ❌

The value of E2E Functional Testing is needless to say, and it is the only practical way to achieve “Release Early, Release Often” is Automated E2E (iva UI) Regression Testing.

According to World Quality Report 2018–19, “The first time ever that `end-user satisfaction” is the top objective of quality assurance and software testing strategy”.

“We shall use the coding language, e.g. JS, for E2E Test Scripts.” ❌

Wrong. End-to-end means 100% independent from the code. A wise manager should know mixing two distinct activities is probably a bad idea, anyway. I have developed a number of successful Web Test Automation using Selenium + RSpec (Ruby) for apps written in Java, C#, Ruby, PHP and JavaScript.

“E2E Test Automation is too expensive, we shall try Cypress Component Testing.” ❌

Wrong. The so-called Cypress Component Testing violates the fundamental raw of software testing: Black-box and White-box. It is a recipe for conflicts. BTW, Cypress.io is dying.

Solution: Simple

Unlike software design or infrastructure setup, a wise IT Executive can confidently avoid mistakes from bad suggestions related to E2E test automation. Why? E2E means

• “End-User”

Can everyone in the team see the test execution? Can an end-user understand (and execute) automated e2e test scripts?

• “End-Results”

whether a big improvement on “release early; release often”, “a big decrease in defects”, or “a huge increase in development productivity”?

which you both understand well.

The idea of E2E (via UI) Test Automation as regression testing is easy to understand and verify. You just need to be aware of the sabotage from your middle management and “technical advisors”.

There are basically two simple and logical approaches:

1. Search for internal talents

Check out “Test Automation and Continuous Testing Competition Week”.

Important note: You must be personally involved, not via your “technical advisors”. This is to make sure it is 100% judged by results.

2. Seek external help

If your company lack the ability, then seek professional help. Be aware, there are many (in fact, most are) fake test automation consultants or coaches. How to tell? Easy, use the same criteria used in “Test Automation and Continuous Testing Competition Week”.

E2E Test automation, at least for the web, has not changed over one decade. One real test automation engineer implemented one once, repeating success (excluding political factors) is almost certain. Moreover, it can verified by hours or days. The rule: See how they do, not what they say.