One Simple Test Automation Scenario Interview Question that Most Candidates Failed

This article is the most liked and popular article on my Medium blog, and one of the “IT Job Interview” series.

During the interview for an automated tester role, after the four screening questions, I usually ask one scenario question (otherwise, too short, not respecting candidates). Personally, I don’t like long, brain-teasers or tricky questions. So, this only test automation knowledge question must be related to real work.

The Question

Describe the steps you would write for a common E2E web automated test case via UI: “User Change Password”.

Everyone understands this simple scenario, the explanation for business logic is unnecessary. It is so common that this would be among the first batch of automated tests written for a web app. To assure the candidate, I will tell the candidate: “There is no trick here, and no functional limitations (ignore any password change restrictions). This is simply about test design. Just say or write down the steps, such as a user logs in, click profile and ‘change password’ link, …, etc.”

Before the candidate started, I said: “A reminder, this is an automated End-to-End test via UI (functional testing). Don’t overthink. Just focus on designing a good automated test that verifies that the user can change password.”

The Results: Most Got it Wrong!

I am not belittling those candidates or many automated testers who were struggling. This is the failure of our IT education on test automation: Universities do not teach; Companies do not provide training. Motivated engineers can learn, though. In fact, I hired some (they all failed on this question), and they learned well under my coaching: can work on real tests in a matter of days.

It is a quick and easy interview question, right? However, not a single interview candidate answered it correctly (regardless of the candidates’ current job titles, such as Senior Test Automation Engineer or Test Automation Architect). Also, I asked this question in my training session and in some conference talks. I’ve only gotten the correct answer once (yelled from the audience at STARWEST 2013 in California; of course, there might be some real test automation engineers, but many seem to prefer to remain silent).

One large IT company adopted this question for interviewing test automation engineers (which they call 'Software Engineer in Test') for a short while, but quickly dropped it. The testing manager told me, "Zhimin, this interview question is too hard. No single candidate can answer it correctly. It made the fake automated testers obvious that I couldn't hire one for two months. Now I got pressure from the director, so I have to drop this question for the interview."

Some readers wonder, how there could be such a high percentage of wrong answers. Is there a trick? NO, definitely not.

Let’s see a typical answer from the interview candidates:

1. User logs in
2. Click the ‘Change Password’ link
3. Enter the current password
4. Enter a new password and confirmation
5. Click the ‘Change Password’ button
6. Log out
7. Login in with the new password
8. Confirm the logged-in successfully

Often, candidates expressed the step in detail (fear there were some tricks) or validations such as confirmed-password-matching. Then I would assure them, don’t worry about validations, alternative paths or functional limits. Just focus on a completed automated test design for the main flow. The more concise (being correct), the better.

Some even stopped at Step 5. But most could get Step 8.

Then I would ask: “Is that all?”

Usually, the candidate would show signs of fear of missing something. Then I will assure them again: “There is no trick, just write down or speak out how you normally test this common scenario?”

However, in dozens of interviews I conducted, no single candidate could work out the missing step.

What's the missing step? Check out the answer (in fact, there are several of them) and explanation in the Medium article.

--

Related reading:

• My eBook: “Practical Web Test Automation with Selenium WebDriver”