BDD Clarified: BDD ≠ “Given-When-Then” (Gherkin)

Cucumber is the first Gherkin BDD framework, and it gained popularity gradually. Cucumber is written in Ruby, naturally, its clones were created, such as SpecFlow and JBehave, in other languages. However, many people think Cucumber (and its clones) is equivalent to BDD, and that is wrong!

RSpec is the first BDD framework

First of all, Cucumber (or Gherkin) is not the first BDD framework, developed in Ruby. In Ruby’s world, the RSpec BDD framework was released nearly two years earlier than Cucumber.

RSpec v1.0 was released on 2009–07–25.

Cucumber v1.0 was released on 2011–06–20.

Putting aside the argument on syntax differences (which I will explain later), without a doubt, RSpec is the first BDD framework. Please note, I am comparing apples to apples here, both frameworks were released on RubyGems.

RSpec is still very active, in fact, its downloads far exceed Cucumber.

Cucumber 7.1 download count: 823,630

RSpec 3.10 download count: 42,526,424

The reason that RSpec downloads are so much higher: RSpec can be used in unit, API and functional testing.

Non-Ruby programmers were not aware of RSpec. When Cucumber clones were available on Java and C#, they mistakenly thought the syntax = BDD, which now you know, is wrong. This is like, many years ago, many people thought Internet Explorer was the first web browser and how a web browser should be, but the reality is: Netscape started web browsing.

What is BDD?

Behaviour-Driven Development. We can’t talk about BDD without mentioning Test-Driven Development (TDD). In fact, BDD was inspired by TDD to apply to functional testing. There was also a short-lived intermediate term: ATDD, Acceptance Test-Driven Development.

In TDD,

• Programmers write automated unit tests,
• and write the test first; let it fail
• then implement the code to pass the test.
• If passed, must run all unit tests immediately.

For more, please read my other article: Unit Testing Clarified.

BDD is actually quite different.

• Software Engineers/ Test Automation Engineers write automated functional test scripts
• Business Analysts/Customers may run (or even edit) automated functional tests
• mostly don’t create the tests beforehand
• preferably, but rarely achieved, run the whole functional tests daily as automated regression testing, typically in a Continuous Testing (not CI) server.

As you can see, The Whos, Whats and Hows are all different. The ‘driven’ in BDD, in practice, has no meaning in the context of automated testing. People might argue that the cucumber features, that Business analysts wrote in Jira, drive the development, i.e. programmers coded based on that. But that’s a different context (requirement-driven), we could say that in software projects using Waterfall methodology three decades ago as well.

In TDD, the unit tests serve two purposes for code:

• as the requirement (upfront)
• as the immediate verification

The concepts in BDD are the same for user stories. However, in reality, many people talk about BDD without enforcing the verification part.

Why do so many people talk about BDD, without seeing automation?

Most IT people know TDD is about execution of automated tests. But BDD in many software projects means:

• a term frequently mentioned but not visible result.
• ‘Given-When-Then’ texts in Jira’s user stories

Of course, it is wrong. BDD ≠ (User Stories are written in a certain syntax). If lacking the meaning of automation and verification, there is no BDD. Frankly, purely for requirement, I don’t see the Given-When-Thensyntax’s advantage over As …, X can syntax introduced in the classic “User Stories Applied: For Agile Software Development” book.

Why can’t we use Gherkin BDD for test automation? The answer is simple, few test engineers are capable to accomplish that.

Let me illustrate with an example. Below is what a BA wrote in Jira for the login user story.

After being assigned, a programmer started to work on that. During the development (for this user story), he tested it (manually) a few times. After being happy about it, he drags the user story to the “To be tested” column in Jira.

A tester might have planned the test design based on the content on Jira. On most projects, they would just test it manually.

Why won’t they create a Cucumber test for it? In theory, they could. In reality, most engineers don’t have the capability. Please read the quotes from the creator of Cucumber.

Why is it hard? When used for test automation, the English text syntax does not come free. Test engineers need to create a parser for it, that’s still relatively easy. The main challenge is maintaining that. Over the last 10 years, every test automation attempts with Gherkin that I witnessed failed badly. That’s the price for not listening to the creator of Cucumber and DHH. For more, check out my other article: Why Gherkin (Cucumber, SpecFlow,…) Always Failed with UI Test Automation?

BDD with RSpec makes sense

We have seen a sample cucumber feature. How does it look in RSpec?

Don’t you think this is also a good (yet simple) way to describe business behaviours? So, BDD ≠ Gherkin. Different from Gherkin, developing/maintaining automated E2E tests in RSpec is a lot easier.

A full (and executable) test script for the above login user story in RSpec:

It is quite readable, isn’t it? Moreover, the test script is in Ruby language, which means many goodies:

• syntax validation
• syntax-highlighting
• auto-reformatting (pretty-printing)
• top flexibility (can’t beat programming)
• good support in testing tools, such as snippets, auto-complete, script navigation, functional test refactorings, …, etc
• dynamic test data generation, such as send_keys(Faker::Internet.email)
• use of utility libraries, such as date manipulation Date.today.next_week and decoding BASE64 strings
• highly maintainable if well designed, such as popular Page Object pattern (see Maintainable Automated Test Design)
• easy to integrate, such as calling API tests
• easy to run in CI servers such as Jenkins, and CT servers such as BuildWise
• …

To wrap it up, ditch Gherkin and use RSpec to do real BDD. If you have to use Gherkin, I suggest doing both RSpec and Cucumber. By starting writing tests in RSpec first, you can maintain them easily before the app’s UI is established. When the time is right, write another set of Cucumbers tests, reusing the helper and page classes created previously (in RSpec), which accounts for the majority of the work.

---

This article was originally published on my Medium blog on 2022-04-19.