Cucumbers Have Layers: A Love Story (AATC 2017)

@geeksam
CUCUMBERS HAVE LAYERS
Sam Livingston-Gray
💚
1
A Love Story
With that out of the way, I'd like to get a quick [ADVANCE] show of hands.

@geeksam
AUDIENCE POLL
How many people in this audience...
Have used Cucumber (on any project)?
Have used Cucumber more than once?
Would use Cucumber again?
Any ﬁrst-timers?
❓
2
[REVEAL EACH]
- Who here has used Cucumber on any project, big or small?

- Who's used Cucumber on more than one project?

- And regardless of how many times you've used it before, how many would use it again?

- Finally, do we have anybody here who's new to Cucumber?

[IF SO:] Welcome! I'll try not to leave you behind, but if you need me to clear anything up, please don't hesitate to ask me after the talk.

@geeksam3
For those of you who haven't used Cucumber, you should go get The Cucumber Book before you start. It'll give you a much better introduction than I possibly could,
even if this were a full 45 minutes of "Cucumber 101."

But, just so you're not completely lost...

@geeksam
CUCUMBER 101
❓
4
Cucumber is often referred to as a tool for writing automated acceptance tests. What I prefer to say is that Cucumber lets you describe the behavior of your software in a
domain-speciﬁc language called [ADVANCE] Gherkin.

@geeksam
CUCUMBER 101
GHERKIN
❓
5
Each separate Gherkin ﬁle is called a feature, and here's a feature that I pulled straight from the Cucumber website.

@geeksam
Feature: Addition
In order to avoid silly mistakes
As a math idiot
I want to be told the sum of two numbers
Scenario: Add two numbers
Given I have entered 50 into the calculator
And I have entered 70 into the calculator
When I press add
Then the result should be 120 on the screen
6
A feature has one or more scenarios, and a scenario has one or more steps: those are the given/when/then that you see toward the bottom left of the screen.

Aside from a few keywords, which I've highlighted here in green, everything else is written in whatever natural language works for you. Gherkin's grammar is extremely
simple: everything from a keyword to the end of the line is basically treated as a single token by the Gherkin parser.

This is quite useful just as documentation, but Cucumber also lets you use these feature files to automate tests, which is why Cucumber folks tend to talk about
"executable specifications".

To go from human-readable documents to running automated tests, you have to write a bunch of [ADVANCE] step definitions.

@geeksam
CUCUMBER 101
STEP DEFINITIONS
❓
7
Given /I have entered (.*) into the calculator/ do |n|
@calculator = Calculator.new
calculator.push(n)
end
This is what a step deﬁnition looks like in Ruby; your local implementation may vary.

This boils down to a [REVEAL] regular expression that gets associated with [REVEAL] a chunk of code. This is how you translate from human-friendly blobs of text to
something that Ruby can execute.

When Cucumber wants to run a step, it tests it against every one of the regular expressions you've given it. When it ﬁnds an expression that matches, it executes the
corresponding code. There's more, but that's all you need to know to follow along with this talk.

@geeksam
Gherkin
Features
Scenarios
Steps 
Cucumber
Parses Gherkin
Steps → Step deﬁnitions
Runs each Scenario 
step by step
CUCUMBER 101
❓
8
In my mind, Gherkin and Cucumber are almost, but not quite, two separate things.

[REVEAL EACH]
- Gherkin gives you a human-friendly way to describe software,

- and Cucumber interprets your Gherkin ﬁles and uses them as a script for automating tests.

So that's the Cucumber crash course. Let's dig into Gherkin a little bit more.

@geeksam
GHERKIN IS: 
AWESOME
😍
9
[PAUSE BRIEFLY] Gherkin is a domain-speciﬁc language where the domain is "talking to other humans about software." It's very free-form, so it lets you talk about your
application using whatever natural language makes sense to you and your team.

Gherkin has just enough structure that Cucumber can use it to drive a lot of machinery for automating tests. But...

@geeksam
GHERKIN IS NOT: 
CODE
💚
10
Gherkin is NOT a programming language! This is a critically important point that can be easy to overlook when you're starting out with Cucumber. Programming
languages are great, but they require us to get a bunch of details right all at once, and getting into that headspace tends to shift our focus onto how to do a thing.

Gherkin, on the other hand, exists to help us think about what thing to do, why we're doing it, and who we're doing it for.

@geeksam
CUCUMBER IS NOT:
TDD
💚
11
I think it's also important to realize that Cucumber is not a tool for doing test-driven development. Cucumber and TDD complement each other nicely, but it's been my
experience that Cucumber works on a very diﬀerent rhythm and timescale than TDD.

@geeksam
Cucumber
Scenario
Unit TestFail
Pass Refactor
Fail
Pass
Refactor
12
I think of Cucumber as a set of guide rails for TDD, and my workflow for using it goes something like this...

[REVEAL EACH]
- I start with a Cucumber scenario. I run the scenario...

- and watch it fail. I look at the error message to find out why it failed, and use that information to go...

- write a unit test.

- I watch the unit test fail...

- make it pass...

- and refactor. At this point, I have a choice. If I know what the next unit test I need to write is,

- I do that, and go back around the red/green/refactor cycle again, usually several times. But if I'm stuck...

- I go back and run the Cucumber test again. It's probably still failing, but it's failing for a different reason. So I go back into that fast TDD cycle again. At some point, I
run the Cucumber test and...

- it passes. I do a little dance,

- refactor,

- and move to the next scenario.

@geeksam
Cucumber
Scenario
Unit TestFail
Pass Refactor
Fail
Pass
Refactor
13
When I'm working this way, I spend most of my time in that tight inner loop doing TDD: red/green/refactor. The fact that the tests and the code under test are both
written in the same language makes it easy to go around the inner TDD loop very quickly—sometimes writing a new test every minute or so. This is where I'm focusing
on how the thing works, and it's good, satisfying, detail-oriented work.

@geeksam
Cucumber
Scenario
Unit TestFail
Pass Refactor
Fail
Pass
Refactor
14
But when I start to lose sight of the forest for the trees, I jump back up to Cucumber. The shift from a programming language back to Gherkin helps remind me to get out
of that hyperfocused how mode, and come back to thinking about what, why, and who... and that helps me ﬁgure out the next thing to do.

@geeksam
"For me, Cucumber works 
more like a mind hack than a testing tool. 
It reminds and invites me to think about the
experience of using software separately from
the details of its implementation."
-Tom Stuart
http://codon.com/notes-on-counting-tree-nodes
MIND HACK
15
Tom Stuart wrote something about Cucumber that really resonated with me. He described it as [REVEAL] "more like a mind hack than a testing tool", because it helps
him think about the big picture rather than the details.

@geeksam
"I wish more people knew it works best 
as a thinking and collaboration tool, 
not just an automated checking tool"
-@mattwynne
A THINKING TOOL
🎓
16
When I first put this talk together, I asked Matt Wynne, co-author of The Cucumber Book, if there was anything he wanted people to know about Cucumber. He tweeted
back that he wished more people viewed Cucumber as [REVEAL] a thinking and collaboration tool, not just something for test automation.

Both of these quotes lead back to something I said a few slides ago.

I asserted that Gherkin is not a programming language, and that Cucumber is not a TDD tool. But negative definitions aren't very useful. Or, to reframe that, a positive
definition is much more useful than a negative one. What are these things for? Well, I already talked about how I use Cucumber as guide rails around an inner TDD loop,
but let's talk about Gherkin some more.

@geeksam
Describing software 
At the level of 
USER INTENT 
(and maybe automating 
tests with Cucumber)
GHERKIN IS FOR:
💚
17
In contrast to the quotes I just showed you, this part is purely my own opinion, based on my own experience. I want to make it clear that I do not speak for the
Cucumber team here.

I think that Gherkin is for: [REVEAL EACH]

- describing software 
- at the level of user intent.

And, at some point, you might choose to use Cucumber to turn your Gherkin artifacts [REVEAL] into automated tests.

Let's unpack that one piece at a time.

@geeksam
Acceptance Criteria
DESCRIBING SOFTWARE
If this doesn't work, 
our users will abandon us.
💚
18
By "describing software", I mean that Gherkin lets you capture

[REVEAL] "acceptance criteria".

And by "acceptance criteria", I mean [REVEAL]

"the system has to do these things or you don't get paid."

@geeksam
When I add a new widget
USER INTENT
NOT:
When I visit "/widgets" 
And I click the "New Widget" link 
And I ﬁll in "Widget Name" with "My Widget" 
And I click the "Create Widget" button
💚
19
By "user intent", I mean that Gherkin lets you paint your picture [REVEAL] in broad strokes, without getting bogged down in a lot of details. Details are what TDD is all
about.

[REVEAL, PAUSE] I've worked with scenarios that look like this, and what I found was that every time I tweaked my user interface, twenty of my Cucumber scenarios
would explode, and I'd have to spend an hour or three adjusting everything, which... is not the best use of my time.

@geeksam
(maybe)
AUTOMATING TESTS
💚
20
Finally, just because Cucumber is often pitched as a tool for writing automated tests, you are under no obligation whatsoever to use it to automate your tests. Personally,
I think Cucumber's greatest value comes from using Gherkin to facilitate conversations between developers and the people who pay us. I've written Gherkin ﬁles, thrown
them away, and felt like my time was very well spent.

@geeksam
💚
21
At the level of 
USER INTENT 
So this is what I now think Cucumber should be used for. But it took me a long time to ﬁgure this out, and I made a lot of mistakes along the way. Some of those
mistakes were rather painful. In the hope that you can learn from them, I'm going to share my mistakes with you.

@geeksam
Inconceivable!
CLASSIC BLUNDERS
💔
22
Yes, this is the part of the talk where you all get to laugh at me. [REVEAL]

This is by no means an exhaustive list of my Cucumber fuckups; these are just some of the more interesting, entertaining, or educational ones.

@geeksam
When I visit "/test/js-tests"
Then I see 44 passing QUnit tests
WTF
💔
23
I'm going to show you a Cucumber scenario that I helped write in a real live codebase.

Before I show it to you, let me reiterate: it is okay to laugh at me. Everybody ready?

[REVEAL AND... WAIT FOR IT]

Pro tip: just because you've already got Cucumber set up to drive a browser doesn't mean you should use it for something like this!

Going back to what I said Gherkin was for...this deﬁnitely describes software, and oh boy is it automated, but it completely fails the "user intent" test. Here's a quick
question that should have clued me in that this did not belong in Cucumber: who cares about this? Is this important to someone using my software?

@geeksam
Given the Rails application
Then CRUD works for the Widgets controller
ACCEPTANCE ASTRONAUT
💔
24
Here's something else I did in an actual project. [REVEAL]

What this did was

- visit the new page for the widgets controller,

- ﬁll out the form with randomly generated data,

- submit the form,

- check that the random data it submitted was displayed on the show page,

- click the edit link,

- change each value on the edit form, click "Save", make sure that the changes were visible... and so on.

Seriously, this was a lot of fun, but while we were gold-plating our Cucumber suite, we were avoiding writing actual features that our actual customer actually cared
about, and pretty soon they actually ﬁred us.

@geeksam
CUKING REGRESSION TESTS
💔
25
When you get a bug report, it's a good idea to write an automated test to reproduce the bug. You might even want to commit that test to your repository so you can
detect any future regressions. However, it is my considered opinion that you should not put these tests in Cucumber. As with so many things in software, it comes down
to communication.

Gherkin is a great way for you to tell the story of your application. Ideally, when you bring on a new developer or manager, you can print out all of your feature ﬁles and
hand them to your new hire. They should be able to read through those in an hour or two and come away with a pretty good high-level idea of what your software does.
Regression tests interrupt that narrative with a bunch of digressions into "remember the bug when?"

Doing this is also a good way to commit the next blunder, which is just plain [ADVANCE] having too many scenarios.

@geeksam
*(to run on every commit)
TOO MANY SCENARIOS*
💔
26
Opinions vary on how long is a reasonable time to wait for a test suite. Personally, I'm willing to wait about ﬁve minutes, up to three or four times a day. Any more than
that and my tests might as well not be running.

If you do ﬁnd yourself in this situation, you might consider tagging a critical subset of your tests to run before every commit, then let your CI server run everything else
after you push.

Another way to commit the "too many scenarios" blunder is [ADVANCE] to automate every feature you write.

@geeksam
@FYI / @TBD
AUTOMATING EVERY FEATURE
💔
27
It's perfectly okay to use Gherkin to facilitate a conversation with someone—possibly even yourself, if you don't happen to have a rubber duck handy. Once you've
learned what you needed to learn from the feature ﬁle, you have my permission to just delete it. It's okay!

Now, if do you feel the need to hang on to it for posterity, you still don't have to automate it. Before you commit your changes, tag your features as [REVEAL] "@FYI" or
"@TBD", and change your conﬁguration so that scenarios with that tag never run.

So far, all of these blunders can be boiled down to a loss of focus on the user intent. But Cucumber also gives you plenty of room for implementation blunders, so I want
to talk about a those before we move on.

@geeksam
ABUSING STEP DEFINITIONS
Too many step definitions
Huge step definitions
Too many huge step definitions
Step definitions that call other s͘te͡p d e҉ fin̕it̕i͏o̕n̨ s
Just about any l ͔̭͎̘͓̣
o̥̯̯ ̭g̝i ̘̪̯̱̙̟
c̗͓̟͕̭̬
wh̛ a̢ ts ̀oe̢ v͡er͡ in a̙̖̙̖͔̤
͓͍s̗͎̩
̩t͓̥͖
e̗̹p͝ ̗̮̩̰̜͍
͉͕͉̲͍͚̞͟
d̀͒
̳̩ͅ
ę̭̹̥͎͈̲
͋́ͯ̾
f ̋̚ ͒ͥ͑͛
̷
̖̼͎̞͓͕
i ̨͔̈́̈́̌̇͗͐ņ̴̖̯̦̞
͗̾͆̆ ͛͊
i͐ͮ̔̄̿ͮͩ͊
̬͉̖
t͌̆̑̄
҉ ͈͙̻̥͚͉̩
i͐́̓́
̦̗̳
ô͆̿̉ͭ͂͊
̶
́
͈͓n̎̆ͥ͐
͏̵̖̗
😡
Given /some rubbish/ do
# rubbish code...
end
28
Basically, step definitions are terrible. They're a necessary implementation detail, but they will get you in SO MUCH TROUBLE.

 
If you get the Cucumber book—and you should get the Cucumber book—it will tell you to define a bunch of helper functions and invoke them from your step definitions,
and that does help. But... I started using Cucumber years before the Cucumber book was published. Which means I've made mistakes like:

[REVEAL EACH. DO NOT AD-LIB NUMBERS; THEY'RE COMING SOON]
Doing either of these last two things turns out to be an excellent way to summon the Elder Gods. Ask me how I know this.

@geeksam
</RANT>
💔
29
After making all of those mistakes and more, I found myself feeling very conﬂicted about Cucumber. I really loved the expressiveness of Gherkin, and I wanted to believe
in this idea that programmers and managers could sit down in a room and write acceptance tests together in universal harmony.

But I struggled to reconcile that with the project I'd been working on, which had hundreds of scenarios backed by 750 step deﬁnitions that contained almost 5,000 lines
of Ruby code, and the whole test suite took about 90 minutes to run. It sucked.

Eventually, I found myself asking an interesting question...

@geeksam
❓
THE QUESTION
How would you write scenarios
if you didn't know
what the UI was going to be?
30
[REVEAL AND READ]
[PAUSE]

I assert that, if you can tell from reading your Cucumber features whether it refers to a web app or a desktop app or a CLI... you're probably letting too much detail leak
into your features. And even though this question doesn't directly say anything about user intent, it turns out to be a great way to drag your attention back to what really
matters about your software.

Here's an example.

@geeksam
❓
When I go to the home screen
And I click on the "Widgets" button
vs.
When I view the list of available widgets
INTERFACE VS. INTENT
31
[PAUSE 5 SECONDS. THE SILENCE WILL NOT KILL YOU.]

Here's another.

@geeksam
❓
When I POST <some JSON> to "/widgets/42"
vs.
When I save my changes to the current widget
32
[PAUSE 5 SECONDS. BREATHE.]

@geeksam
❓
if you didn't know
33
This question ﬂoated around in my head for a while as I worked on other things, until...

@geeksam
THE PROJECT
Salesperson Commissions
Multiple compensation schemes
Schemes change every quarter or two
Described in dense, imprecise quasi-legalese
💚
34
Once upon a time, I was brought in to work on one part of a rather large monolithic Rails app that calculated [REVEAL] salesperson commissions. Now, you might hear
"salesperson commissions" and think "okay, so you add up how much each person sold, multiply the total by some percentage, and cut a check, right?"

But, no. That would be far too simple. There were usually [REVEAL] half a dozen compensation schemes in effect at any one time, with very different incentives for
different kinds of sales staff. These schemes changed [REVEAL] a couple of times a year, sometimes quite dramatically. And they were worked out by the sales
department, who would put together something like fifteen pages of [REVEAL] dense, confusing quasi-legalese to describe each plan, which we then had to read
through and somehow translate into working code.

So, one of my goals as I worked on this project was to be able to describe every aspect of these schemes using Gherkin. I figured that if I could show them how I
translated their documents into clear, precise examples, maybe in a year or two they'd work out how to do it themselves.

@geeksam
This is a work of fiction.
DISCLAIMER
Any resemblance to actual 
sales compensation schemes is... 
unfortunate.
💚
35
I'm going to talk about a simplified, fictionalized version of how just one of those compensation schemes worked. It's not super important for you to catch all of the
details here. I'm just trying to give you a quick taste of how I wrote and organized my features for this project.

@geeksam
TARGETS
Sales Target: $100,000 / month
Target Bonus: $100
💚
36
We'll start with the concept of a "sales target" and a "target bonus". This is, basically, the company saying "if you sell $100k of widgets in a month [REVEAL] (that's the
sales target), we'll pay you $100 over your base salary [REVEAL] (that's the target bonus)."

By the way, these numbers aren't realistic; I just picked them to make it easy to convert between percentages and dollars in my head.

Anyway, there's a scaling factor here: if you miss your target, you get paid less. If you exceed your sales target, you get paid more. So far, so good—but there's a catch.

@geeksam
PAY CURVE
💚
% to Sales Target % of Target Bonus paid
0% 0%
50% 25%
100% 100%
150% 175%
37
The catch is this little thing called a "pay curve". The pay curve is a simple function: you put in what percentage of the sales target you hit, and you get back the
percentage of your target bonus that you'll get paid. To describe the pay curve, the sales department actually gave us a spreadsheet with example rows for every
possible input value from zero to 250.

Fortunately, since it was already in a spreadsheet, it was easy to build a chart so we could see the curve.

@geeksam
PAY CURVE
💚
PercentofBonus
0%
75%
150%
225%
300%
Percent to Target
0% 25% 50% 75% 100% 125% 150% 175% 200% 225% 250% 275%
38
Looking at the chart shows that this is a "piecewise linear function", and that helped me wrap my head around what was happening. So [ADVANCE] I went back to the
spreadsheet...

@geeksam
PAY CURVE
💚
% to Sales Target % of Target Bonus paid
0% 0%
50% 25%
100% 100%
150% 175%
39
and from there, it was quite straightforward to convert that spreadsheet into a Cucumber table, and use it to drive a scenario outline.

@geeksam
Feature: Pay Curve for Sales Associates 
Scenario Outline: "Sales Associate" Pay Curve
Given the "Sales Associate" pay curve
When the rep hits <Percent to Target> of their sales targets
Then their monthly payout is <Percent of Bonus> of target monthly bonus 
Examples:
| Percent to Target | Percent of Bonus |
| 0 | 0.0 |
| 1 | 0.5 |
| 50 | 25.0 |
| 51 | 26.5 |
40
In case you're not familiar with it, a scenario outline is basically a [REVEAL] template for a scenario, followed by [REVEAL] a table. The template gets executed once for
each row in the table, with [REVEAL] values from the appropriate column ﬁlled in wherever you put a placeholder value.

Now, because I knew that this was a piecewise linear function, I was able to get away with not turning all 251 rows of the spreadsheet into a giant Cucumber table. I put
in a few examples around each of the inﬂection points just to make sure I got the boundaries right, and then moved on to the compensation scheme...

@geeksam
Feature: Compensation Scheme for Sales Associates 
 
Background:
* I am a sales rep using the "Sales Associate" scheme
* I have a monthly sales target of $100,000
* I have a monthly target bonus of $100 
 
Scenario Outline: Using the right pay curve
When I make <Sales> in sales
Then my monthly sales bonus is <Bonus>
Examples:
| Sales | Bonus |
| $ 0 | $ 0 |
| $ 50,000 | $ 25 |
| $ 100,000 | $ 100 |
41
At ﬁrst glance, this looks remarkably similar to the scenarios for the pay curves. When I ﬁrst wrote this, it felt like I was repeating myself, but this actually introduces quite
a few new concepts:

[REVEAL EACH]
- compensation schemes,

- sales target and target bonus in dollars instead of percentages,

- actual sales in dollars, and

- bonus amount in dollars.

With those concepts in place, I could then introduce the next feature of this scheme.

@geeksam
Feature: Commissions Plan for Sales Associates 
 
Scenario Outline: Safety Net
Given my safety net IS deployed
And my safety net bonus is <Safety Net> 
Examples:
| Sales | Bonus | Safety Net |
| $ 0 | $ 0 | $ 100 |
| $ 50,000 | $ 25 | $ 75 |
| $ 100,000 | $ 100 | $ 0 |
| $ 175,000 | $ 225 | $ 0 |
42
This is the last one, I promise!

The safety net is a feature to help out new hires as they're getting up to speed for the ﬁrst few months. This is basically a guarantee that you'll always get paid at least
the amount of your target bonus. If you don't hit your sales target, we'll kick in the diﬀerence. If you do better than your sales target, we'll pay you more than your target
bonus -- but you'll never make less, at least until we take your safety net away.

Anyway, I think that's enough to give you a sense for the project. Before I get to the fun twist, though, I want to talk about an underutilized element of Gherkin's grammar.

@geeksam
Feature: Compensation Scheme for Sales Associates  
FREE-FORM TEXT GOES HERE! 
Scenario: Dolor sit amet
43
I omitted this on earlier slides so I could make the text bigger, but Gherkin gives you some space at the top of the ﬁle where you can write whatever you want.

A lot of Cucumber examples show that space being used for [ADVANCE] "As a / I want / so that"...

@geeksam
Feature: Lorem Ipsum  
As a ______
I want ______
So that ______ 
44
...but in practice, I ﬁnd that people tend to [ADVANCE] ﬁll in that template without really thinking about it...

@geeksam
Feature: Lorem Ipsum  
As a manager ﬁlling in this form
I want to make up some plausible BS
So that the developers do what I tell them to, damn it 
45
[WAIT FOR THE CHUCKLE]
So sometimes I just skip this part. For this project, though...

@geeksam
Feature: Compensation Scheme for Sales Associates  
This scheme is based on gross sales, and uses the "Sales Associate" pay curve. 
This scheme features a "safety net". If the appropriate ﬂag is set,
sales reps on this scheme are guaranteed at least 100% of their
target bonus. This tends to be used in a new hire's ﬁrst few months
so they can earn a reasonable living while they get up to speed.
Background:
* I am a sales rep using the "Sales Associate" scheme
*[...]
46
...I used that space to provide some context about why this feature exists, or what makes this feature interesting in comparison to other features that may be similar.

[KEEP GOING]

@geeksam
Feature: Commissions Plan for Sales Associates  
This plan is based on gross sales, and uses the "Sales Associate" pay curve. 
This plan features a "safety net". If the appropriate flag is set,
sales reps on this plan are guaranteed at least 100% of their target
bonus. This tends to be used in a new hire's first few months so
they can earn a reasonable living while they get up to speed.
Background:
* I am a sales rep using the "Sales Associate" plan
*[...]
47
For this scheme, I was worried that the examples of how the safety net worked in specific cases might not fully explain what that aspect of the scheme was for. So I took
a few lines to explain it, using the simplest language I could.

After I handed this project off, one of the bits of feedback I got was that this documentation, in particular, was extremely helpful in making sense of these frankly
ridiculous compensation schemes.

Again, this is the sort of thing you may overlook if you're trying to treat Gherkin as a programming language. If you're thinking of Gherkin as code, then this section of the
file just feels like a big block comment. But if you're thinking about Gherkin as a medium for communicating with other people about your project, this freeform text area
can be really useful, because it gives you a place to talk about things without having to fit it into a step-by-step recipe.

@geeksam
Feature: Compensation Scheme for Sales Associates 
 
Scenario Outline: Safety Net
Given my safety net IS deployed
And my safety net bonus is <Safety Net> 
Examples:
| Sales | Bonus | Safety Net |
| $ 0 | $ 0 | $ 100 |
| $ 50,000 | $ 25 | $ 75 |
| $ 100,000 | $ 100 | $ 0 |
| $ 175,000 | $ 225 | $ 0 |
48
The last thing I want you to notice about these features is that absolutely every word of this is expressed in terms of the domain, not the interface. If you sat down and
read through all of these features, you'd learn a lot about how this organization thinks it can motivate its salespeople, but you won't have any idea what kind of app
they're using to do it.

Now, let's talk very brieﬂy about [ADVANCE] architecture.

@geeksam
ARCHITECTURE
📐
Controllers &Views
ActiveRecord & Services
Core logic in POROs
49
I normally present at Ruby conferences, which means I mostly talk with Rails developers, and while I love Ruby, the fact that Rails dominates the Ruby market means that
the level of discourse around software architecture is... not always the most sophisticated.

The short version is that I departed from Rails orthodoxy by introducing service objects and a layer of plain old Ruby objects. I organized these into three logical layers
as follows:

[REVEAL EACH]
- The UI is standard Rails controllers and views.

- The UI layer talks to a mix of ActiveRecord objects and some service objects.

- And that layer, in turn, talks to a set of plain old Ruby objects, or POROs, that model the rules for the compensation schemes themselves.

Anyway, this is a little bit formal by Rails standards, but it's nothing earth-shattering.

@geeksam
TESTING LAYERS
💚
ActiveRecord & Services
Core logic in POROs
50
Controllers &Views
@model
@ui
@core
The interesting thing I did for this project was to reuse my Cucumber scenarios, running them at both the UI layer and the ActiveRecord layer. Using the tagging feature
of Cucumber, I wanted to be able to mark a scenario as being [REVEAL] "@ui", [REVEAL] "@model", or both.

The scenarios tagged with "@ui" would drive a web browser, which is the way most people in the Rails world think about using Cucumber.

But the scenarios tagged with "@model" would run directly against the model layer, so they could be faster. The ActiveRecord layer, in turn, would exercise the POROs
indirectly, and I thought that would be good enough.

However, I discovered very quickly that the PORO layer was complicated enough on its own that I didn't want to have to think about the relational data model at the
same time, so almost immediately, I added [REVEAL] a "@core" tag as well.

@geeksam
Then their monthly payout is <Percent of Bonus> of their target monthly bonus 
Examples:
| 0 | 0.0 |
| 1 | 0.5 |
| 2 | 1.0 |
51
Here's how that played out in one of the feature ﬁles I showed earlier.

Once I had written a scenario, I would start by tagging it with the name of the layer I wanted to run it at, plus a [ADVANCE] "WIP" suﬃx to indicate that it was work in
progress.

@geeksam
@core_WIP
Examples:
| 0 | 0.0 |
| 1 | 0.5 |
| 2 | 1.0 |
52
I'd run the scenario and watch it fail, and from there, I'd drop down to RSpec and do the usual small, fast TDD cycles until the scenario passed. Once the scenario was
passing, I'd [ADVANCE] remove the "WIP" suﬃx.

@geeksam
@core
Examples:
| 0 | 0.0 |
| 1 | 0.5 |
| 2 | 1.0 |
53
If I wanted to reuse the scenario at the next layer up, I would then add another tag for that layer, again with the [ADVANCE] "WIP" suﬃx.

@geeksam
@core @model_WIP
Examples:
| 0 | 0.0 |
| 1 | 0.5 |
| 2 | 1.0 |
54
And again, I'd write a bunch of RSpec tests until I got the same scenario passing at the new higher layer, at which point I would [ADVANCE] remove the "WIP" suﬃx
from that layer.

@geeksam
@core @model
Examples:
| 0 | 0.0 |
| 1 | 0.5 |
| 2 | 1.0 |
55
So that's how I approached this from the Gherkin side of things.

But it took me a while to ﬁgure out a good way to implement this...

@geeksam
Messing with Cucumber load path
THROWING A SWITCH
💚
56
I wanted Cucumber to run all of the @core scenarios with the load path set to "features/core" to load the core set of step definitions, then run all of the @model scenarios
with the load path set to "features/model", which had its own set of step definitions, and then run all of the @ui scenarios with a *third* load path pointing to a third
independent set of step definitions.

Manipulating the load path was painful, but it basically worked. The problem was that, any time I changed a step, I had to edit three different regular expressions in three
different files and make sure they all matched. Which...was not very much fun.

@geeksam
Swapping in different step drivers
THROWING A SWITCH
💚
Given /(.*?) in sales/ do |amount|
step_driver.given.sales_total(amount)
end
57
So instead, I wound up consolidating down to one set of step definitions that invoked a step driver. [REVEAL, PAUSE]

Again, this is nothing earth shattering. A basic principle of object-oriented programming is that if you send messages to an object that plays a role, you can swap in a
different object without modifying the calling code. It's called "polymorphism."

As an experienced Cucumber user, though, I did find it nicely refreshing that being forced to do this made my step definitions very small and simple.

With this change, I was then able to define three different step drivers to interface with each layer of my application. I put all three of them on the load path, and used an
environment variable to decide which step driver to instantiate.

I had a lot of fun doing this, and used literally every single Ruby trick I had learned in almost a decade to do it... but further details about that are beyond the scope of this
talk. Feel free to ask me about them later if you're curious.

@geeksam
OBSERVATIONS
🔬
58
I just have a few observations to make before I wrap up.

First, a piece of advice from my inner ﬁve-year-old:

@geeksam
THE STEP DEFINITIONS ARE LAVA
59
🔬
Given /(.*?) in sales/ do |amount|
step_driver.given.sales_total(amount)
end
The step definitions are lava.

Because of the way Cucumber works, the step definitions have to be there, but they should be a very thin adapter between Gherkin steps and a custom driver for
automating your application. [REVEAL] Ideally, a step definition should be one annoyingly obvious line of code.

Step definitions exist in a flat namespace with one global scope for sharing variables between them. Moving all of the interesting logic out into a step driver lets me use
the full set of tools my programming language gives me to organize and refactor code, and lets me keep the step definitions so simple that I never have to think about
them.

Using a step driver worked so well for me that I'd do it again on a new project even if I wasn't using a multi-layered approach.

@geeksam
BREAK IT DOWN
🔬
60
These commissions plans involved something like fifteen pages of legalese, which I then had to translate into code. It was just too complicated for me to hold all of it in
my head at once.

Breaking the problem down across architectural layers allowed me to focus on just the core logic, then figure out how to adapt it to my persistence layer, and then fill in
the UI once everything else was already working. And honestly, looking back at it, this was probably the only way I could have completed this app in any reasonable
amount of time.

@geeksam
TAG DISTRIBUTION
🔬
Feature @core @model @ui
Pay Curve ✓
Associate scheme uses right pay curve ✓ ✓
Safety net ✓ ✓
"Big Deal" bonus ✓ ✓
Total monthly bonus ✓ ✓
61
Some scenarios I only tagged at one level, and some of them I tagged at two. In practice, it turned out that the ones that ran at multiple layers always ran at two
adjacent layers. So: @core and @model, or @model and @ui, but I never had a single scenario that ran at both @core and @ui.

I think this was because I had some features that described some lower-level concepts, like the pay curve, that never showed up directly in the user interface. So it
would make sense to describe those concepts at the core and model layers—and I still got a lot of value from describing them in Gherkin—but by the time I made it up
to the UI, they just weren't worth mentioning.

@geeksam
SUSTAINABILITY
"I can see why you did it that way"
"It was really nice to have such clear documentation"
🔬
62
The developers who took over the project from me weren't as enthusiastic about this Cucumber setup as I was... but they did say things like [REVEAL EACH]

- "Yeah, I can see why you did it that way" and

- "it was really nice to have such clear documentation about what these business terms meant."

So I'll call that a win.

@geeksam
🔬
PERFORMANCE
63
I do have to talk about performance.

At Cascadia RubyConf in 2011, Ryan Davis gave a talk called "Size Doesn't Matter," in which he talked about the speed and relative size of various testing frameworks in
Ruby. Cucumber shows up in nine diﬀerent slides from Ryan's slide deck, and they basically all look like this:

@geeksam64Cascadia Ruby Conf 2011 @ Seattle, Cascadia
Ryan Davis, Seattle.rbSize Doesn’t Matter
Lines of Code
(including dependencies)
bacon
minitest
shoulda
test-unit2
rspec2
cucumber 17430
7484
6381
4236
1166
380
+ 62,985
lines of C!
Monday, August 1, 11
In every single metric Ryan chose to present, Cucumber came in dead last. When I saw his talk, I laughed and winced, because at the time, I was dealing with that 90-
minute test suite. So when I started this project, I was fully expecting to pay a huge performance penalty.

@geeksam
PERFORMANCE
Layer # Scenarios Time
Core 64 0.7s
Model 118 7.2s
UI 11 3.0s
Total wall clock time (WIP + done) ~40s
🔬
65
With that in mind, here are the numbers from this project.

There were 64 scenarios tagged with "@core", and they ran in [REVEAL] under a second.

At the model layer, 118 scenarios ran in [REVEAL] under eight seconds.

At the UI layer, I used Capybara to drive the application. I didn't care about Javascript, so I was able to just look at the HTML, but even with that advantage, these were
by far the slowest: [REVEAL] 11 scenarios in three seconds.

Of course, these are the times reported by Cucumber, so they don't include the time it takes to load all of Rails. I had a build task set up to run the WIP versions of all
three layers, then run the non-WIP versions, for a total of six separate test runs, so I loaded Rails six times. The total wall clock time for that build task came in at
[REVEAL] about 40 seconds.

So, yeah. Cucumber isn't the fastest thing out there. It's never gonna run thousands of tests per second. But I was... pretty happy with these numbers.

@geeksam
🎓
IN SUMMATION...
66
I know that was a lot of information to take in. Just as a reminder, my slides are available on the conference website, so feel free to supplement your notes with those.

Before I let you go, I just want to reiterate what I think are the two most important ideas from this talk...

@geeksam67
At the level of 
USER INTENT 
🎓
The ﬁrst of those is, again, what I think is really the sweet spot of Cucumber usage: that focus on communicating with other people about the actual value that your
application is supposed to bring to your users.

@geeksam
THE QUESTION
if you didn't know
68
🎓
And the second thing is this question, which you can use to evaluate your Cucumber scenarios and make sure they're focused at that right level of abstraction.

@geeksam
README
💚
69
@geeksam
Podcast: >Code
GreaterThanCode.com
Finally, If you enjoyed this talk and want to hear more from me, you can ﬁnd me on Twitter at @geeksam.

I'm also on a podcast called Greater Than Code, which is largely about people in tech and how we can get better at the "people" part of that.

@geeksam
FIN.
💚
70
@geeksam
Podcast: >Code
GreaterThanCode.com
Thank you!

@geeksam
Q&A
💚
71
@geeksam
Podcast: >Code
GreaterThanCode.com
We've got a few minutes for Q&A. Before we launch into that, I know some of you are ready to get up and move around a bit, so if you need to leave, please do.

For those of you who are sticking around, I ask that you please be respectful of everyone's time, and remember to phrase your question in the form of a question. If you
have a longer topic you'd like to chat about, feel free to come talk to me after the session. Let's have some questions!

Cucumbers Have Layers: A Love Story (AATC 2017)

Recomendados

Recomendados

Más contenido relacionado

Similar a Cucumbers Have Layers: A Love Story (AATC 2017)

Similar a Cucumbers Have Layers: A Love Story (AATC 2017) (20)

Último

Último (20)

Cucumbers Have Layers: A Love Story (AATC 2017)