Salad? Gimme Steak. (Moving off Cucumber for acceptance tests)(cmdrkeene.tumblr.com)
cmdrkeene.tumblr.com
Salad? Gimme Steak. (Moving off Cucumber for acceptance tests)
http://cmdrkeene.tumblr.com/post/1042767842/group-me-steak
7 comments
Steak doesn't want to be cucumber at all. It's plain ruby. The "DSL" aspects of the library are purely for legibility and structure. The keywords feature, background, scenario perform the exact same function as the RSpec keywords describe, before, and it respectively.
All steak does is provide a structure for acceptance tests and loop in a web automation driver (capybara).
Far less magical than parsing a the magic language of "Gherkin"
All steak does is provide a structure for acceptance tests and loop in a web automation driver (capybara).
Far less magical than parsing a the magic language of "Gherkin"
"Ruby programs are somewhat cumbersome"
How so?
How so?
My strategy for dealing with all these ruby testing fads is to just keep on plugging away with good old Test::Unit and avoid the churn all together. It takes about two minutes to teach somebody how to read asserts and then you're done.
" ... all these ruby testing fads "
What makes something a fad or not?
What makes something a fad or not?
I have a couple issues with Steak:
1. It seems to take something that's already pretty straightfoward (RSpec) and just adds a couple keywords.
2. You're writing code when you should be describing features. Trying to remember where to put an underscore or what should be a symbol makes it harder to focus on describing the feature.
3. It greek to any non-developer. If you tried e-mailing this to a product owner or a user they would ask why you're sending code samples.
I know you addressed #3 in your blog post but why did you decide to use Steak over RSpec alone? Do you have any issue ramping up new developers with the Steak syntax?
1. It seems to take something that's already pretty straightfoward (RSpec) and just adds a couple keywords.
2. You're writing code when you should be describing features. Trying to remember where to put an underscore or what should be a symbol makes it harder to focus on describing the feature.
3. It greek to any non-developer. If you tried e-mailing this to a product owner or a user they would ask why you're sending code samples.
I know you addressed #3 in your blog post but why did you decide to use Steak over RSpec alone? Do you have any issue ramping up new developers with the Steak syntax?
To your points:
1) Agreed. It's simple. But, it does some nice things like 1) hook up capybara 2) create a /spec/acceptance folder and suite hook and 3) add some "behavior-oriented" keywords like feature, background and scenario. Of course, these are just dumb blocks and yes you could use the provided RSpec DSL (describe, it, before, etc) but it helps (if only mildly) to indicate that this is an acceptance test and to maybe put the programmer more in that mindset. This isn't groundbreaking, it's just a nice convention in my opinion.
2) This is a fair criticism. It does put the programmer more in a "code-centric" mindset than a "behavior-centric" one.
To address this I usually write out the feature in comments exactly as I would in cucumber. This might sound like an argument against steak, but it's not. The most useful thing about cucumber to me is the pattern of thinking: "Given this, When I do something, Then I should see that". The trouble is in the details: the maintenance, the awkward step gymnastics, etc. If you write out a pseudo-feature in comments then under each line of comments write the code, it totally remove the layer of feature vs. step definition indirection that plagues large cucumber projects.
3) Totally agreed. Though, very few projects I've used cucumber on ever let customers see feature files, let alone write them. Steak is for programmers, not customers.
There is no issue ramping up because it's just rspec and capybara. This is exactly what's under the sheets of most cucumber (for rails) installs. Anyone who implements cucumber step definitions should be well versed.
1) Agreed. It's simple. But, it does some nice things like 1) hook up capybara 2) create a /spec/acceptance folder and suite hook and 3) add some "behavior-oriented" keywords like feature, background and scenario. Of course, these are just dumb blocks and yes you could use the provided RSpec DSL (describe, it, before, etc) but it helps (if only mildly) to indicate that this is an acceptance test and to maybe put the programmer more in that mindset. This isn't groundbreaking, it's just a nice convention in my opinion.
2) This is a fair criticism. It does put the programmer more in a "code-centric" mindset than a "behavior-centric" one.
To address this I usually write out the feature in comments exactly as I would in cucumber. This might sound like an argument against steak, but it's not. The most useful thing about cucumber to me is the pattern of thinking: "Given this, When I do something, Then I should see that". The trouble is in the details: the maintenance, the awkward step gymnastics, etc. If you write out a pseudo-feature in comments then under each line of comments write the code, it totally remove the layer of feature vs. step definition indirection that plagues large cucumber projects.
3) Totally agreed. Though, very few projects I've used cucumber on ever let customers see feature files, let alone write them. Steak is for programmers, not customers.
There is no issue ramping up because it's just rspec and capybara. This is exactly what's under the sheets of most cucumber (for rails) installs. Anyone who implements cucumber step definitions should be well versed.
I've used cucumber for several years and love it. The author raises these issues:
a) Organization of steps in a huge project is hard.
b) Complex quoting is hard
c) Token links in e-mail are hard
d) Manual poking of background jobs is hard
As a Cucumber user I've run into some of these.
a) Step Organization: In large projects, this is indeed difficult. I would love an os x app that could search step definitions. On the other hand, Ack makes it fairly simple to find steps. Furthermore, you can always organize them into folders.
If you see cucumber steps as methods, the author is basically saying "having methods makes things hard because I don't know if someone has already written the method. Therefore, I have given up on methods and now write everything inline."
b) Quoting: Cucumber allows for multi-line step definitions
c): Token links in e-mail are hard: Not sure exactly what he's talking about. But check out http://github.com/bmabey/email-spec which I helped write / solves e-mail issues for cucumber.
d) I have a step that just says "And the system processes jobs." In this step, I just run all the delayed jobs. It's not that hard.
To be fair, I haven't looked at steak very much, and am just going off of this blog post.
a) Organization of steps in a huge project is hard.
b) Complex quoting is hard
c) Token links in e-mail are hard
d) Manual poking of background jobs is hard
As a Cucumber user I've run into some of these.
a) Step Organization: In large projects, this is indeed difficult. I would love an os x app that could search step definitions. On the other hand, Ack makes it fairly simple to find steps. Furthermore, you can always organize them into folders.
If you see cucumber steps as methods, the author is basically saying "having methods makes things hard because I don't know if someone has already written the method. Therefore, I have given up on methods and now write everything inline."
b) Quoting: Cucumber allows for multi-line step definitions
c): Token links in e-mail are hard: Not sure exactly what he's talking about. But check out http://github.com/bmabey/email-spec which I helped write / solves e-mail issues for cucumber.
d) I have a step that just says "And the system processes jobs." In this step, I just run all the delayed jobs. It's not that hard.
To be fair, I haven't looked at steak very much, and am just going off of this blog post.
To your issues:
1) Yeah, organization is crazy. I think the method comparison is a bit unfair though. In theory, each step should be easily identifiable (like a method) but this is rarely the case given that most steps have complex regex matching (sometimes for grammatical niceness - e.g. (a vs. an), sometimes for other crazy town reasons). This makes not only automated indexing difficult (RubyMine tries...) but trying to keep hundreds of steps in your head is pretty hard. Especially one's you probably didn't write.
The cool thing about steak is that helpers and "reusable steps" are actually just methods! Hello ctags :D
b) Our case was:
The trouble is you have a url:
I click on the link "Link Title or URL". Since the token is random every time (unless you stub it, which is kinda out of the spirit of acceptance testing, but that's another debate) it's hard to say "Click on the link with the big ass token in it".
You can definitely write a step like:
d) Similar to the above, the catch all step can work in cucumber. Though, in a few scenarios, it's desirable to run only specific jobs or more often in our case, to check if jobs are scheduled in a specific way. It avoids the weird scenario of writing a step definition like "And I run only the blah jobs" or "And a blah job should be scheduled with ...". Ideally the job system would be decoupled entirely from the acceptance test, but we can't all do cool stuff like this all the time: http://corner.squareup.com/2010/08/cucumber-and-resque.html
1) Yeah, organization is crazy. I think the method comparison is a bit unfair though. In theory, each step should be easily identifiable (like a method) but this is rarely the case given that most steps have complex regex matching (sometimes for grammatical niceness - e.g. (a vs. an), sometimes for other crazy town reasons). This makes not only automated indexing difficult (RubyMine tries...) but trying to keep hundreds of steps in your head is pretty hard. Especially one's you probably didn't write.
The cool thing about steak is that helpers and "reusable steps" are actually just methods! Hello ctags :D
b) Our case was:
And the user "212-555-1234" should receive a text with:
| Some "double quoted" and 'single quoted' text |
text.should == "..." (you have to do this anyway in the step_definition)
The trouble is you have a url:
http://coolsite.org/something/abcde22424aw3324234
I click on the link "Link Title or URL". Since the token is random every time (unless you stub it, which is kinda out of the spirit of acceptance testing, but that's another debate) it's hard to say "Click on the link with the big ass token in it".
You can definitely write a step like:
And I visit the account confirmation link
d) Similar to the above, the catch all step can work in cucumber. Though, in a few scenarios, it's desirable to run only specific jobs or more often in our case, to check if jobs are scheduled in a specific way. It avoids the weird scenario of writing a step definition like "And I run only the blah jobs" or "And a blah job should be scheduled with ...". Ideally the job system would be decoupled entirely from the acceptance test, but we can't all do cool stuff like this all the time: http://corner.squareup.com/2010/08/cucumber-and-resque.html
I like Steak, but the main reason for using Cucumber in the first place is to interface with your customers, or at least with people that understand the business side of things on a piece of software. Steak doesn't aim to, and won't replace that.
It's interesting. Are there any performance improvements?
It's definitely faster, though I don't have any actual numbers (sorry about that). Developer performance has increased significantly though, since we we're no longer stuck in "step-state" hell.
yum... steak
Ruby seems to be playing with fire a little bit here, similar to the insanity that happened with Java and XML.
Java programs can't handle some edge case, program becomes configurable with XML, huge win. Then XML goes way too far, people start disliking XML, Java falls from grace.
Ruby programs are somewhat cumbersome, but Ruby, being fairly awesome, allows you to define a DSL. People write DSLs that make tasks easier, huge win. Then there are too many DSLs, people get tired of trying to figure out _another_ (groan) new DSL to do something they now know how to do in ruby.
Both start with this simple idea, flavor of the month type thing, that in the beginning is a huge win. After a while though both burn out. I think the saving grace for ruby is that people will just reimplement the DSL as a more straightforward ruby library.