Announcing cucumber-electron(cucumber.io)
cucumber.io
Announcing cucumber-electron
https://cucumber.io/blog/2017/01/23/announcing-cucumber-electron
8 comments
The title is slightly misleading. This just looks like an alternative way to run Cucumber, not a complete alternative to Selenium.
It didn't even occur to me that there were other ways of using Selenium than Selenium-WebDriver!
Well a key property of Selenium is that it's cross-browser. Whilst it's nice to imagine that the web is in a position where running all your tests in some version of Blink means that your site will work on Firefox/Edge/Safari, I fear that the reality is still rather different.
This is not an alternative way of using selenium. It's a way to avoid using selenium. The advantage over selenium is that tests and application code are run in the same process, so they tend to be faster, less brittle and easier to debug. The downside is that they are not cross-browser.
There are also other ways of using Selenium, though, that I had mentally disregarded and ignored as probably not useful.
You can also get this behavior (basically single thread of execution) with cucumber-rails and Selenium (or probably any driver other than Selenium, for that matter). I'm not saying the application code is literally run in the same thread or process as the test executor, but in effect they do. Your Ruby process is in charge of setting up the HTTP server, as well as driving whatever driver (Selenium, Phantom, ...) you're using for Cucumber.
So serendipitously, with (MRI) Ruby being a single-threaded beast, you do still get that behavior and quality of being able to catch errors that pop up on either side, making it much easier to debug misbehaving server or tests.
You can also get this behavior (basically single thread of execution) with cucumber-rails and Selenium (or probably any driver other than Selenium, for that matter). I'm not saying the application code is literally run in the same thread or process as the test executor, but in effect they do. Your Ruby process is in charge of setting up the HTTP server, as well as driving whatever driver (Selenium, Phantom, ...) you're using for Cucumber.
So serendipitously, with (MRI) Ruby being a single-threaded beast, you do still get that behavior and quality of being able to catch errors that pop up on either side, making it much easier to debug misbehaving server or tests.
This sounds very interesting. I have been using cucumber in my development with Selenium WebDriver (cucumber-rails) and the author here has promised things that were missing in PhantomJS and Capybara-WebKit that drove me toward using Selenium and Chrome instead. Namely the option of running in either Headed mode with the inspector available, or Headless.
Does this project require your system-under-test to be a pure Node.Js application to work with it? (I am not sure if I had heard of electron, but it looks to be the case.) Is there any possibility of porting this to be a driver for test systems that use Cucumber for systems built in other languages?
Does this project require your system-under-test to be a pure Node.Js application to work with it? (I am not sure if I had heard of electron, but it looks to be the case.) Is there any possibility of porting this to be a driver for test systems that use Cucumber for systems built in other languages?
Yes you can use it to describe and automate systems built in other technologies and I've done that on a couple of projects. It's a good fit for any web app with a lot of client-side code. But its sweet spot is definitely node.js apps.
You could certainly use electron as a driver for tests written in other tech, but there is no capybara electron driver that I'm aware of. Using cucumber-electron means you write your step definitions (automation code) in JavaScript.
(I'm the author of cucumber-electron)
You could certainly use electron as a driver for tests written in other tech, but there is no capybara electron driver that I'm aware of. Using cucumber-electron means you write your step definitions (automation code) in JavaScript.
(I'm the author of cucumber-electron)
Thanks for showing up and answering questions!
I've gotten used to having my rails application (server) running in the same thread as my cucumber (client-side) steps, thanks to cucumber-rails. So I could see why this would be especially useful for testing node.js applications, with the same advantages. (A fault detected anywhere on either side of the test execution, client or server, can be escalated and reported to stop the entire test, delivering a really good, clear idea of where to look in the code for the violator!)
This looks like a really neat project.
I've gotten used to having my rails application (server) running in the same thread as my cucumber (client-side) steps, thanks to cucumber-rails. So I could see why this would be especially useful for testing node.js applications, with the same advantages. (A fault detected anywhere on either side of the test execution, client or server, can be escalated and reported to stop the entire test, delivering a really good, clear idea of where to look in the code for the violator!)
This looks like a really neat project.
[deleted]
whats the best headless browser for CI/CD? I've always used Phantom, but would be keen to explore alternatives (especially if it could quiet the "nobody uses phantom in real life")
For me so far the most pleasant to work with has (contrary to its name) been Nightmare (http://www.nightmarejs.org/), which is a Node.js library that drives Electron. It has a really pleasant API and its developers claim that it is faster than PhantomJS. Unlike PhantomJS, however, it is not truly headless; it needs an X server like xvfb to run.
While not actually a real-life browser, Phantom 2.5 is _much_ closer to a real-life browser engine than it was previously.
I write tests using chromedriver. Ends up being really simple to write, and if I want to debug something, it's just a chrome instance that I can use normally if I make the tests wait.
Only downside is you have to make it run with xvfb on the CI server, but that's pretty easy to set up in your config file usually.
Only downside is you have to make it run with xvfb on the CI server, but that's pretty easy to set up in your config file usually.
Is headless Chrome working yet?
https://chromium.googlesource.com/chromium/src/+/lkgr/headle...
https://chromium.googlesource.com/chromium/src/+/lkgr/headle...
It works, but there are a number of issues in the remote control protocol.
(note: I am developing a python wrapper for controlling it: https://github.com/fake-name/ChromeController )
I've been meaning to see about wrapping the C++ api as a python module, but not had time.
(note: I am developing a python wrapper for controlling it: https://github.com/fake-name/ChromeController )
I've been meaning to see about wrapping the C++ api as a python module, but not had time.
You can also run Chrome inside a docker container and drive it with the remote driver. Handy if you are running on an aws Linux instance with no X
I wrote something similar to this but 1 level higher by using Karma. It's called `karma-electron` and it allows you to use any test framework inside of Electron as well as any other launcher Karma supports (e.g. Firefox, Chrome, PhantomJS):
https://github.com/twolfson/karma-electron
https://github.com/twolfson/karma-electron
Can anyone explain why you would want to use Cucumber? I heard it enables non-programmers to write the tests themselves but I don't see that as realistic as non-programmers don't have the training to specify requirements rigourously. At least with coded tests you can use IDEs to validate and refactor your tests.
We use cucumber at work for a few weeks now. Our product owner actually writes the initial feature files for our end-to-end tests and we refine them. We keep sending him back on what we changed and why, and the product owner learns from that and tries to do it the next time.
We started doing this around 6 weeks ago and it is helping us developers a lot in my opinion, as well as getting some work off of us. Sure, we need to refine the POs feature definitions, but it's better than nothing.
Before the feature files get written, have a conversation with the PO and other stakeholders.
Involve a QA person and relevant developers.
Catch the changes early, ask for more examples of the feature or goal, "have you thought about ...", "What happens in this case...?".
You will get better quality features with less rework, because you are writing the thing the PO actually wants and cover off edge cases before they crop up in testing.
Involve a QA person and relevant developers.
Catch the changes early, ask for more examples of the feature or goal, "have you thought about ...", "What happens in this case...?".
You will get better quality features with less rework, because you are writing the thing the PO actually wants and cover off edge cases before they crop up in testing.
Cucumber and other tools such as Behat allow you to capture conversations you have with your client, product owner and stakeholders.
They enable you to build the right thing - the thing that helps the users or meets the business goals.
Have a look for BDD, Behaviour Driven Development.
The main thing is to have the conversations. The automated tests are a lovely side effect!
They enable you to build the right thing - the thing that helps the users or meets the business goals.
Have a look for BDD, Behaviour Driven Development.
The main thing is to have the conversations. The automated tests are a lovely side effect!
I can tell you why I'm using it, and it's not to enable non-programmers to write tests. (Not anymore, it was at first...)
The number one advantage is that using Cucumber produces human-readable documentation that is also executable, and through the magic of CI is prevented from becoming stale.
I use Cucumber with Rails, and I've actually heard a lot of really good arguments not to use Cucumber. The one thing these all seem to have in common is, they almost universally do NOT completely rebuke TDD at-large. Nobody (competent) is suggesting you should develop by the seat of your pants. They have issues with Gherkin or with Cucumber's implementation specifically, but if your team had success with Cucumber, you would not give it up without planning to have a replacement.
You may choose to eschew Gherkin style entirely, and you can certainly accomplish much the same thing using feature testing in modern RSpec and scenarios. This may be easier to refactor for you, but for me a well-organized vocabulary of step definitions makes it so easy to copy a scenario, reorganize the steps, and add some new things to create a new feature, adding almost no new test code.
There is also Spinach, which rips out all of Cucumber's regex silliness and declines to enforce a global vocabulary, instead encouraging modular design of scenarios and step definitions (since you might want to say the same thing in two different places and mean two different things.)
There is also Turnip, which integrates Gherkin into RSpec and also has many of the same advantages of Spinach, but includes other options (cleaner options) besides the Regex method for building parameterized step definitions. I think this one is my favorite, but my small project still uses Cucumber proper.
In the beginning of our project, it was very helpful that my PM was writing out requirements in Cucumber style. It meant that while someone (not a coder, but not an inexperienced PM either) had gone to the trouble to specify requirements with rigor, I could focus on coding, just jump in and start writing features, with a clear target of limited scope, that I could easily follow to avoid straying off into the weeds.
Before long my PM got busy with other projects and I took over writing the cucumber stories. Not just me, but the other dev too -- and that was just as well. I often found myself rewriting PM's scenarios anyway, to make them easier to map directly to step definitions, more clear in a context-free sense, or generally less ambiguously specified. These enhancements also made them less human-friendly.
Both approaches to writing Cucumber stories definitely have merit. The one thing you need to have is a strong set of test scenarios (however you've implemented them) with good total coverage of the features and the requirements, so that new or changed requirements can be evaluated in the context of the whole program: what features are broken by your change?
Two things I have found are also critical when using ATDD: your whole team (at least all of the devs) need to be able to run the tests, and (maybe even more important) you need to have a CI solution in place so that no individual needs to be responsible for running all of the tests after each change when your suite gets too heavy for a pre-commit hook. A regression suite is like a pager: when nobody is on-call then everybody is on-call.
The number one advantage is that using Cucumber produces human-readable documentation that is also executable, and through the magic of CI is prevented from becoming stale.
I use Cucumber with Rails, and I've actually heard a lot of really good arguments not to use Cucumber. The one thing these all seem to have in common is, they almost universally do NOT completely rebuke TDD at-large. Nobody (competent) is suggesting you should develop by the seat of your pants. They have issues with Gherkin or with Cucumber's implementation specifically, but if your team had success with Cucumber, you would not give it up without planning to have a replacement.
You may choose to eschew Gherkin style entirely, and you can certainly accomplish much the same thing using feature testing in modern RSpec and scenarios. This may be easier to refactor for you, but for me a well-organized vocabulary of step definitions makes it so easy to copy a scenario, reorganize the steps, and add some new things to create a new feature, adding almost no new test code.
There is also Spinach, which rips out all of Cucumber's regex silliness and declines to enforce a global vocabulary, instead encouraging modular design of scenarios and step definitions (since you might want to say the same thing in two different places and mean two different things.)
There is also Turnip, which integrates Gherkin into RSpec and also has many of the same advantages of Spinach, but includes other options (cleaner options) besides the Regex method for building parameterized step definitions. I think this one is my favorite, but my small project still uses Cucumber proper.
In the beginning of our project, it was very helpful that my PM was writing out requirements in Cucumber style. It meant that while someone (not a coder, but not an inexperienced PM either) had gone to the trouble to specify requirements with rigor, I could focus on coding, just jump in and start writing features, with a clear target of limited scope, that I could easily follow to avoid straying off into the weeds.
Before long my PM got busy with other projects and I took over writing the cucumber stories. Not just me, but the other dev too -- and that was just as well. I often found myself rewriting PM's scenarios anyway, to make them easier to map directly to step definitions, more clear in a context-free sense, or generally less ambiguously specified. These enhancements also made them less human-friendly.
Both approaches to writing Cucumber stories definitely have merit. The one thing you need to have is a strong set of test scenarios (however you've implemented them) with good total coverage of the features and the requirements, so that new or changed requirements can be evaluated in the context of the whole program: what features are broken by your change?
Two things I have found are also critical when using ATDD: your whole team (at least all of the devs) need to be able to run the tests, and (maybe even more important) you need to have a CI solution in place so that no individual needs to be responsible for running all of the tests after each change when your suite gets too heavy for a pre-commit hook. A regression suite is like a pager: when nobody is on-call then everybody is on-call.
Try not to see a feature file as a way to automate tests but to capture the requirements in a language that isn't technical.
Business rules change far less often than implementations, so the feature files shouldn't be tied to the implementation - that tying together can happen in the ruby code that implement the steps. You can refactor that code easily but the feature is denoting the business rule.
Business rules change far less often than implementations, so the feature files shouldn't be tied to the implementation - that tying together can happen in the ruby code that implement the steps. You can refactor that code easily but the feature is denoting the business rule.
It is both, though!
If you're using Cucumber (and not Spinach or Turnip) you need every (mechanically) different step to be (verbally) unique.
So they need to be descriptive enough to distinguish them from something not quite the same. They don't necessarily need to be descriptive enough to mechanically identify every button and selector used in English language, but they should not be so ambiguous that you could choose a different way and reach a different result.
The coolest thing I found is the ability to compose step definitions. So in my customer-level Gherkin, let's say I have some two-man workflow built around activities managing employee users, I can ultimately say:
"When I disable a user"
and I can implement this as
This is the greatest power feature of Cucumber, in my opinion.
If you're using Cucumber (and not Spinach or Turnip) you need every (mechanically) different step to be (verbally) unique.
So they need to be descriptive enough to distinguish them from something not quite the same. They don't necessarily need to be descriptive enough to mechanically identify every button and selector used in English language, but they should not be so ambiguous that you could choose a different way and reach a different result.
The coolest thing I found is the ability to compose step definitions. So in my customer-level Gherkin, let's say I have some two-man workflow built around activities managing employee users, I can ultimately say:
"When I disable a user"
and I can implement this as
When(/^I disable a user$/) do
step "I select the user \"example user\""
step "I enter a termination date of today"
step "I select a termination reason of \"fired\""
step "Review Form is clicked"
step "Submit is clicked"
step "I log in as HR"
step "I pull up the submitted activity"
step "Complete is clicked"
step "I log in as Payroll"
step "I pull up the submitted activity"
step "Complete is clicked"
end
This is the greatest power feature of Cucumber, in my opinion.
wow I actually thought someone had discovered some ridiculous orbital pathway
Pretty soon no word will have its original meaning. An electron is not a subatomic particle. A cucumber isn't a veggie. Rust isn't...
Perhaps we should take a hint from Urbit and start naming our projects Zog-wip-let-bon, etc.
The pharmaceutical industry has an entire sub-industry of coming up with names - I believe there's an algorithm for producing phonetically reasonable names that are unique.
http://www.popsci.com/science/article/2013-04/fyi-how-does-d...
http://www.popsci.com/science/article/2013-04/fyi-how-does-d...
I came here assuming some novel electron orbital geometry had been discovered. It's always something to do with physics and/or food... always.
What is cucumber? What is Electron?
For example: "cucumber-electron is a Javascript module (available via npm) that combines cucumber, a framework for writing Web page tests in plain language, with Electron, a framework for building desktop apps using Web technologies. ..."