Zombie.js: insanely fast, full-stack, headless testing(labnotes.org)
labnotes.org
Zombie.js: insanely fast, full-stack, headless testing
http://labnotes.org/2010/12/30/zombie-js-insanely-fast-full-stack-headless-testing/?utm_source=javascriptweekly&utm_medium=email
18 comments
There needn't be One Test Suite To Rule Them All.
I like having something similar to zombie.js during dev, then Selenium during CI triggered on git push, perhaps with Testswarm scheduled to run daily.
I like having something similar to zombie.js during dev, then Selenium during CI triggered on git push, perhaps with Testswarm scheduled to run daily.
Is test swarm even still up? I can't ever seem to get the website to load in a reasonable amount of time.
See the 'Setting Up Your Own Swarm' section on https://github.com/jeresig/testswarm/wiki
it seems you're saying you run selenium using testswarm? Can you describe how that works in more detail?
Reliable headless DOM implementations are one of major obstacles for JS as a first class language. Kudos and thanks to everyone that is working to solve this!
Does this load external styles, or javascript files? What about the images and does it evaluate any script tags?
I have done a few albeit scattered attempts at headless or semi-headless html/js unit tests. What I found out to be a good middle ground is running is this:
Run selenium server and firefox on a remote server. Trigger tests via a task queue, which at the moment is Gearman, have firefox draw to a virtual frame buffer.
What that does for me is cover pretty much every case that a real browser does, and completeness is key in my point of view. I would like to see how Zombie compares on the completeness side and then speed.
I have done a few albeit scattered attempts at headless or semi-headless html/js unit tests. What I found out to be a good middle ground is running is this:
Run selenium server and firefox on a remote server. Trigger tests via a task queue, which at the moment is Gearman, have firefox draw to a virtual frame buffer.
What that does for me is cover pretty much every case that a real browser does, and completeness is key in my point of view. I would like to see how Zombie compares on the completeness side and then speed.
Any chance this could run in Rhino or does the node.js dependency demand V8?
Feature for feature, how does this compare to HTMLUnit?
Feature for feature, how does this compare to HTMLUnit?
As I understand it, the JS DOM implementation does not have the quirks of the different browsers. So theoretically, you could write a test that passes and works in browser X but does not work in browser Y.
My understanding is that jQuery spends most of its time dealing with these quirks.
My understanding is that jQuery spends most of its time dealing with these quirks.
I like the cleanliness of not needing a browser.
But it seems like you'll have to duplicate a lot of functionality and as soon as there are a couple of things you can't test against or that don't produce the exact same result then this becomes unreliable.
Am I wrong?
But it seems like you'll have to duplicate a lot of functionality and as soon as there are a couple of things you can't test against or that don't produce the exact same result then this becomes unreliable.
Am I wrong?
Simulated-browser tests are more like smoke tests. It's an approximation of the environment you're running tests in.
Zombie and other simulated-browser systems can be useful because they're faster and convenient, but especially because they're faster. If you're not able to test quickly, like the article says, you lose the flow.
Nevertheless, you'll be burned if you don't also test in the browsers you're supporting.
davnola, posted above, has the right idea: use Selenium with your CI system in addition to Zombie.
Another idea: make real-browser testing for development faster. I'm working on a tool that does just that, if you're curious:
https://github.com/yui/yeti http://reid.github.com/decks/2010/yuiconf/yeti.html
Like it's peers, my project won't work for everyone, yet. But it's awesome to see so many people caring about testing web apps in dev quickly. Hooray! :)
Zombie and other simulated-browser systems can be useful because they're faster and convenient, but especially because they're faster. If you're not able to test quickly, like the article says, you lose the flow.
Nevertheless, you'll be burned if you don't also test in the browsers you're supporting.
davnola, posted above, has the right idea: use Selenium with your CI system in addition to Zombie.
Another idea: make real-browser testing for development faster. I'm working on a tool that does just that, if you're curious:
https://github.com/yui/yeti http://reid.github.com/decks/2010/yuiconf/yeti.html
Like it's peers, my project won't work for everyone, yet. But it's awesome to see so many people caring about testing web apps in dev quickly. Hooray! :)
[deleted]
How does this compare to env.js?
http://www.envjs.com/
http://www.envjs.com/
does this thing actually compute the rendered coordinates properly, as a browser would? Can I load jquery and call offset() ?
It uses JSDOM (http://jsdom.org/) under the hood. JSDOM is currently at DOM Level 1 (+additions). It also implements some of Level 2 and Level 3 (https://github.com/tmpvar/jsdom/blob/master/test/testlog.txt).
I would guess no, since it includes only a mock of the DOM. But really, are element locations something you want to test?
Definitely! But it would then also require a CSS renderer. Automated testing of something like Decal CMS (decalcms.com) where we provide javascript which manipulates the DOM but allows the designer to plug whatever markup/CSS they like into the system means that automated testing is incredibly difficult and does rely on actually checking for the position (and other computed styles) of an element after we've run certain pieces of code.
Really, YES. Half of the stuff I want to test is that the right stuff is showing up in the right place. Why, what would you want to test?
To me, this seems to only solve a corner case in the main problem of JS testing - testing the DOM without a browser. The first question that comes to mind is, is a DOM even useful in a headless browser when it comes to JS? For now, grant that we're only testing presence and manipulation of DOM elements and no CSS manipulations or accuracy -- which I'd argue is a decent case for testing. Now, testing presence and manipulation of DOM elements in the abstract isn't very useful, as different browsers might treat DOM manipulations differently. Using a headless browser that no one actually uses doesn't solve this problem.
I'd see a case for this in running something like QUnit[1] test that just test JS completeness, but again, implementations differ so testing in one environment that no one actually is using isn't good enough.
The only thing that I've come up with for a solution for actually testing JavaScript in a robust, real-world way is using QUnit and Selenium[2] to define the tests, with Testswarm[3] actually running the tests. This setup allows unit tests to be run against JS by itself (QUnir), mock interactions to take place to ensure bindings work properly (Selenium), and have these executed against real browsers in a distributed manner (Testswarm).
I concede that this isn't an ideal solution, and I wish there were more people working on solutions like Zombie.js that would combine all of these in one so that JS could finally be apart of our continuous integration testing. The only (sic) thing preventing this solution is that it needs to have headless, identical implementations of all the browser's JS engines and DOM parsers in order to be valid.
[1] http://docs.jquery.com/Qunit [2] http://seleniumhq.org/ [3] https://github.com/jeresig/testswarm/wiki