Functional Tests As A Tree Of Continuations (Erlang)(evanmiller.org)
evanmiller.org
Functional Tests As A Tree Of Continuations (Erlang)
http://www.evanmiller.org/functional-tests-as-a-tree-of-continuations.html
7 comments
The state is being saved in the http example, but you can't see it because it's a functional test at the http client level. Think of the test code as a script controlling a browser. boss_test:get_request and boss_test:follow_link are hitting the http server where all the state saving action occurs.
Yes, but after get_request, you've got some data saved in your session (I'd assume that's the case for any multi-step flow over http). Now how do I guarantee that when I "continue" from step 2 and then "cancel" from step 2, I still have the correct session state to act on step 3?
That depends on the integration between the web server and the test framework. In this case the framework is rolling back some in-memory database. If the session state is in that database, you're good to go. Otherwise the framework needs to track whatever else you're using for session state and roll it back like it does the database.
Cool idea.
In my job we are working on business process engine, and we have implemented many long business processes that involves user interactions and calling plSQL code.
Testing these processes manually is pain, we have tests using soapui, but writing tests in linear fashion is repetative and tedious, our test suite is lagging us. Also we use big databases and our tests unfortunately aren't isolated - we have to prepare data manually for tests to run.
This tree-like approach looks great. One problem is - the only way I see to create "stack of databases" is to use savepoint - rollback to savepoint around every node in tests tree.
We use hibernate, so rollback to savepoint won't be enough - we'll need to disable second level cache, and to do sth about first level cache - maybe flushing after each rollback to savepoint will do.
Anybody has experience with hibernate and rolling back to savepoint?
In my job we are working on business process engine, and we have implemented many long business processes that involves user interactions and calling plSQL code.
Testing these processes manually is pain, we have tests using soapui, but writing tests in linear fashion is repetative and tedious, our test suite is lagging us. Also we use big databases and our tests unfortunately aren't isolated - we have to prepare data manually for tests to run.
This tree-like approach looks great. One problem is - the only way I see to create "stack of databases" is to use savepoint - rollback to savepoint around every node in tests tree.
We use hibernate, so rollback to savepoint won't be enough - we'll need to disable second level cache, and to do sth about first level cache - maybe flushing after each rollback to savepoint will do.
Anybody has experience with hibernate and rolling back to savepoint?
This reminds me a lot of "context blocks" as implemented in the Ruby testing libraries Shoulda and RSpec (and probably elsewhere). A context is a named code block which can contain a number of tests, along with some common setup and teardown that's run for each test. Contexts can contain contexts, and tests in nested contexts get all the setup and teardown from their own context and its parent contexts, in the order you'd expect.
I've used nested contexts to achieve much the same sort of thing as Evan talks about here: high-coverage functional testing of a sequence of actions. They have the same advantage that the length of the test stays linear with the number of actions, although unlike Evan's callback-style tests, they didn't stop the test tree as soon as a step failed.
I've used nested contexts to achieve much the same sort of thing as Evan talks about here: high-coverage functional testing of a sequence of actions. They have the same advantage that the length of the test stays linear with the number of actions, although unlike Evan's callback-style tests, they didn't stop the test tree as soon as a step failed.
I realize the bad example wasn't actually doing anything and the tree example does, but the tree looks like it takes a great deal more effort to actually use.
I have a better idea: Erlang already handles green threads extremely well, how about providing a "test mode" switch in the VM. When a branch is encountered in execution the VM spawns new threads as needed and just takes them all. If spawning a new thread would put us over some thread count threshold then just put it on some queue for when there is space for a new thread. Then you can just put pre and post conditions on all your functions and you'll find out if any possible branch traversal causes one of these conditions to fail.
I have a better idea: Erlang already handles green threads extremely well, how about providing a "test mode" switch in the VM. When a branch is encountered in execution the VM spawns new threads as needed and just takes them all. If spawning a new thread would put us over some thread count threshold then just put it on some queue for when there is space for a new thread. Then you can just put pre and post conditions on all your functions and you'll find out if any possible branch traversal causes one of these conditions to fail.
I know he addresses it in the database example (<<For modern web applications, all the state is stored in a database. Therefore to make a "copy of the universe," we just need a way to make a copy of the database to pass down to the child tests>>), but then he follows with testing over http, where he doesn't save the state in any way. It's just a bit confusing this way.
If you do testing in the wrecking ball approach, at least those things are in plain view. Just kill the state every time. If it doesn't take more than a couple of min. to run all the tests, everything's all right.