This is still a proof-of-concept but criticism and comments are very welcome.
I was fed up (and also very inexperienced) writing system and end-to-end tests so I made this. The implementation is a bit convoluted but the basic idea is:
1. An LLM writes the specs based on READMEs, design docs, references, etc.
2. Another LLM turns specs into a Python DSL
3. An interpreter runs the DSL
You may review and edit the outputs of step 1 and 2 in one text file, and fill in necessary info to properly set up the tests (like describing how data should be validated when the data schema is an implementation detail).
Comparison:
- vs. Cucumber/Gherkin - Unlike Cucumber, (ideally) you write minimal code with Spaceport
- vs. Unit/integration tests - This is not a replacement but a complement because unit and integration tests are cumbersome to write in natural language--lots of rigid actions and data schemas
I was fed up (and also very inexperienced) writing system and end-to-end tests so I made this. The implementation is a bit convoluted but the basic idea is:
1. An LLM writes the specs based on READMEs, design docs, references, etc.
2. Another LLM turns specs into a Python DSL
3. An interpreter runs the DSL
You may review and edit the outputs of step 1 and 2 in one text file, and fill in necessary info to properly set up the tests (like describing how data should be validated when the data schema is an implementation detail).
Comparison:
- vs. Cucumber/Gherkin - Unlike Cucumber, (ideally) you write minimal code with Spaceport
- vs. Unit/integration tests - This is not a replacement but a complement because unit and integration tests are cumbersome to write in natural language--lots of rigid actions and data schemas