Pathod: A pathological HTTP daemon for testing and torturing client software(cortesi.github.com)
cortesi.github.com
Pathod: A pathological HTTP daemon for testing and torturing client software
http://cortesi.github.com/pathod/
6 comments
Drat - this story is just a tad premature. A brand new release of pathod will be out next week, including a publicly accessible pathod instance, pathoc (pathod's evil client-side equivalent), and a huge range of other improvements and bugfixes.
This is a similar tool as http://httpbin.org/ which was written by Kenneth Reitz to test python request http://python-requests.org/. I like that this is written in tornado thanks for sharing.
One difference is that pathod (and pathoc, the client-side equivalent) works hard to make it possible to violate the HTTP protocol specs at will, in pretty much arbitrary ways. The next version of pathod also moves away from Tornado to let it take more direct control of comms.
Ok the client sounds really interesting to me. I am goung to play with it a bit.
bane is a similar tool that pathod-interested users might be interested in.
https://github.com/danielwellman/bane
From the readme: "If you are building an application, you may depend on third-party servers or web services for your data. Most of the time these services are reliable, but at some point they will behave in an unusual manner - such as connecting but never respond, sending data very slowly, or sending an unexpected response. To ensure your application survives these scenarios, you should test your application against these bad behaviors. Bane helps you recreate these scenarios by standing in for your third-party servers and responding in several nefarious ways."
https://github.com/danielwellman/bane
From the readme: "If you are building an application, you may depend on third-party servers or web services for your data. Most of the time these services are reliable, but at some point they will behave in an unusual manner - such as connecting but never respond, sending data very slowly, or sending an unexpected response. To ensure your application survives these scenarios, you should test your application against these bad behaviors. Bane helps you recreate these scenarios by standing in for your third-party servers and responding in several nefarious ways."
Really cool!
A year or two ago, I was working on some automated deployment tools that basically called a bunch of Rest APIs. As I discovered how unreliable the API calls could be at times, I ended up writing something kind of similar (although very application-specific to this) to be used as part of my unit tests for it - it helped immensely.
One thing that would be nice is if I could pre-define reactions to certain requests - ie, some sort of way to specify that when you get a POST at uri /add with a content-type of application/json, I want a 500 error (or maybe I want a 500 error 5% of the time) - rather than having to specify what I want in the actual request. Is this something your client library will make possible?
A year or two ago, I was working on some automated deployment tools that basically called a bunch of Rest APIs. As I discovered how unreliable the API calls could be at times, I ended up writing something kind of similar (although very application-specific to this) to be used as part of my unit tests for it - it helped immensely.
One thing that would be nice is if I could pre-define reactions to certain requests - ie, some sort of way to specify that when you get a POST at uri /add with a content-type of application/json, I want a 500 error (or maybe I want a 500 error 5% of the time) - rather than having to specify what I want in the actual request. Is this something your client library will make possible?
This is already covered. Have a look at the anchor feature for pathod. For instance, starting pathod like this:
pathod -a /foo=200:b@100
Will result in an HTTP 200 response with an body of 100 random bytes if you hit the path /foo.
pathod -a /foo=200:b@100
Will result in an HTTP 200 response with an body of 100 random bytes if you hit the path /foo.
I like the idea, but does the syntax have to be that terse? It's quite expressive but less readable than code (or JSON/YAML), and I'm not sure why that's useful in this context.
Yes, this is something I've pondered too. The language is ugly for a number of reasons - not least the constraint of avoiding syntactically significant characters for both URLs (because of pathod) and shells (because of pathoc).
At the moment, my stance is that the syntax needs to be as terse as possible so that it can comfortably be specified in a URL, or in a snippet in a unit test. In the current master, there's a variant syntax that lets you use newlines (or arbitrary whitespace) wherever the more compact syntax uses colons. In time, I could see an argument for also expanding the single-letter abbreviations in this mode to make a more verbose syntax. There's also no reason why we couldn't add a way of describing requests by manipulating an object, which then compiles down to the request language.
Anyway, this is all still evolving - if you have ideas for keeping the language compact but making it less like line-noise, drop me a line.
At the moment, my stance is that the syntax needs to be as terse as possible so that it can comfortably be specified in a URL, or in a snippet in a unit test. In the current master, there's a variant syntax that lets you use newlines (or arbitrary whitespace) wherever the more compact syntax uses colons. In time, I could see an argument for also expanding the single-letter abbreviations in this mode to make a more verbose syntax. There's also no reason why we couldn't add a way of describing requests by manipulating an object, which then compiles down to the request language.
Anyway, this is all still evolving - if you have ideas for keeping the language compact but making it less like line-noise, drop me a line.
What's the practical usage scenario for this? Would it be only used by http client library developers, or is there a broader audience?
I started work on pathod to improve the testing of mitmproxy (http://mitmproxy.org). It's since become a very capable Swiss army knife with many creative uses. I now routinely use it in pen tests (it's great for exploit delivery), and pathoc has become quite a capable tool for simple fuzzing and stress testing. I'm working on the next release docs as we speak, and I'll try to document a range of different use cases there.
Or you could hand examples of your favorite rogue web bot a whole pile of confusing chunder.