Extreme dogfooding(blog.rainforestqa.com)
blog.rainforestqa.com
Extreme dogfooding
http://blog.rainforestqa.com/2013-11-04-extreme-dogfooding/
6 comments
Your comment was thoroughly written and informative, thanks. Not sure why you've been down-voted. Is discussing meta aspects of submissions frowned upon?
I would think blog writers would like to see tips on how to write well.
I'd be interested to see if the author of the original post revises his entry.
I would think blog writers would like to see tips on how to write well.
I'd be interested to see if the author of the original post revises his entry.
I like the product, but I think this blog post would've been much more interesting if it wasn't written in a "omg we're so aweome we're using our own product and we wrote a script that makes it difficult not to! disruptive!"-way.
Instead, maybe, what trouble did you have when dogfooding? Did you ever run in the situation that you couldn't proceed because of a bug in the then-being-dogfooded software? How did you get by it? Did engineers like it? How about non-engineer team members? Did the dogfooding influence the way you prioritized?
Sure, I like that the second part of the blog post says "we found some new requirements because of this", but for me, as a programmer of another piece of software, it would've been much more interesting to find out how you got to those requirements and how you dealt with them, rather than what the new requirements were. I mean, every software team finds new requirements all the time.
I mean to be constructive here: I strongly doubt blog posts about how awesome your own tool is and how using it yourself made it awesomer is the best approach. By turning it less into promo and more into general experience-sharing, the post may be more generally useful and thus spread further.
Instead, maybe, what trouble did you have when dogfooding? Did you ever run in the situation that you couldn't proceed because of a bug in the then-being-dogfooded software? How did you get by it? Did engineers like it? How about non-engineer team members? Did the dogfooding influence the way you prioritized?
Sure, I like that the second part of the blog post says "we found some new requirements because of this", but for me, as a programmer of another piece of software, it would've been much more interesting to find out how you got to those requirements and how you dealt with them, rather than what the new requirements were. I mean, every software team finds new requirements all the time.
I mean to be constructive here: I strongly doubt blog posts about how awesome your own tool is and how using it yourself made it awesomer is the best approach. By turning it less into promo and more into general experience-sharing, the post may be more generally useful and thus spread further.
Great feedback dude! Thanks. We will try and take this approach in the future.
So hold on, I don't understand why you're calling it extreme, seems just like what everyone else would do to get their product working.
The difference is that because we're using Rainforest in our build process we cannot deploy if Rainforest itself doesn't work... so it's a bit more 'extreme' than regular dogfooding.
Then I guess we used "extreme dogfooding" with our travel company. We colocated other roles with our offshore developers and we used our own site (http://AllTheRooms.com) to find accommodations for designers and product managers who traveled to be with the devs in Medellin, Colombia.
So, if the site didn't work it would have been pretty painful not having a place to stay.
So, if the site didn't work it would have been pretty painful not having a place to stay.
Exactly.
That sounds exactly like dogfooding to me.
What is regular dogfooding?
What is regular dogfooding?
In regular dogfooding, you would give up if your product was so bad it prevented you from working on anything. I think the extreme here means they are trying to make failures as painful for themselves as possible, even if doing so makes it harder to resolve those failures.
It's probably instructive to think about the type of product that you'd be dogfooding.
For most startups, dogfooding is a process of heavily using your own product to figure out what works and what doesn't and better understand the painpoints in the product from your customer's perspective.
That process changes quite significantly - what we're calling 'extreme' dogfooding - when the product you are dogfooding is a crucial part of your build process, because you cannot deploy if the product isn't working properly, whereas with most products that aren't developer tools a problem found through dogfooding isn't going to do something as drastic as preventing your deploy :)
For most startups, dogfooding is a process of heavily using your own product to figure out what works and what doesn't and better understand the painpoints in the product from your customer's perspective.
That process changes quite significantly - what we're calling 'extreme' dogfooding - when the product you are dogfooding is a crucial part of your build process, because you cannot deploy if the product isn't working properly, whereas with most products that aren't developer tools a problem found through dogfooding isn't going to do something as drastic as preventing your deploy :)
The nature of your product means that you are able to use the product as part of creating the product. Much like how a trucking company uses its own trucks to deliver the parts needed to build/repair those same trucks. Or how gcc is used to build gcc. What you call "extreme dogfooding" is more like half plain old dogfooding and half bootstrapping.
It's got nothing to do with bootstrapping.
It's quite similar: http://en.wikipedia.org/wiki/Bootstrapping_(compilers)
The process of getting a compiler to compile itself is not unlike your process of using Rainforest as part of the build process for Rainforest.
The process of getting a compiler to compile itself is not unlike your process of using Rainforest as part of the build process for Rainforest.
Most people don't do it to the extent it stops deploys working. We wanted to as if it's painful for us, it would be painful for our customers.
This seems pretty interesting, but even after reading through your docs, I'm left with a few questions.
How long am I going to wait to get a tester? Does this change over the course of a day? Do you use more than one tester to consider something a 'fail'?
How long am I going to wait to get a tester? Does this change over the course of a day? Do you use more than one tester to consider something a 'fail'?
Hey Adrian, Fred, co-founder here. Yeah, our docs need some love :)
It's totally seamless, so as soon as you run a test we allocate (multiple) human testers automatically and your results come back within 30 mins on average. The median time to being allocated a first tester for a test is (in the last 30 days) is 2min 29s.
We have a massive crowd of testers available, so in general response time is fairly consistent.
The majority of our tech is on the backend for scoring and sorting results and testers. There's a bunch of sorting that happens in the background between a tester submitting a 'result' and Rainforest verifying that result and returning it to the customer.
Does that answer your questions?
It's totally seamless, so as soon as you run a test we allocate (multiple) human testers automatically and your results come back within 30 mins on average. The median time to being allocated a first tester for a test is (in the last 30 days) is 2min 29s.
We have a massive crowd of testers available, so in general response time is fairly consistent.
The majority of our tech is on the backend for scoring and sorting results and testers. There's a bunch of sorting that happens in the background between a tester submitting a 'result' and Rainforest verifying that result and returning it to the customer.
Does that answer your questions?
Note to Rainforest team: When I signed up for the free trial I was sent 3 identical confirmation emails. Something going on there.
Overall your product looks good, definitely going to keep it in mind.
Overall your product looks good, definitely going to keep it in mind.
Thanks! I'd love to look in to that for you. Could you reply to them and I'll check it out?
Done.
I've fixed an am deploying now, was a race condition with two requests happening at the same time. The code to create the user via segment.io on intercom was being fired inside a transaction, which was rolled back. I've moved to an after_commit block instead :)
Did you write a regression test for it? :)
What blog theme did you use for this? I like it.
Thanks Sasha! It's actually custom, based on a bunch of technologies though. I'm going to do a blog post this week about it, and we've open-sourced the template at https://github.com/rainforestapp/middleman-boilerplate. The stack is:
- Scss
- Susy
- Compass
- Haml
- Middleman
- S3
- Scss
- Susy
- Compass
- Haml
- Middleman
- S3
> Confident that our product is fast and accurate enough to solve real-world QA needs, this has been a long journey and many things helped us get to this point.
This sentence has a very common grammatical error called a dangling modifier. In this case, notice that the adjective (i.e. noun-modifier) "Confident" is intended to modify the pronoun "us." But, if we look at the sentence, the subject of the sentence is the pronoun "this"! Because introductory phrases (from Confident... to the first comma) are naturally attached to the subject of the sentence, it sounds like the target of "this", namely "the long journey", is the target of the modifier.
Generally you can catch these by reading your work aloud. Once you read, "Confident that [...], this was a long journey...", hopefully it will throw some red flags that this sentence "doesn't make sense." Another technique (of error location) is shortening the sentence to make the error more pronounced: Confident that we're awesome, this journey has taken us far.
Potential rewordings (of many):
It has been a long journey and many things helped us get to a point where we are confident that our product is fast and accurate enough to solve real-world QA needs.
Confident that our product is fast and accurate enough to solve real-world QA needs, we're reaching a milestone in our long journey.
If we also try to introduce some brevity:
After a long journey, we're confident that our product is fast and accurate enough for real-world QA.
[0]: http://xkcd.com/356/
Edited to add: this is meant to be helpful to the poster, who also wrote the post (I assume). It's not a commentary on their product, since as I said, I never did get past the first paragraph.