Author here: we scraped every script website on the Internet. First we tried to normalize the dataset but only doing stats on the top 1,000 box office, but we were missing too many scripts. So we decided to go big and then display a cut of the data that's only films in the top 2,500 box office (we had about half of those).
We're aware of sampling error and the potential for cherry-picking, but also struggled to figure out what was a representative sample.
This was omitted from the piece, but we count words and then convert to lines at ~10 words/line. When pop culture talks about dialogue in film, we use lines. So that felt more natural than "# of words spoken."
nope. when things are in an inactive tab...browsers slow down the JS and it crawls to a halt. So I just decided to stop it all together. You could have it in another window in the background though!