akshayubhat

     If 15% of the unemployed engineers in Corvallis could start companies that hired 5 full timers, they would (probably) have a booming economy all over again. Shit, maybe we could start exporting to China (I think they value entrepeneurship and industrialization there...).

       While the standard big data stack has made huge strides in making big data more accessible to everyone, it will always fall short against our stack when it comes to the cost of collecting data.  We actually don’t store that much data. Because 80legs users can filter their data on the nodes, they’re able to return the minimum amount of data in their result sets.  The processing (or reduction, pardon the pun) is done on the nodes.  Actual result sets are very small relative to the size of the input set.

      One advantage is optimization — an “off-the-shelf” system is going to have some generalities built into it that can’t be optimized to fit your needs. The opportunity cost of going “standard” is a slew of competitive advantages.