Benchmarking Streaming Computation Engines at Yahoo(yahooeng.tumblr.com)

41 pointsby YAFZ11 anni fa3 comments

yahooeng.tumblr.com

Benchmarking Streaming Computation Engines at Yahoo

http://yahooeng.tumblr.com/post/135321837876/benchmarking-streaming-computation-engines-at

2 comments

kod11 anni fa

If I'm reading this correctly, the Kafka topic only had 5 partitions, but they had 10 workers.

With the Spark direct stream, kafka partitions are 1:1 with spark partitions, which means at most half of the workers would be doing work without a shuffle.

Seems like a pretty basic oversight that should be addressed.

estefan11 anni fa

This is the first mention I've seen of flink on HN.

luckydata11 anni fa

there were a few before but not many.