Indeed that regex was problematic because it had a bug itself. We replaced that line by RegexSplitter and updated the benchmark page. Please shout if you notice something else wrong. Thanks.
Hi rjurney. I would say "Hive, Pig, Cascading" are on the higher level API side and "Pangool, MapReduce" on the low-level side. Pangool is a MapReduce API that aims to make MapReduce simpler. We explain this better in our FAQ: http://pangool.net/faq.html
What this means is that in Pangool, if you worked with 2-sized Tuples, you would be able to do exactly the same that you do now with Java MapReduce - That includes custom RawComparators and arbitrary business logic in any place of the MapReduce chain (Mapper, Combiner, Reducer). Using n-sized Tuples together with Pangool's group & sort by, reduce-side join API will only mean less code, easier code at no loss of performance or flexibility.
Realize that Pangool is still a MapReduce API so it doesn't add any level of abstraction.
We designed Pangool with the aim of offering it as a replacement of the current MapReduce API. Therefore we are not labelling it as a "higher-level API" but as comparable low-level API.
On the other hand we are also benchmarking Pangool to show it doesn't impose a performance overhead: http://pangool.net/benchmark.html