Netflix's first OSS project on Github(techblog.netflix.com)
techblog.netflix.com
Netflix's first OSS project on Github
http://techblog.netflix.com/2011/11/introducing-curator-netflix-zookeeper.html
2 comments
The lack of proper non-Java support in the Hadoop world is maddening, and Zookeeper is a prime example of this. It's a heterogenous world, people! How many people are missing out on Zookeeper because it lacks proper Python or PHP support?
Can't you use Hadoop with other JVM languages like Clojure or Scala?
That was my impression, but I haven't used Hadoop for anything nontrivial or in any language but Java, so I don't really know.
That was my impression, but I haven't used Hadoop for anything nontrivial or in any language but Java, so I don't really know.
Java is certainly the most common language ZK seems to be used with, but there are client bindings in Python, Perl and quite a few other languages:
https://cwiki.apache.org/confluence/display/ZOOKEEPER/ZKClie...
https://cwiki.apache.org/confluence/display/ZOOKEEPER/ZKClie...
Yes, they are binding to a C client, include the existing node-zookeeper:
https://github.com/yfinkelstein/node-zookeeper
The C client isn't what I would call great, and the API surface is fairly complicated for languages to bind to -- you see many reports about the bindings crashing their respective languages.
I'm currently hacking on a pure-Node.js implementation of the Apache Jute RecrodIO and Apache Zookeeper network protocol: https://github.com/racker/node-zk
The core problem is that Zookeeper is using a deprecated serialization layer as its network protocol. Jute predates Apache Thrift or Apache Avro, and it only has code-gen for C and Java. Ideally Zookeeper would add a second network protocol, one which was either a spec for a nice binary protocol, or a more modern serialization framework like Thrift or Avro, which support many more languages.
The ticket to change out Jute is here: https://issues.apache.org/jira/browse/ZOOKEEPER-102
But no one seems to actively be working on it.
The C client isn't what I would call great, and the API surface is fairly complicated for languages to bind to -- you see many reports about the bindings crashing their respective languages.
I'm currently hacking on a pure-Node.js implementation of the Apache Jute RecrodIO and Apache Zookeeper network protocol: https://github.com/racker/node-zk
The core problem is that Zookeeper is using a deprecated serialization layer as its network protocol. Jute predates Apache Thrift or Apache Avro, and it only has code-gen for C and Java. Ideally Zookeeper would add a second network protocol, one which was either a spec for a nice binary protocol, or a more modern serialization framework like Thrift or Avro, which support many more languages.
The ticket to change out Jute is here: https://issues.apache.org/jira/browse/ZOOKEEPER-102
But no one seems to actively be working on it.
The Python bindings are not very robust, don't fit the language well, and have a build process alien to the Python ecosystem. If it's representative of the other non-Java bindings, it's small wonder it doesn't get used by non-Java projects.
I second that. At dotCloud we had to pull away from Zookeeper because of the poor quality of the Python (and therefore C) bindings.
Nothing to say about Hadoop/Java homogeneity, but a Zookeeper alternative I've been looking at for the same reasons is Dozerd from Heroku (https://github.com/ha).
The thinking might be along the line: if you need to use Hadoop at all, you are probably already using Java anyway to maximize single machine performance.
Why Java? Why not C?
Personally, I've never looked to Java as a top performer. It's a middle language---not as expressive as Python, not as fast as C. Even compared to C#, I wouldn't naturally choose Java but I've been forced to because C# wasn't well supported away from a windows' box.
Now adays, I just skip the Java step for CPU intensive work and go to straight C integrated with Python.
Personally, I've never looked to Java as a top performer. It's a middle language---not as expressive as Python, not as fast as C. Even compared to C#, I wouldn't naturally choose Java but I've been forced to because C# wasn't well supported away from a windows' box.
Now adays, I just skip the Java step for CPU intensive work and go to straight C integrated with Python.
Your information is out of date. http://www.azulsystems.com/blog/cliff/2009-09-06-java-vs-c-p...
Why does the Netflix "Tech" Blog have the word tech in quotes? Is there some irony I'm missing?
Probably because it's an informal short form of "Technology." It does seem kind of weird as I'm used to seeing "Tech" used as a word in its own right but it's the likely reason why they did it.