@oellegaard: If You had issues with manual then puppet modules + setup will be the same. If You just want to try You can use Cloudera Manager to setup cloud without any knowledge of hadoop. You just need to have ssh using keys as root on all hosts from host where cloudera manager is installed and it will just happen in seconds.
Agree in regards of Cloud awareness but as in my case it was in house Cloud which amount of nodes is not changed so frequently. Anyway when You add new nodes You have to specify them inside configuration so rack awareness also can be updated during that maintenance.
Regarding which distribution to use - there are lot of them now such as MapR, Hortonworks, Apache and Cloudera. All of them have their advantages and disadvantages but if You take support such we have with Cloudera then even special requests for incorporating fixes from Apache side is possible. And for OS it doesn't matter which to use but I would prefer CentOS because it is almost as Redhat (You do not need anything from RedHat that CentOS cannot give You for Hadoop cluster). Another thing is that CentOS is supported accross all of these vendors as usually it is made for RedHat and then works perfect on CentOS. Also there are lot of things that can improve Hadoop such as:
- Turn of swapping
- ulimit has to be at least 64K
- It is better to have more disks because of failure on one side and disk performances on another side. With having less disks than reading data become bottleneck and so on and so on
Thank You!
For locale yes as You said UTF-8 is standard and related to date and time it is better to use synchronization via NTP because so many troubles can make having different time on servers.
https://ccp.cloudera.com/display/DOC/Documentation#Documenta...