What's your QA environment for Hadoop?

Question

I am trying to build a small QA environment for my product testing which uses Hadoop. My requirements are small, so I am guessing a small 3-4 node cluster do. What I am concerned about is not just the hardware specfiication for building this environment, but the best practices and other tools that should be incorporated from infrastructure and practices perspective. Say you were to start from scratch - what would you QA env have? Here's is what I can think of 1. 3 node serverclass machines, 2. bug tracking tools, code coverage tools, 3. continous integration server 4. MRUnit?)

Answer 1

Here are some articles 1 2 3 to start with for Hadoop hardware sizing.

Answer 2

I am using a combination of MRUnit & ApprovalTests to handle my unit testing part of this scenario. It works seamlessly with my CI.

There's a video on it here: http://www.youtube.com/watch?v=Pjroyrh_gNY