Question

I was thinking of writing a simple agent based framework using Hadoop. This is an idea I have been toying around for quite a while now (but have not put a lot of deep thinking into).

The target application I want to use it for is Genetic Programming. The Idea is that I would spawn a lot of agents, have them perform a task, evaluate their genetic fitness and then breed them, and then repeat the whole exercise.

I wanted to know if folks had any immediate thoughts/critique of doing this using Hadoop.

Thanks in advance.

Was it helpful?

Solution

I think you should look into YARN of hadoop 2.0. It enables you to run any arbitrary logic not restricting you to MR paradigm

OTHER TIPS

It depends on how your agents really fit map/reduce approach. If results produced by agents will be consumed in reduce phase and new jobs will be initiated - you're ok with Hadoop. Buf if your agents depend on results of each other - you probably want to try out something like Storm ( https://github.com/nathanmarz/storm/wiki )

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top