Configure flume in shell/bash script - avoid interactive flume shell console
Question
The normal way you configure flume is via flume master web console, way easy to talk about it here.
OR
via interactive flume shell console, follow the steps below:
1. $ flume shell //this brings you to the interactive flume shell console
2. in the interactive console,connect flume-master-node // this connects you to flume-master
3. in the interactive console, run "exec unconfig your_node" // this unconfig all flume configuration
4. in the interactive console, run "exec config your_node new_config" // apply new flume configuration
5. quit // exit the the interactive console
So far so good.
Then I am trying to write a bash script for my flume configuration. so I want to squeeze 1,2,3,4,5 into a bash and it runs automatically every time without intervention, something like this:
/usr/bin/flume shell << EOF #line1
connect $FLUME_MASTER #line2
exec unconfig your_node #line3
exec config your_node new_config #line4
quit #line5
EOF #line6
But everytime I run this bash script it always stops at #line1 and brings me to interactive flume shell console instead of running it sliently in a non-iteractive mode.
Does anybody know how to ignore the interactive mode and just run it sliently?
Solution 2
Somehow followed flume user guide flume/UserGuide_using_the_flume_command_shell:
echo "connect localhost:35873\ngetconfigs\nquit" | flume shell -q
this doesn't work if you put it in a bash script. But I got way with it by doing this:
/usr/bin/flume shell -q << EOF
connect localhost:35873
getconfigs
quit
EOF
OTHER TIPS
Try the -q
option to flume
to see if that helps.