Rails + XMPP bot in background

Question

I'm building a service which basically allows users to chat with a bot, and the bot then does some strange processing with the chat sent by the user, and eventually reply back with some meaningful data. Basically something similar to how Aardvark used (?) to work.

I've the bot working, and listening right now, and I have a separate rails app that will be doing all the other heavy lifting. Both these parts are working fine individually, and now I'm stuck at interfacing the two. My idea is to interface the bot (which is basically a small ruby script) with the rails app via Resque - anything that comes in goes to a queue, gets picked up, and the results then pushed back again to the queue, and then the script would reply back with the results.

I'm not very clear as to how to establish this interface:

1. Do I need to write a rake task to start / stop / reload the bot
2. If I run it without rake (supposedly as an independent process monitored by Monit) then how do I interface with Resque or access my rails models ?

I know these might be very trivial questions, but I'm having a hard time understanding which works better, and how to get the setup going.

Answer 1

There are three ways to communicate between your Rails app and this bot daemon:

1. By calling the Rails app as an HTTP request (pushing/pulling data from Rails app)
2. By interacting directly with a database the Rails app uses (likely Mysql/Postgres)
3. By interacting with a Resque work queue system, backed by the Redis database

When you are enqueuing and pulling Resque jobs off various Job queues, you're just reading/writing to the shared Redis database through an API. Both the bot and the Rails app talk to the Redis DB over the network.

I recommend running the bot directly as a ruby process or rake task managed by monit. It sounds like you already know how to do this.

Answer 2

I think the main issue here is that you need another solution for messaging (IPC-like, not IM) rather than trying to bend Resque which is “just” a queue. Some of the options are amqp gem (AMQP protocol) or zmq gem (ZeroMQ protocol), but you can also use plain old UNIX sockets via Ruby standard library Socket class (good examples). They all have different pros and cons, so it’s up to you and your needs.

The interaction might look like something like this:

1. Bot starts.
2. Bot starts listening for IPC messages.
3. Bot receives a query from sender (via XMPP).
4. Bot queues a job via Resque.
5. Job calls the Rails app via HTTP.
6. Rails app does its share of work.
7. Someone or something resolves whatever was the query and enters the result via Rails app.
8. Rails app sends the result using some IPC method to the bot.
9. Bot sends the results to the original sender (via XMPP).

There can be some changes as usual. For example, I think you don’t need Resque at all. The bot can simply pass the request immediately to the Rails app. However, it depends on load, time to respond you want to achieve, you current architecture, etc. Maybe the Resque job can wait for the Rails app to return the result and then the job (not the Rails app) would use IPC. There are other variations…

Do I need to write a rake task to start / stop / reload the bot

No, you don’t. It is up to you how and when you run it. After all, Rake can be viewed just as a convenient way to put multiple Ruby scripts together and create dependencies between them. If you think there will be other tasks around the bot than just running it (some cleaning up, deployment, etc.), it will be good to use Rake for convenience. If you haven’t already, refactor bot’s logic to class and use Rake task to initialize it. But it will also be fine if you leave it, and just run your script as-is (using monit, your custom init.d script, ad-hoc, etc.).

If I run it without rake (supposedly as an independent process monitored by Monit) then how do I interface with Resque or access my rails models ?

Rake doesn’t have any effect on this. It doesn’t matter from OS perspective if you run Resque via Rake and your bot via Rake or as a standalone script. They will be different processes anyway. Also, keep in mind that Resque needs Redis to be running somewhere.

I know these might be very trivial questions

No at all. I think it’ll take some time before issues like could be considered trivial.

Answer 3

You may put your code to run on an initializer and have full access to all your Rails models or libs.

This way, you don't need to "communicate" between you bot and your Rails App, because your bot is inside your Rails App.

Boilerplate code would be like:

config/initializers/background_app_tasks.rb

class BackgroundWorker

      #-------------------------------
      def initialize(operation='normal')
        @exit = false
        @lock = Mutex.new  # For thread safety
        @thread = nil
        say "Starting in '#{operation}' mode..."
        case operation
          when 'normal'
            @thread = Thread.new() {    loopme     }
          when 'cleanup'
            @thread = Thread.new() {    cleanup     }
          when 'nothing'
            #startup without threads
        end
        @thread.run if @thread
      end

      #-------------------------------
      def exit!
        begin
            return if @exit # #stop?
            say   "Exiting #{}, waiting for mutex..."
            @lock.synchronize {
                say "exiting thread #{@thread.to_s || '<sem nome>' }..."
                @exit = true # #stop
            }
        rescue Exception => e
            exceptme(e)
        end
      end

      #-------------------------------
      def loopme

        at_exit { exit! }
        i=0;  ok=false;

        nap = 30

        while true do
          begin
              break if @exit
              i+=1

              #lock mutex for processing...
              @lock.synchronize {

                  #.... do some work ....

              }
          rescue StandardError => e

              #....

          end

          sleep(nap)
        end 
      end

end #class

# ------ M A I N --------

Thread.abort_on_exception=false
e = BackgroundWorker.new(OPERATION)