How to combine gems from multiple Gemfiles to run one Ruby program?

Question

My Question

Suppose I have a directory structure like this:

app/
    core/
        bin/
            runner
        Gemfile
        ...
    Gemfile
    lib/

The "core" is an application which has its own Gemfile and Gemfile.lock. I don't want to modify core in any way. app/Gemfile is part of my plugin to core, which also has a Gemfile (listing its own dependencies that are additional to core/Gemfile).

I can "bundle install" from the app/core/ and app/ directories.

How can I run core/bin/runner from the app/ directory in such a way that it will include all the Ruby gems from both app/core/Gemfile and app/Gemfile?

Background

I am writing a plugin for Logstash, using C Ruby. Logstash includes its own Gemfile; once all dependencies are fetched, the total package size is about 40MB.

I want to run Logstash in Heroku. To avoid putting 40MB of stuff into Git, I have forked the Ruby buildpack (https://github.com/inscitiv/heroku-buildpack-logstash) and modified it to download Logstash, unpack it, and use its Gemfile.

This works fine, but I am stuck with the Gemfile provided by Logstash. For my plugins, I want to add new dependencies that my plugins will use; I don't really want to fork Logstash and change its Gemfile in order to accomplish this.

Instead, I want to unpack Logstash into its own directory (logstash/), and then I want to overlay my plugin code, including Gemfile for dependencies, on top of it. Then I want to run a Heroku "worker" process which will run logstash, specifying "." as the plugins directory, and has access to all the gems from both Gemfiles.

Answer 1

The answer is that the top-level Gemfile should include the child directory Gemfiles.

Something like this (from Redmine 2.0 Gemfile)

# Load plugins' Gemfiles
Dir.glob File.expand_path("../plugins/*/Gemfile", __FILE__) do |file|
  puts "Loading #{file} ..." if $DEBUG # `ruby -d` or `bundle -v`
  instance_eval File.read(file)
end

Answer 2

For bundler to work at getting all gem dependencies right, it really needs to have them all in one file. Could you make your plugin into a gem and have your deploy routine add one gem to the parent Gemfile? Your own gem can then list dependencies that would get incorporated into the whole.