Concurrency database connections Heroku Unicorn rails app

Question

I'm running a rails app on Heroku.

I have one dyno. I'm using the Hobby Basic database, the $9 a month one, with a connection limit of 20.

My app is running on Unicorn. But it's still really slow when multiple database calls are being made.

This is what I have in my unicorn.rb file:

# config/unicorn.rb
worker_processes 3
timeout 30
preload_app true

before_fork do |server, worker|

  Signal.trap 'TERM' do
    puts 'Unicorn master intercepting TERM and sending myself QUIT instead'
    Process.kill 'QUIT', Process.pid
  end

  defined?(ActiveRecord::Base) and
    ActiveRecord::Base.connection.disconnect!
end

after_fork do |server, worker|

  Signal.trap 'TERM' do
    puts 'Unicorn worker intercepting TERM and doing nothing. Wait for master to sent QUIT'
  end

  defined?(ActiveRecord::Base) and
    ActiveRecord::Base.establish_connection
end

This article talks about managing the concurrent connections: https://devcenter.heroku.com/articles/concurrency-and-database-connections

But, i'm still confused. With my current setup, how do I allow for multiple database connections at the same time? And at the maximum connections my database allows (20)? I would really appreciate it if someone here that has dealt with scaling a rails app on Heroku could point me in the right direction.

Answer 1

Its fair to make an assumption that the database could be a bottleneck when you use the limited hobby plan. There are likely some optimizations you can do to help improve connection pooling regardless ( like code below). However when this is truly the bottleneck, you would expect to find timeouts occuring often, not just slowness. In order to get a better sense of where your issues lie, I recommend adding the NewRelic plugin next.

The NewRelic Stark plan (recommended) - or the Wayne plan ( a bit too limited in my opinion) is free and will help visualize you Apps performance and trace Errors. You will likely find that dyno queuing time and memory consumption is contributing to the slowness you are getting, and there are a few ways this could improve without ever messing with database connections. Improving your code response time and limiting your database hits (via caching) are good areas to investigate.

With that said, I do recommend adding the following code to your unicorn file as described in the article you mention.

after_fork do |server, worker|

  Signal.trap 'TERM' do
    puts 'Unicorn worker intercepting TERM and doing nothing. Wait for master to sent QUIT'
  end

  # other settings
  if defined?(ActiveRecord::Base)
    config = ActiveRecord::Base.configurations[Rails.env] ||
            Rails.application.config.database_configuration[Rails.env]
    config['reaping_frequency'] = ENV['DB_REAP_FREQ'] || 10 # seconds
    config['pool']            =   ENV['DB_POOL'] || 2
    ActiveRecord::Base.establish_connection(config)
  end
end

Answer 2

Heroku provides managed Postgres databases. Different tiered databases have different connection limits. The Starter Tier “Dev” and “Basic” databases are limited to 20 connections.

and one connect required by on worker. if you have workers more than 20, you can use connection pool like pgbouncer deal the problem. if you use pgbouncer , you shold use transaction pool mode in your environment so server can reused between workers.