504 Gateway timeout error - my rails site won't run

Question

I've set up my rails application to be deployed on a digitalocean droplet using capistrano. However for some odd reason no matter what I do I keep getting a 504 Gateway Timeout nginx error. I've tried restarting unicorn but the error still persists.

I've included here the last 50 lines of the unicorn.log file - what could be the problem here?

/usr/local/rvm/rubies/ruby-1.9.3-p429/lib/ruby/1.9.1/open-uri.rb:29:in `initialize'
/usr/local/rvm/rubies/ruby-1.9.3-p429/lib/ruby/1.9.1/open-uri.rb:29:in `open'
/usr/local/rvm/rubies/ruby-1.9.3-p429/lib/ruby/1.9.1/open-uri.rb:29:in `open'
/usr/local/rvm/rubies/ruby-1.9.3-p429/lib/ruby/1.9.1/open-uri.rb:29:in `open'
/usr/local/rvm/rubies/ruby-1.9.3-p429/lib/ruby/1.9.1/logger.rb:592:in `open_logfile'
/usr/local/rvm/rubies/ruby-1.9.3-p429/lib/ruby/1.9.1/logger.rb:549:in `initialize'
/usr/local/rvm/rubies/ruby-1.9.3-p429/lib/ruby/1.9.1/logger.rb:314:in `new'
/usr/local/rvm/rubies/ruby-1.9.3-p429/lib/ruby/1.9.1/logger.rb:314:in `initialize'
/home/etyme/shared/bundle/ruby/1.9.1/gems/activesupport-3.2.13/lib/active_support/core_ext/logger.rb:72:in `initialize'
/home/etyme/releases/20140120142015/config/environments/development.rb:53:in `new'
/home/etyme/releases/20140120142015/config/environments/development.rb:53:in `block in <top (required)>'
/home/etyme/shared/bundle/ruby/1.9.1/gems/railties-3.2.13/lib/rails/railtie/configurable.rb:24:in `class_eval'
/home/etyme/shared/bundle/ruby/1.9.1/gems/railties-3.2.13/lib/rails/railtie/configurable.rb:24:in `configure'
/home/etyme/releases/20140120142015/config/environments/development.rb:1:in `<top (required)>'
/home/etyme/shared/bundle/ruby/1.9.1/gems/activesupport-3.2.13/lib/active_support/dependencies.rb:251:in `require'
/home/etyme/shared/bundle/ruby/1.9.1/gems/activesupport-3.2.13/lib/active_support/dependencies.rb:251:in `block in require'
/home/etyme/shared/bundle/ruby/1.9.1/gems/activesupport-3.2.13/lib/active_support/dependencies.rb:236:in `load_dependency'
/home/etyme/shared/bundle/ruby/1.9.1/gems/activesupport-3.2.13/lib/active_support/dependencies.rb:251:in `require'
/home/etyme/shared/bundle/ruby/1.9.1/gems/railties-3.2.13/lib/rails/engine.rb:571:in `block in <class:Engine>'
/home/etyme/shared/bundle/ruby/1.9.1/gems/railties-3.2.13/lib/rails/initializable.rb:30:in `instance_exec'
/home/etyme/shared/bundle/ruby/1.9.1/gems/railties-3.2.13/lib/rails/initializable.rb:30:in `run'
/home/etyme/shared/bundle/ruby/1.9.1/gems/railties-3.2.13/lib/rails/initializable.rb:55:in `block in run_initializers'
/home/etyme/shared/bundle/ruby/1.9.1/gems/railties-3.2.13/lib/rails/initializable.rb:54:in `each'
/home/etyme/shared/bundle/ruby/1.9.1/gems/railties-3.2.13/lib/rails/initializable.rb:54:in `run_initializers'
/home/etyme/shared/bundle/ruby/1.9.1/gems/railties-3.2.13/lib/rails/application.rb:136:in `initialize!'
/home/etyme/shared/bundle/ruby/1.9.1/gems/railties-3.2.13/lib/rails/railtie/configurable.rb:30:in `method_missing'
/home/etyme/releases/20140120142015/config/environment.rb:5:in `<top (required)>'
/usr/local/rvm/rubies/ruby-1.9.3-p429/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require'
/usr/local/rvm/rubies/ruby-1.9.3-p429/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require'
config.ru:4:in `block in <main>'
/usr/local/rvm/rubies/ruby-1.9.3-p429/lib/ruby/gems/1.9.1/gems/rack-1.4.5/lib/rack/builder.rb:51:in `instance_eval'
/usr/local/rvm/rubies/ruby-1.9.3-p429/lib/ruby/gems/1.9.1/gems/rack-1.4.5/lib/rack/builder.rb:51:in `initialize'
config.ru:1:in `new'
config.ru:1:in `<main>'
/usr/local/rvm/rubies/ruby-1.9.3-p429/lib/ruby/gems/1.9.1/gems/unicorn-4.6.3/lib/unicorn.rb:48:in `eval'
/usr/local/rvm/rubies/ruby-1.9.3-p429/lib/ruby/gems/1.9.1/gems/unicorn-4.6.3/lib/unicorn.rb:48:in `block in builder'
/usr/local/rvm/rubies/ruby-1.9.3-p429/lib/ruby/gems/1.9.1/gems/unicorn-4.6.3/lib/unicorn/http_server.rb:722:in `call'
/usr/local/rvm/rubies/ruby-1.9.3-p429/lib/ruby/gems/1.9.1/gems/unicorn-4.6.3/lib/unicorn/http_server.rb:722:in `build_app!'
/usr/local/rvm/rubies/ruby-1.9.3-p429/lib/ruby/gems/1.9.1/gems/unicorn-4.6.3/lib/unicorn/http_server.rb:595:in `init_worker_process'
/usr/local/rvm/rubies/ruby-1.9.3-p429/lib/ruby/gems/1.9.1/gems/unicorn-4.6.3/lib/unicorn/http_server.rb:615:in `worker_loop'
/usr/local/rvm/rubies/ruby-1.9.3-p429/lib/ruby/gems/1.9.1/gems/unicorn-4.6.3/lib/unicorn/http_server.rb:500:in `spawn_missing_workers'
/usr/local/rvm/rubies/ruby-1.9.3-p429/lib/ruby/gems/1.9.1/gems/unicorn-4.6.3/lib/unicorn/http_server.rb:511:in `maintain_worker_count'
/usr/local/rvm/rubies/ruby-1.9.3-p429/lib/ruby/gems/1.9.1/gems/unicorn-4.6.3/lib/unicorn/http_server.rb:277:in `join'
/usr/bin/unicorn:121:in `<main>'
E, [2014-01-21T05:42:51.210093 #30994] ERROR -- : reaped #<Process::Status: pid 31079 exit 1> worker=0
I, [2014-01-21T05:42:51.210263 #30994]  INFO -- : worker=0 spawning...
I, [2014-01-21T05:42:51.213421 #31093]  INFO -- : worker=0 spawned pid=31093
I, [2014-01-21T05:42:51.228866 #31093]  INFO -- : Refreshing Gem list
/usr/local/rvm/rubies/ruby-1.9.3-p429/lib/ruby/gems/1.9.1/gems/bundler-1.5.1/lib/bundler/runtime.rb:220: warning: Insecure world writable dir /home/etyme/releases in PATH, mode 040777
/usr/local/rvm/rubies/ruby-1.9.3-p429/lib/ruby/gems/1.9.1/gems/bundler-1.5.1/lib/bundler/runtime.rb:220: warning: Insecure world writable dir /home/etyme/releases in PATH, mode 040777

This is my nginx conf

user www-data;
worker_processes 4;
pid /var/run/nginx.pid;

events { worker_connections 1024; }

http {
        sendfile on;
        tcp_nopush on;
        tcp_nodelay on;
        keepalive_timeout 65;
        types_hash_max_size 2048;
        server_tokens off;

        # server_names_hash_bucket_size 64;
        # server_name_in_redirect off;

        include /etc/nginx/mime.types;
        default_type application/octet-stream;

        access_log /var/log/nginx/access.log;
        error_log /var/log/nginx/error.log;

        gzip on;
        gzip_disable "msie6";
        gzip_types text/plain text/xml text/css text/comma-separated-values;
        upstream app_server { server 127.0.0.1:8080 fail_timeout=0; }

        include /etc/nginx/conf.d/*.conf;
        include /etc/nginx/sites-enabled/*;
}

Answer 1

You're opening an HTTP connection to another site that is timing out. Because that site is not failing, but waiting to properly connect from code within your app, your app request never completes and request to your app now times out.

There are several things you can do to fix this:

• Fix your configuration. Perhaps you have the host/port wrong on this connection? You're hitting the wrong place and it's just hanging instead of failing
• Set a proper HTTP timeout of 10-20sec on this connection. Fail gracefully when it times out.
• Remove this code entirely, since the resource you're trying to use doesn't appear stable