Faraz, thank you so much for the hint on this. Here is the solution and the blog that helped me find the answer
http://blog.seancarpenter.net/2013/09/02/rails-ssl-route-generation-with-nginx-and-unicorn/
I had to tell nginx to keep the https. This is why some of the time it would work and sometimes it would not.
location @unicorn {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://unicorn;
}
Had to be changed to
location @unicorn {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto https; # New header for SSL
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://unicorn_something_server;
}