Here's an older blog post that shows an example of streaming: http://patshaughnessy.net/2010/10/11/activerecord-with-large-result-sets-part-2-streaming-data
You might also have luck with the new Streaming API and Batches. If I'm reading the documentation correctly, you'd need to do your queries and output formatting in a view template rather than your controller in order to take advantage of the streaming.
As for gzipping, it looks like the most common way to do that in Rails is Rack::Deflator. In older versions of Rails, the Streaming API didn't play well Rack::Deflator. That might be fixed now, but if not that SO question has a monkey patch that might help.
Update
Here's some test code that's working for me with JRuby on Torquebox:
# /app/controllers/test_controller.rb
def index
respond_to do |format|
format.csv do
render stream: true, layout: false
end
end
end
# /app/views/test/index.csv.erb
<% 100.times do -%>
<%= (1..1000).to_a.shuffle.join(",") %>
<% end -%>
# /config/application.rb
module StreamTest
class Application < Rails::Application
config.middleware.use Rack::Deflater
end
end
Using that as an example, you should be able to replace your view code with something like this to render your CSV
Name,Created At
<% Model.scope.find_each do |model| -%>
"<%= model.name %>","<%= model.created_at %>"
<% end -%>
As far as I can tell, Rails will continue to generate the response if the user hits stop half-way through. I think this is a limitation with HTTP, but I could be wrong. This should meet the rest of your requirements, though.