Go huge file download and passing to client (proxifying)

Question

I have a small Martini-based application and am faced with an issue that I can't solve.

I want to add an application feature that would allow the user to get files from a 3rd server with some changes in HTTP headers. Some kind of proxy. The files are quite big (200+Mb each) and I want these files to be "streamed" to the client. By "stream" I mean that file data should start to return to the client right after the first bytes were recieved by the application without keeping the whole file on disk and/or in memory.

Is this possible with Martini and Go?

Answer 1

Yes, it is possible in general with Go, I'm not that familiar with Martini specifically. The http response from calling the remote file returns a Reader interface, and your request handler has a Writer interface. This means you can read a stream of data, and write a stream of data. Making your responsibility only to manipulate what you want, and "patch" the forwarded stream to the request stream.

Go even has a ReverseProxy utility built into the standard library:

http://golang.org/pkg/net/http/httputil/#ReverseProxy

You can probably mix Martini and the standard http library if you want.

[EDIT] Reading the martini docs, you can add raw http handlers like the standrad library has, meaning you can indeed do that: https://github.com/codegangsta/martini#service-injection