serving logs via HTTP [closed]

Question

Here is what i am trying to achieve: I have a few Linux servers that have my web App deployed on. Currently my development team access the web app, run their test cases and then SSH into my Linux boxes to view or get the logs. I don't like them to be able to SSH/FTP into any servers and i am looking for a solution to get the logs to them via HTTP and HTTP only ( no Linux solutions such as jail, etc ).

Since the servers are already pretty slow and cannot really handle much more load on them, i decided to go with Python's SimpleHTTPServer. For every directory that has a log they will need, I basically create an index.html file that only has a download link to that log file and then i start a SimpleHTTPServer in that directory. I will need to start 3 SimpleHTTPServers on each box as there are 3 logs they need.

This works fine, except every now and then the http://serverurl:port url used to access one of the logs which sometimes gets around 700MB stops responding ( in google Chrome it says : no data recieved, in IE and FF it just shows a blank page ). At that point SimpleHTTPServer on that port is still up and shows up in processes running.

So far i have been just fixing this problem when it arises by killing and then starting the SimpleHTTPServer on that port, but i am looking for a permanent solution. The weird thing is that only happens with one of the logs and i have tired switching port numbers since i thought maybe there is a conflict or something.

Can anyone suggest a solution that uses HTTP, is as lightweight as SimpleHTTPServer and doesn't need this much maintenance.

Answer 1

Have you considered using a log aggregation and analysis solutions like Splunk? Instead of creating your own software that would require maintenance.

Answer 2

Flask and Bottle are popular choices for this kind of lightweight HTTP work.

At PyCon AU, Richard Jones compared and contrasted about ten such frameworks: http://www.youtube.com/watch?v=AYjPIMe0BhA His video is worth watching.