Internal File Server Design Patterns?

Question

My company processes data that comes from our clients. Our clients submit data by pushing it to an sftp server we maintain. Our code (mostly in Java, Scala and Node.js with a little of Python/shell) then hits this servers in read-only mode and does bunch of data processing after downloading these files.

We are frustrated by this system because not only Java libraries aren't that great for doing sftp but even the node one does not work in Windows which some developers use so we have some hacky shell scripts to sync files from sftp.

What are some good architectural patterns to solve this problem? We want a secure internal file server that can be accessed easily by good OS agnostic libraries and that has good tools for browsing through a file-viewer?

Answer 1

Expanding on Brad's question, "why make you internal interface SFTP"? ...

I'd run rsync against your SFTP server to keep a working copy of the client files in your preferred environment. From there go with whatever tools make your crew most comfortable.