Streaming via SOAP?

Question

I have WS where in the SOAP payload there is a xsd:base64Binary element.

Initially this was designed for max 1MB but the requirement changed and now it should accept 50 MB uploaded files converted to base64 :( In java this becomes HUGE in terms of memory.

My theoretical solution is to stream from one end to the other these 'attachments' (files are uploaded from the web app, then converted to base64 and a web service is called to store in some Adobe app) - very inefficient i say

I read about Stax but that is not for WebServices.

Is there a way to use streaming in combination with Webservices ?

( MTOM ,but did not find a sample with streaming)

Answer 1

I think you could do this with a custom servlet and overriding the post method.

Read enough of the request's InputStream through Stax to find the element containing the base64, then pipe it to some sort of Base64InputStream until you reach the end of the element. Use Stax or JaxB to transform an object into a response and send it back.

Whatever you do, don't buffer up the entire request. Don't use a typical soap framework like CXF or JAXWS without thinking about how it handles buffering. You're going to hit your garbage collector so hard it'll panic, run away, never call you again.

EDIT: Garbage collection

If you're running a 4gb heap, and assuming you're using HotSpot, make sure you're using the latest Java from Oracle (7u5). Next, switch to a collector that's meant to handle large heaps: either g1gc (-XX:+UseG1GC) or CMS (-XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode)

In addition, consider adding the following:

-XX:+OptimizeStringConcat -XX:+AggressiveOpts -XX:+UseFastAccessorMethods -XX:+UseLargePages -XX:+UseStringCache -XX:+UseCompressedOops