OutofmemoryError while uploading a hugefile using cxf webservices

Question

I wrote an webservice using cxf and JAX-WS which runs on Jetty Server. Enabled MTOM in both client and server. I am trying to upload an large file (>2GB). I am trying to upload a file which is of size 2572848KB and from client Application i am trying to upload the file to server. Upto 2572847KB successfully file got uploaded successfully. Got java.lang.outofmemoryerror in client side and some exceptions in server side too. I tried by repeating the steps and faced the same exception in the final stage of uploading.

Client Side Exception:

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Unknown Source)
    at java.lang.AbstractStringBuilder.expandCapacity(Unknown Source)
    at java.lang.AbstractStringBuilder.ensureCapacityInternal(Unknown Source)
    at java.lang.AbstractStringBuilder.append(Unknown Source)
    at java.lang.StringBuilder.append(Unknown Source)
    at org.apache.cxf.io.CachedOutputStream.writeCacheTo(CachedOutputStream.java:350)
    at org.apache.cxf.io.CachedOutputStream.writeCacheTo(CachedOutputStream.java:332)
    at org.apache.cxf.io.CachedOutputStream.writeCacheTo(CachedOutputStream.java:297)
    at org.apache.cxf.interceptor.AbstractLoggingInterceptor.writePayload(AbstractLoggingInterceptor.java:125)
    at org.apache.cxf.interceptor.LoggingOutInterceptor$LoggingCallback.onClose(LoggingOutInterceptor.java:142)
    at org.apache.cxf.io.CachedOutputStream.close(CachedOutputStream.java:182)
    at org.apache.cxf.transport.AbstractConduit.close(AbstractConduit.java:56)
    at org.apache.cxf.transport.http.HTTPConduit.close(HTTPConduit.java:640)
    at org.apache.cxf.interceptor.MessageSenderInterceptor$MessageSenderEndingInterceptor.handleMessage(MessageSenderInterceptor.java:62)
    at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:263)
    at org.apache.cxf.endpoint.ClientImpl.doInvoke(ClientImpl.java:519)
    at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:449)
    at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:352)
    at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:304)
    at org.apache.cxf.frontend.ClientProxy.invokeSync(ClientProxy.java:88)
    at org.apache.cxf.jaxws.JaxWsClientProxy.invoke(JaxWsClientProxy.java:134)
    at com.sun.proxy.$Proxy34.uploadFile(Unknown Source)

Server side exception:

Apr 26, 2014 5:08:50 PM org.apache.cxf.phase.PhaseInterceptorChain doDefaultLogging
WARNING: Interceptor for {http://testupload.java.com/}TestUploadServerImplService#{http://testupload.java.com/}uploadFile has thrown exception, unwinding now
org.apache.cxf.binding.soap.SoapFault: Error writing to XMLStreamWriter.
    at org.apache.cxf.binding.soap.interceptor.SoapOutInterceptor$SoapOutEndingInterceptor.handleMessage(SoapOutInterceptor.java:288)
    at org.apache.cxf.binding.soap.interceptor.SoapOutInterceptor$SoapOutEndingInterceptor.handleMessage(SoapOutInterceptor.java:270)
    at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:263)
    at org.apache.cxf.interceptor.AbstractFaultChainInitiatorObserver.onMessage(AbstractFaultChainInitiatorObserver.java:105)
    at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:323)
    at org.apache.cxf.interceptor.OutgoingChainInterceptor.handleMessage(OutgoingChainInterceptor.java:77)
    at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:263)
    at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121)
    at org.apache.cxf.transport.http_jetty.JettyHTTPDestination.serviceRequest(JettyHTTPDestination.java:319)
    at org.apache.cxf.transport.http_jetty.JettyHTTPDestination.doService(JettyHTTPDestination.java:287)
    at org.apache.cxf.transport.http_jetty.JettyHTTPHandler.handle(JettyHTTPHandler.java:72)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:939)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:875)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
    at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:247)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:110)
    at org.eclipse.jetty.server.Server.handle(Server.java:346)
    at org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:589)
    at org.eclipse.jetty.server.HttpConnection$RequestHandler.content(HttpConnection.java:1065)
    at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:915)
    at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:220)
    at org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:411)
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:535)
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:40)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:529)
    at java.lang.Thread.run(Unknown Source)
Caused by: com.ctc.wstx.exc.WstxIOException: null
    at com.ctc.wstx.sw.BaseStreamWriter._finishDocument(BaseStreamWriter.java:1402)
    at com.ctc.wstx.sw.BaseStreamWriter.writeEndDocument(BaseStreamWriter.java:532)
    at org.apache.cxf.binding.soap.interceptor.SoapOutInterceptor$SoapOutEndingInterceptor.handleMessage(SoapOutInterceptor.java:282)
    ... 25 more
Caused by: org.eclipse.jetty.io.EofException
    at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:149)
    at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:96)
    at org.apache.cxf.io.AbstractWrappedOutputStream.write(AbstractWrappedOutputStream.java:46)
    at com.ctc.wstx.io.UTF8Writer.flush(UTF8Writer.java:100)
    at com.ctc.wstx.sw.BufferingXmlWriter.flush(BufferingXmlWriter.java:225)
    at com.ctc.wstx.sw.BufferingXmlWriter.close(BufferingXmlWriter.java:198)
    at com.ctc.wstx.sw.BaseStreamWriter._finishDocument(BaseStreamWriter.java:1400)
    ... 27 more

In some of the post they mentioned that size of the server response is huge. Any help is appreciated.

client code:

JaxWsProxyFactoryBean factory = new JaxWsProxyFactoryBean();
Map<String, Object> props = new HashMap<String, Object>();
props.put("mtom-enabled", Boolean.TRUE);
factory.setProperties(props);
long timeout = 10000L;

factory.setServiceClass(TestUploadServer.class);
factory.setAddress("http://localhost:8080/TestUploadServer/services/TestUploadServerImplPort");
// logging feature with size limit and output destination.
factory.getFeatures().add(new LoggingFeature(1024));
// factory.getFeatures().add(new LoggingFeature("<stdout>",
// "location"));
// factory.getInInterceptors().add(new LoggingInInterceptor());
// factory.getOutInterceptors().add(new LoggingOutInterceptor());
TestUploadServer uploadWebservice = (TestUploadServer) factory.create();

Client client = ClientProxy.getClient(uploadWebservice);
if (client != null) {
    HTTPConduit conduit = (HTTPConduit) client.getConduit();
    HTTPClientPolicy policy = new HTTPClientPolicy();
    policy.setConnectionTimeout(0);
    policy.setReceiveTimeout(0);
    conduit.setClient(policy);
}
FileUploader file = new FileUploader();

file.setName(FILENAME);
file.setFileType(FILETYPE);

DataSource source = new FileDataSource(new File(FILEPATH));
file.setDfile(new DataHandler(source));
uploadWebservice.uploadFile(file);
System.out.println("Finished Copying......");

Answer 1

Logginginterceptor is causing problem in my client's code. Once server resposes with the attachment, the attachment is placed in my memory which is huge. Because of this i am getting outofmemory error. Just remove the logging interceptor client's code will work.Tried with 12 GB and works fine...