How to send email with large attach? (OutOfMemoryError: java heap space)

StackOverflow https://stackoverflow.com/questions/11290758

Question

I met an OutOfMemoryError: Java Heap Space when I tried to attach a large attachment. When emails with some large files (say 50M) are sent, the error will be thrown.

The code is like this:

//add attaches
if (vo.getAttaches() != null) {
    InputStream iStream = null;
    ByteArrayDataSource bdSource = null;
    String filename = null;

    for (int i = 0; i < vo.getAttaches().length; i++) {
         iStream = new FileInputStream(vo.getAttaches()[i]);
         bdSource = new ByteArrayDataSource(iStream, null);
         filename = vo.getAttachesFileName()[i];
         email.attach(bdSource, MimeUtility.encodeText(filename), filename);
    }
}

It is bdSource = new ByteArrayDataSource(iStream, null) throws the exception above. I had read "out of memory using java mail" but i don't understand. How can I upload large attaches? If I define a buffer like byte[1024], then how to code email.attach()? Should all data in buffer use the same filename?

Update: Thanks for your kindness. I use FileDataSource instead of ByteArrayDataSource, it seems that there's no exception in my send() function. But I still can not send emails with large attach. The apache james got this error: 

    java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Arrays.java:2786)
    at java.io.ByteArrayOutputStream.toByteArray(ByteArrayOutputStream.java:133)
    at com.sun.mail.util.ASCIIUtility.getBytes(ASCIIUtility.java:261)
    at javax.mail.internet.MimeMessage.parse(MimeMessage.java:338)
    at org.apache.james.core.MimeMessageWrapper.parse(MimeMessageWrapper.java:477)
    at org.apache.james.core.MimeMessageWrapper.loadMessage(MimeMessageWrapper.java:205)
    at org.apache.james.core.MimeMessageWrapper.checkModifyHeaders(MimeMessageWrapper.java:414)
    at org.apache.james.core.MimeMessageWrapper.setHeader(MimeMessageWrapper.java:426)
    at org.apache.james.core.MimeMessageCopyOnWriteProxy.setHeader(MimeMessageCopyOnWriteProxy.java:652)
    at org.apache.james.transport.mailets.UsersRepositoryAliasingForwarding.service(UsersRepositoryAliasingForwarding.java:101)
    at org.apache.james.transport.mailets.LocalDelivery.service(LocalDelivery.java:64)
    at org.apache.james.transport.LinearProcessor.service(LinearProcessor.java:424)
    at org.apache.james.transport.JamesSpoolManager.process(JamesSpoolManager.java:405)
    at org.apache.james.transport.JamesSpoolManager.run(JamesSpoolManager.java:309)
    at java.lang.Thread.run(Thread.java:619)
03/07/12 13:08:33 ERROR spoolmanager: An error occurred processing Mail1341292071375-0 through transport
03/07/12 13:08:33 ERROR spoolmanager: Result was error
03/07/12 13:08:33 ERROR spoolmanager: Exception in processor <error>
java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Arrays.java:2786)
    at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94)
    at org.apache.james.core.MimeMessageUtil.copyStream(MimeMessageUtil.java:168)
    at org.apache.james.core.MimeMessageWrapper.writeTo(MimeMessageWrapper.java:276)
    at org.apache.james.core.MimeMessageUtil.writeTo(MimeMessageUtil.java:66)
    at org.apache.james.core.MimeMessageUtil.writeTo(MimeMessageUtil.java:50)
    at org.apache.james.mailrepository.MessageInputStream.writeStream(MessageInputStream.java:131)
    at org.apache.james.mailrepository.MessageInputStream.<init>(MessageInputStream.java:101)
    at org.apache.james.mailrepository.JDBCMailRepository.store(JDBCMailRepository.java:718)
    at org.apache.james.transport.mailets.ToRepository.service(ToRepository.java:98)
    at org.apache.james.transport.LinearProcessor.service(LinearProcessor.java:424)
    at org.apache.james.transport.JamesSpoolManager.process(JamesSpoolManager.java:405)
    at org.apache.james.transport.JamesSpoolManager.run(JamesSpoolManager.java:309)
    at java.lang.Thread.run(Thread.java:619)
03/07/12 13:08:33 ERROR spoolmanager: An error occurred processing Mail1341292071375-0 through error
03/07/12 13:08:33 ERROR spoolmanager: Result was ghost
Was it helpful?

Solution

The problem is that you're trying to store the whole file in memory while composing the message, which actually isn't usually necessary. If you're attaching real files, then you are far better off using a javax.activation.FileDataSource instead of a javax.mail.util.ByteArrayDataSource (both implement the DataSource interface) as that can allow the data to be streamed rather than being held in memory.

OTHER TIPS

For your update: It seems an old (?) variant of JamesSpoolManager reads the whole file into memory when transforming to "mail format (ascii)" in the same thread dated 30/Dec/10 the problem seems fixed.

ByteArrayDataSource will read the full input from the input stream provided, see javadoc:

Create a ByteArrayDataSource with data from the specified InputStream and with the specified MIME type. The InputStream is read completely and the data is stored in a byte array.

So if the file to be read is "larger" than your heap size (JVM limit) it will throw the OutOfMemoryException.

So, to answer your question, you have (at least) two options:

  1. Use a FileDataSource like in this SO answer
  2. Give your program more memory (probably not a good solution in this case..)

I was facing same issue. But, during multiple emails at a same time. Which result in two errors in logs. 

1. OutOfMemoryError: Java Heap Space

2. Maximum number of connections exceeded

I have changed two parameters in wrapper.conf file. 

#wrapper.java.initmemory=16
 wrapper.java.initmemory=32

#wrapper.java.maxmemory=64
wrapper.java.maxmemory=128

After restart of server, Error has gone and mail send-receive is working.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top