Processing all elements in a BlockingQueue

Question

I am parsing an XML document, building an object from the information that I parse, and then putting that object into a blocking queue.

For testing purposes, the size of the blocking queue is 3 and I have 4 pages that I need to process. What I'd like to happen is that after all possible objects have been added they should all be taken off. I've tried using a while loop (as shown below) to take all the elements out of the queue but a there seems to be a null preventing it from going through the rest of the queue.

I'm new to threads in Java and I'm guessing that's where my trouble lies.

Any help would be appreciated.

Runnable parser = new Runnable()
{
    public void run()
    {
        try
        {
            saxParser.parse("file_to_parse.xml", handler);
        }
        catch (Exception e)
        {
            e.printStackTrace();
        }
    }
};

Runnable takeOff = new Runnable()
{
    public void run()
    {
        try
        {
            while(handler.bq.peek() != null)
            {
                handler.bq.take();
            }
        }
        catch (Exception e)
        {
            e.printStackTrace();
        }
    }
};

new Thread(parser).start();
new Thread(takeOff).start();

Answer 1

One problem is that take is going to block on an empty queue (poll is the method that returns null on an empty queue), so your loop won't terminate (unless parser is adding a null to the queue, which ought to be an error). You're also calling take too often - the first take call (in the while loop's guard) is removing an element, and then the second take call (inside the while loop) is removing another element, which means that you're discarding half of the queue's elements. Try something like this instead

try {
    String temp;
    while((temp = handler.bq.poll()) != null) {
        // process temp
    }
}

This will only work if you don't start takeOff until parser is finished adding items to the queue (when offer returns false) - otherwise poll might return null and terminate takeOff while parser is still adding items to the queue. Here's an alternative that lets you run parser and takeOff at the same time

String temp;
try {
    while(true) {
        temp = handler.bq.take();
        // process temp
    }
catch (InterruptedException ex) {
    // ignore exception
}
while((temp = handler.bq.poll()) != null) {
    // process temp
}

Then you need to interrupt() the takeOff thread when parser is finished: this will exit the while(true) loop, after which the thread will go to the while((temp = handler.bq.poll()) != null) loop to finish removing items from the queue.