boost::iostreams::copy - sink - ENOSPC (No space left on device) error handling

Question

In the code fragment below, is there a way to handle ENOSPC?

#include <fstream>
#include <iostream>
#include <boost/iostreams/filtering_streambuf.hpp>
#include <boost/iostreams/copy.hpp>
#include <boost/iostreams/filter/bzip2.hpp>

// open input file stream of the bzip2 file
std::ifstream ifs("file.bz2");

// open output stream to the "full" device
// full device is a "utility-device" to check how applications handle ENOSPC
// more details in "man full"
std::ofstream ofs("/dev/full");

// Setup the iostreams filter
boost::iostreams::filtering_streambuf<boost::iostreams::output> filters;
filters.push(boost::iostreams::bzip2_decompressor());
filters.push(ofs);

// "run" the filter
boost::iostreams::copy(ifs, filters);

If I do strace of the compiled binary, the code seem to infinitely call writev() with the same data and returns ENOSPC error.

writev(4, [{NULL, 0}, {"DATA DATA "..., 4096}], 2) = -1 ENOSPC (No space left on device)

How can this error be handled or made thrown as an error from boost::iostreams::copy().

Is it possible to set appropriate exceptions() on the ofstream object? I tried ofs.exceptions(std::ios::badbit | std::ios::failbit) but it didn't make any difference.

The code above was compiled with GCC and run on Linux. Boost version 1.55.

Answer 1

It's stuck in an infinite loop in non_blocking_adaptor<Device>::write(...):

std::streamsize result = 0;
while (result < n) {
    std::streamsize amt = 
        iostreams::write(device_, s + result, n - result);
    result += amt;
}
return result;    

iostream::write(device_, ... keeps returning 0 (so n stays n, amt and result stay 0).

---

It would appear to be a bug in Boost IOstreams. Perhaps it was introduced when preliminary support for non-blocking (synchronous) IO was added. According to the documentation, this should be a work in progress.

Particularly enlightening was: http://www.boost.org/doc/libs/1_55_0/libs/iostreams/doc/guide/asynchronous.html

Filters

Filters are allowed to propagate temporary failure notifications: if a downstream Device consumes or produces fewer characters than requested by a Filter, and if as a result the Filter is not able to satisfy a read or write request, the Filter may return a value indicating that input or output is temporarily unavailable. It is hoped that this ability will suffice to allow the current Filter concepts to be used with both aynchronous and non-blocking i/o. However, in order to be useful with blocking i/o as well, a Filter must never return a temporary failure notification unless it has received a such notification from a downstream Device. This requirement is summarized by stating that Filters must be blocking-preserving. See Blocking.

(bold mine) It does appear as if IOStreams violated this principle by transforming the E_NOSPC condition into a temporary failure notification.

---

boost::iostreams::copy special-cases the case where source and destination Indirect devices. In this case, both are indirect. Now the special case wraps the sink in a non_blocking_adaptor. I don't know why, and this seems to contradict the following general advice taken from the same documentation page:

Streams and Stream Buffers

Although the Boost.Iostreams Filter and Device concepts can accommodate non-blocking i/o, the C++ standard library stream and stream buffer interfaces cannot, since they lack a means to distinguish between temporary and permanent failures to satisfy a read or write request. As a result, non-blocking Devices do not work properly with the templates stream, stream_buffer, filtering_stream and filtering_streambuf.

I tried to replace the files with file_sink and file_source instead, but there was no change. :(

Here's my reduced test case which still reproduces the problem:

#include <boost/iostreams/copy.hpp>
#include <boost/iostreams/device/file.hpp>
#include <boost/iostreams/filtering_streambuf.hpp>

int main()
{
    using namespace boost::iostreams;
    file_source ifs("/dev/zero");
    file_sink   ofs("/dev/full");

    filtering_streambuf<output> filters(ofs);
    copy(ifs, filters);
}

Perhaps you should report this as a bug with the developers/mailing list.

Answer 2

I think I have found a solution to this problem. What I have done is to make my own sink according to the documentation here.

Basically a sink that would handle write and then check the stream's good state.

struct safe_ofstream_sink
{
    typedef char char_type;
    typedef boost::iostreams::sink_tag category;

    std::ofstream& ofs;

    safe_ofstream_sink(std::ofstream& ofs) :
            ofs(ofs)
    {
    }

    std::streamsize write(const char* s, std::streamsize n)
    {
        ofs.write(s, n);
        if (!ofs)
            throw std::runtime_error("Failed writing to fstream");

        return n;
    }
};


boost::iostreams::filtering_streambuf<boost::iostreams::output> filters;
filters.push(boost::iostreams::bzip2_decompressor());
filters.push(safe_ofstream_sink(ofs));
boost::iostreams::copy(ifs, filters);