Returning Buffer filled with Binary Data from recv

Question

Assuming I have a function that fills a buffer with binary data easily, how can I make that function return said buffer for further use?

The way I am currently doing it is by having it write the buffer (based off the content-length field from recv) write a temp file, then returning the name of that temp file so I can just read the tmp file into memory.

It would be nice however if I could just return the data directly rather than writing it to a tmp file, and using that. The only problem is if I return the binary data I have no way of knowing what the size of the buffer is (it's not static) So my question is: is there a method of returning this binary data as well as maybe it's size, or any way that I can work with it?

Or is my best bet just to stick with using tmp files?

Answer 1

In C you can either use a struct to encapsulate bunch of data items together, and pass that to your function, like:

/* Describes single I/O buffer */
struct buf {
    char* data; /* pointer to dynamically allocated memory */
    size_t mem_size; /* allocation size */
    size_t data_len; /* how much data is in the buffer */
    struct buf* next; /* can be used for buffer chaining */
};

/* Returns 0 on success, -1 on error */
int read_data( int sockfd, struct buf* pb );

Or use value-return arguments, like:

/* buffer receives data, len tells buffer length on input, and
 * how much was read on success. */
int read_data( int sockfd, char* buffer, size_t* len );

Answer 2

You can use the same API as socket recv. Caller provides a buffer and max len, and function returns actual received length.

http://linux.die.net/man/2/recv

Answer 3

Well - the advantage of temp files is indeed - simpler memory management. And on a lot of mature/modern operating systems - the overhead for short lived files like that in /tmp is quite minimal (and your time as a developer is expensive). If you have some sort of idea of file sizes - a pretty common approach is something like below.

But exactly what you want depends on memory management. And it is easy to re-invent the wheel.

A good way to avoid this is to use something like http://apr.apache.org/ APR Commons - i.e. apr_socket_recv() and the associated memory management (http://apr.apache.org/docs/apr/1.4/group_apr_network__io.html#gaa6ee00191f197f64b5a5409f4aff53d1). Generally that is a long term win.

Dw.

// On entry:
//     buffp - pointer to NULL or a malloced buffer.
//     lenp - pointer for the length; set to 0 or the length of the malloced buffer.
// On exit
//     return value < 0 for a fault, 0 for a connection close and > 0 for
//     the number of bytes read.
//     buffp will be filled out with the buffer filled; lenleftp with the bytes left
//     (i.e not yet used). 
//     If post call *buffp != NULL you need to release/clean memory.
//
ssize_t sockread(..., unsigned char * * buffp , ssize_t * lenleftp) {
      if (!buffp || !lenleftp)
           return -1; // invalid params

      // claim memory as needed.
      if (*buffp == 0) {
           *lenleftp = 16K;
           *buffp = malloc(*lenleftp);
      }

      if (!*buffp || !*lenleftp)
          return -2; // invalid params

      ssize_t used = 0;
      while(1) {
         ssize_t l = recv(..., *buffp, *lenleftp - used, ..);
         if (l < 0) {
            // ignore transient errors we can retry without much ado.
            if (errno == EAGAIN || errno == EINTR) 
                continue;

            free(*buffp); *buffp = *lenleftp = NULL;

            // report a fail.
            return -3;
         }
         // we simply assume that a TCP close means we're done.
         if (l == 0)
            break;

         used += l;

         // increase buffer space as/when needed.
         //
         if (used >= lenleftp) {
               *lenleftp += 32K;
               *buffp = realloc(lenleftp);
         };
   }

   // we're assuming that a read == 0-- i.e. tcp stream done means
   // we're done with reading.
   //
   *lenleftp -= used;
   return used;
}