How to read different chunks from a conduit (for example a line until LF and then 10 bytes)?

Question

For one network protocol I need to be able to read flexibly different kinds of chunks from a Source m ByteString. There is lines combinator, which splits input to lines, but I need to be able to combine reading lines and a fixed number of bytes.

My current approach is that I create a helper function:

| Folds a given function on inputs. Repeat while the function returns Left and accumulate its results in a list. When the function returns Right, concatenate the accumulated result (including the last one) and return it, storing what's left using leftover. Returns Nothing if no input is available.

chunk :: (Monad m, Monoid a)
      => (s -> i -> Either (a, s) (a, i))
      -> s
      -> Consumer i m (Maybe a)
chunk f = loop []
  where
    loop xs s = await >>= maybe (emit xs) (go xs s)

    go xs s i = case f s i of
        Left (x, s')    -> loop (x : xs) s'
        Right (x, l)    -> leftover l >> emit (x : xs)

    emit [] = return Nothing
    emit xs = return (Just . mconcat . L.reverse $ xs)
-- Note: We could use `mappend` to combine the chunks directly. But this would
-- often get us O(n^2) complexity (like for `ByteString`s) so we keep a list of
-- the chunks and then use `mconcat`, which can be optimized by the `Monoid`.

Using this function I create specific consumers:

bytes :: (Monad m) => Int -> Consumer ByteString m (Maybe ByteString)
bytes = chunk f
  where
    f n bs | n' > 0     = Left (bs, n')
           | otherwise  = Right $ BS.splitAt n bs
      where n' = n - BS.length bs

line :: (Monad m) => Consumer ByteString m (Maybe ByteString)
line = chunk f ()
  where
    f _ bs = maybe (Left (bs, ()))
                   (\i -> Right $ BS.splitAt (i + 1) bs)
                   (BS.findIndex (== '\n') bs)

Is there a better way? I suppose this problem must have already been solved somewhere.

Answer 1

Looks about right, it's very similar to how Warp parses request headers, though Warp doesn't bother with any higher-level combinators.