Is it safe to reuse a conduit?

Question

Is it safe to perform multiple actions using the same conduit value? Something like

do
  let sink = sinkSocket sock

  something $$ sink
  somethingElse $$ sink

I recall that in the early versions of conduit there were some dirty hacks that made this unsafe. What's the current status?

(Note that sinkSocket doesn't close the socket.)

Answer 1

That usage is completely safe. The issue in older versions had to do with blurring the line between resumable and non-resumable components. With modern versions (I think since 0.4), the line is very clear between the two.

Answer 2

It might be safe to reuse sinks in the sense that the semantics for the "used" sink doesn't change. But you should be aware of another threat: space leaks.

The situation is analogous to lazy lists: you can consume a huge list lazily in a constant space, but if you process the list twice it will be kept in memory. The same thing might happen with a recursive monadic expression: if you use it once it's constant size, but if you reuse it the structure of the computation is kept in memory, resulting in space leak.

Here's an example:

import Data.Conduit
import Data.Conduit.List
import Control.Monad.Trans.Class (lift)

consumeN 0 _ = return ()
consumeN n m = do
  await >>= (lift . m)
  consumeN (n-1) m

main = do
  let sink = consumeN 1000000 (\i -> putStrLn ("Got one: " ++ show i))
  sourceList [1..9000000::Int] $$ sink
  sourceList [1..22000000::Int] $$ sink

This program uses about 150M of ram on my machine, but if you remove the last line or repeat the definition of sink in both places, you get a nice constant space usage.

I agree that this is a contrived example (this was the first that came to my mind), and this is not very likely to happen with most Sinks. For example this will not happen with your sinkSocket. (Why is this contrived: because the control structure of the sink doesn't depend on the values it gets. And that is also why it can leak.) But, for example, for sources this would be much more common. (Many of the common Sources exhibit this behavior. The sourceList would be an obvious example, because it would actually keep the source list in memory. But, enumFromTo is no different, although there is no data to keep in memory, just the structure of the monadic computation.)

So, all in all, I think it's important to be aware of this.