How do I create an in-memory handle in Haskell?

Question

I want something that looks like a file handle but is really backed by an in-memory buffer to use for I/O redirects. How can I do this?

Answer 1

It's not possible without modifying the compiler. This is because Handle is an abstract data type, not a typeclass.

Answer 2

I just wrote a library which provides this, called "knob" [hackage]. You can use it to create Handles which reference/modify a ByteString:

import Data.ByteString (pack)
import Data.Knob
import System.IO

main = do
    knob <- newKnob (pack [])
    h <- newFileHandle knob "test.txt" WriteMode
    hPutStrLn h "Hello world!"
    hClose h
    bytes <- Data.Knob.getContents knob
    putStrLn ("Wrote bytes: " ++ show bytes)

Answer 3

If you can express what you want to do in terms of C or system calls you could use Haskell's Foreign Function Interface (FFI). I started to suggest using mmap, but on second thought I think mmap might be a mapping the wrong way even if you used it with the anonymous option.

You can find more information about the Haskell FFI at the haskell.org wiki.

Answer 4

This may not be possible. GHC, at least, seems to require a handle to have an OS file descriptor that is used for all read/write/seek operations.

See /libraries/base/IOBase.lhs from the GHC sources.

You may be able to get the same effect by enlisting the OS's help: create a temporary file, connect the handle to it and then memory map the file for the I/O redirects. This way, all the handle I/O would become visible in the memory mapped section.

Answer 5

This is actually a bug in the library design, and one that's annoyed me, too. I see two approaches to doing what you want, neither of which is terribly attractive.

1. Create a new typeclass, make the current handle an instance of it, write another instance to do the in-memory-data thing, and change all of your programs that need to use this facility. Possibly this is as simple as importing System.SIO (or whatever you want to call it) instead of System.IO. But if you use the custom I/O routines in libraries such as Data.ByteString, there's more work to be done there.

2. Rewrite the I/O libraries to extend them to support this. Not trivial, and a lot of work, but it wouldn't be particularly difficult work to do. However, then you've got a compatibility issue with systems that don't have this library.

Answer 6

To add a modern answer to this question, you could use createPipe from System.Process:

createPipe :: IO (Handle, Handle)

https://www.stackage.org/haddock/lts-10.3/process-1.6.1.0/System-Process.html#v:createPipe