How to set constant seeds for Haskell's quickCheck function

Question

Every time I run "quickCheck prop_xyz", a new random seed is used. How do I enforce QuickCheck to always use the same random seed?

Thanks!

Answer 1

The functionality you need is in Test.QuickCheck; use quickCheckWith to specify custom Args. In particular, there's the replay :: Maybe (StdGen, Int) field, which allows you to replay tests. So you can use the stdArgs defaults and tweak them; for instance,

ghci> :load Main.hs
ghci> import Test.QuickCheck
ghci> import System.Random -- for mkStdGen
ghci> quickCheckWith stdArgs{replay = Just (mkStdGen 42, 0)} prop_xyz

The second component of the tuple has to do with the size of the test cases, but I forget exactly what.

Answer 2

Use the quickCheckWith function

quickCheckWith (stdArgs{replay = Just (myNewGen, testSize)}) property

If you have a test that's failing and you want to reuse it,

result <- quickCheckResult failing_prop
let gen = usedSeed result
let size = usedSize result

to get the size and seed used in a failing test.

Since you also want reproducible errors, a good shrinking algorithm may help. By default it returns [] but provide a nice enough one, then you can end up with the same (minimal) failures even on different random runs.

Answer 3

Example with QuickCheck >= 2.7 and tf-random.

If the output of your failing test run looks something like this:

Failure {
    usedSeed = TFGenR 1CE4E8B15F9197B60EE70803C62388671B62D6F88720288F5339F7EC521FEBC4 0 70368744177663 46 0,
    USEDSIZE = 75,
    ...        
}

Then you can reproduce the failure as follows:

import Test.QuickCheck.Random (QCGen(..))
import System.Random.TF (TFGen)

qcheck :: Testable a => a -> IO ()
qcheck prop = quickCheckWith args prop
    where
        usedSeed = QCGen (read "TFGenR 1CE4E8B15F9197B60EE70803C62388671B62D6F88720288F5339F7EC521FEBC4 0 70368744177663 46 0"::TFGen)
        usedSize = 75
        args = stdArgs{
            replay=Just(usedSeed, usedSize)
        }

Answer 4

If you want to determine the Generator that caused the failure you could do the following:

import Test.QuickCheck
import System.Random


reproducableTest :: Testable a => a -> Maybe (StdGen, Int) -> IO ()
reproducableTest prop repl = do result <- quickCheckWithResult stdArgs{replay = repl} prop
                                case result of
                                     Failure{interrupted = False} -> do putStrLn $ "Use " ++ show (usedSeed result) ++ " as the initial seed"
                                                                        putStrLn $ "Use " ++ show (usedSize result) ++ " as the initial size"
                                     _       -> return ()

Only the first of the two seed numbers should be used as an argument for the function. So if you have your property prop, your initial call would be reproducableTest prop Nothing. You will get something like

  Use x y as the initial seed
  Use z as the initial size

Then you would call again with reproducableTest prop $ Just (mkStdGen x , z)