Does Splice support Unicode?

StackOverflow https://stackoverflow.com/questions/19329857

Question

This is my code :

 testSplice :: C.Splice Handler
 testSplice = return $ C.yieldRuntimeText $ do
    return "中文"

And I bind it to a tag:

  splices :: Splices (C.Splice Handler)
  splices =
      "testSplice" ## testSplice

And used it on layout.tpl:

   <meta charset="UTF-8"> 
   <testSplice/>

And the output on the browser is 

    -�

What have I done wrong?

Sorry for the delay, I have been busy for a while, and now I come back and I think I may not make the question specific enough @mightybyte Here is the code where the problem occurs, I hope it would make the question more specifier:

test.hs:

{-# LANGUAGE OverloadedStrings #-}
import Snap
import Heist
import qualified Heist.Compiled as C
import Data.Monoid
import Control.Monad.Trans.Either
import Data.Maybe

main :: IO ()
main = quickHttpServe site

site :: Snap ()
site = 
    route [("/", testSnap)]

testSnap :: Snap ()
testSnap = do
    hs <- liftIO $ load "template" splices
    let runtime = fromJust $ C.renderTemplate hs "test"
    builder <-liftIO  (fst runtime)
    writeBuilder builder
  where
    splices :: Splices (C.Splice IO)
    splices = 
        "testSplice" ## testSplice

load :: MonadIO n
    => FilePath
    -> Splices (C.Splice n)
    -> IO (HeistState n)
load baseDir splices = do
    tmap <- runEitherT $ do
        let t = loadTemplates baseDir
            hc = HeistConfig
                    defaultInterpretedSplices
                    defaultLoadTimeSplices
                    splices
                    mempty
                    [t]
        initHeist hc
    either (error . concat) return tmap

testSplice :: C.Splice IO
testSplice = return $ C.yieldRuntimeText $ do return "中文"

template/test.tpl

<html>
  <head>
    <meta charset="UTF-8">
  </head>
 <body>
     <testSplice/>
 </body>
 </html>

Now I tried heist-0.13.0.2 and it works fine now, great work Daniel!

Was it helpful?

Solution

UPDATE: The problem described in this answer has been corrected in heist version 0.13.0.2.

The source code for yieldRuntimeText is:

yieldRuntimeText :: Monad n => RuntimeSplice n Text -> DList (Chunk n)
yieldRuntimeText = yieldRuntime .  liftM fromText

From what module comes that fromText function? In the import section we find:

import           Blaze.ByteString.Builder
import           Blaze.ByteString.Builder.Char8

The documentation for the latter package says:

Note: This package is intended for low-level use like implementing protocols. If you need to serialize Unicode characters use one of the UTF encodings (e.g. 'Blaze.ByteString.Builder.Char.UTF-8').

and also:

fromText :: Text -> BuilderSource

O(n). Serialize the lower 8-bits of all characters in the strict text.

Mmmmm so maybe the problem is that the encoding from Text to Builder is not being done in UTF-8? Try defining your own version of yieldRuntimeText, with exactly the same code but using the following imports instead:

import           Blaze.ByteString.Builder
import           Blaze.ByteString.Builder.Char.Utf8

yieldRuntimeTextUtf8 :: Monad n => RuntimeSplice n Text -> DList (Chunk n)
yieldRuntimeTextUtf8 = yieldRuntime .  liftM fromText
Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top