Tissevert/pandoc
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

Modified fromUTF8 to strip out the BOM (byte order marker)

Browse source 
wherever it is present.  See
http://en.wikipedia.org/wiki/Byte_Order_Mark and
http://six.pairlist.net/pipermail/markdown-discuss/2007-October/000874.html.


git-svn-id: https://pandoc.googlecode.com/svn/trunk@1054 788f1e2b-df1e-0410-8736-df70ead52e1b
This commit is contained in:
fiddlosopher 2007-10-27 22:55:40 +00:00
parent 447b99e35d
commit 65a5db2d41
1 changed files with 1 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

1
src/Text/Pandoc/UTF8.hs
View file

@ -16,6 +16,7 @@ module Text.Pandoc.UTF8 (
-- | Take a UTF-8 string and decode it into a Unicode string.
fromUTF8 :: String -> String
fromUTF8 "" = ""
fromUTF8 ('\xef':'\xbb':'\xbf':cs) = fromUTF8 cs -- skip BOM (byte order marker)
fromUTF8 (c:c':cs) | '\xc0' <= c && c <= '\xdf' &&
'\x80' <= c' && c' <= '\xbf' =
toEnum ((fromEnum c `mod` 0x20) * 0x40 + fromEnum c' `mod` 0x40) : fromUTF8 cs