LaTeX reader: Fixed subtle bug in tokenizer.

Material following `^^` was dropped if it wasn't a character
escape.  This only affected invalid LaTeX, so we didn't see it
in the wild, but it appeared in a QuickCheck test failure
https://travis-ci.org/jgm/pandoc/jobs/319812224
This commit is contained in:
John MacFarlane 2017-12-21 11:50:55 -08:00
parent 117a672c4d
commit 685e90cd4f

View file

@ -376,8 +376,9 @@ totoks pos t =
| d < '\128' ->
Tok pos Esc1 (T.pack ['^','^',d])
: totoks (incSourceColumn pos 3) rest''
_ -> [Tok pos Symbol ("^"),
Tok (incSourceColumn pos 1) Symbol ("^")]
_ -> Tok pos Symbol ("^") :
Tok (incSourceColumn pos 1) Symbol ("^") :
totoks (incSourceColumn pos 2) rest'
_ -> Tok pos Symbol ("^")
: totoks (incSourceColumn pos 1) rest
| otherwise ->