Markdown citation parser improvements and test updates.

Now we handle a suffix after a bare locator, e.g.
@item1 [p. 30, suffix]
The suffix now includes any punctuation that introduces it.
A few tests fail because of problems with citeproc (extra space
before the suffix, missing space after comma separating multiple
page ranges in the locator).

src/Text/Pandoc/Readers/Markdown.hs

@@ -1348,13 +1348,12 @@ bareloc :: Citation -> GenParser Char ParserState [Citation]
 bareloc c = try $ do
   spnl
   char '['
-  spnl
   loc <- locator
-  spnl
+  suff <- suffix
   rest <- option [] $ try $ char ';' >> citeList
   spnl
   char ']'
-  return $ c{ citationLocator = loc } : rest
+  return $ c{ citationLocator = loc, citationSuffix = suff } : rest
 
 normalCite :: GenParser Char ParserState [Citation]
 normalCite = try $ do
@@ -1376,28 +1375,31 @@ citeKey = try $ do
 locator :: GenParser Char st String
 locator = try $ do
   spnl
-  w  <- many1 (noneOf " \t\n;]")
-  spnl
-  ws <- many locatorWord
+  w  <- many1 (noneOf " \t\n;,]")
+  ws <- many (locatorWord <|> locatorComma)
   return $ unwords $ w:ws
 
 locatorWord :: GenParser Char st String
 locatorWord = try $ do
-  wd <- many1 $ (try $ char '\\' >> oneOf "]; \t\n") <|> noneOf "]; \t\n"
   spnl
-  if any isDigit wd
-     then return wd
-     else pzero
+  wd <- many1 $ (try $ char '\\' >> oneOf "];, \t\n") <|> noneOf "];, \t\n"
+  guard $ any isDigit wd
+  return wd
+
+locatorComma :: GenParser Char st String
+locatorComma = try $ do
+  char ','
+  lookAhead $ locatorWord
+  return ","
 
 suffix :: GenParser Char ParserState [Inline]
 suffix = try $ do
-  char ','
   spnl
   liftM normalizeSpaces $ many $ notFollowedBy (oneOf ";]") >> inline
 
 prefix :: GenParser Char ParserState [Inline]
 prefix = liftM normalizeSpaces $
-  manyTill inline (lookAhead citeKey)
+  manyTill inline (char ']' <|> liftM (const ']') (lookAhead citeKey))
 
 citeList :: GenParser Char ParserState [Citation]
 citeList = sepBy1 citation (try $ char ';' >> spnl)
@@ -1407,7 +1409,7 @@ citation = try $ do
   pref <- prefix
   (suppress_author, key) <- citeKey
   loc <- option "" $ try $ blankSpace >> locator
-  suff <- option [] suffix
+  suff <- suffix
   return $ Citation{ citationId        = key
                      , citationPrefix  = pref
                      , citationSuffix  = suff

tests/markdown-citations.plain

@@ -5,12 +5,14 @@ Pandoc with citeproc-hs
 
 @nonexistent
 
-Doe (2005) says blah. Doe (2005, 30) says blah. Doe
-(2005; 2006, 30; see also Doe and Roe 2007) says blah.
+Doe (2005) says blah. Doe (2005, 30) says blah. Doe (2005, 30, suffix)
+says blah. Doe (2005; 2006, 30; see also Doe and Roe 2007) says blah.
 
 In a note.[^1] A citation group
 (see Doe 2005, 34-35; also Doe and Roe 2007, chap. 3). Another one
-(see Doe 2005, 34-35). And another one in a note.[^2]
+(see Doe 2005, 34-35). And another one in a note.[^2] Citation with
+a suffix and locator (Doe 2005, 33, 35-37, and nowhere else).
+Citation with suffix only (Doe 2005, and nowhere else).
 
 Now some modifiers.[^3]
 

tests/markdown-citations.txt

@@ -6,11 +6,15 @@
 
 @item1 says blah.
 @item1 [p. 30] says blah.
+@item1 [p. 30, with suffix] says blah.
 @item1 [-@item2 p. 30; see also @item3] says blah.
 
 In a note.[^1] A citation group [see
 @item1 p. 34-35; also @item3 chap. 3]. Another one [see
 @item1 p. 34-35]. And another one in a note.[^2]
+Citation with a suffix and locator [@item1 pp. 33, 35-37,
+and nowhere else].  Citation with suffix only
+[@item1, and nowhere else].
 
 Now some modifiers.[^3]