Org-mode and Pandoc use different language identifiers, marking source
code as being written in a certain programming language. This adds more
translations from identifiers as used in Org to identifiers used in
Pandoc.
The full list of identifiers used in Org and Pandoc is available through
http://orgmode.org/manual/Languages.html and `pandoc -v`, respectively.
Text such as /*this*/ was not correctly parsed as a strong, emphasised
word. This was due to the end-of-word recognition being to strict as it
did not accept markup chars as part of a word. The fix involves an
additional parser state field, listing the markup chars which might be
parsed as part of a word.
The default pandoc ParserState is replaced with `OrgParserState`. This
is done to simplify the introduction of new state fields required for
efficient Org parsing.
The reader did not correctly parse inline markup. The behavoir is now as follows.
(a) The markup must start at the start of a line, be inside previous
inline markup or be preceeded by whitespace.
(b) The markup can not span across paragraphs (delimited by \n\n)
(c) The markup can not be followed by a alphanumeric character.
(d) Square brackets can be placed around the markup to avoid having
to have white space before it.
In order to make these changes it was either necessary to convert the parser to return a list of inlines or to convert the whole reader to use the builder. The latter approach whilst more work makes a bit more sense as it becomes easy to arbitarily append and prepend elements without changing the type.
Tests are accordingly updated in a later commit to reflect the different normalisation behavoir specified by the builder monoid.
Previously these were typeclasses of monads. They've been changed
to be typeclasses of states. This ismplifies the instance definitions
and provides more flexibility.
This is an API change! However, it should be backwards compatible
unless you're defining instances of HasReaderOptions, HasHeaderMap,
or HasIdentifierList. The old getOption function should work as
before (albeit with a more general type).
The function askReaderOption has been removed.
extractReaderOptions has been added.
getOption has been given a default definition.
In HasHeaderMap, extractHeaderMap and updateHeaderMap have been added.
Default definitions have been given for getHeaderMap, putHeaderMap,
and modifyHeaderMap.
In HasIdentifierList, extractIdentifierList and updateIdentifierList
have been added. Default definitions have been given for
getIdentifierList, putIdentifierList, and modifyIdentifierList.
The ultimate goal here is to allow different parsers to use their
own, tailored parser states (instead of ParserState) while still
using shared functions.
This is to debug backtracking-related parsing bugs.
So far it is only implemented for markdown, but it would
be good to extend it to latex and html readers.
rST parser now supports:
- All built-in rST roles
- New role definition
- Role inheritance
Issues/TODO:
- Silently ignores illegal fields on roles
- Silently drops class annotations for roles
- Only supports :format: fields with a single format for :raw: roles,
requires a change to Text.Pandoc.Definition.Format to support multiple
formats.
- Allows direct use of :raw: role, rST only allows indirect (i.e.,
inherited use of :raw:).
Keys may now start with an underscore as well as a letter.
Underscores do not count as internal punctuation, but are
treated like alphanumerics, so "key:_2008" will work, as
it did not before. (This change was necessary to use keys
generated by zotero.)
Closes#1111, closes#1011.
