as in the mediawiki writer. The dokuwiki markup isn't able
to handle multiple block-level items within a list item, except
in a few special cases (e.g. code blocks, and these must be started
on the same line as the preceding paragraph). So we fall back to
raw HTML for these.
Perhaps there is a better solution. We can "fake" multiple
paragraphs within list items using hard line breaks (`\\`), but
we must keep everything on one line.
(#1398)
- We no longer include trailing spaces and newlines in the
raw blocks.
- We look for closing tags for elements (but without backtracking).
- Each block-level tag is its own RawBlock; we no longer try to
consolidate them (though `--normalize` will do so).
Closes#1330.
This doesn't change the testsuite behaviour, but it does mean that
all the testsuite output files are exactly identical to the
output obtained by running the current pandoc.
* Added normalizeInlines, normalizeBlocks.
* Type signature is now more narrow, `Pandoc -> Pandoc` instead of
`Data a :: a -> a`. Some users may need to change their uses of
`normalize` to the newly exported `normalizeInlines` or
`normalizeBlocks`.
We want to treat it as a plain paragraph if the hanging amount is
greater to or equal to the left indent---i.e., if the first line has
zero indentation. But we still want it to be a block quote if it starts
to the right of the margin. Someone might format verse with wrapping
lines with a hanging indent, for example.
The new code was got from inspecting changes in MediaWiki.hs
This slightly changes the output of Div blocks, but I'm not
convinced the original behaviour was really correct anyway.
The code for handling Span does nothing for now, until I can
work out the desired behaviour, and add tests for it.
This is what seems like the sensible default: read in insertions, and
ignore deletions. In the future, it would be good if options were
available for either taking in deletions or keeping both in some
scriptable format.
Add torture-test for new normalization functions.
One problem that this test demonstrates is that word has a tendency to
turn off formatting at a space, and then turn it back on after. I'm not
sure yet whether this is something we should fix.
This is just a wrapper around Pandoc that doesn't normalize with
`toString`. We want to make sure that our own normalization process
works. If, in the future, we are able to hook into the builder's
normalization, this will be removed.
This brings pandoc's rendering of haddock markup in line
with the new haddock.
Note that we preserve line breaks in `@` code blocks, unlike
the earlier version.
Modified tests pass. More tests would be good.
Closes#1345. Also relabeled 'code' and 'verbatim' parsers
to accord with the org-mode manual.
I'm not sure what the distinction between code and verbatim
is supposed to be, but I'm pretty sure both should be represented
as Code inlines in pandoc. The previous behavior resulted in the
text not appearing in any output format.
Inline LaTeX is now accepted and parsed by the org-mode reader. Both,
math symbols (like \tau) and LaTeX commands (like \cite{Coffee}), can be
used without any further escaping.
Citations are defined via the "normal citation" syntax used in markdown,
with the sole difference that newlines are not allowed between "[...]".
This is for consistency, as org-mode generally disallows newlines
between square brackets.
The extension is turned on by default and can be turned off via the
default syntax-extension mechanism, i.e. by specifying "org-citation" as
the input format.
Move `citeKey` from Readers.Markdown into Parsing
The function can be used by other readers, so it is made accessible for
all parsers.
The reader produced wrong results for block containing non-letter chars
in their parameter arguments. This patch relaxes constraints in that it
allows block header arguments to contain any non-space character (except
for ']' for inline blocks).
Thanks to Xiao Hanyu for noticing this.
The general form of source block headers
(`#+BEGIN_SRC <language> <switches> <header arguments>`) was not
recognized by the reader. This patch adds support for the above form,
adds header arguments to the block's key-value pairs and marks the block
as a rundoc block if header arguments are present.
This closes#1286.
Org's inline code blocks take forms like `src_haskell(print "hi")` and
are frequently used to include results from computations called from
within the document. The blocks are read as inline code and marked with
the special class `rundoc-block`. Proper handling and execution of
these blocks is the subject of a separate library, rundoc, which is
work in progress.
This closes#1278.
Org allows users to define their own custom link types. E.g., in a
document with a lot of links to Wikipedia articles, one can define a
custom wikipedia link-type via
#+LINK: wp https://en.wikipedia.org/wiki/
This allows to write [[wp:Org_mode][Org-mode]] instead of the
equivallent [[https://en.wikipedia.org/wiki/Org_mode][Org-mode]].
Internal links in Org are possible by using an anchor-name as the target
of a link:
[[some-anchor][This]] is an internal link.
It links <<some-anchor>> here.
Footnotes can consist of multiple blocks and end only at a header or at
the beginning of another footnote. This fixes the previous behavior,
which restricted notes to a single paragraph.
Support for standard org-blocks is improved. The parser now handles
"HTML", "LATEX", "ASCII", "EXAMPLE", "QUOTE" and "VERSE" blocks in a
sensible fashion.
These are primarily aimed at testing the new treatment of line breaks,
but hopefully other tests can be added more easily now as features
and changes are implemented in the writer.
Adapted from Tests.Writers.HTML.tests.
* Use a <literallayout> for the entire paragraph, not just for the
newline character
* Don't let LineBreaks inside footnotes influence the enclosing
paragraph
This fixes the org-reader's handling of sub- and superscript
expressions. Simple expressions (like `2^+10`), expressions in
parentheses (`a_(n+1)`) and nested sexp (like `a_(nested()parens)`) are
now read correctly.
Support all of the following variants as valid ways to define inline or
display math inlines:
- `\[..\]` (display)
- `$$..$$` (display)
- `\(..\)` (inline)
- `$..$` (inline)
This closes#1223. Again.
Instead of being ignored, attributes are now parsed and
included in Span inlines.
The output will be a bit different from stock textile:
e.g. for `*(foo)hi*`, we'll get `<em><span class="foo">hi</span></em>`
instead of `<em class="foo">hi</em>`. But at least the data is
not lost.
Text such as /*this*/ was not correctly parsed as a strong, emphasised
word. This was due to the end-of-word recognition being to strict as it
did not accept markup chars as part of a word. The fix involves an
additional parser state field, listing the markup chars which might be
parsed as part of a word.
Previously normalisation was handled by the `normalizeSpaces` function. The behavoir of the builder monoid is slightly different and melds together more items such as consecutive strings and spaces adjacent to line breaks. The tests have been changed to reflect this.
All relevant tests passed when the string melding line of the builder monoid was commented out.
rST parser now supports:
- All built-in rST roles
- New role definition
- Role inheritance
Issues/TODO:
- Silently ignores illegal fields on roles
- Silently drops class annotations for roles
- Only supports :format: fields with a single format for :raw: roles,
requires a change to Text.Pandoc.Definition.Format to support multiple
formats.
- Allows direct use of :raw: role, rST only allows indirect (i.e.,
inherited use of :raw:).
A consequence of this change is that the backtick form will be
preferred in general if both are enabled. I think that is good,
as it is much more widespread than the tilde form.
Closes#1084.
This is reported to be necessary to avoid an error from recent
versions of Libre Office when files contain more than one image.
Closes#1069.
Thanks to wmanley for reporting and diagnosing the problem.
Going forward we'll use pandoc-citeproc, as an external filter.
The `--bibliography`, `--csl`, and `--citation-abbreviation` fields
have been removed. Instead one must include `bibliography`, `csl`,
or `csl-abbrevs` fields in the document's YAML metadata. The filter
can then be used as follows:
pandoc --filter pandoc-citeproc
The `Text.Pandoc.Biblio` module has been removed. Henceforth,
`Text.CSL.Pandoc` from pandoc-citations can be used by library users.
The Markdown and LaTeX readers now longer format bibliographies and
citations. That must be done using `processCites` or `processCites'`
from Text.CSL.Pandoc.
All bibliography-related fields have been removed from `ReaderOptions`
and `WriterOptions`: `writerBiblioFiles`, `readerReferences`,
`readerCitationStyle`.
API change.
The code:
~~~{#test}
asdf
~~~
gets compiled to html:
<pre id="test">
asdf
</pre>
So it is possible to link to the identifier `test`
But this doesn't happen on latex
When using the listings package (`--listings`) it is possible to set the
identifier using the `label=test` property:
\begin{lstlisting}[label=id]
hi
\end{lstlisting}
And this is exactly what this patch is doing.
Modified LaTeX Reader/Writer and added tests for this.
* Add ??? as fallback text for non-resolved citations.
* Biblio: Put references (including a header at the end of
the document, if one exists) inside a Div with class "references".
This gives some control over styling of references, and allows
scripts to manipulate them.
* Markdown writer: Print markdown citation codes, and disable
printing of references, if `citations` extension is enabled.
NOTE: It would be good to improve what citeproc-hs does for
a nonexistent key.
When the original document had text containing //, this was previously
included, unchanged, in the dokuwiki output, and this interacted badly
with later, intended, formating text.
Done because I noticed that in the Autolinks section of writer.dokuwiki, the URL in inlined code was getting auto-linked, when it wasn't supposed to.
This also meant that any inline code examples that had text that looked like dokuwiki syntax could break the formatting of later text.
I've found some incorrect behaviours with the dokuwiki output, for which
extra test cases will be needed - that aren't covered by the standard
pandoc test input files.
* Closed#927 (a bug in which `<pre>` in certain contexts was
not recognized as a code block).
* Remove internal HTML tags in code blocks, rather than printing
them verbatim.
* Parse attributes on `<pre>` tag for code blocks.
Now the `title`, `section`, `header`, and `footer` can all be set
individually in metadata. The `description` variable has been
removed.
Quotes have been added so that spaces are allowed in the title.
If you have a title that begins
COMMAND(1) footer here | header here
pandoc will parse it as before into a title, section, header, and
footer. But you can also specify these elements explicitly.
Closes#885.
* Depend on pandoc 1.12.
* Added yaml dependency.
* `Text.Pandoc.XML`: Removed `stripTags`. (API change.)
* `Text.Pandoc.Shared`: Added `metaToJSON`.
This will be used in writers to create a JSON object for use
in the templates from the pandoc metadata.
* Revised readers and writers to use the new Meta type.
* `Text.Pandoc.Options`: Added `Ext_yaml_title_block`.
* Markdown reader: Added support for YAML metadata block.
Note that it must come at the beginning of the document.
* `Text.Pandoc.Parsing.ParserState`: Replace `stateTitle`,
`stateAuthors`, `stateDate` with `stateMeta`.
* RST reader: Improved metadata.
Treat initial field list as metadata when standalone specified.
Previously ALL fields "title", "author", "date" in field lists
were treated as metadata, even if not at the beginning.
Use `subtitle` metadata field for subtitle.
* `Text.Pandoc.Templates`: Export `renderTemplate'` that takes a string
instead of a compiled template..
* OPML template: Use 'for' loop for authors.
* Org template: '#+TITLE:' is inserted before the title.
Previously the writer did this.
rst2html doesn't add `<p>` tags to list items (even when they are
separated by blank lines) unless there are multiple paragraphs in the
list. This commit changes the RST reader to conform more closely to
what docutils does.
Closes#880.
The _note attribute is supported. This is unofficial, but
used e.g. in OmniOutliner and supported by multimarkdown.
We treat the contents as markdown blocks under a section
header.
Added to documentation and tests.
* Reverts 1.11 change that caused citations to be rendered as
markdown citations, even if `--biblio` was specified, unless
`citation` extension is disabled. Now, formatted citations
are always printed if `--biblio` was specified. If you want to
reformat markdown keeping pandoc markdown citations intact,
just don't specify `--biblio`.
* Reverted now unnecessary changes to Text.Pandoc.Biblio adding the raw
block to mark the bibliography, and to Text.Pandoc.Writers.Markdown
to remove the bibliography if `citations` not specified.
* If the content of a `Cite` inline is a `RawInline "latex"`, which
means that a LaTeX citation command was parsed and `--biblio` wasn't
specified, then render it as a pandoc markdown citation. This means
that `pandoc -f latex -t markdown`, without `--biblio`, will convert
LaTeX citation commands to pandoc markdown citations.
Previously, a LaTeX citation would always be parsed as a Citation
element, with the raw LaTeX in the [Inline] part.
Now, the LaTeX citation is parsed as a Citation element only if
`--biblio` was specified (i.e. only if there is a nonempty set
of references in readerReferences). Otherwise it is parsed as
raw LaTeX.
This will make it possible to simplify some things in the markdown
writer. It also makes the LaTeX reader behave more like the Markdown
reader.
Previously citations were rendered as citeproc-formatted citations
by default. Now we render them as pandoc citations, e.g. `[@item1]`,
unless the `citations` extension is disabled.
If you still want formatted citations in your markdown output,
use `pandoc -t markdown-citations`.
* Moved code for translating listings language names to
highlighting-kate names and back from LaTeX reader to Highlighting.
* Text.Pandoc.Highlighting no longer exposed (API change)
* Text.Pandoc.Highlighting exports toListingsLang, fromListingsLang
Note: The attributes go on the enclosing section or div
if `--section-divs` is specified.
Also fixed a regression (only now noticed) in html+lhs output.
Previously the bird tracks were being omitted.
The 1.10 code assumed that each table header cell contains
exactly one block. That failed for headerless tables (0) and also
for tables with multiple blocks in a header cell.
The code is fixed and tests provided. Thanks to Andrew Lee for
pointing out the bug.
* RTF writer: Export writeRTFWithEmbeddedImages instead of
rtfEmbedImage.
* Text.Pandoc: Use writeRTFWithEmbeddedImages for RTF.
* Moved code for embedding images in RTF out of pandoc.hs.
* It no longer uses Network.URIs URI parser, which is too restrictive
(not allowing unicode URIs unless encoded).
* It allows many more schemes.
* It better handles punctuation so as to avoid capturing trailing
punctuation in bare URLs.
* In markdown reader, add a '\1' character to the beginning
of the title of an image that is alone in its paragraph,
if implicit_figures extension is selected.
* In writers, check for Para [Image alt (src,'\1':tit)] and treat
it as a figure if possible.
* Updated tests.
This is a bit of a hack, but it allows us to make implicit_figures
an extension of the markdown reader, rather than the writers.
We now (a) use anonymous links for links with inline URLs, and
(b) use an inline link instead of a reference link if the
reference link would require a label that has already been
used for a different link.
Closes#511.
Previously header ids were autogenerated by the writers.
Now they are generated (unless supplied explicitly) in the
markdown parser, if the `header_identifiers` extension is
selected.
In addition, the textile reader now supports id attributes on
headers.
Taking into account new context/latex output, and fixing
some bugs in the test suite Tests.Helpers and Tests.Writers.ConTeXt.
(We had the wrong order of expected/actual in the diff output.)
Previously the textile reader and writer incorrectly implented
RST-style autolinks for URLs and email addresses.
This has been fixed. Now an autolink is done this way:
"$":http://myurl.com
Now pandoc correctly handles hard line breaks inside list items.
Previously they broke list parsing. Thanks to Pablo
Rodríguez for pointing out the problem.
* Depend on text.
* Expose Text.Pandoc.UTF8.
* Text.Pandoc.UTF8 now exports toString, fromString,
toStringLazy, fromStringLazy.
* These are used instead of the old utf8-string functions.
Now we insert anchors after each header, and use @ref
instead of @uref for links.
Commas are now escaped as @comma{} only when needed; previously
all commas were escaped. (This change is needed, in part, because @ref
commands must be followed by a real comma or period.)
Also insert a blank line in from of @verbatim environments.
Previously the parser would hang on input like this:
[[[[[[[[[[[[[[[[[[hi
We fixed this by making the link parser parser characters
between balanced brackets (skipping brackets in inline code spans),
then parsing the result as an inline list.
One change is that
[hi *there]* bud](/url)
is now no longer parsed as a link. But in this respect pandoc behaved
differently from most other implementations anyway, so that seems okay.
All current tests pass. Added test for this case.
Closes#620.
This allows the markdown reader to treat '\begin' (not followed
by an argument) as a raw string rather than erroring out when
it doesn't find a '{'.
Closes#622.
Unescaped -'s become hyphens, while \-'s are left as ascii
minus signs. That is preferable for use with command-line
options.
See http://lintian.debian.org/tags/hyphen-used-as-minus-sign.html.
Thanks to Andrea Bolognani for bringing the issue to our
attention.
- Removed writerLiterateHaskell from WriterOptions.
- Removed readerLiterateHaskell from ReaderOptions.
- Added Ext_literate_haskell to Extensions. Test for this
instead of the above.
- Removed failUnlessLHS from Shared.
Note: At this point, +lhs and .lhs extension no longer has any effect.
Need to fix.
* Use Builder's Inlines/Blocks instead of lists.
* Return values in the reader monad, which are then
run (at the end of parsing) against the final
parser state. This allows links, notes, and
example numbers to be resolved without a second
parser pass.
* An effect of using Builder is that everything is
normalized automatically.
* New exports from Text.Pandoc.Parsing:
widthsFromIndices, NoteTable', KeyTable', Key', toKey',
withQuoteContext, singleQuoteStart, singleQuoteEnd, doubleQuoteStart,
doubleQuoteEnd, ellipses, apostrophe, dash
* Updated opendocument tests.
* Don't derive Show for ParserState.
* Benchmarks: markdown reader takes 82% of the time it took before.
Markdown writer takes 92% of the time (here the speedup is probably
due to the fact that everything is normalized by default).
To run tests, configure with --enable-tests, then 'cabal test'.
You can specify particular tests using --test-options='-t markdown'.
No output is shown unless tests fail. In the future, we can move
to the detailed-1.0 interface.
* All tables now require at least one body row.
* Renamed from 'extra' to 'pipe' tables.
* Moved functions from Parsing to Readers.Markdown.
* Cleaned up code; revised to parse in one pass rather than
parsing a raw string, splitting it, and parsing the components.
* Allow pipe tables without pipes on the ends (as PHP Markdown Extra
does).
* Use `:` form instead of `~`, for better compatibility with other
markdown implementations.
* Don't wrap the term, because it breaks definition lists.
This way you can still get the raw latex back, even if you don't
process with citeproc. Previously, cites were not visible at all
unless you specified --biblio on the command line and converted
them using citeproc, or used --natbib or --biblatex.
* The new reader is more robust, accurate, and extensible.
It is still quite incomplete, but it should be easier
now to add features.
* Text.Pandoc.Parsing: Added withRaw combinator.
* Markdown reader: do escapedChar before raw latex inline.
Otherwise we capture commands like \{.
* Fixed latex citation tests for new citeproc.
* Handle \include{} commands in latex.
This is done in pandoc.hs, not the (pure) latex reader.
But the reader exports the needed function, handleIncludes.
* Moved err and warn from pandoc.hs to Shared.
* Fixed tests - raw tex should sometimes have trailing space.
* Updated lhs-test for highlighting-kate changes.
* New module `Text.Pandoc.Docx`.
* New output format `docx`.
* Added reference.docx.
* New option `--reference-docx`.
The writer includes support for highlighted code blocks
and math (which is converted from TeX to OMML using
texmath's new OMML module).
Pandoc previously behaved like Markdown.pl for consecutive
lists of different styles. Thus, the following would be parsed
as a single ordered list, rather than an ordered list followed
by an unordered list:
1. one
2. two
- one
- two
This patch makes pandoc behave more sensibly, parsing this as
two lists. Any change in list type (ordered/unordered) or in
list number style will trigger a new list. Thus, the following
will also be parsed as two lists:
1. one
2. two
a. one
b. two
Since we regard this as a bug in Markdown.pl, and not something
anyone would ever rely on, we do not preserve the old behavior
even when `--strict` is selected.
* `---` is always em-dash, `--` is always en-dash.
* pandoc no longer tries to guess when `-` should be en-dash.
* A new option, `--old-dashes`, is provided for legacy documents.
Rationale: The rules for en-dash are too complex and
language-dependent for a guesser to work reliably. This
change gives users greater control. The alternative of
using unicode isn't very good, since unicode em- and en-
dashes are barely distinguishable in a monospace font.
Inline math uses the :math:`...` construct.
Display math uses
.. math:: ...
or if multilin
.. math::
...
These seem to be supported now by rst2latex.py.
Beamer output uses the default LaTeX template, with some
customizations via variables.
Added `writerBeamer` to `WriterOptions`.
Added `--beamer` option to `markdown2pdf`.
The container element will have the classes, id, and
key-value attributes you specified in the delimited code
block.
Previously these were stripped off.
escapeURI now only escapes space characters, leaving unicode characters
as they are, instead of converting them to octets and URL-encoding them,
as before. This gives more readable URIs. User agents now do the
percent-encoding themselves.
URIs are no longer unescaped at all on conversion to markdown, asciidoc,
rst, org.
Closes#349.
Still TODO:
- documentation in README
- add default.asciidoc to templates/
- lists
- tables
- proper escaping
- footnotes with blank lines - print separately at end?
currently they are just ignored.
- fix header (date gives weird result on pandoc README)
Previously `[@item1 and nowhere else]` yielded the locator ", and nowhere
else", or, with the new citeproc-hs, "and nowhere else".
Now it yields " and nowhere else".
A horizontal rule now gets transformed into an empty H1 header
before 'hierarchicalize' is called.
If the document that does not begin with an H1 header, an
empty one is provided.
This avoids the need for kludgy raw HTML.
Also, the 'titleslide' class is added to any section containing
just a title:
----
----
For example, in
Just a few glitches remaining.
<ul><li> In this situation, one loses the list.
</ul>
And in this, the preformatting.
<pre>Preformatted text not starting with its own blank line.
</pre>
Thansk to Dirk Laurie for noticing the issue.
So, in RST, 'http://google.com.' should be parsed as a link
to 'http://google.com' followed by a period.
The parser is smart enough to recognize balanced parentheses,
as often occur in wikipedia links: 'http://foo.bar/baz_(bam)'.
Also added ()s to RST specialChars, so '(http://google.com)'
will be parsed as a link in parens.
Added test cases.
Resolves Issue #291.
"First paragraph" as opposed to "Text body." This allows
users to specify e.g. that only paragraphs after the first
paragraph of a section are to be indented.
Thanks to Andrea Rossato for the patch.
Closes github Issue #20.
Additional related changes:
* URLs in Code in autolinks now use class "url".
* Require highlighting-kate 0.2.8.2, which omits the final <br/> tag,
essential for inline code.
Field lists now work properly with block content.
(Thanks to Lachlan Musicman for pointing out the bug.)
In addition, definition list items are now always Para instead
of Plain -- which matches behavior of rst2xml.py.
Finally, in image blocks, the alt attribute is parsed properly
and used for the alt, not also the title.
The old TeX, HtmlInline and RawHtml elements have been removed
and replaced by generic RawInline and RawBlock elements.
All modules updated to use the new raw elements.
It needs to be inside the if(strikeout) condition, after
the ulem package is imported; otherwise we try to renewcommand{\sout} when
\sout isn't yet defined.
