Issue #1977
Most markdown processors support the [shortcut format] for reference links.
Pandoc's markdown reader parsed this shortcuts unoptionally.
Pandoc's markdown writer (with --reference-links option) never shortcutted links.
This commit adds an extension `shortcut_reference_links`. The extension is
enabled by default for those markdown flavors that support reading shortcut
reference links, namely:
- pandoc
- strict pandoc
- github flavoured
- PHPmarkdown
If extension is enabled, reader parses the shortcuts in the same way as
it preveously did. Otherwise it would parse them as normal text.
If extension is enabled, writer outputs shortcut reference links unless
doing so would cause problems (see test cases in `tests/Tests/Writers/Markdown.hs`).
The `tabular` environment allows non-empty column separators
with the "@{...}" syntax. Previously, pandoc would fail to
parse tables if a non-empty colsep was present. With this
commit, these separators are still ignored, but the table gets
parsed. A test case is included.
The `tabular` environment takes an optional parameter for
vertical alignment. Previously, pandoc would fail to parse
tables if this parameter was present. With this commit,
the parameter is still ignored, but the table gets
parsed. A test case is included.
Org links like `[[file:target][title]]` were not handled correctly,
parsing the link target verbatim. The org reader is changed such that
the leading `file:` is dropped from the link target.
This is related to issues #756 and #1812.
Move recursive role lookup from renderRole to addNewRole. The Attr value
will be the same for every occurance of this role, so there's no reason
to compute it every time. This allows simplifying the
stateRstCustomRoles map considerably.
We could go even further, and remove the fmt and attr arguments to
renderRole, which are null except for custom roles.
The class directive accepts one or more class names, and creates a Div
value with those classes. If the directive has an indented body, the
body is parsed as the children of the Div. If not, the first block
folowing the directive is made a child of the Div.
This differs from the behavior of rst2xml, which does not create a Div
element. Instead, the specified classes are applied to each child of
the directive. However, most Pandoc Block constructors to not take an
Attr argument, so we can't duplicate this behavior.
closes#65
RST quoted literal blocks are the same as indented literal blocks (which
pandoc already supports) except that the quote character is preserved in
each line.
This includes test cases for the quoted literal block, as well as
additional tests for line blocks and indented literal blocks, to verify
that these are unaffected by the changes.
While empty links are not allowed in Emacs org-mode, Pandoc org-mode
should support them: gitit relies on empty links as they are used to
create wiki links.
Fixesjgm/gitit#471
The org reader was to restrictive when parsing links, some relative
links and links to files given as absolute paths were not recognized
correctly. The org reader's link parsing function was amended to handle
such cases properly.
This fixes#1741
Document trees under a header starting with the word `COMMENT` are
comment trees and should not be exported. Those trees are dropped
silently.
This closes#1678.
Things like `/hello,/` or `/hi'/` were falsy recognized as emphasised
strings. This is wrong, as `,` and `'` are forbidden border chars and
may not occur on the inner border of emphasized text. This patch
enables the reader to matches the reference implementation in that it
reads the above strings as plain text.
Fixes issue with top-level bullet list parsing.
Previously we would use `many1 spaceChars` rather than respecting
the list's indent level. We also permitted `*` bullets on unindented
lists, which should unambiguously parse as `header 1`.
Combined, this meant headers at a different indent level were
being unwittingly slurped into preceding bullet lists, as per
Issue #1650.
Currently, pandoc has hard-coded the following in order to make tight lists in
LaTeX:
```hs
text "\\itemsep1pt\\parskip0pt\\parsep0pt"
```
Which is fine, but does not allow customizations. For example, the `memoir`
class already has a `\tightlist` declaration for this purpose:
```tex
\newcommand{\tightlist}{%
\setlength{\itemsep}{0pt}\setlength{\parskip}{0pt}}
```
I'm proposing to use a similar solution:
```diff
@@ In Writers/LaTeX.hs:
-then text "\\itemsep1pt\\parskip0pt\\parsep0pt"
+then text "\\tightlist"
@@ In templates/default.latex:
+\newcommand{\tightlist}{%
+ \setlength{\itemsep}{1pt}\setlength{\parskip}{0pt}\setlength{\parsep}{0pt}}
```
This allows us to customize the tightness to our needs.
Backward Compatibility
If a person is using a custom LaTeX template (not based upon the `memoir`
class), the `\tightlist` declaration must be added.
Previously text that ended a div would be parsed as Plain
unless there was a blank line before the closing div tag.
Test case:
<div class="first">
This is a paragraph.
This is another paragraph.
</div>
Closes#1591.
Renamed some tests, introducing subsidiary directories
for fb2, docx, epub.
Cleaned up tests in cabal file.
Combined dokuwiki-writer and dokuwiki_inline_formatting tests.
This was just too fragile and dependent on a changing Cabal API
(see #1526).
Instead of passing the bulid directory to the test program, we
now let the test program find itself (using executable-path)
and then find the pandoc executable relative to itself.
Rewrote features test to remove all unimplemented features.
There are now all three examples of where an image can be included in
the test.
1. Cover image
2. As a spine elemnt
3. In the document
Tests have also been added to make sure that the mediabag contains all
these images after processing.
Renamed epub test files so they're identified more clearly as
epub: features.{epub,native} -> epub.features.{epub,native},
and similarly with formatting.{epub,native}.
Added epub test files to cabal file, so they'll be included in
the tarball.
Using `map toUpper` to capitalise text is wrong, as e.g.
“Straße” should be converted to “STRASSE”, which is 1 character
longer. This commit adds a `capitalize` function and replaces
2 identical implementations in different modules (`toCaps` and
`capitalize`) with it.
This will allow us to test the whole mediabag (making sure, for example,
that images are added with the correct keys) instead of just individual
extracted images. We compare each entry in the media bag to an image
extracted on the fly from the docx. As a result, we only need one file
to test with.
The image in the current tests was also replaced with a smaller one.
Moved `MediaBag` definition and functions from Shared:
`lookupMedia`, `mediaDirectory`, `insertMedia`, `extractMediaBag`.
Removed `emptyMediaBag`; use `mempty` instead, since `MediaBag`
is a Monoid.
This will make paragraphs styled with `Author`, `Title`, `Subtitle`,
`Date`, and `Abstract` into pandoc metavalues, rather than text. The
implementation only takes those elements from the beginning of the
document (ignoring empty paragraphs).
Multiple paragraphs in the `Author` style will be made into a metaList,
one paragraph per item. Hard linebreaks (shift-return) in the paragraph
will be maintained, and can be used for institution, email, etc.
* This change brings pandoc's definition list syntax into alignment
with that used in PHP markdown extra and multimarkdown (with the
exception that pandoc is more flexible about the definition markers,
allowing tildes as well as colons).
* Lazily wrapped definitions are now allowed; blank space is required
between list items; and the space before definition is used to
determine whether it is a paragraph or a "plain" element.
* For backwards compatibility, a new extension,
`compact_definition_lists`, has been added that restores the behavior
of pandoc 1.12.x, allowing tight definition lists with no blank space
between items, and disallowing lazy wrapping.
This gives better results for tight lists. Closes#1437.
An alternative solution would be to use Para everywhere, and
never Plain. I am not sufficiently familiar with org to know
which is best. Thoughts, @tarleb?
Adds support to the org reader for conditionally exporting either the code block,
results block immediately following, both, or neither, depending on the value
of the `:exports` header argument. If no such argument is supplied, the default
org behavior (for most languages) of exporting code is used.
- We no longer include trailing spaces and newlines in the
raw blocks.
- We look for closing tags for elements (but without backtracking).
- Each block-level tag is its own RawBlock; we no longer try to
consolidate them (though `--normalize` will do so).
Closes#1330.
* Added normalizeInlines, normalizeBlocks.
* Type signature is now more narrow, `Pandoc -> Pandoc` instead of
`Data a :: a -> a`. Some users may need to change their uses of
`normalize` to the newly exported `normalizeInlines` or
`normalizeBlocks`.
We want to treat it as a plain paragraph if the hanging amount is
greater to or equal to the left indent---i.e., if the first line has
zero indentation. But we still want it to be a block quote if it starts
to the right of the margin. Someone might format verse with wrapping
lines with a hanging indent, for example.
This is what seems like the sensible default: read in insertions, and
ignore deletions. In the future, it would be good if options were
available for either taking in deletions or keeping both in some
scriptable format.
Add torture-test for new normalization functions.
One problem that this test demonstrates is that word has a tendency to
turn off formatting at a space, and then turn it back on after. I'm not
sure yet whether this is something we should fix.
This is just a wrapper around Pandoc that doesn't normalize with
`toString`. We want to make sure that our own normalization process
works. If, in the future, we are able to hook into the builder's
normalization, this will be removed.
Closes#1345. Also relabeled 'code' and 'verbatim' parsers
to accord with the org-mode manual.
I'm not sure what the distinction between code and verbatim
is supposed to be, but I'm pretty sure both should be represented
as Code inlines in pandoc. The previous behavior resulted in the
text not appearing in any output format.
Inline LaTeX is now accepted and parsed by the org-mode reader. Both,
math symbols (like \tau) and LaTeX commands (like \cite{Coffee}), can be
used without any further escaping.
Citations are defined via the "normal citation" syntax used in markdown,
with the sole difference that newlines are not allowed between "[...]".
This is for consistency, as org-mode generally disallows newlines
between square brackets.
The extension is turned on by default and can be turned off via the
default syntax-extension mechanism, i.e. by specifying "org-citation" as
the input format.
Move `citeKey` from Readers.Markdown into Parsing
The function can be used by other readers, so it is made accessible for
all parsers.
The reader produced wrong results for block containing non-letter chars
in their parameter arguments. This patch relaxes constraints in that it
allows block header arguments to contain any non-space character (except
for ']' for inline blocks).
Thanks to Xiao Hanyu for noticing this.
The general form of source block headers
(`#+BEGIN_SRC <language> <switches> <header arguments>`) was not
recognized by the reader. This patch adds support for the above form,
adds header arguments to the block's key-value pairs and marks the block
as a rundoc block if header arguments are present.
This closes#1286.
Org's inline code blocks take forms like `src_haskell(print "hi")` and
are frequently used to include results from computations called from
within the document. The blocks are read as inline code and marked with
the special class `rundoc-block`. Proper handling and execution of
these blocks is the subject of a separate library, rundoc, which is
work in progress.
This closes#1278.
Org allows users to define their own custom link types. E.g., in a
document with a lot of links to Wikipedia articles, one can define a
custom wikipedia link-type via
#+LINK: wp https://en.wikipedia.org/wiki/
This allows to write [[wp:Org_mode][Org-mode]] instead of the
equivallent [[https://en.wikipedia.org/wiki/Org_mode][Org-mode]].
Internal links in Org are possible by using an anchor-name as the target
of a link:
[[some-anchor][This]] is an internal link.
It links <<some-anchor>> here.
Footnotes can consist of multiple blocks and end only at a header or at
the beginning of another footnote. This fixes the previous behavior,
which restricted notes to a single paragraph.
Support for standard org-blocks is improved. The parser now handles
"HTML", "LATEX", "ASCII", "EXAMPLE", "QUOTE" and "VERSE" blocks in a
sensible fashion.
These are primarily aimed at testing the new treatment of line breaks,
but hopefully other tests can be added more easily now as features
and changes are implemented in the writer.
Adapted from Tests.Writers.HTML.tests.
This fixes the org-reader's handling of sub- and superscript
expressions. Simple expressions (like `2^+10`), expressions in
parentheses (`a_(n+1)`) and nested sexp (like `a_(nested()parens)`) are
now read correctly.
Support all of the following variants as valid ways to define inline or
display math inlines:
- `\[..\]` (display)
- `$$..$$` (display)
- `\(..\)` (inline)
- `$..$` (inline)
This closes#1223. Again.
Text such as /*this*/ was not correctly parsed as a strong, emphasised
word. This was due to the end-of-word recognition being to strict as it
did not accept markup chars as part of a word. The fix involves an
additional parser state field, listing the markup chars which might be
parsed as part of a word.
Going forward we'll use pandoc-citeproc, as an external filter.
The `--bibliography`, `--csl`, and `--citation-abbreviation` fields
have been removed. Instead one must include `bibliography`, `csl`,
or `csl-abbrevs` fields in the document's YAML metadata. The filter
can then be used as follows:
pandoc --filter pandoc-citeproc
The `Text.Pandoc.Biblio` module has been removed. Henceforth,
`Text.CSL.Pandoc` from pandoc-citations can be used by library users.
The Markdown and LaTeX readers now longer format bibliographies and
citations. That must be done using `processCites` or `processCites'`
from Text.CSL.Pandoc.
All bibliography-related fields have been removed from `ReaderOptions`
and `WriterOptions`: `writerBiblioFiles`, `readerReferences`,
`readerCitationStyle`.
API change.
The code:
~~~{#test}
asdf
~~~
gets compiled to html:
<pre id="test">
asdf
</pre>
So it is possible to link to the identifier `test`
But this doesn't happen on latex
When using the listings package (`--listings`) it is possible to set the
identifier using the `label=test` property:
\begin{lstlisting}[label=id]
hi
\end{lstlisting}
And this is exactly what this patch is doing.
Modified LaTeX Reader/Writer and added tests for this.
* Add ??? as fallback text for non-resolved citations.
* Biblio: Put references (including a header at the end of
the document, if one exists) inside a Div with class "references".
This gives some control over styling of references, and allows
scripts to manipulate them.
* Markdown writer: Print markdown citation codes, and disable
printing of references, if `citations` extension is enabled.
NOTE: It would be good to improve what citeproc-hs does for
a nonexistent key.
I've found some incorrect behaviours with the dokuwiki output, for which
extra test cases will be needed - that aren't covered by the standard
pandoc test input files.
* Depend on pandoc 1.12.
* Added yaml dependency.
* `Text.Pandoc.XML`: Removed `stripTags`. (API change.)
* `Text.Pandoc.Shared`: Added `metaToJSON`.
This will be used in writers to create a JSON object for use
in the templates from the pandoc metadata.
* Revised readers and writers to use the new Meta type.
* `Text.Pandoc.Options`: Added `Ext_yaml_title_block`.
* Markdown reader: Added support for YAML metadata block.
Note that it must come at the beginning of the document.
* `Text.Pandoc.Parsing.ParserState`: Replace `stateTitle`,
`stateAuthors`, `stateDate` with `stateMeta`.
* RST reader: Improved metadata.
Treat initial field list as metadata when standalone specified.
Previously ALL fields "title", "author", "date" in field lists
were treated as metadata, even if not at the beginning.
Use `subtitle` metadata field for subtitle.
* `Text.Pandoc.Templates`: Export `renderTemplate'` that takes a string
instead of a compiled template..
* OPML template: Use 'for' loop for authors.
* Org template: '#+TITLE:' is inserted before the title.
Previously the writer did this.
The _note attribute is supported. This is unofficial, but
used e.g. in OmniOutliner and supported by multimarkdown.
We treat the contents as markdown blocks under a section
header.
Added to documentation and tests.
* Reverts 1.11 change that caused citations to be rendered as
markdown citations, even if `--biblio` was specified, unless
`citation` extension is disabled. Now, formatted citations
are always printed if `--biblio` was specified. If you want to
reformat markdown keeping pandoc markdown citations intact,
just don't specify `--biblio`.
* Reverted now unnecessary changes to Text.Pandoc.Biblio adding the raw
block to mark the bibliography, and to Text.Pandoc.Writers.Markdown
to remove the bibliography if `citations` not specified.
* If the content of a `Cite` inline is a `RawInline "latex"`, which
means that a LaTeX citation command was parsed and `--biblio` wasn't
specified, then render it as a pandoc markdown citation. This means
that `pandoc -f latex -t markdown`, without `--biblio`, will convert
LaTeX citation commands to pandoc markdown citations.
Previously, a LaTeX citation would always be parsed as a Citation
element, with the raw LaTeX in the [Inline] part.
Now, the LaTeX citation is parsed as a Citation element only if
`--biblio` was specified (i.e. only if there is a nonempty set
of references in readerReferences). Otherwise it is parsed as
raw LaTeX.
This will make it possible to simplify some things in the markdown
writer. It also makes the LaTeX reader behave more like the Markdown
reader.
Previously citations were rendered as citeproc-formatted citations
by default. Now we render them as pandoc citations, e.g. `[@item1]`,
unless the `citations` extension is disabled.
If you still want formatted citations in your markdown output,
use `pandoc -t markdown-citations`.
* Moved code for translating listings language names to
highlighting-kate names and back from LaTeX reader to Highlighting.
* Text.Pandoc.Highlighting no longer exposed (API change)
* Text.Pandoc.Highlighting exports toListingsLang, fromListingsLang
Previously header ids were autogenerated by the writers.
Now they are generated (unless supplied explicitly) in the
markdown parser, if the `header_identifiers` extension is
selected.
In addition, the textile reader now supports id attributes on
headers.
Taking into account new context/latex output, and fixing
some bugs in the test suite Tests.Helpers and Tests.Writers.ConTeXt.
(We had the wrong order of expected/actual in the diff output.)
