Starting with pandoc 2.4, citations and quoted inlines
were no longer recognized after parentheses. This is
because of commit 9b0bd4ec6f,
which is reverted here.
The point of that commit was to allow relocation of
soft line breaks to before an abbreviation, so that
a nonbreaking space could be added after the
abbreviation. Now we simply leave the soft line
break in place, even though this means that
we won't get a nonbreaking space after "Mr."
at the end of a line (and in LaTeX this may
result in a longer intersentential space).
Those who care about this issue should take care
not to end lines with an abbreviation, or to
insert nonbreaking spaces manually.
Closes#5099.
The parameter is Extensions. This allows these functions to
be sensitive to the settings of `Ext_gfm_auto_identifiers` and
`Ext_ascii_identifiers`.
This allows us to use `uniqueIdent` in the CommonMark reader,
replacing some custom code.
It also means that `gfm_auto_identifiers` can now be used
in all formats.
Semantically, `gfm_auto_identifiers` is now a modifier of
`auto_identifiers`; for identifiers to be set, `auto_identifiers`
must be turned on, and then the type of identifier produced
depends on `gfm_auto_identifiers` and `ascii_identifiers` are set.
Closes#5057.
* Lua: allow access to pandoc state
Lua filters and custom writers now have read-only access to most fields
of pandoc's internal state via the global variable `PANDOC_STATE`.
* Lua: allow iterating through fields of PANDOC_STATE
* Lua filters doc: describe CommonState
* Lua filters doc: mention global variable PANDOC_STATE
* Lua: add access to logs
Log messages can currently only be printed, but not decomposed.
T.P.GroffChar: replaced `essentialEscapes` with `manEscapes`,
which includes all the escapes mentioned in the groff_man manual.
T.P.Writers.Groff: removed escapeCode; changed parameter on
escapeString from Bool to new type `EscapeMode`.
Rewrote `escapeString`.
- `--ascii` is now turned on automatically for man output, for
portability. All man output will be escaped to ASCII.
- In T.P.Writers.Groff, `escapeChar`, `escapeString`, and
`escapeCode` now take a boolean parameter that selects
ascii-only output. This is used by the Ms writer for
`--ascii`, instead of doing an extra pass after writing
the document.
- In ms output without `--ascii`, unicode is used whenever
possible (e.g. for double quotes).
- A few escapes are changed: e.g. `\[rs]` instead of `\\` for
backslash, and `\ga]` instead of `` \` `` for backtick.
(unexported module). These are used in both the man and ms
writers.
Moved groffEscape out of Text.Pandoc.Writers.Shared [cancels earlier
API change from adding it, which was after last release].
This fixes strong/code combination on man (should be `\f[CB]` not
`\f[BC]`), mentioned in #4973.
Updated tests.
Closes#4975.
The default is `-raw_tex`, so no raw tex should result
unless we explicitly say `+raw_tex`. Previously some
raw commands did make it through.
Closes#4527.
Previously, the writer would unconditionally emit HTMLish output for
subscripts, superscripts, strikeouts (if the strikeout extension is
disabled) and small caps, even with raw_html disabled.
Now there are plain-text (and, where possible, fancy Unicode)
fallbacks for all of these corresponding (mostly) to the Markdown
fallbacks, and the HTMLish output is only used when raw_html is
enabled.
This commit adds exported functions `toSuperscript` and
`toSubscript` to `Text.Pandoc.Writers.Shared`. [API change]
Closes#4528.
Closes#4284.
Headers with the corresponding tags should not appear in the output.
If one or more of the specified tags contains a non-tag character
like `+`, Org-mode will not treat that as a valid tag, but will
nonetheless continue scanning for valid tags. That behavior is not
replicated in this patch; entering `cat+dog` as one of the entries in
`#+EXCLUDE_TAGS` and running the file through Pandoc will cause the
parser to fail and result in the only excluded tag being the default, `noexport`.
Now the `write*` functions for Docbook, HTML, ICML, JATS,
Man, Ms, OPML are sensitive to `writerPreferAscii`. Previously
the to-ascii translation was done in Text.Pandoc.App, and
thus not available to those using the writer functions
directly.
In addition, the LaTeX writer is now sensitive to
`writerPreferAscii` and to `--ascii`. 100% ASCII
output can't be guaranteed, but the writer will use
commands like `\"{a}` and `\l` whenever possible,
to avoid emiting a non-ASCII character.
A new unexported module, Text.Pandoc.Groff, has been
added to store functions used in the different groff-based
writers.
* Add support for multiprenote and multipostnote arguments.
The multiprenotes occur before the first prefix of a
multicite, and the multipostnotes follow the last suffix.
* Add test for multiprenote and multipostnote.
With this change, autolinks are parsed as Links with
the `uri` class. (The same is true for bare links, if
the `autolink_bare_uris` extension is enabled.) Email
autolinks are parsed as Links with the `email` class.
This allows the distinction to be represented in the
URI.
Formerly the `uri` class was added to autolinks by
the HTML writer, but it had to guess what was an autolink
and could not distinguish `[http://example.com](http://example.com)`
from `<http://example.com>`. It also incorrectly recognized
`[pandoc](pandoc)` as an autolink. Now the HTML writer
simply passes through the `uri` attribute if it is present,
but does not add anything.
The Textile writer has been modified so that the `uri`
class is not explicitly added for autolinks, even if it
is present.
Closes#4913.
This now allows raw LaTeX environments, `\ref`, and `\eqref` to
be parsed (which is helpful for translation HTML documents using
MathJaX).
Closes#1126.
Also foreigncblockquote, hyphenblockquote, hyphencblockquote.
Closes#4848. But note: currently foreignquote will be
parsed as a regular Quoted inline (not using the quotes
appropriate to the foreign language).
Add support for `\|`, `\b`, `\G`, `\h`, `\d`, `\f`,
`\r`, `\t`, `\U`, `\i`, `\j`, `\newtie`, `\textcircled`.
Also fall back to combining characters when composed
characters are not available.
Closes#4652.
...they should only be recognized in siunitx contexts.
For example, `\l` outside of an siunitx context should be l-slash,
not l (for liter)!
Closes#4842.
We can't always tell if it's LaTeX, ConTeXt, or plain TeX.
Better just to use "tex" always.
Also changed:
ConTeXt writer: now outputs raw "tex" blocks as well as "context".
(Closes#969).
RST writer: uses ".. raw:: latex" for "tex" content.
(RST doesn't support raw context anyway.)
Note that if "context" or "latex" specifically is desired,
you can still force that in a markdown document by using
the raw attribute (see MANUAL.txt):
```{=latex}
\foo
```
Note that this change may affect some filters, if they assume that raw
tex parsed by the Markdown reader will be RawBlock (Format "latex").
In most cases it should be trivial to modify the filters to accept
"tex" as well.
For example: `\def\foo#1[#2]{#1 and #2}`.
Closes#4768. Also fixes#4771.
API change: in Text.Pandoc.Readers.LaTeX.Types,
new type ArgSpec added. Second parameter of Macro
constructor is now `[ArgSpec]` instead of `Int`.
This fixes a regression in 2.2.3, which cause boolean values to
be parsed as MetaInlines instead of MetaBool.
Note also an undocumented (but desirable) change in 2.2.3:
numbers are now parsed as MetaInlines rather than MetaString.
Closes#4819.
* Use a Span with class "title-reference" for the default
title-reference role.
* Use B.text to split up contents into Spaces, SoftBreaks, and Strs
for title-reference.
* Use Code with class "interpreted-text" instead of Span and Str for
unknown roles. (The RST writer has also been modified to round-trip
this properly.)
* Disallow blank lines in interpreted text.
* Backslash-escape now works in interpreted text.
* Backticks followed by alphanumerics no longer end interpreted text.
Closes#4811.
RST does not allow nested emphasis, links, or other inline
constructs.
Closes#4581, double parsing of links with URLs as
link text. This supersedes the earlier fix for #4581
in 6419819b46.
Fixes#4561, a bug parsing with URLs inside emphasis.
Closes#4792.
Starting in 2.2.2, everything after an `\input` (or `\include`)
in a markdown file would be parsed as raw LaTeX.
This commit fixes the issue and adds a regression test.
Closes#4781.
Text.Pandoc.Emoji now exports `emojiToInline`, which returns a Span inline containing the emoji character and some attributes with metadata (class `emoji`, attribute `data-emoji` with emoji name). Previously, emojis (as supported in Markdown and CommonMark readers, e.g "😄")
were simply translated into the corresponding unicode code point. By wrapping them in Span
nodes, we make it possible to do special handling such as giving them a special font
in HTML output. We also open up the possibility of treating them differently when the
`--ascii` option is selected (though that is not part of this commit).
Closes#4743.
Non-ascii characters were not stripped from identifiers even if the
`ascii_identifiers` extension was enabled (which is is by default for
gfm).
Closes#4742
* Markdown writer now includes a blank line at the end
of the row in a single-row multiline table, to prevent it from being
interpreted as a simple table. Closes#4578.
* Markdown reader does a better job computing the relative width of
the last column in a multiline table, so we can round-trip tables
without constantly shrinking the last column.
Previously we used an odd mix of 3- and 4-space indentation.
Now we use 3-space indentation, except for ordered lists,
where indentation must depend on the width of the list marker.
Closes#4563.
* Annotate gridTable code with comments and abstract small functions
* Don't wrap lines in tables when `--wrap=none`. Instead, expand cells, even if
it results in cells that don't respect relative widths or surpass page column width.
* This change affects RST, Markdown, and Haddock writers.
and not in the title. If it's in the title, then we get
a titlebar on slides with the `plain` attribute, when
the id is non-null. This fixes a regression from 1.9.x.
Closes#4307.
Fixes#2609.
This PR introduces the new-style section headings: `\section[my-header]{My Header}` -> `\section[title={My Header},reference={my-header}]`.
On top of this, the ConTeXt writer now supports the `--section-divs` option to write sections in the fenced style, with `\startsection` and `\stopsection`.
Closes#4281.
Previously we allowed "nonindent spaces" before the
opening and closing `:::`, but this interfered with
list parsing, so now we require the fences to be
flush with the margin of the containing block.
rst2latex.py uses an align* environment for math in
`.. math::` blocks, so this math may contain line breaks.
If it does, we put the math in an `aligned` environment
to simulate rst2latex.py's behavior.
Closes#4254.
The change both improves performance and fixes a
regression whereby normal citations inside inline notes
were not parsed correctly.
Closesjgm/pandoc-citeproc#315.
even if the `latex_macros` extension is set.
This reverts to earlier behavior and is probably safer
on the whole, since some macros only modify things in
included packages, which pandoc's macro expansion can't
modify.
Closes#4246.
Don't pass through macro definitions themselves when `latex_macros`
is set. The macros have already been applied.
If `latex_macros` is enabled, then `rawLaTeXBlock` in
Text.Pandoc.Readers.LaTeX will succeed in parsing a macro definition,
and will update pandoc's internal macro map accordingly, but the
empty string will be returned.
Together with earlier changes, this closes#4179.
Previously we erroneously included the enclosing
backticks in a reference ID (closes#4156).
This change also disables interpretation of
syntax inside references, as in docutils.
So, there is no emphasis in
`my *link*`_
* Deprecate `--strip-empty-paragraphs` option. Instead we now
use an `empty_paragraphs` extension that can be enabled on
the reader or writer. By default, disabled.
* Add `Ext_empty_paragraphs` constructor to `Extension`.
* Revert "Docx reader: don't strip out empty paragraphs."
This reverts commit d6c58eb836.
* Implement `empty_paragraphs` extension in docx reader and writer,
opendocument writer, html reader and writer.
* Add tests for `empty_paragraphs` extension.
Previously both needed to be specified (unless the image was
being resized to be smaller than its original size).
If height but not width is specified, we now set width to
textwidth (and similarly if width but not height is specified).
Since we have keepaspectratio, this yields the desired result.
This fixes a bug where pandoc would stop parsing a URI with an
empty attribute: for example, `&a=&b=` wolud stop at `a`.
(The uri parser tries to guess which punctuation characters
are part of the URI and which might be punctuation after it.)
Closes#4068.
Previously we got a crash, because we were trying to print
a native cmark STRIKETHROUGH node, and the commonmark writer
in cmark-github doesn't support this. Work around this by
using a raw node to add the strikethrough delimiters.
Closes#4038.
* Move as much as possible to the CSS in the template.
* Ensure that all the HTML-based templates (including epub)
contain the CSS for columns.
* Columns default to 50% width unless they are given a width
attribute.
Closes#4028.
* Remove "width" attribute which is not allowed on div.
* Remove space between `<div class="column">` elements,
since this prevents columns whose widths sum to 100%
(the space takes up space).
Closes#4028.
and other non-HTML formats (`Text.Pandoc.Readers.HTML.htmlTag`).
The parser stopped at the first `>` character, even if it wasn't
the end of the comment.
Closes#4019.
Previously `\include` wouldn't work if the included file
contained, e.g., a begin without a matching end.
We've changed the Tok type so that it stores a full SourcePos,
rather than just a line and column. So tokens keeep track
of the file they came from. This allows us to use a simpler
method for includes, which doesn't require parsing the included
document as a whole.
Closes#3971.
* Options: Added readerStripComments to ReaderOptions.
* Added `--strip-comments` command-line option.
* Made `htmlTag` from the HTML reader sensitive to this feature.
This affects Markdown and Textile input.
Closes#2552.
Div's are difficult to translate into org syntax, as there are multiple
div-like structures (drawers, special blocks, greater blocks) which all
have their advantages and disadvantages. Previously pandoc would
use raw HTML to preserve the full div information; this was rarely
useful and resulted in visual clutter. Div-rendering was changed to
discard the div's classes and key-value pairs if there is no natural way
to translate the div into an org structure.
Closes: #3771
Closes#3511.
Previously pandoc used the four-space rule: continuation paragraphs,
sublists, and other block level content had to be indented 4
spaces. Now the indentation required is determined by the
first line of the list item: to be included in the list item,
blocks must be indented to the level of the first non-space
content after the list marker. Exception: if are 5 or more spaces
after the list marker, then the content is interpreted as an
indented code block, and continuation paragraphs must be indented
two spaces beyond the end of the list marker. See the CommonMark
spec for more details and examples.
Documents that adhere to the four-space rule should, in most cases,
be parsed the same way by the new rules. Here are some examples
of texts that will be parsed differently:
- a
- b
will be parsed as a list item with a sublist; under the four-space
rule, it would be a list with two items.
- a
code
Here we have an indented code block under the list item, even though it
is only indented six spaces from the margin, because it is four spaces
past the point where a continuation paragraph could begin. With the
four-space rule, this would be a regular paragraph rather than a code
block.
- a
code
Here the code block will start with two spaces, whereas under
the four-space rule, it would start with `code`. With the four-space
rule, indented code under a list item always must be indented eight
spaces from the margin, while the new rules require only that it
be indented four spaces from the beginning of the first non-space
text after the list marker (here, `a`).
This change was motivated by a slew of bug reports from people
who expected lists to work differently (#3125, #2367, #2575, #2210,
#1990, #1137, #744, #172, #137, #128) and by the growing prevalance
of CommonMark (now used by GitHub, for example).
Users who want to use the old rules can select the `four_space_rule`
extension.
* Added `four_space_rule` extension.
* Added `Ext_four_space_rule` to `Extensions`.
* `Parsing` now exports `gobbleAtMostSpaces`, and the type
of `gobbleSpaces` has been changed so that a `ReaderOptions`
parameter is not needed.
Acronyms are not resolved by the reader, but acronym and glossary information is put into attributes on Spans so that they can be processed in filters.
The structure expected is:
<div class="columns">
<div class="column" width="40%">
contents...
</div>
<div class="column" width="60%">
contents...
</div>
</div>
Support has been added for beamer and all HTML slide formats.
Closes#1710.
Note: later we could add a more elegant way to create
this structure in Markdown than to use raw HTML div elements.
This would come for free with a "native div syntax" (#168).
Or we could devise something specific to slides
We assume that comments are defined as parsed by the
docx reader:
I want <span class="comment-start" id="0" author="Jesse Rosenthal"
date="2016-05-09T16:13:00Z">I left a comment.</span>some text to
have a comment <span class="comment-end" id="0"></span>on it.
We assume also that the id attributes are unique and properly
matched between comment-start and comment-end.
Closes#2994.
