- With epub extensions, check for epub:type in addition to type.
- Fix problem with noteref parsing which caused block-level
content to be eaten with the noteref.
- Rename pAnyTag to pAny.
- Refactor note resolution.
Version specifiers like `PANDOC_VERSION` and `PANDOC_API_VERSION` are
turned into `Version` objects. The objects simplify version-appropriate
comparisons while maintaining backward-compatibility.
A function `pandoc.types.Version` is added as part of the newly
introduced module `pandoc.types`, allowing users to create version
objects in scripts.
Move up the pattern match to be reachable, closes#5517.
Previously `file:/` URLs were handled wrongly and pandoc attempted
to make HTTP requests, which failed.
add `onlySimpleTableCells` to `Text.Pandoc.Shared`
[API change]
This fixes an inconsistency in the HTML reader, which did not treat tables with `<p>` inside cells as simple.
Previously if labels had integer names, it could produce a conflict
with auto-labeled reference links. Now we test for a conflict and find
the next available integer.
Note that this involves adding a new state variable `stPrevRefs` to
keep track of refs used in other document parts when using
`--reference-location=block|section`
Closes#5495
This makes use of tasty-lua, a package to write tests in Lua
and integrate the results into Tasty output. Test output becomes
more informative: individual tests and test groups become visible
in test output. Failures are reported with helpful error messages.
"Best Practice: When footnotes are grouped at the end of an article,
wrap them in a `<fn-group>` and use an `<xref>` element in the text, as
usual, to tie each footnote in the list to a particular location in the
text."
Closes#5511.
Planning info is now always placed before the subtree contents.
Previously, the planning info was placed after the content if the
header's subtree was converted to a list, which happens with headers of
level 3 and higher per default.
Fixes: #5494
Emacs always uses two spaces when indenting the content of src blocks,
e.g., when exiting a `C-c '` edit-buffer. Pandoc used to indent contents
by the space-equivalent of one tab, but now always uses two spaces, too.
Closes: #5440
The `system` Lua module provides utility functions to interact with the
operating- and file system. E.g.
print(pandoc.system.get_current_directory())
or
pandoc.system.with_temporary_directory('tikz', function (dir)
-- write and compile a TikZ file with pdflatex
end)
iOS chooses to render a number of Unicode entities,
including '↩', as big colorful emoji. This can be
defeated by appending Unicode
VARIATION SELECTOR-15'/'VARIATION SELECTOR-16'.
So we now append this character when escaping
strings, for both '↩' and '↔'.
If other characters prove problematic, they can
simply be added to needsVariationSelector.
Closes#5469.
1) Don't append `.html`
2) Add `wikilink` title
This mirrors behavior of other wiki readers. Generally the
`.html` extension is not wanted. It may be important for
output to HTML in certain circumstances, but it can always
be added using a filter that matches on links with title
`wikilink`.
Note that if you have a workflow that uses pandoc to convert
vimwiki to readable HTML pages, you may need to add such a
filter to reproduce current behavior.
Here is a filter that does the job:
```lua
function Link(el)
if el.title == 'wikilink' then
el.target = el.target .. ".html"
end
return el
end
```
Save this as `fixlinks.lua` and use with `--lua-filter fixlinks.lua`.
Closes#5414.
The previous built-in reference doc had only title and content
layouts. Add in a section-header slide and a two-content slide, so
users can more easily modify it to build their own templates.
Golden files needed to be regenerated. Checked on MS PowerPoint 2013.
Improve the workaround for #1658, adapting a solution by @u-fischer in
<https://github.com/latex3/latex2e/issues/131> that works whether or not
the `indent` variable is enabled.
Remove `subparagraph` variable in LaTeX template. The default is now
to use run-in style for level 4 and 5 headings (`\paragraph` and `\subparagraph`).
To get the previous default behavior (where these were formatted as blocks,
like `\subsubsection`), set the `block-headings` variable.
An example is given in the manual of reformatting the appearance of headings
more thoroughly using KOMA-Script.
Closes#5365.
The HTML writer adds the `data-` prefix for HTML5
for nonstandard attributes. But the attributes are
represented in the AST without the `data-` prefix,
so we should strip this when reading HTML.
Closes#5392.
* LaTeX template: Group graphics-related code
The default figure placement was added in <f3ab4bc2b9>; there does not appear to have been a reason for placing it at the end of the preamble.
* Update tests
This fixes a bug wherein footnotes appeared in the wrong
order, and with duplicate numbers, when in table captions
and cells.
We now use regular `\footnote` commands, even in the table
caption and the minipages containing cells. Apparently
longtable knows how to handle this.
Closes#5367.
+ LaTeX template: Improve template readability
Use `hidelinks` option for hyperref, which has the same effect as `pdfborder={0 0 0}`,
but its purpose is clearer. Use a simpler conditional for Polyglossia/Babel. Format
comments more consistently.
+ Update tests
+ Remove hyperref breaklinks option. According to the documentation, hyperref
sets this automatically as appropriate to the driver.
...in numbering.xml. This caused pandoc-produced docx files to
be uneditable using Word Online.
The problem was that recent versions of reference.docx include
samples of various kinds of text, including lists. The
numering elements for these were getting copied over to
the new docx, where they clashed with the autogenerated
elements produced by pandoc. This didn't confuse Desktop
Word, but it did confuse Word Online.
Closes#5358.
Instead of `$HOME/.pandoc`, the default user data directory is
now `$XDG_DATA_HOME/pandoc`, where `XDG_DATA_HOME` defaults to
`$HOME/.local/share` but can be overridden by setting the environment
variable.
If this directory is missing, then `$HOME/.pandoc` is searched
instead, for backwards compatibility. However, we recommend
moving local pandoc data files from `$HOME/.pandoc` to
`$HOME/.local/share/pandoc`.
On Windows the default user data directory remains the same.
Closes#3582.
This improves on the original fix to #5285 by preventing
other mixed lists (lists with a mix of Plain and Para
elements) that were allowed given the original fix.
This ensures that a figure containing a single image
is parsed as a pandoc "implicit figure" (i.e., a
Para with a single Image whose title attribute begins
with `fig:`). More complex figures will still be parsed
as divs.
Closes#5321.
Previously parsing would break if the code block
contained a string of backticks of sufficient length
followed by something other than end of line.
Closes#5304.
Closes#5285. Previously the algorithm allowed list items
with a mix of Para and Plain, which is never wanted.
compactify in T.P.Shared has been modified so that, if
a list's items contain (at the top level) Para elements
(aside from perhaps at the very end), ALL Plains are
converted to Paras.
Quite a few modules were missing copyright notices.
This commit adds copyright notices everywhere via haddock module
headers. The old license boilerplate comment is redundant with this and has
been removed.
Update copyright years to 2019.
Closes#4592.
Otherwise last block gets parsed as a Plain rather than
a Para.
This is a regression in pandoc 2.x. This patch restores
pandoc 1.19 behavior.
Closes#5271.
`\ldots{}.` doesn't behave as well as `\ldots.` with the latex
ellipsis package. This patch causes pandoc to avoid emitting
the `{}` when it is not necessary. Now `\ldots` and other
control sequences used in escaping will be followed by either
a `{}`, a space, or nothing, depending on context.
Thanks to Elliott Slaughter for the suggestion.
* docx writer: support custom properties. Solves the writer part of #3024.
Also supports additional core properties: `subject`, `lang`, `category`,
`description`.
* odt writer: improve standard properties, including the following core properties:
`generator` (Pandoc/VERSION), `description`, `subject`, `keywords`,
`initial-creator` (from authors), `creation-date` (actual creation date).
Also fix date.
* pptx writer: support custom properties. Also supports additional core
properties: `subject`, `category`, `description`.
* Includes golden tests.
* MANUAL: document metadata support for docx, odt, pptx writers
Otherwise we get an error when trying to compile code
with lua or assembler code.
To change the default dialect (currenty 5.3 for lua
and x86masm for assembler), you can use `--include-in-header`
to inject something like
\lstset{defaultdialect=[5.2]Lua}
Closes#5227.
Ensure that `\part` and `\chapter` are only numbered if `numbersections` is set. To return to the previous behaviour, use `-V numbersections -V secnumdepth=0`.
Notes on secnumdepth:
1 = Number `\section`
0 = Number `\chapter`
-1 = Number `\part`
-2 = No numbering
Incorrect types to pandoc element constructors are automatically
converted to the correct types when possible. This was already done for
most constructors, but conversions are now also done for nested
types (like lists of lists).
Simplify the approach of #5212, ensuring that `mainfont` is used as the base font for scaling and that LuaLaTeX does not display the mono font with TeX ligatures (as it does not use the `Mapping=tex-ansi` option).
With a modified version of `\defaultfontfeatures`, fontspec will continue to report scaling against the old default font in the log, but it nonetheless displays main font at the specified size. Using this rather than setting `Scale=MatchLowercase` for each family individually means that users will not lose scaling when upgrading to the new template if they were using other font options. Scaling can be disabled for an individual family by adding the option `Scale=1` to `sansfontoptions`, `monofontoptions`, etc.
Remove the `\setromanfont` command added in #4665, as this is not documented in the fontspec manual and appears to be a deprecated alias for `\setmainfont`.
For the release notes, I should also add that one can imitate the previous appearance with `-V mainfontoptions="Scale=MatchLowercase"`.
When `Scale=MatchLowercase` is set as a default font option, this scales `mainfont` against the old default, meaning that it resizes whatever is set as the main font to match the metrics of Latin Modern. This can result, for example, in a document set to 12pt appearing in 11pt or 13pt. Setting this option for individual families allows everything to scale against the main font, and permits the user to override the setting if desired. Note that it is not necessary to specify `Ligatures=TeX`, as this is already set by default for the appropriate families. See the `fontspec` manual: <https://ctan.org/pkg/fontspec>.
Those who specify font-options in metadata may need to add `Scale=MatchLowercase`, which will now only be provided if `(roman|sans|math|mono)fontoptions` aren't given explicitly.
* These were added by the RST reader and, for literate Haskell,
by the Markdown and LaTeX readers. There is no point to
this class, and it is not applied consistently by all readers.
See #5047.
* Reverse order of `literate` and `haskell` classes on code blocks
when parsing literate Haskell. Better if `haskell` comes first.
* Use Babel for LuaTeX. There are a number of bugs in Polyglossia under LuaLaTeX with common languages, e.g. <https://github.com/reutenauer/polyglossia/issues/182>.
* Load xcolor. The `xcolor` package must be loaded before the `footnote` package, which we load to fix foonotes in tables. Closes#4861.
* Load xurl if available. This breaks URLs in more locations. Only available with TeX Live 2018 and later.
* Remove obsolete fixltx2e package
* Reindent. Use two spaces to be consistent internally and with other templates.
* Use `bookmark` if available. The `bookmark` package can sometimes correct the levels of headings where `hyperref` cannot: see <https://komascript.de/release3.26>.
* Update LaTeX tests
* FIX#5183 : zimwiki : remove automatic colon prefix before internal images
![](foo.png) should be converted to {{foo.png}} (relative path)
![](/foo.png] should be converted to {{/foo.png}} (absolute path)
Therefore the ':' prefix is useless and must be removed.
I never used the zimwiki, but i submitted the similar dokuwiki fix.
1. The zimwiki syntax is inspired by dokuwiki
2. The zimwiki documentation does not mention the colon character for images
3. The pandoc zimwiki writer seems to be a copy-paste for the dokuwiki writer
If the PR #5184 is applied, I think this one should be applied too.
* FIX#5183 : remove automatic ':' prefix before internal image links
`![](foo.png)` should be converted to `{{foo.png}}` (relative path)
`![](/foo.png]` should be converted to `{{/foo.png}}` (absolute path)
Therefore the ':' prefix is useless and must be removed.
It blocks users from making relative image links.
Update tests for DokuWiki Writer : external images
Previously we used HsYAML's decodeStrict to recognize
boolean values (treating everything else as a string).
This caused problems relating to hvr/HsYAML#7.
We now just check for the recognized boolean values
`true|True|TRUE|false|False|FALSE`, and avoid using
HsYAML.
Closes#5177.
Captions used to have the word 'caption' prepended; this
has been removed.
Also, 'thumb' is used instead of 'frame' to allow images
to be resized.
Closes#5105.
Starting with pandoc 2.4, citations and quoted inlines
were no longer recognized after parentheses. This is
because of commit 9b0bd4ec6f,
which is reverted here.
The point of that commit was to allow relocation of
soft line breaks to before an abbreviation, so that
a nonbreaking space could be added after the
abbreviation. Now we simply leave the soft line
break in place, even though this means that
we won't get a nonbreaking space after "Mr."
at the end of a line (and in LaTeX this may
result in a longer intersentential space).
Those who care about this issue should take care
not to end lines with an abbreviation, or to
insert nonbreaking spaces manually.
Closes#5099.
Previously lists were indented by half an inch on the first line
for each level of nesting. This resulted in lists that looked like
this:
1. The first line of the list point text
the second line of the same list point.
Fix this and bring style into line with libreoffice standards:
1. The first line of the list point text
the second line of the list point text.
Word has a 40 character limit for bookmark names. In
addition, bookmarks must begin with a letter. Since
pandoc's auto-generated identifiers may not respect
these constraints, some internal links did not work.
With this change, pandoc uses a bookmark name based
on the SHA1 hash of the identifier when the identifier
isn't a legal bookmark name.
Closes#5091.
The parameter is Extensions. This allows these functions to
be sensitive to the settings of `Ext_gfm_auto_identifiers` and
`Ext_ascii_identifiers`.
This allows us to use `uniqueIdent` in the CommonMark reader,
replacing some custom code.
It also means that `gfm_auto_identifiers` can now be used
in all formats.
Semantically, `gfm_auto_identifiers` is now a modifier of
`auto_identifiers`; for identifiers to be set, `auto_identifiers`
must be turned on, and then the type of identifier produced
depends on `gfm_auto_identifiers` and `ascii_identifiers` are set.
Closes#5057.
The underline style is now deprecated.
Previously `--atx-headers` would enable the single-line
style; now the single-line style is always used.
Closes#5038.
* Lua: allow access to pandoc state
Lua filters and custom writers now have read-only access to most fields
of pandoc's internal state via the global variable `PANDOC_STATE`.
* Lua: allow iterating through fields of PANDOC_STATE
* Lua filters doc: describe CommonState
* Lua filters doc: mention global variable PANDOC_STATE
* Lua: add access to logs
Log messages can currently only be printed, but not decomposed.
T.P.GroffChar: replaced `essentialEscapes` with `manEscapes`,
which includes all the escapes mentioned in the groff_man manual.
T.P.Writers.Groff: removed escapeCode; changed parameter on
escapeString from Bool to new type `EscapeMode`.
Rewrote `escapeString`.
- Improve escaping of accented characters with `--ascii`.
Combining accents are now handled properly.
- Don't escape spaces and tabs in code blocks. This doesn't
seem to be necessary.
We also introduce a new type ManTokens (a sequence of tokens)
and remove MComment. This allows lexers to return empty strings
of tokens, or multiple tokens (as when macros are resolved).
One test still fails. This needs to be fixed by moving handling
of .BI, .I, etc. to the parsing phase.
- Improved support for custom macro definitions.
- LinePart type has been added. RoffStr is now one
constructor of LinePart (the other being MacroArg).
- MComment has lost its argument.
- MEndMacro has been removed.
- MStr has been removed (we now simply use LinePart).
- Macros now store a list of tokens.
- Each macro argument is a [LinePart], instead of a LinePart.
- .BR now behaves as documented in man (and doesn't create a link).
This should just add to metadata (title, date, section),
and not produce a level-1 header. (That might be done
in the template, depending on the output format.)
- `--ascii` is now turned on automatically for man output, for
portability. All man output will be escaped to ASCII.
- In T.P.Writers.Groff, `escapeChar`, `escapeString`, and
`escapeCode` now take a boolean parameter that selects
ascii-only output. This is used by the Ms writer for
`--ascii`, instead of doing an extra pass after writing
the document.
- In ms output without `--ascii`, unicode is used whenever
possible (e.g. for double quotes).
- A few escapes are changed: e.g. `\[rs]` instead of `\\` for
backslash, and `\ga]` instead of `` \` `` for backtick.
(unexported module). These are used in both the man and ms
writers.
Moved groffEscape out of Text.Pandoc.Writers.Shared [cancels earlier
API change from adding it, which was after last release].
This fixes strong/code combination on man (should be `\f[CB]` not
`\f[BC]`), mentioned in #4973.
Updated tests.
Closes#4975.
The default is `-raw_tex`, so no raw tex should result
unless we explicitly say `+raw_tex`. Previously some
raw commands did make it through.
Closes#4527.
Previously, the writer would unconditionally emit HTMLish output for
subscripts, superscripts, strikeouts (if the strikeout extension is
disabled) and small caps, even with raw_html disabled.
Now there are plain-text (and, where possible, fancy Unicode)
fallbacks for all of these corresponding (mostly) to the Markdown
fallbacks, and the HTMLish output is only used when raw_html is
enabled.
This commit adds exported functions `toSuperscript` and
`toSubscript` to `Text.Pandoc.Writers.Shared`. [API change]
Closes#4528.
Closes#4284.
Headers with the corresponding tags should not appear in the output.
If one or more of the specified tags contains a non-tag character
like `+`, Org-mode will not treat that as a valid tag, but will
nonetheless continue scanning for valid tags. That behavior is not
replicated in this patch; entering `cat+dog` as one of the entries in
`#+EXCLUDE_TAGS` and running the file through Pandoc will cause the
parser to fail and result in the only excluded tag being the default, `noexport`.
Implement table caption numbering with a format
"Table 1: <caption>".
Translations are enabled and numbering is consecutive for
captioned tables, uncaptioned tables are not enumerated.
Captioned figures are now also numbered consecutively
and uncaptioned figures are not enumerated.
Figure captions are now numbered 1, 2, 3, ... The format in
the caption is "Figure 1: <caption>" and so on.
This is necessary in order for libreoffice to generate an
Illustration Index (Table of Figures) for included figures.
This patch eliminates the large gap we used to have between
bullet and text, and also ensures that numbers in numbered
lists will be right-aligned.
Closes#4385.
Now the `write*` functions for Docbook, HTML, ICML, JATS,
Man, Ms, OPML are sensitive to `writerPreferAscii`. Previously
the to-ascii translation was done in Text.Pandoc.App, and
thus not available to those using the writer functions
directly.
In addition, the LaTeX writer is now sensitive to
`writerPreferAscii` and to `--ascii`. 100% ASCII
output can't be guaranteed, but the writer will use
commands like `\"{a}` and `\l` whenever possible,
to avoid emiting a non-ASCII character.
A new unexported module, Text.Pandoc.Groff, has been
added to store functions used in the different groff-based
writers.
* Add support for multiprenote and multipostnote arguments.
The multiprenotes occur before the first prefix of a
multicite, and the multipostnotes follow the last suffix.
* Add test for multiprenote and multipostnote.
This changes the way styles for cells in the header row
and normal rows are handled in ODT tables.
Previously a new (but identical) style was generated for
every table, specifying the style of the cells within the table.
After this change there are two style definitions for table cells,
one for the cells in the header row, one for all other cells.
This doesn't change the actual styles, but makes post-processing
changes to the table styles much simpler as it is no longer
necessary to introduce new styles for header rows and there are
now only two styles where there was previously one per table.
With this change, autolinks are parsed as Links with
the `uri` class. (The same is true for bare links, if
the `autolink_bare_uris` extension is enabled.) Email
autolinks are parsed as Links with the `email` class.
This allows the distinction to be represented in the
URI.
Formerly the `uri` class was added to autolinks by
the HTML writer, but it had to guess what was an autolink
and could not distinguish `[http://example.com](http://example.com)`
from `<http://example.com>`. It also incorrectly recognized
`[pandoc](pandoc)` as an autolink. Now the HTML writer
simply passes through the `uri` attribute if it is present,
but does not add anything.
The Textile writer has been modified so that the `uri`
class is not explicitly added for autolinks, even if it
is present.
Closes#4913.
Inclusion of planning info (*DEADLINE*, *SCHEDULED*, and *CLOSED*) can
be controlled via the `p` export option: setting the option to `t` will
add all planning information in a *Plain* block below the respective
headline.
This now allows raw LaTeX environments, `\ref`, and `\eqref` to
be parsed (which is helpful for translation HTML documents using
MathJaX).
Closes#1126.
Also foreigncblockquote, hyphenblockquote, hyphencblockquote.
Closes#4848. But note: currently foreignquote will be
parsed as a regular Quoted inline (not using the quotes
appropriate to the foreign language).
Add support for `\|`, `\b`, `\G`, `\h`, `\d`, `\f`,
`\r`, `\t`, `\U`, `\i`, `\j`, `\newtie`, `\textcircled`.
Also fall back to combining characters when composed
characters are not available.
Closes#4652.
Inline math in `\(..\)`, display math in `\[..\]`, tex is now used.
Previously we'd "fake it with unicode" and fall back to tex when
that didn't work. But as of
3f50b95532
haddock supports latex math.
...they should only be recognized in siunitx contexts.
For example, `\l` outside of an siunitx context should be l-slash,
not l (for liter)!
Closes#4842.
We can't always tell if it's LaTeX, ConTeXt, or plain TeX.
Better just to use "tex" always.
Also changed:
ConTeXt writer: now outputs raw "tex" blocks as well as "context".
(Closes#969).
RST writer: uses ".. raw:: latex" for "tex" content.
(RST doesn't support raw context anyway.)
Note that if "context" or "latex" specifically is desired,
you can still force that in a markdown document by using
the raw attribute (see MANUAL.txt):
```{=latex}
\foo
```
Note that this change may affect some filters, if they assume that raw
tex parsed by the Markdown reader will be RawBlock (Format "latex").
In most cases it should be trivial to modify the filters to accept
"tex" as well.
For example: `\def\foo#1[#2]{#1 and #2}`.
Closes#4768. Also fixes#4771.
API change: in Text.Pandoc.Readers.LaTeX.Types,
new type ArgSpec added. Second parameter of Macro
constructor is now `[ArgSpec]` instead of `Int`.
This fixes a regression in 2.2.3, which cause boolean values to
be parsed as MetaInlines instead of MetaBool.
Note also an undocumented (but desirable) change in 2.2.3:
numbers are now parsed as MetaInlines rather than MetaString.
Closes#4819.
* Use a Span with class "title-reference" for the default
title-reference role.
* Use B.text to split up contents into Spaces, SoftBreaks, and Strs
for title-reference.
* Use Code with class "interpreted-text" instead of Span and Str for
unknown roles. (The RST writer has also been modified to round-trip
this properly.)
* Disallow blank lines in interpreted text.
* Backslash-escape now works in interpreted text.
* Backticks followed by alphanumerics no longer end interpreted text.
Closes#4811.
Closes#4803
After this commit use `$titleblock$` in order to get what was contained
in `$title$` before, that is a title and subtitle rendered according to
the official rST method:
http://docutils.sourceforge.net/docs/user/rst/quickstart.html#document-title-subtitle. from
With this commit, the `$title$` and `$subtitle$` metadata are available and they
simply carry the metadata values. This opens up more possibilities in templates.
Now we properly parse title and subtitle elements that are
direct children of book and article (as well as children of
bookinfo, articleinfo, or info).
We also now use the "subtitle" metadata field for subtitles,
rather than tacking the subtitle on to the title.
Exposes a function converting which flattenes a list of blocks into a
list of inlines. An example use case would be the conversion of Note
elements into other inlines.
RST does not allow nested emphasis, links, or other inline
constructs.
Closes#4581, double parsing of links with URLs as
link text. This supersedes the earlier fix for #4581
in 6419819b46.
Fixes#4561, a bug parsing with URLs inside emphasis.
Closes#4792.
Emphasis was not parsed when it followed directly after some block types
(e.g., lists).
The org reader uses a wrapper for the `parseFromString` function to
handle org-specific state. The last position of a character allowed
before emphasis was reset incorrectly in this wrapper. Emphasized text
was not recognized when placed directly behind a block which the reader
parses using `parseFromString`.
Fixes: #4784
Starting in 2.2.2, everything after an `\input` (or `\include`)
in a markdown file would be parsed as raw LaTeX.
This commit fixes the issue and adds a regression test.
Closes#4781.
Text.Pandoc.Emoji now exports `emojiToInline`, which returns a Span inline containing the emoji character and some attributes with metadata (class `emoji`, attribute `data-emoji` with emoji name). Previously, emojis (as supported in Markdown and CommonMark readers, e.g "😄")
were simply translated into the corresponding unicode code point. By wrapping them in Span
nodes, we make it possible to do special handling such as giving them a special font
in HTML output. We also open up the possibility of treating them differently when the
`--ascii` option is selected (though that is not part of this commit).
Closes#4743.
Non-ascii characters were not stripped from identifiers even if the
`ascii_identifiers` extension was enabled (which is is by default for
gfm).
Closes#4742