Lists of Inline and Block elements can now be filtered via `Inlines` and
`Blocks` functions, respectively. This is helpful if a filter conversion
depends on the order of elements rather than a single element.
For example, the following filter can be used to remove all spaces
before a citation:
function isSpaceBeforeCite (spc, cite)
return spc and spc.t == 'Space'
and cite and cite.t == 'Cite'
end
function Inlines (inlines)
for i = #inlines-1,1,-1 do
if isSpaceBeforeCite(inlines[i], inlines[i+1]) then
inlines:remove(i)
end
end
return inlines
end
Closes: #6038
The functions `table.insert`, `table.remove`, and `table.sort` are added
to pandoc.List elements. They can be used as methods, e.g.
local numbers = pandoc.List {2, 3, 1}
numbers:sort() -- numbers is now {1, 2, 3}
If parsing fails in a raw environment (e.g. due to special
characters like unescaped `_`), try again as a verbatim
environment, which is less sensitive to special characters.
This allows us to capture special environments that change
catcodes as raw tex when `-f latex+raw_tex` is used.
Closes#6034.
The fix to #6030 actually changed behavior, so that the
2D nesting occurred at slide level N-1 and N, instead of
at the top-level section. This commit restores the 2.7.3 behavior.
If there are more than 2 levels, the top level is horizontal
and the rest are collapsed to vertical.
Closes#6032.
The current DTD for the JATS writer template is for Journal Publishing (JATS-journalpublishing1.dtd), which does not permit ext-link as a valid child (https://jats.nlm.nih.gov/publishing/tag-library/1.1/element/publisher-name.html).
This update modifies the default output template to be the less restrictive JATS archiving and interchange DTD which systems like PubMed use internally to represent their articles.
Pandoc's AST is translated into the Jira AST, which is then rendered by
the dedicated Jira printer.
The following improvements are included in this change:
- non-jira raw blocks are fully discarded instead of showing as blank
lines;
- table cells can contain multiple blocks;
- unnecessary blank lines are removed from the output;
- markup chars within words are properly surrounded by braces;
- preserving soft linebreaks via `--wrap=preserve` is supported.
Note that backslashes are rendered as HTML entities, as there appears no
alternative to produce a plain backslash if it is followed by markup.
This may cause problems when used with confluence, where rendering seems
to fail in this case.
Closes: #5926
Added `\setupinterlinespace` to `title`, `subtitle`, `date` and `author` elements.
Otherwise longer titles that run over multiple lines will look squashed as
`\tfd` etc. won't adapt the line spacing to the font size.
With positive heading shifts, starting in 2.8 this option caused
metadata titles to be removed and changed to regular headings.
This behavior is incompatible with the old behavior of
`--base-header-level` and breaks old workflows, so with this
commit we are rolling back this change.
Now, there is an asymmetry in positive and negative heading
level shifts:
+ With positive shifts, the metadata title stays the same and
does not get changed to a heading in the body.
+ With negative shifts, a heading can be converted into the
metadata title.
I think this is a desirable combination of features, despite
the asymmetry. One might, e.g., want to have a document
with level-1 section headigs, but render it to HTML with
level-2 headings, retaining the metadata title (which pandoc
will render as a level-1 heading with the default template).
Closes#5957.
Revises #5615.
Backslash-escaping is used instead of HTML entities, as escaped
characters are easier to read this way. Furthermore, Confluence, which
seems to use a subset of Jira markup, seems to get confused by HTML
entities.
Previously they appeared centered at the top of the page;
now we put them centered at the bottom, unless the `pagenumbering`
variable is set (this gives users full control over page
number format and position,
https://wiki.contextgarden.net/Command/setuppagenumbering)
All headings now have a uniform color.
Level-1 headings no longer set `w:themeShade="B5"`.
Level-2 headings are now 14 point rather than 16 point.
Level-3 headings are now 12 point rather than 14 point.
Level-4 headings are italic rather than bold.
Closes#5820.
If a simple table would be too wide, we use a grid table.
The code for generating grid tables has been adjusted to
give more intelligent column widths when widths aren't
given. (This also affects the markdown writer.)
Closes#5899.
`Nothing` means: nothing specified.
`Just []` means: an empty list specified (e.g. in defaults).
Potentially these could lead to different behavior: see #5888.
The check whether a complex inline element following a string must be
escaped, now depends on the last character of the string instead of the
first.
Fixes: #5906
PR #5884.
+ Use pandoc-types 1.20 and texmath 0.12.
+ Text is now used instead of String, with a few exceptions.
+ In the MediaBag module, some of the types using Strings
were switched to use FilePath instead (not Text).
+ In the Parsing module, new parsers `manyChar`, `many1Char`,
`manyTillChar`, `many1TillChar`, `many1Till`, `manyUntil`,
`mantyUntilChar` have been added: these are like their
unsuffixed counterparts but pack some or all of their output.
+ `glob` in Text.Pandoc.Class still takes String since it seems
to be intended as an interface to Glob, which uses strings.
It seems to be used only once in the package, in the EPUB writer,
so that is not hard to change.
Superscripts and subscripts cannot contain spaces,
but newlines were previously allowed (unintentionally).
This led to bad interactions in some cases with footnotes.
E.g.
```
foo^[note]
bar^[note]
```
With this change newlines are also not allowed inside
super/subscripts.
Closes#5878.
* Add HTML Reader support for `<dfn>`, parsing this as a Span with class `dfn`.
* Change `htmlSpanLikeElements` implementation to retain classes,
attributes and inline content.
Previously, if a document contained two YAML metadata blocks
that set the same field, the conflict would be resolved in favor
of the first. Now it is resolved in favor of the second (due to
a change in pandoc-types).
This makes the behavior more uniform with other things in pandoc
(such as reference links and `--metadata-file`).
The first list item of a sublist should not resume numbering
from the number of the last sublist item of the same level,
if that sublist was a sublist of a different list item.
That is, we should not get:
```
1. one
1. sub one
2. sub two
2. two
3. sub one
```
* Set dbBook to true when traversing a chapter too.
Currently, a `<title/>` in a chapter and in a `<section/>` below that
chapter have the same level if they're not inside a `<book/>`.
This can happen in a multi-file book project. Also see the example at
https://tdg.docbook.org/tdg/4.5/chapter.html
Co-authored-by: Félix Baylac-Jacqué <felix@alternativebit.fr>
* Add docbook-chapter test
This tests nested `<section/>` and makes sure `<title/>` in the first
`<section/>` below `<chapter/>` is one level deeper than the `<chapter/>`'s
`<title/>`, also when not inside a `<book/>`.
Co-authored-by: Félix Baylac-Jacqué <felix@alternativebit.fr>
The new version of doctemplates adds many features to pandoc's
templating system, while remaining backwards-compatible.
New features include partials and filters. Using template filters,
one can lay out data in enumerated lists and tables.
Templates are now layout-sensitive: so, for example, if a
text with soft line breaks is interpolated near the end of
a line, the text will break and wrap naturally. This makes
the templating system much more suitable for programatically
generating markdown or other plain-text files from metadata.
This package is needed for proper handling of image filenames
containing periods (in addition to the period before the
extension).
Unfortunately, grffile breaks in the latest texlive update.
Until a fix is released (see ho-tex/oberdiek#73) it seems best
to remove this from the default template.
This may cause problems if you have filenames with periods.
The workaround is to put `\usepackage{grffile}` in header-includes,
and be sure you're using an older version of texlive packages.
See #5848. We will leave that issue open to remind us to
check upstream, and restore grffile when it's possible to
do so.
When a div surrounds multiple sections at the same level,
or a section of highre level followed by one of lower level,
then we just leave it as a div and create a new div for the
section.
Closes#5846, closes#5761.
Comment lines in Org-mode can be completely empty; both of these line
should produce no output:
# a comment
#
The reader used to produce a wrong result for the latter, but ignores
that line as well now.
Fixes: #5856
* HTML reader: Handle cite attribute for quotes. If a `<q>` tag has a `cite` attribute, we interpret it as a Quoted element with an inner Span. Closes#5798
* Refactor url canonicalization into a helper function
* Modify HTML writer to handle quote with cite.
[0]: https://developer.mozilla.org/en-US/docs/Web/HTML/Element/q
Parse <mark> elements from HTML as HTML span like elements, with a
single class matching the tag name `mark`. Mark elements are rendered to
HTML using the native <mark> element.
Fixes https://github.com/jgm/pandoc/issues/5797.
* Text.Pandoc.Shared: export `htmlSpanLikeElements` [API change]
This commit also introduces a mapping of HTML span like elements that
are internally represented as a Span with a single class, but that are
converted back to the original element by the html writer. As of now,
only the kbd element is handled this way. Ideally these elements should
be handled as plain AST values, but since that would be a breaking
change with a large impact, we revert to this stop-gap solution.
Fixes https://github.com/jgm/pandoc/issues/5796.
We change to use 0.5pt rather than `\linethickness`, which
apparently only ever worked "by accident" and no longer works
with recent updates to texlive.
Closes#5801.
...even when we must lose width information.
All in all this seems to be people's preferred behavior, even though it
is slightly lossier.
Closes#2608.
Closes#4497.
on conflicting fields. This changes earlier behavior (but not in
a release), where first took precedence.
Note that this may seem inconsistent with the behavior of
multiple YAML blocks within a document, where the first takes
precedence. Still, it is convenient to be able to override
defaults with options later on the command line.
If this is present on a heading with the 'unnumbered' class,
the heading won't appear in the TOC. This class has no
effect if 'unnumbered' is not also specified.
This affects HTML-based writers (including slide shows
and epub), LateX (including beamer), RTF, and PowerPoint.
Other writers do not yet support `unlisted`.
Closes#1762.
Motivation: in a man page there's not much use for relative URLs,
which you can't follow. Absolute URLs are still useful. We previously
suppressed relative URLs starting with '#' (purely internal links),
but it makes sense to go a bit farther.
Closes#5770.
We were requiring consistent indentation, but this
isn't required by RST, as long as each nonblank
line of the block has *some* indentation.
Closes#5753.
Previously we used the following Project Gutenberg conventions
for plain output:
- extra space before and after level 1 and 2 headings
- all-caps for strong emphasis `LIKE THIS`
- underscores surrounding regular emphasis `_like this_`
This commit makes `plain` output plainer. Strong and Emph
inlines are rendered without special formatting. Headings
are also rendered without special formatting, and with only
one blank line following.
To restore the former behavior, use `-t plain+gutenberg`.
API change: Add `Ext_gutenberg` constructor to `Extension`.
See #5741.
Notice this commit updates lists.docx. The old test file contained
references to "ListParagraph" style, which should never leak
outside of pandoc, so I'm not sure what that was supposed to test
for exactly.
Motivating issues: #5523, #5052, #5074
Style name comparisons are case-insensitive, since those are
case-insensitive in Word.
w:styleId will be used as style name if w:name is missing (this should
only happen for malformed docx and is kept as a fallback to avoid
failing altogether on malformed documents)
Block quote detection code moved from Docx.Parser to Readers.Docx
Code styles, i.e. "Source Code" and "Verbatim Char" now honor style
inheritance
Docx Reader now honours "Compact" style (used in Pandoc-generated docx).
The side-effect is that "Compact" style no longer shows up in
docx+styles output. Styles inherited from "Compact" will still
show up.
Removed obsolete list-item style from divsToKeep. That didn't
really do anything for a while now.
Add newtypes to differentiate between style names, ids, and
different style types (that is, paragraph and character styles)
Since docx style names can have spaces in them, and pandoc-markdown
classes can't, anywhere when style name is used as a class name,
spaces are replaced with ASCII dashes `-`.
Get rid of extraneous intermediate types, carrying styleId information.
Instead, styleId is saved with other style data.
Use RunStyle for inline style definitions only (lacking styleId and styleName);
for Character Styles use CharStyle type (which is basicaly RunStyle with styleId
and StyleName bolted onto it).
This commit prevents custom styles on divs and spans from overriding
styles on certain elements inside them, like headings, blockquotes,
and links. On those elements, the "native" style is required for the
element to display correctly. This change also allows nesting of
custom styles; in order to do so, it removes the default "Compact"
style applied to Plain blocks, except when inside a table.
Attr values can now be given as normal Lua tables; this can be used as a
convenient alternative to define Attr values, instead of constructing
values with `pandoc.Attr`. Identifiers are taken from the *id* field,
classes must be given as space separated words in the *class* field. All
remaining fields are included as misc attributes.
With this change, the following lines now create equal elements:
pandoc.Span('test', {id = 'test', class = 'a b', check = 1})
pandoc.Span('test', pandoc.Attr('test', {'a','b'}, {check = 1}))
This also works when using the *attr* setter:
local span = pandoc.Span 'text'
span.attr = {id = 'test', class = 'a b', check = 1}
Furthermore, the *attributes* field of AST elements can now be a plain
key-value table even when using the `attributes` accessor:
local span = pandoc.Span 'test'
span.attributes = {check = 1} -- works as expected now
Closes: #5744
Deprecate --base-heading-level.
The new option does everything the old one does, but also
allows negative shifts. It also promotes the document
metadata (if not null) to a level-1 heading with a +1 shift,
and demotes an initial level-1 heading to document metadata
with a -1 shift. This supports converting documents that
use an initial level-1 heading for the document title.
Closes#5615.
* Org reader: allow the `-i` switch to ignore leading spaces.
* Org reader: handle awkwardly-aligned code blocks within lists.
Code blocks in Org lists must have their #+BEGIN_ aligned in a
reasonable way, but their other components can be positioned otherwise.
Text.Pandoc.Shared:
+ Remove `Element` type [API change]
+ Remove `makeHierarchicalize` [API change]
+ Add `makeSections` [API change]
+ Export `deLink` [API change]
Now that we have Divs, we can use them to represent the structure
of sections, and we don't need a special Element type.
`makeSections` reorganizes a block list, adding Divs with
class `section` around sections, and adding numbering
if needed.
This change also fixes some longstanding issues recognizing
section structure when the document contains Divs.
Closes#3057, see also #997.
All writers have been changed to use `makeSections`.
Note that in the process we have reverted the change
c1d058aeb1
made in response to #5168, which I'm not completely
sure was a good idea.
Lua modules have also been adjusted accordingly.
Existing lua filters that use `hierarchicalize` will
need to be rewritten to use `make_sections`.
Revert "hierarchicalize: ensure that sections get ids..."
This reverts commit 212406a61d.
Revert "Improve detection of headings in Divs by hierarchicalize."
This reverts commit 6e2cfd6c97.
Revert "Shared.hierarchicalize: improve handling of div and section structure."
This reverts commit 345b33762e.
The structure
```
<h1>one</h1>
<div>
<h1>two</h1>
</div>
```
should create two coordinate sections, not a section with
a subsection. Now it does.
Extends #3057.
Previously Divs were opaque to hierarchicalize, so headings
inside divs didn't get into the table of contents, for
example (#3057).
Now hierarchicalize treats Divs as sections when appropriate.
For example, these structures both yield a section and a
subsection:
``` html
<div>
<h1>one</h1>
<div>
<h2>two</h2>
</div>
</div>
```
``` html
<div>
<h1>one</h1>
<div>
<h1>two</h1>
</div>
</div>
```
Note that
``` html
<h1>one</h1>
<div>
<h2>two</h2>
</div>
<h1>three</h1>
```
gets parsed as the structure
one
two
three
which may not always be desirable.
Closes#3057.
Previously we used named character references with html5 output.
But these aren't valid XML, and we aim to produce html5 that is
also valid XHTML (polyglot markup). (This is also needed for
epub3.)
Closes#5718.
+ Remove Text.Pandoc.Pretty; use doclayout instead. [API change]
+ Text.Pandoc.Writers.Shared: remove metaToJSON, metaToJSON'
[API change].
+ Text.Pandoc.Writers.Shared: modify `addVariablesToContext`,
`defField`, `setField`, `getField`, `resetField` to work with
Context rather than JSON values. [API change]
+ Text.Pandoc.Writers.Shared: export new function `endsWithPlain` [API
change].
+ Use new templates and doclayout in writers.
+ Use Doc-based templates in all writers.
+ Adjust three tests for minor template rendering differences.
+ Added indentation to body in docbook4, docbook5 templates.
The main impact of this change is better reflowing of content
interpolated into templates. Previously, interpolated variables
were rendered independently and intepolated as strings, which could lead
to overly long lines. Now the templates interpolated as Doc values
which may include breaking spaces, and reflowing occurs
after template interpolation rather than before.
Changed optMetadataFile from `Maybe FilePath` to `[FilePath]`. This allows
for multiple YAML metadata files to be added. The new default value has
been changed from `Nothing` to `[]`.
To account for this change in `Text.Pandoc.App`, `metaDataFromFile` now
operates on two `mapM` calls (for `readFileLazy` and `yamlToMeta`) and a fold.
Added a test (command/5700.md) which tests this functionality and
updated MANUAL.txt, as per the contributing guidelines.
With the current behavior, using `foldr1 (<>)`, values within files
specified first will be used over those in later files. (If the reverse
of this behavior would be preferred, it should be fixed by changing
foldr1 to foldl1.)
Traversal methods are updated to use the new Walk module such that
sequences with nested Inline (or Block) elements are traversed in the
order in which they appear in the linearized document.
Fixes: #5667
the token string is modified by a parser (e.g. accent when
it only takes part of a Word token).
Closes#5686. Still not ideal, because we get the whole
`\t0BAR` and not just `\t0` as a raw latex inline command.
But I'm willing to let this be an edge case, since you
can easily work around this by inserting a space, braces,
or raw attribute. The important thing is that we no longer
drop the rest of the document after a raw latex inline
command that gobbles only part of a Word token!
* Require recent doctemplates. It is more flexible and
supports partials.
* Changed type of writerTemplate to Maybe Template instead
of Maybe String.
* Remove code from the LaTeX, Docbook, and JATS writers that looked in
the template for strings to determine whether it is a book or an
article, or whether csquotes is used. This was always kludgy and
unreliable. To use csquotes for LaTeX, set `csquotes` in your
variables or metadata. It is no longer sufficient to put
`\usepackage{csquotes}` in your template or header includes.
To specify a book style, use the `documentclass` variable or
`--top-level-division`.
* Change template code to use new API for doctemplates.
The web service passed in to `--webtex` may render formulas using inline
or display style by default. Prefixing formulas with the appropriate
command ensures they are rendered correctly.
This is a followup to the discussion in #5656.
Previously we just omitted these. Now we render them
using `\hfill\break` instead of `\\`. This is a revision
of a PR by @sabine (#5591) who should be credited with the
idea.
Closes#3324.
Workaround for Word Online shortcomming. Fixes#5645
Also, make list para properties go first.
This reordering of properties shouldn't be necessary but
it seems Word Online does not understand the docx correctly otherwise.
The `raw_attribute` will be used to mark raw bits, even HTML
and LaTeX, and even when `raw_html` and `raw_tex` are enabled,
as they are by default.
To get the old behavior, disable `raw_attribute` in the writer.
Closes#4311.
* Let the user choose type of PDF/A generated with ConTeXt (closes#5608)
* Updated ConTeXt test documents for changes in tagging
* Updated color profile settings in accordance with ConTeXt wiki
* Made ICC profile and output intent for PDF/A customizable
* Read pdfa variable from meta (and updated manual)
Now we boldface code but not other things. This matches the
most common style in man pages (particularly option lists).
Also, remove a regression in the last commit in which 'nowrap'
was removed.
This commit alters the way in which the Powerpoint writer treats
inline code and code blocks.
- Inline code is now formatted at the same size as the surrounding
text.
- Code blocks are now given a margin and font size according to their
level.
- Furthermore this commit allows changing the font with which code is
formatted via the `monofont` option.
Tested in
- PowerPoint 365 for Windows - 1808 (Build 10730.20344 Click-to-Run)
- PowerPoint 365 for Mac - 16.26 (19060901)
The JATS spec restricts contents of certain
elements (fn, list-item); this patch wraps
elements that can't go in these contexts inside
p elements with specific-use "wrapper", so the
documents will conform.
Closes#5570.
This was added in pandoc 2.7.2, but it makes it impossible
to use pandoc-crossref. So this has been rolled back for now,
until we find a good solution to make this behavior optional
(or a creative way to let pandoc-crossref and this feature
to coexist).
See #5474.
Previously if the language was not in the list of listings-
supported languages, it would not be added as a class, so
custom syntax highlighting could not be used.
Closes#5540.
Change is in rawLaTeXInline in LaTeX reader, but
it affects the markdown reader and other readers
that allow raw LaTeX.
Previously, trailing `{}` would be included for
unknown commands, but not for known commands.
However, they are sometimes used to avoid a trailing
space after the command. The chances that a `{}`
after a LaTeX command is not part of the command
are very small.
Closes#5439.
- With epub extensions, check for epub:type in addition to type.
- Fix problem with noteref parsing which caused block-level
content to be eaten with the noteref.
- Rename pAnyTag to pAny.
- Refactor note resolution.
Version specifiers like `PANDOC_VERSION` and `PANDOC_API_VERSION` are
turned into `Version` objects. The objects simplify version-appropriate
comparisons while maintaining backward-compatibility.
A function `pandoc.types.Version` is added as part of the newly
introduced module `pandoc.types`, allowing users to create version
objects in scripts.
Move up the pattern match to be reachable, closes#5517.
Previously `file:/` URLs were handled wrongly and pandoc attempted
to make HTTP requests, which failed.
add `onlySimpleTableCells` to `Text.Pandoc.Shared`
[API change]
This fixes an inconsistency in the HTML reader, which did not treat tables with `<p>` inside cells as simple.
Previously if labels had integer names, it could produce a conflict
with auto-labeled reference links. Now we test for a conflict and find
the next available integer.
Note that this involves adding a new state variable `stPrevRefs` to
keep track of refs used in other document parts when using
`--reference-location=block|section`
Closes#5495
This makes use of tasty-lua, a package to write tests in Lua
and integrate the results into Tasty output. Test output becomes
more informative: individual tests and test groups become visible
in test output. Failures are reported with helpful error messages.
"Best Practice: When footnotes are grouped at the end of an article,
wrap them in a `<fn-group>` and use an `<xref>` element in the text, as
usual, to tie each footnote in the list to a particular location in the
text."
Closes#5511.
Planning info is now always placed before the subtree contents.
Previously, the planning info was placed after the content if the
header's subtree was converted to a list, which happens with headers of
level 3 and higher per default.
Fixes: #5494
Emacs always uses two spaces when indenting the content of src blocks,
e.g., when exiting a `C-c '` edit-buffer. Pandoc used to indent contents
by the space-equivalent of one tab, but now always uses two spaces, too.
Closes: #5440
The `system` Lua module provides utility functions to interact with the
operating- and file system. E.g.
print(pandoc.system.get_current_directory())
or
pandoc.system.with_temporary_directory('tikz', function (dir)
-- write and compile a TikZ file with pdflatex
end)
iOS chooses to render a number of Unicode entities,
including '↩', as big colorful emoji. This can be
defeated by appending Unicode
VARIATION SELECTOR-15'/'VARIATION SELECTOR-16'.
So we now append this character when escaping
strings, for both '↩' and '↔'.
If other characters prove problematic, they can
simply be added to needsVariationSelector.
Closes#5469.
1) Don't append `.html`
2) Add `wikilink` title
This mirrors behavior of other wiki readers. Generally the
`.html` extension is not wanted. It may be important for
output to HTML in certain circumstances, but it can always
be added using a filter that matches on links with title
`wikilink`.
Note that if you have a workflow that uses pandoc to convert
vimwiki to readable HTML pages, you may need to add such a
filter to reproduce current behavior.
Here is a filter that does the job:
```lua
function Link(el)
if el.title == 'wikilink' then
el.target = el.target .. ".html"
end
return el
end
```
Save this as `fixlinks.lua` and use with `--lua-filter fixlinks.lua`.
Closes#5414.
The previous built-in reference doc had only title and content
layouts. Add in a section-header slide and a two-content slide, so
users can more easily modify it to build their own templates.
Golden files needed to be regenerated. Checked on MS PowerPoint 2013.
Improve the workaround for #1658, adapting a solution by @u-fischer in
<https://github.com/latex3/latex2e/issues/131> that works whether or not
the `indent` variable is enabled.
Remove `subparagraph` variable in LaTeX template. The default is now
to use run-in style for level 4 and 5 headings (`\paragraph` and `\subparagraph`).
To get the previous default behavior (where these were formatted as blocks,
like `\subsubsection`), set the `block-headings` variable.
An example is given in the manual of reformatting the appearance of headings
more thoroughly using KOMA-Script.
Closes#5365.
