Commit graph

1215 commits

Author SHA1 Message Date
John MacFarlane
45b7636307 Revert "FB2 reader test: better diagnostics on failure."
This reverts commit c65af7d1a2.
2019-09-15 10:27:19 -07:00
John MacFarlane
c65af7d1a2 FB2 reader test: better diagnostics on failure. 2019-09-15 09:06:38 -07:00
John MacFarlane
88a0327579 FB2 reader test: Another attempt to fix test failure on GitHub CI. 2019-09-14 10:37:19 -07:00
John MacFarlane
7ecae69e27 Revert "FB2 reader test: filter CRs."
This reverts commit e35147d715.
2019-09-13 22:08:42 -07:00
John MacFarlane
e35147d715 FB2 reader test: filter CRs.
This may help with the test failure on GitHub CI.

b59e6d0376/checks
2019-09-13 16:50:00 -07:00
John MacFarlane
88dc6fac5d Add --shift-heading-level-by option.
Deprecate --base-heading-level.

The new option does everything the old one does, but also
allows negative shifts.  It also promotes the document
metadata (if not null) to a level-1 heading with a +1 shift,
and demotes an initial level-1 heading to document metadata
with a -1 shift. This supports converting documents that
use an initial level-1 heading for the document title.

Closes #5615.
2019-09-10 23:16:13 -07:00
John MacFarlane
4778d03473 LaTeX reader: Fix parsing of optional arguments that contain braced text.
Closes #5740.
2019-09-09 21:33:16 -07:00
Brian Leung
0558ea9836 Org reader: modify handling of example blocks. (#5717)
* Org reader: allow the `-i` switch to ignore leading spaces.

* Org reader: handle awkwardly-aligned code blocks within lists.

Code blocks in Org lists must have their #+BEGIN_ aligned in a
reasonable way, but their other components can be positioned otherwise.
2019-09-08 22:34:10 -07:00
John MacFarlane
9f984ff26a Replace Element and makeHierarchical with makeSections.
Text.Pandoc.Shared:

+ Remove `Element` type [API change]
+ Remove `makeHierarchicalize` [API change]
+ Add `makeSections` [API change]
+ Export `deLink` [API change]

Now that we have Divs, we can use them to represent the structure
of sections, and we don't need a special Element type.
`makeSections` reorganizes a block list, adding Divs with
class `section` around sections, and adding numbering
if needed.

This change also fixes some longstanding issues recognizing
section structure when the document contains Divs.
Closes #3057, see also #997.

All writers have been changed to use `makeSections`.
Note that in the process we have reverted the change
c1d058aeb1
made in response to #5168, which I'm not completely
sure was a good idea.

Lua modules have also been adjusted accordingly.
Existing lua filters that use `hierarchicalize` will
need to be rewritten to use `make_sections`.
2019-09-08 22:20:19 -07:00
John MacFarlane
1ccff3339d Revert changes to hierarchicalizeWithIds.
Revert "hierarchicalize: ensure that sections get ids..."
This reverts commit 212406a61d.

Revert "Improve detection of headings in Divs by hierarchicalize."
This reverts commit 6e2cfd6c97.

Revert "Shared.hierarchicalize: improve handling of div and section structure."
This reverts commit 345b33762e.
2019-09-08 21:56:42 -07:00
John MacFarlane
212406a61d hierarchicalize: ensure that sections get ids...
even if they're in divs.  Improves #3057.
2019-09-06 09:05:52 -07:00
John MacFarlane
6e2cfd6c97 Improve detection of headings in Divs by hierarchicalize.
The structure

```
<h1>one</h1>
<div>
<h1>two</h1>
</div>
```

should create two coordinate sections, not a section with
a subsection.  Now it does.

Extends #3057.
2019-09-06 08:44:59 -07:00
John MacFarlane
345b33762e Shared.hierarchicalize: improve handling of div and section structure.
Previously Divs were opaque to hierarchicalize, so headings
inside divs didn't get into the table of contents, for
example (#3057).

Now hierarchicalize treats Divs as sections when appropriate.
For example, these structures both yield a section and a
subsection:

``` html
<div>
<h1>one</h1>
<div>
<h2>two</h2>
</div>
</div>
```
``` html
<div>
<h1>one</h1>
<div>
<h1>two</h1>
</div>
</div>
```

Note that

``` html
<h1>one</h1>
<div>
<h2>two</h2>
</div>
<h1>three</h1>
```

gets parsed as the structure

    one
      two
    three

which may not always be desirable.

Closes #3057.
2019-09-05 22:37:13 -07:00
John MacFarlane
381654a704 Add div.hanging-indent CSS to HTML templates. 2019-09-05 12:42:23 -07:00
John MacFarlane
bb362fd76c Add partial styles.html in HTML5 template.
Avoid duplication in HTML templates by using styles.html partial.
Change indentation of styles in template.
2019-09-05 12:39:50 -07:00
John MacFarlane
0e31483d43 asciidoc writer: don't include + in code blocks for regular asciidoc.
This is asciidoctor-specific.

Amends 98ee6ca289.
2019-09-04 14:57:22 -07:00
John MacFarlane
e4cca4cf67 Roff readers: better parsing of groups.
We now allow groups where the closing `\\}` isn't at the
beginning of a line.

Closes #5410.
2019-09-04 09:24:42 -07:00
John MacFarlane
513058a24e XML: change toEntities to emit numerical hex character references.
Previously decimal references were used.
But Polyglot Markup prefers hex.  See #5718.

This affects the output of pandoc with `--ascii`.
2019-09-03 11:28:20 -07:00
John MacFarlane
6b286a1d74 LaTeX reader: don't try to parse includes if raw_tex is set.
When the `raw_tex` extension is set, we just carry through
`\usepackage`, `\input`, etc. verbatim as raw LaTeX.

Closes #5673.
2019-09-02 21:03:05 -07:00
John MacFarlane
d79242796b HTML writer: use numeric character references with --ascii.
Previously we used named character references with html5 output.
But these aren't valid XML, and we aim to produce html5 that is
also valid XHTML (polyglot markup).  (This is also needed for
epub3.)

Closes #5718.
2019-09-02 20:36:57 -07:00
John MacFarlane
5e708eb8ce LaTeX reader: properly handle optional arguments for macros.
Closes #5682.
2019-09-02 18:48:37 -07:00
John MacFarlane
fba1296fd1 LaTeX reader: fix \\ in \parbox inside a table cell.
Closes #5711.
2019-08-27 10:48:02 -07:00
John MacFarlane
167fc4bc87 Markdown reader: Headers: don't parse content over newline boundary.
Closes #5714.
2019-08-27 10:15:00 -07:00
Jesse Rosenthal
4a7dad18b1 PowerPoint writer: Start numbering at appopriate numbers.
Starting numbers for ordered lists were previously ignored. Now we
specify the number if it is something other than 1.

Closes: #5709
2019-08-27 01:24:41 -04:00
John MacFarlane
180f534d21 Add test for issue #5708. 2019-08-26 15:20:22 -07:00
John MacFarlane
1ee6e0e087 Use new doctemplates, doclayout.
+ Remove Text.Pandoc.Pretty; use doclayout instead. [API change]
+ Text.Pandoc.Writers.Shared: remove metaToJSON, metaToJSON'
  [API change].
+ Text.Pandoc.Writers.Shared: modify `addVariablesToContext`,
  `defField`, `setField`, `getField`, `resetField` to work with
  Context rather than JSON values. [API change]
+ Text.Pandoc.Writers.Shared: export new function `endsWithPlain` [API
  change].
+ Use new templates and doclayout in writers.
+ Use Doc-based templates in all writers.
+ Adjust three tests for minor template rendering differences.
+ Added indentation to body in docbook4, docbook5 templates.

The main impact of this change is better reflowing of content
interpolated into templates.  Previously, interpolated variables
were rendered independently and intepolated as strings, which could lead
to overly long lines.  Now the templates interpolated as Doc values
which may include breaking spaces, and reflowing occurs
after template interpolation rather than before.
2019-08-25 14:24:31 -07:00
Owen McGrath
92debe4b9e Change optMetadataFile type from Maybe to List (#5702)
Changed optMetadataFile from `Maybe FilePath` to `[FilePath]`. This allows
for multiple YAML metadata files to be added. The new default value has
been changed from `Nothing` to `[]`.

To account for this change in `Text.Pandoc.App`, `metaDataFromFile` now
operates on two `mapM` calls (for `readFileLazy` and `yamlToMeta`) and a fold.

Added a test (command/5700.md) which tests this functionality and
updated MANUAL.txt, as per the contributing guidelines.

With the current behavior, using `foldr1 (<>)`, values within files
specified first will be used over those in later files. (If the reverse
of this behavior would be preferred, it should be fixed by changing
foldr1 to foldl1.)
2019-08-24 09:41:25 -07:00
John MacFarlane
9d581428f9 Add test for #5690. 2019-08-23 10:15:42 -07:00
John MacFarlane
1c71bd1ff5 Ensure proper nesting when we have long ordered list markers.
Closes #5705.
2019-08-23 09:16:54 -07:00
Albert Krewinkel
2712d3e869
Lua: traverse nested blocks and inlines in correct order
Traversal methods are updated to use the new Walk module such that
sequences with nested Inline (or Block) elements are traversed in the
order in which they appear in the linearized document.

Fixes: #5667
2019-08-16 20:52:15 +02:00
John MacFarlane
79a3449eeb LaTeX reader: improve withRaw so it can handle cases where...
the token string is modified by a parser (e.g. accent when
it only takes part of a Word token).

Closes #5686.  Still not ideal, because we get the whole
`\t0BAR` and not just `\t0` as a raw latex inline command.
But I'm willing to let this be an edge case, since you
can easily work around this by inserting a space, braces,
or raw attribute.  The important thing is that we no longer
drop the rest of the document after a raw latex inline
command that gobbles only part of a Word token!
2019-08-14 14:34:44 -07:00
John MacFarlane
eb23527121 Rename test for 5685 -> 5684 (typo in last commit).
Closes #5684. (Note that #5685 is NOT closed by previous commit.)
2019-08-14 11:13:18 -07:00
John MacFarlane
0b2fb9b8f9 Add thin space when needed in LaTeX quote ligatures.
Closes #5685.
2019-08-14 11:07:02 -07:00
Jan-Otto Kröpke
a0a41c7a8e JIRA writer: Remove escapeStringForJira for code blocks 2019-08-11 21:57:12 +02:00
John MacFarlane
1afea63308 Update muse template to handle multiple authors better. 2019-07-28 19:25:44 -07:00
John MacFarlane
b35fae6511 Use doctemplates 0.3, change type of writerTemplate.
* Require recent doctemplates.  It is more flexible and
  supports partials.
* Changed type of writerTemplate to Maybe Template instead
  of Maybe String.
* Remove code from the LaTeX, Docbook, and JATS writers that looked in
  the template for strings to determine whether it is a book or an
  article, or whether csquotes is used. This was always kludgy and
  unreliable.  To use csquotes for LaTeX, set `csquotes` in your
  variables or metadata. It is no longer sufficient to put
  `\usepackage{csquotes}` in your template or header includes.
  To specify a book style, use the `documentclass` variable or
  `--top-level-division`.
* Change template code to use new API for doctemplates.
2019-07-28 19:25:45 -07:00
Philip Pesca
01a52e2300 HTML writer: ensure TeX formulas are rendered correctly (#5658)
The web service passed in to `--webtex` may render formulas using inline
or display style by default. Prefixing formulas with the appropriate
command ensures they are rendered correctly.

This is a followup to the discussion in #5656.
2019-07-24 10:10:36 -07:00
Philip Pesca
bc508534a6 HTML writer: render inline formulas correctly with --webtex (#5656)
We add `\textstyle` to the beginning of the formula to ensure it will be rendered in inline style.
Closes #5655.
2019-07-23 12:21:31 -07:00
John MacFarlane
db5f6dd4fe Fix error introduced in change to test for 4669. 2019-07-22 15:32:49 -07:00
John MacFarlane
d9960244d8 LaTeX reader: support tex \tt command.
Closes #5654.
2019-07-22 15:29:07 -07:00
Albert Krewinkel
63c65c89da
Org reader: accept ATTR_LATEX in block attributes
Attributes for LaTeX output are accepted as valid block attributes;
however, their values are ignored.

Fixes: #5648
2019-07-22 08:12:22 +02:00
John MacFarlane
91d4283263 LaTeX writer: fix line breaks at start of paragraph.
Previously we just omitted these. Now we render them
using `\hfill\break` instead of `\\`.  This is a revision
of a PR by @sabine (#5591) who should be credited with the
idea.

Closes #3324.
2019-07-20 17:12:53 -07:00
John MacFarlane
465eeece6b LaTeX reader: search for image with list of extensions...
like latex does, if an extension is not provided.

Closes #4933.
2019-07-20 10:17:49 -07:00
John MacFarlane
339392bf54 Markdown: Ensure that expanded latex macros end with space if original did.
Closes #4442.
2019-07-19 10:32:59 -07:00
Agustín Martín Barbero
bd69218451 Change order of ilvl and numId in document.xml (#5647)
Workaround for Word Online shortcomming. Fixes #5645

Also, make list para properties go first.

This reordering of properties shouldn't be necessary but
it seems Word Online does not understand the docx correctly otherwise.
2019-07-19 09:32:43 -07:00
John MacFarlane
28cad16517 Markdown writer: prefer using raw_attribute when enabled.
The `raw_attribute` will be used to mark raw bits, even HTML
and LaTeX, and even when `raw_html` and `raw_tex` are enabled,
as they are by default.

To get the old behavior, disable `raw_attribute` in the writer.

Closes #4311.
2019-07-18 22:31:03 -07:00
John MacFarlane
5c655e86d5 HTML writer: ensure that line numbers in code blocks get id-prefix.
Closes #5650.
2019-07-18 22:08:37 -07:00
John MacFarlane
0d72237e27 Dokuwiki writer: handle mixed lists without HTML fallback.
Closes #5107.
2019-07-16 13:14:37 -07:00
Karl Pettersson
5303791bc4 Customizable type of PDF/A for the ConTeXt writer (issue #5608) (#5610)
* Let the user choose type of PDF/A generated with ConTeXt (closes #5608)
* Updated ConTeXt test documents for changes in tagging
* Updated color profile settings in accordance with ConTeXt wiki
* Made ICC profile and output intent for PDF/A customizable
* Read pdfa variable from meta (and updated manual)
2019-07-15 11:55:04 -07:00
John MacFarlane
968d2046a3 Update test for new skylighting. 2019-07-14 10:48:14 -07:00
Alexander Krotov
0713cb65bc Muse: add RTL support
Closes #5551
2019-07-14 18:22:52 +03:00
Vasily Alferov
f6c92c7523 Fix #4499: add mbox and hbox handling to LaTeX reader (#5586)
When `+raw_tex` is enabled, these are passed through literally.
Otherwise, they are handled in a way that emulates LaTeX's behavior.
2019-07-13 16:55:41 -07:00
John MacFarlane
7bc9eab846
Merge pull request #5589 from blmage/fix-3992
Add support for EPUB2 covers (fix #3992)
2019-07-13 16:48:09 -07:00
John MacFarlane
4a5e727c8c Man writer: Improved definition list term output.
Now we boldface code but not other things. This matches the
most common style in man pages (particularly option lists).

Also, remove a regression in the last commit in which 'nowrap'
was removed.
2019-07-13 16:41:43 -07:00
John MacFarlane
d0bf7efe95 Man writer: fixed boldfacing of definition terms.
Previously the bold-facing would be interrupted by
other formatting, because we used `.B`.

Closes #5620.
2019-07-13 16:12:28 -07:00
John MacFarlane
a16311c225
Merge pull request #5606 from blmage/odt-frames
Improve the parsing of frames in ODT documents
2019-07-13 15:53:58 -07:00
John MacFarlane
1784161946 LaTeX reader: Properly handle \providecommand and environment...
They are now ignored if the corresponding command or environment
is already defined.

Closes #5635.
2019-07-13 15:51:33 -07:00
mb21
6cf5c3f6ac fix filename and issue reference of previous commit 2019-07-13 12:03:45 +02:00
John MacFarlane
6d30d3e0b3 Pass through aria- attributes to HTML5.
Also document addition of data- prefix to unknown attributes.

Closes #5646.
2019-07-12 17:03:01 -07:00
Brian Leung
1d9ff85b45 RST reader: keep name property in imgAttr. (#5637)
Closes #5619.
2019-07-10 18:35:01 -07:00
Arfon Smith
020e2a06d5 Updating JATS template to v1.1dtd (#5632)
* Updating JATS template to v1.1dtd

* Update writer.jats
2019-07-06 23:31:02 +02:00
Brian Leung
9c4ba81357 Markdown reader: handle inline code more eagerly within lists. (#5628)
Closes #5627.
2019-07-06 23:14:21 +02:00
Pete Ryland
24c781039f Fix redundant constraint warnings. (#5625) 2019-07-02 10:29:34 +02:00
martinfrancois
4b73544087 add tests for EPUB2 and EPUB3 cover reader 2019-06-22 22:07:26 +02:00
oquechy
f0edf60364 Support epigraph command in LaTeX Reader.
Closes #3523.
2019-06-21 18:27:26 +02:00
blmage
449c133406 Add a test for MathML formulas in ODT documents 2019-06-20 21:55:31 +02:00
Jeroen de Haas
e67f4c58f2 Powerpoint code formatting is now context dependent (#5573)
This commit alters the way in which the Powerpoint writer treats
inline code and code blocks.

- Inline code is now formatted at the same size as the surrounding
  text.
- Code blocks are now given a margin and font size according to their
  level.
- Furthermore this commit allows changing the font with which code is
  formatted via the `monofont` option.

Tested in
- PowerPoint 365 for Windows - 1808 (Build 10730.20344 Click-to-Run)
- PowerPoint 365 for Mac - 16.26 (19060901)
2019-06-14 08:42:06 -07:00
John MacFarlane
2cd1c7b30f Update test for skylighting 0.8.1.1. 2019-06-13 19:33:52 -07:00
John MacFarlane
fa50da3030 OpenDocument writer: Avoid duplicate attributes.
We use the innermost attribute in nested cases.

Closes #4634.
2019-06-12 21:51:09 -07:00
Albert Krewinkel
11bb862767 Lua: add a clone() method to all AST elements (#5572)
Closes: #5568
2019-06-12 09:58:38 -07:00
John MacFarlane
bec95c97ac LaTeX writer: Don't highlight code in headings.
This causes compilation errors, and I don't know how
to work around them.  Closes #5574.
2019-06-11 20:47:29 -07:00
John MacFarlane
9725522a7f Revert "JATS template: don't include journal-meta unless 'journal'..."
This reverts commit 193b5cceba.

journal-meta is required.  Better to include it even if it
doesn't validate through lack of required fields.
2019-06-11 17:31:52 -07:00
John MacFarlane
193b5cceba JATS template: don't include journal-meta unless 'journal'...
...is set in metadata.

Otherwise we get validation errors.
2019-06-11 17:08:40 -07:00
John MacFarlane
5bbaedac57 JATS writer: ensure validity of pub-date.
We try to parse the date and convert to year, month, day,
as expected in pub-date.  We also add an iso-8601-date attribute
if possible.
2019-06-11 17:01:36 -07:00
John MacFarlane
f7c0d2fb3f JATS writer: don't use break element for LineBreak.
It's only allowed in a few special contexts, and not in
p elements.
2019-06-11 16:19:12 -07:00
John MacFarlane
bb68c183b3 JATS writer: don't embed string-name in string.
That's illegal with this DTD.
2019-06-11 16:16:05 -07:00
Albert Krewinkel
6704b66c33
test/lua/module/pandoc.lua: fix non-determinism in test 2019-06-11 21:17:06 +02:00
Albert Krewinkel
7f9b32e36a
data/pandoc.lua: fix deletion of nonexistent attributes
Fixes: #5569
2019-06-11 19:48:00 +02:00
Albert Krewinkel
03b9f3d5f3
Lua pandoc module: better tests for Attr and AttributeList 2019-06-11 19:48:00 +02:00
John MacFarlane
550d949b9e JATS writer: wrap elements with p when needed.
The JATS spec restricts contents of certain
elements (fn, list-item); this patch wraps
elements that can't go in these contexts inside
p elements with specific-use "wrapper", so the
documents will conform.

Closes #5570.
2019-06-11 10:36:34 -07:00
John MacFarlane
3febd81cbc LaTeX writer: Use mbox to get proper behavior inside \sout.
Closes #5529.
2019-06-10 15:02:48 -07:00
John MacFarlane
59529e408b Asciidoc writer: use doubled ## when necessary for spans.
Closes #5566.
2019-06-10 14:47:04 -07:00
John MacFarlane
2e12106a90 Asciidoc writer: ensure correct nesting strong/emph.
Closes #5565.
2019-06-10 14:42:08 -07:00
Alexander Krotov
814c3af4df Muse reader: test that links inside image descriptions work 2019-06-09 14:08:37 +03:00
John MacFarlane
05ae9f265f Roll back automatic figure/table numbering in ODT/OpenDocument.
This was added in pandoc 2.7.2, but it makes it impossible
to use pandoc-crossref. So this has been rolled back for now,
until we find a good solution to make this behavior optional
(or a creative way to let pandoc-crossref and this feature
to coexist).

See #5474.
2019-06-08 14:17:52 -07:00
John MacFarlane
d1df2b2783 LaTeX reader: pass through unknown listings language as class.
Previously if the language was not in the list of listings-
supported languages, it would not be added as a class, so
custom syntax highlighting could not be used.

Closes #5540.
2019-06-08 12:25:34 -07:00
John MacFarlane
d8b4e45be0 LaTeX writer: Include inline code attributes with --listings.
Closes #5420.
2019-06-07 10:03:10 -07:00
Albert Krewinkel
1c36857465 Add jira writer (#5548)
This adds support for Atlassian's jira markup.

Closes #2497
2019-06-05 17:52:23 -04:00
John MacFarlane
10615420de Include trailing {}s in raw latex commands.
Change is in rawLaTeXInline in LaTeX reader, but
it affects the markdown reader and other readers
that allow raw LaTeX.

Previously, trailing `{}` would be included for
unknown commands, but not for known commands.
However, they are sometimes used to avoid a trailing
space after the command.  The chances that a `{}`
after a LaTeX command is not part of the command
are very small.

Closes #5439.
2019-06-04 21:20:11 -07:00
John MacFarlane
f82d91eb49 Markdown reader: don't create implicit reference for empty header.
Closes #5549.
2019-06-04 08:39:54 -07:00
John MacFarlane
928681ca04 Avoid unwanted interpretation of def list term as other kind of block,
e.g. ordered list item, in Markdown writer.

Closes #554.
2019-06-03 09:51:19 -07:00
Albert Krewinkel
7db3d9ab04
Textile writer: fix closing tag for math output
Opening and closing tag for math output match now.
2019-06-01 10:16:00 +02:00
Albert Krewinkel
f7222370af
Lua modules: test pandoc.mediabag 2019-05-30 08:44:40 +02:00
Albert Krewinkel
3c3e9a12cd
Lua Version type: shorten "version too old" message 2019-05-29 22:59:45 +02:00
John MacFarlane
e871d65b67
Merge pull request #5526 from tarleb/richer-version-type
Lua: add Version type to simplify comparisons
2019-05-29 12:05:04 -04:00
John MacFarlane
1de7b20ebb
Merge pull request #5497 from mb21/html-writer-video-audio
Output HTML5 video and audio elements
2019-05-29 12:01:12 -04:00
John MacFarlane
970b820f47 HTML reader: misc. epub related fixes.
- With epub extensions, check for epub:type in addition to type.
- Fix problem with noteref parsing which caused block-level
  content to be eaten with the noteref.
- Rename pAnyTag to pAny.
- Refactor note resolution.
2019-05-29 08:15:50 -07:00
Albert Krewinkel
505f5bf5d9
Lua: add Version type to simplify comparisons
Version specifiers like `PANDOC_VERSION` and `PANDOC_API_VERSION` are
turned into `Version` objects. The objects simplify version-appropriate
comparisons while maintaining backward-compatibility.

A function `pandoc.types.Version` is added as part of the newly
introduced module `pandoc.types`, allowing users to create version
objects in scripts.
2019-05-29 10:07:43 +02:00
mb21
a58304e00e HTML writer: output video and audio elements
depending on file extension of the image path
2019-05-29 09:43:50 +02:00
John MacFarlane
2ad5dacf87 Remove command test for #5517.
We need a better test that works cross-platform.
Until then, removing this.

Closes #5528.
2019-05-28 12:45:31 -07:00