Commit graph

611 commits

Author SHA1 Message Date
John MacFarlane
59875185b3 Add command test for #7092 2021-02-12 19:04:14 -08:00
John MacFarlane
8ca191604d Add new unexported module T.P.XMLParser.
This exports functions that uses xml-conduit's parser to
produce an xml-light Element or [Content].  This allows
existing pandoc code to use a better parser without
much modification.

The new parser is used in all places where xml-light's
parser was previously used.  Benchmarks show a significant
performance improvement in parsing XML-based formats
(especially ODT and FB2).

Note that the xml-light types use String, so the
conversion from xml-conduit types involves a lot
of extra allocation.  It would be desirable to
avoid that in the future by gradually switching
to using xml-conduit directly. This can be done
module by module.

The new parser also reports errors, which we report
when possible.

A new constructor PandocXMLError has been added to
PandocError in T.P.Error [API change].

Closes #7091, which was the main stimulus.

These changes revealed the need for some changes
in the tests.  The docbook-reader.docbook test
lacked definitions for the entities it used; these
have been added. And the docx golden tests have been
updated, because the new parser does not preserve
the order of attributes.

Add entity defs to docbook-reader.docbook.

Update golden tests for docx.
2021-02-10 22:04:11 -08:00
John MacFarlane
8e9131db4e Markdown reader: improved handling of mmd link attributes in references.
Previously they only worked for links that had titles.  Closes #7080.
2021-02-06 21:52:12 -08:00
John MacFarlane
02d3c71e72 BibTeX writer: use doclayout and doctemplate.
This change allows bibtex/biblatex output to wrap as other
formats do, depending on the settings of `--wrap` and `--columns`.

It also introduces default templates for bibtex and biblatex,
which allow for using the variables `header-include`, `include-before`
or `include-after` (or alternatively the command line options
`--include-in-header`, `--include-before-body`, `--include-after-body`)
to insert content into the generated bibtex/biblatex.

This change requires a change in the return type of the unexported
`T.P.Citeproc.writeBibTeXString` from `Text` to `Doc Text`.

Closes #7068.
2021-02-01 18:05:20 -08:00
John MacFarlane
b239c89a82 BibTeX writer fixes. Closes #7067.
+ Require citeproc 0.3.0.7, which correctly titlecases when titles
  contain non-ASCII characters.
+ Correctly handle 'pages' (= 'page' in CSL).
+ Correctly handle BibLaTeX 'langid' (= 'language' in CSL).
+ In BibTeX output, protect foreign titles since there's no language
  field.
2021-02-01 11:23:07 -08:00
John MacFarlane
d1875b69ec RST reader: fix handling of header in CSV tables.
The interpretation of this line is not affected
by the delim option. Closes #7064.
2021-01-31 12:05:46 -08:00
John MacFarlane
9223788a05 Markdown writer: handle math right before digit.
We insert an HTML comment to avoid a `$` right before
a digit, which pandoc will not recognize as a math delimiter.
2021-01-29 18:29:17 -08:00
John MacFarlane
98c2a52b4e Clean up BibTeX parsing.
Previously there was a messy code path that gave strange
results in some cases, not passing through raw tex but
trying to extract a string content.  This was an artefact
of trying to handle some special bibtex-specific commands
in the BibTeX reader. Now we just handle these in the
LaTeX reader and simplify parsing in the BibTeX reader.
This does mean that more raw tex will be passed through
(and currently this is not sensitive to the `raw_tex`
extension; this should be fixed).

Closes #7049.
2021-01-26 22:45:57 -08:00
John MacFarlane
198ce0cde9 ImageSize: use viewBox for svg if no length, width.
This change allows pandoc to extract size information
from more SVGs.  Closes #7045.
2021-01-22 20:49:41 -08:00
Albert Krewinkel
b4b3560191
JATS writer: allow to use element-citation 2021-01-22 19:35:08 +01:00
John MacFarlane
3d6ebc9051 Use dev version of citeproc.
Change a citation test which had wrong disambiguation
(see jgm/citeproc#44).
2021-01-15 11:51:07 -08:00
Albert Krewinkel
68fa437999
JATS writer: fix citations (#7018)
* JATS writer: keep code lines at 80 chars or below

* JATS writer: fix citations
2021-01-10 15:35:48 -08:00
John MacFarlane
327e1428c5 gfm/commonmark writer: implement start number on ordered lists.
Previously they always started at 1, but according to the spec
the start number is respected. Closes #7009.
2021-01-07 16:42:05 -08:00
John MacFarlane
c0d8b186d1 T.P.Parsing: modify gridTableWith' for headerless tables.
If the table lacks a header, the header row should be an empty
list. Previously we got a list of empty cells, which caused
an empty header to be emitted instead of no header.  In LaTeX/PDF
output that meant we got a double top line with space between.

@tarleb @despres - please let me know if this is problematic
for some reason I'm not grasping.
2021-01-07 11:07:03 -08:00
John MacFarlane
15ba184e6e HTML writer: fix implicit_figure at end of footnotes.
Closes #7006.
2021-01-05 12:07:02 -08:00
David Martschenko
385b6a3b21
Implement defaults file inheritance (#6924)
Allow defaults files to inherit options from other defaults files by
specifying them with the following syntax:
`defaults: [list of defaults files or single defaults file]`.
2021-01-05 10:15:59 -08:00
John MacFarlane
ea479bf28a LaTeX reader: handle filecontents environment.
Closes #7003.
2021-01-04 14:05:03 -08:00
John MacFarlane
9a18cf4b59 LaTeX writer: revert table line height increase in 2.11.3.
In 2.11.3 we started adding `\addlinespace`, which produced less
dense tables.  This wasn't an intentional change; I misunderstood
a comment in the discussion leading up to the change. This commit
restores the earlier default table appearance.

Note that if you want a less dense table, you can use something like
`\def\arraystretch{1.5}` in your header.

Closes #6996.
2021-01-02 07:56:07 -08:00
John MacFarlane
23f964b907 Mediawiki reader: allow space around storng/emph delimiters.
Closes #6993.
2020-12-30 21:31:28 -08:00
John MacFarlane
a7a162ea55 Update test for new citeproc and require it in cabal. 2020-12-28 14:40:23 -08:00
timo-a
668596cc89
Add support for writing nested tables to asciidoc (#6972)
Added field to WriterState that denotes the current nesting level for traversing tables.
Depending on the value of that field nested tables are recognized and written.
Asciidoc supports one level of nesting. If deeper tables are to be written, they are
omitted and a warning is issued.
2020-12-27 18:42:28 -08:00
John MacFarlane
95b15fe6d3 Remove some test files that are no longer used. 2020-12-18 11:22:29 -08:00
John MacFarlane
b4b4e32307 Properly handle boolean values in writing YAML metadata.
(Markdown writer.)
This requires doctemplates >= 0.9.
Closes #6388.
2020-12-15 23:45:34 -08:00
John MacFarlane
7d799bfcda Allow both inline and external references to be used
with `--citeproc`.  This fixes a regression, since pandoc-citeproc
allowed these to be combined.

Closes #6951.
2020-12-15 08:51:43 -08:00
John MacFarlane
c43e2dc0f4 RST writer: better image handling.
- An image alone in its paragraph (but not a figure) is now
  rendered as an independent image, with an `alt` attribute
  if a description is supplied.
- An inline image that is not alone in its paragraph will
  be rendered, as before, using a substitution.
  Such an image cannot have a "center", "left", or
  "right" alignment, so the classes `align-center`,
  `align-left`, or `align-right` are ignored.
  However, `align-top`, `align-middle`, `align-bottom`
  will generate a corresponding `align` attribute.

Closes #6948.
2020-12-13 15:25:46 -08:00
mb21
208cb96196 ICML writer: fix image bounding box for custom widths/heights
fixes #6936
2020-12-12 14:49:11 +01:00
John MacFarlane
0a502e5ff5 HTML reader: retain attribute prefixes and avoid duplicates.
Previously we stripped attribute prefixes, reading
`xml:lang` as `lang` for example. This resulted in
two duplicate `lang` attributes when `xml:lang` and
`lang` were both used.  This commit causes the prefixes
to be retained, and also avoids invald duplicate
attributes.

Closes #6938.
2020-12-10 15:44:10 -08:00
Nils Carlson
c161893f44
OpenDocument writer: Allow references for internal links (#6774)
This commit adds two extensions to the OpenDocument writer,
`xrefs_name` and `xrefs_number`.

Links to headings, figures and tables inside the document are
substituted with cross-references that will use the name or caption
of the referenced item for `xrefs_name` or the number for `xrefs_number`.

For the `xrefs_number` to be useful heading numbers must be enabled
in the generated document and table and figure captions must be enabled using for example the `native_numbering` extension.

In order for numbers and reference text to be updated the generated
document must be refreshed.

Co-authored-by: Nils Carlson <nils.carlson@ludd.ltu.se>
2020-12-05 10:00:04 -08:00
John MacFarlane
ddb76cb356 LaTeX reader: don't apply theorem default styling to a figure inside.
If we put an image in italics, then when rendering to Markdown
we no longer get an implicit figure.

Closes #6925.
2020-12-05 09:53:39 -08:00
John MacFarlane
dc3ef5201f Markdown writer: ensure that a new csl-block begins on a new line.
This just looks better and doesn't affect the semantics.
See #6921.
2020-12-04 10:55:48 -08:00
John MacFarlane
7199d68ba0 EPUB writer: include title page in landmarks.
Closes #6919.

Note that the toc is also included if `--toc` is specified.
2020-12-03 21:39:44 -08:00
Albert Krewinkel
a9c766291f
HTML reader: support body headers, row head columns
Closes: #6312
2020-11-27 10:36:13 +01:00
Igor Pashev
630b1bff2b
LaTeX reader: preserve center environment (#6852)
The contents of the `center` environment are put in a `Div`
with class `center`.
2020-11-26 12:04:31 -08:00
John MacFarlane
70a7c2446e Update tests for LaTeX table changes. 2020-11-25 15:49:17 -08:00
John MacFarlane
815976d537 Fix truncation of [Citation] list in Cite inside footnotes...
This affected author-in-text citations in footnotes.
It didn't cause problems for the printed output, but for
filters that expected the citation id and other information.

Closes #6890.
2020-11-25 09:10:10 -08:00
Albert Krewinkel
c9f98e2bf5
HTML reader: support row or column-spanning table cells 2020-11-24 14:17:35 +01:00
Nils Carlson
75c881e2d9
OpenDocument Writer: Implement Div and Span ident support (#6755)
Spans and Divs containing an ident in the Attr will become bookmarks
or sections with idents in OpenDocument format.
2020-11-22 22:23:30 -08:00
John MacFarlane
b5b5ef92cb LaTeX writer: Improve table spacing.
+ Remove the `\strut` that was added at the end of minipage
  environments in cells.

+ Replace `\tabularnewline` with `\\ \addlinespace`.

Closes #6842, closes #6860.
2020-11-22 10:54:42 -08:00
Nils Carlson
ae52918faa
OpenDocument writer: Table text width support (#6792)
Support for table width as a percentage of text width by summing
width of columns and verifying that the sum is > 0 and <= 1.
2020-11-21 12:42:43 -08:00
John MacFarlane
7db2cf5d2f LaTeX reader: more robust parsing of bracketed options.
Improves on 9a40976.  Closes #6873.
2020-11-21 12:24:37 -08:00
Nils Carlson
56ceaf49dc
DocBook reader: Table text width support (#6791)
Table width in relation to text width is not natively supported
by docbook but is by the docbook fo stylesheets through an XML
processing instruction, <?dbfo table-width="50%"?> .
Implement support for this instruction in the DocBook reader.
2020-11-20 16:05:56 -08:00
John MacFarlane
9a4097640f Improve LaTeX option parsing...
in cases where we run into trouble parsing inlines til the
closing `]`, e.g. quotes, we return a plain string with the
option contents. Previously we mistakenly included the brackets
in this string.

Closes #6869.
2020-11-20 13:40:26 -08:00
John MacFarlane
c647948ff1 commonmark_x: replace auto_identifiers with gfm_auto_identifiers.
`commonmark_x` never actually supported `auto_identifiers` (it
didn't do anything), because the underlying library implements
gfm-style identifiers only.

Attempts to add the `autolink_identifiers` extension to
`commonmark` will now fail with an error.

Closes #6863.
2020-11-20 09:17:14 -08:00
John MacFarlane
0962b30d84 Man reader: improve handling of .IP.
We now better handle `.IP` when it is used with non-bullet,
non-numbered lists, creating a definition list.

We also skip blank lines like groff itself.

Closes #6858.
2020-11-18 22:44:32 -08:00
TEC
0306eec5fa Replace org #+KEYWORDS with #+keywords
As of ~2 years ago, lower case keywords became the standard (though they
are handled case insensitive, as always):
13424336a6

Upper case keywords are exclusive to the manual:
- https://orgmode.org/list/871s50zn6p.fsf@nicolasgoaziou.fr/
- https://orgmode.org/list/87tuuw3n15.fsf@nicolasgoaziou.fr/
2020-11-18 14:48:56 +01:00
John MacFarlane
bf3fea0a8c Markdown reader: fix regression with example list references.
This affects example list references followed by dashes.
Introduced by commit b8d17f7.
Closes #6855.
2020-11-17 20:36:59 -08:00
John MacFarlane
5271c6b3fb Improve fix to siunitx numbers with minus.
- use real minus sign
- use tests contributed by Igor Pashev.
2020-11-16 16:36:16 -08:00
John MacFarlane
734b4c26a9 LaTeX reader: Fix negative numbers in siunitx commands.
The commit a157e1a broke negative numbers, e.g.
`\SI{-33}{\celcius}` or `\num{-3}`. This fixes the regression.
2020-11-16 14:08:29 -08:00
John MacFarlane
d7f905fb63 Markdown reader: fix detection of locators following in-text citations.
Prevously, if we had `@foo [p. 33; @bar]`, the `p. 33` would be
incorrectly parsed as a prefix of `@bar` rather than a suffix
of `@foo`.
2020-11-15 17:51:03 -08:00
Aner Lucero
f63b76e169 Markdown writer: default to using ATX headings.
Previously we used Setext (underlined) headings by default.
The default is now ATX (`##` style).

* Add the `--markdown-headings=atx|setext` option.
* Deprecate `--atx-headers`.
* Add constructor 'ATXHeadingInLHS` constructor to `LogMessage` [API change].
* Support `markdown-headings` in defaults files.
* Document new options in MANUAL.

Closes #6662.
2020-11-14 21:33:32 -08:00