Commit graph

695 commits

Author SHA1 Message Date
Michael Hoffmann
5001fd3f4d
Docx writer: Handle bullets correctly in lists by not reusing numIds (#7822)
Make sure that we only create one bullet per list item in docx.  In
particular, when a div is a list item, its contained paragraphs will
now no longer wrongly get individual bullets.

This is accomplished by making sure that for each list, we only use
the associated numId once.  Any repeated use would add incorrect
bullets to the document.

Closes #7689
2022-01-11 15:48:41 -08:00
Lucas Viana
fb91a91615 Org reader: support alphabetical (fancy) lists
This adds support for alphabetical lists in org by enabling the
extension Ext_fancy_lists, mimicking the behaviour of Org Mode when
org-list-allow-alphabetical is enabled.

Enabling Ext_fancy_lists will also make Pandoc differentiate between the
delimiters of ordered lists (periods or closing parentheses). Org does
this differentiation by default when exporting to some formats (e.g.
plain text) but does not in others (e.g. html and latex), so I decided
to copy Pandoc's markdown reader behaviour.
2022-01-09 09:39:27 -08:00
Lucas Viana
45e2e0d018 Org writer: support starting number cookies
This complements #7806 by supporting writing Org ordered lists that
start at a specific number.
2022-01-07 10:48:28 -08:00
Lucas Viana
4be41e3bb5 Org reader: support counter cookies in lists
This adds support for counter cookies in org lists. Such cookies are
used to override the item counter in ordered lists. In org it is
possible to set the counter at any list item, but since Pandoc AST does
not support this, we restrict the usage to setting an offset for the
entire ordered list, by using the cookie in the first list item.

Note that even though unordered lists do not have counters, Org Mode
still parses such cookies in unordered lists and suppresses them in the
output, so we do the same.

Also, even though org-list-allow-alphabetical is disabled in Emacs by
default, for some reason alphabetical cookies are always parsed and used
in Org Mode regardlessly of whether this option is enabled or the list
style is decimal, so we do the same.

E.g.
 2. test
 3. test
Is parsed as an ordered list starting at 1, as before. This also
conforms to Org Mode behaviour.

 1. [@2] test
 2. test
Is now parsed as an ordered list starting at 2, so that it conforms to
Org Mode behaviour.

Note that when parsing
 1. [@2] test
 2. [@9] test
the second cookie is silenced and the entire list starts at 2. This is
because the current Pandoc AST does not support expressing a change in
the counter at a specific item.
2022-01-06 19:33:13 +01:00
Albert Krewinkel
1f8638fb54 Lua: add pandoc.template module
The module provides a `compile` function to use strings as templates.
2022-01-04 11:55:59 -08:00
Albert Krewinkel
6a5ac90bf1 Lua: add pandoc.WriterOptions constructor 2022-01-04 11:55:59 -08:00
Tuong Nguyen Manh
32297d5677 Odt: Add list-header
The list-header is a type of list-item.
Therefore, it will be treated exactly like one.
2022-01-02 15:05:09 -08:00
Albert Krewinkel
b7a44f9d19 Copyright notices: update for 2022 2022-01-02 11:59:22 -08:00
Albert Krewinkel
eae9be3a48
Org reader: allow trailing spaces after key/value pairs in directives
Ensures that spaces at the end of attribute directives like
`#+ATTR_HTML: :width 100%` (note the trailing spaces) are accepted.
2022-01-01 13:44:14 +01:00
John MacFarlane
c4f6e6cb57 HTML writer: make line breaks more consistent.
- With `--wrap=none`, we now output line breaks between
  block-level elements. Previously they were omitted
  entirely, so the whole document was on one line, unless
  there were literal line breaks in pre sections.  This makes
  the HTML writer's behavior more consistent with that of
  other writers.

- Put newline after `<dd>`.

- Put newlines after block-level elements in footnote section.
2021-12-22 09:45:02 -08:00
Albert Krewinkel
cd2bffee1e
Lua: use more natural representation for Reference values
Omit `false` boolean values, push integers as numbers.
2021-12-20 09:41:03 +01:00
binaarinen
0610f16f7f
Add a writer for Markua 0.10 (#7729)
Markua is a markdown variant used by Leanpub.
More information about Markua can be found at https://leanpub.com/markua/read.

Adds a new exported function `writeMarkua` from T.P.Writers.Markdown.
[API change]

Closes #1871.

Co-authored by Tim Wisotzki and Samuel Lemmenmeier.
2021-12-19 12:10:41 -08:00
John MacFarlane
394fa9d072 Org reader: parse official org-cite citations.
We also support the older org-ref style as a fallback.
We no longer support the "markdown-style" citations.

See #7329.
2021-12-14 11:34:32 -08:00
John MacFarlane
5817e86491 Org reader: remove support for "Berkeley style" citations.
See #7329.
2021-12-14 09:20:26 -08:00
Kolen Cheung
a9a9a2c62a fix(IpynbOutput)!: rank always favors output format
Previously, both `fmt == f` case and Image have a rank of 1.
In the end, e.g. from ipynb to html conversion,
if both html and image exists, it actually prefers the image.
This commit changes this, so that fmt == f is always highest rank,
and rank never collides.
This is achieved by keeping fmt == f case having rank 1,
and every other rank increased by 1.
2021-12-11 09:42:30 -08:00
Albert Krewinkel
bfb3118ebb
Lua tests: remove roundtrip tests
Property tests that roundtrip elements through the Lua stack are
performed in the test-suite of the pandoc-lua-marshal package. No need
to test this here as well.
2021-12-10 18:28:54 +01:00
Albert Krewinkel
a64ea18647
Powerpoint tests: shorten lines by grouping tests
This makes the test output more pleasant to read in narrow terminal
windows.
2021-12-10 18:25:28 +01:00
Kolen Cheung
20eb8ac7fd
ipynb writer: handle cell output with raw block of markdown (#7563)
Write RawBlock of markdown in code-cell output.

#7561 makes the ipynb reader reads code-cell output with mime
"text/markdown" to a RawBlock of markdown

This commit makes the ipynb writer writes this RawBlock of markdown
back inside a code-cell output with the same mime, preserving this
information in round-trip

Add tests of ipynb reader (#7561) and ipynb writer (#7563)'s ability to
handle a "text/markdown" mime type in a code-cell output
2021-12-09 20:36:56 -08:00
Albert Krewinkel
3692a1d1e8
Lua: use package pandoc-lua-marshal (#7719)
The marshaling functions for pandoc's AST are extracted into a separate
package. The package comes with a number of changes:

  - Pandoc's List module was rewritten in C, thereby improving error
    messages.

  - Lists of `Block` and `Inline` elements are marshaled using the new
    list types `Blocks` and `Inlines`, respectively. These types
    currently behave identical to the generic List type, but give better
    error messages. This also opens up the possibility of adding
    element-specific methods to these lists in the future.

  - Elements of type `MetaValue` are no longer pushed as values which
    have `.t` and `.tag` properties. This was already true for
    `MetaString` and `MetaBool` values, which are still marshaled as Lua
    strings and booleans, respectively. Affected values:

      + `MetaBlocks` values are marshaled as a `Blocks` list;

      + `MetaInlines` values are marshaled as a `Inlines` list;

      + `MetaList` values are marshaled as a generic pandoc `List`s.

      + `MetaMap` values are marshaled as plain tables and no longer
        given any metatable.

  - The test suite for marshaled objects and their constructors has
    been extended and improved.

  - A bug in Citation objects, where setting a citation's suffix
    modified it's prefix, has been fixed.
2021-11-27 17:08:01 -08:00
John MacFarlane
db9a73c842 Lua tests: reset path and cpath when testing 'require' fallback. 2021-11-19 21:55:14 -08:00
willj-dev
005dc7ce56
RST reader: handle class attribute for for custom roles (#7700)
Previously the class attribute was ignored, and the name of the role used as the class.
Closes #7699.
2021-11-18 17:33:57 -08:00
Albert Krewinkel
cd91f72843
Lua: set lpeg, re as globals; allow shared lib access via require
The `lpeg` and `re` modules are loaded into globals of the respective
name, but they are not necessarily registered as loaded packages. This
ensures that

- the built-in library versions are preferred when setting the globals,
- a shared library is used if pandoc has been compiled without `lpeg`,
  and
- the `require` mechanism can be used to load the shared library if
  available, falling back to the internal version if possible and
  necessary.
2021-11-17 10:03:04 +01:00
Albert Krewinkel
96a01451ef
JATS writer: ensure figures are wrapped with <p> in list items.
This prevents the generation of invalid output.
2021-11-12 13:29:08 +01:00
Christian Despres
abdfefebdf
Writers.Shared: Improve toLegacyTable.
Closes #7683.
(PR #7684)
2021-11-11 20:55:37 -08:00
Albert Krewinkel
ab0fe676a8
Lua: ensure that 're' module is always available.
The module is shipped with LPeg.
2021-11-08 12:22:33 +01:00
John MacFarlane
c712d13b67 Org reader: allow an initial :PROPERTIES: drawer to add to metadata.
Closes #7520.
2021-10-22 22:10:25 -07:00
Albert Krewinkel
8523bb01b2 Lua: marshal Attr values as userdata
- Adds a new `pandoc.AttributeList()` constructor, which creates the
  associative attribute list that is used as the third component of
  `Attr` values. Values of this type can often be passed to constructors
  instead of `Attr` values.

- `AttributeList` values can no longer be indexed numerically.
2021-10-22 11:16:51 -07:00
Albert Krewinkel
e4287e6c95 Lua: marshal Pandoc values as userdata 2021-10-22 11:16:51 -07:00
Albert Krewinkel
9e74826ba9 Switch to hslua-2.0
The new HsLua version takes a somewhat different approach to marshalling
and unmarshalling, relying less on typeclasses and more on specialized
types. This allows for better performance and improved error messages.

Furthermore, new abstractions allow to document the code and exposed
functions.
2021-10-22 11:16:51 -07:00
Milan Bracke
465c28d28e Docx reader: fix handling of empty fields
Some fields only have an instrText and no content, Pandoc didn't
understand these, causing other fields to be misunderstood because it
seemed like a field was still open when it wasn't.
2021-10-18 19:15:40 -07:00
Milan Bracke
6acc82c5d2 Docx parser: implement PAGEREF fields
These fields, often used in tables of contents, can be a hyperlink.
2021-10-18 19:15:40 -07:00
Milan Bracke
193f6bfeba Docx reader: fix handling of nested fields
Fields delimited by fldChar elements can contain other fields. Before,
the nested fields would be ignored, except for the end, which would be
considered the end of the parent field.

To fix this issue, fields needed to be considered containing ParParts
instead of Runs, since a Run can't represent complex enough structures.
This also impacted Hyperlinks since they can originate from a field.
2021-10-18 19:15:40 -07:00
Emily Bourke
8af15ab345 pptx: Fix list level numbering
In PowerPoint, the content of a top-level list is at the same level as
the content of a top-level paragraph – the only difference is that a
list style has been applied.

At the moment, the pptx writer increments the paragraph level on each
list, turning what should be top-level lists into second-level lists.

This commit changes that logic, only incrementing the paragraph level on
continuation paragraphs of lists.

- Fixes https://github.com/jgm/pandoc/issues/4828
- Fixes https://github.com/jgm/pandoc/issues/4663
2021-10-17 17:24:30 -07:00
Samuel Tardieu
a41c1fe0bb asciidoc writer: translate numberLines attribute to linesnum switch
AsciiDoctor allows to request line numbering on code blocks by
using a switch on the `source` block, such as in:

```
[source%linesnum,haskell]
----
some Haskell code here
----
```
2021-10-14 13:41:12 -07:00
Milan Bracke
0f98cbff4b Avoid blockquote when parent style has more indent
When a paragraph has an indentation different from the parent (named)
style, it used to be considered a blockquote. But this only makes sense
when the paragraph has more indentation. So this commit adds a check
for the indentation of the parent style.
2021-10-10 16:27:32 -07:00
Emily Bourke
aa78765bf9 pptx: Remove excessive layout tests
When I added the tests for moved layouts and deleted layouts, I added
them to all tests. However, this doesn’t really give a lot more info
than having single tests, and the extra tests take up time and disk
space.

This commit removes the moved-layouts and deleted-layouts tests, in
favour of a single test for each of those scenarios.
2021-10-07 08:45:43 -07:00
John MacFarlane
11baeb8850 OOXML tests: use pretty-printed form to display diffs.
Otherwise everything is on one line and the diff is uninformative.
2021-10-04 12:12:16 -07:00
John MacFarlane
6ff04ac52d Fix compareXML helper in Tests.Writers.OOXML.
Given how it is used, we were getting "mine" and "good"
flipped in the test results.
2021-10-02 06:52:40 -07:00
John MacFarlane
c266734448 Use pretty-simple to format native output.
Previously we used our own homespun formatting.  But this
produces over-long lines that aren't ideal for diffs in tests.
Easier to use something off-the-shelf and standard.

Closes #7580.

Performance is slower by about a factor of 10, but this isn't
really a problem because native isn't suitable as a serialization
format. (For serialization you should use json, because the reader
is so much faster than native.)
2021-09-21 12:37:42 -07:00
John MacFarlane
a1ca51c979 Command tests: raise error if command doesn't begin with %. 2021-09-21 10:42:14 -07:00
Emily Bourke
50adea220d pptx: Support footers in the reference doc
In PowerPoint, it’s possible to specify footers across all slides,
containing a date (optionally automatically updated to today’s date),
the slide number (optionally starting from a higher number than 1), and
static text. There’s also an option to hide the footer on the title
slide.

Before this commit, none of that footer content was pulled through from
the reference doc: this commit supports all the functionality listed
above.

There is one behaviour which may not be immediately obvious: if the
reference doc specifies a fixed date (i.e. not automatically updating),
and there’s a date specified in the metadata for the document, the
footer date is replaced by the metadata date.

- Include date, slide number, and static footer content from reference
  doc
- Respect “slide number starts from” option
- Respect “Don’t show on title slide” option
- Add tests
2021-09-18 09:55:45 -07:00
Emily Bourke
7c22c0202e pptx: Support specifying slide background images
In the reveal-js output, it’s possible to use reveal’s
`data-background-image` class on a slide’s title to specify a background
image for the slide.

With this commit, it’s possible to use `background-image` in the same
way for pptx output. Only the “stretch” mode is supported, and the
background image is centred around the slide in the image’s larger axis,
matching the observed default behaviour of PowerPoint.

- Support `background-image` per slide.
- Add tests.
- Update manual.
2021-09-16 19:45:53 -07:00
Emily Bourke
0fb6474a55 pptx: Add support for incremental lists
- Support -i option
- Support incremental/noincremental divs
- Support older block quote syntax
- Add tests

One thing not clear from the manual is what should happen when the input
uses a combination of these things. For example, what should the
following produce?

```md
::: {.incremental .nonincremental}
- are
- these
- incremental?
:::

::: incremental
::::: nonincremental
- or
- these?
:::::
:::

::: nonincremental
> - how
> - about
> - these?
:::
```

In this commit I’ve taken the following approach, matching the observed
behaviour for beamer and reveal.js output:

- if a div with both classes, incremental wins
- the innermost incremental/nonincremental div is the one which takes
  effect
- a block quote containing a list as its first element inverts whether
  the list is incremental, whether or not the quote is inside an
  incremental/non-incremental div

I’ve added some tests to verify this behaviour.

This commit closes issue #5689
(https://github.com/jgm/pandoc/issues/5689).
2021-09-15 09:13:05 -07:00
Emily Bourke
0ebe65e651 pptx: Fix logic for choosing Comparison layout
There was a mistake in the logic used to choose between the Comparison
and Two Content layouts: if one column contained only non-text (an image
or a table) and the other contained only text, the Comparison layout was
chosen instead of the desired Two Content layout.

This commit fixes that logic:

> If either column contains text followed by non-text, use Comparison.
  Otherwise, use Two Content.

It also adds a test asserting this behaviour.
2021-09-13 08:30:36 -07:00
Francesco Mazzoli
99a4d1d0b0
Support --reference-location for HTML output (#7461)
The HTML writer now supports `EndOfBlock`, `EndOfSection`, and
`EndOfDocument` for reference locations.  EPUB and HTML slide
show formats are also affected by this change.

This works similarly to the markdown writer, but with special care
taken to skipping section divs with what regards to the block level.

The change also takes care to not modify the output if `EndOfDocument`
is used.
2021-09-10 09:30:05 -07:00
Emily Bourke
b82a01b688 pptx: Add support for more layouts
Until now, the pptx writer only supported four slide layouts: “Title
Slide” (used for the automatically generated metadata slide), “Section
Header” (used for headings above the slide level), “Two Column” (used
when there’s a columns div containing at least two column divs), and
“Title and Content” (used for all other slides).

This commit adds support for three more layouts: Comparison, Content
with Caption, and Blank.

- Support “Comparison” slide layout

  This layout is used when a slide contains at least two columns, at
  least one of which contains some text followed by some non-text (e.g.
  an image or table). The text in each column is inserted into the
  “body” placeholder for that column, and the non-text is inserted into
  the ObjType placeholder. Any extra content after the non-text is
  overlaid on top of the preceding content, rather than dropping it
  completely (as currently happens for the two-column layout).

  + Accept straightforward test changes

    Adding the new layout means the “-deleted-layouts” tests have an
    additional layout added to the master and master rels.

  + Add new tests for the comparison layout
  + Add new tests to pandoc.cabal

- Support “Content with Caption” slide layout

  This layout is used when a slide’s body contains some text, followed by
  non-text (e.g. and image or a table). Before now, in this case the image
  or table would break onto a new slide: to get that output again, users
  can add a horizontal rule before the image or table.

  + Accept straightforward tests

    The “-deleted-layouts” tests all have an extra layout and relationship
    in the master for the Content with Caption layout.

  + Accept remove-empty-slides test

    Empty slides are still removed, but the Content with Caption layout is
    now used.

  + Change slide-level-0/h1-h2-with-text description

    This test now triggers the content with caption layout, giving a
    different (but still correct) result.

  + Add new tests for the new layout
  + Add new tests to the cabal file

- Support “Blank” slide layout

  This layout is used when a slide contains only blank content (e.g.
  non-breaking spaces). No content is inserted into any placeholders in
  the layout.

  Fixes #5097.

  + Accept straightforward test changes

    Blank layout now copied over from reference doc as well, when
    layouts have been deleted.

  + Add some new tests

    A slide should use the blank layout if:

    - It contains only speaker notes
    - It contains only an empty heading with a body of nbsps
    - It contains only a heading containing only nbsps

- Change ContentType -> Placeholder

  This type was starting to have a constructor for each placeholder on
  each slide (e.g. `ComparisonUpperLeftContent`). I’ve changed it
  instead to identify a placeholder by type and index, as I think that’s
  clearer and less redundant.

- Describe layout-choosing logic in manual
2021-09-01 07:16:17 -07:00
Emily Bourke
8dbea49092 pptx: Restructure tests
- Use dashes consistently rather than underscores
- Make a folder for each set of tests
- List test files explicitly (Cabal doesn’t support ** until version
  2.4)
2021-09-01 07:16:17 -07:00
Emily Bourke
8e5a79f264 pptx: Make first heading title if slide level is 0
Before this commit, the pptx writer adds a slide break before any table,
“columns” div, or paragraph starting with an image, unless the only
thing before it on the same slide is a heading at the slide level. In
that case, the item and heading are kept on the same slide, and the
heading is used as the slide title (inserted into the layout’s “title”
placeholder).

However, if the slide level is set to 0 (as was recently enabled) this
makes it impossible to have a slide with a title which contains any of
those items in its body.

This commit changes this behaviour: now if the slide level is 0, then
items will be kept with a heading of any level, if the heading’s the
only thing before the item on the same slide.
2021-08-27 09:47:03 -07:00
John MacFarlane
2e9a8935fb OOXML tests: silence warnings.
These can make the test output confusing, making people think
tests are failing when they're passing.
2021-08-17 15:33:10 -07:00
Emily Bourke
72823ad947 pptx: Select layouts from reference doc by name
Until now, users had to make sure that their reference doc contains
layouts in a specific order: the first four layouts in the file had to
have a specific structure, or else pandoc would error (or sometimes
successfully produce a pptx file, which PowerPoint would then fail to
open).

This commit changes the layout selection to use the layout names rather
than order: users must make sure their reference doc contains four
layouts with specific names, and if a layout with the right name isn’t
found pandoc will output a warning and use the corresponding layout from
the default reference doc as a fallback.

I believe the use of names rather than order will be clearer to users,
and the clearer errors will help them troubleshoot when things go wrong.

- Add tests for moved layouts
- Add tests for deleted layouts
- Add newly included layouts to slideMaster1.xml to fix tests
2021-08-17 09:35:25 -07:00