Previously, the writer would unconditionally emit HTMLish output for
subscripts, superscripts, strikeouts (if the strikeout extension is
disabled) and small caps, even with raw_html disabled.
Now there are plain-text (and, where possible, fancy Unicode)
fallbacks for all of these corresponding (mostly) to the Markdown
fallbacks, and the HTMLish output is only used when raw_html is
enabled.
This commit adds exported functions `toSuperscript` and
`toSubscript` to `Text.Pandoc.Writers.Shared`. [API change]
Closes#4528.
Closes#4284.
Headers with the corresponding tags should not appear in the output.
If one or more of the specified tags contains a non-tag character
like `+`, Org-mode will not treat that as a valid tag, but will
nonetheless continue scanning for valid tags. That behavior is not
replicated in this patch; entering `cat+dog` as one of the entries in
`#+EXCLUDE_TAGS` and running the file through Pandoc will cause the
parser to fail and result in the only excluded tag being the default, `noexport`.
Implement table caption numbering with a format
"Table 1: <caption>".
Translations are enabled and numbering is consecutive for
captioned tables, uncaptioned tables are not enumerated.
Captioned figures are now also numbered consecutively
and uncaptioned figures are not enumerated.
Figure captions are now numbered 1, 2, 3, ... The format in
the caption is "Figure 1: <caption>" and so on.
This is necessary in order for libreoffice to generate an
Illustration Index (Table of Figures) for included figures.
This patch eliminates the large gap we used to have between
bullet and text, and also ensures that numbers in numbered
lists will be right-aligned.
Closes#4385.
Now the `write*` functions for Docbook, HTML, ICML, JATS,
Man, Ms, OPML are sensitive to `writerPreferAscii`. Previously
the to-ascii translation was done in Text.Pandoc.App, and
thus not available to those using the writer functions
directly.
In addition, the LaTeX writer is now sensitive to
`writerPreferAscii` and to `--ascii`. 100% ASCII
output can't be guaranteed, but the writer will use
commands like `\"{a}` and `\l` whenever possible,
to avoid emiting a non-ASCII character.
A new unexported module, Text.Pandoc.Groff, has been
added to store functions used in the different groff-based
writers.
* Add support for multiprenote and multipostnote arguments.
The multiprenotes occur before the first prefix of a
multicite, and the multipostnotes follow the last suffix.
* Add test for multiprenote and multipostnote.
This changes the way styles for cells in the header row
and normal rows are handled in ODT tables.
Previously a new (but identical) style was generated for
every table, specifying the style of the cells within the table.
After this change there are two style definitions for table cells,
one for the cells in the header row, one for all other cells.
This doesn't change the actual styles, but makes post-processing
changes to the table styles much simpler as it is no longer
necessary to introduce new styles for header rows and there are
now only two styles where there was previously one per table.
With this change, autolinks are parsed as Links with
the `uri` class. (The same is true for bare links, if
the `autolink_bare_uris` extension is enabled.) Email
autolinks are parsed as Links with the `email` class.
This allows the distinction to be represented in the
URI.
Formerly the `uri` class was added to autolinks by
the HTML writer, but it had to guess what was an autolink
and could not distinguish `[http://example.com](http://example.com)`
from `<http://example.com>`. It also incorrectly recognized
`[pandoc](pandoc)` as an autolink. Now the HTML writer
simply passes through the `uri` attribute if it is present,
but does not add anything.
The Textile writer has been modified so that the `uri`
class is not explicitly added for autolinks, even if it
is present.
Closes#4913.
Inclusion of planning info (*DEADLINE*, *SCHEDULED*, and *CLOSED*) can
be controlled via the `p` export option: setting the option to `t` will
add all planning information in a *Plain* block below the respective
headline.
This now allows raw LaTeX environments, `\ref`, and `\eqref` to
be parsed (which is helpful for translation HTML documents using
MathJaX).
Closes#1126.
Also foreigncblockquote, hyphenblockquote, hyphencblockquote.
Closes#4848. But note: currently foreignquote will be
parsed as a regular Quoted inline (not using the quotes
appropriate to the foreign language).
Add support for `\|`, `\b`, `\G`, `\h`, `\d`, `\f`,
`\r`, `\t`, `\U`, `\i`, `\j`, `\newtie`, `\textcircled`.
Also fall back to combining characters when composed
characters are not available.
Closes#4652.
Inline math in `\(..\)`, display math in `\[..\]`, tex is now used.
Previously we'd "fake it with unicode" and fall back to tex when
that didn't work. But as of
3f50b95532
haddock supports latex math.
...they should only be recognized in siunitx contexts.
For example, `\l` outside of an siunitx context should be l-slash,
not l (for liter)!
Closes#4842.
We can't always tell if it's LaTeX, ConTeXt, or plain TeX.
Better just to use "tex" always.
Also changed:
ConTeXt writer: now outputs raw "tex" blocks as well as "context".
(Closes#969).
RST writer: uses ".. raw:: latex" for "tex" content.
(RST doesn't support raw context anyway.)
Note that if "context" or "latex" specifically is desired,
you can still force that in a markdown document by using
the raw attribute (see MANUAL.txt):
```{=latex}
\foo
```
Note that this change may affect some filters, if they assume that raw
tex parsed by the Markdown reader will be RawBlock (Format "latex").
In most cases it should be trivial to modify the filters to accept
"tex" as well.
For example: `\def\foo#1[#2]{#1 and #2}`.
Closes#4768. Also fixes#4771.
API change: in Text.Pandoc.Readers.LaTeX.Types,
new type ArgSpec added. Second parameter of Macro
constructor is now `[ArgSpec]` instead of `Int`.
This fixes a regression in 2.2.3, which cause boolean values to
be parsed as MetaInlines instead of MetaBool.
Note also an undocumented (but desirable) change in 2.2.3:
numbers are now parsed as MetaInlines rather than MetaString.
Closes#4819.
* Use a Span with class "title-reference" for the default
title-reference role.
* Use B.text to split up contents into Spaces, SoftBreaks, and Strs
for title-reference.
* Use Code with class "interpreted-text" instead of Span and Str for
unknown roles. (The RST writer has also been modified to round-trip
this properly.)
* Disallow blank lines in interpreted text.
* Backslash-escape now works in interpreted text.
* Backticks followed by alphanumerics no longer end interpreted text.
Closes#4811.
Closes#4803
After this commit use `$titleblock$` in order to get what was contained
in `$title$` before, that is a title and subtitle rendered according to
the official rST method:
http://docutils.sourceforge.net/docs/user/rst/quickstart.html#document-title-subtitle. from
With this commit, the `$title$` and `$subtitle$` metadata are available and they
simply carry the metadata values. This opens up more possibilities in templates.
Now we properly parse title and subtitle elements that are
direct children of book and article (as well as children of
bookinfo, articleinfo, or info).
We also now use the "subtitle" metadata field for subtitles,
rather than tacking the subtitle on to the title.
Exposes a function converting which flattenes a list of blocks into a
list of inlines. An example use case would be the conversion of Note
elements into other inlines.
RST does not allow nested emphasis, links, or other inline
constructs.
Closes#4581, double parsing of links with URLs as
link text. This supersedes the earlier fix for #4581
in 6419819b46.
Fixes#4561, a bug parsing with URLs inside emphasis.
Closes#4792.
Emphasis was not parsed when it followed directly after some block types
(e.g., lists).
The org reader uses a wrapper for the `parseFromString` function to
handle org-specific state. The last position of a character allowed
before emphasis was reset incorrectly in this wrapper. Emphasized text
was not recognized when placed directly behind a block which the reader
parses using `parseFromString`.
Fixes: #4784
Starting in 2.2.2, everything after an `\input` (or `\include`)
in a markdown file would be parsed as raw LaTeX.
This commit fixes the issue and adds a regression test.
Closes#4781.
Text.Pandoc.Emoji now exports `emojiToInline`, which returns a Span inline containing the emoji character and some attributes with metadata (class `emoji`, attribute `data-emoji` with emoji name). Previously, emojis (as supported in Markdown and CommonMark readers, e.g "😄")
were simply translated into the corresponding unicode code point. By wrapping them in Span
nodes, we make it possible to do special handling such as giving them a special font
in HTML output. We also open up the possibility of treating them differently when the
`--ascii` option is selected (though that is not part of this commit).
Closes#4743.
Non-ascii characters were not stripped from identifiers even if the
`ascii_identifiers` extension was enabled (which is is by default for
gfm).
Closes#4742
Previously it was assumed that the system would have `/bin/false`
and `/bin/sed`, and these tests were skipped otherwise.
On MacOS, these utilities are located in `/usr/bin`.
Fixed by just using `sed` and `false` -- these should always be
in the path. Removed the "skipping" behavior, replaced with a
check for Windows. On Windowns, we use `echo` and `cd`, which
should always exist.
Not yet checked on Windows.
Removed `--latexmathml`, `--gladtex`, `--mimetex`, `--jsmath`, `-m`,
`--asciimathml` options.
Removed `JsMath`, `LaTeXMathML`, and `GladTeX` constructors from
`Text.Pandoc.Options.HTMLMathMethod` [API change].
Removed unneeded data file LaTeXMathML.js and updated tests.
Bumped version to 2.2.
* Markdown writer now includes a blank line at the end
of the row in a single-row multiline table, to prevent it from being
interpreted as a simple table. Closes#4578.
* Markdown reader does a better job computing the relative width of
the last column in a multiline table, so we can round-trip tables
without constantly shrinking the last column.
Previously we used an odd mix of 3- and 4-space indentation.
Now we use 3-space indentation, except for ordered lists,
where indentation must depend on the width of the list marker.
Closes#4563.
* Use `\f[R]` rather than `\f[]` to reset. The latter
returns to the previous font, which gives unintended
results in some cases.
* Use `\f[BI]` and `\f[CB]` in headers, instead of `\f[I]` and `\f[C]`,
since the header font is automatically bold.
* Use `\f[CB]` rather than `\f[BC]` for monospace bold.
Closes#4552.
HorizontalRule corresponds to <hr> element in the default output
format, HTML. Current HTML standard defines <hr> element as
"paragraph-level thematic break". In typography it is often
represented by extra space or centered asterism ("⁂"), but since
FB2 does not support text centering, empty line (similar to extra space)
is the only solution.
Line breaks, on the other hand, don't generate <empty-line />
anymore. Previously line breaks generated <empty-line /> element
inside paragraph, which is not allowed. So, this commit addresses
issue #2424 ("FB2 produced by pandoc doesn't validate").
FB2 does not have a way to represent line breaks inside paragraphs.
They are replaced with LF character, which is not rendered by
FB2 readers, but at least preserves some information.
Only <p> and <empty-line /> are allowed in titles,
but <p> has the same type as an ordinary paragraphs.
Therefore, there is no need to remove emphasis from titles.
Also, don't intersperse paragraph with empty lines.
Tests speaker notes appearing after (and inside of) separating blocks.
Output checked on Windows10 (archlinux virtualbox), PowerPoint
2013. Not corrupted, and output as expected.
* Annotate gridTable code with comments and abstract small functions
* Don't wrap lines in tables when `--wrap=none`. Instead, expand cells, even if
it results in cells that don't respect relative widths or surpass page column width.
* This change affects RST, Markdown, and Haddock writers.
* Provide `bidi` package's option using `\PassOptionsToPackage`. This avoid clash when `polyglossia` loads it first and then it is loaded again for XeLaTeX when `latex-dir-rtl` defined.
* Adjust test suite for LaTeX template change.
In the reader, we use the new Table type in Haddock.
Note that tables with col/rowspans will not translate
well into Pandoc.
In the writer, we now render tables always as grid tables,
since Haddock supports these.
and not in the title. If it's in the title, then we get
a titlebar on slides with the `plain` attribute, when
the id is non-null. This fixes a regression from 1.9.x.
Closes#4307.
This patch fixes some cases where the JATS writer was introducing
semantically significant whitespace by indenting and wrapping tags.
Note that the JATS spec has a content model for `<p>` tags of `(#PCDATA | ...`.
Any tag where `#PCDATA` children are possible should not have any
indentation. The same is true for `<th>`, `<td>`, `<term>`, `<label>`.
Amusewiki disables <literal> tags for security reasons.
If user wants similar behavior in pandoc, RawBlocks and RawInlines
can be removed or replaced with filters.
These were never added when the tests were first created.
Output files checked in MS PowerPoint 2013 (Windows 10, VBox). No
corruption, and output as expected.
Make sure there are no empty slides in the pptx output. Because of the
way that slides were split, these could be accidentally produced by
comments after images.
When animations are added, there will be a way to add an empty slide
with either incremental lists or pauses.
Test outputs checked with MS PowerPoint (Office 2013, Windows 10,
VBox). Both files have expected output and are not corrupted.
The name of the Lua script which is executed is made available in the
global Lua variable `PANDOC_SCRIPT_FILE`, both for Lua filters and
custom writers.
Closes: #4393
It makes more sense not to interpret -- otherwise using the original
document as the reference-doc would produce two of everything: the
interpreted version and the uninterpreted style version.
This commit adjusts the test cases for the Docx writer after the fix of #3930.
- Adjusted test cases with inline images. The inline images now have the correct sizing, title and description.
- Modified the test case to include an image multiple times with different sizing each time.
- Tested on Windows 8.1 with Word 2007 (12.0.6705.5000) The files are not corrupted and display exactly what is expected.
Previously Emph, Strong, etc were outside the custom-style span. This
moves them inside in order to make it easier to write filters that act
on the formatting in these contents.
Tests and MANUAL example are changed to match.
The previous commit had a bug where custom-style spans would be read
with every recurrsion. This fixes that, and changes the example given
in the manual.
The characters allowed before and after emphasis can be configured via
`#+pandoc-emphasis-pre` and `#+pandoc-emphasis-post`, respectively. This
allows to change which strings are recognized as emphasized text on a
per-document or even per-paragraph basis. The allowed characters must be
given as (Haskell) string.
#+pandoc-emphasis-pre: "-\t ('\"{"
#+pandoc-emphasis-post: "-\t\n .,:!?;'\")}["
If the argument cannot be read as a string, the default value is
restored.
Closes: #4378
Modify the PowerPoint tests to run all the tests with
template (--reference-doc) as well. Because there are so many
interlocking pieces, bugs can pop up in weird places when using
templates, since it changes how the writer builds its output
file.
For example, I recently discovered a bug in which speaker notes worked
fine and templating worked fine elsewhere, but templating with speaker
notes produced a file that would crash MS PowerPoint. That particular
bug was fixed, but this will forces us to check for that with each new
change.
Since the template changed, some small elements of these test files
changed as well.
All of these were checked with Powerpoint 2013 on Windows
10 (VirtualBox). All had expected outcomes and no corruption.
Lists are parsed in linear instead of exponential time now.
Contents of block tags, such as <quote>, is parsed directly,
without storing it in a string and parsing with parseFromString.
Fixed a bug: headers did not terminate lists.
Muse allows indentation to indicate quotation or alignment,
but only on the top level, not within a <quote> or list.
This patch also simplifies the code by removing museInQuote
and museInList fields from the state structure.
Headers and indented paragraphs are attempted to be parsed
only at the topmost level, instead of aborting parsing with guards.
Text::Amuse already explicitly requires it anyway.
Supporting block tags on the same line as contents makes
it hard to combine closing tag parsers with indentation parsers.
Being able to combine parsers is required for no-reparsing refactoring
of Muse reader.
These are based off the reader tests, with some removed (where the
reader output was identical, based on different docx inputs). There
are still more to be added. In particular, tests for custom-styles
need to be added.
All golden docx files have been checked in MS Word
2013 (windows). There is no corruption.
There is questionable output in the `tables` test: the three tables
seemed to be joined. This will be addressed in a future commit, and
the golden docx file will be changed.
There is very little pptx-specific in these tests, so we abstract out
the basic testing function so it can be used for docx as well. This
should allow us to catch some errors in the docx writer that slipped
by the roundtrip testing.
Fixes#2609.
This PR introduces the new-style section headings: `\section[my-header]{My Header}` -> `\section[title={My Header},reference={my-header}]`.
On top of this, the ConTeXt writer now supports the `--section-divs` option to write sections in the fenced style, with `\startsection` and `\stopsection`.
We had previously re-read the native file and converted it to
Powerpoint. But we have already done that in constructing the test
archive. So now we just convert the archive back to a bytestring and
write it to disk.
Previously we had tested certain properties of the output PowerPoint
slides. Corruption, though, comes as the result of a numebr of
interrelated issues in the output pptx archive. This is a new
approach, which compares the output of the Powerpoint writer with
files that we know to (a) not be corrupt, and (b) to show the desired
output behavior (details below). This commit introduces three tests
using the new framework. More will follow.
The test procedure: given a native file and a pptx file, we generate a
pptx archive from the native file, and then test:
1. Whether the same files are in the two archives
2. Whether each of the contained xml files is the same. (We skip time
entries in `docProps/core.xml`, since these are derived from IO. We
just check to make sure that they're there in the same way in both
files.)
3. Whether each of the media files is the same.
Note that steps 2 and 3, though they compare multiple files, are one
test each, since the number of files depends on the input file (if
there is a failure, it will only report the first failed file
comparison in the test failure).
Closes#4281.
Previously we allowed "nonindent spaces" before the
opening and closing `:::`, but this interfered with
list parsing, so now we require the fences to be
flush with the margin of the containing block.
Now list item contents is parsed as blocks,
without resorting to parseFromString.
Only the first line of paragraph has to
be indented now, just like in Emacs Muse
and Text::Amuse.
Definition lists are not refactored yet.
See also: issue #3865.
Previously we could get ever-lengthening cell widths
when a table was run repeatedly through `pandoc -f markdown -t
markdown`. This patch stabilizes the relative cell
widths. Closes#4265.
This is difficult to recreate with a modern version of Word, so I'm
using the file submitted with the bug report. It would be preferable
to find a smaller example with Latin characters, though, so as not to
confuse the issue being tested.
rst2latex.py uses an align* environment for math in
`.. math::` blocks, so this math may contain line breaks.
If it does, we put the math in an `aligned` environment
to simulate rst2latex.py's behavior.
Closes#4254.
The change both improves performance and fixes a
regression whereby normal citations inside inline notes
were not parsed correctly.
Closesjgm/pandoc-citeproc#315.
even if the `latex_macros` extension is set.
This reverts to earlier behavior and is probably safer
on the whole, since some macros only modify things in
included packages, which pandoc's macro expansion can't
modify.
Closes#4246.
Elements with attributes got an additional `attr` accessor. Attributes
were accessible only via the `identifier`, `classes`, and `attributes`,
which was in conflict with the documentation, which indirectly states
that such elements have the an `attr` property.
Every constructor which accepts a list of blocks now also accepts a
single block element for convenience. Furthermore, strings are accepted as
shorthand for `{pandoc.Str "text"}` in constructors.
We had previously defaulted to slideLevel 2. Now we use the correct
behavior of defaulting to the highest level header followed by
content. We change an expected test result to match this behavior.
There isn't any reason to have numberous anchors in the same place,
since we can't maintain docx's non-nesting overlapping. So we reduce
to a single anchor, and have all links pointing to one of the
overlapping anchors point to that one. This changes the behavior from
commit e90c714c7 slightly (use the first anchor instead of the last)
so we change the expected test result.
Note that because this produces a state that has to be set after every
invocation of `parPartToInlines`, we make the main function into a
primed subfunction `parPartToInlines'`, and make `parPartToInlines` a
wrapper around that.
Previously we had only read the first child of an sdtContents tag. Now
we replace sdt with all children of the sdtContents tag.
This changes the expected test result of our nested_anchors test,
since now we read docx's generated TOCs.
This gives a pure way to insert an ersatz file into a FileTree.
In addition, we normalize paths both on insertion and on
lookup, so that "foo" and "./foo" will be judged equivalent.
This is the beginning of a test suite for the powerpoint
writer. Initial tests are for the number of slides.
Note that at the moment it does not test against corruption in
Microsoft PowerPoint; it just tests that certain outcomes work as
expected. More tests will be added.
This test framework uses the PandocPure monad introduced with Pandoc 2.0.
The level of headers in included files can be shifted to a higher level
by specifying a minimum header level via the `:minlevel` parameter. E.g.
`#+include: "tour.org" :minlevel 1` will shift the headers in tour.org
such that the topmost headers become level 1 headers.
Fixes: #4154
The org reader test file had grown large, to the point that editor
performance was negatively affected in some cases. The tests are spread
over multiple submodules, and re-combined into a tasty TestTree in the
main org reader test file.
instead of relying on CSS. Closes#4162. HTML-based templates have had
the custom CSS for div.line-block removed. Those maintaining custom
templates will want to remove this too.
We still enclose line blocks in a div with class line-block.
