Removed `--latexmathml`, `--gladtex`, `--mimetex`, `--jsmath`, `-m`,
`--asciimathml` options.
Removed `JsMath`, `LaTeXMathML`, and `GladTeX` constructors from
`Text.Pandoc.Options.HTMLMathMethod` [API change].
Removed unneeded data file LaTeXMathML.js and updated tests.
Bumped version to 2.2.
Previously we used an odd mix of 3- and 4-space indentation.
Now we use 3-space indentation, except for ordered lists,
where indentation must depend on the width of the list marker.
Closes#4563.
HorizontalRule corresponds to <hr> element in the default output
format, HTML. Current HTML standard defines <hr> element as
"paragraph-level thematic break". In typography it is often
represented by extra space or centered asterism ("⁂"), but since
FB2 does not support text centering, empty line (similar to extra space)
is the only solution.
Line breaks, on the other hand, don't generate <empty-line />
anymore. Previously line breaks generated <empty-line /> element
inside paragraph, which is not allowed. So, this commit addresses
issue #2424 ("FB2 produced by pandoc doesn't validate").
FB2 does not have a way to represent line breaks inside paragraphs.
They are replaced with LF character, which is not rendered by
FB2 readers, but at least preserves some information.
Tests speaker notes appearing after (and inside of) separating blocks.
Output checked on Windows10 (archlinux virtualbox), PowerPoint
2013. Not corrupted, and output as expected.
This patch fixes some cases where the JATS writer was introducing
semantically significant whitespace by indenting and wrapping tags.
Note that the JATS spec has a content model for `<p>` tags of `(#PCDATA | ...`.
Any tag where `#PCDATA` children are possible should not have any
indentation. The same is true for `<th>`, `<td>`, `<term>`, `<label>`.
Amusewiki disables <literal> tags for security reasons.
If user wants similar behavior in pandoc, RawBlocks and RawInlines
can be removed or replaced with filters.
These were never added when the tests were first created.
Output files checked in MS PowerPoint 2013 (Windows 10, VBox). No
corruption, and output as expected.
Make sure there are no empty slides in the pptx output. Because of the
way that slides were split, these could be accidentally produced by
comments after images.
When animations are added, there will be a way to add an empty slide
with either incremental lists or pauses.
Test outputs checked with MS PowerPoint (Office 2013, Windows 10,
VBox). Both files have expected output and are not corrupted.
The name of the Lua script which is executed is made available in the
global Lua variable `PANDOC_SCRIPT_FILE`, both for Lua filters and
custom writers.
Closes: #4393
The characters allowed before and after emphasis can be configured via
`#+pandoc-emphasis-pre` and `#+pandoc-emphasis-post`, respectively. This
allows to change which strings are recognized as emphasized text on a
per-document or even per-paragraph basis. The allowed characters must be
given as (Haskell) string.
#+pandoc-emphasis-pre: "-\t ('\"{"
#+pandoc-emphasis-post: "-\t\n .,:!?;'\")}["
If the argument cannot be read as a string, the default value is
restored.
Closes: #4378
Modify the PowerPoint tests to run all the tests with
template (--reference-doc) as well. Because there are so many
interlocking pieces, bugs can pop up in weird places when using
templates, since it changes how the writer builds its output
file.
For example, I recently discovered a bug in which speaker notes worked
fine and templating worked fine elsewhere, but templating with speaker
notes produced a file that would crash MS PowerPoint. That particular
bug was fixed, but this will forces us to check for that with each new
change.
Lists are parsed in linear instead of exponential time now.
Contents of block tags, such as <quote>, is parsed directly,
without storing it in a string and parsing with parseFromString.
Fixed a bug: headers did not terminate lists.
Muse allows indentation to indicate quotation or alignment,
but only on the top level, not within a <quote> or list.
This patch also simplifies the code by removing museInQuote
and museInList fields from the state structure.
Headers and indented paragraphs are attempted to be parsed
only at the topmost level, instead of aborting parsing with guards.
Text::Amuse already explicitly requires it anyway.
Supporting block tags on the same line as contents makes
it hard to combine closing tag parsers with indentation parsers.
Being able to combine parsers is required for no-reparsing refactoring
of Muse reader.
These are based off the reader tests, with some removed (where the
reader output was identical, based on different docx inputs). There
are still more to be added. In particular, tests for custom-styles
need to be added.
All golden docx files have been checked in MS Word
2013 (windows). There is no corruption.
There is questionable output in the `tables` test: the three tables
seemed to be joined. This will be addressed in a future commit, and
the golden docx file will be changed.
There is very little pptx-specific in these tests, so we abstract out
the basic testing function so it can be used for docx as well. This
should allow us to catch some errors in the docx writer that slipped
by the roundtrip testing.
Fixes#2609.
This PR introduces the new-style section headings: `\section[my-header]{My Header}` -> `\section[title={My Header},reference={my-header}]`.
On top of this, the ConTeXt writer now supports the `--section-divs` option to write sections in the fenced style, with `\startsection` and `\stopsection`.
We had previously re-read the native file and converted it to
Powerpoint. But we have already done that in constructing the test
archive. So now we just convert the archive back to a bytestring and
write it to disk.
Previously we had tested certain properties of the output PowerPoint
slides. Corruption, though, comes as the result of a numebr of
interrelated issues in the output pptx archive. This is a new
approach, which compares the output of the Powerpoint writer with
files that we know to (a) not be corrupt, and (b) to show the desired
output behavior (details below). This commit introduces three tests
using the new framework. More will follow.
The test procedure: given a native file and a pptx file, we generate a
pptx archive from the native file, and then test:
1. Whether the same files are in the two archives
2. Whether each of the contained xml files is the same. (We skip time
entries in `docProps/core.xml`, since these are derived from IO. We
just check to make sure that they're there in the same way in both
files.)
3. Whether each of the media files is the same.
Note that steps 2 and 3, though they compare multiple files, are one
test each, since the number of files depends on the input file (if
there is a failure, it will only report the first failed file
comparison in the test failure).
Now list item contents is parsed as blocks,
without resorting to parseFromString.
Only the first line of paragraph has to
be indented now, just like in Emacs Muse
and Text::Amuse.
Definition lists are not refactored yet.
See also: issue #3865.
This is difficult to recreate with a modern version of Word, so I'm
using the file submitted with the bug report. It would be preferable
to find a smaller example with Latin characters, though, so as not to
confuse the issue being tested.
The change both improves performance and fixes a
regression whereby normal citations inside inline notes
were not parsed correctly.
Closesjgm/pandoc-citeproc#315.
Elements with attributes got an additional `attr` accessor. Attributes
were accessible only via the `identifier`, `classes`, and `attributes`,
which was in conflict with the documentation, which indirectly states
that such elements have the an `attr` property.
Every constructor which accepts a list of blocks now also accepts a
single block element for convenience. Furthermore, strings are accepted as
shorthand for `{pandoc.Str "text"}` in constructors.
We had previously defaulted to slideLevel 2. Now we use the correct
behavior of defaulting to the highest level header followed by
content. We change an expected test result to match this behavior.
This gives a pure way to insert an ersatz file into a FileTree.
In addition, we normalize paths both on insertion and on
lookup, so that "foo" and "./foo" will be judged equivalent.
This is the beginning of a test suite for the powerpoint
writer. Initial tests are for the number of slides.
Note that at the moment it does not test against corruption in
Microsoft PowerPoint; it just tests that certain outcomes work as
expected. More tests will be added.
This test framework uses the PandocPure monad introduced with Pandoc 2.0.
The level of headers in included files can be shifted to a higher level
by specifying a minimum header level via the `:minlevel` parameter. E.g.
`#+include: "tour.org" :minlevel 1` will shift the headers in tour.org
such that the topmost headers become level 1 headers.
Fixes: #4154
The org reader test file had grown large, to the point that editor
performance was negatively affected in some cases. The tests are spread
over multiple submodules, and re-combined into a tasty TestTree in the
main org reader test file.
The same init file (`data/init`) that is used to setup the Lua
interpreter for Lua filters is also used to setup the interpreter of
custom writers.lua.
Support writing <fig> and <table-wrap> elements with <title> and
<caption> inside them by using Divs with class set to on of
fig, table-wrap or cation. The title is included as a Heading
so the constraint on where Heading can occur is also relaxed.
Also leaves out empty alt attributes on links.
The integration with Lua's package/module system is improved: A
pandoc-specific package searcher is prepended to the searchers in
`package.searchers`. The modules `pandoc` and `pandoc.mediabag` can now
be loaded via `require`.
* Deprecate `--strip-empty-paragraphs` option. Instead we now
use an `empty_paragraphs` extension that can be enabled on
the reader or writer. By default, disabled.
* Add `Ext_empty_paragraphs` constructor to `Extension`.
* Revert "Docx reader: don't strip out empty paragraphs."
This reverts commit d6c58eb836.
* Implement `empty_paragraphs` extension in docx reader and writer,
opendocument writer, html reader and writer.
* Add tests for `empty_paragraphs` extension.
We now have the `--strip-empty-paragraphs` option for that,
if you want it. Closes#2252.
Updated docx reader tests.
We use stripEmptyParagraphs to avoid changing too
many tests. We should add new tests for empty paragraphs.
Attribute lists are represented as associative lists in Lua. Pure
associative lists are awkward to work with. A metatable is attached to
attribute lists, allowing to access and use the associative list as if
the attributes were stored in as normal key-value pair in table.
Note that this changes the way `pairs` works on attribute lists. Instead
of producing integer keys and two-element tables, the resulting iterator
function now returns the key and value of those pairs. Use `ipairs` to
get the old behavior.
Warning: the new iteration mechanism only works if pandoc has been
compiled with Lua 5.2 or later (current default: 5.3).
The `pandoc.Attr` function is altered to allow passing attributes as
key-values in a normal table. This is more convenient than having to
construct the associative list which is used internally.
Closes#4071
The `text` module is preloaded in lua. The module contains some UTF-8
aware string functions, implemented in Haskell. The module is loaded on
request only, e.g.:
text = require 'text'
function Str (s)
s.text = text.upper(s.text)
return s
end
<annotation> is not allowed inside <body> according to FictionBook2 XML schema. Besides that, the same information is already placed inside <description>.
Related bug: #2424
* Basic skeleton for creole reader.
No real functionality besides preliminary bold and italics yet.
* Creole: add support for bold/italic with implicit end at paragraph end.
* Creole: add support for headings.
* Creole: add support for tilde escaped chars.
* Basic skeleton for creole reader.
No real functionality besides preliminary bold and italics yet.
* Creole: add support for bold/italic with implicit end at paragraph end.
* Creole: add support for headings.
* Creole: add support for tilde escaped chars.
* Add a test suite for the creole parser
So far this covers only things the parser already supports.
* Added simple parsing of flat unordered lists.
* Added tests for unordered lists in creole.
* First, wrong(!) implementation of sublists.
Fails test, as sublists should not be embedded in a list item!
* Implementation of unordered sublists.
* Added support for ordered lists to creole reader.
* Added utility function to append parsers to Creole reader.
* Creole reader: Fixed list item end detection in sub lists.
* Tests for creole reader: added more tests for lists.
Covering ordered and unordered tests, even mixed. Tests for
formatting in list items still missing...
* Added "nowiki" blocks. One exception rule is missing...
* Creole reader: nowiki: implemented exception for curly brackets.
* Creole reader: added inline nowiki.
* Creole reader: added horizontalRule.
* Creole reader: added auto linking of URIs.
* Creole reader: detect horizontalRule as para end.
Used the opportunity for a little refactoring.
* Creole reader: added forced line breaks.
Including test.
* Creole reader: implement wiki links.
* Creole reader: added image support.
* Creole reader: support images as links.
* Creole reader: implemented placeholder -- by simply dropping them.
* Creole reader: added tests for links.
After observing a regression, it was really time... ;-)
* Creole reader: fixed links with names.
* Creole reader: allow space after first of enclosing tags.
Space after the start of formatting tags are allowed with creole,
e.g. "there is // italic text // in here" is legal.
This problem was discovered using the creole1.0test.txt document from
http://www.wikicreole.org/wiki/Creole1.0TestCases
See l.57:
# // italic item 3 //
* Creole reader: fixed links without names.
* Creole reader: Tests, sorted into groups.
* Creole reader: implemented tables.
* Removed redundant import.
* Creole reader: add correct escaping of links.
* Creole reader: allow handling of e.g. links in parenthesis and quotes.
* Creole reader: Modified disclaimer as most of the code is actually by me.
* Creole reader: Tests: added escaped links.
* Creole reader: preserve leading and trailing space in bold/italic.
* Creole reader: detect tables without a leading blank line.
* Creole Reader: added official creole1.0test.txt as "old" test.
The base document was downloaded from
http://www.wikicreole.org/wiki/Creole1.0TestCases.
The Wiki, and therefore the test document is
Copyright (C) by the contributors.
Some rights reserved, license CC BY-SA.
http://creativecommons.org/licenses/by-sa/1.0/
* Added underlineSpan builder function. This can be easily updated if needed. The purpose is for Readers to transform underlines consistently.
* Docx Reader: Use underlineSpan and update test
* Org Reader: Use underlineSpan and add test
* Textile Reader: Use underlineSpan and add test case
* Txt2Tags Reader: Use underlineSpan and update test
* HTML Reader: Use underlineSpan and add test case
Removed `writerSourceURL` from `WriterOptions` (API change).
Added `stSourceURL` to `CommonState`.
It is set automatically by `setInputFiles`.
Text.Pandoc.Class now exports `setInputFiles`, `setOutputFile`.
The type of `getInputFiles` has changed; it now returns `[FilePath]`
instead of `Maybe [FilePath]`.
Functions in Class that formerly took the source URL as a parameter
now have one fewer parameter (`fetchItem`, `downloadOrRead`,
`setMediaResource`, `fillMediaBag`).
Removed `WriterOptions` parameter from `makeSelfContained` in
`SelfContained`.
The org reader was updated to match current org-mode behavior: the set
of characters which are acceptable to occur as the first or last
character in an org emphasis have been changed and now allows all
non-whitespace chars at the inner border of emphasized text (see
`org-emphasis-regexp-components`).
Fixes: #3933
Previously pandoc would sometimes combine two line blocks separated by blanks, and ignore trailing blank lines within the line block.
Test is checked to be consisted with http://rst.ninjs.org/
This change makes it possible to define a catch-all function using lua's
metatable lookup functionality.
function catch_all(el)
…
end
return {
setmetatable({}, {__index = function(_) return catch_all end})
}
A further effect of this change is that the map with filter functions
now only contains functions corresponding to AST element constructors.
Previously we just tacked on a directory to the command
line, but that didn't work when we e.g. used a pipe for round tripping,
with two invocations of pandoc.
Added TikiWiki reader, including tests and documentation.
It's probably not *complete*, but it works pretty well, handles all
the basics (and some not-so-basics).
This rewrite is primarily motivated by the need to
get macros working properly. A side benefit is that the
reader is significantly faster (27s -> 19s in one
benchmark, and there is a lot of room for further
optimization).
We now tokenize the input text, then parse the token stream.
Macros modify the token stream, so they should now be effective
in any context, including math. Thus, we no longer need the clunky
macro processing capacities of texmath.
A custom state LaTeXState is used instead of ParserState.
This, plus the tokenization, will require some rewriting
of the exported functions rawLaTeXInline, inlineCommand,
rawLaTeXBlock.
* Added Text.Pandoc.Readers.LaTeX.Types (new exported module).
Exports Macro, Tok, TokType, Line, Column. [API change]
* Text.Pandoc.Parsing: adjusted type of `insertIncludedFile`
so it can be used with token parser.
* Removed old texmath macro stuff from Parsing.
Use Macro from Text.Pandoc.Readers.LaTeX.Types instead.
* Removed texmath macro material from Markdown reader.
* Changed types for Text.Pandoc.Readers.LaTeX's
rawLaTeXInline and rawLaTeXBlock. (Both now return a String,
and they are polymorphic in state.)
* Added orgMacros field to OrgState. [API change]
* Removed readerApplyMacros from ReaderOptions.
Now we just check the `latex_macros` reader extension.
* Allow `\newcommand\foo{blah}` without braces.
Fixes#1390.
Fixes#2118.
Fixes#3236.
Fixes#3779.
Fixes#934.
Fixes#982.
in Text.Pandoc.Lua. Also to pushPandocModule.
This change allows users to override pandoc.lua with a file
in their local data directory, adding custom functions, etc.
@tarleb, if you think this is a bad idea, you can revert this.
But in general our data files are all overridable.
* New module Text.Pandoc.Readers.Vimwiki, exporting readVimwiki [API change].
* New input format `vimwiki`.
* New data file, `data/vimwiki.css`, for displaying the HTML produced by this reader and pandoc's HTML writer in the style of vimwiki's own HTML export.
* XML.toEntities: changed type to Text -> Text.
* Shared.tabFilter -- fixed so it strips out CRs as before.
* Modified writers to take Text.
* Updated tests, benchmarks, trypandoc.
[API change]
Closes#3731.
