Now list item contents is parsed as blocks,
without resorting to parseFromString.
Only the first line of paragraph has to
be indented now, just like in Emacs Muse
and Text::Amuse.
Definition lists are not refactored yet.
See also: issue #3865.
Previously we could get ever-lengthening cell widths
when a table was run repeatedly through `pandoc -f markdown -t
markdown`. This patch stabilizes the relative cell
widths. Closes#4265.
This is difficult to recreate with a modern version of Word, so I'm
using the file submitted with the bug report. It would be preferable
to find a smaller example with Latin characters, though, so as not to
confuse the issue being tested.
rst2latex.py uses an align* environment for math in
`.. math::` blocks, so this math may contain line breaks.
If it does, we put the math in an `aligned` environment
to simulate rst2latex.py's behavior.
Closes#4254.
The change both improves performance and fixes a
regression whereby normal citations inside inline notes
were not parsed correctly.
Closesjgm/pandoc-citeproc#315.
even if the `latex_macros` extension is set.
This reverts to earlier behavior and is probably safer
on the whole, since some macros only modify things in
included packages, which pandoc's macro expansion can't
modify.
Closes#4246.
Elements with attributes got an additional `attr` accessor. Attributes
were accessible only via the `identifier`, `classes`, and `attributes`,
which was in conflict with the documentation, which indirectly states
that such elements have the an `attr` property.
Every constructor which accepts a list of blocks now also accepts a
single block element for convenience. Furthermore, strings are accepted as
shorthand for `{pandoc.Str "text"}` in constructors.
We had previously defaulted to slideLevel 2. Now we use the correct
behavior of defaulting to the highest level header followed by
content. We change an expected test result to match this behavior.
There isn't any reason to have numberous anchors in the same place,
since we can't maintain docx's non-nesting overlapping. So we reduce
to a single anchor, and have all links pointing to one of the
overlapping anchors point to that one. This changes the behavior from
commit e90c714c7 slightly (use the first anchor instead of the last)
so we change the expected test result.
Note that because this produces a state that has to be set after every
invocation of `parPartToInlines`, we make the main function into a
primed subfunction `parPartToInlines'`, and make `parPartToInlines` a
wrapper around that.
Previously we had only read the first child of an sdtContents tag. Now
we replace sdt with all children of the sdtContents tag.
This changes the expected test result of our nested_anchors test,
since now we read docx's generated TOCs.
This gives a pure way to insert an ersatz file into a FileTree.
In addition, we normalize paths both on insertion and on
lookup, so that "foo" and "./foo" will be judged equivalent.
This is the beginning of a test suite for the powerpoint
writer. Initial tests are for the number of slides.
Note that at the moment it does not test against corruption in
Microsoft PowerPoint; it just tests that certain outcomes work as
expected. More tests will be added.
This test framework uses the PandocPure monad introduced with Pandoc 2.0.
The level of headers in included files can be shifted to a higher level
by specifying a minimum header level via the `:minlevel` parameter. E.g.
`#+include: "tour.org" :minlevel 1` will shift the headers in tour.org
such that the topmost headers become level 1 headers.
Fixes: #4154
The org reader test file had grown large, to the point that editor
performance was negatively affected in some cases. The tests are spread
over multiple submodules, and re-combined into a tasty TestTree in the
main org reader test file.
instead of relying on CSS. Closes#4162. HTML-based templates have had
the custom CSS for div.line-block removed. Those maintaining custom
templates will want to remove this too.
We still enclose line blocks in a div with class line-block.
Don't pass through macro definitions themselves when `latex_macros`
is set. The macros have already been applied.
If `latex_macros` is enabled, then `rawLaTeXBlock` in
Text.Pandoc.Readers.LaTeX will succeed in parsing a macro definition,
and will update pandoc's internal macro map accordingly, but the
empty string will be returned.
Together with earlier changes, this closes#4179.
Previously we erroneously included the enclosing
backticks in a reference ID (closes#4156).
This change also disables interpretation of
syntax inside references, as in docutils.
So, there is no emphasis in
`my *link*`_
The same init file (`data/init`) that is used to setup the Lua
interpreter for Lua filters is also used to setup the interpreter of
custom writers.lua.
Previously we computed the column sizes based on the ratio
between the header lines and the text width (as set by `--columns`).
This meant that tables with very short header lines would be
very narrow. With this change, pipe tables with wrapping cells will
always take up the whole text width. The relative column widths
will still be determined by the ratio of header lines, but they
will be normalized to add up to 1.0.
Support writing <fig> and <table-wrap> elements with <title> and
<caption> inside them by using Divs with class set to on of
fig, table-wrap or cation. The title is included as a Heading
so the constraint on where Heading can occur is also relaxed.
Also leaves out empty alt attributes on links.
This fixes a bug in 2.0.4, whereby pandoc could not
read the theme files generated with `--print-highlight-style`.
It also fixes some CSS issues involving line numbers.
Highlighted code blocks are now enclosed in a div with class
sourceCode.
Highlighting CSS no longer sets a generic color for pre
and code; we only set these for class `sourceCode`.
This will close#4133 and #4128.
The integration with Lua's package/module system is improved: A
pandoc-specific package searcher is prepended to the searchers in
`package.searchers`. The modules `pandoc` and `pandoc.mediabag` can now
be loaded via `require`.
* Deprecate `--strip-empty-paragraphs` option. Instead we now
use an `empty_paragraphs` extension that can be enabled on
the reader or writer. By default, disabled.
* Add `Ext_empty_paragraphs` constructor to `Extension`.
* Revert "Docx reader: don't strip out empty paragraphs."
This reverts commit d6c58eb836.
* Implement `empty_paragraphs` extension in docx reader and writer,
opendocument writer, html reader and writer.
* Add tests for `empty_paragraphs` extension.
We now have the `--strip-empty-paragraphs` option for that,
if you want it. Closes#2252.
Updated docx reader tests.
We use stripEmptyParagraphs to avoid changing too
many tests. We should add new tests for empty paragraphs.
Previously both needed to be specified (unless the image was
being resized to be smaller than its original size).
If height but not width is specified, we now set width to
textwidth (and similarly if width but not height is specified).
Since we have keepaspectratio, this yields the desired result.
Attribute lists are represented as associative lists in Lua. Pure
associative lists are awkward to work with. A metatable is attached to
attribute lists, allowing to access and use the associative list as if
the attributes were stored in as normal key-value pair in table.
Note that this changes the way `pairs` works on attribute lists. Instead
of producing integer keys and two-element tables, the resulting iterator
function now returns the key and value of those pairs. Use `ipairs` to
get the old behavior.
Warning: the new iteration mechanism only works if pandoc has been
compiled with Lua 5.2 or later (current default: 5.3).
The `pandoc.Attr` function is altered to allow passing attributes as
key-values in a normal table. This is more convenient than having to
construct the associative list which is used internally.
Closes#4071
The `text` module is preloaded in lua. The module contains some UTF-8
aware string functions, implemented in Haskell. The module is loaded on
request only, e.g.:
text = require 'text'
function Str (s)
s.text = text.upper(s.text)
return s
end
This fixes a bug where pandoc would stop parsing a URI with an
empty attribute: for example, `&a=&b=` wolud stop at `a`.
(The uri parser tries to guess which punctuation characters
are part of the URI and which might be punctuation after it.)
Closes#4068.
Previously we got a crash, because we were trying to print
a native cmark STRIKETHROUGH node, and the commonmark writer
in cmark-github doesn't support this. Work around this by
using a raw node to add the strikethrough delimiters.
Closes#4038.
* Move as much as possible to the CSS in the template.
* Ensure that all the HTML-based templates (including epub)
contain the CSS for columns.
* Columns default to 50% width unless they are given a width
attribute.
Closes#4028.
The line identifiers are built using the code block's identifier
as a prefix. If the code block has null identifier, we use
"cb1", "cb2", etc.
Closes#4031.
* Remove "width" attribute which is not allowed on div.
* Remove space between `<div class="column">` elements,
since this prevents columns whose widths sum to 100%
(the space takes up space).
Closes#4028.
<annotation> is not allowed inside <body> according to FictionBook2 XML schema. Besides that, the same information is already placed inside <description>.
Related bug: #2424
and other non-HTML formats (`Text.Pandoc.Readers.HTML.htmlTag`).
The parser stopped at the first `>` character, even if it wasn't
the end of the comment.
Closes#4019.
Previously bullet lists interacted in odd way with ordered lists.
For example, bullet lists nested in ordered list had incorrect
indentation. Besides that, indentation with spaces is not rendered
by FBReader and fbless. To avoid this problem, bullet lists are
indented by appending bullets to marker just the same way it is
done for ordered lists.
According to FB2 XML schema <empty-line /> cannot be placed inside
<p>. Linux FBReader can't display such paragraphs, e.g. any "loose"
lists produced by pandoc prior to this commit. Besides that,
FB2 writer placed <p> inside <p> when writing nested lists,
this commit fixes the bug.
Also this commit removes leading non-breaking space from ordered
lists for consistency with bullet lists.
Definition lists are not affected at all.
* Basic skeleton for creole reader.
No real functionality besides preliminary bold and italics yet.
* Creole: add support for bold/italic with implicit end at paragraph end.
* Creole: add support for headings.
* Creole: add support for tilde escaped chars.
* Basic skeleton for creole reader.
No real functionality besides preliminary bold and italics yet.
* Creole: add support for bold/italic with implicit end at paragraph end.
* Creole: add support for headings.
* Creole: add support for tilde escaped chars.
* Add a test suite for the creole parser
So far this covers only things the parser already supports.
* Added simple parsing of flat unordered lists.
* Added tests for unordered lists in creole.
* First, wrong(!) implementation of sublists.
Fails test, as sublists should not be embedded in a list item!
* Implementation of unordered sublists.
* Added support for ordered lists to creole reader.
* Added utility function to append parsers to Creole reader.
* Creole reader: Fixed list item end detection in sub lists.
* Tests for creole reader: added more tests for lists.
Covering ordered and unordered tests, even mixed. Tests for
formatting in list items still missing...
* Added "nowiki" blocks. One exception rule is missing...
* Creole reader: nowiki: implemented exception for curly brackets.
* Creole reader: added inline nowiki.
* Creole reader: added horizontalRule.
* Creole reader: added auto linking of URIs.
* Creole reader: detect horizontalRule as para end.
Used the opportunity for a little refactoring.
* Creole reader: added forced line breaks.
Including test.
* Creole reader: implement wiki links.
* Creole reader: added image support.
* Creole reader: support images as links.
* Creole reader: implemented placeholder -- by simply dropping them.
* Creole reader: added tests for links.
After observing a regression, it was really time... ;-)
* Creole reader: fixed links with names.
* Creole reader: allow space after first of enclosing tags.
Space after the start of formatting tags are allowed with creole,
e.g. "there is // italic text // in here" is legal.
This problem was discovered using the creole1.0test.txt document from
http://www.wikicreole.org/wiki/Creole1.0TestCases
See l.57:
# // italic item 3 //
* Creole reader: fixed links without names.
* Creole reader: Tests, sorted into groups.
* Creole reader: implemented tables.
* Removed redundant import.
* Creole reader: add correct escaping of links.
* Creole reader: allow handling of e.g. links in parenthesis and quotes.
* Creole reader: Modified disclaimer as most of the code is actually by me.
* Creole reader: Tests: added escaped links.
* Creole reader: preserve leading and trailing space in bold/italic.
* Creole reader: detect tables without a leading blank line.
* Creole Reader: added official creole1.0test.txt as "old" test.
The base document was downloaded from
http://www.wikicreole.org/wiki/Creole1.0TestCases.
The Wiki, and therefore the test document is
Copyright (C) by the contributors.
Some rights reserved, license CC BY-SA.
http://creativecommons.org/licenses/by-sa/1.0/
* Added underlineSpan builder function. This can be easily updated if needed. The purpose is for Readers to transform underlines consistently.
* Docx Reader: Use underlineSpan and update test
* Org Reader: Use underlineSpan and add test
* Textile Reader: Use underlineSpan and add test case
* Txt2Tags Reader: Use underlineSpan and update test
* HTML Reader: Use underlineSpan and add test case
This prevents the problem with extra space around highlighted
code blocks (closes#3996).
Note that we no longer put an enclosing div around highlighted
code blocks. The pre is the outer element, just as for unhighlighted
blocks.
Previously `\include` wouldn't work if the included file
contained, e.g., a begin without a matching end.
We've changed the Tok type so that it stores a full SourcePos,
rather than just a line and column. So tokens keeep track
of the file they came from. This allows us to use a simpler
method for includes, which doesn't require parsing the included
document as a whole.
Closes#3971.
Removed `writerSourceURL` from `WriterOptions` (API change).
Added `stSourceURL` to `CommonState`.
It is set automatically by `setInputFiles`.
Text.Pandoc.Class now exports `setInputFiles`, `setOutputFile`.
The type of `getInputFiles` has changed; it now returns `[FilePath]`
instead of `Maybe [FilePath]`.
Functions in Class that formerly took the source URL as a parameter
now have one fewer parameter (`fetchItem`, `downloadOrRead`,
`setMediaResource`, `fillMediaBag`).
Removed `WriterOptions` parameter from `makeSelfContained` in
`SelfContained`.
The org reader was updated to match current org-mode behavior: the set
of characters which are acceptable to occur as the first or last
character in an org emphasis have been changed and now allows all
non-whitespace chars at the inner border of emphasized text (see
`org-emphasis-regexp-components`).
Fixes: #3933
* Options: Added readerStripComments to ReaderOptions.
* Added `--strip-comments` command-line option.
* Made `htmlTag` from the HTML reader sensitive to this feature.
This affects Markdown and Textile input.
Closes#2552.
Div's are difficult to translate into org syntax, as there are multiple
div-like structures (drawers, special blocks, greater blocks) which all
have their advantages and disadvantages. Previously pandoc would
use raw HTML to preserve the full div information; this was rarely
useful and resulted in visual clutter. Div-rendering was changed to
discard the div's classes and key-value pairs if there is no natural way
to translate the div into an org structure.
Closes: #3771
Previously pandoc would sometimes combine two line blocks separated by blanks, and ignore trailing blank lines within the line block.
Test is checked to be consisted with http://rst.ninjs.org/
This change makes it possible to define a catch-all function using lua's
metatable lookup functionality.
function catch_all(el)
…
end
return {
setmetatable({}, {__index = function(_) return catch_all end})
}
A further effect of this change is that the map with filter functions
now only contains functions corresponding to AST element constructors.
Closes#3511.
Previously pandoc used the four-space rule: continuation paragraphs,
sublists, and other block level content had to be indented 4
spaces. Now the indentation required is determined by the
first line of the list item: to be included in the list item,
blocks must be indented to the level of the first non-space
content after the list marker. Exception: if are 5 or more spaces
after the list marker, then the content is interpreted as an
indented code block, and continuation paragraphs must be indented
two spaces beyond the end of the list marker. See the CommonMark
spec for more details and examples.
Documents that adhere to the four-space rule should, in most cases,
be parsed the same way by the new rules. Here are some examples
of texts that will be parsed differently:
- a
- b
will be parsed as a list item with a sublist; under the four-space
rule, it would be a list with two items.
- a
code
Here we have an indented code block under the list item, even though it
is only indented six spaces from the margin, because it is four spaces
past the point where a continuation paragraph could begin. With the
four-space rule, this would be a regular paragraph rather than a code
block.
- a
code
Here the code block will start with two spaces, whereas under
the four-space rule, it would start with `code`. With the four-space
rule, indented code under a list item always must be indented eight
spaces from the margin, while the new rules require only that it
be indented four spaces from the beginning of the first non-space
text after the list marker (here, `a`).
This change was motivated by a slew of bug reports from people
who expected lists to work differently (#3125, #2367, #2575, #2210,
#1990, #1137, #744, #172, #137, #128) and by the growing prevalance
of CommonMark (now used by GitHub, for example).
Users who want to use the old rules can select the `four_space_rule`
extension.
* Added `four_space_rule` extension.
* Added `Ext_four_space_rule` to `Extensions`.
* `Parsing` now exports `gobbleAtMostSpaces`, and the type
of `gobbleSpaces` has been changed so that a `ReaderOptions`
parameter is not needed.
Acronyms are not resolved by the reader, but acronym and glossary information is put into attributes on Spans so that they can be processed in filters.
