Commit graph

1479 commits

Author SHA1 Message Date
John MacFarlane
168b559c96 Use doctemplates 0.4.1 and citeproc 0.10. 2021-07-18 10:55:27 -07:00
John MacFarlane
cfa52e5824 Use skylighting 0.11. 2021-07-17 23:29:15 -07:00
John MacFarlane
ae22b1e977 RST reader: fix regression with code includes.
With the recent changes to include infrastructure,
included code blocks were getting an extra newline.

Closes #7436.  Added regression test.
2021-07-09 12:27:41 -07:00
John MacFarlane
7ac8fcbcac Require commonmark-pandoc >= 0.2.2.1.
This fixes task lists with multiple paragraphs.
2021-06-28 12:20:12 -07:00
John MacFarlane
0c4ed0045a Bump to 2.14.0.3, update changelog, require latest skylighting. 2021-06-20 19:25:43 -07:00
John MacFarlane
a8556ac06c Require commonmark 0.2.1. 2021-06-20 17:46:47 -07:00
John MacFarlane
67b3c36a93 Bump to 2.14.0.2, update chaneglog and manual. 2021-06-10 22:47:53 -07:00
John MacFarlane
17a2f4c49d Require citeproc 0.4.0.1.
This fixes a bug which led to doubled "et al." in some
(rare) circumstances.
2021-06-05 10:04:30 -06:00
John MacFarlane
677ccb7138 Bump to 2.14.0.1; update changelog and man page. 2021-05-31 22:07:12 -06:00
John MacFarlane
f363f00db8 Use commonmark-extensions 0.2.1.2 2021-05-29 19:20:03 -07:00
John MacFarlane
7aade73dce Replace biblatex-exmaples.bib with shorter averroes.bib in tests. 2021-05-29 12:14:37 -07:00
John MacFarlane
e72d2dd962 Use latest commonmark. 2021-05-28 13:14:56 -07:00
John MacFarlane
834da53058 Add rebase_relative_paths extension.
- Add manual entry for (non-default) extension
  `rebase_relative_paths`.
- Add constructor `Ext_rebase_relative_paths` to `Extensions`
  in Text.Pandoc.Extensions [API change]. When enabled, this
  extension rewrites relative image and link paths by prepending
  the (relative) directory of the containing file.
- Make Markdown reader sensitive to the new extension.
- Add tests for #3752.

Closes #3752.

NB. currently the extension applies to markdown and associated
readers but not commonmark/gfm.
2021-05-27 10:38:25 -07:00
Albert Krewinkel
54ab7a0a3b CI: test with GHC 9.0.1 2021-05-25 11:52:49 -07:00
Albert Krewinkel
bb2530caa4 Use haddock-library-1.10.0 2021-05-25 11:52:49 -07:00
Albert Krewinkel
d46ea7d7da
Jira: add support for "smart" links
Support has been added for the new
`[alias|https://example.com|smart-card]` syntax.
2021-05-25 16:54:42 +02:00
Albert Krewinkel
c5f9446646
Use jira-wiki-markup 1.3.5
* Allow spaces and most unicode characters in attachment links.

* No longer require a newline character after `{noformat}`.

* Only allow URI path segment characters in bare links.

* The `file:` schema is no longer allowed in bare links; these
  rarely make sense.

Closes: #7218
2021-05-24 08:45:59 +02:00
John MacFarlane
971a31b4d8 Bump upper-bounds for network-uri, time.
Change stack.yaml to use lts-17.12.
2021-05-21 10:51:35 -07:00
John MacFarlane
b243bf0f44 Bump to 2.14 and update changelog. 2021-05-13 14:10:47 -07:00
John MacFarlane
84fa901fd6 Use texmath 0.12.3. 2021-05-12 12:52:04 -07:00
John MacFarlane
6e45607f99 Change reader types, allowing better tracking of source positions.
Previously, when multiple file arguments were provided, pandoc
simply concatenated them and passed the contents to the readers,
which took a Text argument.

As a result, the readers had no way of knowing which file
was the source of any particular bit of text.  This meant that
we couldn't report accurate source positions on errors or
include accurate source positions as attributes in the AST.
More seriously, it meant that we couldn't resolve resource
paths relative to the files containing them
(see e.g. #5501, #6632, #6384, #3752).

Add Text.Pandoc.Sources (exported module), with a `Sources` type
and a `ToSources` class.  A `Sources` wraps a list of `(SourcePos,
Text)` pairs. [API change] A parsec `Stream` instance is provided for
`Sources`.  The module also exports versions of parsec's `satisfy` and
other Char parsers that track source positions accurately from a
`Sources` stream (or any instance of the new `UpdateSourcePos` class).

Text.Pandoc.Parsing now exports these modified Char parsers instead of
the ones parsec provides.  Modified parsers to use a `Sources` as stream
[API change].

The readers that previously took a `Text` argument have been
modified to take any instance of `ToSources`. So, they may still
be used with a `Text`, but they can also be used with a `Sources`
object.

In Text.Pandoc.Error, modified the constructor PandocParsecError
to take a `Sources` rather than a `Text` as first argument,
so parse error locations can be accurately reported.

T.P.Error: showPos, do not print "-" as source name.
2021-05-09 19:11:34 -06:00
Albert Krewinkel
3da919e35d
Add new internal module Text.Pandoc.Writers.GridTable 2021-05-01 18:52:24 +02:00
John MacFarlane
16b64bda38 Use dev texmath, citeproc. 2021-04-25 21:56:54 -07:00
Albert Krewinkel
0b74bbbdaa
Docx writer: extract Table handling into separate module 2021-04-20 10:57:54 +02:00
John MacFarlane
b7a2632184 Use lastest unicode-collation. 2021-04-18 11:34:12 -07:00
John MacFarlane
aecbf8156e Remove Text.Pandoc.BCP47 module.
[API change]

Use Lang from UnicodeCollation.Lang instead.
This is a richer implementation of BCP 47.
2021-04-17 16:15:14 -07:00
John MacFarlane
7ba8c0d2a5 Move getLang from BCP47 -> T.P.Writers.Shared.
[API change]
2021-04-17 16:15:13 -07:00
John MacFarlane
2e7fee9c3c Use latest xml-conduit. 2021-04-15 14:30:33 -07:00
Roman Beránek
fd0873c907
Require text for trypandoc (#7193) 2021-03-31 13:26:09 -07:00
John MacFarlane
d2495d0c75 Allow attoparsec 0.14.x. 2021-03-24 14:36:29 -07:00
John MacFarlane
88d3d55909 Require latest skylighting (fixes a bug in XML syntax highlighting). 2021-03-22 14:28:03 -07:00
John MacFarlane
99c15f8e85 Bump to 2.13, update changelog 2021-03-20 00:39:38 -07:00
John MacFarlane
ceadf33246 Tests: Use getExecutablePath from base...
avoiding the need to depend on the executable-path package.
2021-03-19 23:35:47 -07:00
John MacFarlane
029de661f4 Narrow version bounds for skylighting, citeproc, and texmath.
This reduces the chance that tests will fail due to behavior
changes in one of these dependencies. (See e.g. #7163)
2021-03-19 14:13:53 -07:00
John MacFarlane
425c2e47b5 Use skylighting 0.10.5.
This fixes a bad regression in Haskell syntax highlighting.
2021-03-19 11:58:56 -07:00
John MacFarlane
f0e4b9cc3c Require safe >= 0.3.18 and remove cpp. 2021-03-18 21:37:56 -07:00
John MacFarlane
2d10f46de5 Don't bake in extra stack size to the executables.
I don't think this is necessary; stack overflows generally
indicate a code problem and should be fixed (and have been when
reported).
2021-03-18 17:18:31 -07:00
John MacFarlane
2e00ed3fde Bump to 2.12.1 and update changelog. 2021-03-18 15:50:13 -07:00
John MacFarlane
8235daf780 Use -A8m for default rtsopts for benchmark 2021-03-18 15:46:20 -07:00
John MacFarlane
c7230182a0 pandoc.cabal: bake in -A8m to rtsopts.
This reserves a larger allocation area and reduces GC,
speeding up execution.
2021-03-18 11:02:46 -07:00
Alexey Kuleshevich
613c070cbd
Update bounds for random (#7156) 2021-03-17 18:57:32 -07:00
John MacFarlane
c6e5cf2e74 Benchmark improvements.
* Build `+RTS -A256m -RTS` into default ghc-options for benchmark,
  so we don't have to specify this separately on the command line.
  This is necessary to get accurate benchmark results; otherwise
  we are largely measuring garbage collecting, some not related
  to the current benchmark.
* Switch back from gauge to tasty-bench.
* Allow specifying BASELINE file in 'make bench' for comparison
  (otherwise the latest is chosen by default).
* Remove obsolete reference to weigh-pandoc from CONTRIBUTING.md.
* Remove `-Rghc-timing` from 'make bench'.
2021-03-17 13:34:17 -07:00
John MacFarlane
1ef3534328 Increase heap space in runtime for benchmarks.
Otherwise we're essentially benchmarking garbage collecting,
which can give very inconsistent results.
2021-03-16 15:59:50 -07:00
John MacFarlane
ff0fcedcb3 Switch to gauge for now for benchmarks.
tasty-bench is displaying odd behavior, with different
timings depending on the `--pattern` specified.
2021-03-15 22:50:18 -07:00
John MacFarlane
5f94dd74f1 Require texmath 0.12.2 2021-03-15 15:36:57 -07:00
John MacFarlane
39934c8851 Require latest doclayout and skylighting. 2021-03-14 15:48:01 -07:00
John MacFarlane
3519d6f3b4 Use eciteproc >= 0.3.0.9 2021-03-13 11:27:05 -08:00
Albert Krewinkel
f8b49e77f8
Use jira-wiki-markup 1.3.4
Jira reader:

* Fixed parsing of autolinks (i.e., of bare URLs in the text).
  Previously an autolink would take up the rest of a line, as spaces
  were allowed characters in these items.

* Emoji character sequences no longer cause parsing failures. This was
  due to missing backtracking when emoji parsing fails.

Jira writer:

* Block quotes are only rendered as `bq.` if they do not contain a
  linebreak.
2021-03-13 14:53:58 +01:00
John MacFarlane
e17127dc28 Re-add a needed dependency for benchmark. 2021-03-09 13:40:24 -08:00
John MacFarlane
a8b2031bb4 Revert "Use -Wunused-packages on ghc >= 8.10."
This reverts commit 7a1d0f01e9.

This option gives confusing output when a build is interrupted,
suggesting that packages aren't required when we just didn't
get to the model that requires them.
2021-03-09 12:49:15 -08:00
John MacFarlane
a9a05110d0 Remove some unused packages from pandoc.cabal. 2021-03-09 12:34:36 -08:00
John MacFarlane
7a1d0f01e9 Use -Wunused-packages on ghc >= 8.10. 2021-03-09 12:34:36 -08:00
John MacFarlane
e649b69564 Bump version to 2.12 2021-03-04 08:57:27 -08:00
John MacFarlane
92ea8a0cb6 Revert "Add T.P.Readers.LaTeX.Include."
This reverts commit b569b0226d.

Memory usage improvement in compilation wasn't very significant.
2021-03-03 19:07:16 -08:00
John MacFarlane
b569b0226d Add T.P.Readers.LaTeX.Include. 2021-03-03 18:47:17 -08:00
John MacFarlane
33e4c8dd6c Remove T.P.Readers.LaTeX.Accent.
Incorporate accentCommands into T.P.Readers.LaTeX.Inline.
2021-03-03 18:21:32 -08:00
John MacFarlane
bbcc1501a5 Split out T.P.Readers.LaTeX.Inline. 2021-03-03 10:34:10 -08:00
John MacFarlane
e8e5ffe1f4 Split out T.P.Writers.LaTeX.Util. 2021-03-02 22:40:45 -08:00
John MacFarlane
fe483c653b Split out T.P.Writers.LaTeX.Citation. 2021-03-02 21:57:37 -08:00
John MacFarlane
827ecdd2de Split out T.P.Writers.LaTeX.Lang. 2021-03-02 21:33:58 -08:00
John MacFarlane
2097411e4f Split up T.P.Writers.Markdown...
with T.P.Writers.Markdown.Types and T.P.Writers.Markdown.Inline.
The module was difficult to compile on low-memory system.s
2021-03-02 21:08:13 -08:00
John MacFarlane
7f1b933aaa Make T.P.Readers.LaTeX.Types an unexported module.
[API change]

This is really an implementation detail that shouldn't be
exposed in the public API.
2021-03-01 09:46:43 -08:00
John MacFarlane
382f0e23d2 Factor out T.P.Readers.LaTeX.Macro. 2021-03-01 09:46:43 -08:00
John MacFarlane
d2bb0c7c8d Factor out T.P.Readers.LaTeX.Math. 2021-02-28 21:05:25 -08:00
John MacFarlane
7e83686d31 trypandoc: add 2 second timeout. 2021-02-28 09:24:37 -08:00
John MacFarlane
2faa57e8e9 Factor out T.P.Readers.LaTeX.Citation. 2021-02-28 09:12:09 -08:00
John MacFarlane
08231f5cdd Factor out T.P.Readers.LaTeX.Table. 2021-02-27 21:40:56 -08:00
John MacFarlane
925815bb33 Split off T.P.Readers.LaTeX.Accent.
To help reduce memory demands compiling the main LaTeX reader.
2021-02-27 17:02:44 -08:00
John MacFarlane
cbc3f034ad Use skylighting 0.10.4.
This version of skylighting uses xml-conduit rather than hxt.
This speeds up parsing of XML syntax definitions fourfold, and
removes four packages from pandoc's dependency graph:

hxt-charproperties
hxt-unicode
hxt-regex-xmlschema
hxt
2021-02-27 14:26:10 -08:00
John MacFarlane
9767386676 Use latest skylighting. 2021-02-22 22:25:10 -08:00
John MacFarlane
d7cfa0ef4c Remove weigh-pandoc.
It's not really useful any more, now that our regular
benchmarks include data on allocation.
2021-02-22 22:10:20 -08:00
John MacFarlane
b2b32d9bb2 'make bench': Create csv files for comparison. 2021-02-18 23:22:18 -08:00
Dmitrii Kovanikov
ef741f3842 Allow base64-bytestring-1.2.* 2021-02-18 18:07:23 +01:00
John MacFarlane
80a1d5c9b6 Revert "Add T.P.XML.Light.Cursor."
This reverts commit d8fc497186.
2021-02-16 19:18:01 -08:00
John MacFarlane
d8fc497186 Add T.P.XML.Light.Cursor. 2021-02-16 18:51:41 -08:00
John MacFarlane
d7a4996b1e Split up T.P.XML.Light into submodules. 2021-02-16 18:40:06 -08:00
John MacFarlane
967e7f5fb9 Rename Text.Pandoc.XMLParser -> Text.Pandoc.XML.Light...
..and add new definitions isomorphic to xml-light's, but with
Text instead of String.  This allows us to keep most of the code in
existing readers that use xml-light, but avoid lots of unnecessary
allocation.

We also add versions of the functions from xml-light's
Text.XML.Light.Output and Text.XML.Light.Proc that operate
on our modified XML types, and functions that convert
xml-light types to our types (since some of our dependencies,
like texmath, use xml-light).

Update golden tests for docx and pptx.

OOXML test: Use `showContent` instead of `ppContent` in `displayDiff`.

Docx: Do a manual traversal to unwrap sdt and smartTag.
This is faster, and needed to pass the tests.

Benchmarks:

A = prior to 8ca191604d (Feb 8)
B = as of 8ca191604d (Feb 8)
C = this commit

| Reader  |  A    | B      | C     |
| ------- | ----- | ------ | ----- |
| docbook | 18 ms | 12 ms  | 10 ms |
| opml    | 65 ms | 62 ms  | 35 ms |
| jats    | 15 ms | 11 ms  |  9 ms |
| docx    | 72 ms | 69 ms  | 44 ms |
| odt     | 78 ms | 41 ms  | 28 ms |
| epub    | 64 ms | 61 ms  | 56 ms |
| fb2     | 14 ms | 5  ms  | 4 ms  |
2021-02-16 16:55:20 -08:00
Albert Krewinkel
1942dc5611
Allow tasty 1.4.* 2021-02-14 14:43:32 +01:00
Albert Krewinkel
8ffd4159d6
Jira: require jira-wiki-markup 1.3.3
* Modified the Doc parser to skip leading blank lines. This fixes
  parsing of documents which start with multiple blank lines.
  (#7095)

* Prevent URLs within link aliases to be treated as autolinks.
  (#6944)

Fixes: #7095
Fixes: #6944
2021-02-12 17:15:12 +01:00
John MacFarlane
8ca191604d Add new unexported module T.P.XMLParser.
This exports functions that uses xml-conduit's parser to
produce an xml-light Element or [Content].  This allows
existing pandoc code to use a better parser without
much modification.

The new parser is used in all places where xml-light's
parser was previously used.  Benchmarks show a significant
performance improvement in parsing XML-based formats
(especially ODT and FB2).

Note that the xml-light types use String, so the
conversion from xml-conduit types involves a lot
of extra allocation.  It would be desirable to
avoid that in the future by gradually switching
to using xml-conduit directly. This can be done
module by module.

The new parser also reports errors, which we report
when possible.

A new constructor PandocXMLError has been added to
PandocError in T.P.Error [API change].

Closes #7091, which was the main stimulus.

These changes revealed the need for some changes
in the tests.  The docbook-reader.docbook test
lacked definitions for the entities it used; these
have been added. And the docx golden tests have been
updated, because the new parser does not preserve
the order of attributes.

Add entity defs to docbook-reader.docbook.

Update golden tests for docx.
2021-02-10 22:04:11 -08:00
Albert Krewinkel
d202f7eb77
Avoid unnecessary use of NoImplicitPrelude pragma (#7089) 2021-02-07 10:02:35 -08:00
Albert Krewinkel
f7be8d0964
pandoc.cabal: use common stanza to reduce duplication (#7086) 2021-02-07 08:33:43 -08:00
Albert Krewinkel
ebb8f23b66 Use hslua-module-path 0.1.0 2021-02-02 21:04:30 -08:00
Albert Krewinkel
61b108d527 Lua: add module "pandoc.path"
The module allows to work with file paths in a convenient and
platform-independent manner.

Closes: #6001
Closes: #6565
2021-02-02 21:04:30 -08:00
John MacFarlane
02d3c71e72 BibTeX writer: use doclayout and doctemplate.
This change allows bibtex/biblatex output to wrap as other
formats do, depending on the settings of `--wrap` and `--columns`.

It also introduces default templates for bibtex and biblatex,
which allow for using the variables `header-include`, `include-before`
or `include-after` (or alternatively the command line options
`--include-in-header`, `--include-before-body`, `--include-after-body`)
to insert content into the generated bibtex/biblatex.

This change requires a change in the return type of the unexported
`T.P.Citeproc.writeBibTeXString` from `Text` to `Doc Text`.

Closes #7068.
2021-02-01 18:05:20 -08:00
John MacFarlane
b239c89a82 BibTeX writer fixes. Closes #7067.
+ Require citeproc 0.3.0.7, which correctly titlecases when titles
  contain non-ASCII characters.
+ Correctly handle 'pages' (= 'page' in CSL).
+ Correctly handle BibLaTeX 'langid' (= 'language' in CSL).
+ In BibTeX output, protect foreign titles since there's no language
  field.
2021-02-01 11:23:07 -08:00
John MacFarlane
a9adb29648 Require citeproc 0.3.0.6. 2021-01-30 19:09:08 -08:00
John MacFarlane
fe06437ba4 Use tasty-bench instead of criterion for benchmarks.
It is much lighter-weight.
2021-01-30 18:01:14 -08:00
John MacFarlane
f5e3c1dad6 Use citeproc 0.3.0.5. 2021-01-22 11:06:35 -08:00
John MacFarlane
83d7804b8f
Merge pull request #7042 from tarleb/jats-element-citations
JATS writer: use element citations
2021-01-22 10:39:58 -08:00
Albert Krewinkel
b4b3560191
JATS writer: allow to use element-citation 2021-01-22 19:35:08 +01:00
John MacFarlane
fa952c8dbe Add biblatex, bibtex as output formats (closes #7040).
* `biblatex` and `bibtex` are now supported as output
  as well as input formats.

* New module Text.Pandoc.Writers.BibTeX, exporting
  writeBibTeX and writeBibLaTeX. [API change]

* New unexported function `writeBibtexString` in
  Text.Pandoc.Citeproc.BibTeX.
2021-01-22 10:08:43 -08:00
John MacFarlane
07c98eae50 Use citeproc >= 0.3.0.4. 2021-01-15 14:17:17 -08:00
John MacFarlane
4a223e68f4 Use commonmark 0.1.1.3. 2021-01-11 12:23:55 -08:00
Albert Krewinkel
68fa437999
JATS writer: fix citations (#7018)
* JATS writer: keep code lines at 80 chars or below

* JATS writer: fix citations
2021-01-10 15:35:48 -08:00
John MacFarlane
c83811773e Bump to 2.11.4.
API change: export getReferences from T.P.Citeproc.
2021-01-10 10:16:15 -08:00
Albert Krewinkel
4f34345867
Update copyright notices for 2021 (#7012) 2021-01-08 09:38:20 -08:00
David Martschenko
385b6a3b21
Implement defaults file inheritance (#6924)
Allow defaults files to inherit options from other defaults files by
specifying them with the following syntax:
`defaults: [list of defaults files or single defaults file]`.
2021-01-05 10:15:59 -08:00
John MacFarlane
886faa3cbc Bump to 2.11.3.2, update changelog and man page 2020-12-29 12:48:55 -08:00
John MacFarlane
5d8b57444e Use citeproc 0.3.0.3.
Fixes an issue in author-only citations when both an
author and translator are present.
2020-12-29 10:43:50 -08:00