Commit graph

1371 commits

Author SHA1 Message Date
John MacFarlane
914cf0b602 Fix citeproc regression with duplicate references.
- Use dev version of citeproc, which handles duplicate
  ids better, preferring the last one in the list
  and discarding the rest.
- Ensure that inline citations take priority over external
  ones.

See jgm/citeproc#36.

This restores the behavior of pandoc-citeproc.
2020-12-16 15:37:40 -08:00
John MacFarlane
b4b4e32307 Properly handle boolean values in writing YAML metadata.
(Markdown writer.)
This requires doctemplates >= 0.9.
Closes #6388.
2020-12-15 23:45:34 -08:00
John MacFarlane
2ce14997ad Require binary >= 0.7.
Needed for runGetOrFail.
2020-12-13 10:33:46 -08:00
Albert Krewinkel
ccd235e31f
LaTeX writer: extract table handling into separate module. 2020-12-12 16:48:28 +01:00
John MacFarlane
248a2a1db5 cabal: remove -Werror=missing-home-modules.
It causes problems using cabal repl.
2020-12-10 10:27:31 -08:00
John MacFarlane
1fd642dd30 Move executable to app directory.
Otherwise we have problems with cabal repl.
2020-12-10 10:08:24 -08:00
John MacFarlane
a3eb87b2ea Add sourcepos extension for commonmarke
* Add `Ext_sourcepos` constructor for `Extension`.
* Add `sourcepos` extension (only for commonmark).
* Bump to 2.11.3

With the `sourcepos` extension set set, `data-pos` attributes are added
to the AST by the commonmark reader. No other readers are affected.  The
`data-pos` attributes are put on elements that accept attributes; for
other elements, an enlosing Div or Span is added to hold the attributes.

Closes #4565.
2020-12-10 08:59:55 -08:00
John MacFarlane
0dd228593f Use latest citeproc release. 2020-12-09 09:34:15 -08:00
John MacFarlane
1489bb8414 Use skylighting 0.10.1. 2020-11-24 21:26:25 -08:00
Albert Krewinkel
41237fcc0e
HTML reader: extract table parsing into separate module 2020-11-24 14:17:35 +01:00
Albert Krewinkel
f9258371dd HTML reader: extract submodules
Reducing module size should reduce memory use during compilation.

This is preparatory work to tackle support for more table features.
2020-11-23 10:12:20 +01:00
John MacFarlane
3f278f580e Remove 'static' flag.
This isn't really necessary and can be misleading
(e.g. on macOS, where a fully static build isn't
possible). cabal's new option
`--enable-executable-static` does the same. On stack
you can add something like this to the options for your
executable in package.yaml:

    ld-options: -static -pthread
2020-11-18 21:08:24 -08:00
John MacFarlane
e17f970ed0 Use citeproc 0.2 2020-11-18 17:49:30 -08:00
John MacFarlane
46bbdad838 Don't allow macos builds with 'static' flag.
Closes #6771.
2020-11-18 15:41:48 -08:00
Albert Krewinkel
94c9028819 JATS writer: move Table handling to separate module
This makes it easier to split the module into smaller parts.
2020-11-17 09:46:30 +01:00
John MacFarlane
79907e5f17 Bump to 2.11.2 for next release (minor API change in Logging). 2020-11-15 08:34:45 -08:00
John MacFarlane
cfb017c76b Bump to 2.11.1.1 and update changelog. 2020-11-07 11:12:19 -08:00
John MacFarlane
e6abf3b8ed Use citeproc 0.1.1.1.
Closes #6813.
2020-11-05 21:23:57 -08:00
John MacFarlane
9de386352a Require latest commonmark, commonmark-extensions.
Fixes a bug with `autolink_bare_uris` and commonmark.
2020-11-05 16:58:36 -08:00
John MacFarlane
391f6e5f80 Use latest commonmark, commonmark-extensions. 2020-11-05 15:05:11 -08:00
John MacFarlane
b5e9c2a7a6 Use citeproc 0.1.1. 2020-11-04 11:15:48 -08:00
John MacFarlane
f502c8d944 Bump version to 2.11.1 and update changelog. 2020-11-02 22:20:44 -08:00
John MacFarlane
992657efaa Use latest commonmark, commonmark-extensions.
This fixes a bug with nested blocks in footnotes with the
`footnote` extension to `commonmark`.  See jgm/commonmark-hs#63.
2020-11-01 10:48:47 -08:00
John MacFarlane
f20ec6b329 Bump to 2.11.0.4. 2020-10-22 22:06:38 -07:00
John MacFarlane
2059c05f0e Require citeproc >= 0.1.0.3.
In the previous release we pointed to this with cabal.project
and stack.yaml, but jumped the gun because citeproc 0.1.0.3
had not yet been officially released.
2020-10-22 21:45:38 -07:00
John MacFarlane
4731fa1d3f Bump to 2.11.0.3 and update changelog. 2020-10-22 17:35:43 -07:00
John MacFarlane
d199abb380 Bump version to 2.11.0.2 2020-10-19 16:32:39 -07:00
Albert Krewinkel
ae4e9d3b38
Relax upper bound on hslua, allow hslua-1.3.* 2020-10-16 21:41:05 +02:00
John MacFarlane
3c8b3eba17 Require citeproc 0.1.0.2. 2020-10-15 13:00:37 -07:00
John MacFarlane
0646873964 Version to 2.11.0.1 2020-10-13 22:47:56 -07:00
John MacFarlane
1122d22a2c Use citeproc 0.1.0.1. 2020-10-13 22:44:05 -07:00
John MacFarlane
b438f0c7a7 pandoc.cabal - recognize new formats in description. 2020-10-10 22:20:56 -07:00
John MacFarlane
eff6b8f27d Use latest citeproc. 2020-09-27 16:03:31 -07:00
John MacFarlane
09d39e0e98 ALlow bytestring 0.11.x. 2020-09-23 22:27:20 -07:00
John MacFarlane
e0984a43a9 Add built-in citation support using new citeproc library.
This deprecates the use of the external pandoc-citeproc
filter; citation processing is now built in to pandoc.

* Add dependency on citeproc library.
* Add Text.Pandoc.Citeproc module (and some associated unexported
  modules under Text.Pandoc.Citeproc).  Exports `processCitations`.
  [API change]
* Add data files needed for Text.Pandoc.Citeproc:  default.csl
  in the data directory, and a citeproc directory that is just
  used at compile-time.  Note that we've added file-embed as a mandatory
  rather than a conditional depedency, because of the biblatex
  localization files. We might eventually want to use readDataFile
  for this, but it would take some code reorganization.
* Text.Pandoc.Loging: Add `CiteprocWarning` to `LogMessage` and use it
  in `processCitations`. [API change]
* Add tests from the pandoc-citeproc package as command tests (including
  some tests pandoc-citeproc did not pass).
* Remove instructions for building pandoc-citeproc from CI and
  release binary build instructions.  We will no longer distribute
  pandoc-citeproc.
* Markdown reader: tweak abbreviation support.  Don't insert a
  nonbreaking space after a potential abbreviation if it comes right before
  a note or citation.  This messes up several things, including citeproc's
  moving of note citations.
* Add `csljson` as and input and output format. This allows pandoc
  to convert between `csljson` and other bibliography formats,
  and to generate formatted versions of CSL JSON bibliographies.
* Add module Text.Pandoc.Writers.CslJson, exporting `writeCslJson`. [API
  change]
* Add module Text.Pandoc.Readers.CslJson, exporting `readCslJson`. [API
  change]
* Added `bibtex`, `biblatex` as input formats.  This allows pandoc
  to convert between BibLaTeX and BibTeX and other bibliography formats,
  and to generated formatted versions of BibTeX/BibLaTeX bibliographies.
* Add module Text.Pandoc.Readers.BibTeX, exporting `readBibTeX` and
  `readBibLaTeX`. [API change]
* Make "standalone" implicit if output format is a bibliography format.
  This is needed because pandoc readers for bibliography formats put
  the bibliographic information in the `references` field of metadata;
  and unless standalone is specified, metadata gets ignored.
  (TODO: This needs improvement. We should trigger standalone for the
  reader when the input format is bibliographic, and for the writer
  when the output format is markdown.)
* Carry over `citationNoteNum` to `citationNoteNumber`.  This was just
  ignored in pandoc-citeproc.
* Text.Pandoc.Filter: Add `CiteprocFilter` constructor to Filter.
  [API change] This runs the processCitations transformation.
  We need to treat it like a filter so it can be placed
  in the sequence of filter runs (after some, before others).
  In FromYAML, this is parsed from `citeproc` or `{type: citeproc}`,
  so this special filter may be specified either way in a defaults file
  (or by `citeproc: true`, though this gives no control of positioning
  relative to other filters).  TODO: we need to add something to the
  manual section on defaults files for this.
* Add deprecation warning if `upandoc-citeproc` filter is used.
* Add `--citeproc/-C` option to trigger citation processing.
  This behaves like a filter and will be positioned
  relative to filters as they appear on the command line.
* Rewrote the manual on citatations, adding a dedicated Citations
  section which also includes some information formerly found in
  the pandoc-citeproc man page.
* Look for CSL styles in the `csl` subdirectory of the pandoc user data
  directory.  This changes the old pandoc-citeproc behavior, which looked
  in `~/.csl`.  Users can simply symlink `~/.csl` to the `csl`
  subdirectory of their pandoc user data directory if they want
  the old behavior.
* Add support for CSL bibliography entry formatting to LaTeX, HTML,
  Ms writers.  Added CSL-related CSS to styles.html.
2020-09-21 10:15:50 -07:00
John MacFarlane
89c577befb Bump to 2.11. 2020-09-21 10:15:49 -07:00
Albert Krewinkel
ba591ba365
pandoc.cabal: sort build depends alphabetically (#6691) 2020-09-21 09:28:25 -07:00
Albert Krewinkel
acbea6b8c6
Lua filters: add SimpleTable for backwards compatibility (#6575)
A new type `SimpleTable` is made available to Lua filters. It is
similar to the `Table` type in pandoc versions before 2.10;
conversion functions from and to the new Table type are provided.

Old filters using tables now require minimal changes and can use,
e.g.,

    if PANDOC_VERSION > {2,10,1} then
      pandoc.Table = pandoc.SimpleTable
    end

and

    function Table (tbl)
      tbl = pandoc.utils.to_simple_table(tbl)
      …
      return pandoc.utils.from_simple_table(tbl)
    end

to work with the current pandoc version.
2020-09-20 15:48:31 -07:00
Albert Krewinkel
b2decdfd13
CI: bump tested GHC versions to 8.8.4 and 8.10.2
Besides being newer, GHC version 8.10.2 comes preinstalled on GitHub
Actions environments; using it slightly speeds up CI tests.
2020-09-20 22:57:51 +02:00
Albert Krewinkel
c9e7ccba6f
pandoc.cabal: allow hslua 1.2 again
Sporadic test failures also happen with hslua-1.1.*, so there is no need
to exclude the newer version.

This reverts commit 315b5a4836.
2020-09-20 09:20:59 +02:00
John MacFarlane
1dd9f8b654 Use released pandoc-types 1.22. 2020-09-19 09:58:16 -07:00
Albert Krewinkel
a400d0dc62
HTML writer: render table footers if present
Part of: #6314
2020-09-12 21:49:01 +02:00
Christian Despres
22babd5382
[API change] Rename Writers.Tables and its contents (#6679)
Writers.Tables is now Writers.AnnotatedTable. All of the types and
functions in it have had the "Ann" removed from them. Now it is
expected that the module be imported qualified.
2020-09-12 08:50:36 -07:00
Albert Krewinkel
315b5a4836
pandoc.cabal: disallow hslua 1.2
See #6674
2020-09-11 09:50:33 +02:00
Christian Despres
10c6c411f9
Add Writers.Tables helper functions and types, add tests for those (#6655)
Add Writers.Tables helper functions and types, add tests for those

The Writers.Tables module contains an AnnTable type that is a pandoc
Table with added inferred information that should be enough for
writers (in particular the HTML writer) to operate on without having
to lay out the table themselves.

The toAnnTable and fromAnnTable functions in that module convert
between AnnTable and Table. In addition to producing an AnnTable with
coherent and well-formed annotations, the toAnnTable function also
normalizes its input Table like the table builder does.

Various tests ensure that toAnnTable normalizes tables exactly like
the table builder, and that its annotations are coherent.
2020-09-05 14:36:51 -07:00
John MacFarlane
81fe8ebf36 LaTeX reader: Factored out siunitx stuff into separate module. 2020-09-02 10:10:55 -07:00
John MacFarlane
613828e497 Use skylighting 0.10. 2020-08-30 22:54:36 -07:00
Albert Krewinkel
333c553548
Relax version constraints for hslua, hslua-module-text
Allow `hslua-1.2.*` and `hslua-module-text-0.3.*`.
2020-08-15 13:52:37 +02:00
John MacFarlane
57417feaf4 Use commonmark >= 0.1.0.2. Closes #6589. 2020-08-05 12:58:29 -07:00
Felix Yan
48487f287f
Remove duplicated exceptions (#6591) 2020-08-05 12:12:01 -07:00
John MacFarlane
57e29610ac Use skylighting 0.9. 2020-07-31 08:56:59 -07:00
John MacFarlane
ac1f45c553 Update changelog and bump to 2.10.1. 2020-07-23 17:30:57 -07:00
John MacFarlane
c37a56e060 Use commonmark-0.1.0.1.
Partially addresses #6556.
2020-07-23 16:36:44 -07:00
John MacFarlane
942e3ee1f9 RST reader: fix csv tables with multiline cells.
Closes #6549.
2020-07-21 10:20:15 -07:00
John MacFarlane
e17b4718d4 Use commonmark-extensions, commonmark-pandoc 0.2. 2020-07-19 22:51:59 -07:00
John MacFarlane
d6b7b1dc77 Remove use of cmark-gfm for commonmark/gfm rendering.
Instead rely on the markdown writer with appropriate extensions.

Export writeCommonMark variant from Markdown writer.
This changes a few small things in rendering markdown,
e.g. w/r/t requiring backslashes before spaces inside
super/subscripts.
2020-07-19 22:51:59 -07:00
John MacFarlane
0db4702042 Use commonmark-hs to parse commonmark/gfm...
...instead of cmark-gfm (a wrapper around a C library).

We can now support many more pandoc extensions for
commonmark and gfm.

Add fenced_code_attributes to gfm/commonmark extensions.
2020-07-19 22:51:59 -07:00
John MacFarlane
c2cc4a0504 CI: Drop testing on ghc 8.0.2. 2020-07-19 22:51:59 -07:00
Michael Hoffmann
09ea10e2b1 Escape starting periods in ms writer code blocks
If a line of ms code block output starts with a period (.), it should
be prepended by '\&' so that it is not interpreted as a roff command.

Fixes #6505
2020-07-08 23:52:28 +02:00
John MacFarlane
6716d1c702 Bump version to 2.10. Update tested-with.
We need a major version bump because of the new pandoc-types.
2020-06-22 16:59:20 -07:00
Albert Krewinkel
064303e2c9
Jira writer: always escape braces
Braces are now always escaped, even within words or when surrounded by
whitespace. Jira and Confluence treat braces specially.

Package jira-wiki-markup must be version 1.3.2 or later.

Fixes: #6478
2020-06-22 16:30:11 +02:00
John MacFarlane
112e98def6 Use latest skylighting. 2020-06-19 15:31:31 -07:00
Laurent P. René de Cotret
e8f532780b
Remove unused dependency vector (#6462) 2020-06-15 10:03:41 -07:00
Albert Krewinkel
16889a01b9
Jira reader: require jira-wiki-markup 1.3.1 or later
This solves the following problems of the Jira reader:

 * Two consecutive markup chars are now parsed verbatim; styled text
   must not be empty.
 * Styled text may not contain newlines.
 * Links to anchors are now parsed as links.

Fixes: #6343
Fixes: #6325
Fixes: #6407
2020-06-14 11:46:59 +02:00
John MacFarlane
287a3fcc51 Allow http-client 0.7. 2020-05-28 11:25:01 -07:00
Felix Yan
46cb70a30c
Allow aeson 1.5 (#6400)
Builds fine and all tests pass here.
2020-05-25 22:02:54 -07:00
John MacFarlane
1e031ae63a Allow tasty 1.3.x. 2020-05-21 09:57:24 -07:00
Lila
57fa394eb0
Bump cabal-version to 2.2 (#6377)
In b3cfdc2c7 the license was changed to GPL-2.0-or-later which is an
SPDX expression, however cabal only interprets the license field as an
SPDX expression if cabal-version is 2.2 or later.

Starting with 2.2 cabal-version also has to be the first statement in
the .cabal file.
2020-05-18 14:29:43 -07:00
Lila
b3cfdc2c7a Correct license listed in cabal
`GPL-2` is interpreted as GPLv2 only, however the actual license is GPLv2 or later
2020-05-13 22:27:44 +02:00
John MacFarlane
46179d5b3e Use latest skylighting.
This adds `aria-hidden="true"` to the empty a elements, which
helps people who use screen readers.
2020-05-12 14:37:07 -07:00
John MacFarlane
3324412d72 Allow base64-bytestring 1.1. 2020-04-25 12:55:17 -07:00
John MacFarlane
61771e2c9b Use pandoc-types 1.21, new texmath. 2020-04-19 08:17:32 -07:00
Albert Krewinkel
62cf21cbaa
API change: use new type PandocLua for all pandoc Lua operations
The new type `PandocLua` is an instance of the `PandocMonad` typeclass
and can thus be used in a way similar to `PandocIO`.
2020-04-17 23:05:44 +02:00
Albert Krewinkel
eceb8eaf47
Class: generalize PandocIO functions to MonadIO 2020-04-17 23:05:31 +02:00
Albert Krewinkel
fb54f3d679
API change: use PandocError for exceptions in Lua subsystem
The PandocError type is used throughout the Lua subsystem, all Lua
functions throw an exception of this type if an error occurs. The
`LuaException` type is removed and no longer exported from
`Text.Pandoc.Lua`. In its place, a new constructor `PandocLuaError` is
added to PandocError.
2020-04-17 21:52:48 +02:00
John MacFarlane
a9ef15bbd5 Revert 0e48a02 and dependency on base-noprelude...
which hasn't been updated for ghc 8.10.
See discussion at #6187.
2020-04-17 09:50:55 -07:00
Cédric Couralet
34775b4128
Add an option to disable certificate validation (#6156)
This commit adds the option `--no-check-certificate`, which disables certificate
checking when resources are fetched by HTTP.

Co-authored-by: Cécile Chemin <cecile.chemin@insee.fr>
Co-authored-by: Juliette Fourcot <juliette.fourcot@insee.fr>
2020-04-13 14:58:42 -07:00
John MacFarlane
58cd731e51 Require doctemplates 0.8.2. 2020-04-12 22:25:31 -07:00
John MacFarlane
3f7f741297 Allow haddock-library 1.9.x. 2020-04-06 21:01:04 -07:00
Albert Krewinkel
c3f539364a
Jira: support citations, attachment links, and user links
Closes: #6231
Closes: #6238
Closes: #6239
2020-04-04 14:27:27 +02:00
Albert Krewinkel
d867cac8ca
Jira reader: resolve parsing issues of blockquote, color
Parsing problems occurring with block quotes and colored text have been
resolved.

Fixes: #6233
Fixes: #6235
2020-04-03 13:25:52 +02:00
Felix Yan
f6615ad85e
Allow hslua 1.1 (#6243)
Builds fine and all tests pass.
2020-04-01 08:52:14 -07:00
Albert Krewinkel
ff9be6b384
Jira writer: convert spans with class underline to inserted text
Spans with class `underline` as converted into Jira text marked as
`+inserted+`, i.e. surrounded by plus-signs.
2020-03-31 09:57:59 +02:00
Albert Krewinkel
09a5a1f447
Jira reader: fix multiple parsing problems
Multiple parsing problems are resolved, including issues with empty
table cells, faulty recognition of closing emphasis characters, and
parsing of image attributes.

Fixes: #6212
Fixes: #6219
Fixes: #6220
2020-03-28 15:57:00 +01:00
John MacFarlane
e1d302b3ef Bump QuickCheck upper bound. 2020-03-27 22:23:13 -07:00
John MacFarlane
b5a3c3292a Bump version to 2.9.1. 2020-03-23 14:38:58 -07:00
Albert Krewinkel
2a042ff711
Text.Pandoc.Class: extract submodules PandocIO, PandocPure 2020-03-22 14:59:38 +01:00
Albert Krewinkel
44f8c2725e
Jira reader: fix parsing of tables without preceding blankline
A bug was fixed which caused faulty parsing if a table was not preceded
by a newline and the first table cell had no space after the initial `|`
characters.

Fixes: #6198
2020-03-19 21:27:35 +01:00
Albert Krewinkel
81d46435f6
Jira reader: fix parsing of strikeout, emphasis
A bug was fixed which caused non-emphasized text containing digits and/or
non-special symbols (like dots) to sometimes be parsed incorrectly.

Fixes: #6196
2020-03-18 21:32:05 +01:00
Albert Krewinkel
a5fa55969f
Use implicit Prelude (#6187)
* Use implicit Prelude

The previous behavior was introduced as a fix for #4464. It seems that
this change alone did not fix the issue, and `stack ghci` and `cabal
repl` only work with GHC 8.4.1 or newer, as no custom Prelude is loaded
for these versions. Given this, it seems cleaner to revert to the
implicit Prelude.

* PandocMonad: remove outdated check for base version

Only base versions 4.9 and later are supported, the check for
`MIN_VERSION_base(4,8,0)` is therefore unnecessary.

* Always use custom prelude

Previously, the custom prelude was used only with older GHC versions, as
a workaround for problems with ghci. The ghci problems are resolved by
replacing package `base` with `base-noprelude`, allowing for consistent
use of the custom prelude across all GHC versions.
2020-03-15 09:45:44 -07:00
Albert Krewinkel
ec49643d64
Subdivide Text.Pandoc.Class into small modules (#6106)
* Extract CommonState into submodule

* Extract PandocMonad into submodule

* PandocMonad: ensure all functions have Haddock documentation
2020-03-14 20:49:36 -07:00
Albert Krewinkel
11b5f1e40b
Update copyright year (#6186)
* Update copyright year

* Copyright: add notes for Lua and Jira modules
2020-03-13 09:52:47 -07:00
Albert Krewinkel
7eb9914841
Jira reader: support colored inline text, indented lists
* Support for colored inlines has been added.
* Lists are now allowed to be indented; i.e., lists are still recognized
  if list markers are preceded by spaces.

Closes: #6183, #6184
2020-03-13 09:52:28 +01:00
John MacFarlane
33fcac40d5 Use latest texmath. 2020-02-15 09:12:56 -08:00
John MacFarlane
3a181f0a97 Add Text.Pandoc.Image with unexported svgToPng. 2020-02-12 21:57:00 -08:00
Albert Krewinkel
f5ea5f0aad
Introduce new format variants for JATS (#6067)
New formats:

- `jats_archiving` for the "Archiving and Interchange Tag Set",
- `jats_publishing` for the "Journal Publishing Tag Set", and
- `jats_articleauthoring` for the "Article Authoring Tag Set."

The "jats" output format is now an alias for "jats_archiving".

Closes: #6014
2020-02-12 20:36:02 -08:00
John MacFarlane
f9514ccb9e Add Text.Pandoc.Readers.CSV (readCSV).
This adds csv as an input format.
The CSV table is converted into a pandoc simple table.

Closes #6100.
2020-01-31 21:14:21 -08:00
John MacFarlane
3b4dc3a2b0 Use skylighting 0.8.3.2. 2020-01-21 17:57:23 -08:00
Albert Krewinkel
672a4bdd1d Lua filters: allow filtering of element lists (#6040)
Lists of Inline and Block elements can now be filtered via `Inlines` and
`Blocks` functions, respectively. This is helpful if a filter conversion
depends on the order of elements rather than a single element.

For example, the following filter can be used to remove all spaces
before a citation:

    function isSpaceBeforeCite (spc, cite)
      return spc and spc.t == 'Space'
       and cite and cite.t == 'Cite'
    end

    function Inlines (inlines)
      for i = #inlines-1,1,-1 do
        if isSpaceBeforeCite(inlines[i], inlines[i+1]) then
          inlines:remove(i)
        end
      end
      return inlines
    end

Closes: #6038
2020-01-15 14:26:00 -08:00
John MacFarlane
9009bda179 Update versions for doclayout, doctemplates.
Closes #6031.  The new version of doclayout fixes a
memory leak that affected `--include-in-header` with
large files (and possibly other cases involving extremely
long lines).
2020-01-13 08:39:25 -08:00