Commit graph

964 commits

Author SHA1 Message Date
Kolen Cheung
3f0ceaf6ee Use CamelCase for JavaScript for uniformity
other instances when javascript is refered to in pandoc is also in CamelCase, the official casing of JavaScript.
2017-10-26 22:43:19 -07:00
John MacFarlane
7d60e9e0eb Bring MANUAL, README, and pandoc.cabal description in sync. 2017-10-26 16:30:37 -07:00
Sascha Wilde
66fd3247ea Creole reader (#3994)
This is feature complete but not very thoroughly tested yet.
2017-10-26 19:19:28 -04:00
John MacFarlane
76886678a6 Use skylighting 0.4.2.
This prevents the problem with extra space around highlighted
code blocks (closes #3996).

Note that we no longer put an enclosing div around highlighted
code blocks.  The pre is the outer element, just as for unhighlighted
blocks.
2017-10-26 15:57:55 -07:00
John MacFarlane
a27dc79073 Updated Glob upper bound. 2017-10-23 11:20:31 -07:00
John MacFarlane
dc92c34827 Use doctemplates 0.2.1: allows $-- line comments in templates.
Closes #3806.
2017-10-20 22:44:49 -07:00
John MacFarlane
181d7370bb Export Text.Pandoc.BCP47 2017-10-17 22:21:32 -07:00
John MacFarlane
99c1481c12 Split list of authors from MANUAL.txt into separate file, AUTHORS.md.
It was getting too long for a man page.
2017-10-17 20:30:20 -07:00
John MacFarlane
c40857b389 Improved handling of include files in LaTeX reader.
Previously `\include` wouldn't work if the included file
contained, e.g., a begin without a matching end.

We've changed the Tok type so that it stores a full SourcePos,
rather than just a line and column.  So tokens keeep track
of the file they came from. This allows us to use a simpler
method for includes, which doesn't require parsing the included
document as a whole.

Closes #3971.
2017-10-16 22:05:34 -07:00
John MacFarlane
2f66d57616 Remove openURL from Shared (API change).
Now all the guts of openURL have been put into openURL from
Class.  openURL is now sensitive to stRequestHeaders in CommonState
and will add these custom headers when making a request.
It no longer looks at the USER_AGENT environment variable,
since you can now set the `User-Agent` header directly.
2017-10-15 22:11:38 -07:00
Albert Krewinkel
2262f005ce
Use hslua v0.9.0 2017-10-05 11:30:44 +02:00
John MacFarlane
fc25e3a5b7 Added ghc 8.2.1 to tested-with stanza. 2017-09-24 13:12:41 -07:00
John MacFarlane
1d6e651e5a Require pandoc-types 1.17.2. 2017-09-24 12:04:15 -07:00
John MacFarlane
04aba6bd51 Use skylighting 0.4. 2017-09-15 09:36:54 -07:00
John MacFarlane
64472a468c FromJSON/ToJSON instances for Reader, WriterOptions.
Depends on skylighting 0.3.5.
2017-09-14 22:38:23 -07:00
John MacFarlane
cbdeed9cfd Require skylighting 0.3.4.1 or higher. 2017-09-09 21:06:07 -07:00
John MacFarlane
3421f3eac7 Removed old beamer template.
We now use the default.latex template for both latex and beamer.
It contains conditionals for the beamer-specific things.

`pandoc -D beamer` will return this template.
2017-09-08 10:48:02 -07:00
John MacFarlane
de93744179 cabal: add custom-setup stanza, lowercase field names. 2017-09-07 10:13:40 -07:00
John MacFarlane
aa90919fe9 Add 'static' Cabal flag, use it for linux statically linked. 2017-09-04 17:07:20 -07:00
John MacFarlane
8fcf66453c RST reader: Fixed ..include:: directive.
Closes #3880.
2017-08-27 17:09:55 -07:00
John MacFarlane
d70b89c0d9 Use pandoc-types 1.17.1. Tests updated for new simpleTable behavior...
with empty headers.
2017-08-20 23:24:51 -07:00
John MacFarlane
b1f6fb4af5 HTML reader: support column alignments.
These can be set either with a `width` attribute or
with `text-width` in a `style` attribute.

Closes #1881.
2017-08-17 12:08:32 -07:00
Albert Krewinkel
9b31835530 Update to hslua-0.8.0
hslua no longer provides lua stack instances for Int and Double, the
necessary instances are added to the Custom writer and the lua filtering
system.
2017-08-16 15:47:05 +02:00
John MacFarlane
319d7ed6ff Changed command test for #2994 so it actually tests the writer. 2017-08-14 00:00:50 -07:00
Albert Krewinkel
3d87e2080a
Delete Text.Pandoc.Lua.SharedInstances
Stack instances for common data types are now provides by hslua. The
instance for Either was useful only for a very specific case; the
function that was using the `ToLuaStack Either` instance was rewritten
to work without it.

Closes: #3805
2017-08-13 17:48:43 +02:00
Albert Krewinkel
2dc3dbd68b Use hslua >= 0.7, update Lua code 2017-08-13 14:23:54 +02:00
John MacFarlane
418bda8128 Docx writer: pass through comments.
We assume that comments are defined as parsed by the
docx reader:

    I want <span class="comment-start" id="0" author="Jesse Rosenthal"
    date="2016-05-09T16:13:00Z">I left a comment.</span>some text to
    have a comment <span class="comment-end" id="0"></span>on it.

We assume also that the id attributes are unique and properly
matched between comment-start and comment-end.

Closes #2994.
2017-08-12 22:59:53 -07:00
John MacFarlane
622c3f2fa6 Change to yaml for translation files. 2017-08-12 12:17:38 -07:00
John MacFarlane
74212eb1b0 Added support for translations (localization) (see #3559).
* readDataFile, readDefaultDataFile, getReferenceDocx,
  getReferenceODT have been removed from Shared and
  moved into Class.  They are now defined in terms of
  PandocMonad primitives, rather than being primitve
  methods of the class.

* toLang has been moved from BCP47 to Class.

* NoTranslation and CouldNotLoudTranslations have
  been added to LogMessage.

* New module, Text.Pandoc.Translations, exporting
  Term, Translations, readTranslations.

* New functions in Class: translateTerm, setTranslations.
  Note that nothing is loaded from data files until
  translateTerm is used; setTranslation just sets the
  language to be used.

* Added two translation data files in data/translations.

* LaTeX reader: Support `\setmainlanguage` or `\setdefaultlanguage`
  (polyglossia) and `\figurename`.
2017-08-11 22:22:31 -07:00
John MacFarlane
ed4f6b348a Added data file to pandoc.cabal. 2017-08-10 18:26:06 -07:00
John MacFarlane
a5790dd308 RST reader: Basic support for csv-table directive.
* Added Text.Pandoc.CSV, simple CSV parser.
* Options still not supported, and we need tests.

See #3533.
2017-08-10 11:12:41 -07:00
John MacFarlane
14a44f53c8 Build config that works with lts-9.0 and nightly. 2017-08-09 12:14:35 -07:00
John MacFarlane
6e7ef6fbfc Don't put fancy warnings in pandoc.cabal ghc-options. 2017-08-08 22:44:47 -07:00
John MacFarlane
2f0bff0f54 Use extra ghc warnings.
In addition to `-Wall`:
 `-Wincomplete-uni-patterns -Wincomplete-record-updates -Wredundant-constraints -Wcompat -Wnoncanonical-monad-instances -Wnoncanonical-monadfail-instances`
2017-08-08 20:29:37 -07:00
John MacFarlane
91c989d622 Remove GFM modules; use CMarkGFM for both gfm and commonmark.
We no longer have a separate readGFM and writeGFM;
instead, we'll use readCommonMark and writeCommonMark
with githubExtensions.

It remains to implement these extensions conditionally.

Closes #3841.
2017-08-07 23:11:14 -07:00
John MacFarlane
2c81c4c218 Added gfm (GitHub-flavored CommonMark) as an input and output format.
This uses bindings to GitHub's fork of cmark, so it should parse
gfm exactly as GitHub does (excepting certain postprocessing
steps, involving notifications, emojis, etc.).

* Added Text.Pandoc.Readers.GFM (exporting readGFM)
* Added Text.Pandoc.Writers.GFM (exporting writeGFM)
* Added `gfm` as input and output forma

Note that tables are currently always rendered as HTML
in the writer; this can be improved when CMarkGFM supports
tables in output.
2017-08-07 16:59:31 -07:00
rlpowell
2ae75e23dd Added TikiWiki reader (#3800)
Added TikiWiki reader, including tests and documentation.

It's probably not *complete*, but it works pretty well, handles all
the basics (and some not-so-basics).
2017-07-21 10:09:54 +02:00
John MacFarlane
3a36441b61 Use foldrWithKey instead of deprecated foldWithKey. 2017-07-13 23:37:21 +02:00
John MacFarlane
0feb7504b1 Rewrote LaTeX reader with proper tokenization.
This rewrite is primarily motivated by the need to
get macros working properly.  A side benefit is that the
reader is significantly faster (27s -> 19s in one
benchmark, and there is a lot of room for further
optimization).

We now tokenize the input text, then parse the token stream.

Macros modify the token stream, so they should now be effective
in any context, including math. Thus, we no longer need the clunky
macro processing capacities of texmath.

A custom state LaTeXState is used instead of ParserState.
This, plus the tokenization, will require some rewriting
of the exported functions rawLaTeXInline, inlineCommand,
rawLaTeXBlock.

* Added Text.Pandoc.Readers.LaTeX.Types (new exported module).
  Exports Macro, Tok, TokType, Line, Column.  [API change]
* Text.Pandoc.Parsing: adjusted type of `insertIncludedFile`
  so it can be used with token parser.
* Removed old texmath macro stuff from Parsing.
  Use Macro from Text.Pandoc.Readers.LaTeX.Types instead.
* Removed texmath macro material from Markdown reader.
* Changed types for Text.Pandoc.Readers.LaTeX's
  rawLaTeXInline and rawLaTeXBlock.  (Both now return a String,
  and they are polymorphic in state.)
* Added orgMacros field to OrgState.  [API change]
* Removed readerApplyMacros from ReaderOptions.
  Now we just check the `latex_macros` reader extension.
* Allow `\newcommand\foo{blah}` without braces.

Fixes #1390.
Fixes #2118.
Fixes #3236.
Fixes #3779.
Fixes #934.
Fixes #982.
2017-07-07 12:36:00 +02:00
John MacFarlane
69388d52e3 Use latest texmath. 2017-06-30 20:39:53 +02:00
John MacFarlane
ac9423eccc Moved BCP47 specific functions from Writers.Shared to new module.
Text.Pandoc.BCP47 (unexported, internal module).
`getLang`, `Lang(..)`, `parseBCP47`.
2017-06-25 21:00:35 +02:00
John MacFarlane
9ac0a99292 Added mention of vimwiki raeder more places. 2017-06-20 16:57:39 +02:00
Yuchen Pei
564c77964d Added Vimwiki reader (#3705).
* New module Text.Pandoc.Readers.Vimwiki, exporting readVimwiki [API change].
* New input format `vimwiki`.
* New data file, `data/vimwiki.css`, for displaying the HTML produced by this reader and pandoc's HTML writer in the style of vimwiki's own HTML export.
2017-06-19 22:15:12 +02:00
John MacFarlane
aface549a0 Mention muse reader in README, MANUAL, debian control, cabal description. 2017-06-19 10:50:59 +02:00
Alexander Krotov
a91b9b2a1d Add Muse reader (#3620) 2017-06-19 10:46:02 +02:00
John MacFarlane
6166655b52 Allow QuickCheck 2.10 2017-06-17 23:58:12 +02:00
Bartosz Nitka
ec01d26a1b Relax time and process in preparation for GHC 8.2 (#3739)
GHC 8.2 is very likely to ship with process-1.6.0.0
and time-1.8.0.1.

Consult:
https://ghc.haskell.org/trac/ghc/wiki/Commentary/Libraries/VersionHistory
2017-06-17 23:48:01 +02:00
John MacFarlane
25bfa26554 Fixed weigh-pandoc for Text readers. 2017-06-10 21:10:52 +02:00
John MacFarlane
f43504f9f4 Fixed benchmark to work with Text readers. 2017-06-10 20:57:38 +02:00
John MacFarlane
610f1706b6 Remove cpphs build requirement -- it is no longer needed.
It was required when we used hsb2hs but no longer seemes
needed with file-embed.
2017-06-04 21:01:34 +02:00
John MacFarlane
afb551429b Update criterion upper bound. 2017-05-28 12:35:50 +02:00
Marc Schreiber
03cb05f4c6 Improve SVG image size code.
The old code made some unwise assumptions about
how the svg file would look.

See #3580.
2017-05-20 23:09:08 +02:00
Albert Krewinkel
a27e2e8a4e
Org reader: put tree parsing code into dedicated module 2017-05-16 22:42:34 +02:00
Albert Krewinkel
af4bf91c59
Org reader: add basic file inclusion mechanism
Support for the `#+INCLUDE:` file inclusion mechanism was added.
Recognized include types are *example*, *export*, *src*, and normal org
file inclusion.  Advanced features like line numbers and level selection
are not implemented yet.

Closes: #3510
2017-05-14 12:45:31 +02:00
John MacFarlane
d414b2543a Remove https flag.
Supporting two completely different libraries for fetching
from URLs makes it difficult to trap errors, because of
different error types expected from the libraries.

There's no clear reason not to build with these https-capable
libraires.
2017-05-07 12:49:25 +02:00
Albert Krewinkel
c2567b2bd0 API change: move writer functions to Text.Pandoc.Writers
Writer helper functions were defined in the top-level Text.Pandoc
module. These functions are moved to the Writer submodule as to enable
reuse in other submodules.
2017-04-26 23:28:40 +02:00
Albert Krewinkel
0e107a305f API change: move reader functions to Text.Pandoc.Readers
Reader helper functions were defined in the top-level Text.Pandoc
module. These functions are moved to the Readers submodule as to enable
reuse in other submodules.
2017-04-26 23:28:40 +02:00
John MacFarlane
b68135d34d Allow aeson 1.2.0.0. 2017-04-22 17:59:13 +02:00
John MacFarlane
e30a25701f Use skylighitng >= 0.3.3. 2017-04-22 11:15:09 +02:00
Albert Krewinkel
3aeed816e1
Lua filter: allow shorthand functions for math and quoted
Allow to use functions named `SingleQuoted`, `DoubleQuoted`,
`DisplayMath`, and `InlineMath` in filters.
2017-04-14 23:43:59 +02:00
Albert Krewinkel
0add4253e6
Avoid repeating StackValue instances definitions
The lua filters and custom lua writer system defined very similar
StackValue instances for strings and tuples.  These instance definitions
are extracted to a separate module to enable sharing.
2017-04-14 19:07:55 +02:00
Albert Krewinkel
feb1c1a930
Extract lua helper functions into Lua.Util module 2017-04-14 18:26:42 +02:00
Albert Krewinkel
0516b5127c
Drop dependency on hslua-aeson
Pushing values to the lua stack via custom functions is faster and more
flexible.
2017-04-14 11:21:12 +02:00
Albert Krewinkel
d412c38c71
Ensure correctness of StackValue instances 2017-04-06 21:00:38 +02:00
John MacFarlane
e62fbc1c3c Merge pull request #3550 from tarleb/lua-readers-submodule
Lua module: add readers submodule
2017-04-03 09:47:16 +02:00
John MacFarlane
ff991d1e21 Revert "Revert "Use file-embed instead of hsb2hs to embed data files.""
This reverts commit 1fa15c225b.
2017-04-02 23:10:10 +02:00
John MacFarlane
913db947a9 Text.Pandoc.App: Throw errors rather than exiting.
These are caught (and lead to exit) in pandoc.hs, but
other uses of Text.Pandoc.App may want to recover in another
way.

Added PandocAppError to PandocError (API change).
This is a stopgap:  later we should have a separate constructor
for each type of error.

Also fixed uses of 'exit' in Shared.readDataFile, and
removed 'err' from Shared (API change).

Finally, removed the dependency on extensible-exceptions.

See #3548.
2017-04-02 23:04:48 +02:00
Albert Krewinkel
e7eb21ecca
Lua module: add readers submodule
Plain text readers are exposed to lua scripts via the `pandoc.reader`
submodule, which is further subdivided by format.  Converting e.g. a
markdown string into a pandoc document is possible from within lua:

    doc = pandoc.reader.markdown.read_doc("Hello, World!")

A `read_block` convenience function is provided for all formats,
although it will still parse the whole string but return only the first
block as the result.

Custom reader options are not supported yet, default options are used
for all parsing operations.
2017-04-02 17:28:07 +02:00
John MacFarlane
420e3eb26e Allow a theme file as argument to --highlight-style.
Also include a sample, `default.theme`, in `data/`.
2017-04-01 22:27:00 +02:00
John MacFarlane
48039a1d80 Add JATS to description in pandoc.cabal. 2017-03-30 23:00:12 +02:00
John MacFarlane
6ad486c3c3 Automatically include URI-encoded jats.csl for jats output.
This way people can do

    pandoc -s -t jats --filter pandoc-citeproc

and it will just work.  If they want to specify a stylesheet,
they still can.
2017-03-30 16:43:17 +02:00
John MacFarlane
831e1c5edd Added JATS writer.
* New module Text.Pandoc.Writer.JATS exporting writeJATS.
* New output format `jats`.
* Added tests.
* Revised manual.
2017-03-30 01:16:34 +02:00
John MacFarlane
64fe39c255 Added default.jats template.
This is copied from Martin Fenner's pandoc-jats project:
https://github.com/mfenner/pandoc-jats
2017-03-28 09:38:00 +02:00
John MacFarlane
b2f3d8e759 Add auxiliary files for command test for #3530 to pandoc.cabal.
Otherwise builds fail.
2017-03-27 22:03:38 +02:00
John MacFarlane
1fa15c225b Revert "Use file-embed instead of hsb2hs to embed data files."
This reverts commit 10d91c1479.
2017-03-26 20:48:17 +02:00
John MacFarlane
10d91c1479 Use file-embed instead of hsb2hs to embed data files.
I think template haskell is robust enough now across platforms
that this will work.

Motivation: file-embed gives us better dependency tracking:  if a data
file changes, ghc/stack/cabal know to recompile the Data module.

This also removes hsb2hs as a build dependency.
2017-03-26 17:22:00 +02:00
Albert Krewinkel
1e13e98ecf
Ensure compatibility with hslua 0.5.*
The 0.5.0 release of hslua fixes problems with lua C modules on linux.
The signature of the `loadstring` function changed, so a compatibility
wrapper is introduced to allow both 0.4.* and 0.5.* versions to be used.
2017-03-24 21:31:35 +01:00
John MacFarlane
f0abbe7533 Allow creation of pdf via groff ms and pdfroff.
pandoc -t ms -o output.pdf input.txt
2017-03-23 21:24:01 +01:00
John MacFarlane
6c204ea2bd Initial addition of groff ms writer.
* New module: Text.Pandoc.Writers.Ms.
* New template: default.ms.
* The writer uses texmath's new eqn writer to convert math
  to eqn format, so a ms file produced with this writer
  should be processed with `groff -ms -e` if it contains
  math.
2017-03-23 10:14:16 +01:00
John MacFarlane
2d94d48332 Added two lua test files to extra-source-files in pandoc.cabal.
This was caught by our new .travis.yml, which builds from
an extracted sdist tarball instead of the repository.
2017-03-20 17:32:35 +01:00
Albert Krewinkel
f2f6851713 Lua filters (#3514)
* Add `--lua-filter` option.  This works like `--filter` but takes pathnames of special lua filters and uses the lua interpreter baked into pandoc, so that no external interpreter is needed.  Note that lua filters are all applied after regular filters, regardless of their position on the command line.
* Add Text.Pandoc.Lua, exporting `runLuaFilter`.  Add `pandoc.lua` to data files.
* Add private module Text.Pandoc.Lua.PandocModule to supply the default lua module.
* Add Tests.Lua to tests.
* Add data/pandoc.lua, the lua module pandoc imports when processing its lua filters.
* Document in MANUAL.txt.
2017-03-20 15:17:03 +01:00
John MacFarlane
090165d714 Added test for #256. 2017-03-16 22:31:36 +01:00
John MacFarlane
c93d069d49 Add default abbreviations file (data/abbreviations).
This contains a list of strings that will be recognized by pandoc's
Markdown parser as abbreviations.  (A nonbreaking space will
be inserted after the period, preventing a sentence space in
formats like LaTeX.)

Users can override the default by putting a file abbreviations
in their user data directory (`~/.pandoc` on *nix).
2017-03-16 22:16:41 +01:00
John MacFarlane
cc57e36180 Removed another unused cabal dependency. 2017-03-15 15:20:37 +01:00
John MacFarlane
befaf037ce Removed some unnecessary build-depends. 2017-03-15 15:12:14 +01:00
John MacFarlane
2235c2a8f7 Use tasty-golden for golden tests in Old. 2017-03-15 00:27:39 +01:00
John MacFarlane
6ecc5b96a9 Use tasty for tests rather than test-framework. 2017-03-14 17:07:23 +01:00
John MacFarlane
6bf3f89d69 Better handling of \part in LaTeX.
Closes #1905.

Removed stateChapters from ParserState.

Now we parse chapters as level 0 headers, and parts as level -1 headers.
After parsing, we check for the lowest header level, and if it's
less than 1 we bump everything up so that 1 is the lowest header level.
So `\part` will always produce a header; no command-line options
are needed.
2017-03-13 22:11:10 +01:00
Alexander Krotov
d037c5019d Add Muse writer (#3489)
* Add Muse writer

* Advertise new Muse writer

* Muse writer: add regressions tests
2017-03-10 10:16:27 +01:00
John MacFarlane
c46febaaee Expand \newenvironment macros.
Closes #987.

Depends on still unreleased texmath 0.9.3.
2017-03-10 09:46:32 +01:00
John MacFarlane
9925739ba9 Require skylighting 0.3.1. 2017-03-04 10:33:52 +01:00
John MacFarlane
81028e2dce Bumped syb upper bound. 2017-03-02 10:42:07 +01:00
John MacFarlane
da792c63ca Expose some unexposed modules.
+ Text.Pandoc.Writers.Shared
+ Text.Pandoc.Parsing
+ Text.Pandoc.Asciify
+ Text.Pandoc.Emoji
+ Text.Pandoc.ImageSize

[API change]

These are often helpful to people writing their own
reader or writer modules.  Closes #3260.
2017-02-25 22:07:01 +01:00
John MacFarlane
d199d58124 We need process >= 1.2.3 for readCreateProcessWithExitCode. 2017-02-20 16:51:47 +01:00
John MacFarlane
767bd77ec4 Updated test-with field. 2017-02-20 16:35:19 +01:00
John MacFarlane
59666f79f3 Use latest skylighting (0.3). 2017-02-20 13:31:15 +01:00
John MacFarlane
42e2ac8536 Use new skylighting-0.2. 2017-02-20 00:31:57 +01:00
John MacFarlane
7b6273362a pandoc.cabal: use cpphs if embed_data_files and darwin. 2017-02-12 19:59:47 +01:00
Alexander Krotov
48f809384a Add Org writer unit tests 2017-02-12 17:09:07 +03:00
John MacFarlane
a6c649cfc8 Added --log option to save log messages in JSON format to a file.
See #3392.
2017-02-11 09:59:54 +01:00