Commit graph

5060 commits

Author SHA1 Message Date
John MacFarlane
d1444b4ecd RST reader/writer: support unknown interpreted text roles...
...by parsing them as Span with "role" attributes.
This way they can be manipulated in the AST.

Closes #3407.
2017-08-17 16:01:44 -07:00
John MacFarlane
b1f6fb4af5 HTML reader: support column alignments.
These can be set either with a `width` attribute or
with `text-width` in a `style` attribute.

Closes #1881.
2017-08-17 12:08:32 -07:00
John MacFarlane
b9b35059f6 LaTeX reader: support \lq, \rq. 2017-08-17 12:08:32 -07:00
ickc
2a0e2e6fd6 slidy uses https instead of http (#3848)
grep -rl 'http://www.w3.org/Talks/Tools/Slidy2' . | xargs sed -i 's/http:\/\/www\.w3\.org\/Talks\/Tools\/Slidy2/https:\/\/www\.w3\.org\/Talks\/Tools\/Slidy2/g'
2017-08-17 10:11:49 -07:00
John MacFarlane
c175317d03 LaTeX reader: support \textquoteleft|right, \textquotedblleft|right.
Closes #3849.
2017-08-17 10:09:35 -07:00
John MacFarlane
ae61d5f57d LaTeX reader: rudimentary support for \hyperlink. 2017-08-16 10:56:16 -07:00
John MacFarlane
db715ca847 LaTeX reader: use Link instead of Span for \ref.
This makes more sense semantically and avoids unnecessary
Span [Link] nestings when references are resolved.
2017-08-16 10:56:12 -07:00
John MacFarlane
c6ec189a96 Revision to binary format output to stdout:
We now allow default output to stdout when it can be
determined that the output is being piped.  (On Windows,
as mentioned before, this can't be determined.)

Using '-o -' forces output to stdout regardless.
2017-08-16 10:39:34 -07:00
schrieveslaach
cf4b40162d LaTeX reader: add Support for glossaries and acronym package (#3589)
Acronyms are not resolved by the reader, but acronym and glossary information is put into attributes on Spans so that they can be processed in filters.
2017-08-16 10:24:46 -07:00
John MacFarlane
708bb8afe4 Fix import in PDF. 2017-08-16 10:13:16 -07:00
John MacFarlane
61cf3affa9 Change behavior with binary format output to stdout.
Previously, for binary formats, output to stdout was disabled
unless we could detect that the output was being piped (and not
sent to the terminal).  Unfortunately, such detection is not
possible on Windows, leaving windows users no way to pipe binary
output.  So we have changed the behavior in the following way:

* If the -o option is not used, binary output is never sent
  to stdout by default; instead, an error is raised.
* IF '-o -' is used, binary output is sent to stdout, regardless
  of whether it is being piped. This works on Windows too.
2017-08-16 09:45:12 -07:00
Albert Krewinkel
9b31835530 Update to hslua-0.8.0
hslua no longer provides lua stack instances for Int and Double, the
necessary instances are added to the Custom writer and the lua filtering
system.
2017-08-16 15:47:05 +02:00
John MacFarlane
f8b6a224ae Remove initial check for pdf creating program.
Instead, just try running it and raise the exception if it
isn't found at that point.

This improves things for users of Cygwin on Windows, where
the executable won't be found by `findExecutable` unless
`.exe` is added.

The same exception is raised as before, but at a later
point.

Closes #3819.
2017-08-15 21:17:20 -07:00
John MacFarlane
97fe6c35b5 Docx writer: fixed a regression (infinite loop on certain lists).
Bug was introduced by commit a868b238f2.
2017-08-15 14:05:10 -07:00
John MacFarlane
892a4edeb1 Implement multicolumn support for slide formats.
The structure expected is:

    <div class="columns">
      <div class="column" width="40%">
        contents...
      </div>
      <div class="column" width="60%">
        contents...
      </div>
    </div>

Support has been added for beamer and all HTML slide formats.

Closes #1710.

Note:  later we could add a more elegant way to create
this structure in Markdown than to use raw HTML div elements.
This would come for free with a "native div syntax" (#168).
Or we could devise something specific to slides
2017-08-14 23:17:44 -07:00
John MacFarlane
6aef1bd228 Better handle complex \def macros as raw latex. 2017-08-13 12:45:04 -07:00
John MacFarlane
425b731050 LaTeX reader: Allow @ as a letter in control sequences.
@ is commonly used in macros using `\makeatletter`.
Ideally we'd make the tokenizer sensitive to `\makeatletter`
and `\makeatother`, but until then this seems a good change.
2017-08-13 12:24:06 -07:00
John MacFarlane
bf9ec6dfd8 LaTeX reader: fix \let\a=0 case, with single character token. 2017-08-13 12:16:51 -07:00
John MacFarlane
f9656ece4e Resolve references to section numbers in LaTeX reader. 2017-08-13 11:48:44 -07:00
John MacFarlane
253a7c6201 LaTeX reader: track header numbers and correlate with labels. 2017-08-13 11:30:17 -07:00
schrieveslaach
2845ab5976 Put content of \ref, \label commands into span… (#3639)
* Put content of `\ref` and `\label` commands into Span elements so they can be used in filters.
* Add support for `\eqref`
2017-08-13 10:58:45 -07:00
John MacFarlane
8f65590ce9 CommonMark writer: prefer pipe tables to HTML tables...
...even if it means losing relative column width information.
See #3734.
2017-08-13 10:43:43 -07:00
John MacFarlane
506866ef73 Markdown writer: Use pipe tables if raw_html disabled...
and `pipe_tables` enabled, even if the table has relative
width information.

Closes #3734.
2017-08-13 10:37:24 -07:00
John MacFarlane
39066eba1d Added some Functor constraints needed for ghc 7.8. 2017-08-13 10:00:46 -07:00
Albert Krewinkel
3d87e2080a
Delete Text.Pandoc.Lua.SharedInstances
Stack instances for common data types are now provides by hslua. The
instance for Either was useful only for a very specific case; the
function that was using the `ToLuaStack Either` instance was rewritten
to work without it.

Closes: #3805
2017-08-13 17:48:43 +02:00
Albert Krewinkel
6e6cee454e Text.Pandoc.Lua: cleanup element walking code
WalkM is general enough to work in any monad, not just IO. Also get rid
of the LuaException type, sufficient to use the one defined in hslua.
2017-08-13 17:24:17 +02:00
Albert Krewinkel
b9c7adf02e
Text.Pandoc.Lua: Optimize performance by using raw table access
Raw table accessing functions never call back into haskell, which allows
the compiler to use more aggressive optimizations. This improves lua
filter performance considerably (⪆5% speedup).
2017-08-13 14:25:36 +02:00
Albert Krewinkel
2dc3dbd68b Use hslua >= 0.7, update Lua code 2017-08-13 14:23:54 +02:00
John MacFarlane
418bda8128 Docx writer: pass through comments.
We assume that comments are defined as parsed by the
docx reader:

    I want <span class="comment-start" id="0" author="Jesse Rosenthal"
    date="2016-05-09T16:13:00Z">I left a comment.</span>some text to
    have a comment <span class="comment-end" id="0"></span>on it.

We assume also that the id attributes are unique and properly
matched between comment-start and comment-end.

Closes #2994.
2017-08-12 22:59:53 -07:00
John MacFarlane
be9957bddc Escape MetaString values (as added with --metadata flag).
Previously they would be transmitted to the template without
any escaping.

Note that `--M title='*foo*'` yields a different result from

    ---
    title: *foo*
    ---

In the latter case, we have emphasis; in the former case, just
a string with literal asterisks (which will be escaped
in formats, like Markdown, that require it).

Closes #3792.
2017-08-12 20:27:42 -07:00
John MacFarlane
0ab8670a0e LaTeX reader: Fixed space after \figurename etc. 2017-08-12 13:40:28 -07:00
John MacFarlane
3897df868a LaTeX reader: support \chaptername, \partname, \abstractname, etc.
See #3559.
Obsoletes #3560.
2017-08-12 13:28:18 -07:00
John MacFarlane
f7346bbfc1 Added Listing to Term.
So far only added to English.
2017-08-12 13:19:50 -07:00
John MacFarlane
78e61cedd0 Added Encl, Glossary to Term 2017-08-12 13:14:27 -07:00
John MacFarlane
f035f0ffe3 LaTeX reader: have \setmainlanguage set lang in metadata. 2017-08-12 12:34:36 -07:00
John MacFarlane
444f8e9569 Improved error report on loading translation file. 2017-08-12 12:25:33 -07:00
John MacFarlane
622c3f2fa6 Change to yaml for translation files. 2017-08-12 12:17:38 -07:00
John MacFarlane
b6e0add76a Set user data dir at beginning, so readDataFile has access to it. 2017-08-12 12:15:40 -07:00
John MacFarlane
95f7dc6573 Fixed import. 2017-08-11 23:57:35 -07:00
John MacFarlane
1002178752 More redundant imports. 2017-08-11 23:24:55 -07:00
John MacFarlane
b3bb9a4384 Avoid some warnings when compiled wo/ embed_data_files flag. 2017-08-11 23:11:37 -07:00
John MacFarlane
6c5952e746 Add getFileName to PandocMonad.
We need this for getDefaultDataFile.
2017-08-11 23:09:51 -07:00
John MacFarlane
2c85c678f2 Removed redundant import. 2017-08-11 22:26:23 -07:00
John MacFarlane
74212eb1b0 Added support for translations (localization) (see #3559).
* readDataFile, readDefaultDataFile, getReferenceDocx,
  getReferenceODT have been removed from Shared and
  moved into Class.  They are now defined in terms of
  PandocMonad primitives, rather than being primitve
  methods of the class.

* toLang has been moved from BCP47 to Class.

* NoTranslation and CouldNotLoudTranslations have
  been added to LogMessage.

* New module, Text.Pandoc.Translations, exporting
  Term, Translations, readTranslations.

* New functions in Class: translateTerm, setTranslations.
  Note that nothing is loaded from data files until
  translateTerm is used; setTranslation just sets the
  language to be used.

* Added two translation data files in data/translations.

* LaTeX reader: Support `\setmainlanguage` or `\setdefaultlanguage`
  (polyglossia) and `\figurename`.
2017-08-11 22:22:31 -07:00
John MacFarlane
0283616a4a Remove writerUserDataDir from WriterOptions.
It is now carried in CommonState in PandocMonad instances.
(And thus it can be used by readers too.)
2017-08-10 23:46:59 -07:00
John MacFarlane
737d09e325 Removed datadir param from readDataFile and getDefaultTemplate.
In Text.Pandoc.Class and Text.Pandoc.Template, resp.
We now get the datadir from CommonState.
2017-08-10 23:37:44 -07:00
John MacFarlane
13abd97ac0 Class: add setUserDataDir and getUserDataDir.
* Add stUserDataDir to CommonState.
* Rename stUserDataDir/stCabalDataDir in PureState to
  stUserDataFiles/stCabalDataFiles.
2017-08-10 23:16:10 -07:00
John MacFarlane
6aaf8f4770 Expose getDefaultDataFile in both Shared and Class. 2017-08-10 23:04:14 -07:00
John MacFarlane
dee4cbc854 RST reader: implement csv-table directive.
Most attributes are supported, including `:file:` and `:url:`.
A (probably insufficient) test case has been added.

Closes #3533.
2017-08-10 15:01:14 -07:00
John MacFarlane
a5790dd308 RST reader: Basic support for csv-table directive.
* Added Text.Pandoc.CSV, simple CSV parser.
* Options still not supported, and we need tests.

See #3533.
2017-08-10 11:12:41 -07:00
John MacFarlane
e9eaf84215 Slidy writer: use h1 for all slides...
even if they were originally level 2 headers.
Otherwise the built-in table of contents in Slidy breaks.

Closes #3566.
2017-08-10 07:48:28 -07:00
John MacFarlane
f4bff5d359 RST reader: reorganize block parsers for ~20% faster parsing. 2017-08-09 21:16:17 -07:00
John MacFarlane
8c1ae7ddaf RST writer: don't wrap term in definition list.
Wrapping is not allowed.
2017-08-09 20:59:25 -07:00
John MacFarlane
1dcecffef4 Removed spurious comments. 2017-08-09 20:53:42 -07:00
John MacFarlane
2581f97620 EPUB writer: don't strip formatting in TOC.
Closes #1611.
2017-08-09 18:02:38 -07:00
John MacFarlane
6b72c5e35b Support svg in PDF output, converting with rsvg2pdf.
Closes #1793.
2017-08-09 12:13:49 -07:00
John MacFarlane
ac18ff90b2 Org reader: use org-language attribute rather than data-org-language. 2017-08-09 09:45:17 -07:00
John MacFarlane
96933c6043 Org reader: use tag-name attribute instead of data-tag-name. 2017-08-09 09:26:57 -07:00
John MacFarlane
09b7df472d LaTeX reader: Use label instead of data-label for label in caption.
See d441e656db, #3639.
2017-08-09 09:15:50 -07:00
bucklereed
db55f7c1b2 HTML reader: parse <main> like <div role=main>. (#3791)
* HTML reader: parse <main> like <div role=main>.

* <main> closes <p> and behaves like a block element generally
2017-08-09 09:10:12 -07:00
Alexander
cfa597fc2a Muse reader: simplify tableCell implementation (#3846) 2017-08-09 09:09:05 -07:00
John MacFarlane
606a8e2af4 RST reader: support :widths: attribute for table directive. 2017-08-08 20:48:30 -07:00
John MacFarlane
34d3f25e87 Parsing: added gobbleSpaces.
This is a utility function to use in list parsing.
2017-08-08 20:07:06 -07:00
John MacFarlane
b1c2ada4e4 Removed redundant import. 2017-08-08 16:46:12 -07:00
Alexander
b50de96502 Muse writer: insert two blanklines between lists of the same type (#3844) 2017-08-08 14:05:49 -07:00
John MacFarlane
a1cd7c3bfd Templates: Have gfm use commonmark template. 2017-08-08 14:05:04 -07:00
John MacFarlane
1ad9679dc9 CommonMark writer: avoid excess blank lines at end of output. 2017-08-08 14:00:13 -07:00
John MacFarlane
3752298d91 Thread options through CommonMark reader.
This is more efficient than doing AST traversals for
emojis and hard breaks.

Also make behavior sensitive to `raw_html` extension.
2017-08-08 13:55:19 -07:00
John MacFarlane
54658b923a Support hard_line_breaks in CommonMark reader. 2017-08-08 13:30:53 -07:00
John MacFarlane
b6f7c4930b CommonMark writer: support hard_line_breaks, smart.
Add tests.
2017-08-08 13:18:27 -07:00
John MacFarlane
ccd4f13a4a Writers.Shared.unsmartify: undo literal double curly quotes.
Previously we left these.
2017-08-08 13:16:45 -07:00
John MacFarlane
714d8a6377 CommonMark reader: support emoji extension. 2017-08-08 12:05:20 -07:00
John MacFarlane
73caf92871 CommonMark reader: support gfm_auto_identifiers.
Added `Ext_gfm_auto_identifiers`: new constructor for `Extension`
in `Text.Pandoc.Extensions` [API change].

Use this in githubExtensions.

Closes #2821.
2017-08-08 11:43:35 -07:00
John MacFarlane
312349bbcc CommonMark writer: Support pipe tables.
We bypass the commonmark writer from cmark and construct our
own pipe tables, with better results.  (Note also that cmark-gfm
currently doesn't support rendering table nodes; see
kivikakk/cmark-gfm-hs#3.)
2017-08-08 11:01:05 -07:00
John MacFarlane
56a680c305 CommonMark writer: support table, strikethrough extensions...
when enabled (as with gfm).  Note: because of limitations in
cmark-gfm, which will hopefully soon be corrected, this currently
gives an error on Tables.

Also properly support `--wrap=none`.
2017-08-08 09:14:13 -07:00
Alexander
c95cc813cc Fix `blanklines' documentation (#3843) 2017-08-08 08:44:18 -07:00
John MacFarlane
d752f85582 CommonMark reader: make exts depend on extensions. 2017-08-07 23:20:29 -07:00
John MacFarlane
91c989d622 Remove GFM modules; use CMarkGFM for both gfm and commonmark.
We no longer have a separate readGFM and writeGFM;
instead, we'll use readCommonMark and writeCommonMark
with githubExtensions.

It remains to implement these extensions conditionally.

Closes #3841.
2017-08-07 23:11:14 -07:00
John MacFarlane
6a9db1fde3 Issue deprecation warning for markdown_github.
Advise to use `gfm` instead.
2017-08-07 22:34:38 -07:00
John MacFarlane
834e9498f9 Logging: Added Deprecated constructor to LogMessage. 2017-08-07 22:34:23 -07:00
John MacFarlane
3bf11cae27 Man writer: removed some unneeded imports. 2017-08-07 22:04:08 -07:00
John MacFarlane
ea6be2e4e7 Added Deprecated warning to LogMessage. 2017-08-07 21:25:07 -07:00
John MacFarlane
5064241b24 Man writer: avoid error for def lists with no definitions.
Closes #3832.
2017-08-07 21:18:50 -07:00
John MacFarlane
2c0e989f9d Markdown reader: fixed spurious parsing as citation as reference def.
We now disallow reference keys starting with `@` if the
`citations` extension is enabled.  Closes #3840.
2017-08-07 21:00:57 -07:00
John MacFarlane
fd23b6dbce Revert "Logging: Made SkippedContent an INFO level message..."
This reverts commit 8995281691.
2017-08-07 17:07:20 -07:00
John MacFarlane
2c81c4c218 Added gfm (GitHub-flavored CommonMark) as an input and output format.
This uses bindings to GitHub's fork of cmark, so it should parse
gfm exactly as GitHub does (excepting certain postprocessing
steps, involving notifications, emojis, etc.).

* Added Text.Pandoc.Readers.GFM (exporting readGFM)
* Added Text.Pandoc.Writers.GFM (exporting writeGFM)
* Added `gfm` as input and output forma

Note that tables are currently always rendered as HTML
in the writer; this can be improved when CMarkGFM supports
tables in output.
2017-08-07 16:59:31 -07:00
John MacFarlane
8995281691 Logging: Made SkippedContent an INFO level message...
rather than WARNING.
2017-08-07 16:53:59 -07:00
John MacFarlane
190f36d2fd Small tweak to previous commit. 2017-08-07 16:11:13 -07:00
John MacFarlane
c806ef1b15 LaTeX reader: Support simple \def macros.
Note that we still don't support macros with fancy parameter
delimiters, like

    \def\foo#1..#2{...}
2017-08-07 16:06:19 -07:00
John MacFarlane
9e6b9cdc5f LaTeX reader: Support \let.
Also, fix regular macros so they're expanded at the
point of use, and NOT also the point of definition.
`\let` macros, by contrast, are expanded at the
point of definition.  Added an `ExpansionPoint`
field to `Macro` to track this difference.
2017-08-07 13:38:15 -07:00
John MacFarlane
3504915e63 LaTeX writer: Don't use figure inside table cell.
Closes #3836.
2017-08-06 22:31:15 -07:00
Alexander
1b5bfced55 Muse reader: debug indented paragraph support (#3839)
Take only first line indentation into account
and do not start new paragraph on indentation change.
2017-08-06 21:43:59 -07:00
Jesse Rosenthal
a36a56b8ac Docx reader: Avoid 0-level headers.
We used to parse paragraphs styled with "HeadingN" as "nth-level
header." But if a document has a custom style named "Heading0", this
will produce a 0-level header, which shouldn't exist. We only parse
this style if N>0. Otherwise we treat it as a normal style name, and
follow its dependencies, if any.

Closes #3830.
2017-08-06 19:35:03 -07:00
Alexander
8164a005c0 Muse reader: debug list and list item separation rules (#3837) 2017-08-06 13:19:59 -07:00
bucklereed
685788cd4b LaTeX reader: plainbreak, fancybreak et al from the memoir class (#3833) 2017-08-05 10:03:31 -07:00
Alexander Krotov
7a3a8790de Muse reader: do not allow headers in blockquotes (#3831) 2017-08-03 15:41:45 -07:00
Alexander Krotov
38b6adaac0 Muse reader: do not parse blocks inside comments (#3828) 2017-08-03 09:11:00 -07:00
John MacFarlane
ced834076d DokuWiki reader: better handling for code block in list item.
Closes #3824.
2017-08-02 10:33:08 -07:00
John MacFarlane
b8afec05e0 Markdown writer: better escaping of < and >.
If `all_symbols_escapable` is set, we backslash escape these.
Otherwise we use entities as before.
2017-07-30 13:45:22 -07:00
John MacFarlane
8cbc28415e Class: more haddocks. 2017-07-30 07:43:55 -07:00
John MacFarlane
7f9e950d8d Class: Removed unnecessary withMedia, improved haddocks. 2017-07-30 07:43:55 -07:00
Wandmalfarbe
200b5fb60c Sorted the list of supported HTML5 attributes and removed duplicates. (#3817) 2017-07-26 18:50:10 +02:00
John MacFarlane
e0ab09611a HTML writer: render raw inline environments when --mathjax used.
We previously did this only with raw blocks, on the assumption
that math environments would always be raw blocks. This has changed
since we now parse them as inline environments.

Closes #3816.
2017-07-26 12:50:36 +02:00
John MacFarlane
d441e656db HTML writer: insert data- in front of unsupported attributes.
Thus, a span with attribute 'foo' gets written to HTML5
with 'data-foo', so it is valid HTML5.

HTML4 is not affected.

This will allow us to use custom attributes in pandoc without
producing invalid HTML.
2017-07-25 13:13:24 +02:00
John MacFarlane
2b039acb4e Merge branch 'textcolor-support' of https://github.com/schrieveslaach/pandoc into schrieveslaach-textcolor-support 2017-07-25 11:42:10 +02:00
John MacFarlane
329b61ff5c LaTeX reader: support etoolbox's ifstrequal. 2017-07-24 11:20:59 +02:00
John MacFarlane
e7876d4320 LaTeX reader: removed code for non-exported 'macro'. 2017-07-24 00:25:35 +02:00
John MacFarlane
159d31e80f LaTeX reader: Removed 'macro'.
It is no longer necessary, since the rawLaTeXBlock parser
will parse macro definitions.

This also avoids the need for a separate latexMacro parser
in the Markdown reader.
2017-07-24 00:02:55 +02:00
John MacFarlane
be14e2b501 LaTeX reader: some improvements in macro parsing.
Fixed applyMacros so that it operates on the whole
string, not just the first token!

Don't remove macro definitions from the output,
even if Ext_latex_macros is set, so that macros will
be applied.  Since they're only applied to math in
Markdown, removing the macros can have bad effects.
Even for math macros, keeping them should be harmless.
2017-07-24 00:02:55 +02:00
John MacFarlane
d453b3319a Refactored some common code in LaTeX reader. 2017-07-24 00:02:54 +02:00
Mauro Bieg
7d9b782f73 HTML Reader: parse figure and figcaption (#3813) 2017-07-22 19:22:56 +02:00
John MacFarlane
f9309bc46e LaTeX reader: improved heuristic for raw block/inline.
An unknown command at the beginning of the line that could
be either block or inline is treated as block if we have
a sequence of block commands followed by a newline or a
`\startXXX` command (which might start a raw ConTeXt environment).
2017-07-21 23:27:54 +02:00
rlpowell
2ae75e23dd Added TikiWiki reader (#3800)
Added TikiWiki reader, including tests and documentation.

It's probably not *complete*, but it works pretty well, handles all
the basics (and some not-so-basics).
2017-07-21 10:09:54 +02:00
Alexander Krotov
335a1c7f48 Muse reader: fix reading of lists inside tags (#3802) 2017-07-21 10:04:13 +02:00
John MacFarlane
7191fe1f29 LaTeX reader: handle optional args in raw \titleformat.
Closes #3804.
2017-07-21 09:28:36 +02:00
John MacFarlane
992943d98e Templates: change signature of getDefaultTemplate.
Now it runs in any instance of PandocMonad, and returns
a String rather than an Either value.
2017-07-19 21:46:28 +02:00
John MacFarlane
2ce6b492e1 Class: started adding haddocks. 2017-07-19 21:31:46 +02:00
John MacFarlane
56f63af3f6 LaTeX reader: fixed regression with starred environment names.
Closes #3803.
2017-07-19 17:30:22 +02:00
John MacFarlane
dcf79c4188 Don't rely on listDirectory, which is only in newer versions...
of directory.
2017-07-15 11:38:43 +02:00
John MacFarlane
df5a00990e Class: make addToFileTree handle directories recursively. 2017-07-14 17:28:13 +02:00
John MacFarlane
f8fef1ebb0 Class: remove stFontFiles in PureState, 'glob' searches stFiles. 2017-07-14 16:35:29 +02:00
John MacFarlane
f140784870 Class: added addToFileTree 2017-07-14 16:24:30 +02:00
John MacFarlane
42fd536a0e Fix ghc 8.2.1 compiler warnings. 2017-07-14 11:17:46 +02:00
John MacFarlane
f6d97bc8cc Revert "Fixed some ghc 8.2 compiler warnings."
This reverts commit e22dc98a70.
2017-07-14 11:00:42 +02:00
John MacFarlane
e22dc98a70 Fixed some ghc 8.2 compiler warnings.
(Unnecessary type constraints.)
2017-07-14 09:57:18 +02:00
John MacFarlane
3a36441b61 Use foldrWithKey instead of deprecated foldWithKey. 2017-07-13 23:37:21 +02:00
schrieveslaach
911b63dfc3 Add LaTeX xspace support (#3797) 2017-07-13 20:56:59 +02:00
Marc Schreiber
f93d7d06f6 Merge branch 'master' of https://github.com/jgm/pandoc into textcolor-support 2017-07-13 11:51:40 +02:00
Yuchen Pei
8b502dd50f Fixed #3760. (#3784)
Using the same solution as in the LaTeX reader:
equation -> displaymath
align -> displaymath \begin{aligned} ... \end{aligned}
etc..
2017-07-12 17:19:49 +02:00
Alexander Krotov
de117fbd9e Muse writer: indent lists inside <quote> with at least one space (#3795) 2017-07-12 17:16:02 +02:00
Alexander Krotov
e0025cf4f1 Remove redundant imports (#3796) 2017-07-12 17:14:10 +02:00
John MacFarlane
050036c036 Print informative message when failing with use of --normalize.
We may want to think of some kind of graceful fallback, but
the present behavior has the advantage of forcing people to
update scripts when updating to pandoc 2.0.

See #3786.
2017-07-12 16:51:30 +02:00
John MacFarlane
013fd1c6b6 Make sure \write18 is parsed as raw LaTeX.
The change is in the LaTeX reader's treatment of raw commands,
but it also affects the Markdown reader.
2017-07-12 14:50:49 +02:00
John MacFarlane
41209ea676 HTML reader: Ensure that paragraphs are closed properly...
when the parent block element closes, even without `</p>`.

Closes #3794.
2017-07-11 15:52:38 +02:00
John MacFarlane
770e0cccc1 Use takeP in LaTeX reader. 2017-07-07 12:37:21 +02:00
John MacFarlane
6f6e83a06e Parsing: added takeP, takeWhileP for efficient parsing of [Char]. 2017-07-07 12:37:21 +02:00
John MacFarlane
0feb7504b1 Rewrote LaTeX reader with proper tokenization.
This rewrite is primarily motivated by the need to
get macros working properly.  A side benefit is that the
reader is significantly faster (27s -> 19s in one
benchmark, and there is a lot of room for further
optimization).

We now tokenize the input text, then parse the token stream.

Macros modify the token stream, so they should now be effective
in any context, including math. Thus, we no longer need the clunky
macro processing capacities of texmath.

A custom state LaTeXState is used instead of ParserState.
This, plus the tokenization, will require some rewriting
of the exported functions rawLaTeXInline, inlineCommand,
rawLaTeXBlock.

* Added Text.Pandoc.Readers.LaTeX.Types (new exported module).
  Exports Macro, Tok, TokType, Line, Column.  [API change]
* Text.Pandoc.Parsing: adjusted type of `insertIncludedFile`
  so it can be used with token parser.
* Removed old texmath macro stuff from Parsing.
  Use Macro from Text.Pandoc.Readers.LaTeX.Types instead.
* Removed texmath macro material from Markdown reader.
* Changed types for Text.Pandoc.Readers.LaTeX's
  rawLaTeXInline and rawLaTeXBlock.  (Both now return a String,
  and they are polymorphic in state.)
* Added orgMacros field to OrgState.  [API change]
* Removed readerApplyMacros from ReaderOptions.
  Now we just check the `latex_macros` reader extension.
* Allow `\newcommand\foo{blah}` without braces.

Fixes #1390.
Fixes #2118.
Fixes #3236.
Fixes #3779.
Fixes #934.
Fixes #982.
2017-07-07 12:36:00 +02:00
John MacFarlane
1dd769e558 Logging: added MacroAlreadyDefined. 2017-07-06 14:50:51 +02:00
John MacFarlane
20103ac2bc Allow ibooks-specific metadata in epubs. Closes #2693.
You can now have the following fields in your YAML metadata,
and it will be treated appropriately in the generated
EPUB.

```
ibooks:
  version: 1.3.4
  specified-fonts: false
  ipad-orientation-lock: portrait-only
  iphone-orientation-lock: landscape-only
  binding: true
  scroll-axis: vertical
```

This commit also fixes a regression in stylesheet paths.
2017-06-30 23:33:51 +02:00
John MacFarlane
d3dae1200a Removed hard_line_breaks extension from markdown_github.
GitHub has two Markdown modes, one for long-form documents like READMEs
and one for short things like issue coments. In issue comments, a line
break is treated as a hard line break. In README, wikis, etc., it is
treated as a space as in regular Markdown.

Since pandoc is more likely to be used to convert long-form
documents from GitHub Markdown, `-hard_line_breaks` is a better
default.

Closes #3594.
2017-06-30 22:26:42 +02:00
John MacFarlane
69b2cb38a8 Make east_asian_line_breaks affect all readers/writers.
Closes #3703.
2017-06-30 22:23:15 +02:00
John MacFarlane
e574d50b1c Markdown writer: Ensure that + and - are escaped properly...
so they don't cause spurious lists.  Previously they were only
if succeeded by a space, not if they were at end of line.

Closes #3773.
2017-06-30 17:41:25 +02:00
John MacFarlane
5e00cf8086 Added parameter for user data directory to runLuaFilter.
in Text.Pandoc.Lua.  Also to pushPandocModule.

This change allows users to override pandoc.lua with a file
in their local data directory, adding custom functions, etc.

@tarleb, if you think this is a bad idea, you can revert this.
But in general our data files are all overridable.
2017-06-29 17:13:19 +02:00
John MacFarlane
cb25326fa3 Text.Pandoc.Lua: more code simplification.
Also, now we check before running walkM that the function
table actually does contain something relevant.  E.g. if
your filter just defines Str, there's no need to run walkM
for blocks, meta, or the whole document. This should
help performance a bit (and it does, in my tests).
2017-06-29 17:07:30 +02:00
John MacFarlane
780a65f8a8 Lua filters: Remove special treatment of Quoted, Math.
No more SingleQuoted, DoubleQuoted, InlineMath, DisplayMath.
This makes everything uniform and predictable, though it does
open up a difference btw lua filters and custom writers.
2017-06-29 15:47:27 +02:00
John MacFarlane
5c80aca0e2 Text.Pandoc.Lua: refactored to remove duplicated code. 2017-06-29 14:31:02 +02:00
John MacFarlane
6ad74815f6 Text.Pandoc.Lua: use generics to reduce boilerplate.
I tested this with the str.lua filter on MANUAL.txt, and
I could see no significant performance degradation.

Doing things this way will ease maintenance, as we won't
have to manually modify this module when types change.

@tarleb, do we really need special cases for things like
DoubleQuoted and InlineMath?
2017-06-29 14:00:26 +02:00
John MacFarlane
2902260b63 Make papersize: a4 work regardless of the case of a4.
It is converted to `a4` in LaTeX and `A4` in ConTeXt.
2017-06-28 15:07:35 +02:00
Alexander Krotov
79cc56726c Muse reader: parse indented blockquotes (#3769) 2017-06-28 14:32:53 +02:00
John MacFarlane
cd690d0401 LaTeX writer: fixed detection of otherlangs.
We weren't recursing into inline contexts.

Closes #3770.
2017-06-28 14:20:53 +02:00
Albert Krewinkel
f5f8485923
Text.Pandoc.Lua: catch lua errors in filter functions
Replace lua errors with `LuaException`s.
2017-06-27 17:55:47 +02:00
Albert Krewinkel
4282abbd07
Text.Pandoc.Lua: keep element unchanged if filter returns nil
This was suggested by jgm and is consistent with the behavior of other
filtering libraries.
2017-06-27 17:11:42 +02:00
Albert Krewinkel
beb78a552c
Text.Pandoc.Lua: simplify filter function runner
The code still allowed to pass an arbitrary number of arguments to the
filter function, as element properties were passed as function arguments
at some point. Now we only pass the element as the single arg, so the
code to handle multiple arguments is no longer necessary.
2017-06-27 17:11:42 +02:00
John MacFarlane
7d9d77ca44 Require nonempty alt text for implicit_figures.
A figure with an empty caption doesn't make sense.

Closes #2844.
2017-06-27 15:25:37 +02:00
John MacFarlane
33a29fbf87 RST reader: support anchors.
E.g.

    `hello`

    .. _hello:

    paragraph

This is supported by putting "paragraph" in a Div with
id `hello`.

Closes #262.
2017-06-27 15:03:16 +02:00
John MacFarlane
563c9c8687 RST reader: Handle chained link definitions.
For example,

    .. _hello:
    .. _goodbye: example.com

Here both `hello` and `goodbye` should link to `example.com`.

Fixes the first part of #262.
2017-06-27 14:35:03 +02:00
John MacFarlane
a868b238f2 Docx writer: Allow 9 list levels.
Closes #3519.
2017-06-27 12:42:56 +02:00
bucklereed
460b6c470b HTML reader: Use the lang value of <html> to set the lang meta value. (#3765)
* HTML reader: Use the lang value of <html> to set the lang meta value.

* Fix for pre-AMP environments.
2017-06-27 10:19:37 +02:00
John MacFarlane
19d9482fc4 OpenDocument/ODT writer: Added support for table of contents.
Closes #2836.

Thanks to @anayrat.
2017-06-26 16:46:56 +02:00
John MacFarlane
75f4e41d7d Use table-of-contents for contents of toc, make toc a boolean.
Changed markdown, rtf, and HTML-based templates accordingly.

This allows you to set `toc: true` in the metadata; this
previously produced strange results in some output formats.

Closes #2872.

For backwards compatibility, `toc` is still set to the
toc contents.  But it is recommended that you update templates
to use `table-of-contents` for the toc contents and `toc`
for a boolean flag.
2017-06-26 16:20:09 +02:00
Alexander Krotov
fa515e46f3 Muse writer: fix hlint errors (#3764) 2017-06-26 15:07:45 +02:00
John MacFarlane
b2fe009d8f LaTeX writer: use BCP47 parser. 2017-06-26 15:04:22 +02:00
John MacFarlane
700a0843b2 parseBCP47: Parse extensions and private-use as variants.
Even though officially they aren't.  This suffices
for our purposes.
2017-06-26 15:03:51 +02:00
Yuchen Pei
f09473eab7 minor updates to vimwiki reader. (#3759)
- updated comments in Vimwiki.hs to reflect current status of
implementation
- added vimwiki to trypandoc
2017-06-26 08:41:51 +02:00
Alexander Krotov
492b3b1291 Muse reader: fix horizontal rule parsing (#3762)
Do not parse 3 dashes as horizontal rule and allow whitespace after rule
2017-06-26 08:41:17 +02:00
Alexander Krotov
b95f391beb Muse reader: simplify para implementation (#3761) 2017-06-26 08:40:53 +02:00
John MacFarlane
4cbbc9dd58 BCP47: split toLang from getLang, rearranged types. 2017-06-25 23:16:55 +02:00
John MacFarlane
d0d2443f2e Refactored ConTeXt writer to use BCP47.
BCP47 - consistent case for BCP47 fields (e.g. uppercase
for region).
2017-06-25 21:56:29 +02:00
John MacFarlane
ac9423eccc Moved BCP47 specific functions from Writers.Shared to new module.
Text.Pandoc.BCP47 (unexported, internal module).
`getLang`, `Lang(..)`, `parseBCP47`.
2017-06-25 21:00:35 +02:00
John MacFarlane
643cbdf104 Writers.Shared: improve type of Lang and bcp47 parser.
Use a real parsec parser for BCP47, include variants.
2017-06-25 18:31:59 +02:00
John MacFarlane
a85d833576 Fixed log message for InvalidLang. 2017-06-25 15:52:30 +02:00
John MacFarlane
e7cd3cb466 Writers.Shared: refactored getLang, splitLang...
into `Lang(..)`, `getLang`, `parceBCP47`.
2017-06-25 15:36:30 +02:00
John MacFarlane
3ae4105d14 Fixed support for lang attribute in OpenDocument and ODT writers.
This improves on the last commit, which didn't work in
some important ways.

See #1667.
2017-06-25 13:49:40 +02:00
John MacFarlane
083a224d1e Support lang attribute in OpenDocument and ODT writers.
This adds the required attributes to the temporary styles,
and also replaces existing language attributes in styles.xml.

Support for lang attributes on Div and Span has also been
added.

Closes #1667.
2017-06-25 12:46:26 +02:00
John MacFarlane
a02f08c9fc Added InvalidLang to LogMessage. 2017-06-25 12:46:26 +02:00
John MacFarlane
0c993a6c7b Text.Pandoc.Writers.Shared: export splitLang. 2017-06-25 12:46:26 +02:00
John MacFarlane
87e6235fe7 Text.Pandoc.Writers.Shared: added getLang. 2017-06-25 10:37:49 +02:00
Alexander Krotov
f8877516e0 Muse reader: Require space before and after '=' for code (#3758) 2017-06-25 10:01:43 +02:00
John MacFarlane
743419af5c Readers.getReader, Writers.getWriter API change.
Now these functions return a pair of a reader/writer and an
Extensions, instead of building the extensions into the
reader/writer.  The calling code must explicitly set
readerExtensions or writerExtensions using the Extensions
returned.

The point of the change is to make it possible for the
calling code to determine what extensions are being used.

See #3659.
2017-06-24 13:47:10 +02:00
John MacFarlane
a9259c1501 Extensions: Monoid instance for Extensions.
[API change]
2017-06-24 13:20:42 +02:00
John MacFarlane
a20302d9cf Added comment in source. 2017-06-23 22:36:43 +02:00
John MacFarlane
5812ac0390 Markdown reader: interpret YAML metadata as Inlines when possible.
If the metadata field is all on one line, we try to interpret
it as Inlines, and only try parsing as Blocks if that fails.

If it extends over one line (including possibly the `|` or
`>` character signaling an indented block), then we parse as
Blocks.

This was motivated by some German users finding that

    date: '22. Juin 2017'

got parsed as an ordered list.

Closes #3755.
2017-06-23 22:31:08 +02:00
John MacFarlane
57cc9a391c Markdown writer: make sure plain, markdown_github, etc. work for raw.
Previously only `markdown` worked.

Note: currently a raw block labeled `markdown_github` will
be printed for any `markdown` format.
2017-06-23 11:51:44 +02:00
John MacFarlane
da7d9ef295 HTML writer: make sure html4, html5 formats work for raw blocks/inlines. 2017-06-23 11:51:26 +02:00
John MacFarlane
2b34337a9c Text.Pandoc.Extensions: Added Ext_raw_attribute.
Documented in MANUAL.txt.

This is enabled by default in pandoc markdown and multimarkdown.
2017-06-23 00:37:13 +02:00
John MacFarlane
4a6868885d EPUB writer: put title_page.xhtml in text/. 2017-06-22 12:42:21 +02:00
John MacFarlane
2b3e8cb718 EPUB writer: Fixed various things with new EPUB structure. 2017-06-22 12:38:08 +02:00
John MacFarlane
24d215acf5 Added --epub-subdirectory option.
This specifies the subdirectory in the OCF container that
holds the EPUB specific content.

Closes #3720.
2017-06-22 12:01:33 +02:00
John MacFarlane
379b99f63a Added writerEpubSubdirectory to WriterOptions.
[API change]

The EPUB writer now takes its EPUB subdirectory from this option.

Also added `PandocEpubSubdirectoryError` to `PandocError`.
This is raised if the EPUB subdirectory is not all ASCII
alphanumerics.

See #3720.
2017-06-22 11:43:50 +02:00
John MacFarlane
242e2a064f Change default EPUB directory structure in OCF container.
See #3720.
We now put all EPUB related content in an EPUB/ subdirectory
by default (later this will be configurable).

    mimetype
    META-INF/
      com.apple.ibooks.display-options.xml
      container.xml
    EPUB/ <<--configurable-->>
      fonts/ <<--static-->>
      font.otf
    media/ <<--static-->>
      cover.jpg
      fig1.jpg
    styles/ <<--static-->>
      stylesheet.css
    content.opf
    toc.ncx
    text/ <<--static-->>
      ch001.xhtml
2017-06-21 23:54:16 +02:00
John MacFarlane
6e6324bade Removed an 'error' bomb. 2017-06-20 22:44:09 +02:00
John MacFarlane
c349f0b0ba Writers: adjusted for renderTemplate' changes.
Now we raise a proper error on template failure.
2017-06-20 22:43:48 +02:00
John MacFarlane
21c4281b13 Odt reader: replaced collectRights with rights from Data.Either. 2017-06-20 22:43:06 +02:00
John MacFarlane
c0a1286025 Text.Pandoc.Templates: change type of renderTemplate'.
Now it runs in PandocMonad and raises a proper
PandocTemplateError if there are problems, rather
than failing with uncatchable 'error'.
2017-06-20 22:41:56 +02:00
John MacFarlane
8f8f505fd4 Text.Pandoc.Error: added PandocTemplateError. 2017-06-20 22:41:34 +02:00
John MacFarlane
2363e6a15b Move CR filtering from tabFilter to the readers.
The readers previously assumed that CRs had been filtered
from the input.  Now we strip the CRs in the readers themselves,
before parsing.  (The point of this is just to simplify the
parsers.)

Shared now exports a new function `crFilter`. [API change]
And `tabFilter` no longer filters CRs.
2017-06-20 21:52:13 +02:00
John MacFarlane
4ba5ef46ae Updated code example. 2017-06-20 21:25:39 +02:00
John MacFarlane
5ec84bfeb4 Text.Pandoc.Lua - added DeriveDataTypeable for ghc 7.8. 2017-06-20 21:11:21 +02:00
Albert Krewinkel
f4c12606e1
Lua: use registry to store function references
Using the registry directly instead of a custom table is cleaner and
more efficient. The performance improvement is especially noticable when
filtering on frequent elements like Str.
2017-06-20 20:51:10 +02:00
Albert Krewinkel
bd5a7e5258
Lua: apply hslint suggestions 2017-06-20 19:20:50 +02:00
John MacFarlane
429c4620df Removed redundant import. 2017-06-20 16:44:05 +02:00
John MacFarlane
32f86067ec App: issue proper errors instead of using 'error'. 2017-06-20 16:19:59 +02:00
John MacFarlane
b78afbd980 Text.Pandoc.Lua: throw LuaException instead of using 'error'.
Text.Pandoc.App: trap LuaException and issue a PandocFilterError.
2017-06-20 16:19:59 +02:00
Alexander Krotov
2192528424 Muse reader: check that headers start at the first column (#3749) 2017-06-20 14:48:00 +02:00
John MacFarlane
ea1724e35e Docbook, JATS, TEI writers: print INFO message when omitting interior header.
This only applies to section headers inside list items, e.g.,
which were otherwise silently omitted.

See #3750.
2017-06-20 14:22:19 +02:00
John MacFarlane
b26d3c4522 FB2 writer: don't fail with an error on interior headers (e.g. in list).
Instead, omit them with an INFO message.

Closes #3750.
2017-06-20 14:21:43 +02:00
John MacFarlane
6a077ac9c7 Fixed footnotes in table captions.
Note that if the table has a first page header and a
continuation page header, the notes will appear only
on the first occurrence of the header.

Closes #2378.
2017-06-20 11:21:32 +02:00
John MacFarlane
328655e863 Tracing: give less misleading line information with parseWithString.
Previously positions would be reported past the end of the chunk.
We now reset the source position within the chunk and report
positions "in chunk."
2017-06-19 22:41:09 +02:00
John MacFarlane
b6a38ed111 Vimwiki reader: adjusted for changes in trace. 2017-06-19 22:29:01 +02:00
John MacFarlane
814ac51d32 Separated tracing from logging.
Formerly tracing was just log messages with a DEBUG log
level.  We now make these things independent.  Tracing
can be turned on or off in PandocMonad using `setTrace`;
it is independent of logging.

* Removed `DEBUG` from `Verbosity`.
* Removed `ParserTrace` from `LogMessage`.
* Added `trace`, `setTrace` to `PandocMonad`.
2017-06-19 22:17:43 +02:00
Alexander Krotov
4929d027dc Muse reader: fix list item continuation parsing (#3747) 2017-06-19 22:16:21 +02:00
Yuchen Pei
564c77964d Added Vimwiki reader (#3705).
* New module Text.Pandoc.Readers.Vimwiki, exporting readVimwiki [API change].
* New input format `vimwiki`.
* New data file, `data/vimwiki.css`, for displaying the HTML produced by this reader and pandoc's HTML writer in the style of vimwiki's own HTML export.
2017-06-19 22:15:12 +02:00
John MacFarlane
b3041de2fc Text.Pandoc.Writers.Math: export defaultMathJaxURL, defaultKaTeXURL.
This will ensure that we only need to update these in one place.
(Currently, for example, the mathjax URL is used in both
App and trypandoc.)

Closes #3685.
2017-06-19 11:10:29 +02:00
Alexander Krotov
a91b9b2a1d Add Muse reader (#3620) 2017-06-19 10:46:02 +02:00
John MacFarlane
ec3992b2f0 Use revealjs's math plugin for mathjax.
This is a thin wrapper around mathjax that makes math look better
on revealjs.

See https://github.com/hakimel/reveal.js/#mathjax

We do this by setting the 'mathjax' boolean variable and
using it in the revealjs template. Also, for revealjs
and mathjax, we don't assign the usual thing to the 'math'
variable, since it's handled by mathjax config.

Closes #3743.
2017-06-18 11:44:21 +02:00
John MacFarlane
abd2e94f5a In producing PDFs, warn if the font is missing some characters.
* Added `MissingCharacter` to `LogMessage` in Text.Pandoc.Logging.
* Parse the (xe)latex log for missing character warnings and issue
  the warning.

Closes #3742.
2017-06-18 11:17:00 +02:00
John MacFarlane
9849ba7fd7 Use Control.Monad.State.Strict throughout.
This gives 20-30% speedup and reduction of memory
usage in most of the writers.
2017-06-17 07:45:28 +02:00
schrieveslaach
635f299b44 Merge branch 'master' into textcolor-support 2017-06-12 15:52:29 +02:00
John MacFarlane
23f3c2d7b4 Changed "extracting..." warning to a regular log message.
This makes it sensitive to proper verbosity settings.
(It is now treated as INFO rather than WARNING, so one
doesn't get these messages for creation of tmp images
while making a pdf.)

API changes:

* Removed extractMediaBag from Text.Pandoc.MediaBag.
* Added Extracting as constructor for LogMessage.
2017-06-12 15:28:39 +02:00
John MacFarlane
8a000e3ecc Markdown writer: don't allow soft break in header.
Closes #3736.
2017-06-12 09:23:30 +02:00
John MacFarlane
b466152d61 Don't allow backslash + newline to affect block structure.
Note that as a result of this change, the following,
which formerly produced a header with two lines separated
by a line break, will now produce a header followed by a
paragraph:

    # Hi\
    there

This may affect some existing documents that relied on
this undocumented and unintended behavior.

This change makes pandoc more consistent with other
Markdown implementations, and with itself (since the two-space
version of a line break doesn't work inside ATX headers, and
neither version works inside Setext headers).

Closes #3730.
2017-06-11 22:24:20 +02:00
John MacFarlane
d1da54a4c3 Properly decode source from stdin.
This should fix the appveyor failures.
2017-06-11 21:22:44 +02:00
John MacFarlane
49b738de4e Rewrote HTML reader to use Text throughout.
- Export new NamedTag class from HTML reader.
- Effect on memory usage is modest (< 10%).
2017-06-11 21:15:58 +02:00
schrieveslaach
f36de77a25 Support for \faCheck and \faClose (#3727) 2017-06-11 07:47:42 +02:00
John MacFarlane
fa719d0264 Switched Writer types to use Text.
* XML.toEntities: changed type to Text -> Text.
* Shared.tabFilter -- fixed so it strips out CRs as before.
* Modified writers to take Text.
* Updated tests, benchmarks, trypandoc.

[API change]

Closes #3731.
2017-06-11 00:46:31 +02:00
John MacFarlane
0c2a509dfb Writers.Shared: metaToJSON, generalized type so it can take a Text.
Previously a String was needed as argument; now any ToJSON
instance will do.

API change.
2017-06-10 23:10:33 +02:00
John MacFarlane
e8cc9faa41 Writers: changed StringWriter -> TextWriter. 2017-06-10 21:54:26 +02:00
John MacFarlane
94b3dacb4e Changed all readers to take Text instead of String.
Readers: Renamed StringReader -> TextReader.

Updated tests.

API change.
2017-06-10 18:26:44 +02:00
John MacFarlane
d6822157e7 Readers: Changed StringReader -> TextReader. 2017-06-10 16:06:15 +02:00
John MacFarlane
d1e78d96b6 UTF8: export fromText, fromTextLazy. 2017-06-10 16:05:56 +02:00
John MacFarlane
627e27fc1e App: change readSource(s) to use Text instead of String. 2017-06-10 15:55:18 +02:00
John MacFarlane
c691b97506 UTF8: export toText, toTextLazy.
Define toString, toStringLazy in terms of them.
2017-06-10 15:54:35 +02:00
John MacFarlane
72b45f05ed Rewrote convertTabs to use Text not String. 2017-06-10 15:22:25 +02:00
Albert Krewinkel
55d679e382
Improve code style in lua and org modules 2017-06-03 13:35:19 +02:00
Albert Krewinkel
d55f01c65f
Org reader: apply hlint suggestions 2017-06-03 00:05:20 +02:00
John MacFarlane
b61a51ee15 hlint suggestions. 2017-06-02 15:25:39 +02:00
John MacFarlane
18f86a0c02 Fixed keywords in docx writer.
(See #3719)
2017-06-02 10:17:19 +02:00
John MacFarlane
a063ba58fd Merge pull request #3719 from iandol/patch-2
Add keywords metadata to docx core.xml document properties
2017-06-02 10:11:37 +02:00
John MacFarlane
e43ea03410 Fixed HTML reader. 2017-06-02 10:10:31 +02:00
Ian
b2fe1015d9 Add keywords metadata to docx document properties
Hi, I don't know haskell so possibly this is wrong, but DOCX stores keywords in cp:keywords in core.xml, and this should be easy to add from the pandoc metadata (I copy and paste the author code). As far as I can tell (no clear documentation, just a few refs), keywords should be separated with a comma.
2017-06-02 10:47:30 +08:00
John MacFarlane
eb6fb62e55 HTML reader: Use sets instead of lists for block tag lookup. 2017-06-01 19:15:00 +02:00
John MacFarlane
0f07404daf HTML reader: Removed "button" from block tag list.
It is already in the eitherBlockOrInlineTag list, and
should be both places.

Closes #3717.

Note: the result of this change is that there will be
p tags around the whole paragraph.  That is the right
result, because the `button` tags are treated as inline
HTML here, and the whole chunk of text is a Markdown
paragraph.
2017-06-01 18:59:21 +02:00
John MacFarlane
8218bdb95c HTML writer: Avoid two class attributes when adding 'uri' class.
Closes #3716.
2017-06-01 18:41:54 +02:00
John MacFarlane
0cf6511f16 Some hlint refactoring. 2017-06-01 15:09:38 +02:00
John MacFarlane
b1a9b567aa Trivial reformatting. 2017-06-01 14:19:43 +02:00
John MacFarlane
c2eb7d0857 Use isNothing. 2017-06-01 14:16:17 +02:00
John MacFarlane
00d8585d8f Trivial renaming. 2017-06-01 14:14:42 +02:00
John MacFarlane
c366fab2cb Markdown writer: Avoid inline surround-marking with empty content.
E.g. we don't want `<strong></strong>` to become `****`.
Similarly for emphasis, super/subscript, strikeout.

Closes #3715.
2017-06-01 12:30:58 +02:00
John MacFarlane
9396f1fb67 LaTeX reader: handle some width specifiers on table columns.
Currently we only handle the form `0.9\linewidth`.
Anything else would have to be converted to a percentage,
using some kind arbitrary assumptions about line widths.

See #3709.
2017-06-01 12:08:28 +02:00
John MacFarlane
af6e8414c7 LaTeX reader: more table refactoring. 2017-06-01 11:56:59 +02:00
John MacFarlane
58cfac84f0 LaTeX reader: Small refactoring of table parsing code.
This makes room for doing something with widths.
2017-06-01 11:35:03 +02:00
John MacFarlane
1e7ba5ccd7 LaTeX reader: Handle block structure inside table cells.
minipage is no longer required.

Closes #3709.
2017-06-01 11:16:28 +02:00
John MacFarlane
a61dce88e8 Merge pull request #3714 from tarleb/odt-reader-cleanup
Odt reader: remove dead code
2017-06-01 10:26:25 +02:00
Marc Schreiber
181c56d400 Add \colorbox support 2017-06-01 09:50:51 +02:00
Albert Krewinkel
e1a0666689
Org reader: respect export option for tags
Tags are appended to headlines by default, but will be omitted when the
`tags` export option is set to nil.

Closes: #3713
2017-05-31 21:26:07 +02:00
Albert Krewinkel
33a1e4ae1a
Org reader: include tags in headlines
The Emacs default is to include tags in the headline when exporting.
Instead of just empty spans, which contain the tag name as attribute,
tags are rendered as small caps and wrapped in those spans.
Non-breaking spaces serve as separators for multiple tags.
2017-05-31 20:43:30 +02:00
Albert Krewinkel
7852cd5603
Org reader: recognize babel result blocks with attributes
Babel result blocks can have block attributes like captions and names.
Result blocks with attributes were not recognized and were parsed as
normal blocks without attributes.

Fixes: #3706
2017-05-31 20:01:04 +02:00
Albert Krewinkel
4b98d0459a
Org reader: fix module names in haddock comments
Copy-pasting had lead to haddock module descriptions containing the
wrong module names.
2017-05-31 20:01:04 +02:00
Albert Krewinkel
f955af58e6
Odt reader: remove dead code
The ODT reader contained a lot of general code useful for working with
arrows. However, many of these utils weren't used and are hence removed.
2017-05-31 19:59:34 +02:00
John MacFarlane
774075c3e2 Added eastAsianLineBreakFilter to Shared.
This used to live in the Markdown reader.
2017-05-30 10:22:48 +02:00
John MacFarlane
5ec384eb60 LaTeX reader: handle escaped & inside table cell.
Closes #3708.
2017-05-29 22:47:04 +02:00
John MacFarlane
230a1b89e8 LaTeX reader: don't crash on empty enumerate environment.
Closes #3707.
2017-05-29 15:09:24 +02:00
John MacFarlane
d461b29d9d Merge pull request #3704 from labdsf/anylinenewline
Markdown reader: use anyLineNewline
2017-05-29 09:25:53 +02:00
Alexander Krotov
efc069de5d Markdown reader: use anyLineNewline 2017-05-28 22:52:35 +03:00
Herwig Stuetz
bfd5c6b172 Org reader: Fix cite parsing behaviour
Until now, org-ref cite keys included special characters also at the
end. This caused problems when citations occur right before colons or
at the end of a sentence.

With this change, all non alphanumeric characters at the end of a cite
key are ignored.

This also adds `,` to the list of special characters that are legal
in cite keys to better mirror the behaviour of org-export.
2017-05-28 18:08:11 +02:00
Herwig Stuetz
5a71632d11 Parsing: many1Till: Check for the end condition before parsing
By not checking for the end condition before the first parse, the
parser was applied too often, consuming too much of the input.

This fixes the behaviour of

  `testStringWith (many1Till (oneOf "ab") (string "aa")) "aaa"`

which before incorrectly returned `Right "a"`. With this change, it
instead correctly fails with `Left (PandocParsecError ...)` because it
is not able to parse at least one occurence of `oneOf "ab"` that is
not `"aa"`.

Note that this only affects `many1Till p end` where `p` matches on a
prefix of `end`.
2017-05-28 18:08:11 +02:00
Alexander Krotov
c38d5966ed RST reader: use anyLineNewline in rawListItem (#3702) 2017-05-28 09:29:37 +02:00
John MacFarlane
8614902234 Markdown writer: changes to --reference-links.
With `--reference-location` of `section` or `block`, pandoc
will now repeat references that have been used in earlier
sections.

The Markdown reader has also been modified, so that *exactly*
repeated references do not generate a warning, only
references with the same label but different targets.

The idea is that, with references after every block,
one  might want to repeat references sometimes.

Closes #3701.
2017-05-27 23:18:45 +02:00
John MacFarlane
4dabcc27f6 Pretty: Eq instance for Doc. 2017-05-27 23:18:45 +02:00
Albert Krewinkel
bf93c07267
Org reader: subject full doc tree to headline transformations
Emacs parses org documents into a tree structure, which is then
post-processed during exporting. The reader is changed to do the same,
turning the document into a single tree of headlines starting at
level 0.

Fixes: #3695
2017-05-27 15:38:08 +02:00
John MacFarlane
8ec03cfc87 HTML writer: Removed unused parameter in dimensionsToAttributeList. 2017-05-26 10:21:55 +02:00
John MacFarlane
cb7b0a6985 Allow em for image height/width in HTML, LaTeX.
- Export `inEm` from ImageSize [API change].
- Change `showFl` and `show` instance for `Dimension` so
  extra decimal places are omitted.
- Added `Em` as a constructor of `Dimension` [API change].
- Allow `em`, `cm`, `in` to pass through without conversion
  in HTML, LaTeX.

Closes #3450.
2017-05-25 22:48:27 +02:00
John MacFarlane
708973a33a Added spaced_reference_links extension.
This is now the default for pandoc's Markdown.
It allows whitespace between the two parts of a
reference link:  e.g.

    [a] [b]

    [b]: url

This is now forbidden by default.

Closes #2602.
2017-05-25 12:57:31 +02:00
John MacFarlane
650e1ac1fd Docx writer: Use Table rather than "Table Normal" for table style.
"Table Normal" is the default table style and can't be modified.

Closes #3275, further testing welcome.
2017-05-25 12:11:46 +02:00
John MacFarlane
8f2c803f97 Markdown reader: warn for notes defined but not used.
Closes #1718.

Parsing.ParserState: Make stateNotes' a Map, add stateNoteRefs.
2017-05-25 11:34:51 +02:00
John MacFarlane
41db9e826e MediaWiki reader: don't do curly quotes inside <tt> contexts.
Even if `+smart`.

See #3585.
2017-05-25 09:35:25 +02:00
John MacFarlane
e6f4636a2c MediaWiki reader: Make smart double quotes depend on smart extension.
Closes #3585.
2017-05-25 09:19:34 +02:00
John MacFarlane
b9a30ef959 Markdown reader: fixed smart quotes after emphasis.
E.g. in

    *foo*'s 'foo'

Closes #2228.
2017-05-24 23:23:08 +02:00
John MacFarlane
8f718b0883 LaTeX reader: Fixed failures on \ref{}, \label{} with +raw_tex.
Now these commands are parsed as raw if `+raw_tex`;
otherwise, their argument is parsed as a bracketed string.
2017-05-24 23:04:49 +02:00
John MacFarlane
bc6aac7b47 Parsing: Provide parseFromString'.
This is a verison of parseFromString specialied to
ParserState, which resets stateLastStrPos at the end.
This is almost always what we want.

This fixes a bug where `_hi_` wasn't treated as emphasis in
the following, because pandoc got confused about the
position of the last word:

    - [o] _hi_

Closes #3690.
2017-05-24 22:41:47 +02:00
John MacFarlane
1288a50380 LaTeX reader: parse tikzpicture as raw verbatim environment...
if `raw_tex` extension is selected.
Otherwise skip with a warning.

This is better than trying to parse it as text!

Closes #3692.
2017-05-24 21:46:53 +02:00
John MacFarlane
19d3a2bbe5 Logging: Made SkippedContent WARNING not INFO. 2017-05-24 21:46:43 +02:00
John MacFarlane
7174776c19 HTML reader: Add details tag to list of block tags.
Closes #3694.
2017-05-24 12:11:12 +02:00
Marc Schreiber
29a4bdc681 Add suggestions of @jgm: parse bracketed stuff as inlines 2017-05-23 17:31:42 -03:00
John MacFarlane
5844af67b4 RST reader: reformatting (code line length). 2017-05-23 21:00:51 +02:00
keiichiro shikano
c0c54b7906 RST Reader: parse list table directive (#3688)
Closes #3432.
2017-05-23 20:53:04 +02:00
John MacFarlane
8edeaa9349 Fixed handling of soft hyphen (0173) in docx writer.
Closes #3691.
2017-05-23 16:58:24 +02:00
John MacFarlane
66fa38ed1c Shared.isURI: allow uppercase versions of known schemes. 2017-05-23 09:49:56 +02:00
Albert Krewinkel
5debb0da0f Shared: Provide custom isURI that rejects unknown schemes [isURI]
We also export the set of known `schemes`.

The new function replaces the function of the same name
from `Network.URI`, as the latter did not check whether a scheme is
well-known.  E.g. MediaWiki wikis frequently feature pages with names
like `User:John`. These links were interpreted as URIs, thus turning
internal links into global links. This is prevented by also checking
whether the scheme of a URI is frequently used (i.e. is IANA registered
or an otherwise well-known scheme).

Fixes: #2713

Update set of well-known URIs from IANA list
All official IANA schemes (as of 2017-05-22) are included in the set of
known schemes.  The four non-official schemes doi, isbn, javascript, and
pmid are kept.
2017-05-23 09:48:11 +02:00
John MacFarlane
4d1e9b8e41 Let --eol take native as an argument.
Add `Native` to the `LineEnding` type.
Make `optEol` a `Native` rather than `Maybe Native`.
2017-05-22 10:15:03 +02:00
Alexander Krotov
30a3deadcc Move indentWith to Text.Pandoc.Parsing (#3687) 2017-05-22 10:10:15 +02:00
John MacFarlane
aa1e39858d Text.Pandoc.App: ToJSON and FromJSON instances for Opts.
This can be used e.g. to pass options via web interface,
such as trypandoc.
2017-05-21 11:42:50 +02:00
John MacFarlane
8c1b81bbef Finished implemtation of --resource-path.
* Default is just working directory.
* Working directory must be explicitly specifide if
  `--resource-path` option is used.
2017-05-21 09:02:01 +02:00
John MacFarlane
6a7f980247 PDF: Got --resource-path working with pdf output.
See #852.
2017-05-20 23:46:51 +02:00
John MacFarlane
d109c8be8f PDF: better error message for non-converted svg images. 2017-05-20 23:24:20 +02:00
Alexander Krotov
753d5811e2 RST reader: make use of anyLineNewline (#3686) 2017-05-20 23:14:08 +02:00
Marc Schreiber
03cb05f4c6 Improve SVG image size code.
The old code made some unwise assumptions about
how the svg file would look.

See #3580.
2017-05-20 23:09:08 +02:00
John MacFarlane
5c44fd554f PDF: Refactoring, makePDF is now in PandocIO [API change]. 2017-05-20 22:42:50 +02:00
John MacFarlane
fd6e65b00f Added --resource-path=SEARCHPATH command line option.
SEARCHPATH is separated by the usual character,
depending on OS (: on unix, ; on windows).

Note: This does not yet work for PDF output, because the
routine that creates PDFs runs outside PandocMonad.
(This has to do with its use of inTemporaryDirectory and
its interaction with our exceptions.)

The best solution would be to figure out how to move the
PDF creation routines into PandocMonad.  Second-best,
just pass an extra parameter in?

See #852.
2017-05-20 21:47:10 +02:00
John MacFarlane
93eaf33e6e SelfContained: handle @import with quoted string. 2017-05-20 17:32:46 +02:00
John MacFarlane
8d4fbe6a2a SelfContained: fixed problem with embedded fonts.
Closes #3629.

However, there is still room for improvement.

`@import` with following media declaration is not
handled.

Also `@import` with a simple filename (rather than
`url(...)` is not handled.
2017-05-20 17:09:47 +02:00
John MacFarlane
ca77f0a95e RST writer: add empty comments when needed...
to avoid including a blocquote in the indented content
of a preceding block.

Closes #3675.
2017-05-19 21:05:15 +02:00
Albert Krewinkel
7a09b7b21d
Org reader: fix smart parsing behavior
Parsing of smart quotes and special characters can either be enabled via
the `smart` language extension or the `'` and `-` export options. Smart
parsing is active if either the extension or export option is enabled.
Only smart parsing of special characters (like ellipses and en and em
dashes) is enabled by default, while smart quotes are disabled.

This means that all smart parsing features will be enabled by adding the
`smart` language extension. Fine-grained control is possible by leaving
the language extension disabled. In that case, smart parsing is
controlled via the aforementioned export OPTIONS only.

Previously, all smart parsing was disabled unless the language extension
was enabled.
2017-05-18 23:25:11 +02:00
John MacFarlane
f870a2d8ea Don't render LaTeX images with data: URIs.
LaTeX can't handle these.

Note that --extract-media can be used when the input contains
data: URIs.  Closes #3636.
2017-05-18 22:50:07 +02:00
Ian
b9185b0216 Docx writer: Change FigureWithCaption to CaptionedFigure (#3658)
Edit styles.xml as part of the fix for #3656
2017-05-18 22:34:13 +02:00
John MacFarlane
0f6458c0c1 Don't double extract images from docx.
This fixes a regression that was introduced when `--extract-media`
was generalized to work with any input format.  We were getting
two versions of each image extracted from a docx, one with a hash,
one with the original filename, though only the hash one was used.
This patch restores the original behavior (using the original
filename).

Pointed out in comments on #3674. Thanks to @laperouse.
2017-05-18 13:38:19 +02:00
John MacFarlane
818d5c2f35 Markdown: allow attributes in reference links to start on next line.
This addresses a subsidiary issue in #3674.
2017-05-18 13:20:32 +02:00
Stefan Dresselhaus
6b8240fc2f Add --eol flag and writer option to control line endings.
* Add `--eol=crlf|lf` CLI option.
* Add `optEol` to `WriterOptions` [API change]
* In `Text.Pandoc.UTF8`, add new functions parameterized on `Newline`:
  `writeFileWith`, `putStrWith`, `putStrLnWith`, `hPutStrWith`,
  `hPutStrLnWith`. [API change]
* Document option in MANUAL.txt.

Closes #3663.
Closes #2097.
2017-05-18 11:55:45 +02:00
John MacFarlane
7b3aaee15a Markdown writer: Fixed duplicated reference links
with `--reference-links` and `--reference-location=section`.
Also ensure that there are no empty link references `[]`.

Closes #3674.
2017-05-17 16:23:33 +02:00
John MacFarlane
61e965b117 Merge pull request #3676 from labdsf/space-char
Txt2Tags parser: newline is not indentation
2017-05-17 12:47:27 +02:00
John MacFarlane
377733e08f Merge pull request #3677 from labdsf/anylinenewline
Move anyLineNewline to Parsing.hs
2017-05-17 12:47:03 +02:00
Alexander Krotov
55ce47d050 Move anyLineNewline to Parsing.hs 2017-05-17 11:02:38 +03:00
Alexander Krotov
e74bd06cc8 Txt2Tags parser: newline is not indentation
space parses '\n', while spaceChar parses only ' ' and '\t'
2017-05-17 02:12:24 +03:00
Albert Krewinkel
602cd6a327
Org reader: replace sequence . map with mapM 2017-05-16 22:49:52 +02:00
Albert Krewinkel
a27e2e8a4e
Org reader: put tree parsing code into dedicated module 2017-05-16 22:42:34 +02:00
John MacFarlane
fbce4228a5 Merge pull request #3671 from WUUUGI/horizont-spacing
Added support for horizontal spacing in LaTeX
2017-05-16 09:18:57 +02:00
John MacFarlane
37189667cc Textile reader: fix bug for certain links in table cells.
Closes #3667.
2017-05-15 20:36:11 +02:00
Henri Werth
2de5208311 Added support for horizontal spacing in LaTeX: parse \, to \8198 (six-per-em space) 2017-05-15 16:37:08 +02:00
Albert Krewinkel
af4bf91c59
Org reader: add basic file inclusion mechanism
Support for the `#+INCLUDE:` file inclusion mechanism was added.
Recognized include types are *example*, *export*, *src*, and normal org
file inclusion.  Advanced features like line numbers and level selection
are not implemented yet.

Closes: #3510
2017-05-14 12:45:31 +02:00
Albert Krewinkel
9d295f4527
Parsing: add insertIncludedFilesF which returns F blocks
The `insertIncludeFiles` function was generalized and renamed to
`insertIncludedFiles'`; the specialized versions are based on that.
2017-05-14 12:40:16 +02:00
Albert Krewinkel
5ff6108b4c
Parsing: introduce HasIncludeFiles type class
The `insertIncludeFile` function is generalized to work with all parser
states which are instances of that class.
2017-05-14 10:00:58 +02:00
Albert Krewinkel
7a17c3eb9f
Parsing: replace partial with total function
Calling `tail` on an empty list raises an exception, while calling the
otherwise equivalent `drop 1` will return the empty list again.
2017-05-14 09:28:08 +02:00
Albert Krewinkel
965f1ddd4a
Update dates in copyright notices
This follows the suggestions given by the FSF for GPL licensed software.
<https://www.gnu.org/prep/maintain/html_node/Copyright-Notices.html>
2017-05-13 23:30:13 +02:00
Albert Krewinkel
1cbb3bad2b
Add haddock module description to FB2 writer
Copyright, maintainer etc. were missing in haddock docs for this module.
2017-05-13 23:18:50 +02:00
Albert Krewinkel
62d34c79b9
Change maintainer line in Org writer module 2017-05-13 23:18:10 +02:00
Alexander Krotov
2a291e437a Replace repeat' and take' with `replicate' once more 2017-05-12 16:31:57 +02:00
Albert Krewinkel
4b9fb7a128 Combine grid table parsers
The grid table parsers for markdown and rst was combined into one single
parser, slightly changing parsing behavior of both parsers:

- The markdown parser now compactifies block content cell-wise: pure
  text blocks in cells are now treated as paragraphs only if the cell
  contains multiple paragraphs, and as plain blocks otherwise. Before,
  this was true only for single-column tables.

- The rst parser now accepts newlines and multiple blocks in header
  cells.

Closes: #3638
2017-05-11 00:17:56 +02:00
David A Roberts
cff6d2dd73 Markdown writer: missing \n (#3647) 2017-05-07 23:11:57 +02:00
Alexander Krotov
69110cde81 Muse writer: Indent tables with one space (#3649)
It is required to trigger Muse table rendering.
2017-05-07 21:41:38 +02:00
John MacFarlane
6b086acae8 Rename fillMedia -> fillMediaBag. 2017-05-07 21:03:18 +02:00
John MacFarlane
a902109c6d PDF: use fillMedia and extractMedia to extract media to tmp dir.
This reduces code duplication.
We should be able to do something similar in ODT, Docx, EPUB writers.
2017-05-07 20:57:16 +02:00
John MacFarlane
af7215a048 Moved fillMedia, extractMedia from App to Class.
Also generalized type of fillMedia to any instance of PandocMonad.
2017-05-07 20:42:32 +02:00
John MacFarlane
99be906101 Added PandocHttpException, trap exceptions in fetching from URLs.
Closes #3646.
2017-05-07 13:11:04 +02:00
John MacFarlane
d414b2543a Remove https flag.
Supporting two completely different libraries for fetching
from URLs makes it difficult to trap errors, because of
different error types expected from the libraries.

There's no clear reason not to build with these https-capable
libraires.
2017-05-07 12:49:25 +02:00
John MacFarlane
f8e125f42d fillMediaBag: don't cause fatal error if resource not found.
Report warning instead and change image to its alt text.
2017-05-07 12:16:14 +02:00
John MacFarlane
400fe3188e Allow --extract-media to work with non-binary input formats.
If `--extract-media` is supplied with a non-binary input format,
pandoc will attempt to extract the contents of all linked images,
whether in local files, data: uris, or external uris.

They will be named based on the sha1 hash of the contents.

Closes #1583, #2289.

Notes:

- One thing that is slightly subideal with this commit is that
  identical resources will be downloaded multiple times.  To improve
  this we could have mediabag store an original filename/url +
  a new name.

- We might think about reusing some of this code, since more or less the
  same thing is done in the Docx, EPUB, PDF writers (with slight
  variations).
2017-05-07 11:45:33 +02:00
John MacFarlane
e15a4badff Simplify plumbing for document transformation. 2017-05-07 10:34:04 +02:00
John MacFarlane
82cc7fb0d4 Markdown reader: improved parsing of indented raw HTML blocks.
Previously we inadvertently interpreted indented HTML as
code blocks.  This was a regression.

We now seek to determine the indentation level of the contents
of an HTML block, and (optionally) skip that much indentation.

As a side effect, indentation may be stripped off of raw
HTML blocks, if `markdown_in_html_blocks` is used. This
is better than having things interpreted as indented code
blocks.

Closes #1841.
2017-05-06 22:56:16 +02:00
John MacFarlane
f20c89e243 LaTeX reader: Better handling of comments inside math environments.
This solves a problem with commented out `\end{eqnarray}` inside
an eqnarray (among other things).

Closes #3113.
2017-05-06 22:16:43 +02:00
schrieveslaach
ddf2524477 Fix keyval funtion: pandoc did not parse options in braces correctly.… (#3642)
* Fix keyval funtion: pandoc did not parse options in braces correctly. Additionally, dot, dash, and colon were no valid characters

* Add | as possible option value

* Improved code
2017-05-06 15:09:29 +02:00
Albert Krewinkel
bf44b88522
Drop redundant import of sort
This was left in accidentally.
2017-05-06 11:32:38 +02:00
Albert Krewinkel
da8c153a68
Org reader: support macros
Closes: #3401
2017-05-06 11:00:32 +02:00
John MacFarlane
9f0a80457f Revert "SelfContained: special handling for css @import."
This reverts commit 89b3fcc8e0.
2017-05-05 23:23:49 +02:00
John MacFarlane
89b3fcc8e0 SelfContained: special handling for css @import.
We now avoid creating a data URI for the url under an
@import.
2017-05-05 23:03:31 +02:00
John MacFarlane
c1b45adda0 SelfContained: Handle url() inside material retrieved from url().
This can happen e.g. with an @import of a google web font.
(What is imported is some CSS which contains an url reference
to the font itself.)

Also, allow unescaped pipe (|) in URL.

This is intended to help with #3629, but it doesn't seem to
work.
2017-05-05 17:03:27 +02:00
Marc Schreiber
4ed6d91656 \textcolor will be parse as span at the beginning of a paragraph 2017-05-04 16:48:27 +02:00
John MacFarlane
1668998c46 Include backtick_code_blocks extension in mardkown_mmd.
Closes #3637.
2017-05-04 16:36:35 +02:00
Alexander Krotov
430e6be1f4 Muse writer: omit automatic header identifiers (#3633) 2017-05-04 11:36:52 +02:00
Albert Krewinkel
57cba3f1d5
Org reader: support table.el tables
Closes #3314
2017-05-03 22:43:34 +02:00
Marc Schreiber
1728d4e609 \textcolor works as inline and block command 2017-05-03 13:39:38 +02:00
Marc Schreiber
d9439808f2 Add block version of \textcolor 2017-05-03 12:00:30 +02:00
Albert Krewinkel
df23d96c89
Generalize tableWith, gridTableWith
The parsing functions `tableWith` and `gridTableWith` are generalized to
work with more parsers. The parser state only has to be an instance of
the `HasOptions` class instead of requiring a concrete type. Block
parsers are required to return blocks wrapped into a monad, as this
makes it possible to use parsers returning results wrapped in `Future`s.
2017-05-02 23:41:45 +02:00
David A Roberts
79855ef934 Markdown writer: better escaping for links (#3628)
Previously the Markdown writer would sometimes create links where there
were none in the source.  This is now avoided by selectively escaping bracket
characters when they occur in a place where a link might be created.

Closes #3619.
2017-05-03 12:19:45 +02:00
schrieveslaach
6e55e6837a LaTeX reader: Add support for tabularx environment (#3632) 2017-05-03 12:16:48 +02:00
Mauro Bieg
e02cfcdeac Markdown Writer: put space before reference link definitions
Fixes #3630 (#3631).

Previously the attributes in link reference definitions did not have a space preceding.
2017-05-03 12:13:25 +02:00
John MacFarlane
cd2551c16c Added PandocResourceNotFound error.
Use this instead of PandocIOError when a resource is not
found in path.

This improves the error message in this case, see #3629.
2017-05-02 16:00:04 +02:00
Marc Schreiber
49336ee6ee Add basic \textcolor support to LaTeX reader 2017-05-02 10:48:57 +02:00
David A Roberts
c0192132cf Markdown writer: Case-insensitive reference links. (#3616)
Ensure that we do not generate reference links
whose labels differ only by case.

Also allow implicit reference links when the link
text and label are identical up to case.

Closes #3615.
2017-05-02 09:00:37 +02:00
Alexander Krotov
5d529e30c7 FB2 writer: Add support for "lang" metadata (#3625) 2017-05-02 08:57:08 +02:00
Albert Krewinkel
ae21a8bb2a
Lua filter: fall-back to global filters when none is returned
The implicitly defined global filter (i.e. all element filtering
functions defined in the global lua environment) is used if no filter is
returned from a lua script. This allows to just write top-level
functions in order to define a lua filter. E.g

    function Emph(elem) return pandoc.Strong(elem.content) end
2017-04-30 17:06:54 +02:00
Albert Krewinkel
2e916b3abf
Lua module: simplify Attributes, rename to Attr
Attributes was written to behave much like a normal table, in order to
simplify working with it. However, all Attr containing elements were
changed to provide panflute-like accessors to Attr components, rendering
the previous approach unnecessary.
2017-04-30 11:50:09 +02:00
Albert Krewinkel
3362cb89d9
Lua module: make Header argument order consistent
Attributes are always passed as the last element, making it possible to
omit this argument.  Argument order for `Header` was wrong and is fixed.
2017-04-30 11:28:03 +02:00
Albert Krewinkel
31caa616a9 Provide shared F monad functions for Markdown and Org readers
The `F` monads used for delayed evaluation of certain values in the
Markdown and Org readers are based on a shared data type capturing the
common pattern of both `F` types.
2017-04-30 10:59:20 +02:00
Alexander Krotov
97addc2a17 Add returnF to Text.Pandoc.Parsing 2017-04-30 10:59:20 +02:00
Alexander Krotov
540f3d49ef Org reader: Avoid creating nullMeta by applying setMeta directly 2017-04-30 10:58:19 +02:00
John MacFarlane
730796ee31 LaTeX writer: Fix problem with escaping in lstinline.
Previously the LaTeX writer created invalid LaTeX
when `--listings` was specified and a code span occured
inside emphasis or another construction.

This is because the characters `%{}\` must be escaped
in lstinline when the listinline occurs in another
command, otherwise they must not be escaped.

To deal with this, adoping Michael Kofler's suggestion,
we always wrap lstinline in a dummy command `\passthrough`,
now defined in the default template if `--listings` is
specified.  This way we can consistently escape the
special characters.

Closes #1629.
2017-04-29 11:05:44 +02:00
John MacFarlane
e76b672414 LaTeX writer: don't use lstinline it \item[..].
If you do, the contents of item disappear or are misplaced.
Use `\texttt` instead.

Closes #645.
2017-04-28 12:03:59 +02:00
Alexander Krotov
c346a8a7be Ms writer: make use of already defined render' (#3613) 2017-04-28 10:04:09 +02:00
Alexander Krotov
e97e9cd6a3 Muse writer: Do not reflow directives (#3614)
Directives at the beginning of documents cannot
span multiple lines so they must not be reflown.
2017-04-28 10:03:39 +02:00
Albert Krewinkel
24ef672132 Lua module: provide simple read format parser
A single `read` function parsing pandoc-supported formats is added to
the module. This is simpler and more convenient than the previous method
of exposing all reader functions individually.
2017-04-26 23:28:40 +02:00
Albert Krewinkel
c2567b2bd0 API change: move writer functions to Text.Pandoc.Writers
Writer helper functions were defined in the top-level Text.Pandoc
module. These functions are moved to the Writer submodule as to enable
reuse in other submodules.
2017-04-26 23:28:40 +02:00
Albert Krewinkel
0e107a305f API change: move reader functions to Text.Pandoc.Readers
Reader helper functions were defined in the top-level Text.Pandoc
module. These functions are moved to the Readers submodule as to enable
reuse in other submodules.
2017-04-26 23:28:40 +02:00
Albert Krewinkel
3ac23ab615 API change: move extension handling to Text.Pandoc.Extensions
Extension parsing and processing functions were defined in the top-level
Text.Pandoc module.  These functions are moved to the Extensions
submodule as to enable reuse in other submodules.
2017-04-26 23:28:40 +02:00
Albert Krewinkel
9cd20c9b8b Lua filter: allow filtering of meta data only 2017-04-26 23:28:40 +02:00
Albert Krewinkel
5416fb14ab Lua filter: allow natural access to meta elements
Meta elements that are treated as lua tables (i.e. MetaList,
MetaInlines, MetaBlocks, and MetaMap), are no longer wrapped in an
additional table but simply marked via a metatable.  This allows
treating those meta values just like normal tables, while still making
empty elements of those values distinguishable.
2017-04-26 23:28:40 +02:00
schrieveslaach
a29fa15a7b LaTeX reader: Add basic support for hyphenat package (#3603) 2017-04-26 12:05:13 +02:00
Alexander Krotov
256e3a6a5d Simplify linkText in Text.Pandoc.Readers.TWiki (#3605)
Avoid constructing list of one element ([B.str url]) just to flatten it
back with mconcat.
2017-04-26 12:04:28 +02:00
schrieveslaach
81548960d5 LaTeX reader: Add support for \vdots (#3607) 2017-04-26 12:03:07 +02:00
John MacFarlane
66b08391b3 HTML line block: Use class instead of style attribute.
We now issue `<div class="line-block">` and include a
default definition for `line-block` in the default
templates, instead of hard-coding a `style` on the
div.

Closes #1623.
2017-04-25 23:07:30 +02:00
John MacFarlane
ee160d7c4c LaTeX writer: fix error with line breaks after empty content.
LaTeX requires something before a line break, so we insert a
`~` if no printable content has yet been emitted.

Closes #2874.
2017-04-25 15:00:27 +02:00
John MacFarlane
d17f0dab84 LaTeX reader: better support for subfigure package.
A figure with two subfigures turns into two pandoc
figures; the subcaptions are used and the main caption
ignored, unless there are no subcaptions.

Closes #3577.
2017-04-24 23:39:14 +02:00
Alexander Krotov
624e5b2f92 TWiki reader: remove unnecessary $ (#3597) 2017-04-24 10:17:53 +02:00
Albert Krewinkel
04cb602d79
Org reader: allow multi-word arguments to src block params
The reader now correctly parses src block parameter list even if
parameter arguments contain multiple words.

Closes: #3477
2017-04-23 13:58:16 +02:00
Albert Krewinkel
2e43e27e5c
Org reader: stop adding rundoc prefix to src params
Source block parameter names are no longer prefixed with *rundoc*. This
was intended to simplify working with the rundoc project, a babel
runner. However, the rundoc project is unmaintained, and adding those
markers is not the reader's job anyway.

The original language that is specified for a source element is now
retained as the `data-org-language` attribute and only added if it
differs from the translated language.
2017-04-23 12:56:11 +02:00
Albert Krewinkel
04658c491b
Org reader: handle line numbering switch for src blocks
The line-numbering switch that can be given to source blocks (`-n` with
an start number as an optional parameter) is parsed and translated to a
class/key-value combination used by highlighting and other readers and
writers.
2017-04-23 11:54:36 +02:00
John MacFarlane
51a46b7e31 HTML reader: Revise treatment of li with id attribute.
Previously we always added an empty div before the list
item, but this created problems with spacing in tight
lists.  Now we do this:

If the list item contents begin with a Plain block,
we modify the Plain block by adding a Span around
its contents.

Otherwise, we add a Div around the contents of the
list item (instead of adding an empty Div to the
beginning, as before).

Closes #3596.
2017-04-23 11:03:48 +02:00
schrieveslaach
020dc63e23 Add siunitx Support (#3588)
For example:


```latex
\SI[round-precision=2]{1}{m} is equal to \SI{1000}{mm}.

\SI[round-precision=2]{1}[\$]{} is equal to \SI{0.938094}{\euro}
```
2017-04-22 21:57:21 +02:00
Albert Krewinkel
57a0759def
Lua: drop useless filter function type parameter
The return-type parameter for lua filter functions is removed.  It only
complicated the code without introducing any additional type safety.
2017-04-18 19:21:03 +02:00
Albert Krewinkel
464db59394
Org reader: allow emphasized text to be followed by [
Closes: #3577
2017-04-16 21:19:35 +02:00
schrieveslaach
1c8683f205 Read image size of Inkscape SVGs (#3580) 2017-04-16 19:39:24 +02:00
Albert Krewinkel
56dddcc3f5
Org reader: convert markup at beginning of footnotes
Closes: #3576
2017-04-16 17:22:47 +02:00
Albert Krewinkel
e6a536befc
Lua filter: revert to non-destructuring filters
We want to provide an interface familiar to users of other filtering
libraries.
2017-04-15 21:40:48 +02:00
John MacFarlane
629c6494a5 Small fix to error reporting in App.
Closes #3548.
2017-04-15 13:46:46 +02:00
John MacFarlane
bcc848d773 Avoid parsing "Notes:**" as a bare URI.
This avoids parsing bare URIs that start with a scheme
+ colon + `*`, `_`, or `]`.

Closes #3570.
2017-04-15 13:32:28 +02:00
John MacFarlane
1fe1c162ac Error: Added PandocCouldNotFindDataFileError.
Use this instead of PandocAppError when appropriate.
Removed exit code from PandocAppError, use 1 for all.
2017-04-15 12:05:58 +02:00
John MacFarlane
76b051f7b4 Fixed shadowing warnings 2017-04-15 11:59:48 +02:00
John MacFarlane
8e0032940f Docx reader: removed readDocxWithWarnings (API change).
No longer necessary with pandoc 2.0 framework.
2017-04-15 11:53:46 +02:00
John MacFarlane
35e9da28b8 App: use PandocOptionError instead of PandocAppError where appropriate. 2017-04-15 11:53:45 +02:00
John MacFarlane
d722d93b61 Error: Added PandocFilterError. 2017-04-15 11:53:45 +02:00
John MacFarlane
306dc624d9 Error: Added PandocPDFError 2017-04-15 11:53:45 +02:00
John MacFarlane
d5701e625c Text.Pandoc.Error: added new constructors.
- PandocSyntaxMapError String
- PandocFailOnWarningError
- PandocPDFProgramNotFoundError String
2017-04-15 11:53:45 +02:00
John MacFarlane
dd4110fb09 Revised error message for pandoc -t pdf. 2017-04-15 11:53:45 +02:00
Albert Krewinkel
3d6edbd9e3
Lua filter: use Attributes constructor for Attrs
Element attributes are pushed to the stack via the `Attributes`
function. `Attributes` creates an Attr like triple, but the triple also
allows table-like access to key-value pairs.
2017-04-15 09:31:09 +02:00
Albert Krewinkel
3aeed816e1
Lua filter: allow shorthand functions for math and quoted
Allow to use functions named `SingleQuoted`, `DoubleQuoted`,
`DisplayMath`, and `InlineMath` in filters.
2017-04-14 23:43:59 +02:00
Albert Krewinkel
eb8de6514b
Lua filter: Re-order code of stack value instances 2017-04-14 22:58:00 +02:00
Albert Krewinkel
d671b69b87
Lua filter: use lua strings for nullary constructors
Lua string are used to represent nullary data constructors. The previous
table-based representation was based on the JSON serialization, but can
be simplified. This also matches the way those arguments are passed to
custom writers.
2017-04-14 22:32:08 +02:00
Albert Krewinkel
0add4253e6
Avoid repeating StackValue instances definitions
The lua filters and custom lua writer system defined very similar
StackValue instances for strings and tuples.  These instance definitions
are extracted to a separate module to enable sharing.
2017-04-14 19:07:55 +02:00
Albert Krewinkel
feb1c1a930
Extract lua helper functions into Lua.Util module 2017-04-14 18:26:42 +02:00
Albert Krewinkel
1d9742bb5d
Use lua bools and strings for MetaBool, MetaString
Native lua booleans and strings are used to represent MetaBool and
MetaString values.  This is more natural than the previous table-based
representation.  The old lua representation can still be read back to
haskell, ensuring compatibility with the `pandoc.MetaBool` and
`pandoc.MetaString` lua constructors.
2017-04-14 14:59:17 +02:00
Albert Krewinkel
07f41a5515
Lua filter: use destructured functions for block filters
Filtering functions take element components as arguments instead of the
whole block elements. This resembles the way elements are handled in
custom writers.
2017-04-14 11:34:44 +02:00
Albert Krewinkel
0516b5127c
Drop dependency on hslua-aeson
Pushing values to the lua stack via custom functions is faster and more
flexible.
2017-04-14 11:21:12 +02:00
Albert Krewinkel
540a3e80ad
Push blocks via lua constructors and constants
All element creation tasks are handled by lua functions defined in the
pandoc module.
2017-04-14 10:33:38 +02:00
Albert Krewinkel
0085251ec7
Push inlines via lua constructors and constants
All element creation tasks are handled in the lua module.
2017-04-14 00:35:38 +02:00
Albert Krewinkel
425df8fff4
Use lua constructors to push meta values 2017-04-13 22:57:50 +02:00
Albert Krewinkel
2761a38e53
Lua filter: use destructured functions for inline filters
Instead of taking the whole inline element, forcing users to destructure it
themselves, the components of the elements are passed to the filtering
functions.
2017-04-12 20:48:44 +02:00
Alexander Krotov
624ccbd45e s/safed/saved/ 2017-04-14 08:27:56 +02:00
John MacFarlane
31a36cf186 Man writer: Fix handling of nested font commands.
Previously pandoc emitted incorrect markup for bold + italic, for example,
or bold + code.

Closes #3568.
2017-04-12 12:23:29 +02:00
Albert Krewinkel
7e3705c1c4
Lua filter: use custom StackValue Inline instance
Inline elements are no longer pushed and pulled via aeson's Value.
2017-04-11 23:31:55 +02:00
John MacFarlane
d4e5fe02b0 Docx writer: don't take "distArchive" from datadir.
The docx writer takes components from the distribution's
version of reference.docx when it can't find them in a
user's custom reference.docx.  (This sometimes happens
because Word will sometimes omit components needed for larger
documents when saving a simple one.)

Previously, we allowed a reference.docx in the data directory
(e.g. `~/.pandoc`) to be used as the distribution's reference.docx.
This led to a bizarre situation where pandoc would produce a
good docx using `--template ~/.pandoc/ref.docx`, but if `ref.docx`
were moved to `~/.pandoc/reference.docx`, it would then produce
a corrupted docx.

Closes #3322 (I think).
2017-04-10 23:59:32 +02:00
Albert Krewinkel
41ebdee5df
Lua filter: improve doc filter performance
Pandoc elements are pushed and pulled from the lua stack via custom
instances.
2017-04-07 21:04:22 +02:00
Albert Krewinkel
9278a6325d
Lua filter: Improve block filter performance
Reading of simple block values from the lua stack is handled manually,
but most block constructors are still handled via instances of aeson's
Value type.
2017-04-06 19:32:21 +02:00
Albert Krewinkel
dd00163a35 Lua filter: Improve inline filter performance
Getting inline instances from the lua stack is handled manually for some
simple inline constructors, including the `Str` constructor. This avoids
the indirect route through aeson's Value type and improves performance
considerably (approx. 30% speedup for some filters).
2017-04-06 19:25:42 +02:00
Albert Krewinkel
fca93efb62
Use lua registry instead of named globals
This is slightly cleaner while keeping performance approximately the
same.
2017-04-04 21:51:51 +02:00
John MacFarlane
6b0d3d1582 Ms writer: wider indents for lists.
Previously some indents weren't wide enough, leading
the list item to start on a line after the marker.
2017-04-06 12:45:23 +02:00
John MacFarlane
12ae1df5bf Allow raw latex commands starting with \start in Markdown.
Previously these weren't allowed because they were interpreted
as starting ConTeXt environments, even without a corresponding
\stop...

Closes #3558.
2017-04-06 11:30:03 +02:00
John MacFarlane
12a3481632 Ms writer: respect text wrapping options. 2017-04-05 15:17:35 +02:00
John MacFarlane
48729f9715 Ms writer improvements:
- added some variables to the default template.
- cleaner output for images (stringify alt text).
2017-04-04 17:21:02 +02:00
John MacFarlane
1ebb766aff Ms writer: ensure that @ is escaped in URIs.
Otherwise we may get unescaped @s that give eqn fits,
with @ as the delimiter character.
2017-04-04 16:13:24 +02:00
John MacFarlane
e650d1fbfd Error: Added PandocOptionError. 2017-04-04 14:35:00 +02:00
Timm Albers
f1eb3b3169 Add original classes to JS obfuscated links (#3554)
HTML links containing classes originally now preserve them when using
javascript email obfuscation.

Fixes #2989
2017-04-04 10:36:00 +02:00
Timm Albers
3e817124fe Add class to footnote back references
The HTML writer now also adds the class footnoteBack to back references
of footnotes. This allows for easier CSS styling.
2017-04-03 17:02:16 +02:00
Timm Albers
6e2019d588 Include \VerbatimFootnotes for highlighted code blocks
Updated the LaTeX writer to also include \VerbatimFootnotes in the
preamble for highlighted code blocks. Previously this was only done for
raw code blocks.
2017-04-03 14:36:36 +02:00
John MacFarlane
e3e52aa4c1 Removed unused import. 2017-04-03 10:20:51 +02:00
John MacFarlane
e62fbc1c3c Merge pull request #3550 from tarleb/lua-readers-submodule
Lua module: add readers submodule
2017-04-03 09:47:16 +02:00
John MacFarlane
e281a7cda0 Make sure docx/_rels/.rels gets into data files.
embedDir in file-embed excludes hidden files, so we need
to add this manually.
2017-04-02 23:29:58 +02:00
John MacFarlane
ff991d1e21 Revert "Revert "Use file-embed instead of hsb2hs to embed data files.""
This reverts commit 1fa15c225b.
2017-04-02 23:10:10 +02:00
John MacFarlane
913db947a9 Text.Pandoc.App: Throw errors rather than exiting.
These are caught (and lead to exit) in pandoc.hs, but
other uses of Text.Pandoc.App may want to recover in another
way.

Added PandocAppError to PandocError (API change).
This is a stopgap:  later we should have a separate constructor
for each type of error.

Also fixed uses of 'exit' in Shared.readDataFile, and
removed 'err' from Shared (API change).

Finally, removed the dependency on extensible-exceptions.

See #3548.
2017-04-02 23:04:48 +02:00
Albert Krewinkel
e7eb21ecca
Lua module: add readers submodule
Plain text readers are exposed to lua scripts via the `pandoc.reader`
submodule, which is further subdivided by format.  Converting e.g. a
markdown string into a pandoc document is possible from within lua:

    doc = pandoc.reader.markdown.read_doc("Hello, World!")

A `read_block` convenience function is provided for all formats,
although it will still parse the whole string but return only the first
block as the result.

Custom reader options are not supported yet, default options are used
for all parsing operations.
2017-04-02 17:28:07 +02:00
John MacFarlane
420e3eb26e Allow a theme file as argument to --highlight-style.
Also include a sample, `default.theme`, in `data/`.
2017-04-01 22:27:00 +02:00
John MacFarlane
1c84a03509 Ms writer: added syntax highlighting.
Closes #3547.

Macro definitions are inserted in the template when there is highlighted
code.

Limitations: background colors and underline currently not
supported.
2017-04-01 22:05:38 +02:00
John MacFarlane
34b9bee5a4 OpenDocument writer: wider labels for lists.
This avoids overly narrow labels for ordered lists with
() delimiters.

However, arguably it creates overly wide labels for bullets.

Also, lists now start flush with the margin, rather than
indented.

Fixes #2421.
2017-04-01 12:27:40 +02:00
John MacFarlane
8761d57750 Change MathJax CDN default since old one is shutting down.
New URL: https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js
Announcement: https://www.mathjax.org/cdn-shutting-down/

NOTE:  The new URL requires a version number, which we'll have
to update manually in subsequent pandoc releases in order to
take advantage of mathjax improvements.

Closes #3544.
2017-04-01 11:05:12 +02:00
John MacFarlane
80c3c93273 JATS writer: don't include jats.csl in metadata if csl already specified. 2017-03-31 15:15:49 +02:00
John MacFarlane
3217bc192e JATS writer: put references in <back>.
Modified template to include a `<back>` and `<body>` section.
This should give authors more flexibility, e.g. to put
acknowledgements metadata in `<back>`.  References are
automatically extracted and put into `<back>`.
2017-03-31 11:07:09 +02:00
John MacFarlane
136a53edc8 Fix compiler warning. 2017-03-30 23:03:57 +02:00
John MacFarlane
80d093843b Allow dynamic loading of syntax definitions.
See #3334.

* Add writerSyntaxMap to WriterOptions.
* Highlighting: added parameter for SyntaxMap to highlight.
* Implemented --syntax-definition option.

TODO:

[ ] Figure out whether we want to have the xml parsing
    depend on the dtd (it currently does, and fails unless
    the language.dtd is found in the same directory).
[ ] Add an option to read a KDE syntax highlighting theme
    as a custom style.
[ ] Add tests.
2017-03-30 22:36:36 +02:00
John MacFarlane
ea84cd0842 ZimWiki writer: put in PandocMonad, added warnings for raw. 2017-03-30 21:43:22 +02:00
John MacFarlane
d8a3228617 Textile writer: moved into PandocMonad.
Warnings for omitted raw content.
2017-03-30 21:37:13 +02:00
John MacFarlane
b27836666f Org writer: move everything into PandocMonad. 2017-03-30 21:31:43 +02:00
John MacFarlane
8d50f37d53 Don't read jats.csl unless we actually need it. 2017-03-30 17:23:16 +02:00