Commit graph

1932 commits

Author SHA1 Message Date
Albert Krewinkel
f56792927f
Org reader: support special strings export option
Parsing of special strings (like '...' as ellipsis or '--' as en dash)
can be toggled using the `-` option.
2016-06-03 11:41:23 +02:00
Albert Krewinkel
d4de8451b9 Org reader: support emphasized text export option
Parsing of emphasized text can be toggled using the `*` option.  This
influences parsing of text marked as emphasized, strong, strikeout, and
underline.  Parsing of inline math, code, and verbatim text is not
affected by this option.
2016-06-03 11:17:02 +02:00
Albert Krewinkel
952a7dac58 Org reader: support smart quotes export option
Reading of smart quotes can be toggled using the `'` option.
2016-06-03 11:16:35 +02:00
Albert Krewinkel
729fca311f Org reader: drop unused field from parser state
The `OrgParserState` contained both an `orgStateMeta` and
`orgStateMeta'` field, the former for plain meta information and the
latter for F-monad wrapped meta info.  The plain meta info is only used
to make `OrgParserState` an instance of the `HasMeta` class, which in
turn is never used in the reader.  The (F Meta) version is hence renamed
to the "un-primed" version while the other one is dropped.
2016-06-02 15:30:21 +02:00
Albert Krewinkel
512bf2eebf Org reader: undo code duplication
Some code was duplicated (copy-pasted) or placed in an inappropriate
module during the modularization refactoring.  Those functions are moved
into a `Shared` module, as was originally intended but forgotten.
Better documentation of the respective functions is a positive
side-effect.
2016-06-02 15:30:20 +02:00
John MacFarlane
061bc60f70 Merge pull request #2950 from tarleb/org-ref-support
Org reader: support org-ref style citations
2016-05-31 12:44:29 -07:00
Albert Krewinkel
c17c62a2c7 Org reader: support new syntax for export blocks
Org-mode version 9 usees a new syntax for export blocks.  Instead of
`#+BEGIN_<FORMAT>`, where `<FORMAT>` is the format of the block's
content, the new format uses `#+BEGIN_export <FORMAT>` instead.  Both
types are supported.
2016-05-29 21:08:50 +02:00
Albert Krewinkel
4f84cf02c7 Org reader: refactor BEGIN…END block parsing
- Reorder functions, grouping related functions together.

- Demote simple functions to local functions if they are used just once.

- Rename and document functions to increase code readability.

- Fix handling of whitespace in blocks, allowing content to be indented
  less then the block header.
2016-05-29 21:08:36 +02:00
Albert Krewinkel
bc82123122 Org reader: rename parseInlines to inlines
Having a function starting with `parse` in a parsing library is overly
redundant.  Let's use a nicer, shorter name more in line with the rest
of the library.
2016-05-29 21:08:31 +02:00
Albert Krewinkel
f226cb88b0 Org reader: support org-ref style citations
The *org-ref* package is an org-mode extension commonly used to manage
citations in org documents.  Basic support for the `cite:citeKey` and
`[[cite:citeKey][prefix text::suffix text]]` syntax is added.
2016-05-27 21:19:28 +02:00
Albert Krewinkel
eea6d6568f Org reader: extract blocks parser to module
Block parsing code is moved to a separate module.

This is part of the Org-mode reader cleanup effort.
2016-05-25 23:21:40 +02:00
Albert Krewinkel
39e8b4276e Org reader: extract inline parser to module
Inline parsing code is moved to a separate module.  Parsers for block
starts are extracted as well, as those are used in the `endline` parser.

This is part of the Org-mode reader cleanup effort.
2016-05-25 22:54:45 +02:00
Albert Krewinkel
a340c7249f Org reader: extract parsing function to module
The Org-mode reader uses many functions defined in the
`Text.Pandoc.Parsing` utility module.  Some of the functions are
overwritten with versions adapted to Org-mode idiosyncrasies.  These
special functions, as well as the normal Pandoc versions, are combined
in a single module to increase the ease of use.

This leads to decoupling of Org-mode and Pandoc and hence to slightly
cleaner code.  The downside is code-bloat due to repeated import/export
statements.
2016-05-25 22:53:55 +02:00
mb21
340f0aaef8 EPUB Reader: normalise Link id as well 2016-05-24 17:42:37 +02:00
Albert Krewinkel
a4717c2fc5 Org reader: respect drawer export setting
The `d` export option can be used to control which drawers are exported
and which are discarded.  Basic support for this option is added here.
2016-05-23 09:44:37 +02:00
Albert Krewinkel
f3d27e4c80 Org reader/writer: use CUSTOM_ID in properties
The `ID` property is reserved for internal use by Org-mode and should
not be used.  The `CUSTOM_ID` property is to be used instead, it is
converted to the `ID` property for certain export format.

The reader and writer erroneously used `ID`.  This is corrected by using
`CUSTOM_ID` where appropriate.
2016-05-22 23:01:47 +02:00
John MacFarlane
446cf6a1cf HTML reader: fixed bug in pClose.
This caused exponential parsing behavior in documnets
with unclosed tags in dl, dd, dt.
2016-05-21 23:05:00 -07:00
Albert Krewinkel
68d388f833 Org reader: add :PROPERTIES: drawer support
Headers can have optional `:PROPERTIES:` drawers associated with them.
These drawers contain key/value pairs like the header's `id`.  The
reader adds all listed pairs to the header's attributes; `id` and
`class` attributes are handled specially to match the way `Attr` are
defined.

This also changes behavior of how drawers of unknown type are handled.
Instead of including all unknown drawers, those are not read/exported,
thereby matching current Emacs behavior.

This closes #1877.
2016-05-20 17:01:26 +02:00
John MacFarlane
0958f2f5d0 Merge pull request #2927 from tarleb/org-attr-html
Org reader support for ATTR_HTML statements
2016-05-19 10:44:11 -07:00
Albert Krewinkel
16e233475a Org reader: add support for ATTR_HTML attributes
Arbitrary key-value pairs can be added to some block types using a
`#+ATTR_HTML` line before the block.  Emacs Org-mode only includes these
when exporting to HTML, but since we cannot make this distinction here,
the attributes are always added.

The functionality is now supported for figures.

This closes #1906.
2016-05-19 09:55:12 +02:00
Albert Krewinkel
26e8d98be2 Org reader: use custom anyLine
Additional state changes need to be made after a newline is parsed,
otherwise markup may not be recognized correctly.

This fixes a bug where markup after certain block-types would not be
recognized. E.g. `/emph/` in the following snippet was not parsed as
emphasized.

    foo
    # comment
    /emph/
2016-05-19 09:35:47 +02:00
Albert Krewinkel
1dda535378 Org reader: refactor block attribute handling
A parser state attribute was used to keep track of block attributes
defined in meta-lines.  Global state is undesirable, so block attributes
are no longer saved as part of the parser state.  Old functions and the
respective part of the parser state are removed.
2016-05-19 09:33:51 +02:00
John MacFarlane
847167804a EPUB reader: unescape URIs in spine.
This should fix #2924.

Testing on the epub that caused the problem originally
would be welcome.
2016-05-17 09:38:52 -07:00
John MacFarlane
344412cba8 Merge pull request #2894 from sid-kap/rst-code-class
Add class option for code block in RST reader
2016-05-12 00:03:14 -07:00
Albert Krewinkel
be5cccf248 Org reader: parse but ignore export options
All known export options are parsed but ignored.
2016-05-11 19:13:43 +02:00
Albert Krewinkel
76143de97e Org reader: add support for sub/superscript export options
Org-mode allows to specify export settings via `#+OPTIONS` lines.
Disabling simple sub- and superscripts is one of these export options,
this options is now supported.
2016-05-11 19:13:43 +02:00
Albert Krewinkel
7a0729ea09 Org reader: move parser state into separate module
The org reader code has become large and confusing.  Extracting smaller
parts into submodules should help to clean things up.
2016-05-11 19:13:42 +02:00
John MacFarlane
fd9ec835ec Merge pull request #2907 from tarleb/org-fixes
Org fixes (reader and writer)
2016-05-09 10:17:56 -07:00
Albert Krewinkel
10a809f126 Org reader: fix inline-LaTeX regression
The last fix for whitespace handling of inline LaTeX commands was
incorrect, preventing correct recognition of inline LaTeX commands which
contain spaces.  This fix ensures that only trailing whitespace is cut
off.
2016-05-09 19:06:04 +02:00
roblabla
acd492c7f4 Allow spaces before '!' in MediaWiki table header 2016-05-09 17:54:40 +02:00
John MacFarlane
21d1a3b57c Merge pull request #2898 from tarleb/org-table-refactoring
Org reader: table parsing code refactoring and fixes
2016-05-05 16:22:56 -07:00
Albert Krewinkel
405c3e9c36 Org reader: fix spacing after LaTeX-style symbols
The org-reader was droping space after unescaped LaTeX-style symbol
commands: `\ForAll \Auml` resulted in `∀Ä` but should give `∀ Ä`
instead.  This seems to be because the LaTeX-reader treats the
command-terminating space as part of the command.  Dropping the trailing
space from the symbol-command fixes this issue.
2016-05-04 23:16:23 +02:00
Albert Krewinkel
2d825603c6 Org reader: fix handling of empty table cells, rows
This fixes Org mode parsing of some corner cases regarding empty cells
and rows.  Empty cells weren't parsed correctly, e.g. `|||` should be
two empty cells, but would be parsed as a single cell containing a pipe
character.  Empty rows where parsed as alignment rows and dropped from
the output.

This fixes #2616.
2016-05-04 16:02:03 +02:00
Albert Krewinkel
a51e4e8215 Org reader: refactor rows-to-table conversion
This refactores the codes conversing a list table lines to an org table
ADT.  The old code was simplified and is now slightly less ugly.
2016-05-04 16:01:22 +02:00
Albert Krewinkel
d5e4bc179c Org reader: stop padding short table rows
Emacs Org-mode doesn't add any padding to table rows.  The first
row (header or first body row) is used to determine the column count, no
other magic is performed.

The org reader was padding rows to the length of the longest table row.
This was done due to a misunderstanding of how Org handles tables.  This
feature reflected how Org-mode handles tables when pressing <TAB>.  The
Org exporter however, which is what the reader should implement, doesn't
do any of this.  So this was a mis-feature that made the reader more
complex and reduced comparability.  It was hence removed.
2016-05-04 15:48:07 +02:00
Sidharth Kapur
490c2b543d Add class option for code block in RST reader
According to http://docutils.sourceforge.net/docs/ref/rst/directives.html#code,
the code directive supports the ":class:" option.
2016-05-01 21:42:58 -05:00
Jesse Rosenthal
99eac312fe Binary fmts throw PandocError on zip-archive fail
Commit 91dc3342 made `readDocx` throw PandocError if there was an
unarchiving error. This extends that fix to `readOdt` and `readEPUB`.
2016-05-01 18:27:20 -04:00
Jesse Rosenthal
91dc334249 Docx Reader: Throw PandocError on unzip failure
Previously, readDocx would error out if zip-archive failed. We change
the archive extraction step from `toArchive` to `toArchiveOrFail`, which
returns an Either value.
2016-05-01 12:17:12 -04:00
Emanuel Evans
1bfe39e24c
Ignore leading space in org code blocks
Fixes #2862

Also fix up tab handling for leading whitespace in code blocks.
2016-04-26 10:29:59 -07:00
Jesse Rosenthal
a385ee1d4f Docx Reader: parse moveTo and moveFrom
`moveTo` and `moveFrom` are track-changes tags that are used when a
block of text is moved in the document. We now recognize these tags and
treat them the same as `insert` and `delete`, respectively. So,
`--track-changes=accept` will show the moved version, while
`--track-changes=reject` will show the original version.
2016-04-15 14:09:18 -04:00
John MacFarlane
4b49f923cb Markdown reader: Fix pandoc title blocks with lines ending in 2 spaces.
Closes #2799.

Also added -s to markdown-reader-more test.
2016-04-10 09:13:53 -07:00
John MacFarlane
773bbb8fc7 Markdown + HTML readers: be more forgiving about unescaped &.
We are now more forgiving about parsing invalid HTML with
unescaped `&` as raw HTML.  (Previously any unescaped `&`
would cause pandoc not to recognize the string as raw HTML.)

Closes #2410.
2016-04-10 07:39:36 -07:00
John MacFarlane
b1ffdf3b01 Fixed bug in Markdown raw HTML parsing.
This was a regression, with the rewrite of `htmlInBalanced`
(from `Text.Pandoc.Readers.HTML`) in 1.17.

It caused newlines to be omitted in raw HTML blocks.

Closes #2804.
2016-03-22 16:56:10 -07:00
Jesse Rosenthal
28c7617f19 Docx reader: Handle alternate content
Some word functions -- especially graphics -- give various choices for
content so there can be backwards compatibility. This follows the
largely undocumented feature by working through the choices until we
find one that works.

Note that we had to split out the processing of child elems of runs into
a separate function so we can recurse properly. Any processing of an
element *within* a run (other than a plain run) should go into
`childElemToRun`.
2016-03-18 09:38:26 -04:00
Jesse Rosenthal
855c8b43f0 Docx reader: Don't make numbered heads into lists.
Word uses list numbering styles to number its headings. We only call
something a numbered list if it does not also heave a heading style.
2016-03-16 12:50:32 -04:00
Jesse Rosenthal
ee03e954d0 Add readDocxWithWarnings
The regular readDocx just becomes a special case.
2016-03-12 17:08:20 -05:00
Jesse Rosenthal
102ba9ecb8 Docx Reader: Add state to the parser, for warnings
In order to be able to collect warnings during parsing, we add a state
monad transformer to the D monad. At the moment, this only includes a
list of warning strings (nothing currently triggers them, however). We
use StateT instead of WriterT to correspond more closely with the
warnings behavior in T.P.Parsing.
2016-03-12 17:08:20 -05:00
John MacFarlane
a485c42d78 Fixed behavior of base tag.
+ If the base path does not end with slash, the last component
  will be replaced.  E.g. base = `http://example.com/foo`
  combines with `bar.html` to give `http://example.com/bar.html`.
+ If the href begins with a slash, the whole path of the base
  is replaced.  E.g. base = `http://example.com/foo/` combines
  with `/bar.html` to give `http://example.com/bar.html`.

Closes #2777.
2016-03-10 19:59:55 -08:00
John MacFarlane
2b55b76ebe Markdown reader: Improved pipe table parsing.
Fixes #2765.
Added test case.
2016-03-09 11:46:00 -08:00
John MacFarlane
54a68616d7 Markdown reader: Clean up pipe table parsing. 2016-03-09 10:11:32 -08:00