Commit graph

74 commits

Author SHA1 Message Date
John MacFarlane
328655e863 Tracing: give less misleading line information with parseWithString.
Previously positions would be reported past the end of the chunk.
We now reset the source position within the chunk and report
positions "in chunk."
2017-06-19 22:41:09 +02:00
John MacFarlane
814ac51d32 Separated tracing from logging.
Formerly tracing was just log messages with a DEBUG log
level.  We now make these things independent.  Tracing
can be turned on or off in PandocMonad using `setTrace`;
it is independent of logging.

* Removed `DEBUG` from `Verbosity`.
* Removed `ParserTrace` from `LogMessage`.
* Added `trace`, `setTrace` to `PandocMonad`.
2017-06-19 22:17:43 +02:00
John MacFarlane
9849ba7fd7 Use Control.Monad.State.Strict throughout.
This gives 20-30% speedup and reduction of memory
usage in most of the writers.
2017-06-17 07:45:28 +02:00
John MacFarlane
23f3c2d7b4 Changed "extracting..." warning to a regular log message.
This makes it sensitive to proper verbosity settings.
(It is now treated as INFO rather than WARNING, so one
doesn't get these messages for creation of tmp images
while making a pdf.)

API changes:

* Removed extractMediaBag from Text.Pandoc.MediaBag.
* Added Extracting as constructor for LogMessage.
2017-06-12 15:28:39 +02:00
John MacFarlane
b61a51ee15 hlint suggestions. 2017-06-02 15:25:39 +02:00
John MacFarlane
0cf6511f16 Some hlint refactoring. 2017-06-01 15:09:38 +02:00
John MacFarlane
b1a9b567aa Trivial reformatting. 2017-06-01 14:19:43 +02:00
John MacFarlane
0f6458c0c1 Don't double extract images from docx.
This fixes a regression that was introduced when `--extract-media`
was generalized to work with any input format.  We were getting
two versions of each image extracted from a docx, one with a hash,
one with the original filename, though only the hash one was used.
This patch restores the original behavior (using the original
filename).

Pointed out in comments on #3674. Thanks to @laperouse.
2017-05-18 13:38:19 +02:00
John MacFarlane
6b086acae8 Rename fillMedia -> fillMediaBag. 2017-05-07 21:03:18 +02:00
John MacFarlane
af7215a048 Moved fillMedia, extractMedia from App to Class.
Also generalized type of fillMedia to any instance of PandocMonad.
2017-05-07 20:42:32 +02:00
John MacFarlane
99be906101 Added PandocHttpException, trap exceptions in fetching from URLs.
Closes #3646.
2017-05-07 13:11:04 +02:00
John MacFarlane
cd2551c16c Added PandocResourceNotFound error.
Use this instead of PandocIOError when a resource is not
found in path.

This improves the error message in this case, see #3629.
2017-05-02 16:00:04 +02:00
John MacFarlane
1ec6a19223 Changed display format for messages. 2017-03-09 10:30:57 +01:00
John MacFarlane
beb94c53e9 Add hanging indent to log messages.
This makes them easier to read.
2017-03-04 11:26:49 +01:00
John MacFarlane
72af7b4ee5 Shared: remove 'warn'.
PDF writer: Use 'report' instead of 'warn', make it sensitive
to verbosity settings.
2017-02-24 14:29:56 +01:00
John MacFarlane
1c84855aab Class: Add stResourcePath to CommonState, getResourcePath, setResourcePath.
To be used in implementing `\graphicspath` in LaTeX, and possibly
in things like PDF production via context.

Use resource path in fetchItem.

Issue an info message if we get a resource from somewhere other
than ".".

Added UsingResourceFrom to log message.
2017-02-24 14:15:10 +01:00
John MacFarlane
61d3376a45 Restore "Fetching..." message with openURL if --verbose. 2017-02-23 16:24:20 +01:00
John MacFarlane
e08e93e844 Use PandocIOError in Class. 2017-02-23 16:21:59 +01:00
John MacFarlane
e0d21dbb82 Class.report: Save all log messages in state.
Verbosity level only affects which are printed to stdout.
(Exception: DEBUG messages are only printed, never saved to
state.)
2017-02-11 09:06:49 +01:00
John MacFarlane
76c55466d3 Use new warnings throughout the code base. 2017-02-11 00:14:44 +01:00
John MacFarlane
5e1249481b Added Text.Pandoc.Logging (exported module).
This now contains the Verbosity definition previously
in Options, as well as a new LogMessage datatype that
will eventually be used instead of raw strings for
warnings.

This will enable us, among other things, to provide
machine-readable warnings if desired.

See #3392.
2017-02-10 20:59:54 +01:00
John MacFarlane
87507e1b9c LaTeX reader: Issue warnings when skipping unknown latex commands.
See #3392.
2017-02-09 22:21:07 +01:00
John MacFarlane
0a4ba91994 Reverted deferred media bag code.
This was not actually being used. Since it adds considerable
complexity, it's best not to include it unless we are
actually going to use it.

The original thought was that we could do all loading in the
readers, always deferred and thus costless.  This was supposed
to eliminate the need to traverse trees loading resources in
the docx, epub, odt writers and in PDF and SelfContained.
(It would also have the side effect that --extract-media could
be used with all input formats.  This wasn't an intended side
effect, and it could be debated whether it's desirable, since
--extract-media was originally designed to extract the media
contained in a docx or odt or epub container.)

However, we never actually took the step of moving all of this
work to the readers, for a couple of reasons. The main reason
is that we'd still need to fetch resources in the docx,
epub, odt, pdf and self-contained writers, since the Pandoc AST might
have been built programatically and hence not generated by a reader.
So it's not clear that doing lazy loading in the readers would have
any real advantage.

I'm still not completely sure about this --- if we change our
minds it would be easy to undo this commit.

@jkr comments welcome.
2017-02-09 21:26:24 +01:00
John MacFarlane
857d35fce4 Refactored some files formerly in LaTeX reader.
* Export readFileFromDirs from Class.
* Export insertIncludedFile from Parsing.

Simplified code in LaTeX/RST readers.
2017-02-07 22:33:05 +01:00
John MacFarlane
1ecc48e9f9 Moved readFileFromDirs to Text.Pandoc.Class.
This can be used in several different modules, not just
LaTeX reader.
2017-02-07 21:42:35 +01:00
John MacFarlane
4ccbdf4e8d Expose FileTree in Class 2017-01-27 11:15:42 +01:00
John MacFarlane
73e343cfcd Fixed small mistake in instance for logOutput. 2017-01-25 17:07:43 +01:00
John MacFarlane
bc7e846da6 More logging-related changes.
Class:

* Removed getWarnings, withWarningsToStderr
* Added report
* Added logOutput to PandocMonad
* Make logOutput streaming in PandocIO monad
* Properly reverse getLog output

Readers:

* Replaced use of trace with report DEBUG.

TWiki Reader:  Put everything inside PandocMonad m.

API changes.
2017-01-25 17:07:43 +01:00
John MacFarlane
4e97efe857 Class: Changes around logging.
* Export getLog, setVerbosity
* Add report to PandocMonad methods.
* Redefine warning and getWarnings in terms of getLog and report.
* Remove stWarnings from CommonState, add stLog and stVerbosity.
2017-01-25 17:07:43 +01:00
Jesse Rosenthal
5b3bfa28f4 Class: Warn instead or erroring if we can't fetch media
If deferred media can't be fetched, we catch the error and warn
instead. We add an internal function for fetching which returns a Maybe
value, and then run catMaybes to only keep the Just's.
2017-01-25 17:07:42 +01:00
Jesse Rosenthal
613588a0dc Class: Refactor fetchItem.
Move the downloading/reading-in logic out of fetchItem, so we can use it
to fill the MediaBag. Now when other modules use `fetchItem` it will
fill the MediaBag as expected.
2017-01-25 17:07:42 +01:00
Jesse Rosenthal
4b953720c8 Class: Add insertDeferredMedia function. 2017-01-25 17:07:42 +01:00
Jesse Rosenthal
55dbc00d55 Integrate DeferredMediaBag into CommonState
The DeferredMediaBag is now the object that is held in state. It should
not be visible to users, who will still deal with MediaBag through
exported getters and setters.

We now have a function `fetchDeferredMedia` which returns () but
downloads/reads in all of the deferred media.

Note that getMediaBag first fetches all deferred media.
2017-01-25 17:07:42 +01:00
Jesse Rosenthal
5814096d79 Introduce DeferredMediaBag.
This is a lazy MediaBag, that will only be evaluated (downloaded/read
in) upon demand.

Note that we use fetchItem in getDefferedMedia at the moment to read
in/download. This means that we don't need to distinguish between URIs
and FilePaths. But there is an inefficiency here: `fetchItem` will pull
an item out of the mediaBag if it's already there, and then we'll
reinsert it. We could separate out `fetchItem` into the function that
checks the MediaBag and the underlying downloader/read-inner.
2017-01-25 17:07:42 +01:00
John MacFarlane
994d431172 Class: have pure instance of openURL throw an error, for now.
Later we may want to include a map of URLs and mime type, bytestring
pairs in pure state to serve as a fake internet.
2017-01-25 17:07:42 +01:00
John MacFarlane
4cb124d147 Add openURL and readFileStrict to PandocMonad.
Removed fetchItem and fetchItem'.
Provide fetchItem in PandocMonad (it uses openURL and readFileStrict).

TODO:

- PandocPure instance for openURL.
- Fix places where fetchItem is used so that we trap the
  exception instead of checking for a Left value.  (At least
  in the places where we want a warning rather than a failure.)
2017-01-25 17:07:41 +01:00
John MacFarlane
08110c3714 Class: Removed getDefaultReferenceDocx/ODT from PandocMonad.
We don't need these, since the default docx and odt can be
retrieved using `readDataFile datadir "reference.docx"` (or odt).
2017-01-25 17:07:41 +01:00
John MacFarlane
143d1a2113 Removed commented-out vestigaes of fail in Class. 2017-01-25 17:07:41 +01:00
John MacFarlane
b5d1567022 Class: removed 'fail' from PandocMonad.
Do we need this?  I don't see why.

There's a name clash which would better be avoided.
2017-01-25 17:07:41 +01:00
John MacFarlane
f328cfe6a7 Removed unneeded pragmas. 2017-01-25 17:07:41 +01:00
John MacFarlane
4111fdbaf0 Instances of PandocMonad for common transformers. 2017-01-25 17:07:41 +01:00
John MacFarlane
54932ade67 Class: no more MonadState CommonState.
- Added getCommonState, putCommonState, getsCommonState, modifyCommonState
  to PandocMonad interface.

- Removed MonadState CommonState instances.
2017-01-25 17:07:41 +01:00
John MacFarlane
40ac0cf133 Whitespace. 2017-01-25 17:07:41 +01:00
John MacFarlane
8753a91252 Add PandocMonad m instance for ParserT based on PandocMonad.
This will avoid the need for lift.
2017-01-25 17:07:41 +01:00
John MacFarlane
e1d2da4c22 Have warningWithPos take a SourcePos rather than Maybe SourcePos.
After all, we have warning if you don't want the source pos info.
2017-01-25 17:07:41 +01:00
Jesse Rosenthal
57cff4b8ae Class: Functions for dealing with PureState
There are two states in PandocPure, but it is only easy to deal with
CommonState. In the past, to do state monad operations on
PureState (the state specific to PandocPure) you had to add (lift
. lift) to the monadic operation and then rewrap in the newtype. This
adds four functions ({get,gets,put,modify}PureState) corresponding to
normal state monad operations. This allows the user to modify
PureState in PandocPure without worrying about where it sits in the
monad stack or rewrapping the newtype.
2017-01-25 17:07:40 +01:00
Jesse Rosenthal
15708f0b0f Class: rename env* prefixed fields to st*.
This was left over from when they were part of an environment.
2017-01-25 17:07:40 +01:00
John MacFarlane
d7583f3659 Error: change type of handleError.
It now lives in IO and gives a proper message + exit
instead of calling 'error'.

We shouldn't be making it easier for people to raise error on
pure code.  And this is better for the main application
in IO.
2017-01-25 17:07:40 +01:00
John MacFarlane
a4bd650277 Class: rename addWarning[WithPos] to warning[WithPos].
There's already a function addWarning in Parsing!

Maybe we can dispense with that now, but I still like
'warning' better as a name.
2017-01-25 17:07:40 +01:00
John MacFarlane
2710fc4261 Class: Renamed 'warn' to 'addWarning' and consolidated RTF writer.
* Renaming Text.Pandoc.Class.warn to addWarning avoids conflict
  with Text.Pandoc.Shared.warn.
* Removed writeRTFWithEmbeddedImages from Text.Pandoc.Writers.RTF.
  This is no longer needed; we automatically handle embedded images
  using the PandocM functions.  [API change]
2017-01-25 17:07:40 +01:00