* New module: Text.Pandoc.Writers.Ms.
* New template: default.ms.
* The writer uses texmath's new eqn writer to convert math
to eqn format, so a ms file produced with this writer
should be processed with `groff -ms -e` if it contains
math.
* Add `--lua-filter` option. This works like `--filter` but takes pathnames of special lua filters and uses the lua interpreter baked into pandoc, so that no external interpreter is needed. Note that lua filters are all applied after regular filters, regardless of their position on the command line.
* Add Text.Pandoc.Lua, exporting `runLuaFilter`. Add `pandoc.lua` to data files.
* Add private module Text.Pandoc.Lua.PandocModule to supply the default lua module.
* Add Tests.Lua to tests.
* Add data/pandoc.lua, the lua module pandoc imports when processing its lua filters.
* Document in MANUAL.txt.
This contains a list of strings that will be recognized by pandoc's
Markdown parser as abbreviations. (A nonbreaking space will
be inserted after the period, preventing a sentence space in
formats like LaTeX.)
Users can override the default by putting a file abbreviations
in their user data directory (`~/.pandoc` on *nix).
Closes#1905.
Removed stateChapters from ParserState.
Now we parse chapters as level 0 headers, and parts as level -1 headers.
After parsing, we check for the lowest header level, and if it's
less than 1 we bump everything up so that 1 is the lowest header level.
So `\part` will always produce a header; no command-line options
are needed.
+ Text.Pandoc.Writers.Shared
+ Text.Pandoc.Parsing
+ Text.Pandoc.Asciify
+ Text.Pandoc.Emoji
+ Text.Pandoc.ImageSize
[API change]
These are often helpful to people writing their own
reader or writer modules. Closes#3260.
This now contains the Verbosity definition previously
in Options, as well as a new LogMessage datatype that
will eventually be used instead of raw strings for
warnings.
This will enable us, among other things, to provide
machine-readable warnings if desired.
See #3392.
The App module provides a function that does a pandoc conversion,
based on option settings. The program (pandoc.hs) now does nothing
more than parse options and pass them to this function, which can
easily be used by other applications (e.g. a GUI wrapper).
The Opt structure has been further simplified.
API changes:
* New exposed module Text.Pandoc.App
* Text.Pandoc.Highlighting has been exposed.
* highlightingStyles has been moved to Text.Pandoc.Highlighting.
Removed writerDocbookVersion in WriterOptions.
Renamed default.docbook template to default.docbook4.
Allow docbook4 as an output format.
But alias docbook = docbook4.
* Text.Pandoc.Writers.HTML: removed writeHtml, writeHtmlString,
added writeHtml4, writeHtml4String, writeHtml5, writeHtml5String.
* Removed writerHtml5 from WriterOptions.
* Renamed default.html template to default.html4.
* "html" now aliases to "html5"; to get the old HTML4 behavior,
you must now specify "-t html4".
The type is implemented in terms of an underlying bitset
which should be more efficient.
API change: from Text.Pandoc.Extensions export Extensions,
emptyExtensions, extensionsFromList, enableExtension, disableExtension,
extensionEnabled.
* Remove exported module `Text.Pandoc.Readers.TeXMath`
* Add exported module `Text.Pandoc.Writers.Math`
* The function `texMathToInlines` now lives in `Text.Pandoc.Writers.Math`
* Export helper function `convertMath` from `Text.Pandoc.Writers.Math`
* Use these functions in all writers that do math conversion.
This ensures that warnings will always be issued for failed
math conversions.
Introduce a new module, Text.Pandoc.Free, with pure versions, based on
the free monad, of numerous IO functions used in writers and
readers. These functions are in a pure
Monad (PandocAction). PandocAction takes as a parameter the type of
IORefs in it. It can be aliased in individual writers and readers to
avoid this parameter.
Note that this means that at the moment a reader can only use one type
of IORef. If possible, it would be nice to remove this limitation.
So far this just reproduces capacity.
Later we'll be able to add features like warning
messages, dynamic loading of xml syntax definitions,
and dynamic loading of themes.
+ Removed Text.Pandoc.Readers.Docx.Fonts
+ Moved its code to texmath; we now use (from texmath 0.9)
Text.TeXMath.Unicode.Fonts
+ Use texmath 0.9 (currently from git).
+ Updated epub tests because texmath now handles more mathml.
We already lower-bound tagsoup at 0.13.7, which means we were always
running the compatibility layer (it was conditional on min value
0.13). Better to just use `lookupEntity` from the library directly, and
convert a string to a char if need be.
Some code was duplicated (copy-pasted) or placed in an inappropriate
module during the modularization refactoring. Those functions are moved
into a `Shared` module, as was originally intended but forgotten.
Better documentation of the respective functions is a positive
side-effect.
Inline parsing code is moved to a separate module. Parsers for block
starts are extracted as well, as those are used in the `endline` parser.
This is part of the Org-mode reader cleanup effort.
The Org-mode reader uses many functions defined in the
`Text.Pandoc.Parsing` utility module. Some of the functions are
overwritten with versions adapted to Org-mode idiosyncrasies. These
special functions, as well as the normal Pandoc versions, are combined
in a single module to increase the ease of use.
This leads to decoupling of Org-mode and Pandoc and hence to slightly
cleaner code. The downside is code-bloat due to repeated import/export
statements.
Now instead of using `findExecutable`, which has limitations
on Windows, we just do `progname --version` and see if it
returns successfully. Closes#2903.
The docx reader used to use a Modifiable typeclass to combine both
Blocks and Inlines. But all the work was in the inlines. So most of the
generality was wasted, at the expense of making the code harder to
understand. This gets rid of the generality, and adds functions for
Blocks and Inlines. It should be a bit easier to work with going forward.
* Bumped version to 1.16.
* Added Attr field to Link and Image.
* Added `common_link_attributes` extension.
* Updated readers for link attributes.
* Updated writers for link attributes.
* Updated tests
* Updated stack.yaml to build against unreleased versions of
pandoc-types and texmath.
* Fixed various compiler warnings.
Closes#261.
TODO:
* Relative (percentage) image widths in docx writer.
* ODT/OpenDocument writer (untested, same issue about percentage widths).
* Update pandoc-citeproc.
Don't use custom prelude for latest ghc.
This is a better approach to making 'stack ghci' and 'cabal repl'
work. Instead of using NoImplicitPrelude, we only use the custom
prelude for older ghc versions. The custom prelude presents a
uniform API that matches the current base version's prelude.
So, when developing (presumably with latest ghc), we don't
use a custom prelude at all and hence have no trouble with ghci.
The custom prelude no longer exports (<>): we now want to
match the base 4.8 prelude behavior.
- The (non-exported) prelude is in prelude/Prelude.hs.
- It exports Monoid and Applicative, like base 4.8 prelude,
but works with older base versions.
- It exports (<>) for mappend.
- It hides 'catch' on older base versions.
This allows us to remove many imports of Data.Monoid
and Control.Applicative, and remove Text.Pandoc.Compat.Monoid.
It should allow us to use -Wall again for ghc 7.10.
Autgenerated using make_travis_yml.hs.
This script has been modified to add GHCOPTS.
'make .travis.yml' regenerates it based on the tested-with
field of the cabal file.
'xref' is used to create cross references to other parts of the
document. It is an empty element - the cross reference text depends on
various attributes. Quoting 'DocBook: The Definitive Guide':
1. If the endterm attribute is specified on xref, the content of the
element pointed to by endterm will be used as the text of the
cross-reference.
2. Otherwise, if the object pointed to has a specified XRefLabel, the
content of that attribute will be used as the cross-reference text.
The process was too fragile. It made too many assumptions about
available libraries (which failed sometimes when sandboxes were
used). This is a low-tech solution. The only drawback is that
`man/pandoc.1` is a generated file in the repository. It will need
to be regenerated periodically when README changes.
The pandoc.1 man page is generated automatically after the cabal
build process. It goes in `data/pandoc.1`. It can be obtained
by the user who installs pandoc via cabal thus:
pandoc --print-default-data-file pandoc.1 > pandoc.1
+ Removed `--man1`, `--man5` options (breaking change).
+ Removed `Text.Pandoc.ManPages` module (breaking API change).
+ Version bump to 1.15 because of the breaking changes, even
though they involve features that have only been in pandoc
for a day.
+ Makefile target for `man/man1/pandoc.1`. This uses pandoc to
create the man page from README using a custom template and filters.
+ Added `man/` directory with template and filters needed to build
man page.
+ We no longer have two man pages: pandoc.1 and pandoc_markdown.5.
Now there is just pandoc.1, which has all the content from README.
This change was needed because of the extensive cross-references
between parts of the README.
+ Removed old `data/pandoc.1.template` and
`data/pandoc_markdown.5.template`.
This change adds `--man1` and `--man5` options to pandoc, so
pandoc can generate its own man pages.
It removes the old overly complex method of building a separate
executable (but not installing it) just to create the man pages.
The man pages are no longer automatically created in the build
process.
The man/ directory has been removed. The man page templates
have been moved to data/.
New unexported module: Text.Pandoc.ManPages.
Text.Pandoc.Data now exports readmeFile, and `readDataFile`
knows how to find README.
Closes#2190.
In 1b44acf0c5 we replaced some
hackish CSS parsing with css-text, which I thought was a complete
CSS parser. It turns out that it is very buggy, which results
in lots of things being silently dropped from CSS when
`--self-contained` is used (#2224).
This commit replaces the use of css-text with a small but
more principled css preprocessor, which only removes whitespace
and replaces URLs with base 64 data when possible.
Closes#2224.
* Reverted kludgy change to make-windows-installer.bat.
* Removed make-reference-fiels.hs.
* Moved the individual ingredients of reference.docx and
reference.odt to the data directory.
* Removed reference.docx and reference.odt from data directory.
* We now build the reference archives from their ingredient pieces
in the docx and odt writers, instead of having a reference.docx
or reference.odt intermediary.
This should fix#2187.
It also simplifies the bulid procedure.
The one thing users may notice is different is that you can
no longer get the reference.docx or reference.odt using
`--print-default-data-file`. Instead, simply generate a
docx or odt using pandoc with a blank or minimal input,
and use that (or a customized version) with `--reference-docx`
or `--reference-odt`.
We only support the href attribute, as there's no place for
"target" in the Pandoc document model for links.
Added HTML reader test module, with tests for this feature.
Closes#1751.
This ensures that all code blocks will be wrapped in a div
with class sourceCode. Also, the default highlighting CSS
now adds `div.sourceCode { x-overflow: auto; }`, which means
that code blocks (even with line numbers) will acquire a scroll
bar on screens too small to display them (e.g. mobile phones).
See #1903 and jgm/highlighting-kate#65.
- Added commonmark as an input format.
- Added `Text.Pandoc.Readers.CommonMark.readCommonMark`.
- For now, we use the markdown writer to generate benchmark
text for the CommonMark reader. We can change this when we
get a writer.
This allows the test suite to be run using "+RTS -N".
Doing so improves the performance of the test suite on my quad-core Mac laptop as follows:
Before: 8.2 seconds
After: 2.5 seconds
Build dependencies of the trypandoc executable are required, regardless of the
trypandoc flag was set to either True or False. Correct package description
to make them truly optional.
Renamed some tests, introducing subsidiary directories
for fb2, docx, epub.
Cleaned up tests in cabal file.
Combined dokuwiki-writer and dokuwiki_inline_formatting tests.
This was just too fragile and dependent on a changing Cabal API
(see #1526).
Instead of passing the bulid directory to the test program, we
now let the test program find itself (using executable-path)
and then find the pandoc executable relative to itself.
Rewrote features test to remove all unimplemented features.
There are now all three examples of where an image can be included in
the test.
1. Cover image
2. As a spine elemnt
3. In the document
Tests have also been added to make sure that the mediabag contains all
these images after processing.
Renamed epub test files so they're identified more clearly as
epub: features.{epub,native} -> epub.features.{epub,native},
and similarly with formatting.{epub,native}.
Added epub test files to cabal file, so they'll be included in
the tarball.
This will allow us to test the whole mediabag (making sure, for example,
that images are added with the correct keys) instead of just individual
extracted images. We compare each entry in the media bag to an image
extracted on the fly from the docx. As a result, we only need one file
to test with.
The image in the current tests was also replaced with a smaller one.
Moved `MediaBag` definition and functions from Shared:
`lookupMedia`, `mediaDirectory`, `insertMedia`, `extractMediaBag`.
Removed `emptyMediaBag`; use `mempty` instead, since `MediaBag`
is a Monoid.
http://txt2tags.org/
There are two points which currently do not match the official
implementation.
1. In the official implementation lists can not be nested like the
following but the reader would interpret this as a bullet list with the
first item being a numbered list.
```
- + This is not a list
```
2. The specification describes how URIs automatically becomes links.
Unfortunately as is often the case, their definitiong of URI is not
clear. I tried three solutions but was unsure about which to adopt.
* Using isURI from Network.URI, this matches far too many strings and is
therefore unsuitable
* Using uri from Text.Pandoc.Shared, this doesn't match all strings that
the reference implementation matches
* Try to simulate the regex which is used in the native code
I went with the third approach but it is not perfect, for example
trailing punctuation is captured in Urls.
mtl switched from ErrorT to ExceptT, but we're not sure which mtl we'll
be dealing with. This should make errors work with both.
The main difference (beside the name of the module and the monad
transformer) is that Except doesn't require an instance of an Error
Typeclass. So we define that for compatability. When we switch to a
later mtl, using Control.Monad.Exception, we can just erase the instance
declaration, and all should work fine.
With the move from parsec to attoparsec, we lost good error
reporting. In fact, since we weren't testing for end of input,
malformed templates would fail silently. Here we revert back to
Parsec for better error messages.
This has fixes for unicode path names. Note that compiling
pandoc against zip-archive 0.2.3 or 0.2.3.1 will lead to invalid
zip containers, causing LibreOffice (e.g.) to regard ODTs as corrupt.
They aren't needed at runtime.
We keep README and COPYRIGHT in data to ensure that they'll be
available on all systems on which pandoc is installed.
Closes#1123.
Users of s5 and slideous will have to download the needed
files, as has been documented for some time in the README.
slidy files will be sought on the web, as before.
Previously we tried to remove make-pandoc-man-pages from the list
of packages to be haddocked, installed, copied, etc.
It works better to set 'Buildable: False' on make-pandoc-man-pages,
then have the buildHook temporarily set Buildable to True. This
allows make-pandoc-man-pages to be built (and used in generating
the man pages), but not installed.
Note that anything not parseable as a YAML boolean or string
is treated as a literal string.
Note that you can still get a string value with "yes" or any
of the strings interpretable as booleans:
-M boolvalue=yes -M stringvalue='"yes"'
Going forward we'll use pandoc-citeproc, as an external filter.
The `--bibliography`, `--csl`, and `--citation-abbreviation` fields
have been removed. Instead one must include `bibliography`, `csl`,
or `csl-abbrevs` fields in the document's YAML metadata. The filter
can then be used as follows:
pandoc --filter pandoc-citeproc
The `Text.Pandoc.Biblio` module has been removed. Henceforth,
`Text.CSL.Pandoc` from pandoc-citations can be used by library users.
The Markdown and LaTeX readers now longer format bibliographies and
citations. That must be done using `processCites` or `processCites'`
from Text.CSL.Pandoc.
All bibliography-related fields have been removed from `ReaderOptions`
and `WriterOptions`: `writerBiblioFiles`, `readerReferences`,
`readerCitationStyle`.
API change.
This way filters can figure out what the target format is
and react appropriately.
Example:
#!/usr/bin/env runghc
import Text.Pandoc.JSON
import Data.Char
main = toJSONFilter cap
where cap (Just "html") (Str xs) = Str $ map toUpper xs
cap _ x = x
This capitalizes text only for html output.
* Text.Pandoc.Writers.Shared contains shared functions used
only in writers.
* metaToJSON now takes a WriterOptions parameter, and will
return an empty object if standalone is not specified.
* Depend on pandoc 1.12.
* Added yaml dependency.
* `Text.Pandoc.XML`: Removed `stripTags`. (API change.)
* `Text.Pandoc.Shared`: Added `metaToJSON`.
This will be used in writers to create a JSON object for use
in the templates from the pandoc metadata.
* Revised readers and writers to use the new Meta type.
* `Text.Pandoc.Options`: Added `Ext_yaml_title_block`.
* Markdown reader: Added support for YAML metadata block.
Note that it must come at the beginning of the document.
* `Text.Pandoc.Parsing.ParserState`: Replace `stateTitle`,
`stateAuthors`, `stateDate` with `stateMeta`.
* RST reader: Improved metadata.
Treat initial field list as metadata when standalone specified.
Previously ALL fields "title", "author", "date" in field lists
were treated as metadata, even if not at the beginning.
Use `subtitle` metadata field for subtitle.
* `Text.Pandoc.Templates`: Export `renderTemplate'` that takes a string
instead of a compiled template..
* OPML template: Use 'for' loop for authors.
* Org template: '#+TITLE:' is inserted before the title.
Previously the writer did this.
Currently the library is set up with a shim for association
lists, for compatibility, but this can change when the writers
are changed.
New export: `varListToJSON`.
Removed `Empty`.
Simplified template type to a newtype.
Reason: the path to the system temp directory may contain tildes,
which causes problems in LaTeX when the username is more than
eight characters.
Closes#777.
pandoc -t data/sample.lua
will load the script sample.lua and use it as a custom writer.
data/sample.lua is provided as an example.
Added `--print-custom-lua-writer` option to print the sample
script.
The _note attribute is supported. This is unofficial, but
used e.g. in OmniOutliner and supported by multimarkdown.
We treat the contents as markdown blocks under a section
header.
Added to documentation and tests.
Previously, a LaTeX citation would always be parsed as a Citation
element, with the raw LaTeX in the [Inline] part.
Now, the LaTeX citation is parsed as a Citation element only if
`--biblio` was specified (i.e. only if there is a nonempty set
of references in readerReferences). Otherwise it is parsed as
raw LaTeX.
This will make it possible to simplify some things in the markdown
writer. It also makes the LaTeX reader behave more like the Markdown
reader.
* Moved code for translating listings language names to
highlighting-kate names and back from LaTeX reader to Highlighting.
* Text.Pandoc.Highlighting no longer exposed (API change)
* Text.Pandoc.Highlighting exports toListingsLang, fromListingsLang
Pandoc will compile with older versions, but there will be bugs
relating to these functions. Making the current versions dependencies
is the only way to get packages updated in the distros.
Previously header ids were autogenerated by the writers.
Now they are generated (unless supplied explicitly) in the
markdown parser, if the `header_identifiers` extension is
selected.
In addition, the textile reader now supports id attributes on
headers.
* Added `embed_data_files` flag. (not yet used)
* Shared no longer exports `findDataFile`.
* `readDataFile` now returns a strict bytestring.
* Shared now exports `readDataFileUTF8` which returns a string like
the old `readDataFile`.
* Rewrote modules to use new data file functions and to avoid
using functions from Paths_pandoc directly.
* Remove executable and library flags.
* Expose `Text.Pandoc.XML` and `Text.Pandoc.Biblio`.
* Depend on pandoc library in executable, so we don't recompile
everything.
* Move pandoc.hs from src/ to .
* Previously there were three different templates involved in
epub production. There is now just one template, default.epub
or default.epub3.
* It can now be overridden using `--template`, just like other
templates.
* MakeManPage.hs has been transformed into
man/make-pandoc-man-pages.hs.
* There is now a cabal stanza for this, so the dependencies are
handled by cabal.
* Special treatment in Setup.hs ensures that this never gets installed;
it is built and used to create the man pages.
* Setup.hs cleaned up.
* Depend on text.
* Expose Text.Pandoc.UTF8.
* Text.Pandoc.UTF8 now exports toString, fromString,
toStringLazy, fromStringLazy.
* These are used instead of the old utf8-string functions.
- Removed writerLiterateHaskell from WriterOptions.
- Removed readerLiterateHaskell from ReaderOptions.
- Added Ext_literate_haskell to Extensions. Test for this
instead of the above.
- Removed failUnlessLHS from Shared.
Note: At this point, +lhs and .lhs extension no longer has any effect.
Need to fix.
To run tests, configure with --enable-tests, then 'cabal test'.
You can specify particular tests using --test-options='-t markdown'.
No output is shown unless tests fail. In the future, we can move
to the detailed-1.0 interface.
* All tables now require at least one body row.
* Renamed from 'extra' to 'pipe' tables.
* Moved functions from Parsing to Readers.Markdown.
* Cleaned up code; revised to parse in one pass rather than
parsing a raw string, splitting it, and parsing the components.
* Allow pipe tables without pipes on the ends (as PHP Markdown Extra
does).
Now you can use def (which is re-exported by Text.Pandoc) instead of
defaultParserState or defaultWriterOptions. For now, these
are still defined too, so existing code need not change.
Closes#546.
Use of the threaded runtime imposes a slight performance penalty,
and there is really no need for it, since pandoc doesn't run faster
on multiple cores.
Otherwise even those who don't build with the tests flag need
to have dependencies like test-framework.
This restores the old behavior; I wrongly assumed that by now cabal
was smart enough not to bring in build-depends for components with
Buildable = False.
* New module `Text.Pandoc.Docx`.
* New output format `docx`.
* Added reference.docx.
* New option `--reference-docx`.
The writer includes support for highlighted code blocks
and math (which is converted from TeX to OMML using
texmath's new OMML module).
This is the unaltered file from the dzslides repository.
Pandoc now reads it, looking for the core part, and includes
this in the new dzslides template via the 'dzslides-core'
variable.
When dzslides is updated, you can just put the new
template.html in your `~/.pandoc/dzslides` directory,
and things should work -- provided the core part can
be identified as everything from
<!-- {{{{ dzslides core
to the end of the file.
This should make it a bit easier to keep up to date.
* This is a breaking API change for `writeHtml`.
* It introduces a new dependency on blaze-html.
* Pandoc now depends on highlighting-kate >= 0.4, which
also uses blaze-html.
* The --ascii option has been removed, because of differences
in blaze-html's and xhtml's escaping.
* Pandoc will no longer transform leading newlines in code
blocks to `<br/>` tags.
Still TODO:
- documentation in README
- add default.asciidoc to templates/
- lists
- tables
- proper escaping
- footnotes with blank lines - print separately at end?
currently they are just ignored.
- fix header (date gives weird result on pandoc README)
In general I don't like adding generated content to the repository,
but I also want to make it possible to clone the repository and
'cabal install'. THe current system with Setup.hs calling
MakeManPage.hs is too fragile.
* Use non-minimized version of `slidy.css` with `--offline` option,
so users can more easily edit it.
* Fixed bug in slidy css that prevented proper centering of title.
Reason: new naming scheme for templates may require users
to rename default templates they have put in ~./pandoc/templates.
A template named FORMAT.template should be renamed default.FORMAT.
Instead of latex.template, we now have default.latex.
An appropriate extension is added automatically if the value of
`--template` has no extension. So, `pandoc --template=special -t latex`
looks for `special.latex`, while `pandoc --template=special -t man`
looks for `special.man`.
* Markdown syntax description from README now goes in pandoc_markdown.5.
* Refactored man page construction functions, putting more of
the work in MakeManPages.hs.
Additional related changes:
* URLs in Code in autolinks now use class "url".
* Require highlighting-kate 0.2.8.2, which omits the final <br/> tag,
essential for inline code.
The old TeX, HtmlInline and RawHtml elements have been removed
and replaced by generic RawInline and RawBlock elements.
All modules updated to use the new raw elements.
readNative can now read full pandoc documents, block lists, blocks,
inline lists, or inlines. It will interpret
Str "hi"
as if it were
Pandoc (Meta [] [] []) [Plain [Str "hi"]]
This should make testing easier.
The man pages are produced by a post-build hook, using
MakeManPage.hs. However, there seem to be problems running
this program on some systems; see
https://github.com/jgm/pandoc/issues/#issue/15.
This change means that the generated man pages will be included in
the tarball produced by 'cabal sdist', and hence in releases.
They will still be updated by the build process if the sources
change, but in most builds, the man page won't need to be regenerated.
+ You can now specify glob patterns after 'cabal test';
e.g. 'cabal test latex' will only run the latex tests.
+ Instead of detecting highlighting support in Setup.hs,
we now detect it in test-pandoc, by looking to see if
'languages' is null.
+ We now verify the lhs readers against the lhs-test.native,
normalizing with 'normalize'. This makes more sense than
verifying against HTML, which also brings in the HTML writer.
+ Added lhsn-test.nohl.{html,html+lhs}, so we can do the lhs
tests whether or not highlighting has been installed.
