Commit graph

157 commits

Author SHA1 Message Date
fiddlosopher
4399db4fd2 Bug fixes in readers:
+ LaTeX reader:  skip anything after \end{document}
+ HTML reader: fixed bug skipping material after </html> -- previously,
  stuff at the end was skipped even if no </html> was present, which
  meant only part of the file would be parsed and no error issued
+ HTML reader: added new constant eitherBlockOrInline with elements that
  may count either as block-level or inline
+ Modified isInline and isBlock to take this into account
+ modified rawHtmlBlock to accept any tag (even an inline tag);
  this is innocuous, because rawHtmlBlock is tried only if a regular
  inline element can't be parsed.


git-svn-id: https://pandoc.googlecode.com/svn/trunk@862 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-08-18 23:44:26 +00:00
fiddlosopher
e48f046aa0 + Fixed bug in markdown ordered list parsing. The problem was
that anyOrderedListStart did not check for a space following the
  ordered list marker. So, 'A.B. 2007' would be parsed as a list item,
  then fail because of the lack of space after 'A.' (required by
  orderedListStart). Resolves Issue #22.
+ Fixed a similar problem in RST reader.
+ Added regression test.


git-svn-id: https://pandoc.googlecode.com/svn/trunk@861 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-08-18 15:26:29 +00:00
fiddlosopher
3904078e39 Cosmetic changes.
git-svn-id: https://pandoc.googlecode.com/svn/trunk@851 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-08-15 21:20:02 +00:00
fiddlosopher
1a4489ef30 LaTeX reader: parse \texttt{} as code, as long as there's
nothing fancy inside.


git-svn-id: https://pandoc.googlecode.com/svn/trunk@846 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-08-15 17:30:37 +00:00
fiddlosopher
6cc5f6b199 Allow htmlComments as rawHtmlInline in HTML reader.
git-svn-id: https://pandoc.googlecode.com/svn/trunk@844 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-08-15 17:28:21 +00:00
fiddlosopher
a8e2199034 Major code cleanup in all modules. (Removed unneeded imports,
reformatted, etc.)  More major changes are documented below:

+ Removed Text.Pandoc.ParserCombinators and moved all its definitions
  to Text.Pandoc.Shared.
+ In Text.Pandoc.Shared:
  - Removed unneeded 'try' in blanklines.
  - Removed endsWith function and rewrote functions to use isSuffixOf instead.
  - Added >>~ combinator.
  - Rewrote stripTrailingNewlines, removeLeadingSpaces.
+ Moved Text.Pandoc.Entities -> Text.Pandoc.CharacterReferences.
  - Removed unneeded functions charToEntity, charToNumericalEntity.
  - Renamed functions using proper terminology (character references,
    not entities).  decodeEntities -> decodeCharacterReferences,
    characterEntity -> characterReference.
  - Moved escapeStringToXML to Docbook writer, which is the only thing
    that uses it.
  - Removed old entity parser in HTML and Markdown readers; replaced with
    new charRef parser in Text.Pandoc.Shared.
+ Fixed accent bug in Text.Pandoc.Readers.LaTeX:  \^{} now correctly
  parses as a '^' character.
+ Text.Pandoc.ASCIIMathML is no longer an exported module.


git-svn-id: https://pandoc.googlecode.com/svn/trunk@835 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-08-15 06:00:58 +00:00
fiddlosopher
e814a3f6d2 Major change in the way ordered lists are handled:
+ The changes are documented in README, under Lists.
+ The OrderedList block element now stores information
  about list number style, list number delimiter, and
  starting number.
+ The readers parse this information, when possible.
+ The writers use this information to style ordered
  lists.
+ Test suites have been changed accordingly.

Motivation:  It's often useful to start lists with
numbers other than 1, and to have control over the
style of the list.

Added to Text.Pandoc.Shared:
+ camelCaseToHyphenated
+ toRomanNumeral
+ anyOrderedListMarker
+ orderedListMarker
+ orderedListMarkers

Added to Text.Pandoc.ParserCombinators:
+ charsInBalanced'
+ withHorizDisplacement
+ romanNumeral

RST writer:
+ Force blank line before lists, so that sublists will be handled
  correctly.

LaTeX reader:
+ Fixed bug in parsing of footnotes containing multiple paragraphs,
  introduced by use of charsInBalanced.  Fix: use charsInBalanced'
  instead.

LaTeX header:
+ use mathletters option in ucs package, so that basic unicode Greek
  letters will work properly.


git-svn-id: https://pandoc.googlecode.com/svn/trunk@834 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-08-08 02:43:15 +00:00
fiddlosopher
22a6538557 Added parsing for \url to LaTeX reader.
git-svn-id: https://pandoc.googlecode.com/svn/trunk@833 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-08-03 22:44:47 +00:00
fiddlosopher
44b136740b Changes to Markdown reader:
+ added try to def of indentSpaces
+ in def of 'reference', check to make sure it's not a note reference
  first.


git-svn-id: https://pandoc.googlecode.com/svn/trunk@827 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-07-28 19:14:50 +00:00
fiddlosopher
5e13f8c320 Removed examplep specific stuff in LaTeX reader.
git-svn-id: https://pandoc.googlecode.com/svn/trunk@826 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-07-28 18:23:13 +00:00
fiddlosopher
d3404581d4 Changes in LaTeX reader to accommodate Pandoc's own use of
examplep.


git-svn-id: https://pandoc.googlecode.com/svn/trunk@818 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-07-28 05:45:27 +00:00
fiddlosopher
9939b0f07e Make URLs and emails in autolinks appear as Code.
git-svn-id: https://pandoc.googlecode.com/svn/trunk@810 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-07-28 02:12:40 +00:00
fiddlosopher
d8762eb436 In HTML reader, filter Nulls in lists of blocks. (These can
be caused by raw HTML when the parse-raw option isn't selected.)


git-svn-id: https://pandoc.googlecode.com/svn/trunk@787 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-07-23 02:50:29 +00:00
fiddlosopher
9f871ab1ec Fixed bug in spanStrikeout: case was not exhaustive.
git-svn-id: https://pandoc.googlecode.com/svn/trunk@786 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-07-23 02:28:15 +00:00
fiddlosopher
9a410e1635 README: Removed the statement that the RST reader doesn't parse
definition lists.
HTML reader:  Added failIfStrict to the definitionList parser, so
definition lists will be passed through as raw HTML if --strict
specified.


git-svn-id: https://pandoc.googlecode.com/svn/trunk@783 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-07-23 01:41:37 +00:00
fiddlosopher
6a6b1a5842 Added support for definition lists to RST reader.
Added a relevant test to the test suite.


git-svn-id: https://pandoc.googlecode.com/svn/trunk@782 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-07-23 01:35:03 +00:00
fiddlosopher
f69d9efc83 Added definition list support to HTML reader.
Added a test for definition lists to the html-reader test suite.


git-svn-id: https://pandoc.googlecode.com/svn/trunk@781 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-07-23 01:21:21 +00:00
fiddlosopher
1296273c85 Added support for definition lists to LaTeX reader.
git-svn-id: https://pandoc.googlecode.com/svn/trunk@780 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-07-23 00:19:37 +00:00
fiddlosopher
caef362065 Renamed parseFromStr -> parseFromString.
git-svn-id: https://pandoc.googlecode.com/svn/trunk@779 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-07-23 00:19:00 +00:00
fiddlosopher
4beb0b9130 Superscript and Subscript support for RST reader.
git-svn-id: https://pandoc.googlecode.com/svn/trunk@777 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-07-22 21:26:22 +00:00
fiddlosopher
c1f397622d Added a "try" to the end parser in enclosed (Text.Pandoc.ParserCombinators).
This makes errors in its use less likely. Removed some now-unneeded try's
in calling code.


git-svn-id: https://pandoc.googlecode.com/svn/trunk@776 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-07-22 21:01:17 +00:00
fiddlosopher
3ca9932ca6 Added subscript and superscript support to LaTeX reader.
git-svn-id: https://pandoc.googlecode.com/svn/trunk@775 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-07-22 20:30:07 +00:00
fiddlosopher
4905ebadb3 LaTeX reader: Added clauses for tilde and caret.
Tilde is \ensuremath{\sim}, and caret is \^{},
not \^ as before.


git-svn-id: https://pandoc.googlecode.com/svn/trunk@764 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-07-22 18:26:10 +00:00
fiddlosopher
b19c36970e Removed an extra occurance of escapedChar in definition
of inline.


git-svn-id: https://pandoc.googlecode.com/svn/trunk@762 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-07-22 17:51:15 +00:00
fiddlosopher
d03ec5a4a2 Added support for strikeout (\sout) to latex
reader.  (Thanks to Bradley Sif for the patch.)


git-svn-id: https://pandoc.googlecode.com/svn/trunk@754 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-07-22 05:14:43 +00:00
fiddlosopher
a2194f23db Added support for Strikeout, Superscript, and Subscript to
HTML reader.  Thanks to Bradley Sif for the patch for
Strikeout (Issue #18).


git-svn-id: https://pandoc.googlecode.com/svn/trunk@753 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-07-21 22:54:40 +00:00
fiddlosopher
44b11214ba + Added support for Strikeout, Superscript, and
Subscript in markdown reader.
+ Also replaced constants like emphStart with literals. 


git-svn-id: https://pandoc.googlecode.com/svn/trunk@752 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-07-21 22:52:07 +00:00
fiddlosopher
48d67d0d1f Simplified inlinesInBalanced, using lookAhead.
git-svn-id: https://pandoc.googlecode.com/svn/trunk@732 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-07-16 07:22:17 +00:00
fiddlosopher
7e1370aa87 Markdown reader: Added inlinesInBalanced parser combinator to
unify treatment of embedded brackets in links and inline footnotes.
Note that the solution adopted here causes one of John Gruber's
markdown tests to fail:
  [with_underscore](/url/with_underscore)
Here the whole phrase "underscore](/url/with" is treated as
emphasized. The previous version of the markdown reader handled
this the way Gruber's script handles it, but it ran into trouble
on the following:
  [link with verbatim `]`](/url)
where the inner ] was treated as the end of the reference link
label. I don't see any good way to handle both cases in the framework
of pandoc, so I choose to require an escape in the first example:
  [with\_underscore](/url/with_underscore)


git-svn-id: https://pandoc.googlecode.com/svn/trunk@729 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-07-15 23:53:22 +00:00
fiddlosopher
e962c76f05 HTML reader: haddock comment fix.
git-svn-id: https://pandoc.googlecode.com/svn/trunk@690 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-07-12 08:31:39 +00:00
fiddlosopher
1780228b7c Markdown reader: Parse bracketed text in inline footnotes. Previously,
"test^[my [note] contains brackets]" would yield a note with contents
"my [note". Now it yields a note with contents "my [note] contains
brackets".  New function: inlinesInBrackets.

Resolves Issue 14. 


git-svn-id: https://pandoc.googlecode.com/svn/trunk@665 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-07-09 06:51:06 +00:00
fiddlosopher
d58dca502e RST reader: Allow hyperlink target URIs to be split over multiple
lines, and to start on the line after the reference.
Resolves Issue 7. 


git-svn-id: https://pandoc.googlecode.com/svn/trunk@664 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-07-09 06:23:59 +00:00
fiddlosopher
655363da51 Moved Text.ParserCombinators.Pandoc ->
Text.Pandoc.ParserCombinators.  This way, all the
Pandoc modules are in one place.


git-svn-id: https://pandoc.googlecode.com/svn/trunk@663 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-07-09 03:39:25 +00:00
fiddlosopher
a414ec9d63 Adjusted copyright notices to 2006-7; use
real email address instead of lamely attempting
to obfuscate.


git-svn-id: https://pandoc.googlecode.com/svn/trunk@640 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-07-07 22:51:55 +00:00
fiddlosopher
f9c988e703 Fixed bug in Markdown reader: links in footnotes were not
being processed.  Solution:  three-stage parse.  First, get
all the reference keys and add information to state.  Next,
get all the notes and add information to state.  (Reference
keys may be needed at this stage.)  Finally, parse everything
else.


git-svn-id: https://pandoc.googlecode.com/svn/trunk@625 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-07-06 06:46:31 +00:00
fiddlosopher
61024d93ed Require blankspace (but not multiple lines) between URL and
title in links and reference keys.  (Markdown reader.)


git-svn-id: https://pandoc.googlecode.com/svn/trunk@599 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-05-10 22:06:13 +00:00
fiddlosopher
cf081435ff Changed definition list syntax in markdown reader and simplified
the parsing code. A colon is now required before every block in a
definition. This fixes a problem with the old syntax, in which the last
block in the following was ambiguous between a regular paragraph in the
definition and a code block following the definition list:

term
:   definition

    is this code or more definition?



git-svn-id: https://pandoc.googlecode.com/svn/trunk@589 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-05-03 14:42:40 +00:00
fiddlosopher
485fa81559 Resolved issue #10: instead of adding "\n\n" to the
end of strings in Main, do it in readMarkdown and readRST.
(Note: the point of this is to ensure that a block at the
end of the file gets treated as if it has blank space after
it, which is generally what is wanted.)


git-svn-id: https://pandoc.googlecode.com/svn/trunk@588 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-04-22 04:38:05 +00:00
fiddlosopher
23df0ed176 Extensive changes stemming from a rethinking of the Pandoc data
structure. Key and Note blocks have been removed. Link and image URLs
are now stored directly in Link and Image inlines, and note blocks
are stored in Note inlines. This requires changes in both parsers
and writers. Markdown and RST parsers need to extract data from key
and note blocks and insert them into the relevant inline elements.
Other parsers can be simplified, since there is no longer any need to
construct separate key and note blocks. Markdown, RST, and HTML writers
need to construct lists of notes; Markdown and RST writers need to
construct lists of link references (when the --reference-links option
is specified); and the RST writer needs to construct a list of image
substitution references. All writers have been rewritten to use the
State monad when state is required.  This rewrite yields a small speed
boost and considerably cleaner code. 

* Text/Pandoc/Definition.hs:
  + blocks:  removed Key and Note
  + inlines:  removed NoteRef, added Note
  + modified Target:  there is no longer a 'Ref' target; all targets
    are explicit URL, title pairs

* Text/Pandoc/Shared.hs:

  + Added 'Reference', 'isNoteBlock', 'isKeyBlock', 'isLineClump',
    used in some of the readers.
  + Removed 'generateReference', 'keyTable', 'replaceReferenceLinks',
    'replaceRefLinksBlockList', along with some auxiliary functions
    used only by them.  These are no longer needed, since
    reference links are resolved in the Markdown and RST readers.
  + Moved 'inTags', 'selfClosingTag', 'inTagsSimple', and 'inTagsIndented'
    to the Docbook writer, since that is now the only module that uses
    them.
  + Changed name of 'escapeSGMLString' to 'escapeStringForXML'
  + Added KeyTable and NoteTable types
  + Removed fields from ParserState;  'stateKeyBlocks', 'stateKeysUsed',
    'stateNoteBlocks', 'stateNoteIdentifiers', 'stateInlineLinks'. 
    Added 'stateKeys' and 'stateNotes'.
  + Added clause for Note to 'prettyBlock'.
  + Added 'writerNotes', 'writerReferenceLinks' fields to WriterOptions.

* Text/Pandoc/Entities.hs: Renamed 'escapeSGMLChar' and
  'escapeSGMLString' to 'escapeCharForXML' and 'escapeStringForXML'

* Text/ParserCombinators/Pandoc.hs: Added lineClump parser: parses a raw
  line block up to and including following blank lines.

* Main.hs:  Replaced --inline-links with --reference-links.

* README: 
  + Documented --reference-links and removed description of --inline-links.
  + Added note that footnotes may occur anywhere in the document, but must
    be at the outer level, not embedded in block elements.
  
* man/man1/pandoc.1, man/man1/html2markdown.1: Removed --inline-links
  option, added --reference-links option

* Markdown and RST readers:
  + Rewrote to fit new Pandoc definition.  Since there are no longer
    Note or Key blocks, all note and key blocks are parsed on a first pass
    through the document.  Once tables of notes and keys have been constructed,
    the remaining parts of the document are reassembled and parsed.
  + Refactored link parsers.

* LaTeX and HTML readers: Rewrote to fit new Pandoc definition. Since
  there are no longer Note or Key blocks, notes and references can be
  parsed in a single pass through the document.

* RST, Markdown, and HTML writers: Rewrote using state monad new Pandoc
  and definition. State is used to hold lists of references footnotes to
  and be printed at the end of the document.

* RTF and LaTeX writers: Rewrote using new Pandoc definition. (Because
  of the different treatment of footnotes, the "notes" parameter is no
  longer needed in the block and inline conversion functions.)

* Docbook writer:
  + Moved the functions 'attributeList', 'inTags', 'selfClosingTag',
    'inTagsSimple', 'inTagsIndented' from Text/Pandoc/Shared, since
    they are now used only by the Docbook writer.
  + Rewrote using new Pandoc definition.  (Because of the different
    treatment of footnotes, the "notes" parameter is no longer needed
    in the block and inline conversion functions.)

* Updated test suite

* Throughout:  old haskell98 module names replaced by hierarchical module
  names, e.g. List by Data.List.

* debian/control: Include libghc6-xhtml-dev instead of libghc6-html-dev
  in "Build-Depends."

* cabalize: 
  + Remove haskell98 from BASE_DEPENDS (since now the new hierarchical
    module names are being used throughout)
  + Added mtl to BASE_DEPENDS (needed for state monad)
  + Removed html from GHC66_DEPENDS (not needed since xhtml is now used)



git-svn-id: https://pandoc.googlecode.com/svn/trunk@580 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-04-10 01:56:50 +00:00
fiddlosopher
571c3b4173 Removed Blank block element as unnecessary.
git-svn-id: https://pandoc.googlecode.com/svn/trunk@578 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-03-17 20:23:47 +00:00
fiddlosopher
9b3d5d88c2 Consolidated 'text', 'special', and 'inline' into 'inline'.
git-svn-id: https://pandoc.googlecode.com/svn/trunk@577 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-03-17 17:25:28 +00:00
fiddlosopher
cdf7c78b5f Added trys to two list start routines. Reason:
<|> only parses second parser when first hasn't consumed
input.


git-svn-id: https://pandoc.googlecode.com/svn/trunk@576 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-03-16 06:05:43 +00:00
fiddlosopher
7bf7aba7e7 Changes to Markdown reader relating to definition lists:
+ fixed bug in indentSpaces (which didn't properly handle
  cases with mixed spaces and tabs)
+ rewrote definition list code to conform to new syntax
+ include definition lists in list block
+ failIfStrict on definition lists


git-svn-id: https://pandoc.googlecode.com/svn/trunk@572 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-03-11 07:56:29 +00:00
fiddlosopher
115cad8882 Changed syntax of definition lists in Markdown parser:
+ definition blocks must be indented throughout (not
    just in first line)
  + compact lists can be formed by leaving no blank line
    between a definition and the next term


git-svn-id: https://pandoc.googlecode.com/svn/trunk@568 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-03-10 20:45:19 +00:00
fiddlosopher
5ec31cc727 Added parser for definition lists, derived from reStructuredText
syntax:

term 1
    Definition 1

    Paragraph 2 of definition 1.

term 2
    There must be whitespace between entries.
    Any kind of block may serve as a definition,
    but the first line of each block must be indented.

terms can contain any *inline* elements
    If you want to be lazy, you can just
indent the first line of the definition block.



git-svn-id: https://pandoc.googlecode.com/svn/trunk@566 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-03-10 17:48:16 +00:00
fiddlosopher
d5f9b1dbb4 Change in ordered lists in Markdown reader:
+ Lists may begin with lowercase letters only, and only 'a' through
    'n'. Otherwise first initials and page references (e.g., p. 400)
    are too easily parsed as lists.
  + Numbers beginning list items must end with '.' (not ')', which is
    now allowed only after letters).
NOTE:  This change may cause documents to be parsed differently.
Users should take care in upgrading.


git-svn-id: https://pandoc.googlecode.com/svn/trunk@561 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-03-09 02:37:49 +00:00
fiddlosopher
f1191f029a More smart quote adjustments:
+ remove support for all-caps contractions (too
  much potential for conflict with things like
  'M. Mitterand')
+ add support for 'm as a contraction


git-svn-id: https://pandoc.googlecode.com/svn/trunk@560 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-03-07 20:53:37 +00:00
fiddlosopher
42ade05ebe Smart quote parsing in Markdown reader:
treat ' followed by ll, re, ve, then a
non-letter, as a contraction.  (e.g.
I've, you're, he'll)


git-svn-id: https://pandoc.googlecode.com/svn/trunk@559 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-03-07 03:01:43 +00:00
fiddlosopher
21ba6b08f2 Made image parsing in HTML reader sensitive to the
--inline-links option.


git-svn-id: https://pandoc.googlecode.com/svn/trunk@556 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-03-03 18:25:49 +00:00
fiddlosopher
31c030e3a5 Added --inline-links option to force links in HTML to be parsed
as inline links, rather than reference links.  (Addresses Issue
#4.)


git-svn-id: https://pandoc.googlecode.com/svn/trunk@554 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-03-03 18:19:31 +00:00