Change --metadata-file parsing...

...so that, when the input format is not markdown or a markdown
variant, pandoc's markdown is used.  When the input format is
a markdown variant, the same format is used.  Reason for the change:
it doesn't make sense to run the markdown parser with a set of
extensions designed for a non-markdown format, and this dramatically
limits what people can do in metadata files.

Refines #6832.  Closes #7926.

Perhaps this can be reconsidered if we come up with a way
of specifying an arbitrary format for the metadata file (#5914).

MANUAL.txt

@@ -641,10 +641,12 @@ header when requesting a document from a URL:
 
 :   Read metadata from the supplied YAML (or JSON) file. This
     option can be used with every input format, but string scalars
-    in the YAML file will always be parsed as Markdown. Generally,
-    the input will be handled the same as in [YAML metadata
-    blocks][Extension: `yaml_metadata_block`]. This option can be
-    used repeatedly to include multiple metadata files; values in
+    in the YAML file will always be parsed as Markdown. (If the
+    input format is Markdown or a Markdown variant, then the
+    same variant will be used to parse the metadata file;
+    if it is a non-Markdown format, pandoc's default Markdown
+    extensions will be used.) This option can be used
+    repeatedly to include multiple metadata files; values in
     files specified later on the command line will be preferred
     over those specified in earlier files. Metadata values
     specified inside the document, or by using `-M`, overwrite

src/Text/Pandoc/App.hs

@@ -218,9 +218,16 @@ convertWithOpts opts = do
     metadataFromFile <-
       case optMetadataFiles opts of
         []    -> return mempty
-        paths -> mconcat <$>
-           mapM (\path -> do raw <- readMetadataFile path
-                             yamlToMeta readerOpts (Just path) raw) paths
+        paths -> do
+          -- If format is markdown or commonmark, use the enabled extensions,
+          -- otherwise treat metadata as pandoc markdown (see #7926, #6832)
+          let readerOptsMeta =
+                if readerNameBase == "markdown" || readerNameBase == "commonmark"
+                   then readerOpts
+                   else readerOpts{ readerExtensions = pandocExtensions }
+          mconcat <$> mapM
+                (\path -> do raw <- readMetadataFile path
+                             yamlToMeta readerOptsMeta (Just path) raw) paths
 
     let transforms = (case optShiftHeadingLevelBy opts of
                           0             -> id