Pandoc extensions?

[haoyun]

I would like to ask if it's possible to use any pandoc extensions and how.

For example, https://pandoc.org/MANUAL.html#extension-east_asian_line_breaks is quite useful when writing notes in CJK.

Currently, a markdown

中文中文中文中文中文中文中文中文中文中文中文中文中文
断行断行断行断行断行断行断行断行断行断行断行断行断行

will be generated as

中文中文中文中文中文中文中文中文中文中文中文中文中文 断行断行断行断行断行断行断行断行断行断行断行断行断行

that is, there is an extra space in between.

This is just to show an example, there are many other extensions that are useful too.

As far as I know, internally emanote uses Pandoc AST, so I think there should be some solutions.

[srid]

Definitely worth adding this to Emanote. Perhaps let user specify it in YAML meta.

https://hackage.haskell.org/package/pandoc-2.17.1.1/docs/Text-Pandoc-Extensions.html#v:Ext_east_asian_line_breaks

But emanote uses commonmark-hs (from same author) to parse Markdown. I'm okay with us switching to parsing with Pandoc. The Wikilink extension, as well as the HashTag extension, would be the main thing to port over.

[haoyun]

Thank you!

Personally I prefer Pandoc Reader being used instead of commonmark-hs since Pandoc could do more. I would also be happy that there will be some interfaces available in emanote, so that in some cases, I could even use Pandoc Writer or Pandoc filter. That's related to the discussions here at #195.

Morover, as said in the link you provided, the extension Ext_east_asian_line_breaks is actually a Pandoc filter which is applied after parsing. That's the reason why I am asking if there is any way to provide a Pandoc filter to transform the document.

[srid]

That's the reason why I am asking if there is any way to provide a Pandoc filter to transform the document.

Yes, we can let the user specify these filters in their index.yaml, and have Emanote run those filters (Pandoc -> Pandoc?) automatically. I'd be open to a PR on that.

It should be a fairly simple PR I guess. We want to call this function: https://github.com/jgm/pandoc/blob/d9ec95e7abc71a642dd8e53b92f2bd3ae1609f10/src/Text/Pandoc/Shared.hs#L703-L715

[srid]

Or maybe we should just switch from commonmark-hs to Pandoc, then a larger range of reader and writer extensions can be enabled from index.yaml.

[Ninlives]

Would definitely be nice to have a configuration for user defined filters an extensions!
My personal requirement is to have the superscript and subscript extension, currently it seems the only way is to patch the source code (subscript does not work though).

[haoyun]

Just found out this related comment from pandoc

jgm/pandoc#2923 (comment)

[srid]

Emanote definitely should support lua filters so that users can include these filters in their notebook along with .md and .tpl files.

You should be able to apply these filters on per-note or per-folder basis via specifying them in .yml files. Just like you can specify a .tpl template.

#263

[srid]

Lua filters are now supported! In a beta form though, the implementation is not complete but things basically work.

https://emanote.srid.ca/demo/lua-filters

Try it out, and give me feedback. :-)

[fl0pedro]

This is quite a late reply to the discussion, but what about other extensions? I am looking to enable tex_math_single_backslash or rather Ext_tex_math_single_backslash in haskell. I didn't see any docs on this on any config and I don't see where it could be easily added in, though I am not too familiar with haskell. Is this something I am missing? Otherwise, would this be something interesting to add support to? Could this maybe be specified via some config, to add support to a variety of pandoc extensions?

[srid]

@fl0pedro Extensions are specified here:

emanote/emanote/src/Emanote/Pandoc/Markdown/Parser.hs

Lines 31 to 51 in 58a7dad

	where
	baseExtsSansPipeTable =
	mconcat
	[ CE.fancyListSpec
	, CE.footnoteSpec
	, CE.mathSpec
	, CE.smartPunctuationSpec
	, CE.definitionListSpec
	, CE.attributesSpec
	, CE.rawAttributeSpec
	, CE.fencedDivSpec
	, CE.bracketedSpanSpec
	, CE.autolinkSpec
	, CM.defaultSyntaxSpec
	]
	gfmExtensionsSansPipeTable =
	CE.emojiSpec
	<> CE.strikethroughSpec
	<> CE.autolinkSpec
	<> CE.autoIdentifiersSpec
	<> CE.taskListSpec

We use commonmark-hs (which Pandoc uses, AFAICT):

https://github.com/jgm/commonmark-hs/tree/master/commonmark-extensions

[fl0pedro]

Yeah I got that far as well. What I have gathered is that although Pandoc supports commonmark-hs, their implementations are still different. For one Pandoc has more extensions (the topic of discussion). For example, commonmark only supports one type of math mode, while Pandoc supports three different kinds:

https://github.com/jgm/pandoc/blob/main/src/Text/Pandoc/Parsing/Math.hs

I am not sure how easy/difficult it may be to make these extensions customizable by some config, but its the first step IMO.

I am still figuring out how to actually replace commonmark with Pandoc, at least for the extensions. As for hyperlinks you mentioned in #264 that the wikilinks in commonmark might be faster which, as with my idea, I agree that it should be tested.

Also in case my tone comes off dry, I want to reassure you that I have quite enjoyed poking around at your code base. I am enjoying using emanote and hope to contribute something meaningful to the project too!

[srid]

Thanks for reminding me about #264 - actually it makes sense to switch to Pandoc first. I'll accept a PR on that.