Repository navigation
Make inline parsing extensible #113
Description
Similar to how custom block parser can be registered today, it would be good to make inline parsing extensible as well. One use would be to improve autolinking, see #112.
The way this could work:
- When we match a character during inline parsing, the last fallback is to treat it as text. Before we do that, if there are custom inline parsers, we could call each one (with some priority maybe?) and give them the opportunity to handle that.
- In a custom inline parser, it could parse the input and return/add Nodes. It could also advance the current parsing position. So inline parsing needs to expose some API (or the custom inline parser would return instructions).
To support autolinking, a custom inline parser would also need to be able to get the text before the matching character, probably all the plain text characters up to that point. It would also need a way to detect if any brackets are active or not ("are we inside a link?").
Reacted by Samuel P
Activity
robinst commentedon Nov 10, 2017
I'm pretty sure I'm not gonna pick this up in the near future. But if someone else wants to have a go at it, I'm willing to mentor. Help wanted :).
HiuKwok commentedon Apr 20, 2018
Hi @robinst, I am willing to work on it. However as a newbie of both codebase and Commonmark spec. Just wanna clarify several points first to make sure I m on the same page as you guys.
inlineParserImplone central class into multiplesinlineParserImplfor all inline components that listed on spec. So each of the components impl can have chance to parser the content in a whole line lv instead of how it handle at the moment whereInlinePraserImpl.parseInline()?BlockParser.parserInlines()would have a new signature which take a list of InlineParser as a input?HiuKwok commentedon Apr 25, 2018
Hi everyone, several commit have been made for let Both parser and doc parser class to accept multiple instances of InlineParser.
01f4149
9b28f86
e145fc5
My next thought would be alter InlineParser interface to make it return a parsing progress object which is similar to
BlockContinueclass, so when block parser attempt to parser it content via lists of InlineParser. Every inlineParser would be able to pick where last InlineParser left up to.Am I on the right track? Plz advise
robinst commentedon Apr 26, 2018
Yes, that's the rough idea (similar to block parsers).
Ideally yes.
Not sure about this bit, I think it would still be useful to have a single class for parsing all the inlines. Not sure what to call that though, if we use
InlineParserfor the thing that knows how to parse a single inline element syntax.. MaybeInlineContentParser?HiuKwok commentedon Apr 26, 2018
@robinst First of all, thx for the reply.
So more like a new parser
InlineContentParsercreated which only know how to parse a specific inline object type?Let say having a
CodeSpanInlineContentParserwhich implement fromInlineContentParserand only responsible to parse Code Span item and nothing more than that. Then return aInlineContinueObject which would let the nextInlineContentParserknow where to resume the parsing.While all of above is happening under
InlineParser.parse( )so in one wayInlineParsermethod signature stay unchanged. And the parsing process which be able to split into multiple class which implementsInlineContentParser.HiuKwok commentedon Apr 27, 2018
Hi everyone,
I drafted a sample of how a
InlineContentParsercan be implements (32478ca).Basically what have done are first created a new
InlineContentParsewhich have aparse()method. Which then implemented byAbstractInlineContentParse.Then from this point onward a specific
inlineContentParserclass can be created to parse specific type of inlineItem. For this example, created a class callNewLineBreakParserto handle soft & hard LineBreak parsing which originally handled byInlineParserImplinternally.noties commentedon Dec 11, 2019
Hello everyone who might be interested in this!
I've been able to achieve the customization of inline parsing in my library recently. I won't say it is perfect but it is fairly simple and implemented in a way so it's relatively easy to keep it upstream. The implementation keeps all current logic and the way inline parsing is done in general but introduces
InlineProcessorabstraction. EachInlineProcessorhandles uniquecharand is a direct copy of what can be found in specific methods ofInlineParserImpl. You can see the code hereAt this point I don't think
commonmark-javashould jump at having one inline parsing implementation, extensible or not. But having an alternative is always a nice thing. UnfortunatelyInlineParserImplis using some classes that are in theinternalpackage (although they are public and can be accessed) which makes it questionable for replication. They are:There is also the
StaggeredDelimiterProcessorthat is not available outside of its package which forces possible replicators ofInlineParserImplto copy the source code.So, I think that it would be a good thing to move mentioned above classes to public surface. At least it would make possible custom implementations of
InlineParserAny feedback on my solution is welcome and if someone finds it helpful I can convert it to a Java module (currently it is published as an Android library)
robinst commentedon Sep 18, 2024
Support for custom inline content parsers has been released in 0.23.0 now, see
customInlineContentParserFactoryinParser.Builder. Full changelog: https://github.com/commonmark/commonmark-java/blob/main/CHANGELOG.md#0230---2024-09-16