Escape text that should not be interpreted as Wiki syntax

[th-we]

This is not intended as a ready-to-merge pull request – it still needs some work and I'd like some feedback.

When I type __foo//bar%% into the WYSIWYG editor, then a JSON structure as in the new test file unformatted.json is sent. This needs to be escaped. I modified getInnerSyntax() to do just that. Ist das im Sinne des Erfinders?

This works, but it can not be tested in the same way as the other pairs of txt/json files because roundtripping will result in a different JSON structure that splits up the string into five sections, three of them marked as "unformatted".

I see several options how to make this work:

• Write a separate test in PHP for this feature
• Introduce files with "double extensions" .renderer.json and .jsonParser.json that are only intended for renderer.test.php or jsonParser.test.php
• A quite radical approach: Get rid of the "unformatted" mark altogether because unformatted text will render correctly when sent like in unformatted.json with the getInnerSyntax() function from this pull request.

[splitbrain]

I understand the sentiment, but the implementation will not solve the underlying problem.

In a WYSIWYG editor you would expect to be able to type and have everything treated as "normal" text unless you apply some special formatting. To do that, everything that has "special" in wiki syntax needs to be escaped when typed in the WYSIWYG editor.

Your approach is too naive to solve this. You hardcode what is "special" but in reality it is not as simple because every plugin can introduce its own syntax. The regex would need to be built from the syntax definitions of all plugins (and core syntax).

I don't think it would be impossible to do so, but it is much more complex than this. Basically we'd need a PHP component that builds a compound regex for everything that is "special" eg, the open/close/special expressions plugins register in their connectTo methods (and maybe the postConnect method, too). Basically we need to do what the Lexer class in our Parser does (it probably could be reused for that).

[th-we]

Here is an example why I think this needs to be addressed. Let's say a user inputs this:

What is saved is this:

Check the following files:

  * __foo.txt
  * __bar.txt

And as expected, that's parsed back to the following by the Prosemirror plugin:

But it gets a little worse because the next time this is edited with the WYSIWYG plugin, this is what's saved:

Check the following files:

  * __foo.txt   * __bar.txt

My point is: Native syntax (that the WYSIWYG plugin recognizes and parses into the proper markup) should be understood and escaped properly. Anything else should be left alone. For example, I'm not saying that something like {{tag>foo}} should be parsed or escaped in any way (though it would be great to allow plugins to make their syntax known to the editor).

Do I understand your argument correctly that you don't want to escape core syntax because plugins might use syntax that's ambiguous and in conflict with core syntax – say they use three underscores ___? Then the WYSIWYG editor will misrender and potentially mess it up when saving anyway, as can be seen above.