Repository navigation
[3.0] Fixes BBC and Markdown parsing bugs and adds unit tests for parsing - #9817
Conversation
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Previously we used line feed characters, which made things awkward when a line feed was wanted for its normal purpose. Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
This fixes an issue parsing the unparsed_equals version of the spoiler tag. Signed-off-by: Jon Stovell <jonstovell@gmail.com>
687af63 to
94f94f4
Compare
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
94f94f4 to
6ef7db6
Compare
|
I have been building tests for the parser, too. Mine are more focused and don't cover every single BBCode. I also noticed that there are several repeatable patterns that make that test file cumbersome to read. I'll need to condense those before adding my tests. I also have some performance tweaks for the BBCode parser, but those will be put on the back burner to give way to the tests. |
|
By all means add more tests. However, all of the existing ones, even when they seem repetitive, should stay as they are. I wrote a script to programmatically generate tests using every possible combination of Here's the script I used to generate these tests. I just commented and uncommented the relevant strings to create the tests for <?php
namespace SMF;
use SMF\BBCode\BBCode;
use SMF\Parsers\BBCodeParser;
use SMF\Parsers\MarkdownParser;
use SMF\Parsers\SmileyParser;
require_once 'tests/bootstrap.php';
// Ensure everything is enabled.
Config::$modSettings['enableBBC'] = 1;
Config::$modSettings['enablePostHTML'] = 1;
Config::$modSettings['enableMarkdown'] = 1;
Config::$modSettings['smiley_enable'] = 1;
// Smiley settings.
Config::$modSettings['smileys_url'] = Config::$boardurl . '/Smileys';
Config::$modSettings['smiley_sets_default'] = 'fugue';
// Populate the smiley data manually in order to avoid calling the database.
$smiley_parser = new \ReflectionClass(SmileyParser::class);
$smiley_parser_data = $smiley_parser->getProperty('data');
$smiley_parser_data->setAccessible(true);
$smiley_parser_data->setValue(null, getSmileyData());
// Populate the TLD regex with a default value.
Config::$modSettings['tld_regex'] ??= Utils::buildRegex(array_merge(Url::$basic_tlds, Url::$cc_tlds, Url::$special_use_tlds));
// Need a user.
User::load(0, dataset: UserDataset::None);
User::setMe(0);
// Make the tests.
$tests = [];
$test_num = 0;
$strings = [
// Smileys.
'Lorem ipsum :)',
// BBCode.
// No tests for the acronym BBCode because it just an alias of the abbr BBCode.
// No tests for the attach BBCode because it requires a database connection.
// No tests for the ftp BBCode because it just an alias of the url BBCode.
// No tests for the html BBCode because the test user isn't an admin.
'Lorem [abbr=ipsum]dolor[/abbr] sit amet.',
'Lorem [anchor=ipsum]ipsum[/anchor] dolor sit amet.',
'Lorem [b]ipsum dolor[/b] sit amet.',
'[bdo=ltr]Lorem ipsum dolor sit amet.[/bdo]',
'Lorem [black]ipsum dolor[/black] sit amet.',
'Lorem [blue]ipsum dolor[/blue] sit amet.',
'Lorem [br]ipsum dolor sit amet.',
'[center]Lorem ipsum dolor sit amet.[/center]',
'[code]Lorem ipsum dolor sit amet.[/code]',
'[code=foobar]Lorem ipsum dolor sit amet.[/code]',
'Lorem [color=red]ipsum dolor[/color] sit amet.',
'[details]Lorem ipsum dolor sit amet.[/details]',
'[details summary=unquoted summary]Lorem ipsum dolor sit amet.[/details]',
'[details summary="quoted summary"]Lorem ipsum dolor sit amet.[/details]',
'Lorem [email]ipsum@email.invalid[/email] dolor sit amet.',
'Lorem [email=ipsum@email.invalid]dolor[/email] sit amet.',
'Lorem [flash=1,2]http://example.com/ipsum.swf[/flash] dolor sit amet.',
'[float=left]Lorem ipsum dolor sit amet.[/float]',
'Lorem [font=Arial]ipsum dolor[/font] sit amet.',
'Lorem [glow=#fff,123]ipsum dolor[/glow] sit amet.',
'Lorem [green]ipsum dolor[/green] sit amet.',
'[h1]Lorem ipsum dolor sit amet.[/h1]',
'[h2]Lorem ipsum dolor sit amet.[/h2]',
'[h3]Lorem ipsum dolor sit amet.[/h3]',
'[h4]Lorem ipsum dolor sit amet.[/h4]',
'[h5]Lorem ipsum dolor sit amet.[/h5]',
'[h6]Lorem ipsum dolor sit amet.[/h6]',
'Lorem ipsum [hr]dolor sit amet.',
'Lorem ipsum' . "\n" . '[hr]' . "\n" . 'dolor sit amet.',
'Lorem [i]ipsum dolor[/i] sit amet.',
'Lorem ipsum [img]http://example.com/path/to/file.png[/img] dolor sit amet.',
'Lorem [iurl]http://example.com/path/to/page.html[/iurl] dolor sit amet.',
'Lorem ipsum [iurl=http://example.com/path/to/page.html]dolor[/iurl] sit amet.',
'Lorem ipsum [iurl="http://example.com/path/to/page.html"]dolor[/iurl] sit amet.',
'[justify]Lorem ipsum dolor sit amet.[/justify]',
'[right]Lorem ipsum dolor sit amet.[/right]',
'[list]' . "\n" . '[li]Lorem ipsum dolor sit amet.[/li]' . "\n" . '[li]Lorem ipsum dolor sit amet.' . "\n\n" . 'Lorem ipsum dolor sit amet.[/li]' . "\n" . '[/list]',
'[list type=disc]' . "\n" . '[li]Lorem ipsum dolor sit amet.[/li]' . "\n" . '[li]Lorem ipsum dolor sit amet.' . "\n\n" . 'Lorem ipsum dolor sit amet.[/li]' . "\n" . '[/list]',
'[list type=decimal]' . "\n" . '[li]Lorem ipsum dolor sit amet.[/li]' . "\n" . '[li]Lorem ipsum dolor sit amet.' . "\n\n" . 'Lorem ipsum dolor sit amet.[/li]' . "\n" . '[/list]',
'[ltr]Lorem ipsum dolor sit amet.[/ltr]',
'[me=foobar]Lorem ipsum dolor sit amet.[/me]',
'Lorem [member=123]dolor[/member] sit amet.',
'[move]Lorem ipsum dolor sit amet.[/move]',
'Lorem [nobbc][b]ipsum[/b][/nobbc] dolor sit amet.',
'Lorem [nolink]http://example.com/path/to/page.html[/nolink] sit amet.',
'Lorem [php]ipsum[/php] dolor sit amet.',
'Lorem [pre]ipsum dolor[/pre] sit amet.',
'[quote]Lorem ipsum dolor sit amet.[/quote]',
'[quote="[url]http://example.com[/url]"]Lorem ipsum dolor sit amet.[/quote]',
'[quote="[url]http://example.com[/url]"]Lorem ipsum dolor sit amet.[/quote]',
'[quote="foobar"]Lorem ipsum dolor sit amet.[/quote]',
'[quote=foobar]Lorem ipsum dolor sit amet.[/quote]',
'[quote author=foobar]Lorem ipsum dolor sit amet.[/quote]',
'[quote author=foobar link=msg=123 date=1793143433]Lorem ipsum dolor sit amet.[/quote]',
'Lorem [red]ipsum dolor[/red] sit amet.',
'[right]Lorem ipsum dolor sit amet.[/right]',
'[rtl]Lorem ipsum dolor sit amet.[/rtl]',
'Lorem [s]ipsum dolor[/s] sit amet.',
'Lorem [shadow=#000,bottom]ipsum dolor[/shadow] sit amet.',
'Lorem [size=large]dolor[/size] sit amet.',
'Lorem [size=4]dolor[/size] sit amet.',
'Lorem [spoiler]ipsum dolor[/spoiler] sit amet.',
'[spoiler=foobar]Lorem ipsum dolor sit amet.[/spoiler]',
'Lorem [sub]ipsum dolor[/sub] sit amet.',
'Lorem [sup]ipsum dolor[/sup] sit amet.',
'[table][tr][td]Lorem ipsum dolor sit amet.[/td][td]Lorem ipsum dolor sit amet.[/td][/tr][tr][td]Lorem ipsum dolor sit amet.[/td][td]Lorem ipsum dolor sit amet.[/td][/tr][/table]',
'Lorem [time=' . date_create('@1793143433')->format('c') . ']dolor[/time] sit amet.',
'Lorem ipsum [time]' . date_create('@1793143433')->format('c') . '[/time] dolor sit amet.',
'Lorem [tt]ipsum[/tt] dolor sit amet.',
'Lorem [u]ipsum dolor[/u] sit amet.',
'Lorem ipsum [url]http://example.com/path/to/page.html[/url] dolor sit amet.',
'Lorem [url=http://example.com/path/to/page.html]ipsum[/url] dolor sit amet.',
'Lorem [url="http://example.com/path/to/page.html"]ipsum[/url] dolor sit amet.',
'Lorem [white]ipsum dolor[/white] sit amet.',
'Lorem ipsum [youtube]76Gopgo-4PY[/youtube] dolor sit amet.',
// Markdown tests taken from the GFM spec.
'paragraph' . "\n" . ' code?' . "\n",
'1. one' . "\n\n" . '2. two' . "\n" . '3. three' . "\n",
'1. one' . "\n" . ' - a' . "\n\n" . ' - b' . "\n" . '2. two' . "\n",
' 8. item 1' . "\n" . ' 9. item 2' . "\n" . '10. item 2a' . "\n",
'* a' . "\n" . '* * * * *' . "\n" . '* b' . "\n",
'1. fee' . "\n" . '2. fie' . "\n" . '- foe' . "\n" . '- fum' . "\n",
'[a backtick (`)](/url) and [another backtick (`)](/url).' . "\n",
'*foo *bar* baz*' . "\n",
'- `a long code span can contain a hyphen like this' . "\n" . ' - and it can screw things up`' . "\n",
'- # Heading' . "\n",
'* a' . "\n" . '*' . "\n" . '* b' . "\n",
'> Blockquote [foo].' . "\n" . '>' . "\n" . '> [foo]: /url' . "\n",
'[foo]: /url1' . "\n" . '[foo]: /url2' . "\n\n" . '[foo][]' . "\n",
"\t" . 'foo' . "\t" . 'baz' . "\t\t" . 'bim' . "\n",
'>' . "\t\t" . 'foo' . "\n",
'-' . "\t\t" . 'foo' . "\n",
'#' . "\t" . 'Foo' . "\n",
'*' . "\t" . '*' . "\t" . '*' . "\t\n",
'***' . "\n" . '---' . "\n" . '___' . "\n",
'+++' . "\n",
'===' . "\n",
'--' . "\n" . '**' . "\n" . '__' . "\n",
'_____________________________________' . "\n",
'Foo' . "\n" . '***' . "\n" . 'bar' . "\n",
'Foo' . "\n" . '---' . "\n" . 'bar' . "\n",
'Foo' . "\n" . '-------------------------' . "\n\n" . 'Foo' . "\n" . '=' . "\n",
'Foo\\' . "\n" . '----' . "\n",
'Foo' . "\n" . 'Bar' . "\n" . '---' . "\n",
'---' . "\n" . 'Foo' . "\n" . '---' . "\n" . 'Bar' . "\n" . '---' . "\n" . 'Baz' . "\n",
"\n" . '====' . "\n",
'---' . "\n" . '---' . "\n",
'Foo' . "\n" . 'bar' . "\n" . '---' . "\n" . 'baz' . "\n",
'Foo' . "\n\n" . 'bar' . "\n" . '---' . "\n" . 'baz' . "\n",
'Foo' . "\n" . 'bar' . "\n\n" . '---' . "\n\n" . 'baz' . "\n",
'Foo' . "\n" . 'bar' . "\n" . '\\---' . "\n" . 'baz' . "\n",
'``' . "\n" . 'foo' . "\n" . '``' . "\n",
'```' . "\n" . 'aaa' . "\n" . '~~~' . "\n" . '```' . "\n",
'~~~' . "\n" . 'aaa' . "\n" . '```' . "\n" . '~~~' . "\n",
'````' . "\n" . 'aaa' . "\n" . '```' . "\n" . '``````' . "\n",
'~~~~' . "\n" . 'aaa' . "\n" . '~~~' . "\n" . '~~~~' . "\n",
'```' . "\n",
'`````' . "\n\n" . '```' . "\n" . 'aaa' . "\n",
'```' . "\n" . '```' . "\n",
'foo' . "\n" . '```' . "\n" . 'bar' . "\n" . '```' . "\n" . 'baz' . "\n",
'````;' . "\n" . '````' . "\n",
'<table><tr><td>' . "\n" . '<pre>' . "\n" . '**Hello**,' . "\n\n" . '_world_.' . "\n" . '</pre>' . "\n" . '</td></tr></table>' . "\n",
'</div>' . "\n" . '*foo*' . "\n",
'<div>' . "\n" . '*foo*' . "\n\n" . '*bar*' . "\n",
'<table><tr><td>' . "\n" . 'foo' . "\n" . '</td></tr></table>' . "\n",
'<Warning>' . "\n" . '*bar*' . "\n" . '</Warning>' . "\n",
'</ins>' . "\n" . '*bar*' . "\n",
'<del>' . "\n" . '*foo*' . "\n" . '</del>' . "\n",
'<del>' . "\n\n" . '*foo*' . "\n\n" . '</del>' . "\n",
'<del>*foo*</del>' . "\n",
'<style>p{color:red;}</style>' . "\n" . '*foo*' . "\n",
'Foo' . "\n" . '<div>' . "\n" . 'bar' . "\n" . '</div>' . "\n",
'<div>' . "\n" . 'bar' . "\n" . '</div>' . "\n" . '*foo*' . "\n",
'<table>' . "\n\n" . '<tr>' . "\n\n" . '<td>' . "\n" . 'Hi' . "\n" . '</td>' . "\n\n" . '</tr>' . "\n\n" . '</table>' . "\n",
'[foo]:' . "\n" . '/url' . "\n\n" . '[foo]' . "\n",
'[foo]:' . "\n\n" . '[foo]' . "\n",
'aaa' . "\n\n" . 'bbb' . "\n",
'aaa' . "\n" . 'bbb' . "\n\n" . 'ccc' . "\n" . 'ddd' . "\n",
'aaa' . "\n\n\n" . 'bbb' . "\n",
'> foo' . "\n" . '> ---' . "\n",
'> - foo' . "\n" . '> - bar' . "\n",
'> foo' . "\n" . '> - bar' . "\n",
'>' . "\n",
'-one' . "\n\n" . '2.two' . "\n",
'*' . "\n",
'foo' . "\n" . '*' . "\n\n" . 'foo' . "\n" . '1.' . "\n",
'- foo' . "\n\n" . ' bar' . "\n\n" . ' - baz' . "\n",
' - one' . "\n\n" . ' two' . "\n",
'> - one' . "\n" . '>' . "\n" . '> two' . "\n",
' 10. foo' . "\n\n" . ' bar' . "\n",
'1. foo' . "\n\n" . ' indented code' . "\n",
'The number of windows in my house is' . "\n" . '14. The number of doors is 6.' . "\n",
'I need to buy' . "\n" . '- new shoes' . "\n" . '- a coat' . "\n" . '- a plane ticket' . "\n",
' * I need to buy' . "\n" . ' - new shoes' . "\n" . ' - a coat' . "\n" . ' - a plane ticket' . "\n",
'I need to buy' . "\n" . '- new shoes' . "\n" . '- a coat' . "\n" . '- a plane ticket' . "\n",
'`hi`lo`' . "\n",
'\\!\\"\\#\\$\\%\\&\'\\(\\)\\*\\+\\,\\-\\.\\/\\:\\;\\<\\=\\>\\?\\@\\[\\\\]\\^\\_\\`\\{\\|\\}\\~' . "\n",
'\\*emphasis*' . "\n",
'foo\\' . "\n" . 'bar' . "\n",
'~~~' . "\n" . '\\[\\]' . "\n" . '~~~' . "\n",
'<http://example.com?find=\\*>' . "\n",
'&copy' . "\n",
'&MadeUpEntity;' . "\n",
'`f&ouml;&ouml;`' . "\n",
'&#42;foo&#42;' . "\n" . '*foo*' . "\n",
'foo&#10;&#10;bar' . "\n",
'&#9;foo' . "\n",
'`foo`' . "\n",
'` b `' . "\n",
'`foo\\`bar`' . "\n",
'``foo`bar``' . "\n",
'*foo`*`' . "\n",
'`<http://foo.bar.`baz>`' . "\n",
'<http://foo.bar.`baz>`' . "\n",
'```foo``' . "\n",
'`foo' . "\n",
'`foo``bar``' . "\n",
'***strong emph***' . "\n" . '***strong** in emph*' . "\n" . '***emph* in strong**' . "\n" . '**in strong *emph***' . "\n" . '*in emph **strong***' . "\n",
'*emph *with emph* in it*' . "\n" . '**strong **with strong** in it**' . "\n",
'internal emphasis: foo*bar*baz' . "\n" . 'no emphasis: foo_bar_baz' . "\n",
'a*"foo"*' . "\n",
'* a *' . "\n",
'foo*bar*' . "\n",
'5*6*78' . "\n",
'a_"foo"_' . "\n",
'foo_bar_' . "\n",
'5_6_78' . "\n",
'пристаням_стремятся_' . "\n",
'aa_"bb"_cc' . "\n",
'foo-_(bar)_' . "\n",
'_foo*' . "\n",
'*(*foo)' . "\n",
'*(*foo*)*' . "\n",
'*foo*bar' . "\n",
'_(_foo)' . "\n",
'_(_foo_)_' . "\n",
'_foo_bar' . "\n",
'_пристаням_стремятся' . "\n",
'_foo_bar_baz_' . "\n",
'_(bar)_.' . "\n",
'a**"foo"**' . "\n",
'foo**bar**' . "\n",
'a__"foo"__' . "\n",
'foo__bar__' . "\n",
'5__6__78' . "\n",
'пристаням__стремятся__' . "\n",
'foo-__(bar)__' . "\n",
'**(**foo)' . "\n",
'*(**foo**)*' . "\n",
'**foo**bar' . "\n",
'__(__foo)' . "\n",
'_(__foo__)_' . "\n",
'__foo__bar' . "\n",
'__пристаням__стремятся' . "\n",
'__foo__bar__baz__' . "\n",
'__(bar)__.' . "\n",
'*foo' . "\n" . 'bar*' . "\n",
'*foo**bar**baz*' . "\n",
'<p><em>foo</em><em>bar<em></em>baz</em></p>' . "\n",
'*foo**bar*' . "\n",
'*foo**bar***' . "\n",
'foo***bar***baz' . "\n",
'foo******bar*********baz' . "\n",
'**foo' . "\n" . 'bar**' . "\n",
'**foo*bar*baz**' . "\n",
'**foo*' . "\n",
'*foo**' . "\n",
'***foo**' . "\n",
'****foo*' . "\n",
'**foo***' . "\n",
'*foo****' . "\n",
'__foo_' . "\n",
'_foo__' . "\n",
'___foo__' . "\n",
'____foo_' . "\n",
'__foo___' . "\n",
'_foo____' . "\n",
'**foo**' . "\n",
'*_foo_*' . "\n",
'__foo__' . "\n",
'_*foo*_' . "\n",
'****foo****' . "\n",
'____foo____' . "\n",
'******foo******' . "\n",
'***foo***' . "\n",
'_____foo_____' . "\n",
'*[bar*](/url)' . "\n",
'**a<http://foo.bar/?q=**>' . "\n",
'__a<http://foo.bar/?q=__>' . "\n",
'[link](/uri)' . "\n",
'[link]()' . "\n",
'[link](<>)' . "\n",
'[link](foo' . "\n" . 'bar)' . "\n",
'[link](<foo' . "\n" . 'bar>)' . "\n",
'[a](<b)c>)' . "\n",
'[link](<foo\\>)' . "\n",
'[a](<b)c' . "\n" . '[a](<b)c>' . "\n" . '[a](<b>c)' . "\n",
'[link](\\(foo\\))' . "\n",
'[link](foo(and(bar)))' . "\n",
'[link](foo\\(and\\(bar\\))' . "\n",
'[link](<foo(and(bar)>)' . "\n",
'[link](foo\\)\\:)' . "\n",
'[link](#fragment)' . "\n\n" . '[link](http://example.com#fragment)' . "\n\n" . '[link](http://example.com?foo=3#frag)' . "\n",
'[link](foo\\bar)' . "\n",
'[link](foo%20b&auml;)' . "\n",
'[link]("title")' . "\n",
'[link](/url "title")' . "\n",
'[](/uri)' . "\n",
'](uri2)](uri3)' . "\n",
'*[foo*](/uri)' . "\n",
'[foo`](/uri)`' . "\n",
'[foo<http://example.com/?search=](uri)>' . "\n",
'[foo]' . "\n" . '[bar]' . "\n\n" . '[foo]: /url1' . "\n" . '[bar]: /url2' . "\n",
'' . "\n",
'' . "\n",
'' . "\n",
'<http://foo.bar.baz>' . "\n",
'<http://foo.bar.baz/test?q=hello&id=22&boolean>' . "\n",
'<irc://foo.bar:2233/baz>' . "\n",
'<MAILTO:FOO@BAR.BAZ>' . "\n",
'<a+b+c:d>' . "\n",
'<made-up-scheme://foo,bar>' . "\n",
'<http://../>' . "\n",
'<localhost:5001/foo>' . "\n",
'<http://example.com/\\[\\>' . "\n",
'<foo@bar.example.com>' . "\n",
'<foo+special@Bar.baz-bar0.com>' . "\n",
'<foo\\+@bar.example.com>' . "\n",
'<>' . "\n",
'<m:abc>' . "\n",
'<foo.bar.baz>' . "\n",
'http://example.com' . "\n",
'foo@bar.example.com' . "\n",
'www.commonmark.org' . "\n",
'www.google.com/search?q=Markup+(business)' . "\n\n" . 'www.google.com/search?q=Markup+(business)))' . "\n\n" . '(www.google.com/search?q=Markup+(business))' . "\n\n" . '(www.google.com/search?q=Markup+(business)' . "\n",
'www.google.com/search?q=(business))+ok' . "\n",
'www.google.com/search?q=commonmark&hl=en' . "\n\n" . 'www.google.com/search?q=commonmark&hl;' . "\n",
'www.commonmark.org/he<lp' . "\n",
'foo@bar.baz' . "\n",
'a.b-c_d@a.b' . "\n\n" . 'a.b-c_d@a.b.' . "\n\n" . 'a.b-c_d@a.b-' . "\n\n" . 'a.b-c_d@a.b_' . "\n",
'mailto:foo@bar.baz' . "\n\n" . 'mailto:a.b-c_d@a.b' . "\n\n" . 'mailto:a.b-c_d@a.b.' . "\n\n" . 'mailto:a.b-c_d@a.b/' . "\n\n" . 'mailto:a.b-c_d@a.b-' . "\n\n" . 'mailto:a.b-c_d@a.b_' . "\n\n" . 'xmpp:foo@bar.baz' . "\n\n" . 'xmpp:foo@bar.baz.' . "\n",
'xmpp:foo@bar.baz/txt' . "\n\n" . 'xmpp:foo@bar.baz/txt@bin' . "\n\n" . 'xmpp:foo@bar.baz/txt@bin.com' . "\n",
'xmpp:foo@bar.baz/txt/bin' . "\n",
'<a><bab><c2c>' . "\n",
'<a/><b2/>' . "\n",
'foo\\' . "\n" . 'baz' . "\n",
'*foo\\' . "\n" . 'bar*' . "\n",
'`code\\' . "\n" . 'span`' . "\n",
'foo\\' . "\n",
'foo' . "\n" . 'baz' . "\n",
'> Lorem ipsum dolor' . "\n" . 'sit amet.' . "\n" . '> - Qui *quodsi iracundia*' . "\n" . '> - aliquando id' . "\n",
'> Lorem ipsum dolor' . "\n",
'sit amet.' . "\n",
'> - Qui *quodsi iracundia*' . "\n",
'> - aliquando id' . "\n",
// Protect content of inline code spans.
'Here is some [tt]inline **code**[/tt].',
'Here is some `inline [b]code[/b]`.',
'Here is some `inline [b]code`.',
'Here is some [tt]inline [b]code[/b][/tt].',
'Here is some [tt]inline [b]code[/tt].',
];
foreach ($strings as $string) {
$first_test_num_for_string = $test_num;
for (
$input_types = (Parser::INPUT_BBC | Parser::INPUT_MARKDOWN | Parser::INPUT_SMILEYS);
$input_types >= Parser::INPUT_BBC;
$input_types--
) {
if (
($input_types & Parser::INPUT_SMILEYS)
&& $input_types !== (Parser::INPUT_BBC | Parser::INPUT_MARKDOWN | Parser::INPUT_SMILEYS)
&& !str_contains($string, ':)')
) {
continue;
}
foreach ([Parser::OUTPUT_HTML, Parser::OUTPUT_BBC] as $output_type) {
for (
$hard_breaks = 0;
$hard_breaks <= ($input_types & Parser::INPUT_MARKDOWN) ? 3 : 0;
$hard_breaks++
) {
$options['hard_breaks'] = $hard_breaks;
foreach ([false, true] as $for_print) {
$options['for_print'] = $for_print;
$output = Parser::transform(
$string,
input_types: $input_types,
output_type: $output_type,
options: $options,
);
if ($string === $output && $output_type === Parser::OUTPUT_BBC) {
continue;
}
for ($i = $first_test_num_for_string; $i < $test_num; $i++) {
if (isset($tests[$i]) && $output === $tests[$i][4]) {
continue 2;
}
}
$tests[$test_num++] = [
$string,
$input_types,
$output_type,
$options,
$output,
];
}
}
}
}
}
$tests = str_replace(
[
'1 => 1',
'1 => 2',
'1 => 3',
'1 => 4',
'1 => 5',
'1 => 6',
'1 => 7',
'2 => 0',
'2 => 1',
'2 => 2',
'2 => 3',
'\'hard_breaks\' => 1',
'\'hard_breaks\' => 2',
'\'hard_breaks\' => 3',
Utils::TAB_SUBSTITUTE . Utils::TAB_SUBSTITUTE . Utils::TAB_SUBSTITUTE,
Utils::TAB_SUBSTITUTE . Utils::TAB_SUBSTITUTE,
Utils::TAB_SUBSTITUTE,
'\'\' . Utils::TAB_SUBSTITUTE',
'Utils::TAB_SUBSTITUTE . \'\'',
],
[
'Parser::INPUT_BBC',
'Parser::INPUT_MARKDOWN',
'Parser::INPUT_BBC | Parser::INPUT_MARKDOWN',
'Parser::INPUT_SMILEYS',
'Parser::INPUT_BBC | Parser::INPUT_SMILEYS',
'Parser::INPUT_MARKDOWN | Parser::INPUT_SMILEYS',
'Parser::INPUT_BBC | Parser::INPUT_MARKDOWN | Parser::INPUT_SMILEYS',
'Parser::OUTPUT_HTML',
'Parser::OUTPUT_TEXT',
'Parser::OUTPUT_BBC',
'MarkdownParser::OUTPUT_HTML_STRICT',
'\'hard_breaks\' => MarkdownParser::BR_LINES',
'\'hard_breaks\' => MarkdownParser::BR_IN_PARAGRAPHS',
'\'hard_breaks\' => MarkdownParser::BR_LINES | MarkdownParser::BR_IN_PARAGRAPHS',
'\' . Utils::TAB_SUBSTITUTE . Utils::TAB_SUBSTITUTE . Utils::TAB_SUBSTITUTE . \'',
'\' . Utils::TAB_SUBSTITUTE . Utils::TAB_SUBSTITUTE . \'',
'\' . Utils::TAB_SUBSTITUTE . \'',
'Utils::TAB_SUBSTITUTE',
'Utils::TAB_SUBSTITUTE',
],
'return ' . Config::varExport($tests) . ';' . "\n",
);
$tests = preg_replace(
[
'/\t\t\K\d+ => /',
'/' . preg_quote(Config::$modSettings['smileys_url'] . '/' . Config::$modSettings['smiley_sets_default'], '/') . '/',
'/' . preg_quote(Config::$modSettings['smileys_url'], '/') . '/',
'/' . preg_quote(Config::$scripturl, '/') . '/',
'/' . preg_quote(Config::$boardurl, '/') . '/',
],
[
'',
"'" . ' . Config::$modSettings[\'smileys_url\'] . \'/\' . Config::$modSettings[\'smiley_sets_default\'] . ' . "'",
"'" . ' . Config::$modSettings[\'smileys_url\'] . \'' . "'",
"'" . ' . Config::$scripturl . ' . "'",
"'" . ' . Config::$boardurl . ' . "'",
],
$tests,
);
echo $tests;
/**
* Gets default data to populate SmileyParser::$data with.
*
* @return array Data to populate SmileyParser::$data with.
*/
function getSmileyData(): array
{
foreach (
(new \ReflectionClass(\SMF\Db\Schema\v3_0\Smileys::class))->getDefaultProperties()['initial_data'] as $row
) {
$data[''][(int) $row['id_smiley']] = [
'code' => $row['code'],
'filename' => '',
'description' => Lang::getTxt(substr($row['description'], 2, -1), file: 'Maintenance'),
];
}
foreach (
(new \ReflectionClass(\SMF\Db\Schema\v3_0\SmileyFiles::class))->getDefaultProperties()['initial_data'] as $row
) {
if ($row['smiley_set'] === Config::$modSettings['smiley_sets_default']) {
$data[''][(int) $row['id_smiley']]['filename'] = $row['filename'];
}
}
$data[Config::$modSettings['smiley_sets_default']] = $data[''];
return $data;
} |
This adds over 1200 unit tests for parsing BBCode and Markdown. In the process of creating the unit tests, I also found and fixed a bunch of bugs.
Also Gets rid of the SMF\Time::$user_tz static property in order to avoid giving SMF\tests\Unit\TimeTest a bellyache.