Skip to content

Precompute low-codepoint Unicode property bitmaps - #1055

Open
subotac wants to merge 2 commits into
PCRE2Project:mainfrom
subotac:perf/precompute-unicode-property-bitmaps
Open

subotac wants to merge 2 commits into
PCRE2Project:mainfrom
subotac:perf/precompute-unicode-property-bitmaps

Conversation

@subotac

@subotac subotac commented Oct 25, 2026

Copy link
Copy Markdown

Fixes #930.

Generate and deduplicate property bitmaps for code points below 256 with the Unicode tables, then reuse them when compiling character classes. Preserve the PT_ANY fast path and existing matching semantics.

For five local patterns across 8/16/32-bit builds, paired three-run medians improved compilation by 2.48-4.62x, adding 3136 bytes of read-only tables per width. All-width interpreter/JIT tests, 1608 property-bitmap equivalence cases per width, generator regeneration, GCC ASan/UBSan, and the Unicode/JIT-disabled build passed. Clang/Valgrind configurations were unavailable locally.

Comment thread src/pcre2_ucd.c Outdated
// clang-format on

#if PT_LAMP != 0
#error Please update low_properties in maint/GenerateUcd.py

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it worth to detect which one is failed when you need to run a script anyway? Just use &&

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Combined the property-number checks into one && guard and regenerated the table in 26eb3e4. @zherczeg, could you take another look and approve the CI run?

Comment thread src/pcre2_compile_class.c
case PT_GC:
set_bit = (PRIV(ucp_gentype)[prop->chartype] == pdata);
break;
PCRE2_ASSERT(ptype <= PT_PXXDIGIT && ptype != PT_CLIST);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There should be an assert here that ucd_low_bitmap_indexes[ptype] + pdata <= ucd_low_bitmap_indexes[ptype + 1], except for the last ptype.

Comment thread src/pcre2_ucd.c
{ 0x00, 0x00, 0x00, 0x00, 0xfe, 0xff, 0x00, 0xfc, 0x01, 0x00, 0x00, 0xf8, 0x01, 0x00, 0x00, 0x78, 0x00, 0x00, 0x00, 0x00, 0x82, 0x08, 0xc0, 0x88, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 },
};

const uint16_t PRIV(ucd_low_bitmap_indexes)[] = {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These could be two sets, first uses 16 bit values, the second uses 8 bit values.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

performance regression for Unicode properties

2 participants