Repository navigation
Conversation
seperman
reviewed
Oct 15, 2026
seperman
left a comment
Member
There was a problem hiding this comment.
Static review: #632 Keep regex escapes intact in case-insensitive DeepSearch
Summary: This changes use_regexp=True searches with a string pattern when case_sensitive=False, which is the default. The pattern and the searched text are no longer lowercased. The pattern is compiled with re.IGNORECASE instead. This fixes the reported problem: uppercase escapes were inverted (\S→\s, \D→\d, \W→\w) and \N{...}/\U... escapes were corrupted.
Overlap with other PRs
- The production change is functionally identical to #631, "Preserve regex syntax in case-insensitive DeepSearch". #631 was closed unmerged by its author on 2026-09-23, with no comments.
- The only differences are how the
flagsexpression is formatted and the tests. - #631's tests also covered
\Band ran the syntax cases withcase_sensitive=Trueas well.
- The only differences are how the
- #638 (open) adds an exclusion check just before the
new_parent_casedline this PR changes (deepdiff/search.py:240).git merge-treereports a textual conflict between the two. Resolving it means keeping both changes. - Nothing equivalent is on
dev.
Security
No security concerns found in static review.
- The user-supplied pattern is still compiled once, as before. Only the way case-insensitivity is applied has changed:
re.IGNORECASEinstead of lowercasing. - This adds no regex-injection path and does not meaningfully change backtracking.
- Compiling a caller-supplied pattern, with whatever ReDoS exposure that brings, is existing behavior of
use_regexp=True.
Correctness
deepdiff/search.py:109,130-131: string patterns are no longer lowercased. Whenself.case_sensitiveis false they are compiled withre.IGNORECASE.- For a precompiled pattern, line 108 forces
self.case_sensitivetoTrue, soflagsis0andre.compile(pattern, flags=0)returns the pattern unchanged. This avoids theValueErrorthatre.compileraises when non-zero flags are passed with a compiled pattern. User-supplied flags are kept. - Numbers converted to strings under
strict_checking=Falsealso getflags=0, as before.
- For a precompiled pattern, line 108 forces
deepdiff/search.py:240and:286-289: whenuse_regexpis set, the dictionary path and the string value are no longer lowercased. This is consistent with compiling withIGNORECASE. Literal (non-regex) searches keep their.lower()handling.- Other patterns this also fixes, which the description doesn't list and the tests don't cover. These follow from the old
item.lower():\Abecame\a(BEL).\Bbecame\b.\Zbecame\z, which is a bad escape before Python 3.14.- Named groups:
(?P<name>...)and(?P=name)became(?p<..., whichrerejects. So ondev, a default (case-insensitive) regex search with a named group raisesre.error.
- Side effect in
__search_numbers(deepdiff/search.py:304, used only withstrict_checking=False):str(obj)was never lowercased there. Ondev, an uppercase pattern such as'E'therefore failed to matchDecimal('1E+5'). WithIGNORECASEit is now truly case-insensitive. This looks like an improvement, but it is a change in behavior. - Behavior change worth a changelog note: scoped flags such as
(?-i:ABC)are now honored. Ondevthey were lowercased together with the text, so they matched case-insensitively.
Tests
test_case_insensitive_regex_preserves_syntax(tests/test_search.py:382) andtest_case_insensitive_regex_preserves_dictionary_paths(:397) cover the bug. Followingdev's code, these cases should fail ondevand pass with this change:\S+,\W+,\D+,\N{...},\U0001F680and(?-i:...)- the
\D+and(?-i:Name)path cases
:387,:403and:412are guard tests for case sensitivity, compiled-pattern flags and literal search. They should pass both before and after the change, which is fine.
Suggestions
- Consider adding cases for named groups (
(?P<name>...)), which raise ondev, and for the anchors\A,\Zand\B. - A short changelog note that
(?-i:...)is now respected in case-insensitive searches may help users. - If #638 lands first, this will need a small rebase around
deepdiff/search.py:238-241.
AI-assisted static review; the code was not executed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
DeepSearch lowercases a regular expression before using it for a case-insensitive search. That changes the meaning of escapes: searching for
\S+(non-whitespace) instead searches for\s+(whitespace), and\D+becomes a digit search.Use
re.IGNORECASEinstead, keeping both the pattern and the searched text intact. This also preserves Unicode escapes and lets groups such as(?-i:ABC)keep their explicitly case-sensitive behavior. Literal searches and the flags of precompiled patterns retain their existing behavior. This addresses the problem also described in the closed PR #631.The search tests pass all 78 cases, including regressions for escaped characters, dictionary paths, scoped flags, and compiled patterns. Pyright reports no errors for the changed module. On Windows with Python 3.12, the broader run passes 1,319 tests with 8 skips and one failure in
test_delta_dump_and_read3; that same text-mode pickle failure occurs on unchangeddev. The security test module and two restricted-pickle tests were excluded from this local run. Other operating systems and Python versions have not been tested locally.