Skip to content

Respect exclusions before reporting DeepSearch path matches - #638

Open
FanWu-ai wants to merge 1 commit into
qlustered:devfrom
FanWu-ai:fix/deepsearch-excluded-path-reporting
Open

FanWu-ai wants to merge 1 commit into
qlustered:devfrom
FanWu-ai:fix/deepsearch-excluded-path-reporting

Conversation

@FanWu-ai

@FanWu-ai FanWu-ai commented Oct 3, 2026

Copy link
Copy Markdown

DeepSearch reports matching dictionary keys and attribute names before the recursive search checks exclusions. For example, DeepSearch({'needle': 'x'}, 'needle', exclude_paths=["root['needle']"]) currently reports the excluded path in matched_paths.

Check the child object and its path with the existing exclusion predicate before reporting a path match. Add ten regression cases covering exact, wildcard and regex path exclusions, attribute names, excluded child types, retained siblings, and both verbosity levels.

This complements #629: that change fixes the recursive current-object type check, while this fixes the earlier path-reporting step. All eight initial path-exclusion regressions still fail with #629's production change alone.

Validation on Linux, CPython 3.12.14:

  • Full suite including slow tests: pytest --cov=deepdiff --runslow --benchmark-disable -q: 1,319 passed, 8 skipped; 95% aggregate coverage.
  • Search suite: 72 passed; 100% coverage of deepdiff/search.py.
  • pyright deepdiff/search.py: 0 errors, 0 warnings.
  • CI-blocking Flake8 rules E9,F63,F7,F82 and git diff --check: passed.
  • Full Flake8 on the touched files reports seven pre-existing diagnostics, compared against the unchanged baseline; no new diagnostics.

Other Python versions, operating systems, and hosted CI have not been validated locally.

@FanWu-ai
FanWu-ai marked this pull request as ready for review October 3, 2026 09:14

@seperman seperman left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Static review: #638 Respect exclusions before reporting DeepSearch path matches

Summary: Adds self.__skip_this(obj_child, new_parent) to DeepSearch.__search_dict, before a dictionary key or attribute name is compared with the search term. Excluded paths (exact, glob and regex) and children of excluded types no longer show up in matched_paths. This fixes the reported problem.

Overlap with other PRs

  • #629 (open) changes the type check at the top of __search from __skip_this(item, ...) to __skip_this(obj, ...). As the description says, the two PRs complement each other:
    • This PR alone also skips dictionary values and attributes of excluded types, because the new check is given obj_child. But the root object is never type-checked, and __search (deepdiff/search.py:324) still checks the type of the search item. For example, DeepSearch([1], 1, exclude_types=[list]) still reports root[0], and #629's test_skip_root_object_type cases would still fail.
    • #629 alone still reports excluded keys in matched_paths, so the new tests here would fail with only #629.
    • The two merge without a textual conflict (checked with git merge-tree).
  • #632 (open) changes the line right after the new check (new_parent_cased = ..., deepdiff/search.py:241 here), and git merge-tree reports a textual conflict there. The fix is to keep both: this check, then #632's or self.use_regexp condition.
  • Nothing equivalent is on dev.

Security

No security concerns found in static review.

Correctness

  • deepdiff/search.py:239-240: on dev, __search_dict reports a path match (lines 244-250 here) before any exclusion check, because exclusions are only checked inside __search. So DeepSearch({'needle': 'x'}, 'needle', exclude_paths=["root['needle']"]) returns matched_paths: {"root['needle']"}. Running the check before the report fixes this. The fix also covers attributes and namedtuples, since __search_obj goes through __search_dict(print_as_attribute=True).
  • Because the check is given obj_child, a key whose value has an excluded type is no longer reported in matched_paths either.
  • Minor: each dictionary child now goes through __skip_this twice, once here and once at the top of __search (line 324). Glob parsing (_path_to_elements) is lru_cached, and __search_iterable already checks twice in the same way, so the extra cost should be small.
  • Optional: parents_ids_added (line 236) could be computed after the new check, which avoids a frozenset union for every skipped child.

Tests

  • The new tests at tests/test_search.py:263, :272 and :281 cover:

    • exact, glob (root[*]['needle']) and regex path exclusions
    • attribute names
    • an excluded value type

    Each runs at verbosity levels 1 and 2. Following dev's code, each case should produce an extra matched_paths entry there, so the tests fail on dev and pass with this change.

Suggestions

  • Land this together with, or after, #629 so the root-object case and the item-type check in __search are fixed as well.
  • If both this and #632 are merged, the conflict on the neighbouring line needs resolving (see Overlap above).

AI-assisted static review; the code was not executed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants