Skip to content

Skip excluded objects before calling custom operators - #636

Open
EchoJohn wants to merge 1 commit into
qlustered:devfrom
EchoJohn:fix/633-skip-excluded-custom-operators
Open

EchoJohn wants to merge 1 commit into
qlustered:devfrom
EchoJohn:fix/633-skip-excluded-custom-operators

Conversation

@EchoJohn

@EchoJohn EchoJohn commented Oct 2, 2026

Copy link
Copy Markdown

When a path is excluded, DeepDiff still calls custom operators' match() and give_up_diffing() methods before discarding the comparison result. This can run expensive comparisons or raise exceptions for objects that the caller explicitly excluded.

Apply the existing exclusion check before invoking custom operators. Preserve the identity shortcut when there are no custom operators, and continue allowing custom operators to inspect identical objects on included paths.

Closes #633.

Adds 10 regression cases covering shorthand and full paths, regex and callback exclusions, excluded roots and types, both operator return values, and custom reports for identical objects on included paths. All 10 fail on the unchanged dev baseline. The original issue example now compares only root['a'] and still returns an empty diff.

Validation on Windows, Python 3.12.7, with dependencies from uv.lock:

  • python -X utf8 -m pytest tests --benchmark-disable --cov=deepdiff --cov-report=term --tb=short -q: 1,281 passed, 45 skipped, 1 failed; 92% coverage.
  • The remaining failure, TestDeltaClassPollution.test_remote_code_execution, expects posix.system but receives nt.system on Windows. The same failure was reproduced on baseline 59acdd9cbb50fdbc30ef7cde2aa6f290ac0a5192 using the same environment.
  • python -m flake8 deepdiff tests/test_operators.py --count --select=E9,F63,F7,F82 --show-source --statistics: passed.
  • git diff --check: passed.

The full Python/platform CI matrix and optional slow tests have not been run locally.

@seperman seperman left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Static review: #636 Skip excluded objects before calling custom operators

Summary: This reorders the start of DeepDiff._diff so that _skip_this(level) runs before _use_custom_operator(level). Custom operators' match()/give_up_diffing() are no longer called for paths excluded by exclude_paths, exclude_regex_paths, exclude_types, the *_obj_callback options, or paths outside include_paths. As far as I can tell from reading the code, this fixes the scenario in #633.

Overlap with other PRs

  • I found no other PR, open or merged, for #633. Nothing merged on dev addresses it.
  • The patch merges cleanly (git merge-tree) with #634, #620 and #630. It only touches the top of _diff (diff.py:2139-2149), which none of those PRs modify.

Security

No security concerns found in static review.

Correctness

  • New order in deepdiff/diff.py:2139-2149:

    1. identity shortcut, only when there are no custom operators
    2. _skip_this
    3. custom operators
    4. identity shortcut

    Without custom operators the behaviour is the same as before: both early returns still happen, and the cheap identity check now comes first.

  • With custom operators, the identity check still runs after the operators (diff.py:2148). Operators therefore still see identical objects on included paths. The existing test_custom_operators_should_not_equal relies on that: it reports 'unexpected:still' for 10 vs 10. It keeps working.

  • Minor behaviour change: when custom operators are set, _skip_this now runs for identical objects too. So exclude_obj_callback/include_obj_callback may be called more often than before. This only matters for callbacks with side effects.

Tests

  • tests/test_operators.py:18 and :51 look like they would fail on dev. On dev the operator's match() is called for root['b'], and in the excluded-root case the UnexpectedOperator.match raises. They pass with the reorder.
  • The tests cover the shorthand and full exclude_paths, regex, callback, exclude_types, and both prevent_default values.
  • I don't see a CI run for this branch yet.

Suggestions

  • Optional: add a case with include_paths, checking that operators are not invoked outside the included subtree, since _skip_this covers that too.
  • Optional: a CHANGELOG line, and a sentence in custom.rst saying that operators are not called for excluded paths. Some users may have relied on the old behaviour.

AI-assisted static review; the code was not executed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants