Skip to content

fix: fail fast when sharing segmenter weights without a compressor - #77

Open
ClaireXi99 wants to merge 1 commit into
zjunlp:mainfrom
ClaireXi99:fix/topic-segmenter-shared-without-compressor
Open

fix: fail fast when sharing segmenter weights without a compressor#77
ClaireXi99 wants to merge 1 commit into
zjunlp:mainfrom
ClaireXi99:fix/topic-segmenter-shared-without-compressor

Conversation

@ClaireXi99

Copy link
Copy Markdown

What breaks

LlmLingua2Segmenter.__init__ has two initialization paths:

if shared is False:
    self.model = AutoModel.from_pretrained(...)      # loads its own weights
    self.tokenizer = ...
    self.buffer_len = ...
elif compressor is not None:
    self.model = compressor.inner_compressor.model   # shares the compressor's weights
    self.tokenizer = ...
    self.buffer_len = ...

When shared is True and no compressor is supplied, neither branch runs. The object is constructed successfully but without model, tokenizer or buffer_len, so the failure surfaces later and far from its cause — LightMemory.__init__ immediately reads self.segmenter.buffer_len, giving:

AttributeError: 'LlmLingua2Segmenter' object has no attribute 'buffer_len'

That message points at the segmenter's internals rather than at the actual problem, which is a configuration one: precomp_topic_shared=True asks the segmenter to share a pre-compressor's weights while pre_compress=False means there is no pre-compressor to share.

Reproduced directly against the class:

>>> seg = LlmLingua2Segmenter(config={"model_name": "dummy"}, shared=True, compressor=None)
>>> [hasattr(seg, a) for a in ("model", "tokenizer", "buffer_len")]
[False, False, False]
>>> seg.buffer_len
AttributeError: 'LlmLingua2Segmenter' object has no attribute 'buffer_len'

Fix

Add the missing else branch so an unsatisfiable configuration fails immediately with an actionable message naming both ways out:

precomp_topic_shared=True requires a pre-compressor to share weights with, but none
was provided. Either set pre_compress=True and configure pre_compressor, or set
precomp_topic_shared=False to let the segmenter load its own model.

This is intentionally narrow — it only converts a silent half-initialization into a clear error. Neither existing branch changes behavior.

Relationship to PR #75

PR #75 fixes a related crash one layer up, where LightMemory passed an unset self.compressor to the segmenter factory. The two changes are complementary rather than overlapping: #75 touches only lightmem.py, this one only llmlingua_2.py, and I have deliberately left #75's one-line fix out of this branch to avoid duplicating it.

Worth noting that #75 makes this bug more reachable. Once self.compressor correctly defaults to None, the precomp_topic_shared=True plus pre_compress=False combination stops failing on the missing attribute and starts flowing into the segmenter as compressor=None with shared=True — landing exactly on the branchless path above. That is the same cascade described in PR #63, where PR #59's fix removed an early return and exposed an uninitialized variable underneath.

With precomp_topic_shared at its default of False nothing changes: shared is False takes the first branch and the segmenter loads its own model, so #75's scenario and its assertion of (topic_segmenter, False, None) are unaffected.

Tests

Added tests/test_topic_segmenter_shared_weights.py covering both sharing paths:

  • shared=True with no compressor raises the new ValueError
  • shared=True with a compressor still adopts its model, tokenizer and max_position_embeddings-derived buffer_len

The shared=False path is unchanged and untested here because it downloads a real model. The module imports torch, so the file uses pytest.importorskip("torch").

$ python -m pytest tests/ -v
tests/test_sensory_memory.py::test_oversized_single_user_message_is_consumed PASSED
tests/test_sensory_memory.py::test_force_segment_flushes_remaining_buffer PASSED
tests/test_topic_segmenter_shared_weights.py::test_shared_without_compressor_raises_clear_error PASSED
tests/test_topic_segmenter_shared_weights.py::test_shared_with_compressor_reuses_its_weights PASSED

4 passed

Verified on Python 3.11 with the pinned transformers==4.57.0; the pre-existing sensory-memory tests still pass.

LlmLingua2Segmenter.__init__ loads its own model when shared is False and
adopts the compressor's weights when a compressor is given, but when shared
is True and no compressor is supplied neither branch runs. The instance is
then built without model, tokenizer or buffer_len, and the first access
fails with a confusing AttributeError far from the real cause.

Raise a ValueError naming both ways to resolve the configuration instead,
and cover the sharing paths with tests.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant