Describe the bug
SQLiteMemory stringifies every prompt_metadata value before comparing it with JSON_EXTRACT, so a filter on an integer value matches nothing — and the same query with the value as a string does not match either, because SQLite will not compare an integer against text.
MemoryInterface.get_message_pieces types the filter as dict[str, str | int], and PyRIT itself stores an integer there: pyrit/prompt_target/common/prompt_target.py:179 sets piece.prompt_metadata[RequestTraceContext.REQUEST_METADATA_KEY] = 1 on every outgoing request. So asking for the pieces of a request returns an empty list on SQLite, while the same call on AzureSQLMemory returns them.
The fix looks local to sqlite_memory.py:174. Two neighbouring implementations already do the right thing:
sqlite_memory.py:186-188, the seed path in the same file, deliberately does not stringify: # Note: We do NOT convert values to string here, to allow integer comparison in JSON
azure_sql_memory.py:318-319 stringifies because of a SQL Server premise that does not hold in SQLite: JSON_VALUE always returns nvarchar in SQL Server, so we must convert all values to strings
SQLite's json_extract is JSON-type-preserving (select json_extract('{"a":5}','$.a') = 5 → 1, = '5' → 0), so the SQL Server rule appears to have been carried over without its premise.
Steps/Code to Reproduce
from pyrit.memory import SQLiteMemory
from pyrit.models.messages.message_piece import MessagePiece
from pyrit.models.seeds.seed_prompt import SeedPrompt
import uuid
memory = SQLiteMemory()
memory.add_message_pieces_to_memory(
message_pieces=[
MessagePiece(
conversation_id=str(uuid.uuid4()),
role="user",
original_value="hi",
prompt_metadata={"pyrit_target_request": 1},
)
]
)
await memory.add_seeds_to_memory_async(
seeds=[SeedPrompt(value="hi", metadata={"pyrit_target_request": 1})], added_by="repro"
)
print(len(memory.get_message_pieces(prompt_metadata={"pyrit_target_request": 1}))) # pieces, int
print(len(memory.get_message_pieces(prompt_metadata={"pyrit_target_request": "1"}))) # pieces, str
print(len(memory.get_seeds(metadata={"pyrit_target_request": 1}))) # seeds, int
Expected Results
1, 0 (an int-valued filter cannot match a text bind parameter) and 1 — i.e. the seed path and the message-piece path should agree, and the integer filter should find the row.
Actual Results
The third line is the same JSON column and the same dict[str, str | int] contract, read through the helper that does not stringify; the first two go through the one that does.
Screenshots
Not applicable.
Versions
- OS: macOS (Darwin arm64)
- Python version: 3.11.15
- PyRIT version: 1.2.0.dev0, installed from
main at 9c26cd7
- Relevant packages: sqlalchemy 2.0.51, pydantic 2.13.4
Describe the bug
SQLiteMemorystringifies everyprompt_metadatavalue before comparing it withJSON_EXTRACT, so a filter on an integer value matches nothing — and the same query with the value as a string does not match either, because SQLite will not compare an integer against text.MemoryInterface.get_message_piecestypes the filter asdict[str, str | int], and PyRIT itself stores an integer there:pyrit/prompt_target/common/prompt_target.py:179setspiece.prompt_metadata[RequestTraceContext.REQUEST_METADATA_KEY] = 1on every outgoing request. So asking for the pieces of a request returns an empty list on SQLite, while the same call onAzureSQLMemoryreturns them.The fix looks local to
sqlite_memory.py:174. Two neighbouring implementations already do the right thing:sqlite_memory.py:186-188, the seed path in the same file, deliberately does not stringify:# Note: We do NOT convert values to string here, to allow integer comparison in JSONazure_sql_memory.py:318-319stringifies because of a SQL Server premise that does not hold in SQLite:JSON_VALUE always returns nvarchar in SQL Server, so we must convert all values to stringsSQLite's
json_extractis JSON-type-preserving (select json_extract('{"a":5}','$.a') = 5→ 1,= '5'→ 0), so the SQL Server rule appears to have been carried over without its premise.Steps/Code to Reproduce
Expected Results
1,0(an int-valued filter cannot match a text bind parameter) and1— i.e. the seed path and the message-piece path should agree, and the integer filter should find the row.Actual Results
The third line is the same JSON column and the same
dict[str, str | int]contract, read through the helper that does not stringify; the first two go through the one that does.Screenshots
Not applicable.
Versions
mainat 9c26cd7