Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/contribute.rst
Original file line number Diff line number Diff line change
Expand Up @@ -33,13 +33,13 @@ Tests live inside the package, in ``src/calendaring_sync/tests/``. ``test_classi
Checking performance
====================

If your change touches ingesting, local edits or a store, run the benchmarks on ``main`` and on your branch, on the same machine, and compare:
If your change touches ingesting, local edits, pushes or a store, run the benchmarks on ``main`` and on your branch, on the same machine, and compare:

.. code-block:: bash

python -m calendaring_sync.tests.benchmark

It times a full listing of a large scope, a second listing over local edits and conflicts, and a run of single edits, each on a fresh SQLite store. Times differ between machines, so only the comparison means anything. Explain any slowdown beyond run-to-run noise in your pull request. ``--size`` and ``--runs`` change the scope's size and the number of runs.
It times a full listing of a large scope, a second listing over local edits and conflicts, a run of single edits, and pushes of those edits, each on a fresh SQLite store. Times differ between machines, so only the comparison means anything. Explain any slowdown beyond run-to-run noise in your pull request. ``--size`` and ``--runs`` change the scope's size and the number of runs.

Previewing docs
===============
Expand Down
16 changes: 8 additions & 8 deletions docs/explanation/design.rst
Original file line number Diff line number Diff line change
Expand Up @@ -122,22 +122,22 @@ A session is ``ingesting`` while pages remain and ``ingested`` once the token is
note right of ingesting: no pushes while pages remain
note right of pushing: no ingests until it ends

Before a push goes out, the core marks it :term:`in flight` in the store, in one step that fails if another process got there first, so two processes can't send the same edit. Each push has one of five outcomes:
Before a push goes out, the adapter :term:`claims <claim>` it: the core marks it :term:`in flight` in the store, in one step that fails if another process got there first, so two processes can't send the same edit. Each push has one of five outcomes:

- applied: the edit becomes the synced content
- rejected: the server refused the write because something changed since your last sync, and the edit stays pending
- gone: the object no longer exists on the server
- refused because another object holds its :term:`logical key`, which becomes a conflict
- unknown: the request may or may not have reached the server

If the process stops while a push is in flight, the push is unknown the next time you open the store. The next complete :term:`ingest` settles each unknown push:
An unknown push stays in flight, and so does one whose process stopped before the answer arrived. The next complete :term:`ingest` of its scope, in any process, settles it:

- a create that comes back under the id it chose reached the server, and the core stores the server's copy; where the server picks the id, a create that comes back with the logical key it was sent with counts the same, but only if the adapter says that key is unique
- a create that comes back with what you sent, under the id it chose, reached the server; where the server picks the id, a new object with the same logical key and what you sent counts the same, but only if the adapter says that key is unique. A different object at the id the create chose is a conflict, and one that only shares its logical key is another object, so the create goes out again
- an update that comes back with your fingerprint reached the server, and one that comes back with another fingerprint is a conflict, since someone else changed the object too
- a delete that comes back removed reached the server
- a push the change set doesn't mention didn't reach the server, and its edit is pending again

Treating an unknown push as failed could create the same object twice.
Treating an unknown push as failed could create the same object twice. An outcome that arrives after an ingest settled its push still counts while the record holds the same edit and no one has claimed it again, and changes nothing otherwise. Two processes syncing one scope at the same moment can therefore send one edit twice. A second change or delete is rejected where the adapter guards it with the object's version. A create whose id the server picks has no such guard, so the server can end up with the object twice, unless the adapter says logical keys are unique: then the server refuses the second create for its key, which becomes a conflict. Nothing you edited is lost either way.

Why the server's answer to a write isn't trusted
================================================
Expand Down Expand Up @@ -191,7 +191,7 @@ The table lists every transition with its conditions, including the ones the dia
- Pending
* - pending
- Edit back to the synced content, or discarded
- Synced
- Synced; a create the server never had is deleted from the store instead
* - pending
- Remote change or removal ingested
- Conflict, unless both sides arrived at the same content or both deleted
Expand All @@ -218,7 +218,7 @@ The table lists every transition with its conditions, including the ones the dia
- Conflict with the new remote side; synced if it now equals your edit, pending again if the server is back at the synced content, or removed if both sides deleted the object
* - conflict
- Resolved
- Synced, pending, or removed, depending on the resolution
- Synced, pending, or removed, depending on the resolution; a create the server refused is deleted from the store when you discard it or push it as an update to the object holding its logical key
* - removed
- Remote change: the server holds an object at that id again
- Synced with the new content, as the same record
Expand Down Expand Up @@ -268,9 +268,9 @@ Each :term:`conflict record` has one of five kinds:

Later remote changes update the record's remote side. The conflict resolves itself when the server reaches your edit, goes back to the synced content, or removes an object you deleted too.

The core stores conflicts instead of raising them: an exception for one object would abort every other change in the sync, and a :term:`resolver` called during a sync would run again on every retry. You resolve a record with your edit, the server's version, merged content, or a delete, and the core pushes the result as a change to the server's current version. You can't push a create the server refused for its UID again unchanged, so you can discard it, push it as an update to the object that holds the UID, or replace it with merged content, possibly with a new UID.
The core stores conflicts instead of raising them: an exception for one object would abort every other change in the sync, and a :term:`resolver` called during a sync would run again on every retry. You resolve a record with your edit (``KeepEdit``), the server's version (``TakeServer``), merged content (``Merge``), or a delete (``Delete``), and the core pushes the result as a change to the server's current version. You can't push a create the server refused for its UID again unchanged, so you can discard it (``TakeServer`` or ``Delete``), push it as an update to the object that holds the UID (``UpdateHolder``), or replace it with merged content, possibly with a new UID (``Merge``).

``resolve_conflicts(resolver)`` passes each open record to your resolver, outside any sync. Each resolution is its own transaction: ``None`` leaves a record open, and an exception stops the loop and keeps the resolutions before it.
``resolve(record, resolution)`` resolves one record you read, and refuses if its conflict changed since, so a decision is never applied to news you haven't seen. ``resolve_conflicts(resolver)`` passes each open record to your resolver, never during an ingest, and resolves each answer the same way. Each resolution is its own transaction: ``None`` leaves a record open, a record that changed while your resolver ran waits for the next call, and an exception stops the loop and keeps the resolutions before it.

A removal on the server against a local edit is always a conflict, never resolved automatically, because that would delete your edit without asking.

Expand Down
5 changes: 4 additions & 1 deletion docs/reference/glossary.rst
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,9 @@ These are the sync terms calendaring-sync's pages use. :doc:`../explanation/desi
checkpoint
The sync token a :term:`truncated page` ends at. The next request continues from it.

claim
To mark a :term:`pending edit` :term:`in flight` just before pushing it, in one step that fails if another process got there first.

conflict
An object where your :term:`pending edit` and the server disagree: both sides changed it, one side deleted it while the other edited it, or the server refused your create because another object holds its :term:`UID`. calendaring-sync reports it and leaves the decision to you.

Expand Down Expand Up @@ -56,7 +59,7 @@ These are the sync terms calendaring-sync's pages use. :doc:`../explanation/desi
A :term:`change set` that lists every object in a scope, not only the ones that changed.

in flight
The state of a :term:`push` the core has marked as sent, before the adapter reports how it ended.
The state of a :term:`push` that was :term:`claimed <claim>`, until the adapter reports how it ended or the next complete :term:`ingest` settles it.

incremental change set
A :term:`change set` that holds only what changed since the scope's :term:`sync token`.
Expand Down
14 changes: 10 additions & 4 deletions docs/reference/objects.rst
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
.. SPDX-FileCopyrightText: 2026 calendaring-sync contributors
.. SPDX-License-Identifier: AGPL-3.0-or-later

:mod:`objects`: Change sets, records, edits, conflicts, scopes, and feed entries
================================================================================
:mod:`objects`: Change sets, records, edits, conflicts, push outcomes, resolutions, scopes, and feed entries
============================================================================================================

.. meta::
:description: The data objects calendaring-sync takes and returns: change sets and their changes, records with their local edits and conflicts, scopes, feed entries, and dropped scopes.
:description: The data objects calendaring-sync takes and returns: change sets and their changes, records with their local edits and conflicts, push outcomes, resolutions, scopes, feed entries, and dropped scopes.

The data an :term:`adapter` hands the core, and what the core returns.
The data an :term:`adapter` hands the core, such as change sets and how each :term:`push` ended, the resolutions your application chooses for :term:`conflicts <conflict>`, and what the core returns.

.. automodule:: calendaring_sync.objects.change_set
:members:
Expand All @@ -24,6 +24,12 @@ The data an :term:`adapter` hands the core, and what the core returns.
.. automodule:: calendaring_sync.objects.conflict
:members:

.. automodule:: calendaring_sync.objects.push
:members:

.. automodule:: calendaring_sync.objects.resolution
:members:

.. automodule:: calendaring_sync.objects.scope
:members:

Expand Down
8 changes: 4 additions & 4 deletions docs/reference/replica.rst
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
.. SPDX-FileCopyrightText: 2026 calendaring-sync contributors
.. SPDX-License-Identifier: AGPL-3.0-or-later

:mod:`replica`: Sessions, ingesting, local edits, and the change feed
=====================================================================
:mod:`replica`: Sessions, ingesting, local edits, pushing, resolving conflicts, and the change feed
===================================================================================================

.. meta::
:description: Replica and Session, where calendaring-sync applies change sets, records local edits, stores conflicts, and serves the change feed.
:description: Replica and Session, where calendaring-sync applies change sets, records local edits, pushes them, stores and resolves conflicts, and serves the change feed.

:class:`~calendaring_sync.replica.Replica` is the entry point. Record your :term:`local edits <local edit>` as you make them. To sync, open a :term:`session` for a :term:`scope` and :term:`ingest` the :term:`change sets <change set>` an :term:`adapter` fetched. Then read the :term:`change feed` to learn what changed and which :term:`conflicts <conflict>` came up.
:class:`~calendaring_sync.replica.Replica` is the entry point. Record your :term:`local edits <local edit>` as you make them. To sync, open a :term:`session` for a :term:`scope` and :term:`ingest` the :term:`change sets <change set>` an :term:`adapter` fetched. After the last page, the adapter pushes your edits in the same session: it :term:`claims <claim>` each one, sends it, and settles it with how the :term:`push` ended. Then read the :term:`change feed` to learn what changed and which :term:`conflicts <conflict>` came up, and resolve them with :meth:`~.Replica.resolve` or :meth:`~.Replica.resolve_conflicts`.

.. automodule:: calendaring_sync.replica
:members:
1 change: 1 addition & 0 deletions news/9.feature.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Added pushes and conflict resolution. A ``Session`` lists the edits ready to push, claims each one so no other process sends it, and records how the push ended; the next complete ingest settles a push whose answer never arrived. ``Replica.resolve`` and ``Replica.resolve_conflicts`` resolve conflicts with your edit, the server's side, merged content, or a delete.
36 changes: 36 additions & 0 deletions src/calendaring_sync/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,14 @@
DuplicateObjectError,
EditRefusedError,
OpenConflictError,
PendingEditError,
PushInFlightError,
RecordValueError,
RemovedRecordError,
ScopeMismatchError,
SessionError,
StaleChangeSetError,
StaleConflictError,
StoreBusyError,
StoreError,
StoreVersionError,
Expand All @@ -28,6 +31,7 @@
TransactionRequiredError,
TrimmedFeedError,
UnexpectedStoreError,
UnknownHolderError,
UnknownRecordError,
UnknownScopeError,
)
Expand All @@ -37,53 +41,85 @@
from calendaring_sync.objects.dropped_scope import DroppedScope
from calendaring_sync.objects.edit import EditKind, LocalEdit
from calendaring_sync.objects.feed import FeedEntry, FeedEntryKind
from calendaring_sync.objects.push import (
Applied,
Gone,
LogicalKeyTaken,
PushOutcome,
Rejected,
Unknown,
)
from calendaring_sync.objects.record import Record, RecordState
from calendaring_sync.objects.resolution import (
Delete,
KeepEdit,
Merge,
Resolution,
TakeServer,
UpdateHolder,
)
from calendaring_sync.objects.scope import Scope
from calendaring_sync.replica import Replica, Session, SessionState
from calendaring_sync.store.base import StateStore
from calendaring_sync.store.memory import MemoryStore
from calendaring_sync.store.sqlite import SQLiteStore

__all__ = [
"Applied",
"CalendaringSyncError",
"Change",
"ChangeSet",
"Changed",
"Conflict",
"ConflictKind",
"Delete",
"DroppedScope",
"DuplicateObjectError",
"EditKind",
"EditRefusedError",
"FeedEntry",
"FeedEntryKind",
"Gone",
"KeepEdit",
"LocalEdit",
"LogicalKeyTaken",
"MemoryStore",
"Merge",
"OpenConflictError",
"PendingEditError",
"PushInFlightError",
"PushOutcome",
"Record",
"RecordState",
"RecordValueError",
"Rejected",
"Removed",
"RemovedRecordError",
"Replica",
"Resolution",
"SQLiteStore",
"Scope",
"ScopeMismatchError",
"Session",
"SessionError",
"SessionState",
"StaleChangeSetError",
"StaleConflictError",
"StateStore",
"StoreBusyError",
"StoreError",
"StoreVersionError",
"TakeServer",
"TransactionError",
"TransactionPermissionError",
"TransactionRequiredError",
"TrimmedFeedError",
"Unchanged",
"UnexpectedStoreError",
"Unknown",
"UnknownHolderError",
"UnknownRecordError",
"UnknownScopeError",
"UpdateHolder",
"__version__",
]
24 changes: 13 additions & 11 deletions src/calendaring_sync/_classify.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,15 +7,12 @@

from enum import StrEnum

from calendaring_sync.objects._data import REMOTE, data
from calendaring_sync.objects.change import Changed, Removed
from calendaring_sync.objects.conflict import Conflict, ConflictKind
from calendaring_sync.objects.edit import EditKind, LocalEdit
from calendaring_sync.objects.record import Record

#: What "the same data" compares, in every copy of an object. A conflict
#: holds the server's copy under the same names with ``remote_`` in front.
_DATA_FIELDS = ("fingerprint", "content", "containers", "logical_key")

#: A copy of an object: the synced copy, a server's change, or a local edit.
Copy = Record | Changed | LocalEdit

Expand All @@ -39,10 +36,6 @@ class ChangeClass(StrEnum):
CONFLICT = "conflict"


def _data(copy: object, prefix: str = "") -> tuple[object, ...]:
return tuple(getattr(copy, prefix + name) for name in _DATA_FIELDS)


def _same(a: tuple[object, ...], b: tuple[object, ...]) -> bool:
"""Whether two copies' data match.

Expand All @@ -59,18 +52,27 @@ def _same(a: tuple[object, ...], b: tuple[object, ...]) -> bool:

def same_data(a: Copy, b: Copy) -> bool:
"""Whether two copies of an object hold the same data."""
return _same(_data(a), _data(b))
return _same(data(a), data(b))


def same_remote(conflict: Conflict, change: Changed | Removed) -> bool:
"""Whether the server reports again what an open conflict already holds."""
if conflict.remote_content is None or isinstance(change, Removed):
return conflict.remote_content is None and isinstance(change, Removed)
return _same(_data(conflict, "remote_"), _data(change))
return _same(data(conflict, REMOTE), data(change))


def same_conflict(a: Conflict, b: Conflict) -> bool:
"""Whether two copies of a conflict say the same thing, whatever the server's version."""
if (a.kind, a.holder) != (b.kind, b.holder):
return False
return _same(data(a, REMOTE), data(b, REMOTE))


def classify(
synced: Record | None, remote: Changed | Removed | None, local: LocalEdit | None
synced: Record | None,
remote: Changed | Removed | None,
local: LocalEdit | None,
) -> tuple[ChangeClass, ConflictKind | None]:
"""Classify one object from its synced side, the server's report and the local edit.

Expand Down
20 changes: 20 additions & 0 deletions src/calendaring_sync/_text.py
Original file line number Diff line number Diff line change
Expand Up @@ -44,3 +44,23 @@ def check_content(content: object) -> None:
"""
if not isinstance(content, bytes):
raise TypeError(f"content must be bytes, not {type(content).__name__}")


def check_data(
content: object,
fingerprint: str | None,
containers: frozenset[str],
logical_key: str | None,
**values: str | None,
) -> None:
"""Check a copy of an object: its content with :func:`check_content`, and its strings.

Args:
content: The copy's content.
fingerprint: Its fingerprint.
containers: Its containers.
logical_key: Its logical key.
values: Any other strings that come with it, by name.
"""
check_content(content)
check_text(fingerprint=fingerprint, containers=containers, logical_key=logical_key, **values)
Loading
Loading