Complete pending children with their parent; plan recurring task trees

CHANGELOG.md

@@ -15,6 +15,8 @@ and I do try to adhere to [Semantic Versioning](https://semver.org/spec/v2.0.0.h
 
 ### Changed
 
+* Completing a task (`edit --complete`, `complete` in the interactive and multi-edit modes, `completed!` in `set-task-attribs`) now also completes its pending child tasks, recursively, and prints a warning for each.  A child has to be done before the parent can be completed.
+* `completed!` in `set-task-attribs` moves a recurring task on to its next recurrence, like `edit --complete` does, instead of completing the whole series.
 * Shell tab completion scripts (bash, zsh, fish) now ship with the package.  They work without shell configuration for system-wide installs (and for bash user/venv installs with bash-completion 2.12+); see the README for other setups.  The `make install-completion` targets are gone.
 * icalendar 6.2.0 or newer is required.
 

TASK_MANAGEMENT.md

@@ -187,6 +187,12 @@ The current default logic is to duplicate/split the completed task into a comple
 
 There is not so much support for recurrences outside the task completion code, as for now the rrule has to be manually added when creating or editing the task, or the recurrence has to be set through some another caldav client tool.  I believe recurring tasks is an important functionality, so I will implement better support for this at some point.
 
+#### Recurring task trees
+
+A recurring job is often a tree - "varnish the boat every spring and autumn" breaks down into areas, and each area into layers that have to be done in order.  Putting an `RRULE` on every node of the tree works with the completion logic above, but it breaks the rule that a parent is due after its children: once a child is completed it advances to its next occurrence, while the parent is still pending.  `RELATED-TO` points to a UID, not to a single occurrence, so the `RECURRENCE-ID` model cannot link the right occurrences to each other either.
+
+The prior art (OmniFocus repeating projects and action groups; Todoist re-adding the sub-tasks of a recurring parent, one level deep; see the task model survey in calendaring-client) puts the recurrence on the root only.  The plan for plann is the same: an `RRULE` only on the root.  Completing the root completes its pending descendants (plann does that already) and splits the completed occurrence off as a copy with a new UID, as the "safe" completion mode does.  Then the subtree is regenerated for the next occurrence: every descendant is copied with a new UID, its `DTSTART`/`DUE` is shifted by the same delta as the root, and each copy is linked to the copy of its own parent, so deeper trees keep their shape.  Copying carries every `RELATED-TO` along, and plann stores the parent/child links on both ends (`PARENT` on the child, `CHILD` on the parent), so every link - ordering links such as RFC 9253 `DEPENDS-ON` between descendants included - is rewritten to the corresponding copy, or dropped when there is none.  After copying, the completed copy of the root already lists the old children as `CHILD`, and the regenerated first-level copies already say `PARENT` to the root; the old children's `PARENT` must be pointed at the completed copy, the root's `CHILD` set replaced by the new copies, and deeper copies remapped within their generation.  If the root has a parent of its own, the completed copy drops that `PARENT` link, rather than piling up one child per occurrence on the grandparent.  The caldav library builds the completed copy in the private `_build_recurring_safe_completed()`, but `complete()` returns nothing, so it has to expose that copy first.  This is not implemented yet.
+
 ### Timestamps
 **TLDR:** Timestamps are important.  DUE should indicate when we need or want to be done with the task, DURATION should be the estimated time for doing the task.  Since DURATION and DUE cannot be combined, let rather DTSTART indicate the last possible time one can start working with the task and still have a hope to get done before the DUE timestamp.
 

USER_GUIDE.md

@@ -157,6 +157,8 @@ plann select --todo --category computer-work --start=2022-04-04 --end=2022-05-05
 plann select --todo --category computer-work --overdue edit --postpone=5d
 ```
 
+A child task has to be done before its parent can be completed, so completing a parent also completes its pending children and grandchildren, with a warning for each.  A recurring child moves on to its next recurrence, the same as a recurring task completed on its own.  Children are found through the parent's own `RELATED-TO;RELTYPE=CHILD` lines; a child that is linked only from its own side (`RELTYPE=PARENT`, as some other clients write it) is not found.  When a recurring parent moves on to its next recurrence, its non-recurring children stay completed and linked to it, and no fresh ones are made for the next recurrence (regenerating them is planned, see [TASK_MANAGEMENT.md](TASK_MANAGEMENT.md)).
+
 ## Sharing busy time between calendars
 
 `freebusy-sync` copies every busy period in the selected calendars (`--config-section`, plus any given with `--caldav-url`/`--calendar-url`) into the calendar of the config section given with `--to-section`, which must resolve to exactly one calendar (set `calendar_url` or `calendar_name` in it).  The copies hold only the time, in UTC, and a fixed summary - no description, location, attendees, timezone or UID from the original.  Run it regularly, e.g. from a systemd timer: copies are moved or deleted when the original changes, and other events in the target calendar are left alone.  Copies are never copied onward, so syncing both ways is safe:

docs/ROADMAP.md

@@ -275,6 +275,7 @@ the first priority will always be to get food on the table.
 | Item | Rough estimate | Notes | Issue |
 |---|---|---|---|
 | Creating and editing recurring tasks from the CLI | 16-24 h | `TASK_MANAGEMENT.md` calls recurring tasks "important functionality"; today the RRULE must be written by hand or set from another client.  The completion logic already handles them | |
+| Recurring task trees | 12-20 h | `RRULE` on the root only; completing the root regenerates the subtree for the next occurrence, shifted by the same delta.  See "Recurring task trees" in `TASK_MANAGEMENT.md` | |
 | `pin` - sticking tasks to the calendar | 12-20 h | Fully specified in `DESIGN.md`, never implemented.  The conceptual core of the VTODO/VEVENT pairing idea | |
 | Loadable module support in `select` | 12-16 h | Arbitrary Python for filtering, editing and ical transformation | [#6](https://github.com/pycalendar/plann/issues/6) |
 | Mergecal support - privacy-preserving calendar merging | 12-16 h | Copy busy-time between work and private calendars without leaking detail | [#24](https://github.com/pycalendar/plann/issues/24) |

plann/commands.py

@@ -26,11 +26,13 @@
     _add_comma_list,
     _comma_list_canonical,
     _comma_list_tokens,
+    _complete,
     _component_type,
     _is_comma_list_attr,
     _list,
     _process_set_arg,
     _procrastinate,
+    _RelativeCache,
     _relships_by_type,
     _set_something,
     _summary,
@@ -298,9 +300,13 @@ def _edit(ctx, cancel=None, interactive_ical=False, interactive_relations=False,
         ## TODO: should be possible to combine this with other opitions
         return
 
+    complete_cache = _RelativeCache()
+    complete_cache.add_objects(ctx.obj['objs'])
+    completed = set()
     for obj in ctx.obj['objs']:
-        if interactive:
-            _interactive_edit(obj)
+        if interactive and str(obj.icalendar_component['UID']) not in completed:
+            ## (a task completed along with its parent is not asked about again)
+            _interactive_edit(obj, complete_cache, completed)
         comp = obj.icalendar_component
         if kwargs.get('pdb'):
             _pdb_edit(obj)
@@ -309,7 +315,7 @@ def _edit(ctx, cancel=None, interactive_ical=False, interactive_relations=False,
         for canonical, tokens in add_args:
             _add_comma_list(obj, canonical, tokens)
         if complete:
-            obj.complete(handle_rrule=complete_recurrence_mode, rrule_mode=complete_recurrence_mode)
+            _complete(obj, handle_rrule=complete_recurrence_mode, rrule_mode=complete_recurrence_mode, err_callback=lambda msg: click.echo(msg, err=True), cache=complete_cache, _seen=completed)
         elif complete is False:
             obj.uncomplete()
         if cancel:
@@ -589,6 +595,11 @@ def _set_task_attribs(ctx):
     ## rather than issuing a fresh server select for each attribute (E4).
     _select(ctx=ctx, todo=True, sort_key=['{DTSTART:?{DUE:?(0000)?}?%F %H:%M:%S}', '{PRIORITY:?0?}'])
     todos = list(ctx.obj['objs'])
+    ## completing a task completes its children - make them resolve to the
+    ## objects in todos, so a stale copy is neither prompted for nor saved
+    complete_cache = _RelativeCache()
+    complete_cache.add_objects(todos)
+    completed = set()
 
     def _set_something_(something, help_text, help_url=None, default=None, objs=None):
         something_ = _comma_list_canonical(something) if _is_comma_list_attr(something) else something
@@ -620,15 +631,17 @@ def _set_something_(something, help_text, help_url=None, default=None, objs=None
             click.echo(f"For each task, {help_text}")
             click.echo('(or enter "completed!" with bang but without quotes if the task is already done)')
             for obj in objs:
+                if str(obj.icalendar_component['UID']) in completed:
+                    continue
                 comp = obj.icalendar_component
                 summary = _summary(comp)
                 value = click.prompt(summary, default=default)
                 if value == 'completed!':
-                    obj.complete()
+                    _complete(obj, err_callback=lambda msg: click.echo(msg, err=True), cache=complete_cache, _seen=completed)
                     obj.save()
-                    ## todos is fetched once - drop the task so later
-                    ## attributes don't prompt for it again
-                    todos.remove(obj)
+                    ## todos is fetched once - drop the task and the children
+                    ## completed with it, so later attributes don't prompt for them
+                    todos[:] = [x for x in todos if str(x.icalendar_component['UID']) not in completed]
                     continue
                 if _is_comma_list_attr(something):
                     comp.add(something_, _comma_list_tokens(something, value))

plann/interactive.py

@@ -25,12 +25,14 @@
 
 from plann.lib import (
     _adjust_relations,
+    _complete,
     _component_type,
     _list,
     _now,
     _process_set_arg,
     _procrastinate,
     _relationship_text,
+    _RelativeCache,
     _set_something,
     _split_vcal,
     _summary,
@@ -49,7 +51,7 @@ def _pdb_edit(obj, interactive=True):
         click.echo("happy hacking")
     breakpoint()
 
-def command_edit(obj, command, interactive=True):
+def command_edit(obj, command, interactive=True, complete_cache=None, completed=None):
     if command == 'ignore':
         return
     elif command in ('part', 'partially-complete'):
@@ -76,7 +78,7 @@ def command_edit(obj, command, interactive=True):
         ## TODO: we probably shouldn't be doing this interactively here?
         _procrastinate([obj], command.split(' ')[1], **with_params)
     elif command == 'complete':
-        obj.complete(handle_rrule=True)
+        _complete(obj, handle_rrule=True, err_callback=click.echo, cache=complete_cache, _seen=completed)
     elif command == 'cancel':
         obj.icalendar_component['STATUS'] = 'CANCELLED'
     elif command.lower().startswith('set rrule='):
@@ -213,7 +215,7 @@ def _abort(message):
     click.echo(message)
     raise click.Abort(message)
 
-def _interactive_edit(obj):
+def _interactive_edit(obj, complete_cache=None, completed=None):
     if _component_type(obj) != 'VTODO':
         raise NotImplementedError("interactive editing only implemented for tasks")
     comp = obj.icalendar_component
@@ -228,7 +230,7 @@ def _interactive_edit(obj):
     click.echo(f"pri={pri} {dtstart:%F %H:%M:%S %Z} - {due:%F %H:%M:%S %Z}: {summary}")
     input = click.prompt("postpone <n>d / ignore / part(ially-complete) / complete / split / cancel / set foo=bar / edit / family / start / pdb?", default='ignore')
     try:
-        command_edit(obj, input, interactive=True)
+        command_edit(obj, input, interactive=True, complete_cache=complete_cache, completed=completed)
     except NotImplementedError as e:
         ## e.g. `start` when extra_config.time_tracking is not configured -
         ## the prompt advertises the command, so report why it did not work
@@ -237,7 +239,7 @@ def _interactive_edit(obj):
         return
     if input == 'start':
         ## time tracking has been started - re-prompt so a follow-up command can be given for the same task
-        _interactive_edit(obj)
+        _interactive_edit(obj, complete_cache, completed)
 
 def _mass_reprioritize(objs):
     text = """\
@@ -298,10 +300,15 @@ def _mass_interactive_edit(objs, default='ignore'):
         ## TODO: this is bad design
         template=default + " {UID}: due={DUE} Pri={PRIORITY:?0?} {SUMMARY:?{DESCRIPTION:?(no summary given)?}?} (STATUS={STATUS:-})", filter=lambda obj: obj.icalendar_component.get('STATUS', 'NEEDS-ACTION')=='NEEDS-ACTION'))
     edited = _editor(text)
+    ## tasks are fetched once the editor is closed, and shared between
+    ## lines, so a task completed along with its parent is not completed
+    ## again on its own line
+    complete_cache = _RelativeCache()
+    completed = set()
     for line in edited.split('\n'):
         ## TODO: BUG: does not work if the source data comes from multiple calendars!
         ## (possible fix: make a dict from uid to calendar(s))
-        _command_line_edit(line, interactive=True, calendar=objs[0].parent)
+        _command_line_edit(line, interactive=True, calendar=objs[0].parent, complete_cache=complete_cache, completed=completed)
 
 def interactive_split_task(obj, partially_complete=False, too_big=True):
     comp = obj.icalendar_component
@@ -366,7 +373,7 @@ def _strip_line(line):
     line = line.strip()
     return line
 
-def _get_obj_from_line(line, calendar):
+def _get_obj_from_line(line, calendar, cache=None):
     ## TODO: the calendar we get here may be invalid.
     ## We need to map uids to objects when fetching events
     ## this should be done globally to reduce code duplication
@@ -377,18 +384,20 @@ def _get_obj_from_line(line, calendar):
     found = uid_re.match(line)
     assert found
     uid = found.group(1)
+    if cache is not None:
+        return cache.get_object(calendar, uid)
     obj = calendar.object_by_uid(uid)
     return obj
 
-def _command_line_edit(line, calendar, interactive=True):
+def _command_line_edit(line, calendar, interactive=True, complete_cache=None, completed=None):
     regexp = re.compile(rf"((?:set [^ ]*=[^ ]*)|(?:postpone (?:[0-9]+[{DURATION_UNITS}]|20[0-9][0-9]-[0-9][0-9]-[0-9][0-9]))|[^ ]*) (.*)$")
     line = _strip_line(line)
     if not line:
         return
     splitted = regexp.match(line)
     assert splitted
     command = splitted.group(1)
-    obj = _get_obj_from_line(splitted.group(2), calendar)
+    obj = _get_obj_from_line(splitted.group(2), calendar, complete_cache)
     assert obj
-    command_edit(obj, command, interactive)
+    command_edit(obj, command, interactive, complete_cache, completed)
 

plann/lib.py

@@ -459,6 +459,12 @@ def __init__(self):
         self._objects = {}
         self._relships = {}
 
+    def add_objects(self, objs):
+        """Seed the cache with objects already in memory, so relatives
+        resolve to those very objects rather than to fresh copies"""
+        for obj in objs:
+            self._objects[(getattr(obj.parent, 'url', None), str(obj.icalendar_component['UID']))] = obj
+
     def get_object(self, calendar, uid):
         key = (getattr(calendar, 'url', None), str(uid))
         if key not in self._objects:
@@ -516,6 +522,50 @@ def _relships_by_type(obj, reltype_wanted=None, cache=None):
     cache.store_relships(obj, reltype_wanted, ret)
     return ret
 
+def _complete(obj, handle_rrule=True, rrule_mode='safe', err_callback=print, cache=None, _seen=None):
+    """Mark obj as completed.
+
+    A child has to be done before the parent can be completed, so
+    completing a parent means the pending children (and grandchildren)
+    are done as well - they get auto-completed first, with a warning for
+    each.  Recurring children are handled the same way as obj.
+
+    When completing several objects, pass the same cache (seeded with
+    the objects) and the same _seen set to every call, so that an object
+    that is also a child of another one is completed exactly once.
+    """
+    if cache is None:
+        cache = _RelativeCache()
+    if _seen is None:
+        _seen = set()
+    uid = str(obj.icalendar_component['UID'])
+    if uid in _seen:
+        return
+    _seen.add(uid)
+    try:
+        pending = obj.is_pending()
+    except AssertionError:
+        ## caldav refuses a STATUS outside the RFC
+        pending = False
+    if not pending:
+        ## checked before the children, so a task that cannot be
+        ## completed does not leave its tree half done
+        err_callback(f"Not completing '{_summary(obj)}': STATUS is {obj.icalendar_component.get('STATUS')}")
+        return
+    for child in _relships_by_type(obj, 'CHILD', cache=cache).get('CHILD', []):
+        if str(child.icalendar_component['UID']) in _seen:
+            continue
+        try:
+            if not child.is_pending():
+                continue
+        except AssertionError:
+            ## caldav refuses a STATUS outside the RFC
+            err_callback(f"Not completing child task '{_summary(child)}' of '{_summary(obj)}': unknown STATUS {child.icalendar_component.get('STATUS')}")
+            continue
+        err_callback(f"Completing child task '{_summary(child)}' of '{_summary(obj)}'")
+        _complete(child, handle_rrule, rrule_mode, err_callback, cache, _seen)
+    obj.complete(handle_rrule=handle_rrule, rrule_mode=rrule_mode)
+
 def _relationship_text(obj, reltype_wanted=None):
     rels = obj.get_relatives(reltypes=reltype_wanted)
     if not rels: