Skip to content

Add entity audit logging and diver certification card ID - #286

Merged
kargig merged 3 commits into
mainfrom
feature/entity-audit-logging-and-cert-card-id
Oct 24, 2026
Merged

kargig merged 3 commits into
mainfrom
feature/entity-audit-logging-and-cert-card-id

Conversation

@kargig

@kargig kargig commented Oct 24, 2026

Copy link
Copy Markdown
Owner

Provide an immutable mutation trail for core entities and store diver card credentials so centers can verify certifications at booking time.

Entity audit logging:

  • Add entity_audit_logs (migration 0108) for create, update, soft_delete, hard_delete, restore, import, approve, reject, confirm, cancel, extend, and ownership_* actions across dive sites, dives, centers, routes, OCTO bookings, certifications, comments, settings, tags, managers, ownership requests, and dive-site edit requests.
  • Instrument CRUD and governance paths so audit rows share the mutation transaction (including RouteDeletionService commit=False for hide, hard-delete, and restore).
  • Record structured before/after diffs via snapshot_orm_fields (aligned field sets) and pre-deletion snapshots; distinguish soft vs hard delete in the action vocabulary.
  • On edit-request approve, log proposed_data and a dive_site-scoped update so site history includes community-approved changes.
  • Cover admin dive update/delete, ownership assign/revoke, dive-site approve/reject, and OCTO confirm/cancel/extend (status metadata only; no contact/passport PII).
  • Propagate auth provenance on request.state for required and optional auth: jwt, pat (id/name), or guest, plus IP and User-Agent forensics.
  • Redact secrets by field name (passwords, tokens including access_token, API keys) and settings-shaped {key, value} payloads when the setting name is sensitive; coerce Decimal/float/date/Enum for stable diffs.
  • Expose admin GET /entity-audit-logs and expand the Entity Lifecycle admin tab with filters, visual diffs, and the new entity/action types.

Certification card ID:

  • Add nullable card_id and certification_date on user_certifications.
  • Keep card_id/issue date on private my-certifications responses only; public profile and public cert list use UserCertificationPublicResponse without those fields.
  • Coerce empty-string card_id/date to null in schemas and Profile submit.
  • Surface Card ID and Issue Date on the profile form and badges; emit card_id in booking checkout context and auto-fill BookingWizard (clear stale numbers when a cert has no card ID).

Tests and polish:

  • Add unit/integration coverage for the audit service, directory and governance instrumentation, admin API authz, auth context (JWT/PAT/ guest), and certification card_id privacy/empty-string behavior.
  • Fix admin dark-mode header contrast and Ant Design deprecation noise where touched by the audit UI work.

Provide an immutable mutation trail for core entities and store diver
card credentials so centers can verify certifications at booking time.

Entity audit logging:
- Add entity_audit_logs (migration 0108) for create, update, soft_delete,
  hard_delete, restore, import, approve, reject, confirm, cancel, extend,
  and ownership_* actions across dive sites, dives, centers, routes,
  OCTO bookings, certifications, comments, settings, tags, managers,
  ownership requests, and dive-site edit requests.
- Instrument CRUD and governance paths so audit rows share the mutation
  transaction (including RouteDeletionService commit=False for hide,
  hard-delete, and restore).
- Record structured before/after diffs via snapshot_orm_fields (aligned
  field sets) and pre-deletion snapshots; distinguish soft vs hard
  delete in the action vocabulary.
- On edit-request approve, log proposed_data and a dive_site-scoped
  update so site history includes community-approved changes.
- Cover admin dive update/delete, ownership assign/revoke, dive-site
  approve/reject, and OCTO confirm/cancel/extend (status metadata only;
  no contact/passport PII).
- Propagate auth provenance on request.state for required and optional
  auth: jwt, pat (id/name), or guest, plus IP and User-Agent forensics.
- Redact secrets by field name (passwords, tokens including access_token,
  API keys) and settings-shaped {key, value} payloads when the setting
  name is sensitive; coerce Decimal/float/date/Enum for stable diffs.
- Expose admin GET /entity-audit-logs and expand the Entity Lifecycle
  admin tab with filters, visual diffs, and the new entity/action types.

Certification card ID:
- Add nullable card_id and certification_date on user_certifications.
- Keep card_id/issue date on private my-certifications responses only;
  public profile and public cert list use UserCertificationPublicResponse
  without those fields.
- Coerce empty-string card_id/date to null in schemas and Profile submit.
- Surface Card ID and Issue Date on the profile form and badges; emit
  card_id in booking checkout context and auto-fill BookingWizard
  (clear stale numbers when a cert has no card ID).

Tests and polish:
- Add unit/integration coverage for the audit service, directory and
  governance instrumentation, admin API authz, auth context (JWT/PAT/
  guest), and certification card_id privacy/empty-string behavior.
- Fix admin dark-mode header contrast and Ant Design deprecation noise
  where touched by the audit UI work.
@github-actions

Copy link
Copy Markdown

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 4 🔵🔵🔵🔵⚪
🧪 PR contains tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Possible Issue

In delete_route, the call to deletion_service.hard_delete_route was changed to pass route_id, current_user, migration_route_id, commit=False. The old hunk shows the previous call was hard_delete_route(route_id, current_user, migration_route_id) (three positional args). If the service signature is hard_delete_route(self, route_id, user, migration_route_id=None, commit=True), this is fine, but the diff for route_deletion_service.py is not shown, so verify the parameter order/name matches. If commit is not a keyword the service accepts, this raises TypeError on every hard delete.

success = deletion_service.hard_delete_route(
    route_id, current_user, migration_route_id, commit=False
)
Possible Issue

restore_route now calls restore_route(..., commit=False) then queries the route and logs, then commits. But the route query happens after the service call; if the service's restore sets deleted_at = None but the session isn't flushed, the subsequent db.query(DiveRoute).filter(DiveRoute.id == route_id).first() may return a stale/expired object or the query may autoflush. More importantly, if restore_route returns True but the route was already active, the audit log records a spurious restore. Confirm the service returns False when nothing was restored.

success = deletion_service.restore_route(route_id, current_user, commit=False)
if not success:
    raise HTTPException(
        status_code=status.HTTP_400_BAD_REQUEST,
        detail="Failed to restore route or insufficient permissions",
    )

route = db.query(DiveRoute).filter(DiveRoute.id == route_id).first()
log_entity_mutation(
    db,
    "dive_route",
    route_id,
    "restore",
    current_user,
    request=request,
    after_state={
        "name": route.name if route else None,
        "description": route.description if route else None,
    },
)
db.commit()
if route:
    db.refresh(route)
return route
PII in Audit Logs

_extract_user_agent and _extract_ip store raw client IP and full User-Agent in every audit row. Combined with the admin GET endpoint, this is a privacy/GDPR concern and can bloat the table. Consider truncating user_agent length and documenting retention. Also _infer_client_type treats any UA containing "python" as api_client, which will misclassify legitimate browser traffic (e.g. some bots), but that is minor.

def _extract_ip(request: Request | None) -> str | None:
    """Extract client IP address handling proxies using project utility."""
    if not request:
        return None
    try:
        from app.utils import get_client_ip

        ip = get_client_ip(request)
        return ip if ip and ip != "-" else None
    except Exception:
        headers = getattr(request, "headers", {})
        x_forwarded_for = headers.get("x-forwarded-for")
        if x_forwarded_for:
            return x_forwarded_for.split(",")[0].strip()
        client = getattr(request, "client", None)
        return client.host if client else None


def _extract_user_agent(request: Request | None) -> str | None:
    """Extract client User-Agent string."""
    if not request:
        return None
    headers = getattr(request, "headers", {})
    return headers.get("user-agent")


def _infer_client_type(request: Request | None, auth_method: str) -> str:
    """
    Infer client type from headers and authentication method.
    Returns: 'web', 'android_app', 'api_client', or 'system'.
    """
    if not request:
        return "system"

    headers = getattr(request, "headers", {})
    user_agent = (headers.get("user-agent") or "").lower()
    x_requested_with = (headers.get("x-requested-with") or "").lower()

    if "com.divemap.twa" in x_requested_with or "divemapapp" in user_agent:
        return "android_app"

    if auth_method == "pat":
        return "api_client"

    api_indicators = ("curl", "python", "postman", "httpie", "axios", "urllib")
    if any(indicator in user_agent for indicator in api_indicators):
        return "api_client"

    return "web"

⚠️ Review coverage: The following files were not included in this review because of the token budget:

  • backend/app/schemas/__init__.py
  • backend/app/services/route_deletion_service.py
  • backend/tests/test_audit_governance_integration.py
  • backend/tests/test_auth_context_propagation.py
  • backend/app/routers/system.py
  • backend/app/routers/admin/dive_sites.py
  • backend/app/routers/dives/dives_admin.py
  • backend/app/routers/tags.py
  • backend/app/models.py
  • backend/migrations/versions/0108_add_entity_audit_logs_and_cert_card_id.py
  • backend/app/routers/settings.py
  • backend/app/routers/dives/imports/confirm.py
  • backend/tests/test_admin_entity_audit_api.py
  • backend/tests/test_entity_audit_model.py
  • backend/app/routers/bookings.py
  • frontend/src/utils/formHelpers.js
  • frontend/src/pages/AdminAuditLogs.jsx
  • frontend/src/pages/Profile.jsx
  • frontend/src/components/AdminLayout.jsx
  • frontend/src/components/booking/BookingWizard.jsx
  • frontend/src/pages/Profile.test.jsx
  • REPO_MAP.md

Fix CI Ruff format check failures and address PR review findings on
route lifecycle handling and audit payload hygiene.

- Run ruff format across modified backend files to satisfy CI checks
  in backend-lint-format.yml.
- Use explicit keyword arguments for route hard deletion in dive_routes
  to ensure robust parameter binding with RouteDeletionService.
- Add session flush prior to querying restored route in restore_route
  to guarantee up-to-date state before audit logging.
- Truncate client User-Agent strings to 500 characters in audit log
  extraction to prevent database table bloat from oversize headers.
- Refine API client indicator list in entity_audit_service to prevent
  false positives from generic User-Agent strings.
Keep all diver booking preferences (including qualification details,
certNumber, and emergency contacts) in persistent browser cookies while
removing redundant cleartext localStorage storage (remediates CodeQL
CWE-312 alert).
@kargig
kargig merged commit 5b1add6 into main Oct 24, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant