Skip to content

docs: clarify reusable flush and terminal shutdown guidance #1043

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
marandaneto wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

docs: clarify reusable flush and terminal shutdown guidance #1043

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .sampo/changesets/flush-shutdown-lifetime-guidance.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
pypi/posthog: patch
---

Clarify when to flush reusable clients versus shut down disposable clients, including timeout and delivery limitations.
47 changes: 40 additions & 7 deletions posthog/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1226,16 +1226,39 @@ def load_feature_flags():

def flush(timeout_seconds: Optional[float] = 10) -> None:
"""
Tell the client to flush all queued events.
Attempt immediate delivery of queued events and ended spans, keeping the global client reusable.

Call ``flush()`` before a short-lived request or serverless invocation returns
when using the shared module-level client. Use ``shutdown()`` only for final
process cleanup or a client that will not be reused. Flush does not stop the
background workers or feature flag poller, and does not end open spans.

Returning does not guarantee server receipt: pending work can remain after a
timeout or span export failure, and failed events may be dropped by consumers.
Calls made directly from SDK callbacks such as ``on_error`` are deferred
rather than blocking the worker that invoked the callback.

Args:
timeout_seconds: Maximum seconds to wait for the queue to flush.
Defaults to 10 seconds. Pass ``None`` to wait indefinitely.
timeout_seconds: Seconds to wait for event queues to drain, shared across
event lanes. Defaults to 10 seconds. Pass ``None`` to wait without a
time limit; this does not guarantee successful delivery. Queued spans
flush concurrently, waiting up to this many seconds for an in-flight
flush before starting a fresh drain budget. A lock wait timeout leaves
the in-flight flush to finish. At least one span request is attempted
after acquiring the lock, even with no budget left; retriable failures
are retried after backoff while budget remains, with one last attempt
at the deadline. The last span request is not cut short and is bounded
by the client's ``timeout`` in seconds,
so this is not a strict wall-clock limit for the whole call. With
``None``, a retriable span failure is left for a later flush rather than
retried within this call.

Examples:
```python
from posthog import flush
flush()
import posthog
posthog.capture('event_name', distinct_id='user_id')
# Before a serverless invocation returns; keep the global client open.
posthog.flush(timeout_seconds=10)
```

Category:
Expand Down Expand Up @@ -1343,11 +1366,21 @@ def get_active_span() -> Optional[Span]:

def shutdown() -> None:
"""
Flush all messages and cleanly shutdown the client.
Attempt final delivery and permanently shut down the global client.

Call this for final process cleanup, not at the end of each request or
serverless invocation that shares the module-level client. Use ``flush()``
before returning from those invocations instead. Shutdown closes capture
lanes, stops background workers and the feature flag poller, and tears down
integrations. The global client is not recreated automatically after shutdown.

This normally blocks until queued events have been attempted and cleanup
finishes. Failed or undrainable events may be dropped and reported through
logging or ``on_error``; returning does not guarantee server receipt.
logging or ``on_error``; returning does not guarantee server receipt. Event
draining has no timeout. Queued spans get a final flush with up to 30 seconds
waiting for an in-flight flush, then a fresh 30-second drain budget; the last
HTTP request is not cut short. Unsent queued spans and still-open spans are
discarded with a warning.
Lifecycle cleanup is attempted once, and cleanup failures are logged without
retry. Calls made directly from SDK callbacks such as ``on_error`` are deferred
to avoid deadlocking the worker. If blocking completion is required, signal an application-owned
Expand Down
60 changes: 42 additions & 18 deletions posthog/client.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2715,25 +2715,41 @@ def get_active_span(self) -> Optional[Span]:

def flush(self, timeout_seconds: Optional[float] = 10) -> None:
"""
Force a flush from the internal queue to the server. Do not use directly, call `shutdown()` instead.
Attempt immediate delivery of queued events and ended spans, keeping the client reusable.

For a client shared across requests or serverless invocations (including
the module-level client), call ``flush()`` before a short-lived invocation
returns. Use ``shutdown()`` instead only for final process cleanup or when
disposing of a client that will not be reused. Flush does not stop the
background workers or feature flag poller, and does not end open spans.

Returning does not guarantee server receipt: pending work can remain after
a timeout or span export failure, and failed events may be dropped by the
consumers. Calls made directly from SDK callbacks such as ``on_error`` are
deferred rather than blocking the worker that invoked the callback.

Args:
timeout_seconds: Maximum seconds to wait for the queue to flush.
Defaults to 10 seconds. Pass ``None`` to wait indefinitely.
Queued spans are sent at the same time, within the same
budget: at least one span request is attempted even when the
budget is already spent, a retriable failure is retried after
its backoff while budget remains, no other request starts once
it is spent, and each request is bounded by ``timeout``. The
wait for the last request is not cut short, so a flush can
take up to ``timeout_seconds`` plus ``timeout`` in the worst
case. A span flush already in flight for the whole wait is
left to finish instead.
timeout_seconds: Seconds to wait for event queues to drain, shared
across event lanes. Defaults to 10 seconds. Pass ``None`` to wait
without a time limit; this does not guarantee successful delivery.
Queued spans flush concurrently. The span exporter waits up to
this many seconds for an in-flight flush, then uses a fresh drain
budget once it acquires the lock. If the lock wait times out, the
in-flight flush is left to finish. At least one span request is
attempted after acquiring the lock, even with no budget left;
retriable failures are retried after backoff while budget remains,
with one last attempt at the deadline. Each span request is
bounded by the client's ``timeout`` in
seconds, and the last request is not cut short. This is therefore
not a strict wall-clock limit for the whole call. With ``None``,
a retriable span failure is left for a later flush rather than
retried within this call.

Examples:
```python
posthog.capture('event_name')
posthog.flush() # Ensures the event is sent immediately
posthog.capture('event_name', distinct_id='user_id')
# Before a serverless invocation returns; keep the shared client open.
posthog.flush(timeout_seconds=10)
```
"""
if self._defer_flush_from_callback(timeout_seconds):
Expand Down Expand Up @@ -3130,14 +3146,22 @@ def join(self) -> None:
@no_throw()
def shutdown(self) -> None:
"""
Flush all messages and cleanly shutdown the client. Call this before the process ends in serverless environments to avoid data loss.
Attempt final delivery and permanently shut down the client.

Call this for final process cleanup or when disposing of a client that
will not be reused. It closes capture lanes, stops background workers and
the feature flag poller, and tears down integrations. Do not call it at the
end of each request or serverless invocation when the client is shared;
call ``flush()`` before returning instead. A shut-down client cannot be
reused; create a new client if needed.

Normally this method blocks until queued events have been attempted and
cleanup finishes. Failed or undrainable events may be dropped and
reported through logging or ``on_error``; returning does not guarantee
server receipt. Queued spans get one final flush of up to 30 s (plus a
request already in flight); any it cannot send are discarded with a
warning, as are spans still open.
server receipt. Event draining has no timeout. Queued spans get one final
flush with up to 30 seconds waiting for an in-flight flush, then a fresh
30-second drain budget; the last HTTP request is not cut short. Any spans
it cannot send are discarded with a warning, as are spans still open.
Lifecycle cleanup is attempted once, and cleanup failures
are logged without retry. When called directly from an SDK callback such as
``on_error``, shutdown is deferred to avoid blocking the worker that
Expand Down
16 changes: 8 additions & 8 deletions references/posthog-python-references-7.67.0.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1001,13 +1001,13 @@
{
"id": "flush",
"title": "flush",
"description": "Force a flush from the internal queue to the server. Do not use directly, call `shutdown()` instead.",
"description": "Attempt immediate delivery of queued events and ended spans, keeping the client reusable. For a client shared across requests or serverless invocations (including the module-level client), call ``flush()`` before a short-lived invocation returns. Use ``shutdown()`` instead only for final process cleanup or when disposing of a client that will not be reused. Flush does not stop the background workers or feature flag poller, and does not end open spans. Returning does not guarantee server receipt: pending work can remain after a timeout or span export failure, and failed events may be dropped by the consumers. Calls made directly from SDK callbacks such as ``on_error`` are deferred rather than blocking the worker that invoked the callback.",
"details": "",
"category": null,
"params": [
{
"name": "timeout_seconds",
"description": "Maximum seconds to wait for the queue to flush. Defaults to 10 seconds. Pass ``None`` to wait indefinitely. Queued spans are sent at the same time, within the same",
"description": "Seconds to wait for event queues to drain, shared across event lanes. Defaults to 10 seconds. Pass ``None`` to wait without a time limit; this does not guarantee successful delivery. Queued spans flush concurrently. The span exporter waits up to this many seconds for an in-flight flush, then uses a fresh drain budget once it acquires the lock. If the lock wait times out, the in-flight flush is left to finish. At least one span request is attempted after acquiring the lock, even with no budget left; retriable failures are retried after backoff while budget remains, with one last attempt at the deadline. Each span request is bounded by the client's ``timeout`` in seconds, and the last request is not cut short. This is therefore not a strict wall-clock limit for the whole call. With ``None``, a retriable span failure is left for a later flush rather than retried within this call.",
"isOptional": true,
"type": "float"
}
Expand All @@ -1022,7 +1022,7 @@
{
"id": "example_1",
"name": "Example 1",
"code": "posthog.capture('event_name')\nposthog.flush() # Ensures the event is sent immediately"
"code": "posthog.capture('event_name', distinct_id='user_id')\n# Before a serverless invocation returns; keep the shared client open.\nposthog.flush(timeout_seconds=10)"
}
]
},
Expand Down Expand Up @@ -2050,7 +2050,7 @@
{
"id": "shutdown",
"title": "shutdown",
"description": "Flush all messages and cleanly shutdown the client. Call this before the process ends in serverless environments to avoid data loss. Normally this method blocks until queued events have been attempted and cleanup finishes. Failed or undrainable events may be dropped and reported through logging or ``on_error``; returning does not guarantee server receipt. Queued spans get one final flush of up to 30 s (plus a request already in flight); any it cannot send are discarded with a warning, as are spans still open. Lifecycle cleanup is attempted once, and cleanup failures are logged without retry. When called directly from an SDK callback such as ``on_error``, shutdown is deferred to avoid blocking the worker that invoked the callback. If the callback must coordinate a blocking shutdown, have it signal an application-owned thread and return before that thread calls shutdown. Do not wait inside the callback for another thread or task that calls a lifecycle method.",
"description": "Attempt final delivery and permanently shut down the client. Call this for final process cleanup or when disposing of a client that will not be reused. It closes capture lanes, stops background workers and the feature flag poller, and tears down integrations. Do not call it at the end of each request or serverless invocation when the client is shared; call ``flush()`` before returning instead. A shut-down client cannot be reused; create a new client if needed. Normally this method blocks until queued events have been attempted and cleanup finishes. Failed or undrainable events may be dropped and reported through logging or ``on_error``; returning does not guarantee server receipt. Event draining has no timeout. Queued spans get one final flush with up to 30 seconds waiting for an in-flight flush, then a fresh 30-second drain budget; the last HTTP request is not cut short. Any spans it cannot send are discarded with a warning, as are spans still open. Lifecycle cleanup is attempted once, and cleanup failures are logged without retry. When called directly from an SDK callback such as ``on_error``, shutdown is deferred to avoid blocking the worker that invoked the callback. If the callback must coordinate a blocking shutdown, have it signal an application-owned thread and return before that thread calls shutdown. Do not wait inside the callback for another thread or task that calls a lifecycle method.",
"details": "",
"category": null,
"params": [],
Expand Down Expand Up @@ -2489,13 +2489,13 @@
{
"id": "flush",
"title": "flush",
"description": "Tell the client to flush all queued events.",
"description": "Attempt immediate delivery of queued events and ended spans, keeping the global client reusable. Call ``flush()`` before a short-lived request or serverless invocation returns when using the shared module-level client. Use ``shutdown()`` only for final process cleanup or a client that will not be reused. Flush does not stop the background workers or feature flag poller, and does not end open spans. Returning does not guarantee server receipt: pending work can remain after a timeout or span export failure, and failed events may be dropped by consumers. Calls made directly from SDK callbacks such as ``on_error`` are deferred rather than blocking the worker that invoked the callback.",
"details": "",
"category": "Client management",
"params": [
{
"name": "timeout_seconds",
"description": "Maximum seconds to wait for the queue to flush. Defaults to 10 seconds. Pass ``None`` to wait indefinitely.",
"description": "Seconds to wait for event queues to drain, shared across event lanes. Defaults to 10 seconds. Pass ``None`` to wait without a time limit; this does not guarantee successful delivery. Queued spans flush concurrently, waiting up to this many seconds for an in-flight flush before starting a fresh drain budget. A lock wait timeout leaves the in-flight flush to finish. At least one span request is attempted after acquiring the lock, even with no budget left; retriable failures are retried after backoff while budget remains, with one last attempt at the deadline. The last span request is not cut short and is bounded by the client's ``timeout`` in seconds, so this is not a strict wall-clock limit for the whole call. With ``None``, a retriable span failure is left for a later flush rather than retried within this call.",
"isOptional": true,
"type": "float"
}
Expand All @@ -2510,7 +2510,7 @@
{
"id": "example_1",
"name": "Example 1",
"code": "from posthog import flush\nflush()"
"code": "import posthog\nposthog.capture('event_name', distinct_id='user_id')\n# Before a serverless invocation returns; keep the global client open.\nposthog.flush(timeout_seconds=10)"
}
]
},
Expand Down Expand Up @@ -3421,7 +3421,7 @@
{
"id": "shutdown",
"title": "shutdown",
"description": "Flush all messages and cleanly shutdown the client. This normally blocks until queued events have been attempted and cleanup finishes. Failed or undrainable events may be dropped and reported through logging or ``on_error``; returning does not guarantee server receipt. Lifecycle cleanup is attempted once, and cleanup failures are logged without retry. Calls made directly from SDK callbacks such as ``on_error`` are deferred to avoid deadlocking the worker. If blocking completion is required, signal an application-owned thread, return from the callback, and call ``shutdown()`` from that thread. Do not wait inside a callback for another thread or task calling a lifecycle method.",
"description": "Attempt final delivery and permanently shut down the global client. Call this for final process cleanup, not at the end of each request or serverless invocation that shares the module-level client. Use ``flush()`` before returning from those invocations instead. Shutdown closes capture lanes, stops background workers and the feature flag poller, and tears down integrations. The global client is not recreated automatically after shutdown. This normally blocks until queued events have been attempted and cleanup finishes. Failed or undrainable events may be dropped and reported through logging or ``on_error``; returning does not guarantee server receipt. Event draining has no timeout. Queued spans get a final flush with up to 30 seconds waiting for an in-flight flush, then a fresh 30-second drain budget; the last HTTP request is not cut short. Unsent queued spans and still-open spans are discarded with a warning. Lifecycle cleanup is attempted once, and cleanup failures are logged without retry. Calls made directly from SDK callbacks such as ``on_error`` are deferred to avoid deadlocking the worker. If blocking completion is required, signal an application-owned thread, return from the callback, and call ``shutdown()`` from that thread. Do not wait inside a callback for another thread or task calling a lifecycle method.",
"details": "",
"category": "Client management",
"params": [],
Expand Down
Loading
Loading