feat: sync .NET client with Apify OpenAPI spec v2-2026-10-08T073338Z

CHANGELOG.md

@@ -1,5 +1,29 @@
 # Changelog
 
+## 0.5.0
+
+- Bumped `ApifyClientVersion.ApiSpecVersion` to the Apify OpenAPI spec `v2-2026-10-08T073338Z` and
+  the project version to `0.5.0`.
+- Added a `WaitForResources` option to `ActorStartOptions`/`TaskStartOptions`: `ActorClient.StartAsync`/
+  `CallAsync` and `TaskClient.StartAsync`/`CallAsync` retry a start rejected with
+  `actor-memory-limit-exceeded`/`concurrent-runs-limit-exceeded` every 10 seconds instead of throwing
+  immediately. `WaitForResources.UntilAvailable` retries until the run starts; `WaitForResources.For(seconds)`
+  bounds the retrying, after which the last error is thrown. Matches the reference client's
+  `waitForResources` start option.
+- Fixed the auto-paging collection iterators (`IterateAsync` on every collection client) to stop only once
+  a page returns no items, instead of once `offset` reached an earlier page's `total` — a stale bound that
+  cut the iteration short when the listed collection grew while it ran (e.g. a dataset collection while
+  another process created more datasets).
+- Fixed `RunClient.WaitForFinishAsync`/`BuildClient.WaitForFinishAsync` to throw `NotFoundException` after
+  3 seconds of a run/build id answering 404 continuously, instead of polling it as replica lag for the
+  entire (possibly multi-day) `waitSecs` budget.
+- Fixed `StreamedLog` (`RunClient.GetStreamedLog`) to reopen the log stream (with growing backoff up to 5s)
+  when it ends empty before the run has logged anything, instead of treating that empty stream as the
+  run's whole log. Stopping before any bytes have streamed now falls back to one plain log request instead
+  of redirecting nothing.
+- `WaitForResources.For` now rejects `NaN`/infinite budgets (use `WaitForResources.UntilAvailable` instead
+  of `For(double.PositiveInfinity)`).
+
 ## 0.4.0
 
 - Bumped `ApifyClientVersion.ApiSpecVersion` to the Apify OpenAPI spec `v2-2026-10-01T153946Z` and

docs/actors.md

@@ -42,7 +42,8 @@ foreach (var actor in page.Items)
 - `StartAsync(object? input = null, ActorStartOptions? options = null)` → `ActorRun` (returns immediately).
 - `CallAsync(object? input = null, ActorStartOptions? options = null, int? waitSecs = null, Action<string>? log = null)`
   → `ActorRun` (starts then waits; `waitSecs` bounds the wait, `null` waits indefinitely; `log`, if set,
-  redirects the run's live log to that sink for the duration of the wait).
+  redirects the run's live log to that sink for the duration of the wait). Time spent retrying
+  `ActorStartOptions.WaitForResources` does not count toward `waitSecs`.
 - `ValidateInputAsync(object? input = null, ValidateInputOptions? options = null)` → `bool`.
 
 `input` on `StartAsync`/`CallAsync`/`ValidateInputAsync` (and `RunClient.MetamorphAsync`, see
@@ -68,6 +69,7 @@ without re-encoding it as JSON.
 | `MaxTotalChargeUsd` | `double?` | Maximum total USD the run is allowed to charge. |
 | `ContentType` | `string?` | Content type of the `input` body (defaults to `application/json`). |
 | `RestartOnError` | `bool?` | Automatically restart the run's container if it exits with an error. |
+| `WaitForResources` | `WaitForResources?` | Retry the start while the account has no free memory or concurrent-run slot for it, instead of throwing immediately. `WaitForResources.UntilAvailable` retries until the run starts; `WaitForResources.For(seconds)` bounds the retrying, after which the last error is thrown. `null` (the default) disables retrying. |
 | `ForcePermissionLevel` | `string?` | Override the Actor's permission level (`LIMITED_PERMISSIONS`/`FULL_PERMISSIONS`). |
 | `Webhooks` | `object?` | Ad-hoc webhooks (any JSON-serializable list) to attach to this run. |
 

docs/builds.md

@@ -15,7 +15,8 @@ Access the account-wide build collection with `client.Builds()`, an Actor's buil
 - `AbortAsync()` → `Build`.
 - `DeleteAsync()`.
 - `WaitForFinishAsync(int? waitSecs = null)` → `Build` — client-side polling until terminal (`null`
-  waits indefinitely).
+  waits indefinitely). A build id that keeps answering 404 for 3 seconds straight (replica lag resolves
+  faster than that) throws `NotFoundException` instead of polling for the rest of `waitSecs`.
 - `GetOpenApiDefinitionAsync()` → `JsonObject?`.
 - `Log()` → `LogClient` (its `GetAsync`/`StreamAsync` methods are documented under [Logs in misc.md](misc.md#logs--clientlogbuildorrunid); on this build-nested client they throw on a 404 rather than resolve `null`, since the build itself could be what's missing).
 

docs/runs.md

@@ -24,7 +24,9 @@ collections).
 - `RebootAsync()` → `ActorRun`.
 - `ResurrectAsync(RunResurrectOptions? options = null)` → `ActorRun`.
 - `ChargeAsync(RunChargeOptions options)` → `Task` — record pay-per-event charges (idempotent).
-- `WaitForFinishAsync(int? waitSecs = null)` → `ActorRun`.
+- `WaitForFinishAsync(int? waitSecs = null)` → `ActorRun` — client-side polling until terminal (`null`
+  waits indefinitely). A run id that keeps answering 404 for 3 seconds straight (replica lag resolves
+  faster than that) throws `NotFoundException` instead of polling for the rest of `waitSecs`.
 - `Dataset()`, `KeyValueStore()`, `RequestQueue()` — the run's default storages. Their own
   `GetAsync()`/`DeleteAsync()` throw on a 404 instead of resolving `null`/no-op, since the run itself
   (not just the storage) could be what's missing; a record/request looked up by its own key or id

docs/tasks.md

@@ -31,8 +31,10 @@ a specific task with `client.Task(id)`.
 
 `TaskStartOptions` overrides the task's stored run settings for a single start: `Build`,
 `MemoryMbytes`, `TimeoutSecs`, `WaitForFinish` (server-side wait on the start call, max 60),
-`MaxItems`, `MaxTotalChargeUsd`, `RestartOnError`, and `Webhooks` (ad-hoc webhooks for this run). See
-[Actors](actors.md) for the meaning of each field.
+`MaxItems`, `MaxTotalChargeUsd`, `RestartOnError`, `WaitForResources` (retry the start while the
+account has no free memory or concurrent-run slot for it; time spent retrying does not count toward
+`CallAsync`'s `waitSecs`), and `Webhooks` (ad-hoc webhooks for this run). See [Actors](actors.md) for
+the meaning of each field.
 
 The model is named `ActorTask` (not `Task`) to avoid colliding with `System.Threading.Tasks.Task`.
 

src/Apify.Client/Apify.Client.csproj

@@ -6,7 +6,7 @@
 
     <!-- NuGet package metadata (see the publish workflow). -->
     <PackageId>Apify.Client</PackageId>
-    <Version>0.4.0</Version>
+    <Version>0.5.0</Version>
     <Authors>Apify</Authors>
     <Company>Apify</Company>
     <Product>Apify API client for .NET</Product>

src/Apify.Client/ApifyClientVersion.cs

@@ -14,11 +14,11 @@ public static class ApifyClientVersion
     /// The semantic version of this client library (see https://semver.org/). Changes to the public
     /// interface other than additive ones are considered breaking changes.
     /// </summary>
-    public const string ClientVersion = "0.4.0";
+    public const string ClientVersion = "0.5.0";
 
     /// <summary>
     /// The version of the Apify OpenAPI specification this client was generated and verified against.
     /// Corresponds to the <c>info.version</c> field of the Apify OpenAPI document.
     /// </summary>
-    public const string ApiSpecVersion = "v2-2026-10-01T153946Z";
+    public const string ApiSpecVersion = "v2-2026-10-08T073338Z";
 }

src/Apify.Client/Internal/ResourceContext.cs

@@ -38,6 +38,15 @@ internal sealed class ResourceContext
     /// </summary>
     private const int MaxWaitForFinishSecs = 999999;
 
+    /// <summary>
+    /// How long <see cref="WaitForFinishAsync"/> keeps polling a resource that keeps answering 404 (to ride
+    /// out database replica lag just after creation) before giving up and throwing, instead of running out
+    /// the full wait budget. Matches the reference client's <c>MAX_WAIT_WHEN_JOB_NOT_EXIST_MILLIS</c> — a
+    /// job that never existed (e.g. a typo'd run id) would otherwise hang for up to
+    /// <see cref="MaxWaitForFinishSecs"/> (~11.5 days) when <c>waitSecs</c> is left unset.
+    /// </summary>
+    private const int MaxWaitWhenJobNotExistMillis = 3000;
+
     private readonly string _apiOrigin;
     private string _publicOrigin;
     private TimeSpan? _requestTimeout;
@@ -211,6 +220,14 @@ public async Task<PaginationList<T>> ListResourceAsync<T>(string subPath, QueryP
     /// if <paramref name="limit"/> is set, yields at most that many items in total. <paramref name="baseQuery"/>
     /// carries the caller's filters; its <c>offset</c>/<c>limit</c> are overwritten per page.
     /// </summary>
+    /// <remarks>
+    /// Paging stops once a page returns no items (or <paramref name="limit"/> is reached), never by
+    /// comparing the offset to a page's reported <c>total</c>: the listed resource can grow while the
+    /// iteration runs (e.g. a dataset collection a running Actor keeps adding to), and a reported total
+    /// only ever reflects the moment its own page was fetched. Stopping at an early page's total would cut
+    /// the iteration off before items added later, so — like the reference client — this always issues one
+    /// more request at the end to confirm there is nothing left, unless <paramref name="limit"/> is hit first.
+    /// </remarks>
     public async IAsyncEnumerable<T> IterateListAsync<T>(
         string subPath,
         QueryParams baseQuery,
@@ -238,7 +255,7 @@ public async IAsyncEnumerable<T> IterateListAsync<T>(
             }
 
             offset += (int)page.Count;
-            if (page.Count == 0 || offset >= page.Total || (limit is not null && yielded >= limit.Value))
+            if (page.Count == 0 || (limit is not null && yielded >= limit.Value))
             {
                 yield break;
             }
@@ -408,8 +425,10 @@ private int ServerWaitCapSecs()
     /// <summary>
     /// Polls a GET endpoint with <c>waitForFinish</c> until the resource reaches a terminal state or the
     /// wait budget elapses. <paramref name="waitSecs"/> == <c>null</c> means "wait indefinitely",
-    /// implemented as a finite but very large bound so the loop always terminates. A transient 404 (replica
-    /// lag) is treated as "not yet available".
+    /// implemented as a finite but very large bound so the loop always terminates. A 404 is treated as "not
+    /// yet available" (database replica lag just after creation) for up to
+    /// <see cref="MaxWaitWhenJobNotExistMillis"/> of continuous 404s, after which it is reported as a
+    /// genuine not-found rather than retried for the rest of the (possibly multi-day) wait budget.
     /// </summary>
     public async Task<JsonObject> WaitForFinishAsync(int? waitSecs, string resourceName, Func<JsonObject, bool> isTerminal, CancellationToken ct)
     {
@@ -421,6 +440,8 @@ public async Task<JsonObject> WaitForFinishAsync(int? waitSecs, string resourceN
         var serverWaitCap = ServerWaitCapSecs();
 
         JsonObject? resource = null;
+        long? notFoundSinceMillis = null;
+        ApifyApiException? notFoundError = null;
 
         while (true)
         {
@@ -431,18 +452,35 @@ public async Task<JsonObject> WaitForFinishAsync(int? waitSecs, string resourceN
             var p = new QueryParams();
             p.AddInt("waitForFinish", requestSecs);
 
-            // Long tier (the overall per-request budget): the server may hold the connection open for up to
-            // requestSecs (derived from that same budget via ServerWaitCapSecs), so the HTTP timeout for this
-            // call must cover the full budget, not a shorter tier.
-            var data = await GetResourceAsync(string.Empty, p, RequestTier.Long, ct).ConfigureAwait(false);
-            if (data is JsonObject obj)
+            var isNotFoundNow = false;
+            try
             {
-                resource = obj;
-                if (isTerminal(obj))
+                // Long tier (the overall per-request budget): the server may hold the connection open for
+                // up to requestSecs (derived from that same budget via ServerWaitCapSecs), so the HTTP
+                // timeout for this call must cover the full budget, not a shorter tier.
+                var data = await GetResourceRequiredAsync(string.Empty, p, RequestTier.Long, ct).ConfigureAwait(false);
+                if (data is JsonObject obj)
                 {
-                    return obj;
+                    resource = obj;
+                    notFoundSinceMillis = null;
+                    notFoundError = null;
+                    if (isTerminal(obj))
+                    {
+                        return obj;
+                    }
                 }
             }
+            catch (ApifyApiException e) when (HttpClientCore.IsNotFound(e))
+            {
+                isNotFoundNow = true;
+                notFoundSinceMillis ??= stopwatch.ElapsedMilliseconds;
+                notFoundError = e;
+            }
+
+            if (isNotFoundNow && stopwatch.ElapsedMilliseconds - notFoundSinceMillis!.Value >= MaxWaitWhenJobNotExistMillis)
+            {
+                throw notFoundError!;
+            }
 
             if (stopwatch.ElapsedMilliseconds >= budgetMillis)
             {
@@ -457,6 +495,11 @@ public async Task<JsonObject> WaitForFinishAsync(int? waitSecs, string resourceN
             return resource;
         }
 
+        if (notFoundError is not null)
+        {
+            throw notFoundError;
+        }
+
         throw new InvalidOperationException(
             $"waiting for {resourceName} to finish failed: cannot fetch {resourceName} details from the server");
     }

src/Apify.Client/Internal/ResourceLimitRetry.cs

@@ -0,0 +1,83 @@
+using System;
+using System.Threading;
+using System.Threading.Tasks;
+using Apify.Client.Exceptions;
+using Apify.Client.Options;
+
+namespace Apify.Client.Internal;
+
+/// <summary>
+/// Implements <see cref="ActorStartOptions.WaitForResources"/>/<see cref="TaskStartOptions.WaitForResources"/>:
+/// retries a run start while the API rejects it for lack of resources. Matches the reference client's
+/// <c>startWaitingForResources()</c>.
+/// </summary>
+internal static class ResourceLimitRetry
+{
+    /// <summary>
+    /// Error types the API rejects a run start with while the account has no free memory or concurrent-run
+    /// slot for it. Both clear as other runs or builds finish.
+    /// </summary>
+    private const string MemoryLimitErrorType = "actor-memory-limit-exceeded";
+
+    private const string ConcurrentRunsLimitErrorType = "concurrent-runs-limit-exceeded";
+
+    /// <summary>Cooldown between two attempts to start a run that was rejected for lack of resources.</summary>
+    private const int CooldownMillis = 10_000;
+
+    /// <summary>
+    /// Calls <paramref name="start"/>, retrying it every <paramref name="cooldownMillis"/> while it fails
+    /// with a resource-limit error. <paramref name="waitForResources"/> <c>null</c> disables retrying
+    /// entirely (the first error is thrown as-is); otherwise <see cref="WaitForResources.MaxWaitSecs"/>
+    /// <c>null</c> retries until the call succeeds, and a value bounds the retrying, after which the last
+    /// error is thrown. Any other error is thrown right away.
+    /// </summary>
+    /// <param name="start">Makes one start attempt.</param>
+    /// <param name="waitForResources">The retry configuration, or <c>null</c> to disable retrying.</param>
+    /// <param name="cancellationToken">A token to cancel the wait between attempts.</param>
+    /// <param name="cooldownMillis">
+    /// The cooldown between attempts; defaults to <see cref="CooldownMillis"/>. Overridable only so unit
+    /// tests can exercise multi-attempt retries without a real 10-second wait per attempt.
+    /// </param>
+    public static async Task<T> StartAsync<T>(
+        Func<Task<T>> start,
+        WaitForResources? waitForResources,
+        CancellationToken cancellationToken,
+        int cooldownMillis = CooldownMillis)
+    {
+        if (waitForResources is null)
+        {
+            return await start().ConfigureAwait(false);
+        }
+
+        var budgetMillis = waitForResources.Value.MaxWaitSecs is { } secs ? (long)(secs * 1000) : (long?)null;
+        var stopwatch = budgetMillis is not null ? System.Diagnostics.Stopwatch.StartNew() : null;
+
+        while (true)
+        {
+            try
+            {
+                return await start().ConfigureAwait(false);
+            }
+            catch (ApifyApiException e) when (IsResourceLimitError(e))
+            {
+                if (budgetMillis is not null)
+                {
+                    var remainingMillis = budgetMillis.Value - stopwatch!.ElapsedMilliseconds;
+                    if (remainingMillis <= 0)
+                    {
+                        throw;
+                    }
+
+                    await Task.Delay((int)Math.Min(cooldownMillis, remainingMillis), cancellationToken).ConfigureAwait(false);
+                }
+                else
+                {
+                    await Task.Delay(cooldownMillis, cancellationToken).ConfigureAwait(false);
+                }
+            }
+        }
+    }
+
+    private static bool IsResourceLimitError(ApifyApiException e) =>
+        e.Type is MemoryLimitErrorType or ConcurrentRunsLimitErrorType;
+}

src/Apify.Client/Options/ActorStartOptions.cs

@@ -31,6 +31,13 @@ public sealed class ActorStartOptions
     /// <summary>If <c>true</c>, restart the run if it fails.</summary>
     public bool? RestartOnError { get; init; }
 
+    /// <summary>
+    /// Retries the start while the account lacks the memory or a concurrent-run slot for the run (an
+    /// <c>actor-memory-limit-exceeded</c>/<c>concurrent-runs-limit-exceeded</c> error), instead of throwing
+    /// immediately. <c>null</c> (the default) disables retrying.
+    /// </summary>
+    public WaitForResources? WaitForResources { get; init; }
+
     /// <summary>
     /// Override the Actor's permission level for this run (<c>LIMITED_PERMISSIONS</c>/<c>FULL_PERMISSIONS</c>).
     /// </summary>

src/Apify.Client/Options/TaskStartOptions.cs

@@ -32,6 +32,13 @@ public sealed class TaskStartOptions
     /// <summary>If <c>true</c>, restart the run if it fails.</summary>
     public bool? RestartOnError { get; init; }
 
+    /// <summary>
+    /// Retries the start while the account lacks the memory or a concurrent-run slot for the run (an
+    /// <c>actor-memory-limit-exceeded</c>/<c>concurrent-runs-limit-exceeded</c> error), instead of throwing
+    /// immediately. <c>null</c> (the default) disables retrying.
+    /// </summary>
+    public WaitForResources? WaitForResources { get; init; }
+
     /// <summary>Ad-hoc webhooks to attach (a JSON-serializable list serialized to base64-encoded JSON).</summary>
     public object? Webhooks { get; init; }