Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 33 additions & 10 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -159,6 +159,24 @@ open a second one for a new topic. Fatal and
handled errors forward to the `CrashSink` set on `Log` — Crashlytics, in a
prerelease build (see [Logging](#logging)).

Redundancy is a property of the **tier**, not of the feature: `lbApi`,
`lbStatic`, `coreApi` and `coreStatic` name both regions and fail over, the
`*Exclusive` tiers and `legacyApi` name one origin. Which family lives where is
the backend's decision and changes without a client release, so the tier is
chosen per datasource from what the endpoint actually does — an endpoint that
speaks SSE stays on `lbApi` (only the `lb-*` nodes terminate it), the archive
and static families follow whichever regions have a copy (see
`api.md#curl-可用性2026-11-20-實測http-狀態碼` and the multi-active table).

Two requests fall outside that model and must not be forced into it: the
basemap/OSM/terrain tiles are fetched by MapLibre, not `ApiClient`, so they are
**pinned to the selected region** (`staticLbOrigin(regions.lb)` — a pin, no
failover, which is why `etag_interceptor.dart` keeps recognising the bare host
of rows cached before the pin existed); and notify/location are primary on
tnn1 with tyo1 as a **hot standby** that answers `307` back to the primary
while it is healthy, so they stay `coreExclusiveApi` and rely on Dio following
the redirect.

## Data & errors

models are `@freezed` value types with generated
Expand Down Expand Up @@ -233,16 +251,21 @@ the `ServerClock` corrects device time via real **SNTP**
timezone change can't move it (only a resync does).

When both SNTP hosts fail, `fallback_time_source.dart` drops to
`https://api.lb.exptech.dev/ntp` (`http_time_source.dart`; the iOS widget
mirrors it in `WidgetHTTPTimeSource.swift`) — corporate Wi-Fi, hotel and campus
networks block UDP/123, and there the alternative is no calibration at all. The
order is precision, not preference: SNTP resolves "now" to about a millisecond,
one HTTPS round trip only to ±RTT/2. This is the one bare host the repository
requests, because a clock reading has no region to pin; the body is never
trusted on its own but checked against the Cloudflare-stamped `Date` header of
the same response, which is what catches a backend whose own clock is wrong —
see [api.md](api.md) for the 15.7-second measurement that made the check
necessary.
`https://api.lb-<region>.exptech.dev/ntp` (`http_time_source.dart`; the iOS
widget mirrors it in `WidgetHTTPTimeSource.swift`) — corporate Wi-Fi, hotel and
campus networks block UDP/123, and there the alternative is no calibration at
all. The order is precision, not preference: SNTP resolves "now" to about a
millisecond, one HTTPS round trip only to ±RTT/2.

The hosts are the app's own region pair, its own region first
(`HttpTimeSource.hostsFor(regions.lbOrder)`), so the fallback follows the region
the user picked instead of leaving the choice to Cloudflare — that bare name
was retired because the `lb-*` pool it hides contains backends 15.7 seconds
slow. It is not an `ApiTier`: no `ApiClient`, no tier failover, and a per-host
retry of its own when the body disagrees with the response's `Date` header. The
body is never trusted on its own but checked against that header, which
Cloudflare stamps at the edge and which is what catches a backend whose clock
is wrong — see [api.md](api.md) for the measurement.
`RealtimeService` resyncs it every 60s (paused in background, resynced on
resume). Read corrected time anywhere via the global `AppTime` facade
(`AppTime.utc` / `AppTime.utc8`, the fixed-offset Taipei wall clock) — never
Expand Down
314 changes: 187 additions & 127 deletions api.md

Large diffs are not rendered by default.

41 changes: 24 additions & 17 deletions ios/DPIPWidgets/WidgetHTTPTimeSource.swift
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,9 @@ enum WidgetHTTPTimeError: Error, Equatable {
case responseTooLarge
}

/// `WidgetServerTimeSource` over `https://api.lb.exptech.dev/ntp`, for networks
/// that block SNTP's UDP/123 — corporate Wi-Fi, hotel and campus networks, some
/// carriers.
/// `WidgetServerTimeSource` over `https://api.lb-<region>.exptech.dev/ntp`, for
/// networks that block SNTP's UDP/123 — corporate Wi-Fi, hotel and campus
/// networks, some carriers.
///
/// Strictly a fallback behind `WidgetSNTPClient`: one HTTPS round trip resolves
/// "now" to roughly ±RTT/2 (±150 ms measured), where the SNTP exchange resolves
Expand All @@ -39,31 +39,38 @@ enum WidgetHTTPTimeError: Error, Equatable {
/// ## Why the `Date` header is checked
///
/// The body is the *origin's* clock, and not every origin's clock is right: of
/// the four `lb-*` nodes behind this name, two were 15.7 seconds behind at the
/// time of writing — consistently, and in agreement with each other, so a
/// quorum across them would not have caught it either.
/// the four `lb-*` nodes, two were 15.7 seconds behind at the time of writing —
/// consistently, and in agreement with each other, so a quorum across them would
/// not have caught it either.
///
/// What does catch it is the `Date` header of the very same response. This host
/// sits behind Cloudflare, so `Date` is stamped at the edge, independently of
/// whichever backend answered — which makes every response self-checking: a body
/// more than `maximumDateSkew` from its own `Date` is a bad backend (or a frozen
/// cache), and is discarded without needing to know which node it came from.
/// What does catch it is the `Date` header of the very same response. Every
/// `lb-*` name sits behind Cloudflare, so `Date` is stamped at the edge,
/// independently of whichever backend answered — which makes every response
/// self-checking: a body more than `maximumDateSkew` from its own `Date` is a bad
/// backend (or a frozen cache), and is discarded without needing to know which
/// node it came from.
///
/// Every guard fails closed: installing a wrong offset is worse than not
/// calibrating, so a rejected response throws and `WidgetServerClock` keeps its
/// last anchor.
struct WidgetHTTPTimeSource: WidgetServerTimeSource {
/// The DNS-balanced LB name — the one place the repository uses a bare
/// host.
/// The region-pinned LB names, `tpe1` first.
///
/// Region-pinned API traffic deliberately never does, because region
/// selection and failover belong to the app. A clock reading has no region
/// to select, and the bare name is what Cloudflare terminates with a valid
/// These used to be the DNS-balanced bare `api.lb.exptech.dev`, on the
/// reasoning that a clock reading has no region to select. The bare name is
/// balanced without regard for which region the app's other traffic picked,
/// so the sample could come from a machine whose clock disagrees with the
/// edge the app is talking to. The widget extension has no region setting
/// to read, so it takes the app's default order: `tpe1` first, `khh1` as the
/// fallback. The bare name is what Cloudflare terminates with a valid
/// certificate — `time.exptech.com.tw`, SNTP's own primary, serves `/ntp`
/// too but presents an expired certificate for the wrong name, and its
/// `Date` comes from the same machine as its body, so it can neither be
/// reached over HTTPS nor check itself.
static let defaultHosts = ["api.lb.exptech.dev"]
static let defaultHosts = [
"api.lb-tpe1.exptech.dev",
"api.lb-khh1.exptech.dev",
]

/// How far the body may sit from its own `Date` header.
///
Expand Down
16 changes: 10 additions & 6 deletions ios/RunnerTests/WidgetHTTPTimeSourceTests.swift
Original file line number Diff line number Diff line change
Expand Up @@ -225,18 +225,22 @@ final class WidgetHTTPTimeSourceTests: XCTestCase {
)
}

func testAsksTheDNSBalancedLBNameOverHTTPS() {
// The one bare host the repository uses: a clock reading has no region
// to pin, and this is the name Cloudflare terminates with a valid cert.
func testAsksTheRegionPinnedLBNamesOverHTTPS() {
// No bare host: the reading comes from the same edge the app talks to,
// and the widget has no region setting, so it takes the app's default
// order with the other region as the fallback.
XCTAssertEqual(
WidgetHTTPTimeSource.defaultHosts,
["api.lb.exptech.dev"]
[
"api.lb-tpe1.exptech.dev",
"api.lb-khh1.exptech.dev",
]
)
XCTAssertEqual(
WidgetHTTPTimeSource.url(
host: "api.lb.exptech.dev"
host: WidgetHTTPTimeSource.defaultHosts[0]
)?.absoluteString,
"https://api.lb.exptech.dev/ntp"
"https://api.lb-tpe1.exptech.dev/ntp"
)
}

Expand Down
14 changes: 12 additions & 2 deletions lib/bootstrap.dart
Original file line number Diff line number Diff line change
Expand Up @@ -285,7 +285,12 @@ Future<void> bootstrap() async {
// one query and no asset reads. The tile cache holds its first answers until
// this lands, so a fresh install offline still draws.
if (cache != null) {
mapTileCache?.servesAfter(BasemapSeeder(cache.etag).run());
// Seeded under the region-pinned template the style asks for: the tiles
// are the same everywhere, but a row keyed by another host would never be
// read, and a first launch offline would draw nothing.
mapTileCache?.servesAfter(
BasemapSeeder(cache.etag, region: regions.lb).run(),
);
}
// Turn off the OS-level disk HTTP cache (iOS NSURLCache) and drop its
// residue: every cached byte now lives in the app's own SQLite, so a second
Expand All @@ -311,7 +316,12 @@ Future<void> bootstrap() async {
final serverClock = ServerClock(
const SystemClock(),
SystemElapsed(),
FallbackServerTimeSource([NtpTimeSource(), HttpTimeSource()]),
FallbackServerTimeSource([
NtpTimeSource(),
// The HTTP fallback asks the region the user picked, its own region
// first — see [HttpTimeSource.hostsFor].
HttpTimeSource(hosts: HttpTimeSource.hostsFor(regions.lbOrder)),
]),
);
AppTime.install(serverClock);
AppTime.sync().ignore();
Expand Down
8 changes: 7 additions & 1 deletion lib/core/geo/location_api.dart
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,13 @@ import 'package:dpip/core/network/api_region.dart';
/// Device-location endpoint on the region-aware [ApiClient].
///
/// Migrated from the legacy `api-1` host to `api.core-tnn1`
/// ([ApiTier.coreExclusiveApi]).
/// ([ApiTier.coreExclusiveApi]). tnn1 is the **primary**: the tyo1 copy is a hot
/// standby, so a healthy tyo1 answers `307` back to tnn1 rather than serving the
/// write itself — it takes the traffic only when the primary is gone. That is
/// why the tier stays exclusive instead of becoming region-redundant: there is
/// no second origin to fail over to, only a redirect that names the first one.
/// Dio follows it (the default `followRedirects`), and the redirect never
/// reaches `validateStatus`, which accepts 2xx and 304 only.
class LocationApi {
const LocationApi(this._client);

Expand Down
6 changes: 3 additions & 3 deletions lib/core/network/api_client.dart
Original file line number Diff line number Diff line change
Expand Up @@ -77,9 +77,9 @@ class ApiClient {

/// Absolute-URL GET as bytes with ETag (no region failover).
///
/// For hosts already baked into the MapLibre style (basemap
/// `static.lb.exptech.dev`, glyphs CDN) that are not an [ApiTier]. Prefer
/// [getBytes] for region-pinned ExpTech paths.
/// For hosts already baked into the MapLibre style (the region-pinned basemap
/// host from `staticLbOrigin`, the glyphs CDN) that are not an [ApiTier].
/// Prefer [getBytes] for region-pinned ExpTech paths.
Future<BytePayload> getBytesAbsolute(
String url, {
CancelToken? cancelToken,
Expand Down
30 changes: 21 additions & 9 deletions lib/core/network/api_region.dart
Original file line number Diff line number Diff line change
Expand Up @@ -38,19 +38,31 @@ enum CoreRegion {
final String label;
}

/// The origin for the region-pinned LB static host — `static.lb-<code>`.
///
/// Not an [ApiTier]: nothing here goes through `ApiClient`, because these are
/// tiles MapLibre fetches itself. What is left is the pinning — basemap, terrain
/// and OSM used to come from the bare `static.lb.exptech.dev`, which balances
/// without regard for the region the rest of the app selected, so a client could
/// draw every tile from the far end while its API traffic came from the near
/// one. Both hosts carry the same dataset; the pin is about one honest
/// selection, not about which copy exists.
String staticLbOrigin(LbRegion region) =>
'https://static.lb-${region.code}.exptech.dev';

/// The host family and redundancy a request targets.
///
/// Only concrete, region-pinned hosts are ever used; the DNS-balanced bare
/// hosts (`api.lb.exptech.dev`, `api.core.exptech.dev`) are deliberately never
/// requested **for tiered traffic**, so region selection and failover stay
/// under app control.
/// hosts (`api.lb.exptech.dev`, `api.core.exptech.dev`) are never requested, so
/// region selection and failover stay under app control.
///
/// One request outside this enum does use a bare host: the HTTP time fallback
/// in `core/realtime/http_time_source.dart` asks
/// `https://api.lb.exptech.dev/ntp`. It is not an [ApiTier] request at all —
/// a clock reading has no region to select, so there is nothing for the app to
/// pin and no per-region failover to preserve, and the bare name is the one
/// Cloudflare terminates with a valid certificate.
/// Two requests fall outside this enum, and both are pinned by region too: the
/// map host from [staticLbOrigin], and the HTTP time fallback in
/// `core/realtime/http_time_source.dart`, which builds its hosts from the
/// selected region's order. Neither is an [ApiTier] request — a clock reading
/// and a MapLibre tile have no `ApiClient` between them and the network — but
/// neither goes to a bare name either, because a bare name is balanced without
/// regard for the region the rest of the app selected.
enum ApiTier {
/// LB API — multi-active across [LbRegion]s (`api.lb-{tpe1,khh1}`).
lbApi,
Expand Down
7 changes: 7 additions & 0 deletions lib/core/network/dio_client.dart
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,13 @@ Dio createDio({EtagCacheStore? etagCache, NetworkUsageStore? usage}) {
// If-None-Match, so this is inert without a cache.
validateStatus: (status) =>
status != null && ((status >= 200 && status < 300) || status == 304),
// Redirects are followed by the adapter (`followRedirects` defaults to
// true, `maxRedirects` to 5), which matters for one endpoint family:
// notify/location are primary-on-tnn1 with a hot standby, and a standby
// that is up while the primary is not answers `307` at the *interim*
// node, not at tnn1. Following it means the write lands on tnn1, where it
// belongs. A 3xx never reaches [validateStatus] — by the time the status
// is checked the chain has already settled on its final response.
),
);
if (etagCache != null) {
Expand Down
21 changes: 16 additions & 5 deletions lib/core/network/endpoint_health.dart
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,9 @@
///
/// Outcomes are bucketed by **service × tier × host**: the same region carries
/// different services (EEW/RTS on `lbApi` — one shared stream feeding both
/// rows, see [EndpointService.allOfPath] — radar lists on
/// `coreExclusiveApi` → `api.core-tnn1`), and one service's dead host may be
/// another's healthy one. The More → 伺服器狀態 screen renders this map as a
/// rows, see [EndpointService.allOfPath] — radar lists on `coreApi` and radar
/// tiles on `coreStatic`), and one service's dead host may be another's
/// healthy one. The More → 伺服器狀態 screen renders this map as a
/// table — rows are services, columns are tier groups, cells are the regions
/// each service was observed on.
library;
Expand Down Expand Up @@ -45,7 +45,12 @@ enum EndpointService {
/// path after `/api/` decides; the families share the `ApiPaths` constants.
static EndpointService ofPath(String path) {
if (path.startsWith('/api/v2/eq/eew')) return eew;
if (path.startsWith('/api/v2/trem/rts')) return rts;
// Both the live v2 topic and the v3 archive are the same family: the
// status table should not care which generation a caller used.
if (path.startsWith('/api/v3/trem/rts') ||
path.startsWith('/api/v2/trem/rts')) {
return rts;
}
if (path.contains('/tiles/radar')) return radar;
if (path.contains('/tiles/satellite')) return satellite;
if (path.contains('/tiles/qpesums')) return qpesums;
Expand All @@ -58,7 +63,13 @@ enum EndpointService {
if (path.startsWith('/api/v5/meteor/lightning')) return lightning;
if (path.startsWith('/api/v5/meteor/typhoon')) return typhoon;
if (path.startsWith('/api/v2/eq/report')) return report;
if (path.startsWith('/api/v1/trem/')) return tremStation;
// The station directory is a static CSV (`/resource/station`); the
// legacy `/api/v1/trem/*` spelling is kept only so old call sites still
// land on the same row.
if (path.startsWith('/resource/station') ||
path.startsWith('/api/v1/trem/')) {
return tremStation;
}
if (path.startsWith('/api/v1/dpip')) return event;
if (path.startsWith('/api/v2/location')) return location;
if (path.startsWith('/api/v2/notify')) return notify;
Expand Down
12 changes: 10 additions & 2 deletions lib/core/network/etag_interceptor.dart
Original file line number Diff line number Diff line change
Expand Up @@ -90,9 +90,17 @@ class EtagInterceptor extends Interceptor {
static bool _isBytes(RequestOptions o) =>
o.responseType == ResponseType.bytes;

/// Bare-host static map vector tiles (no server ETag).
/// Region-pinned static map vector tiles (no server ETag).
///
/// The basemap/OSM copy is pinned to one LB host per session —
/// `static.lb-<region>.exptech.dev` — so this matches the family, not one
/// name. The bare `static.lb.exptech.dev` is still accepted: cached tile
/// entries written before the pin hold those URLs, and a store lookup has to
/// recognise what is already in it.
static bool isBasemapPbf(Uri uri) =>
uri.host == 'static.lb.exptech.dev' &&
(uri.host == 'static.lb.exptech.dev' ||
(uri.host.startsWith('static.lb-') &&
uri.host.endsWith('.exptech.dev'))) &&
(uri.path.contains(ApiPaths.mapTilesV1) ||
uri.path.contains(ApiPaths.mapOsmV1)) &&
uri.path.endsWith('.pbf');
Expand Down
Loading
Loading