Skip to content

feat: add database healthcheck callback - #39

Open
goshander wants to merge 5 commits into
mainfrom
dlops-619-healthcheck-status-callback
Open

feat: add database healthcheck callback#39
goshander wants to merge 5 commits into
mainfrom
dlops-619-healthcheck-status-callback

Conversation

@goshander

@goshander goshander commented Sep 11, 2026

Copy link
Copy Markdown
Member

Summary

  • expose an onHealthcheck callback through initDB
  • report a typed snapshot for every configured host after each healthcheck
  • support both primary-replica and proxy topology modes
  • report unavailable primary-replica hosts with the unknown role
  • invoke the callback independently of suppressStatusLogs
  • keep the callback synchronous and isolate its failures from database routing
  • stop callback notifications when dispatcher termination starts
  • preserve the existing Database current status log payload

This API lets consumers publish host availability, role, and latency metrics without parsing log records or running a second healthcheck loop.

Tests

  • npm run build
  • npm run test:unit
  • npm run lint

The full npm test command completed the package and demo builds plus unit tests, but the integration suite could not start locally because /var/run/docker.sock is unavailable. CI is expected to run that suite.

@jhoncool

Copy link
Copy Markdown
Collaborator

@goshander pls use commit message conventions

@goshander
goshander marked this pull request as draft September 11, 2026 13:19
@goshander goshander changed the title DLOPS-619: add database healthcheck callback feat: add database healthcheck callback Sep 11, 2026
Comment thread lib/dispatcher.ts Outdated
topologyMode: 'primary-replica',
connections: this.connections.map((connection) => ({
host: connection.host,
primary: connection.primary,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Preserve an unknown role for failed checks

When a primary becomes unreachable, checkDatabase sets primary = false, and this public snapshot emits it as a non-primary host. In primary-replica mode, consumers have no state other than false, so availability metrics will misclassify an unreachable primary as a replica during the outage. Please model the role as primary | replica | unknown (or expose a separate last-known role) instead of collapsing a failed role check to false.

AI generated

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 7e244d1. The callback snapshot now exposes role as primary, replica, or unknown; unavailable hosts use unknown. The existing status log keeps its primary boolean for backward compatibility.

Comment thread lib/dispatcher.ts
}),
);
Promise.all(checkups).then(() => {
const status = this.getHealthcheckStatus();

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Build the snapshot from one health-check cycle

Health-check cycles can overlap when healthcheckInterval is shorter than a check. Each cycle mutates the shared this.connections, while getHealthcheckStatus() reads that shared state only after its own Promise.all completes. A slower older cycle can therefore publish a mixture of its own results and values written by a newer cycle. Please serialize cycles or build the snapshot from per-cycle local results.

AI generated

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The callback intentionally mirrors the existing Database current status semantics and snapshots shared current connection state rather than cycle-local results. I added a short comment to make this explicit. Serializing healthcheck cycles would change existing dispatcher behavior and is better handled separately.

Comment thread lib/dispatcher.ts
connections: status.connections,
},
});
this.notifyHealthcheck(status);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Do not notify after terminate() has completed

terminate() clears the interval but does not cancel or await an already-running health check. Its Promise.all can finish after teardown and call onHealthcheck after terminate() has resolved, when the consumer may already have closed its exporter or registry. Please track the lifecycle and either await in-flight work or suppress notifications once termination starts.

AI generated

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 7e244d1. Termination now marks the dispatcher before clearing its timer, prevents initialization from starting a healthcheck after termination, and suppresses callback notification from an in-flight check.

Comment thread lib/dispatcher.ts Outdated
}

try {
Promise.resolve(this.onHealthcheck(status)).catch((error) => {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Bound concurrent async callbacks

PGHealthcheckHandler may return a Promise, but that Promise is only given a rejection handler; the next interval starts another callback regardless of whether the previous one is still pending. A slow or stuck metrics exporter can therefore accumulate unbounded concurrent operations. Please serialize, coalesce, or otherwise bound callback execution while keeping database routing independent.

AI generated

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 7e244d1 by making the callback synchronous and removing Promise handling. Callback exceptions remain isolated from database routing.

@goshander
goshander marked this pull request as ready for review September 11, 2026 13:55
@goshander
goshander requested a review from jhoncool September 11, 2026 17:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants