Repository navigation
Conversation
新增 swanlab.Histogram 数据类型。 - transforms/histogram:Histogram 转换类,支持原始序列分桶(默认 64 桶、 上限 512 桶)与预分桶结果两种输入,完整校验计数与边界,大整数计数不经 float64 转换以保精度 - transformer:新增 TransformHistogram 基类,区分 histogram 表后端 - protocol/core、core_python:实现 upsert_histograms,列注册含自动建列 与 define_metric 两条路径 - consumer:新增 histogram_batch 批次与 builder 分发注册 - sender/api:编码上传 POST /house/metrics,请求体携带 epoch、_type、 values、bins,重试由 transport 层统一调度 - sync 回放、swanlab.Histogram 导出(含 .pyi)同步支持
- 移除 Histogram 上传载荷和 UploadHistogram 类型中的 epoch 字段,因为发现scalar和media都对其进行了忽略 - 更新 sender 和 HTTP 测试,验证请求不包含 epoch
Nexisato
reviewed
Oct 29, 2026
Nexisato
left a comment
Collaborator
There was a problem hiding this comment.
审核结论:通过(4 条改进建议,不阻塞合并)
本 PR 在 SDK 侧打通 swanlab.Histogram → HistogramRecord → 本地持久化 → 上传的完整链路,实现完整,本地与 CI 均通过;以下建议供参考。
| # | 级别 | 位置 | 问题 | 建议 |
|---|---|---|---|---|
| 1 | 建议 | swanlab/sdk/internal/run/transforms/histogram/__init__.py:90 |
bins 只接受 Python int / list / tuple,np.ndarray、np.int64、range 都会被拒绝;np.int64 的报错为 must be an int ... got int64,信息自相矛盾。同时 np_histogram 的边界接受 ndarray,同一参数的两条输入通路行为不一致。 |
整数判断改用 numbers.Integral,边界统一交给 _validate_edges 处理;同步调整 test_bins_wrong_type 中把 np.array 视为非法类型的断言。 |
| 2 | 建议 | swanlab/sdk/internal/run/transforms/histogram/__init__.py:121 |
sequence 为 requires_grad=True 的 torch.Tensor 时,np.asarray 会抛 RuntimeError;Histogram(...) 在 log() 之前构造,不在 consumer 的 safe.block 保护范围内,会直接中断训练。 |
对带 requires_grad 的对象先 detach(),或在 docstring 中明确只支持无梯度张量。 |
| 3 | 建议 | swanlab/sdk/internal/core_python/sync.py:281 |
新增的 _handle_histogram 没有测试,而它是 swanlab sync 离线回放的唯一入口,逻辑与 _handle_metric 平行复制。 |
参照 tests/unit/sdk/internal/core_python/test_core_sync.py 中 media 的用例补一条 histogram 版本,覆盖自动补列与 step 过滤。 |
| 4 | 建议 | swanlab/sdk/internal/core_python/transport/sender/__init__.py:248 |
512 桶时单条 histogram 载荷约 20KB,而 record_batch 默认 10000,理论上一轮 flush 可拼出约 200MB 的单请求。 |
为 histogram 单独设置较小的批次上限,或按序列化字节数切批。 |
验证
uv run pytest -q:2109 passed, 21 skippeduv run ruff check .:通过uv run basedpyright:0 errors, 0 warnings- PR Actions:19 个 job 通过,1 个 skipping
Collaborator
|
另外还有几项需要等待后端上线后再实现:
|
Nexisato
requested changes
Oct 29, 2026
Nexisato
marked this pull request as draft
October 29, 2026 08:07
- 将 Histogram 参数类型定义移至 typings - 支持 NumPy 整数桶数及 ndarray、range 边界 - 补充 Tensor 需先执行 detach().cpu().numpy() 的说明 - 限制每批最多上传 512 条 Histogram,重试时排除已成功批次 - 补充参数校验、上传分批和失败重试测试
- 验证 _handle_histogram 的回放处理逻辑 - 补充 Histogram 回放时自动建列和重复 step 去重测试 - 覆盖 step 和时间范围过滤,以及全部被过滤时不创建列的行为 - 使用真实 RunMetrics 验证指标注册与 step 状态
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
概要
在 SDK 侧打通 swanlab.Histogram 的完整链路:用户在 log() 里传入 Histogram 对象,后台线程分批转换为 HistogramRecord,本地持久化后上传。proto 定义见 #1808。
数据流
变更内容
swanlab/sdk/internal/run/transforms/histogram/swanlab/sdk/internal/context/transformer.pyswanlab/sdk/protocol/core.py、core_python/core.pyswanlab/sdk/internal/core_python/metrics/swanlab/sdk/internal/core_python/pkg/builder/swanlab/sdk/internal/run/components/consumer/swanlab/sdk/internal/core_python/transport/sender/swanlab/sdk/internal/core_python/api/upload.py、typings DTOswanlab/sdk/internal/core_python/sync.pyswanlab/__init__.py等验证