Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions research/finance_program_v1/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,3 +58,12 @@
这是开发验证集、单 seed,仅 8/12 类有支持,且来源可能共享表;不是独立 Test 或跨领域泛化结论。教育与钢铁数据集不在本轮结果内。该更新只替换结果展示和学生公开轨迹,不修改实验代码、权重或训练设置。

真实数据集、gold、权重、凭证和实验服务器地址均不上传。clone/import 不会启动任何训练、GPU 或 API 作业。复现说明见 [REPRODUCE.md](REPRODUCE.md)。

## 发布依赖修复(2026-10-19)

补回历史批次审计模块与源代码/官方模型哈希清单、教师录制调用回放助手,以及评估旧后端的SHA清单工具。未改变模型、实验协议、训练参数、结果数字或公开轨迹。

`pool_metadata.py` 位于 `nontraining/vendor/` 和 `evaluation/runtime/vendor/`,不是仓库根目录模块。通过 `program_loop` 导入时会初始化对应的vendor路径;若只使用该工具,请按 [REPRODUCE.md](REPRODUCE.md) 设置明确的导入路径。新增隔离进程回归覆盖批次入口、完整合成回放和请求篡改拒绝。

**代码依赖补齐不等于发布了私有数据/历史执行目录。** 历史 `teacher/collect.py`、`teacher/detached.py`、评估watcher/legacy环境身份仍要求经授权的原始配置、manifest、ledger或数据;不得直接运行它们来启动付费请求/GPU。公开CPU测试不需要这些私有资产。

29 changes: 29 additions & 0 deletions research/finance_program_v1/REPRODUCE.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,3 +44,32 @@ The prior partial-validation result tables and standalone partial trajectory fil
## Interpretation

Val422 is a development set with eight supported classes out of twelve, one seed and potentially correlated table sources. It is not held-out Test or independent generalization. Supported-class and fixed-twelve-class Macro-F1 are reported separately; legal termination is not business correctness. Historical non-training interfaces are not interchangeable with this matched F66 comparison.

## Publication dependency repair (2026-10-19)

The public nontraining audit now includes `reviews/independent_audit_01.py`, its exact historical source SHA inventory, the public official-model file-hash inventory, and the two omitted source files needed by that inventory. This preserves the historical 121-file/109-core-file signature; it does not fabricate a historical run or ship model weights. `batch_audit.py` imports and a real-controller recorded-I/O replay with synthetic source data are covered by fresh isolated-process tests, including tampered-request rejection.

Additional restored code: `teacher/pilot_audit.py` contains only the recorded retry/I-O replay helpers used by `detached.py`, not the private-gold historical quality aggregator. The public teacher runtime resolves to the already included `evaluation/runtime` when no controlled local `teacher/runtime` is present. `evaluation/legacy/tools/freeze_revision.py` restores the original pure SHA inventory helper, not the private revision manifests or a new deployment authorization. Legacy environment-identity functions still require their controlled snapshot layout; this code snapshot is not a turnkey historical deployment.

### Import paths and CPU checks

From `research/finance_program_v1/nontraining`:

```powershell
python -B -c "import batch_audit; import program_loop; import pool_metadata; print(pool_metadata.__file__)"
```

If using only the standalone metadata helper in your own Python program, add the explicit snapshot directory before importing it:

```python
import sys
from pathlib import Path
snapshot = Path("research/finance_program_v1/nontraining").resolve()
sys.path.insert(0, str(snapshot / "vendor"))
from pool_metadata import PoolMetadata
```

Do not copy files between different historical snapshots or rely on another checkout's PYTHONPATH. Run the teacher synthetic suite from `teacher/` with `python -B -m pytest tests -q -p no:cacheprovider`. Run the publication checks from this directory with `python -B -m unittest discover -s publication_tests -v`; dependency checks use `-I` fresh processes so private/local import paths cannot mask missing modules.

`python -B -m reviews.independent_audit_01 --help` and `python -B batch_audit.py --help` are safe CPU-only entrypoint checks. Actual audit commands need complete, authorized, unredacted recorded runs and the matching study/model manifests; redacted public trajectories cannot substitute for them. Cloning/importing/testing does not authorize or start new model downloads, API calls, training, or GPU work.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
"""Historical SHA-inventory helpers; private execution assets are not included."""
177 changes: 177 additions & 0 deletions research/finance_program_v1/evaluation/legacy/tools/freeze_revision.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,177 @@
"""Pure SHA inventories. No model acceptance, transport, launch, or approval.

Create: python -B -m tools.freeze_revision --root . --output reviews/revision-identity
Verify: python -B -m tools.freeze_revision --root . --output reviews/revision-identity --verify
The output directory is created exclusively; a failed/old freeze is never reused.
"""
import argparse
import hashlib
import json
from pathlib import Path, PurePosixPath
import re

ROOT = Path(__file__).resolve().parents[1]
DEFAULT_IDENTITY = ROOT / 'reviews/revision-identity'
SOURCE_DIRS = ('program_loop', 'vendor', 'tools', 'tests', 'tests_batch')
CORE_DIRS = ('program_loop', 'vendor', 'tools')
SOURCE_FILES = ('reviews/independent_audit_01.py', 'reviews/remote_gate.py')
EXECUTION_FILES = ('data/study.json', 'data/study-manifest.json', 'reviews/trusted-model-manifest-01.json')
BASELINE_FILES = {'source_manifest_sha256': 'reviews/baseline-source-manifest-01.json',
'execution_manifest_sha256': 'reviews/baseline-execution-manifest-01.json'}


def sha(path):
return hashlib.sha256(Path(path).read_bytes()).hexdigest()


def signature(entries):
return hashlib.sha256(''.join(f"{e['path']}\t{e['bytes']}\t{e['sha256']}\n"
for e in sorted(entries, key=lambda e: e['path'])).encode()).hexdigest()


def safe_path(root, name):
if (not isinstance(name, str) or not name or '\\' in name or ':' in name or
PurePosixPath(name).is_absolute() or PurePosixPath(name).as_posix() != name or
any(p in ('', '.', '..') or p.endswith((' ', '.')) for p in name.split('/')) or
any(ord(c) < 32 for c in name)):
raise ValueError(f'unsafe inventory path: {name!r}')
path = root
for part in name.split('/'):
path = path / part
if path.is_symlink() or (hasattr(path, 'is_junction') and path.is_junction()):
raise ValueError(f'symlink inventory path: {name}')
# Canonical relative components and the ancestor link checks above prevent
# escaping root without re-resolving the full Windows path for every file.
return path


def inventory(root, names):
entries = []
seen = set()
for name in sorted(names):
path = safe_path(root, name)
if name.casefold() in seen:
raise ValueError('duplicate inventory path')
seen.add(name.casefold())
if not path.is_file():
raise ValueError(f'missing inventory file: {name}')
entries.append({'path': name, 'bytes': path.stat().st_size, 'sha256': sha(path)})
return entries


def source_names(root):
names = [p.name for p in root.glob('*.py')]
def walk(directory):
for path in directory.iterdir():
relative = path.relative_to(root).as_posix()
safe_path(root, relative)
if path.name == '__pycache__':
continue
if path.is_dir():
walk(path)
elif path.is_file():
names.append(relative)
else:
raise ValueError(f'nonregular inventory entry: {relative}')
for name in SOURCE_DIRS:
directory = safe_path(root, name)
if not directory.is_dir():
raise ValueError(f'missing source directory: {name}')
walk(directory)
return names + list(SOURCE_FILES)


def manifests(root):
root = Path(root)
if root.is_symlink() or (hasattr(root, 'is_junction') and root.is_junction()):
raise ValueError('symlink root')
provenance = {key: sha(safe_path(root, name)) for key, name in BASELINE_FILES.items()}
source = inventory(root, source_names(root))
execution = sorted(source + inventory(root, EXECUTION_FILES), key=lambda e: e['path'])
shared = {'schema': 1, 'baseline_provenance': provenance}
return ({**shared, 'kind': 'source', 'files': source, 'signature_sha256': signature(source)},
{**shared, 'kind': 'execution', 'files': execution, 'signature_sha256': signature(execution),
'source_signature_sha256': signature(source), 'study_sha256': sha(safe_path(root, 'data/study.json'))})


def environment_identity(root=ROOT):
source, execution = manifests(root)
return identity_from_manifests(source, execution)


def identity_from_manifests(source, execution):
return {'revision_identity': {'schema': 1, 'source_signature_sha256': source['signature_sha256'],
'execution_signature_sha256': execution['signature_sha256'],
'study_sha256': execution['study_sha256'], 'baseline_provenance': source['baseline_provenance']},
'core_source_sha256': {e['path']: e['sha256'] for e in source['files'] if e['path'].split('/')[0] in CORE_DIRS}}


def check_entries(root, manifest):
entries = manifest.get('files')
if not isinstance(entries, list) or not entries:
raise ValueError('empty or invalid manifest inventory')
for entry in entries:
if (not isinstance(entry, dict) or set(entry) != {'path', 'bytes', 'sha256'} or
type(entry['bytes']) is not int or entry['bytes'] < 0 or
not isinstance(entry['sha256'], str) or not re.fullmatch('[0-9a-f]{64}', entry['sha256'])):
raise ValueError('invalid inventory entry')
actual = inventory(root, [e['path'] for e in entries])
if actual != entries or signature(entries) != manifest.get('signature_sha256'):
raise ValueError('inventory bytes/hash/order/signature mismatch')


def no_duplicate_keys(pairs):
result = {}
for key, value in pairs:
if key in result:
raise ValueError('duplicate manifest key')
result[key] = value
return result


def verify_revision(root=ROOT, output=None):
root = Path(root); output = Path(output) if output is not None else DEFAULT_IDENTITY
expected = manifests(root)
for name, wanted in zip(('source.json', 'execution.json'), expected):
# Also reject linked manifest directories, not just linked final files.
for parent in (output, *output.parents):
if parent.is_symlink() or (hasattr(parent, 'is_junction') and parent.is_junction()):
raise ValueError('symlink manifest directory')
path = safe_path(output, name)
actual = json.loads(path.read_text(encoding='utf-8'), object_pairs_hook=no_duplicate_keys)
check_entries(root, actual)
if actual != wanted:
raise ValueError(f'exact {name} inventory or provenance mismatch')
return identity_from_manifests(*expected)


def freeze(root=ROOT, output=DEFAULT_IDENTITY):
root = Path(root); output = Path(output)
source, execution = manifests(root)
for parent in (output, *output.parents):
if parent.is_symlink() or (hasattr(parent, 'is_junction') and parent.is_junction()):
raise ValueError('symlink output path')
relative = output.resolve().relative_to(root.resolve()) if output.resolve().is_relative_to(root.resolve()) else None
if relative and (relative.parts[0] in SOURCE_DIRS or relative.as_posix() in SOURCE_FILES + EXECUTION_FILES):
raise ValueError('output would enter its own inventory')
output.mkdir() # Parent must exist; never merge or overwrite another revision.
for name, value in (('source.json', source), ('execution.json', execution)):
with (output / name).open('x', encoding='utf-8') as stream:
json.dump(value, stream, ensure_ascii=False, indent=2, allow_nan=False)
stream.write('\n')
return verify_revision(root, output)


def main(argv=None):
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--root', type=Path, default=ROOT)
parser.add_argument('--output', type=Path, required=True)
parser.add_argument('--verify', action='store_true')
args = parser.parse_args(argv)
result = verify_revision(args.root, args.output) if args.verify else freeze(args.root, args.output)
print(json.dumps(result, ensure_ascii=False, sort_keys=True))
return 0


if __name__ == '__main__':
raise SystemExit(main())
24 changes: 24 additions & 0 deletions research/finance_program_v1/nontraining/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# Program 动态引用与事实性恢复(2026-10-11)

这是用户批准的独立新版本,基于上一份115项冻结源码。历史实验及失败结果只读,不替换历史身份。本轮只加强显式动态参数引用约束,以及Program结束后的事实性恢复上下文。

Planner 优先使用 `arguments:{}`,由原binder补齐required参数;optional省略仍不注入。显式动态参数(canonical reference以`$state.`开头)必须是确切规范引用,不得手写对象、列表、null、数字或普通字符串。初始domain/field_name等保持原契约。无效计划在执行前拒绝,只计Planner尝试预算,保留原文和具体错误。

模型仍从全部 14 个可选 Skill 中自行选择步骤和顺序,每次 Planner 调用生成一个完整有序 Program。没有必选 Skill、固定完整流程示例、自动插入步骤或改写模型答案。可能生产者只表示静态可能性;元数据取得、输出发布、状态绑定以及最终终止,都必须由真实执行和冻结 guard 验证。`semantic_query`、`semantic_evidence` 与 `field_semantics` 保持各自契约含义,描述语义不能替代字段名语义。

核心入口文件:

- `program_loop/planning_contracts.py`:根据 catalog 和共享校验规则生成无状态值的规划说明。
- `program_loop/catalog_context.py`:把说明附加到每个 Skill 的原始契约库中,供 Planner 读取。
- `program_loop/planner.py`:说明参数省略、合法引用、依赖顺序和条件性执行的使用规则。
- `program_loop/validation.py`:静态依赖与显式动态引用约束,共享取值策略。
- `program_loop/recovery_context.py`:纯事实性恢复摘要,不执行Skill、不改state。
- `program_loop/controller.py`:Program级执行/反馈与全局预算循环;不在Skill间规划。

恢复消息从真实state与observation陈述实际成功/失败步骤、失败位置、尚未执行的尾部、证据失效与真实剩余预算。尾部是历史意图,不是强制路线;字段存在不等于已验证ready。最后一份Program也仍允许部分计划,代码不自动完成分类或赠送预算。合法终止仍由原terminal guard确认。

新增CPU测试只脚本化模型I/O,parser、binder、executor和guard用真实实现。`vendor/`、底层execution、Registry、P14池及`local_smoke.py`/`download_model.py`字节保持不变。完整变更与未改SHA由新source manifest记录,不以人工描述替代哈希证据。

用户已授权测试/审查门通过后在新远程目录单次运行同三条无标签输入、同9B基础模型、同生成参数和预算。无新adapter/训练、Val调策略、Test、依赖升级或Git提交推送。工程通过不等于模型成功;真实试跑结果及原始调用见交付后的`实验结果.md`与`SMOKE_README.md`,保留全部失败,不调prompt重复挑结果。

RED/GREEN、独立Spec/Quality审查、全CPU验收、完整diff、冻结新签名和证据ZIP在`reviews/`。两项修改同时施加,与历史三条只作描述性比较,不能分别归因或推广为准确率提升。DCG全文输入模式、池规模和训练方式不属本轮改动,不声称已实现导师全部行动指南。
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
"""Public, CPU-only historical audit helpers; no generated private receipts."""
Loading
Loading