Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,7 @@ Encrypted linear head test acc: 0.492

## 📚 Documentation
- [roadmap.md](roadmap.md) – Project roadmap and development phases
- [Research archive (10 October 2026)](docs/research/2026-10-10/README.md) – 2025–2026 literature findings, numerical results, and a proposed roadmap review
- [README_INSTRUCTION.md](README_INSTRUCTION.md) – Detailed PoC documentation
- [CONTRIBUTING.md](CONTRIBUTING.md) – Contribution guidelines
- [CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md) – Community guidelines
Expand Down
82 changes: 82 additions & 0 deletions docs/research/2026-10-10/DESIGN-audit.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
# DESIGN: source audit for PrivNetAI

Review date: 2026-10-10. This is a literature and static source review, not a reproduced benchmark, security proof, or code execution audit. No PrivNetAI source was changed.

## Verified identity and publication status

- Paper: **DESIGN: Encrypted GNN Inference via Server-Side Input Graph Pruning**, Kaixiang Zhao, Joseph Yousry Attalla, Qian Lou, and Yushun Dong.
- [Original arXiv record](https://arxiv.org/abs/2507.05649): first submitted **8 July 2025**, revised **14 July 2025**, current accessible version **v2**. The arXiv comment says “Under Review in Conference on Neural Information Processing Systems (NeurIPS 2025).” This does not establish conference acceptance.
- [Hugging Face paper page](https://huggingface.co/papers/2507.05649) displays “Published on Jul 14, 2025,” corresponding to the revision, and links the author repository. Its presence is a paper listing; the page does not provide a model, dataset, or Space implementing the method.
- [Author GitHub repository](https://github.com/LabRAI/DESIGN), inspected at commit `859225550559b96dc025b7ceaa5819c919da1c5e`. The repository README also identifies the work as NeurIPS 2025 under review.
- The original paper, Hugging Face page and pinned author source were inspected on the review date. Persistent primary-source and commit-pinned code links are provided below; downloaded source snapshots are omitted from this archive.

## What the authors propose

The paper targets **inference**, with a model trained in plaintext. A client supplies an encrypted adjacency matrix and encrypted node features; the server computes encrypted node degrees, compares them homomorphically with thresholds, and forms encrypted importance masks. It then masks less important nodes and incident edges and combines polynomial activations of different degrees using encrypted level masks. The client decrypts the result. The appendix specifies a two-layer GCN with only **two hidden channels**, initially trained with a quadratic activation.

This is directly relevant prior art for graph-private HE inference and degree-guided adaptive polynomial execution. It does not demonstrate encrypted training, post-quantum transport/authentication, differential privacy of outputs, or verified correctness against a malicious server.

## Strengths and useful ideas

1. It treats graph topology as confidential rather than limiting encryption to a final classifier.
2. Degree-based importance is cheaper to compute than many richer encrypted graph statistics.
3. It explicitly acknowledges the cost and approximation errors of encrypted comparisons.
4. It releases inspectable source and describes ablations and polynomial-degree tradeoffs.

These are research ideas and author-reported evaluations, not independently established performance for our application.

## Central efficiency issue: encrypted masking does not itself skip work

The paper explicitly describes **logical** pruning. Equation (3) multiplies features and adjacency entries by encrypted keep masks. Equation (5) evaluates multiple activation polynomials and combines them with encrypted importance masks.

The pinned implementation confirms the fixed-size computation:

- [`design_model.py`, lines 254–269](https://github.com/LabRAI/DESIGN/blob/859225550559b96dc025b7ceaa5819c919da1c5e/design_model.py#L254) encrypts every adjacency diagonal and each feature vector.
- [Lines 394–426](https://github.com/LabRAI/DESIGN/blob/859225550559b96dc025b7ceaa5819c919da1c5e/design_model.py#L394) mask every feature vector and every adjacency diagonal, append every result, and preserve their counts.
- [Lines 272–330](https://github.com/LabRAI/DESIGN/blob/859225550559b96dc025b7ceaa5819c919da1c5e/design_model.py#L272) aggregate over all `self.N` diagonals. This dimension is not reduced after pruning.
- [Lines 458–479](https://github.com/LabRAI/DESIGN/blob/859225550559b96dc025b7ceaa5819c919da1c5e/design_model.py#L458) evaluate **each** importance level's polynomial on the entire packed hidden-feature ciphertext, then multiply by masks and sum. They do not execute only the chosen polynomial for each hidden node.

**Our assessment:** The shown implementation does not physically compact the graph, omit ciphertexts, or provide a public schedule that skips the work represented by encrypted zeros. Its encrypted pruning adds operations, while its adaptive activation computes all polynomial branches before selection. A secret encrypted zero is not a known plaintext zero that an evaluator can exploit to skip an operation. Consequently, the claimed reduction in graph-processing volume and average activation work needs a reproducible explanation beyond the visible masking implementation.

This finding does not prove that every possible encrypted-pruning scheme is inefficient. It means this specific implementation does not establish the proposed mechanism's claimed cost reduction. Efficient compaction, public preprocessing, public workload schedules, or an explicit leakage allowance would need separate specification and measurement.

## Numerical and reproducibility caveats

### Comparison approximation domain

The implementation uses `p(t) = -0.25*t^3 + 0.75*t + 0.5` on degree minus threshold. Its own [lines 198–200](https://github.com/LabRAI/DESIGN/blob/859225550559b96dc025b7ceaa5819c919da1c5e/design_model.py#L198) note that the difference must be approximately in `[-1, 1]` and that normalization may be needed. The inspected path uses raw degrees and thresholds such as `[5.0, 2.0]`; no corresponding normalization is evident. For example, `p(3) = -4`, which cannot act as an approximate binary selection mask. Approximation-domain coverage and resulting mask error must be validated before relying on pruning results.

### Accuracy claims conflict across artifacts

The repository README says “up to 2× latency reduction with ≤2 pp accuracy loss vs. plaintext.” The paper's Table 2 instead reports:

| Dataset | Plaintext | DESIGN | Difference |
|---|---:|---:|---:|
| Cora | 84.00% | 74.00% | −10.00 pp |
| CiteSeer | 60.00% | 45.00% | −15.00 pp |
| PubMed | 80.00% | 55.00% | −25.00 pp |

Report these as conflicting **author claims**, not as a verified ≤2 pp fidelity result. Table 1 also does not show DESIGN winning every comparison: on ogbn-proteins it reports 8.49 s versus OpenFHE's 7.69 s.

### Evaluation scope

- [`utils.py`, lines 70–76](https://github.com/LabRAI/DESIGN/blob/859225550559b96dc025b7ceaa5819c919da1c5e/utils.py#L70) computes classification accuracy over **all nodes**, despite comments elsewhere describing test-mask evaluation. The multi-label metric likewise uses all flattened labels and thresholded predictions.
- [Lines 106–113](https://github.com/LabRAI/DESIGN/blob/859225550559b96dc025b7ceaa5819c919da1c5e/utils.py#L106) subsample PubMed, Yelp, and ogbn-proteins to 4,000 nodes and generate new random splits. This matters when interpreting large-dataset scalability claims.
- The visible code calls `to_dense_adj` without importing it in `utils.py`, and calls `get_A_bar` without importing it in `design_model.py`. These are static reproducibility blockers; we did not install bindings or run/fix the authors' code.
- Code sets ring degree 8,192 with coefficient-modulus bit sizes totaling 230, whereas the paper describes degrees 32,768/65,536. Official SEAL's default classical 128-bit limit at degree 8,192 is 218 bits ([source](https://github.com/microsoft/SEAL/blob/main/native/src/seal/util/hestdparms.h#L29)); its default context requests `tc128` ([source](https://github.com/microsoft/SEAL/blob/main/native/src/seal/context.h#L500)). Parameter validity and the selected Python binding must therefore be checked. A classical 128-bit setting should also not be silently relabeled as 128-bit quantum security.

## Threat-model interpretation

The intended server sees encrypted adjacency, features, scores, and masks and lacks the client's decryption key. Graph dimensions, ciphertext counts, public thresholds, model architecture, execution schedule, and inference output disclosure are not hidden merely by encrypting numerical contents. The demonstration generates a decryptor in the same process for benchmarking; that does not by itself establish or refute an independently deployed client/server boundary. A real implementation must separate client key material from server execution.

The reviewed paper inherits confidentiality claims from CKKS; the material examined does not establish a separate malicious-server correctness protocol, a model-extraction guarantee, or an output differential-privacy guarantee. Any server-visible schedule that changes based on secret pruning decisions needs explicit leakage analysis.

## Implications for PrivNetAI

1. **Cite DESIGN as 2025 prior art** for encrypted degree scoring, graph masking, and adaptive polynomial selection. Do not claim those broad ideas as ours alone.
2. **Reproduce pruning complexity before importing its speedup.** Count ciphertext multiplications, rotations, depth, packing occupancy, transmitted ciphertexts, preprocessing, and elapsed time. Distinguish logical masked gating from a physical reduction of work.
3. Start with a correct complete polynomial GCN under an explicit threat model and identical float-versus-HE architecture. Our fixed-point ablation already shows why head changes and arithmetic errors must be separated from cryptographic costs.
4. If introducing adaptive schedules, define whether topology, retained-node counts, importance assignments, or schedule decisions become public. Compare against fixed-schedule masking and any proposed compaction protocol using the same leakage and security conditions.
5. A promising new research question is **which graph-aware optimizations actually lower observable HE work while preserving a stated leakage policy and classifier fidelity**. That is a proposed question, not an established novelty claim.

Keep equivariant obfuscation literature separate: a secret feature transformation is not automatically semantically secure HE and requires its own attack/threat analysis.
53 changes: 53 additions & 0 deletions docs/research/2026-10-10/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# PrivNetAI research archive — 10 October 2026

This archive preserves the research findings, numerical diagnosis and roadmap proposal reviewed on **2026-10-10** for future work. The repository baseline was [`be0400f8e8d22fcfff2d011bc1c1dd9b7a2f4767`](https://github.com/aliaslany/PrivnetAI/tree/be0400f8e8d22fcfff2d011bc1c1dd9b7a2f4767). The [existing roadmap](../../../roadmap.md) and application code remain unchanged; the proposed milestones below are not implemented or adopted by this archive.

## Findings to carry forward

- Build a matched-model, real-CKKS inference baseline before combining encrypted training, private topology and verifiable computation. Specify the permitted disclosures, output recipients and security parameters first.
- Separate model-adaptation loss from arithmetic error. The current CLI compares a complete GCN with a different ridge classifier, and its simulator rounds weights without a weight scale.
- Compare with recent work on sparse graph packing, interactive protocols, compiler layouts and bounded activation ranges. The review covers 2025–2026 papers, GitHub artifacts and targeted Hugging Face searches; it is not exhaustive and does not establish novelty.
- Assign cryptographic primitives distinct roles. SIDH/SIKE cannot underpin the proposal; CKKS computation, standardized post-quantum communication and signatures address different requirements.

## Documents and evidence

| Artifact | Contents |
|---|---|
| [Research review](research-review.md) | Overall proposal improvements, implementation findings and numerical diagnosis |
| [2025–2026 review](research-2025-2026-addendum.md) | Recent papers, GitHub/Hugging Face coverage and comparison limits |
| [Graph literature notes](recent-graph-2025-2026-addendum.md) | FicGCN, CryptGNN, PrivGNN and TGHE primary-paper details |
| [Arithmetic and compiler notes](pqc-2025-2026/literature-notes.md) | EinHops, 2026 overflow/range work and PKC 2025 output-security considerations |
| [DESIGN static audit](DESIGN-audit.md) | Commit-pinned source findings; third-party code was not executed |
| [Roadmap assessment](roadmap-review.md) | Changes recommended to the current roadmap |
| [Proposed roadmap](roadmap-proposed.md) | Future deliverables, dependency order and proposed acceptance gates |
| [Numerical results](numerical-ablation.json) | Original three-seed measurements and configuration |
| [Reproduction helper](numerical_ablation.py) | Portable replay of the arithmetic experiment |
| [Search provenance](search-provenance.json) | Review date, baseline commit, Hugging Face queries and selected metadata |

Primary-source links and access limitations are recorded in the notes. Publication, submission and revision dates are distinguished. Third-party timings and accuracy figures are author-reported and were not reproduced. Raw downloaded papers, source snapshots and large search dumps are omitted.

## Numerical result

The Cora experiment uses the standard Planetoid split, 50 training epochs and the best validation checkpoint for seeds 42, 43 and 44. The corrected simulation uses feature and weight scales of 4096.

| Seed | Full float GCN accuracy | Float ridge accuracy | Current shim accuracy | Corrected simulation accuracy | Test prediction agreement with float ridge |
|---|---:|---:|---:|---:|---:|
| 42 | 82.7% | 77.7% | 51.2% | 77.7% | 100.0% |
| 43 | 82.4% | 79.3% | 57.0% | 79.3% | 100.0% |
| 44 | 81.5% | 77.4% | 67.9% | 77.4% | 99.8% |

These are **fixed-point simulations, not homomorphic-encryption results or privacy guarantees**. Bias is added in plaintext to match the existing CLI. The full GCN and ridge classifier have different architectures. Three seeds on one dataset provide an exploratory diagnosis; roadmap accuracy thresholds are future targets. The original training traces are retained for [seed 42](numerical-ablation-seed-42.log), [seed 43](numerical-ablation-seed-43.log) and [seed 44](numerical-ablation-seed-44.log).

## Reproduction

Run from the repository root using an environment with the repository dependencies installed. The recorded environment used Python 3.12.14, PyTorch 2.6.0+cpu, PyTorch Geometric 2.8.0.post1, NumPy 2.5.3 and SciPy 1.18.1, with two CPU threads. These versions document the run; the repository dependency manifest is not a frozen environment.

```bash
OMP_NUM_THREADS=2 MKL_NUM_THREADS=2 python docs/research/2026-10-10/numerical_ablation.py
```

The helper resolves the repository root from its own location, reads the Cora dataset through the existing loader and writes new JSON/results to a fresh temporary directory, whose path it prints. If Cora is absent, the PyTorch Geometric loader downloads and processes it under `data/Cora`; network access to its dataset source is then required. The recorded run used cached Cora data. To retain a replay, supply `--output-dir /path/to/fresh-results`. The helper rejects the archive directory and existing result files to preserve recorded evidence. Timestamps and training times vary; dependency/platform changes can also change numerical values.

The portable helper was replayed on the recorded environment before archiving. All three seeds matched every archived numerical metric exactly, excluding timestamps and training times. Local Markdown links and JSON validity were also checked.

The original onboarding test run reported **7 passed, 1 failed**. The diagnosed failure is the existing simulator precision assertion in `tests/test_encryption.py`; the encrypted-evaluation CLI also has an import-path issue. This archive documents those application defects and does not fix them. Real CKKS execution, encrypted training, private topology and proof-system integration remain unvalidated.
6 changes: 6 additions & 0 deletions docs/research/2026-10-10/numerical-ablation-seed-42.log
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
Epoch 001 | loss=1.9458 | train=0.257 | val=0.156 | test=0.170
Epoch 010 | loss=1.7470 | train=0.957 | val=0.708 | test=0.725
Epoch 020 | loss=1.3528 | train=0.964 | val=0.774 | test=0.786
Epoch 030 | loss=0.9377 | train=0.971 | val=0.804 | test=0.811
Epoch 040 | loss=0.6351 | train=0.979 | val=0.806 | test=0.829
Epoch 050 | loss=0.4529 | train=0.979 | val=0.802 | test=0.825
6 changes: 6 additions & 0 deletions docs/research/2026-10-10/numerical-ablation-seed-43.log
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
Epoch 001 | loss=1.9461 | train=0.543 | val=0.356 | test=0.399
Epoch 010 | loss=1.7685 | train=0.943 | val=0.714 | test=0.733
Epoch 020 | loss=1.4218 | train=0.950 | val=0.760 | test=0.785
Epoch 030 | loss=1.0071 | train=0.957 | val=0.792 | test=0.809
Epoch 040 | loss=0.7109 | train=0.971 | val=0.804 | test=0.824
Epoch 050 | loss=0.4916 | train=0.979 | val=0.802 | test=0.822
6 changes: 6 additions & 0 deletions docs/research/2026-10-10/numerical-ablation-seed-44.log
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
Epoch 001 | loss=1.9461 | train=0.279 | val=0.180 | test=0.167
Epoch 010 | loss=1.7645 | train=0.971 | val=0.742 | test=0.753
Epoch 020 | loss=1.3958 | train=0.964 | val=0.768 | test=0.786
Epoch 030 | loss=0.9843 | train=0.964 | val=0.782 | test=0.803
Epoch 040 | loss=0.6633 | train=0.971 | val=0.786 | test=0.811
Epoch 050 | loss=0.4986 | train=0.986 | val=0.794 | test=0.815
Loading
Loading