chore: standardize configuration, logging, and error handling across blockchain node and coordinator API
- Add infrastructure.md and workflow files to .gitignore to prevent sensitive info leaks - Change blockchain node mempool backend default from memory to database for persistence - Refactor blockchain node logger with StructuredLogFormatter and AuditLogger (consistent with coordinator) - Add structured logging fields: service, module, function, line number - Unify coordinator config with Database
This commit is contained in:
oib
609
docs/5_reference/5_zk-proofs.md
Normal file
609
docs/5_reference/5_zk-proofs.md
Normal file
@@ -0,0 +1,609 @@
|
||||
# ZK Receipt Attestation Implementation Summary
|
||||
|
||||
## Overview
|
||||
|
||||
Successfully implemented a zero-knowledge proof system for privacy-preserving receipt attestation in AITBC, enabling confidential settlements while maintaining verifiability.
|
||||
|
||||
## Components Implemented
|
||||
|
||||
### 1. ZK Circuits (`apps/zk-circuits/`)
|
||||
- **Basic Circuit**: Receipt hash preimage proof in circom
|
||||
- **Advanced Circuit**: Full receipt validation with pricing (WIP)
|
||||
- **Build System**: npm scripts for compilation, setup, and proving
|
||||
- **Testing**: Proof generation and verification tests
|
||||
- **Benchmarking**: Performance measurement tools
|
||||
|
||||
### 2. Proof Service (`apps/coordinator-api/src/app/services/zk_proofs.py`)
|
||||
- **ZKProofService**: Handles proof generation and verification
|
||||
- **Privacy Levels**: Basic (hide computation) and Enhanced (hide amounts)
|
||||
- **Integration**: Works with existing receipt signing system
|
||||
- **Error Handling**: Graceful fallback when ZK unavailable
|
||||
|
||||
### 3. Receipt Integration (`apps/coordinator-api/src/app/services/receipts.py`)
|
||||
- **Async Support**: Updated create_receipt to support async ZK generation
|
||||
- **Optional Privacy**: ZK proofs generated only when requested
|
||||
- **Backward Compatibility**: Existing receipts work unchanged
|
||||
|
||||
### 4. Verification Contract (`contracts/ZKReceiptVerifier.sol`)
|
||||
- **On-Chain Verification**: Groth16 proof verification
|
||||
- **Security Features**: Double-spend prevention, timestamp validation
|
||||
- **Authorization**: Controlled access to verification functions
|
||||
- **Batch Support**: Efficient batch verification
|
||||
|
||||
### 5. Settlement Integration (`apps/coordinator-api/aitbc/settlement/hooks.py`)
|
||||
- **Privacy Options**: Settlement requests can specify privacy level
|
||||
- **Proof Inclusion**: ZK proofs included in settlement messages
|
||||
- **Bridge Support**: Works with existing cross-chain bridges
|
||||
|
||||
## Key Features
|
||||
|
||||
### Privacy Levels
|
||||
1. **Basic**: Hide computation details, reveal settlement amount
|
||||
2. **Enhanced**: Hide all amounts, prove correctness mathematically
|
||||
|
||||
### Performance Metrics
|
||||
- **Proof Size**: ~200 bytes (Groth16)
|
||||
- **Generation Time**: 5-15 seconds
|
||||
- **Verification Time**: <5ms on-chain
|
||||
- **Gas Cost**: ~200k gas
|
||||
|
||||
### Security Measures
|
||||
- Trusted setup requirements documented
|
||||
- Circuit audit procedures defined
|
||||
- Gradual rollout strategy
|
||||
- Emergency pause capabilities
|
||||
|
||||
## Testing Coverage
|
||||
|
||||
### Unit Tests
|
||||
- Proof generation with various inputs
|
||||
- Verification success/failure scenarios
|
||||
- Privacy level validation
|
||||
- Error handling
|
||||
|
||||
### Integration Tests
|
||||
- Receipt creation with ZK proofs
|
||||
- Settlement flow with privacy
|
||||
- Cross-chain bridge integration
|
||||
|
||||
### Benchmarks
|
||||
- Proof generation time measurement
|
||||
- Verification performance
|
||||
- Memory usage tracking
|
||||
- Gas cost estimation
|
||||
|
||||
## Usage Examples
|
||||
|
||||
### Creating Private Receipt
|
||||
```python
|
||||
receipt = await receipt_service.create_receipt(
|
||||
job=job,
|
||||
miner_id=miner_id,
|
||||
job_result=result,
|
||||
result_metrics=metrics,
|
||||
privacy_level="basic" # Enable ZK proof
|
||||
)
|
||||
```
|
||||
|
||||
### Cross-Chain Settlement with Privacy
|
||||
```python
|
||||
settlement = await settlement_hook.initiate_manual_settlement(
|
||||
job_id="job-123",
|
||||
target_chain_id=2,
|
||||
use_zk_proof=True,
|
||||
privacy_level="enhanced"
|
||||
)
|
||||
```
|
||||
|
||||
### On-Chain Verification
|
||||
```solidity
|
||||
bool verified = verifier.verifyAndRecord(
|
||||
proof.a,
|
||||
proof.b,
|
||||
proof.c,
|
||||
proof.publicSignals
|
||||
);
|
||||
```
|
||||
|
||||
## Current Status
|
||||
|
||||
### Completed ✅
|
||||
1. Research and technology selection (Groth16)
|
||||
2. Development environment setup
|
||||
3. Basic circuit implementation
|
||||
4. Proof generation service
|
||||
5. Verification contract
|
||||
6. Settlement integration
|
||||
7. Comprehensive testing
|
||||
8. Performance benchmarking
|
||||
|
||||
### Pending ⏳
|
||||
1. Trusted setup ceremony (production requirement)
|
||||
2. Circuit security audit
|
||||
3. Full receipt validation circuit
|
||||
4. Production deployment
|
||||
|
||||
## Next Steps for Production
|
||||
|
||||
### Immediate (Week 1-2)
|
||||
1. Run end-to-end tests with real data
|
||||
2. Performance optimization based on benchmarks
|
||||
3. Security review of implementation
|
||||
|
||||
### Short Term (Month 1)
|
||||
1. Plan and execute trusted setup ceremony
|
||||
2. Complete advanced circuit with signature verification
|
||||
3. Third-party security audit
|
||||
|
||||
### Long Term (Month 2-3)
|
||||
1. Production deployment with gradual rollout
|
||||
2. Monitor performance and gas costs
|
||||
3. Consider PLONK for universal setup
|
||||
|
||||
## Risks and Mitigations
|
||||
|
||||
### Technical Risks
|
||||
- **Trusted Setup**: Mitigate with multi-party ceremony
|
||||
- **Performance**: Optimize circuits and use batch verification
|
||||
- **Complexity**: Maintain clear documentation and examples
|
||||
|
||||
### Operational Risks
|
||||
- **User Adoption**: Provide clear UI indicators for privacy
|
||||
- **Gas Costs**: Optimize proof size and verification
|
||||
- **Regulatory**: Ensure compliance with privacy regulations
|
||||
|
||||
## Documentation
|
||||
|
||||
- [ZK Technology Comparison](zk-technology-comparison.md)
|
||||
- [Circuit Design](zk-receipt-attestation.md)
|
||||
- [Development Guide](./5_zk-proofs.md)
|
||||
- [API Documentation](../6_architecture/3_coordinator-api.md)
|
||||
|
||||
## Conclusion
|
||||
|
||||
The ZK receipt attestation system provides a solid foundation for privacy-preserving settlements in AITBC. The implementation balances privacy, performance, and usability while maintaining backward compatibility with existing systems.
|
||||
|
||||
The modular design allows for gradual adoption and future enhancements, making it suitable for both testing and production deployment.
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
This document outlines the design for adding zero-knowledge proof capabilities to the AITBC receipt attestation system, enabling privacy-preserving settlement flows while maintaining verifiability.
|
||||
|
||||
## Goals
|
||||
|
||||
1. **Privacy**: Hide sensitive transaction details (amounts, parties, specific computations)
|
||||
2. **Verifiability**: Prove receipts are valid and correctly signed without revealing contents
|
||||
3. **Compatibility**: Work with existing receipt signing and settlement systems
|
||||
4. **Efficiency**: Minimize proof generation and verification overhead
|
||||
|
||||
## Architecture
|
||||
|
||||
### Current Receipt System
|
||||
|
||||
The existing system has:
|
||||
- Receipt signing with coordinator private key
|
||||
- Optional coordinator attestations
|
||||
- History retrieval endpoints
|
||||
- Cross-chain settlement hooks
|
||||
|
||||
Receipt structure includes:
|
||||
- Job ID and metadata
|
||||
- Computation results
|
||||
- Pricing information
|
||||
- Miner and coordinator signatures
|
||||
|
||||
### Privacy-Preserving Flow
|
||||
|
||||
```
|
||||
1. Job Execution
|
||||
↓
|
||||
2. Receipt Generation (clear text)
|
||||
↓
|
||||
3. ZK Circuit Input Preparation
|
||||
↓
|
||||
4. ZK Proof Generation
|
||||
↓
|
||||
5. On-Chain Settlement (with proof)
|
||||
↓
|
||||
6. Verification (without revealing data)
|
||||
```
|
||||
|
||||
## ZK Circuit Design
|
||||
|
||||
### What to Prove
|
||||
|
||||
1. **Receipt Validity**
|
||||
- Receipt was signed by coordinator
|
||||
- Computation was performed correctly
|
||||
- Pricing follows agreed rules
|
||||
|
||||
2. **Settlement Conditions**
|
||||
- Amount owed is correctly calculated
|
||||
- Parties have sufficient funds/balance
|
||||
- Cross-chain transfer conditions met
|
||||
|
||||
### What to Hide
|
||||
|
||||
1. **Sensitive Data**
|
||||
- Actual computation amounts
|
||||
- Specific job details
|
||||
- Pricing rates
|
||||
- Participant identities
|
||||
|
||||
### Circuit Components
|
||||
|
||||
```circom
|
||||
// High-level circuit structure
|
||||
template ReceiptAttestation() {
|
||||
// Public inputs
|
||||
signal input receiptHash;
|
||||
signal input settlementAmount;
|
||||
signal input timestamp;
|
||||
|
||||
// Private inputs
|
||||
signal input receipt;
|
||||
signal input computationResult;
|
||||
signal input pricingRate;
|
||||
signal input minerReward;
|
||||
|
||||
// Verify receipt signature
|
||||
component signatureVerifier = ECDSAVerify();
|
||||
// ... signature verification logic
|
||||
|
||||
// Verify computation correctness
|
||||
component computationChecker = ComputationVerify();
|
||||
// ... computation verification logic
|
||||
|
||||
// Verify pricing calculation
|
||||
component pricingVerifier = PricingVerify();
|
||||
// ... pricing verification logic
|
||||
|
||||
// Output settlement proof
|
||||
settlementAmount <== minerReward + coordinatorFee;
|
||||
}
|
||||
```
|
||||
|
||||
## Implementation Plan
|
||||
|
||||
### Phase 1: Research & Prototyping
|
||||
1. **Library Selection**
|
||||
- snarkjs for development (JavaScript/TypeScript)
|
||||
- circomlib2 for standard circuits
|
||||
- Web3.js for blockchain integration
|
||||
|
||||
2. **Basic Circuit**
|
||||
- Simple receipt hash preimage proof
|
||||
- ECDSA signature verification
|
||||
- Basic arithmetic operations
|
||||
|
||||
### Phase 2: Integration
|
||||
1. **Coordinator API Updates**
|
||||
- Add ZK proof generation endpoint
|
||||
- Integrate with existing receipt signing
|
||||
- Add proof verification utilities
|
||||
|
||||
2. **Settlement Flow**
|
||||
- Modify cross-chain hooks to accept proofs
|
||||
- Update verification logic
|
||||
- Maintain backward compatibility
|
||||
|
||||
### Phase 3: Optimization
|
||||
1. **Performance**
|
||||
- Trusted setup for Groth16
|
||||
- Batch proof generation
|
||||
- Recursive proofs for complex receipts
|
||||
|
||||
2. **Security**
|
||||
- Audit circuits
|
||||
- Formal verification
|
||||
- Side-channel resistance
|
||||
|
||||
## Data Flow
|
||||
|
||||
### Proof Generation (Coordinator)
|
||||
|
||||
```python
|
||||
async def generate_receipt_proof(receipt: Receipt) -> ZKProof:
|
||||
# 1. Prepare circuit inputs
|
||||
public_inputs = {
|
||||
"receiptHash": hash_receipt(receipt),
|
||||
"settlementAmount": calculate_settlement(receipt),
|
||||
"timestamp": receipt.timestamp
|
||||
}
|
||||
|
||||
private_inputs = {
|
||||
"receipt": receipt,
|
||||
"computationResult": receipt.result,
|
||||
"pricingRate": receipt.pricing.rate,
|
||||
"minerReward": receipt.pricing.miner_reward
|
||||
}
|
||||
|
||||
# 2. Generate witness
|
||||
witness = generate_witness(public_inputs, private_inputs)
|
||||
|
||||
# 3. Generate proof
|
||||
proof = groth16.prove(witness, proving_key)
|
||||
|
||||
return {
|
||||
"proof": proof,
|
||||
"publicSignals": public_inputs
|
||||
}
|
||||
```
|
||||
|
||||
### Proof Verification (On-Chain/Settlement Layer)
|
||||
|
||||
```solidity
|
||||
contract SettlementVerifier {
|
||||
// Groth16 verifier
|
||||
function verifySettlement(
|
||||
uint256[2] memory a,
|
||||
uint256[2][2] memory b,
|
||||
uint256[2] memory c,
|
||||
uint256[] memory input
|
||||
) public pure returns (bool) {
|
||||
return verifyProof(a, b, c, input);
|
||||
}
|
||||
|
||||
function settleWithProof(
|
||||
address recipient,
|
||||
uint256 amount,
|
||||
ZKProof memory proof
|
||||
) public {
|
||||
require(verifySettlement(proof.a, proof.b, proof.c, proof.inputs));
|
||||
// Execute settlement
|
||||
_transfer(recipient, amount);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Privacy Levels
|
||||
|
||||
### Level 1: Basic Privacy
|
||||
- Hide computation amounts
|
||||
- Prove pricing correctness
|
||||
- Reveal participant identities
|
||||
|
||||
### Level 2: Enhanced Privacy
|
||||
- Hide all amounts
|
||||
- Zero-knowledge participant proofs
|
||||
- Anonymous settlement
|
||||
|
||||
### Level 3: Full Privacy
|
||||
- Complete transaction privacy
|
||||
- Ring signatures or similar
|
||||
- Confidential transfers
|
||||
|
||||
## Security Considerations
|
||||
|
||||
1. **Trusted Setup**
|
||||
- Multi-party ceremony for Groth16
|
||||
- Documentation of setup process
|
||||
- Toxic waste destruction proof
|
||||
|
||||
2. **Circuit Security**
|
||||
- Constant-time operations
|
||||
- No side-channel leaks
|
||||
- Formal verification where possible
|
||||
|
||||
3. **Integration Security**
|
||||
- Maintain existing security guarantees
|
||||
- Fail-safe verification
|
||||
- Gradual rollout with monitoring
|
||||
|
||||
## Migration Strategy
|
||||
|
||||
1. **Parallel Operation**
|
||||
- Run both clear and ZK receipts
|
||||
- Gradual opt-in adoption
|
||||
- Performance monitoring
|
||||
|
||||
2. **Backward Compatibility**
|
||||
- Existing receipts remain valid
|
||||
- Optional ZK proofs
|
||||
- Graceful degradation
|
||||
|
||||
3. **Network Upgrade**
|
||||
- Coordinate with all participants
|
||||
- Clear communication
|
||||
- Rollback capability
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. **Research Task**
|
||||
- Evaluate zk-SNARKs vs zk-STARKs trade-offs
|
||||
- Benchmark proof generation times
|
||||
- Assess gas costs for on-chain verification
|
||||
|
||||
2. **Prototype Development**
|
||||
- Implement basic circuit in circom
|
||||
- Create proof generation service
|
||||
- Build verification contract
|
||||
|
||||
3. **Integration Planning**
|
||||
- Design API changes
|
||||
- Plan data migration
|
||||
- Prepare rollout strategy
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
Analysis of zero-knowledge proof systems for AITBC receipt attestation, focusing on practical considerations for integration with existing infrastructure.
|
||||
|
||||
## Technology Options
|
||||
|
||||
### 1. zk-SNARKs (Zero-Knowledge Succinct Non-Interactive Argument of Knowledge)
|
||||
|
||||
**Examples**: Groth16, PLONK, Halo2
|
||||
|
||||
**Pros**:
|
||||
- **Small proof size**: ~200 bytes for Groth16
|
||||
- **Fast verification**: Constant time, ~3ms on-chain
|
||||
- **Mature ecosystem**: circom, snarkjs, bellman, arkworks
|
||||
- **Low gas costs**: ~200k gas for verification on Ethereum
|
||||
- **Industry adoption**: Used by Aztec, Tornado Cash, Zcash
|
||||
|
||||
**Cons**:
|
||||
- **Trusted setup**: Required for Groth16 (toxic waste problem)
|
||||
- **Longer proof generation**: 10-30 seconds depending on circuit size
|
||||
- **Complex setup**: Ceremony needs multiple participants
|
||||
- **Quantum vulnerability**: Not post-quantum secure
|
||||
|
||||
### 2. zk-STARKs (Zero-Knowledge Scalable Transparent Argument of Knowledge)
|
||||
|
||||
**Examples**: STARKEx, Winterfell, gnark
|
||||
|
||||
**Pros**:
|
||||
- **No trusted setup**: Transparent setup process
|
||||
- **Post-quantum secure**: Resistant to quantum attacks
|
||||
- **Faster proving**: Often faster than SNARKs for large circuits
|
||||
- **Transparent**: No toxic waste, fully verifiable setup
|
||||
|
||||
**Cons**:
|
||||
- **Larger proofs**: ~45KB for typical circuits
|
||||
- **Higher verification cost**: ~500k-1M gas on-chain
|
||||
- **Newer ecosystem**: Fewer tools and libraries
|
||||
- **Less adoption**: Limited production deployments
|
||||
|
||||
## Use Case Analysis
|
||||
|
||||
### Receipt Attestation Requirements
|
||||
|
||||
1. **Proof Size**: Important for on-chain storage costs
|
||||
2. **Verification Speed**: Critical for settlement latency
|
||||
3. **Setup Complexity**: Affects deployment timeline
|
||||
4. **Ecosystem Maturity**: Impacts development speed
|
||||
5. **Privacy Needs**: Moderate (hiding amounts, not full anonymity)
|
||||
|
||||
### Quantitative Comparison
|
||||
|
||||
| Metric | Groth16 (SNARK) | PLONK (SNARK) | STARK |
|
||||
|--------|----------------|---------------|-------|
|
||||
| Proof Size | 200 bytes | 400-500 bytes | 45KB |
|
||||
| Prover Time | 10-30s | 5-15s | 2-10s |
|
||||
| Verifier Time | 3ms | 5ms | 50ms |
|
||||
| Gas Cost | 200k | 300k | 800k |
|
||||
| Trusted Setup | Yes | Universal | No |
|
||||
| Library Support | Excellent | Good | Limited |
|
||||
|
||||
## Recommendation
|
||||
|
||||
### Phase 1: Groth16 for MVP
|
||||
|
||||
**Rationale**:
|
||||
1. **Proven technology**: Battle-tested in production
|
||||
2. **Small proofs**: Essential for cost-effective on-chain verification
|
||||
3. **Fast verification**: Critical for settlement performance
|
||||
4. **Tool maturity**: circom + snarkjs ecosystem
|
||||
5. **Community knowledge**: Extensive documentation and examples
|
||||
|
||||
**Mitigations for trusted setup**:
|
||||
- Multi-party ceremony with >100 participants
|
||||
- Public documentation of process
|
||||
- Consider PLONK for Phase 2 if setup becomes bottleneck
|
||||
|
||||
### Phase 2: Evaluate PLONK
|
||||
|
||||
**Rationale**:
|
||||
- Universal trusted setup (one-time for all circuits)
|
||||
- Slightly larger proofs but acceptable
|
||||
- More flexible for circuit updates
|
||||
- Growing ecosystem support
|
||||
|
||||
### Phase 3: Consider STARKs
|
||||
|
||||
**Rationale**:
|
||||
- If quantum resistance becomes priority
|
||||
- If proof size optimizations improve
|
||||
- If gas costs become less critical
|
||||
|
||||
## Implementation Strategy
|
||||
|
||||
### Circuit Complexity Analysis
|
||||
|
||||
**Basic Receipt Circuit**:
|
||||
- Hash verification: ~50 constraints
|
||||
- Signature verification: ~10,000 constraints
|
||||
- Arithmetic operations: ~100 constraints
|
||||
- Total: ~10,150 constraints
|
||||
|
||||
**With Privacy Features**:
|
||||
- Range proofs: ~1,000 constraints
|
||||
- Merkle proofs: ~1,000 constraints
|
||||
- Additional checks: ~500 constraints
|
||||
- Total: ~12,650 constraints
|
||||
|
||||
### Performance Estimates
|
||||
|
||||
**Groth16**:
|
||||
- Setup time: 2-5 hours
|
||||
- Proving time: 5-15 seconds
|
||||
- Verification: 3ms
|
||||
- Proof size: 200 bytes
|
||||
|
||||
**Infrastructure Impact**:
|
||||
- Coordinator: Additional 5-15s per receipt
|
||||
- Settlement layer: Minimal impact (fast verification)
|
||||
- Storage: Negligible increase
|
||||
|
||||
## Security Considerations
|
||||
|
||||
### Trusted Setup Risks
|
||||
|
||||
1. **Toxic Waste**: If compromised, can forge proofs
|
||||
2. **Setup Integrity**: Requires honest participants
|
||||
3. **Documentation**: Must be publicly verifiable
|
||||
|
||||
### Mitigation Strategies
|
||||
|
||||
1. **Multi-party Ceremony**:
|
||||
- Minimum 100 participants
|
||||
- Geographically distributed
|
||||
- Public livestream
|
||||
|
||||
2. **Circuit Audits**:
|
||||
- Formal verification where possible
|
||||
- Third-party security review
|
||||
- Public disclosure of circuits
|
||||
|
||||
3. **Gradual Rollout**:
|
||||
- Start with low-value transactions
|
||||
- Monitor for anomalies
|
||||
- Emergency pause capability
|
||||
|
||||
## Development Plan
|
||||
|
||||
### Week 1-2: Environment Setup
|
||||
- Install circom and snarkjs
|
||||
- Create basic test circuit
|
||||
- Benchmark proof generation
|
||||
|
||||
### Week 3-4: Basic Circuit
|
||||
- Implement receipt hash verification
|
||||
- Add signature verification
|
||||
- Test with sample receipts
|
||||
|
||||
### Week 5-6: Integration
|
||||
- Add to coordinator API
|
||||
- Create verification contract
|
||||
- Test settlement flow
|
||||
|
||||
### Week 7-8: Trusted Setup
|
||||
- Plan ceremony logistics
|
||||
- Prepare ceremony software
|
||||
- Execute multi-party setup
|
||||
|
||||
### Week 9-10: Testing & Audit
|
||||
- End-to-end testing
|
||||
- Security review
|
||||
- Performance optimization
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. **Immediate**: Set up development environment
|
||||
2. **Research**: Deep dive into circom best practices
|
||||
3. **Prototype**: Build minimal viable circuit
|
||||
4. **Evaluate**: Performance with real receipt data
|
||||
5. **Decide**: Final technology choice based on testing
|
||||
Reference in New Issue
Block a user