chore(security): enhance environment configuration, CI workflows, and wallet daemon with security improvements

- Restructure .env.example with security-focused documentation, service-specific environment file references, and AWS Secrets Manager integration
- Update CLI tests workflow to single Python 3.13 version, add pytest-mock dependency, and consolidate test execution with coverage
- Add comprehensive security validation to package publishing workflow with manual approval gates, secret scanning, and release
This commit is contained in:
oib
2026-03-03 10:33:46 +01:00
parent 00d00cb964
commit f353e00172
220 changed files with 42506 additions and 921 deletions

View File

@@ -0,0 +1,112 @@
# GPU Acceleration Migration Checklist
## ✅ Pre-Migration Preparation
- [ ] Review existing CUDA-specific code
- [ ] Identify all files that import CUDA modules
- [ ] Document current CUDA usage patterns
- [ ] Create backup of existing code
- [ ] Test current functionality
## ✅ Code Migration
### Import Statements
- [ ] Replace `from high_performance_cuda_accelerator import ...` with `from gpu_acceleration import ...`
- [ ] Replace `from fastapi_cuda_zk_api import ...` with `from gpu_acceleration import ...`
- [ ] Update all CUDA-specific imports
### Function Calls
- [ ] Replace `accelerator.field_add_cuda()` with `gpu.field_add()`
- [ ] Replace `accelerator.field_mul_cuda()` with `gpu.field_mul()`
- [ ] Replace `accelerator.multi_scalar_mul_cuda()` with `gpu.multi_scalar_mul()`
- [ ] Update all CUDA-specific function calls
### Initialization
- [ ] Replace `HighPerformanceCUDAZKAccelerator()` with `GPUAccelerationManager()`
- [ ] Replace `ProductionCUDAZKAPI()` with `create_gpu_manager()`
- [ ] Add proper error handling for backend initialization
### Error Handling
- [ ] Add fallback handling for GPU failures
- [ ] Update error messages to be backend-agnostic
- [ ] Add backend information to error responses
## ✅ Testing
### Unit Tests
- [ ] Update unit tests to use new interface
- [ ] Test backend auto-detection
- [ ] Test fallback to CPU
- [ ] Test performance regression
### Integration Tests
- [ ] Test API endpoints with new backend
- [ ] Test multi-backend scenarios
- [ ] Test configuration options
- [ ] Test error handling
### Performance Tests
- [ ] Benchmark new vs old implementation
- [ ] Test performance with different backends
- [ ] Verify no significant performance regression
- [ ] Test memory usage
## ✅ Documentation
### Code Documentation
- [ ] Update docstrings to be backend-agnostic
- [ ] Add examples for new interface
- [ ] Document configuration options
- [ ] Update error handling documentation
### API Documentation
- [ ] Update API docs to reflect backend flexibility
- [ ] Add backend information endpoints
- [ ] Update performance monitoring docs
- [ ] Document migration process
### User Documentation
- [ ] Update user guides with new examples
- [ ] Document backend selection options
- [ ] Add troubleshooting guide
- [ ] Update installation instructions
## ✅ Deployment
### Configuration
- [ ] Update deployment scripts
- [ ] Add backend selection environment variables
- [ ] Update monitoring for new metrics
- [ ] Test deployment with different backends
### Monitoring
- [ ] Update monitoring to track backend usage
- [ ] Add alerts for backend failures
- [ ] Monitor performance metrics
- [ ] Track fallback usage
### Rollback Plan
- [ ] Document rollback procedure
- [ ] Test rollback process
- [ ] Prepare backup deployment
- [ ] Create rollback triggers
## ✅ Validation
### Functional Validation
- [ ] All existing functionality works
- [ ] New backend features work correctly
- [ ] Error handling works as expected
- [ ] Performance is acceptable
### Security Validation
- [ ] No new security vulnerabilities
- [ ] Backend isolation works correctly
- [ ] Input validation still works
- [ ] Error messages don't leak information
### Performance Validation
- [ ] Performance meets requirements
- [ ] Memory usage is acceptable
- [ ] Scalability is maintained
- [ ] Resource utilization is optimal

View File

@@ -0,0 +1,49 @@
#!/usr/bin/env python3
"""
API Migration Example
Shows how to migrate FastAPI endpoints to use the new abstraction layer.
"""
# BEFORE (CUDA-specific API)
# from fastapi_cuda_zk_api import ProductionCUDAZKAPI
#
# cuda_api = ProductionCUDAZKAPI()
# if not cuda_api.initialized:
# raise HTTPException(status_code=500, detail="CUDA not available")
# AFTER (Backend-agnostic API)
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
from gpu_acceleration import GPUAccelerationManager, create_gpu_manager
import numpy as np
app = FastAPI(title="Refactored GPU API")
# Initialize GPU manager (auto-detects best backend)
gpu_manager = create_gpu_manager()
class FieldOperation(BaseModel):
a: list[int]
b: list[int]
@app.post("/field/add")
async def field_add(op: FieldOperation):
"""Perform field addition with any available backend."""
try:
a = np.array(op.a, dtype=np.uint64)
b = np.array(op.b, dtype=np.uint64)
result = gpu_manager.field_add(a, b)
return {"result": result.tolist()}
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
@app.get("/backend/info")
async def backend_info():
"""Get current backend information."""
return gpu_manager.get_backend_info()
@app.get("/performance/metrics")
async def performance_metrics():
"""Get performance metrics."""
return gpu_manager.get_performance_metrics()

View File

@@ -0,0 +1,40 @@
#!/usr/bin/env python3
"""
Basic Migration Example
Shows how to migrate from direct CUDA calls to the new abstraction layer.
"""
# BEFORE (Direct CUDA)
# from high_performance_cuda_accelerator import HighPerformanceCUDAZKAccelerator
#
# accelerator = HighPerformanceCUDAZKAccelerator()
# if accelerator.initialized:
# result = accelerator.field_add_cuda(a, b)
# AFTER (Abstraction Layer)
import numpy as np
from gpu_acceleration import GPUAccelerationManager, create_gpu_manager
# Method 1: Auto-detect backend
gpu = create_gpu_manager()
gpu.initialize()
a = np.array([1, 2, 3, 4], dtype=np.uint64)
b = np.array([5, 6, 7, 8], dtype=np.uint64)
result = gpu.field_add(a, b)
print(f"Field addition result: {result}")
# Method 2: Context manager (recommended)
from gpu_acceleration import GPUAccelerationContext
with GPUAccelerationContext() as gpu:
result = gpu.field_mul(a, b)
print(f"Field multiplication result: {result}")
# Method 3: Quick functions
from gpu_acceleration import quick_field_add
result = quick_field_add(a, b)
print(f"Quick field addition: {result}")

View File

@@ -0,0 +1,38 @@
#!/usr/bin/env python3
"""
Configuration Migration Example
Shows how to migrate configuration to use the new abstraction layer.
"""
# BEFORE (CUDA-specific config)
# cuda_config = {
# "lib_path": "./liboptimized_field_operations.so",
# "device_id": 0,
# "memory_limit": 8*1024*1024*1024
# }
# AFTER (Backend-agnostic config)
from gpu_acceleration import ZKOperationConfig, GPUAccelerationManager, ComputeBackend
# Configuration for any backend
config = ZKOperationConfig(
batch_size=2048,
use_gpu=True,
fallback_to_cpu=True,
timeout=60.0,
memory_limit=8*1024*1024*1024 # 8GB
)
# Create manager with specific backend
gpu = GPUAccelerationManager(backend=ComputeBackend.CUDA, config=config)
gpu.initialize()
# Or auto-detect with config
from gpu_acceleration import create_gpu_manager
gpu = create_gpu_manager(
backend="cuda", # or None for auto-detect
batch_size=2048,
fallback_to_cpu=True,
timeout=60.0
)