oib/aitbc
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: add marketplace metrics, privacy features, and service registry endpoints

Browse Source 
- Add Prometheus metrics for marketplace API throughput and error rates with new dashboard panels
- Implement confidential transaction models with encryption support and access control
- Add key management system with registration, rotation, and audit logging
- Create services and registry routers for service discovery and management
- Integrate ZK proof generation for privacy-preserving receipts
- Add metrics instru
This commit is contained in:
oib
2025-12-22 10:33:23 +01:00
parent d98b2c7772
commit c8be9d7414
260 changed files with 59033 additions and 351 deletions
Download Patch File Download Diff File Expand all files Collapse all files

403
docs/reference/architecture/cross-chain-settlement-design.md Normal file
View File

@ -0,0 +1,403 @@
# Cross-Chain Settlement Hooks Design
## Overview
This document outlines the architecture for cross-chain settlement hooks in AITBC, enabling job receipts and proofs to be settled across multiple blockchains using various bridge protocols.
## Architecture
### Core Components
```
┌─────────────────┐ ┌──────────────────┐ ┌─────────────────┐
│ AITBC Chain │ │ Settlement Hooks │ │ Target Chains │
│ │ │ │ │ │
│ - Job Receipts │───▶│ - Bridge Manager │───▶│ - Ethereum │
│ - Proofs │ │ - Adapters │ │ - Polygon │
│ - Payments │ │ - Router │ │ - BSC │
│ │ │ - Validator │ │ - Arbitrum │
└─────────────────┘ └──────────────────┘ └─────────────────┘
```
### Settlement Hook Interface
```python
from abc import ABC, abstractmethod
from typing import Dict, Any, List
from dataclasses import dataclass
@dataclass
class SettlementMessage:
"""Message to be settled across chains"""
source_chain_id: int
target_chain_id: int
job_id: str
receipt_hash: str
proof_data: Dict[str, Any]
payment_amount: int
payment_token: str
nonce: int
signature: str
class BridgeAdapter(ABC):
"""Abstract interface for bridge adapters"""
@abstractmethod
async def send_message(self, message: SettlementMessage) -> str:
"""Send message to target chain"""
pass
@abstractmethod
async def verify_delivery(self, message_id: str) -> bool:
"""Verify message was delivered"""
pass
@abstractmethod
async def estimate_cost(self, message: SettlementMessage) -> Dict[str, int]:
"""Estimate bridge fees"""
pass
@abstractmethod
def get_supported_chains(self) -> List[int]:
"""Get list of supported target chains"""
pass
@abstractmethod
def get_max_message_size(self) -> int:
"""Get maximum message size in bytes"""
pass
```
### Bridge Manager
```python
class BridgeManager:
"""Manages multiple bridge adapters"""
def __init__(self):
self.adapters: Dict[str, BridgeAdapter] = {}
self.default_adapter: str = None
def register_adapter(self, name: str, adapter: BridgeAdapter):
"""Register a bridge adapter"""
self.adapters[name] = adapter
async def settle_cross_chain(
self,
message: SettlementMessage,
bridge_name: str = None
) -> str:
"""Settle message across chains"""
adapter = self._get_adapter(bridge_name)
# Validate message
self._validate_message(message, adapter)
# Send message
message_id = await adapter.send_message(message)
# Store settlement record
await self._store_settlement(message_id, message)
return message_id
def _get_adapter(self, bridge_name: str = None) -> BridgeAdapter:
"""Get bridge adapter"""
if bridge_name:
return self.adapters[bridge_name]
return self.adapters[self.default_adapter]
```
## Bridge Implementations
### 1. LayerZero Adapter
```python
class LayerZeroAdapter(BridgeAdapter):
"""LayerZero bridge adapter"""
def __init__(self, endpoint_address: str, chain_id: int):
self.endpoint = endpoint_address
self.chain_id = chain_id
self.contract = self._load_contract()
async def send_message(self, message: SettlementMessage) -> str:
"""Send via LayerZero"""
# Encode settlement data
payload = self._encode_payload(message)
# Estimate fees
fees = await self._estimate_fees(message)
# Send transaction
tx = await self.contract.send(
message.target_chain_id,
self._get_target_address(message.target_chain_id),
payload,
message.payment_amount,
message.payment_token,
fees
)
return tx.hash
def _encode_payload(self, message: SettlementMessage) -> bytes:
"""Encode message for LayerZero"""
return abi.encode(
['uint256', 'bytes32', 'bytes', 'uint256', 'address'],
[
message.job_id,
message.receipt_hash,
json.dumps(message.proof_data),
message.payment_amount,
message.payment_token
]
)
```
### 2. Chainlink CCIP Adapter
```python
class ChainlinkCCIPAdapter(BridgeAdapter):
"""Chainlink CCIP bridge adapter"""
def __init__(self, router_address: str, chain_id: int):
self.router = router_address
self.chain_id = chain_id
self.contract = self._load_contract()
async def send_message(self, message: SettlementMessage) -> str:
"""Send via Chainlink CCIP"""
# Create CCIP message
ccip_message = {
'receiver': self._get_target_address(message.target_chain_id),
'data': self._encode_payload(message),
'tokenAmounts': [{
'token': message.payment_token,
'amount': message.payment_amount
}]
}
# Estimate fees
fees = await self.contract.getFee(ccip_message)
# Send transaction
tx = await self.contract.ccipSend(ccip_message, {'value': fees})
return tx.hash
```
### 3. Wormhole Adapter
```python
class WormholeAdapter(BridgeAdapter):
"""Wormhole bridge adapter"""
def __init__(self, bridge_address: str, chain_id: int):
self.bridge = bridge_address
self.chain_id = chain_id
self.contract = self._load_contract()
async def send_message(self, message: SettlementMessage) -> str:
"""Send via Wormhole"""
# Encode payload
payload = self._encode_payload(message)
# Send transaction
tx = await self.contract.publishMessage(
message.nonce,
payload,
message.payment_amount
)
return tx.hash
```
## Integration with Coordinator
### Settlement Hook in Coordinator
```python
class SettlementHook:
"""Settlement hook for coordinator"""
def __init__(self, bridge_manager: BridgeManager):
self.bridge_manager = bridge_manager
async def on_job_completed(self, job: Job) -> None:
"""Called when job completes"""
# Check if cross-chain settlement needed
if job.requires_cross_chain_settlement:
await self._settle_cross_chain(job)
async def _settle_cross_chain(self, job: Job) -> None:
"""Settle job across chains"""
# Create settlement message
message = SettlementMessage(
source_chain_id=await self._get_chain_id(),
target_chain_id=job.target_chain,
job_id=job.id,
receipt_hash=job.receipt.hash,
proof_data=job.receipt.proof,
payment_amount=job.payment_amount,
payment_token=job.payment_token,
nonce=await self._get_nonce(),
signature=await self._sign_message(job)
)
# Send via appropriate bridge
await self.bridge_manager.settle_cross_chain(
message,
bridge_name=job.preferred_bridge
)
```
### Coordinator API Endpoints
```python
@app.post("/v1/settlement/cross-chain")
async def initiate_cross_chain_settlement(
request: CrossChainSettlementRequest
):
"""Initiate cross-chain settlement"""
job = await get_job(request.job_id)
if not job.completed:
raise HTTPException(400, "Job not completed")
# Create settlement message
message = SettlementMessage(
source_chain_id=request.source_chain,
target_chain_id=request.target_chain,
job_id=job.id,
receipt_hash=job.receipt.hash,
proof_data=job.receipt.proof,
payment_amount=request.amount,
payment_token=request.token,
nonce=await generate_nonce(),
signature=await sign_settlement(job, request)
)
# Send settlement
message_id = await settlement_hook.settle_cross_chain(message)
return {"message_id": message_id, "status": "pending"}
@app.get("/v1/settlement/{message_id}/status")
async def get_settlement_status(message_id: str):
"""Get settlement status"""
status = await bridge_manager.get_settlement_status(message_id)
return status
```
## Configuration
### Bridge Configuration
```yaml
bridges:
layerzero:
enabled: true
endpoint_address: "0x..."
supported_chains: [1, 137, 56, 42161]
default_fee: "0.001"
chainlink_ccip:
enabled: true
router_address: "0x..."
supported_chains: [1, 137, 56, 42161]
default_fee: "0.002"
wormhole:
enabled: false
bridge_address: "0x..."
supported_chains: [1, 137, 56]
default_fee: "0.0015"
settlement:
default_bridge: "layerzero"
max_retries: 3
retry_delay: 30
timeout: 3600
```
## Security Considerations
### Message Validation
- Verify signatures on all settlement messages
- Validate chain IDs and addresses
- Check message size limits
- Prevent replay attacks with nonces
### Bridge Security
- Use reputable audited bridge contracts
- Implement bridge-specific security checks
- Monitor for bridge vulnerabilities
- Have fallback mechanisms
### Economic Security
- Validate payment amounts
- Check token allowances
- Implement fee limits
- Monitor for economic attacks
## Monitoring
### Metrics to Track
- Settlement success rate per bridge
- Average settlement time
- Cost per settlement
- Failed settlement reasons
- Bridge health status
### Alerts
- Settlement failures
- High settlement costs
- Bridge downtime
- Unusual settlement patterns
## Testing
### Test Scenarios
1. **Happy Path**: Successful settlement across chains
2. **Bridge Failure**: Handle bridge unavailability
3. **Message Too Large**: Handle size limits
4. **Insufficient Funds**: Handle payment failures
5. **Replay Attack**: Prevent duplicate settlements
### Test Networks
- Ethereum Sepolia
- Polygon Mumbai
- BSC Testnet
- Arbitrum Goerli
## Migration Path
### Phase 1: Single Bridge
- Implement LayerZero adapter
- Basic settlement functionality
- Test on testnets
### Phase 2: Multiple Bridges
- Add Chainlink CCIP
- Implement bridge selection logic
- Add cost optimization
### Phase 3: Advanced Features
- Add Wormhole support
- Implement atomic settlements
- Add settlement routing
## Future Enhancements
1. **Atomic Settlements**: Ensure all-or-nothing settlements
2. **Settlement Routing**: Automatically select optimal bridge
3. **Batch Settlements**: Settle multiple jobs together
4. **Cross-Chain Governance**: Governance across chains
5. **Privacy Features**: Confidential settlements
---
*Document Version: 1.0*
*Last Updated: 2025-01-10*
*Owner: Core Protocol Team*

618
docs/reference/architecture/python-sdk-transport-design.md Normal file
View File

@ -0,0 +1,618 @@
# Python SDK Transport Abstraction Design
## Overview
This document outlines the design for a pluggable transport abstraction layer in the AITBC Python SDK, enabling support for multiple networks and cross-chain operations.
## Architecture
### Current SDK Structure
```
AITBCClient
├── Jobs API
├── Marketplace API
├── Wallet API
├── Receipts API
└── Direct HTTP calls to coordinator
```
### Proposed Transport-Based Structure
```
AITBCClient
├── Transport Layer (Pluggable)
│ ├── HTTPTransport
│ ├── WebSocketTransport
│ └── CrossChainTransport
├── Jobs API
├── Marketplace API
├── Wallet API
├── Receipts API
└── Settlement API (New)
```
## Transport Interface
### Base Transport Class
```python
from abc import ABC, abstractmethod
from typing import Dict, Any, Optional, Union
import asyncio
class Transport(ABC):
"""Abstract base class for all transports"""
def __init__(self, config: Dict[str, Any]):
self.config = config
self._connected = False
@abstractmethod
async def connect(self) -> None:
"""Establish connection"""
pass
@abstractmethod
async def disconnect(self) -> None:
"""Close connection"""
pass
@abstractmethod
async def request(
self,
method: str,
path: str,
data: Optional[Dict[str, Any]] = None,
params: Optional[Dict[str, Any]] = None,
headers: Optional[Dict[str, str]] = None
) -> Dict[str, Any]:
"""Make a request"""
pass
@abstractmethod
async def stream(
self,
method: str,
path: str,
data: Optional[Dict[str, Any]] = None
) -> AsyncIterator[Dict[str, Any]]:
"""Stream responses"""
pass
@property
def is_connected(self) -> bool:
"""Check if transport is connected"""
return self._connected
@property
def chain_id(self) -> Optional[int]:
"""Get the chain ID this transport is connected to"""
return self.config.get('chain_id')
```
### HTTP Transport Implementation
```python
import aiohttp
from typing import AsyncIterator
class HTTPTransport(Transport):
"""HTTP transport for REST API calls"""
def __init__(self, config: Dict[str, Any]):
super().__init__(config)
self.base_url = config['base_url']
self.session: Optional[aiohttp.ClientSession] = None
self.timeout = config.get('timeout', 30)
async def connect(self) -> None:
"""Create HTTP session"""
connector = aiohttp.TCPConnector(
limit=100,
limit_per_host=30,
ttl_dns_cache=300,
use_dns_cache=True,
)
timeout = aiohttp.ClientTimeout(total=self.timeout)
self.session = aiohttp.ClientSession(
connector=connector,
timeout=timeout,
headers=self.config.get('default_headers', {})
)
self._connected = True
async def disconnect(self) -> None:
"""Close HTTP session"""
if self.session:
await self.session.close()
self.session = None
self._connected = False
async def request(
self,
method: str,
path: str,
data: Optional[Dict[str, Any]] = None,
params: Optional[Dict[str, Any]] = None,
headers: Optional[Dict[str, str]] = None
) -> Dict[str, Any]:
"""Make HTTP request"""
if not self.session:
await self.connect()
url = f"{self.base_url}{path}"
async with self.session.request(
method=method,
url=url,
json=data,
params=params,
headers=headers
) as response:
if response.status >= 400:
error_data = await response.json()
raise APIError(error_data.get('error', 'Unknown error'))
return await response.json()
async def stream(
self,
method: str,
path: str,
data: Optional[Dict[str, Any]] = None
) -> AsyncIterator[Dict[str, Any]]:
"""Stream HTTP responses (not supported for basic HTTP)"""
raise NotImplementedError("HTTP transport does not support streaming")
```
### WebSocket Transport Implementation
```python
import websockets
import json
from typing import AsyncIterator
class WebSocketTransport(Transport):
"""WebSocket transport for real-time updates"""
def __init__(self, config: Dict[str, Any]):
super().__init__(config)
self.ws_url = config['ws_url']
self.websocket: Optional[websockets.WebSocketServerProtocol] = None
self._subscriptions: Dict[str, Any] = {}
async def connect(self) -> None:
"""Connect to WebSocket"""
self.websocket = await websockets.connect(
self.ws_url,
extra_headers=self.config.get('headers', {})
)
self._connected = True
# Start message handler
asyncio.create_task(self._handle_messages())
async def disconnect(self) -> None:
"""Disconnect WebSocket"""
if self.websocket:
await self.websocket.close()
self.websocket = None
self._connected = False
async def request(
self,
method: str,
path: str,
data: Optional[Dict[str, Any]] = None,
params: Optional[Dict[str, Any]] = None,
headers: Optional[Dict[str, str]] = None
) -> Dict[str, Any]:
"""Send request via WebSocket"""
if not self.websocket:
await self.connect()
message = {
'id': self._generate_id(),
'method': method,
'path': path,
'data': data,
'params': params
}
await self.websocket.send(json.dumps(message))
response = await self.websocket.recv()
return json.loads(response)
async def stream(
self,
method: str,
path: str,
data: Optional[Dict[str, Any]] = None
) -> AsyncIterator[Dict[str, Any]]:
"""Stream responses from WebSocket"""
if not self.websocket:
await self.connect()
# Subscribe to stream
subscription_id = self._generate_id()
message = {
'id': subscription_id,
'method': 'subscribe',
'path': path,
'data': data
}
await self.websocket.send(json.dumps(message))
# Yield messages as they come
async for message in self.websocket:
data = json.loads(message)
if data.get('subscription_id') == subscription_id:
yield data
async def _handle_messages(self):
"""Handle incoming WebSocket messages"""
async for message in self.websocket:
data = json.loads(message)
# Handle subscriptions and other messages
pass
```
### Cross-Chain Transport Implementation
```python
from ..settlement.manager import BridgeManager
from ..settlement.bridges.base import SettlementMessage, SettlementResult
class CrossChainTransport(Transport):
"""Transport for cross-chain settlements"""
def __init__(self, config: Dict[str, Any]):
super().__init__(config)
self.bridge_manager = BridgeManager(config.get('storage'))
self.base_transport = config.get('base_transport')
async def connect(self) -> None:
"""Initialize bridge manager"""
await self.bridge_manager.initialize(config.get('bridges', {}))
if self.base_transport:
await self.base_transport.connect()
self._connected = True
async def disconnect(self) -> None:
"""Disconnect all bridges"""
if self.base_transport:
await self.base_transport.disconnect()
self._connected = False
async def request(
self,
method: str,
path: str,
data: Optional[Dict[str, Any]] = None,
params: Optional[Dict[str, Any]] = None,
headers: Optional[Dict[str, str]] = None
) -> Dict[str, Any]:
"""Handle cross-chain requests"""
if path.startswith('/settlement/'):
return await self._handle_settlement_request(method, path, data)
# Forward to base transport for other requests
if self.base_transport:
return await self.base_transport.request(
method, path, data, params, headers
)
raise NotImplementedError(f"Path {path} not supported")
async def settle_cross_chain(
self,
message: SettlementMessage,
bridge_name: Optional[str] = None
) -> SettlementResult:
"""Settle message across chains"""
return await self.bridge_manager.settle_cross_chain(
message, bridge_name
)
async def estimate_settlement_cost(
self,
message: SettlementMessage,
bridge_name: Optional[str] = None
) -> Dict[str, Any]:
"""Estimate settlement cost"""
return await self.bridge_manager.estimate_settlement_cost(
message, bridge_name
)
async def _handle_settlement_request(
self,
method: str,
path: str,
data: Optional[Dict[str, Any]]
) -> Dict[str, Any]:
"""Handle settlement-specific requests"""
if method == 'POST' and path == '/settlement/cross-chain':
message = SettlementMessage(**data)
result = await self.settle_cross_chain(message)
return {
'message_id': result.message_id,
'status': result.status.value,
'transaction_hash': result.transaction_hash
}
elif method == 'GET' and path.startswith('/settlement/'):
message_id = path.split('/')[-1]
result = await self.bridge_manager.get_settlement_status(message_id)
return {
'message_id': message_id,
'status': result.status.value,
'error_message': result.error_message
}
else:
raise ValueError(f"Unsupported settlement request: {method} {path}")
```
## Multi-Network Client
### Network Configuration
```python
@dataclass
class NetworkConfig:
"""Configuration for a network"""
name: str
chain_id: int
transport: Transport
is_default: bool = False
bridges: List[str] = None
class MultiNetworkClient:
"""Client supporting multiple networks"""
def __init__(self):
self.networks: Dict[int, NetworkConfig] = {}
self.default_network: Optional[int] = None
def add_network(self, config: NetworkConfig) -> None:
"""Add a network configuration"""
self.networks[config.chain_id] = config
if config.is_default or self.default_network is None:
self.default_network = config.chain_id
def get_transport(self, chain_id: Optional[int] = None) -> Transport:
"""Get transport for a network"""
network_id = chain_id or self.default_network
if network_id not in self.networks:
raise ValueError(f"Network {network_id} not configured")
return self.networks[network_id].transport
async def connect_all(self) -> None:
"""Connect to all configured networks"""
for config in self.networks.values():
await config.transport.connect()
async def disconnect_all(self) -> None:
"""Disconnect from all networks"""
for config in self.networks.values():
await config.transport.disconnect()
```
## Updated SDK Client
### New Client Implementation
```python
class AITBCClient:
"""AITBC client with pluggable transports"""
def __init__(
self,
transport: Optional[Union[Transport, Dict[str, Any]]] = None,
multi_network: bool = False
):
if multi_network:
self._init_multi_network(transport or {})
else:
self._init_single_network(transport or {})
def _init_single_network(self, transport_config: Dict[str, Any]) -> None:
"""Initialize single network client"""
if isinstance(transport_config, Transport):
self.transport = transport_config
else:
# Default to HTTP transport
self.transport = HTTPTransport(transport_config)
self.multi_network = False
self._init_apis()
def _init_multi_network(self, configs: Dict[str, Any]) -> None:
"""Initialize multi-network client"""
self.multi_network_client = MultiNetworkClient()
# Configure networks
for name, config in configs.get('networks', {}).items():
transport = self._create_transport(config)
network_config = NetworkConfig(
name=name,
chain_id=config['chain_id'],
transport=transport,
is_default=config.get('default', False)
)
self.multi_network_client.add_network(network_config)
self.multi_network = True
self._init_apis()
def _create_transport(self, config: Dict[str, Any]) -> Transport:
"""Create transport from config"""
transport_type = config.get('type', 'http')
if transport_type == 'http':
return HTTPTransport(config)
elif transport_type == 'websocket':
return WebSocketTransport(config)
elif transport_type == 'crosschain':
return CrossChainTransport(config)
else:
raise ValueError(f"Unknown transport type: {transport_type}")
def _init_apis(self) -> None:
"""Initialize API clients"""
if self.multi_network:
self.jobs = MultiNetworkJobsAPI(self.multi_network_client)
self.settlement = MultiNetworkSettlementAPI(self.multi_network_client)
else:
self.jobs = JobsAPI(self.transport)
self.settlement = SettlementAPI(self.transport)
# Other APIs remain the same but use the transport
self.marketplace = MarketplaceAPI(self.transport)
self.wallet = WalletAPI(self.transport)
self.receipts = ReceiptsAPI(self.transport)
async def connect(self) -> None:
"""Connect to network(s)"""
if self.multi_network:
await self.multi_network_client.connect_all()
else:
await self.transport.connect()
async def disconnect(self) -> None:
"""Disconnect from network(s)"""
if self.multi_network:
await self.multi_network_client.disconnect_all()
else:
await self.transport.disconnect()
```
## Usage Examples
### Single Network with HTTP Transport
```python
from aitbc import AITBCClient, HTTPTransport
# Create client with HTTP transport
transport = HTTPTransport({
'base_url': 'https://api.aitbc.io',
'timeout': 30,
'default_headers': {'X-API-Key': 'your-key'}
})
client = AITBCClient(transport)
await client.connect()
# Use APIs normally
job = await client.jobs.create({...})
```
### Multi-Network Configuration
```python
from aitbc import AITBCClient
config = {
'networks': {
'ethereum': {
'type': 'http',
'chain_id': 1,
'base_url': 'https://api.aitbc.io',
'default': True
},
'polygon': {
'type': 'http',
'chain_id': 137,
'base_url': 'https://polygon-api.aitbc.io'
},
'arbitrum': {
'type': 'crosschain',
'chain_id': 42161,
'base_transport': HTTPTransport({
'base_url': 'https://arbitrum-api.aitbc.io'
}),
'bridges': {
'layerzero': {'enabled': True},
'chainlink': {'enabled': True}
}
}
}
}
client = AITBCClient(config, multi_network=True)
await client.connect()
# Create job on specific network
job = await client.jobs.create({...}, chain_id=137)
# Settle across chains
settlement = await client.settlement.settle_cross_chain(
job_id=job['id'],
target_chain_id=42161,
bridge_name='layerzero'
)
```
### Cross-Chain Settlement
```python
# Create job on Ethereum
job = await client.jobs.create({
'name': 'cross-chain-ai-job',
'target_chain': 42161, # Arbitrum
'requires_cross_chain_settlement': True
})
# Wait for completion
result = await client.jobs.wait_for_completion(job['id'])
# Settle to Arbitrum
settlement = await client.settlement.settle_cross_chain(
job_id=job['id'],
target_chain_id=42161,
bridge_name='layerzero'
)
# Monitor settlement
status = await client.settlement.get_status(settlement['message_id'])
```
## Migration Guide
### From Current SDK
```python
# Old way
client = AITBCClient(api_key='key', base_url='url')
# New way (backward compatible)
client = AITBCClient({
'base_url': 'url',
'default_headers': {'X-API-Key': 'key'}
})
# Or with explicit transport
transport = HTTPTransport({
'base_url': 'url',
'default_headers': {'X-API-Key': 'key'}
})
client = AITBCClient(transport)
```
## Benefits
1. **Flexibility**: Easy to add new transport types
2. **Multi-Network**: Support for multiple blockchains
3. **Cross-Chain**: Built-in support for cross-chain settlements
4. **Backward Compatible**: Existing code continues to work
5. **Testable**: Easy to mock transports for testing
6. **Extensible**: Plugin architecture for custom transports
---
*Document Version: 1.0*
*Last Updated: 2025-01-10*
*Owner: SDK Team*

600
docs/reference/bootstrap/aitbc_tech_plan.md Normal file
View File

@ -0,0 +1,600 @@
# AITBC – Artificial Intelligence Token Blockchain
## Overview (Recovered)
- AITBC couples decentralized blockchain control with asset‑backed value derived from AI computation.
- No pre‑mint: tokens are minted by providers **only after** serving compute; prices are set by providers (can be free at bootstrap).
## Staged Development Roadmap
### Stage 1: Client–Server Prototype (no blockchain, no hub)
- Direct client → server API.
- API‑key auth; local job logging.
- Goal: validate AI service loop and throughput.
### Stage 2: Blockchain Integration
- Introduce **AIToken** and minimal smart contracts for minting + accounting.
- Mint = amount of compute successfully served; no premint.
### Stage 3: AI Pool Hub
- Hub matches requests to multiple servers (sharding/parallelization), verifies outputs, and accounts contributions.
- Distributes payments/minted tokens proportionally to work.
### Stage 4: Marketplace
- Web DEX/market to buy/sell AITokens; price discovery; reputation and SLAs.
## System Architecture: Actors
- **Client** – requests AI jobs (e.g., image/video generation).
- **Server/Provider** – runs models (Stable Diffusion, PyTorch, etc.).
- **Blockchain Node** – ledger + minting rules.
- **AI Pool Hub** – orchestration, metering, payouts.
## Token Minting Logic (Genesis‑less)
- No tokens at boot.
- Provider advertises price/unit (e.g., 1 AIToken per image or per N GPU‑seconds).
- After successful job → provider mints that amount. Free jobs mint 0.
---
# Stage 1 – Technischer Implementierungsplan (Detail)
## Ziele
- Funktionierender End‑to‑End‑Pfad: Prompt → Inferenz → Ergebnis.
- Authentifizierung, Rate‑Limit, Logging.
## Architektur
```
[ Client ] ⇄ HTTP/JSON ⇄ [ FastAPI AI‑Server (GPU) ]
```
- Server hostet Inferenz-Endpunkte; Client sendet Aufträge.
- Optional: WebSocket für Streaming‑Logs/Progress.
## Technologie‑Stack
- **Server**: Python 3.10+, FastAPI, Uvicorn, PyTorch, diffusers (Stable Diffusion), PIL.
- **Client**: Python CLI (requests / httpx) oder schlankes Web‑UI.
- **Persistenz**: SQLite oder JSON Log; Artefakte auf Disk/S3‑ähnlich.
- **Sicherheit**: API‑Key (env/secret file), CORS policy, Rate‑Limit (slowapi), timeouts.
## API‑Spezifikation (v0)
### POST `/v1/generate-image`
Request JSON:
```json
{
"api_key": "<KEY>",
"prompt": "a futuristic city skyline at night",
"steps": 30,
"guidance": 7.5,
"width": 512,
"height": 512,
"seed": 12345
}
```
Response JSON:
```json
{
"status": "ok",
"job_id": "2025-09-26-000123",
"image_base64": "data:image/png;base64,....",
"duration_ms": 2180,
"gpu_seconds": 1.9
}
```
### GET `/v1/health`
- Rückgabe von `{ "status": "ok", "gpu": "RTX 2060", "model": "SD1.5" }`.
## Server‑Ablauf (Pseudocode)
```python
@app.post("/v1/generate-image")
def gen(req: Request):
assert check_api_key(req.api_key)
rate_limit(req.key)
t0 = now()
img = stable_diffusion.generate(prompt=req.prompt, ...)
log_job(user=req.key, gpu_seconds=measure_gpu(), ok=True)
return {"status":"ok", "image_base64": b64(img), "duration_ms": ms_since(t0)}
```
## Betriebliche Aspekte
- **Logging**: strukturierte Logs (JSON) inkl. Prompt‑Hash, Laufzeit, GPU‑Sekunden, Exit‑Code.
- **Observability**: Prometheus‑/OpenTelemetry‑Metriken (req/sec, p95 Latenz, VRAM‑Nutzung).
- **Fehler**: Retry‑Policy (idempotent), Graceful shutdown, Max batch/queue size.
- **Sicherheit**: Input‑Sanitization, Upload‑Limits, tmp‑Verzeichnis säubern.
## Setup‑Schritte (Linux, NVIDIA RTX 2060)
```bash
sudo apt update && sudo apt install -y python3-venv git
python3 -m venv venv && source venv/bin/activate
pip install fastapi uvicorn[standard] torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
pip install diffusers transformers accelerate pillow safetensors xformers slowapi httpx
# Start
uvicorn app:app --host 127.0.0.2 --port 8000 --workers 1
```
## Akzeptanzkriterien
- `GET /v1/health` liefert GPU/Model‑Infos.
- `POST /v1/generate-image` liefert innerhalb < 5s ein 512×512 PNG (bei RTX 2060, SD‑1.5, \~30 steps).
- Logs enthalten pro Job mindestens: job\_id, duration\_ms, gpu\_seconds, bytes\_out.
## Nächste Schritte zu Stage 2
- Job‑Quittungsschema definieren (hashbare Receipt für On‑Chain‑Mint später).
- Einheit „Compute‑Einheit“ festlegen (z. B. GPU‑Sekunden, Token/Prompt).
- Nonce/Signatur im Request zur späteren On‑Chain‑Verifikation.
---
## Kurze Stage‑2/3/4‑Vorschau (Implementierungsnotizen)
- **Stage 2 (Blockchain)**: Smart Contract mit `mint(provider, units, receipt_hash)`, Off‑Chain‑Orakel/Attester.
- **Stage 3 (Hub)**: Scheduler (priority, price, reputation), Sharding großer Jobs, Konsistenz‑Checks, Reward‑Split.
- **Stage 4 (Marketplace)**: Orderbook, KYC/Compliance Layer (jurisdictions), Custody‑freie Wallet‑Anbindung.
## Quellen (Auszug)
- Ethereum Smart Contracts: [https://ethereum.org/en/smart-contracts/](https://ethereum.org/en/smart-contracts/)
- PoS Überblick: [https://ethereum.org/en/developers/docs/consensus-mechanisms/pos/](https://ethereum.org/en/developers/docs/consensus-mechanisms/pos/)
- PyTorch Deploy: [https://pytorch.org/tutorials/](https://pytorch.org/tutorials/)
- FastAPI Docs: [https://fastapi.tiangolo.com/](https://fastapi.tiangolo.com/)
---
# Stage 1 – Referenz‑Implementierung (Code)
## `.env`
```
API_KEY=CHANGE_ME_SUPERSECRET
MODEL_ID=runwayml/stable-diffusion-v1-5
BIND_HOST=127.0.0.2
BIND_PORT=8000
```
## `requirements.txt`
```
fastapi
uvicorn[standard]
httpx
pydantic
python-dotenv
slowapi
pillow
torch
torchvision
torchaudio
transformers
diffusers
accelerate
safetensors
xformers
```
## `server.py`
```python
import base64, io, os, time, hashlib
from functools import lru_cache
from typing import Optional
from dotenv import load_dotenv
from fastapi import FastAPI, HTTPException, Request
from pydantic import BaseModel, Field
from slowapi import Limiter
from slowapi.util import get_remote_address
from PIL import Image
load_dotenv()
API_KEY = os.getenv("API_KEY", "CHANGE_ME_SUPERSECRET")
MODEL_ID = os.getenv("MODEL_ID", "runwayml/stable-diffusion-v1-5")
app = FastAPI(title="AITBC Stage1 Server", version="0.1.0")
limiter = Limiter(key_func=get_remote_address)
class GenRequest(BaseModel):
api_key: str
prompt: str
steps: int = Field(30, ge=5, le=100)
guidance: float = Field(7.5, ge=0, le=25)
width: int = Field(512, ge=256, le=1024)
height: int = Field(512, ge=256, le=1024)
seed: Optional[int] = None
@lru_cache(maxsize=1)
def load_pipeline():
from diffusers import StableDiffusionPipeline
import torch
pipe = StableDiffusionPipeline.from_pretrained(MODEL_ID, torch_dtype=torch.float16, safety_checker=None)
pipe = pipe.to("cuda" if torch.cuda.is_available() else "cpu")
pipe.enable_attention_slicing()
return pipe
@app.get("/v1/health")
def health():
gpu = os.getenv("NVIDIA_VISIBLE_DEVICES", "auto")
return {"status": "ok", "gpu": gpu, "model": MODEL_ID}
@app.post("/v1/generate-image")
@limiter.limit("10/minute")
def generate(req: GenRequest, request: Request):
if req.api_key != API_KEY:
raise HTTPException(status_code=401, detail="invalid api_key")
t0 = time.time()
pipe = load_pipeline()
generator = None
if req.seed is not None:
import torch
generator = torch.Generator(device=pipe.device).manual_seed(int(req.seed))
result = pipe(req.prompt, num_inference_steps=req.steps, guidance_scale=req.guidance, width=req.width, height=req.height, generator=generator)
img: Image.Image = result.images[0]
buf = io.BytesIO()
img.save(buf, format="PNG")
b64 = base64.b64encode(buf.getvalue()).decode("ascii")
dur_ms = int((time.time() - t0) * 1000)
job_id = hashlib.sha256(f"{t0}-{req.prompt[:64]}".encode()).hexdigest()[:16]
log_line = {"job_id": job_id, "duration_ms": dur_ms, "bytes_out": len(b64), "prompt_hash": hashlib.sha256(req.prompt.encode()).hexdigest()}
print(log_line, flush=True)
return {"status": "ok", "job_id": job_id, "image_base64": f"data:image/png;base64,{b64}", "duration_ms": dur_ms}
if __name__ == "__main__":
import uvicorn, os
uvicorn.run("server:app", host=os.getenv("BIND_HOST", "127.0.0.2"), port=int(os.getenv("BIND_PORT", "8000")), reload=False)
```
## `client.py`
```python
import base64, json, os
import httpx
API = os.getenv("API", "http://localhost:8000")
API_KEY = os.getenv("API_KEY", "CHANGE_ME_SUPERSECRET")
payload = {
"api_key": API_KEY,
"prompt": "a futuristic city skyline at night, ultra detailed, neon",
"steps": 30,
"guidance": 7.5,
"width": 512,
"height": 512,
}
r = httpx.post(f"{API}/v1/generate-image", json=payload, timeout=120)
r.raise_for_status()
resp = r.json()
print("job:", resp.get("job_id"), "duration_ms:", resp.get("duration_ms"))
img_b64 = resp["image_base64"].split(",",1)[1]
open("out.png","wb").write(base64.b64decode(img_b64))
print("saved out.png")
```
---
# OpenAPI 3.1 Spezifikation (Stage 1)
```yaml
openapi: 3.1.0
info:
title: AITBC Stage1 Server
version: 0.1.0
servers:
- url: http://localhost:8000
paths:
/v1/health:
get:
summary: Health check
responses:
'200':
description: OK
content:
application/json:
schema:
type: object
properties:
status: { type: string }
gpu: { type: string }
model: { type: string }
/v1/generate-image:
post:
summary: Generate image from text prompt
requestBody:
required: true
content:
application/json:
schema:
type: object
required: [api_key, prompt]
properties:
api_key: { type: string }
prompt: { type: string }
steps: { type: integer, minimum: 5, maximum: 100, default: 30 }
guidance: { type: number, minimum: 0, maximum: 25, default: 7.5 }
width: { type: integer, minimum: 256, maximum: 1024, default: 512 }
height: { type: integer, minimum: 256, maximum: 1024, default: 512 }
seed: { type: integer, nullable: true }
responses:
'200':
description: Image generated
content:
application/json:
schema:
type: object
properties:
status: { type: string }
job_id: { type: string }
image_base64: { type: string }
duration_ms: { type: integer }
```
---
# Stage 2 – Receipt‑/Quittungs‑Schema & Hashing
## JSON Receipt (off‑chain, signierbar)
```json
{
"job_id": "2025-09-26-000123",
"provider": "0xProviderAddress",
"client": "client_public_key_or_id",
"units": 1.90,
"unit_type": "gpu_seconds",
"model": "runwayml/stable-diffusion-v1-5",
"prompt_hash": "sha256:...",
"started_at": 1695720000,
"finished_at": 1695720002,
"artifact_sha256": "...",
"nonce": "b7f3...",
"hub_id": "optional-hub",
"chain_id": 11155111
}
```
## Hashing
- Kanonische Serialisierung (minified JSON, Felder in alphabetischer Reihenfolge).
- `receipt_hash = keccak256(bytes(serialized))` (für EVM‑Kompatibilität) **oder** `sha256` falls kettenagnostisch.
## Signatur
- Signatur über `receipt_hash`:
- **secp256k1/ECDSA** (Ethereum‑kompatibel, EIP‑191/EIP‑712) **oder** Ed25519 (falls Off‑Chain‑Attester bevorzugt).
- Felder zur Verifikation on‑chain: `provider`, `units`, `receipt_hash`, `signature`.
## Double‑Mint‑Prevention
- Smart Contract speichert `used[receipt_hash] = true` nach erfolgreichem Mint.
---
# Stage 2 – Smart‑Contract‑Skeleton (Solidity)
```solidity
// SPDX-License-Identifier: MIT
pragma solidity ^0.8.24;
interface IERC20Mint {
function mint(address to, uint256 amount) external;
}
contract AITokenMinter {
IERC20Mint public token;
address public attester; // Off‑chain Hub/Oracle, darf Quittungen bescheinigen
mapping(bytes32 => bool) public usedReceipt; // receipt_hash → consumed
event Minted(address indexed provider, uint256 units, bytes32 receiptHash);
event AttesterChanged(address indexed oldA, address indexed newA);
constructor(address _token, address _attester) {
token = IERC20Mint(_token);
attester = _attester;
}
function setAttester(address _attester) external /* add access control */ {
emit AttesterChanged(attester, _attester);
attester = _attester;
}
function mintWithReceipt(
address provider,
uint256 units,
bytes32 receiptHash,
bytes calldata attesterSig
) external {
require(!usedReceipt[receiptHash], "receipt used");
// Verify attester signature over EIP‑191 style message: keccak256(abi.encode(provider, units, receiptHash))
bytes32 msgHash = keccak256(abi.encode(provider, units, receiptHash));
require(_recover(msgHash, attesterSig) == attester, "bad sig");
usedReceipt[receiptHash] = true;
token.mint(provider, units);
emit Minted(provider, units, receiptHash);
}
function _recover(bytes32 msgHash, bytes memory sig) internal pure returns (address) {
bytes32 ethHash = keccak256(abi.encodePacked("\x19Ethereum Signed Message:\n32", msgHash));
(bytes32 r, bytes32 s, uint8 v) = _split(sig);
return ecrecover(ethHash, v, r, s);
}
function _split(bytes memory sig) internal pure returns (bytes32 r, bytes32 s, uint8 v) {
require(sig.length == 65, "sig len");
assembly {
r := mload(add(sig, 32))
s := mload(add(sig, 64))
v := byte(0, mload(add(sig, 96)))
}
}
}
```
> Hinweis: In Produktion Access‑Control (Ownable/Role‑based), Pausable, Reentrancy‑Guard und EIP‑712‑Typed‑Data einführen.
---
# Stage 3 – Hub‑Spezifikation (Kurz)
- **Scheduler**: Round‑robin + Preis/VRAM‑Filter; optional Reputation.
- **Split**: Große Jobs shard‑en; `units` aus Subjobs aggregieren.
- **Verification**: Stichprobenhafte Re‑Auswertung / Konsistenz‑Hashes.
- **Payout**: Proportionale Verteilung; ein Receipt je Gesamtjob.
# Stage 4 – Marketplace (Kurz)
- **Orderbook** (limit/market), **Wallet‑Connect**, Non‑custodial.
- **KYC/Compliance** optional je Jurisdiktion.
- **Reputation/SLAs** on‑/off‑chain verknüpfbar.
---
# Deployment ohne Docker (Bare‑Metal / VM)
## Voraussetzungen
- Ubuntu/Debian mit NVIDIA Treiber (535+) und CUDA/CuDNN passend zur PyTorch‑Version.
- Python 3.10+ und `python3-venv`.
- Öffentliche Ports: **8000/tcp** (API) – optional Reverse‑Proxy auf 80/443.
## Treiber & CUDA (Kurz)
```bash
# NVIDIA Treiber (Beispiel Ubuntu)
sudo apt update && sudo apt install -y nvidia-driver-535
# Nach Reboot: nvidia-smi prüfen
# PyTorch bringt eigenes CUDA-Toolkit über Wheels (empfohlen). Kein System-CUDA zwingend nötig.
```
## Benutzer & Verzeichnisstruktur
```bash
sudo useradd -m -r -s /bin/bash aitbc
sudo -u aitbc mkdir -p /opt/aitbc/app /opt/aitbc/logs
# Code nach /opt/aitbc/app kopieren
```
## Virtualenv & Abhängigkeiten
```bash
sudo -u aitbc bash -lc '
cd /opt/aitbc/app && python3 -m venv venv && source venv/bin/activate && \
pip install --upgrade pip && pip install -r requirements.txt
'
```
## Konfiguration (.env)
```
API_KEY=<GEHEIM>
MODEL_ID=runwayml/stable-diffusion-v1-5
BIND_HOST=127.0.0.1 # hinter Reverse Proxy
BIND_PORT=8000
```
## Systemd‑Unit (Uvicorn)
`/etc/systemd/system/aitbc.service`
```ini
[Unit]
Description=AITBC Stage1 FastAPI Server
After=network-online.target
Wants=network-online.target
[Service]
User=aitbc
Group=aitbc
WorkingDirectory=/opt/aitbc/app
EnvironmentFile=/opt/aitbc/app/.env
ExecStart=/opt/aitbc/app/venv/bin/python -m uvicorn server:app --host ${BIND_HOST} --port ${BIND_PORT} --workers 1
Restart=always
RestartSec=3
# GPU/VRAM limits optional per nvidia-visible-devices
StandardOutput=append:/opt/aitbc/logs/stdout.log
StandardError=append:/opt/aitbc/logs/stderr.log
[Install]
WantedBy=multi-user.target
```
Aktivieren & Starten:
```bash
sudo systemctl daemon-reload
sudo systemctl enable --now aitbc.service
sudo systemctl status aitbc.service
```
## Reverse Proxy (optional, ohne Docker)
### Nginx (TLS via Certbot)
```bash
sudo apt install -y nginx certbot python3-certbot-nginx
sudo tee /etc/nginx/sites-available/aitbc <<'NG'
server {
listen 80; server_name example.com;
location / {
proxy_pass http://127.0.0.1:8000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
NG
sudo ln -s /etc/nginx/sites-available/aitbc /etc/nginx/sites-enabled/aitbc
sudo nginx -t && sudo systemctl reload nginx
sudo certbot --nginx -d example.com
```
## Firewall/Netzwerk
```bash
sudo ufw allow OpenSSH
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw enable
```
## Monitoring ohne Docker
- **systemd**: `journalctl -u aitbc -f`
- **Metriken**: Prometheus Node Exporter, `nvtop`/`nvidia-smi dmon` für GPU.
- **Alerts**: systemd `Restart=always`, optional Monit.
## Zero‑Downtime Update (Rolling ohne Container)
```bash
sudo systemctl stop aitbc
sudo -u aitbc bash -lc 'cd /opt/aitbc/app && git pull && source venv/bin/activate && pip install -r requirements.txt'
sudo systemctl start aitbc
```
## Härtung & Best Practices
- Starker API‑Key, IP‑basierte Allow‑List am Reverse‑Proxy.
- Rate‑Limit (slowapi) aktivieren; Request‑Body‑Limits setzen (`client_max_body_size`).
- Temporäre Dateien regelmäßig bereinigen (systemd tmpfiles).
- Separate GPU‑Workstation vs. Edge‑Expose (API hinter Proxy).
> Hinweis: Diese Anleitung vermeidet bewusst jeglichen Docker‑Einsatz und nutzt **systemd + venv** für einen reproduzierbaren, schlanken Betrieb.

392
docs/reference/bootstrap/blockchain_node.md Normal file
View File

@ -0,0 +1,392 @@
# blockchain-node/ — Minimal Chain (asset-backed by compute)
## 0) TL;DR boot path for Windsurf
1. Create the service: `apps/blockchain-node` (Python, FastAPI, asyncio, uvicorn).
2. Data layer: `sqlite` via `SQLModel` (later: PostgreSQL).
3. P2P: WebSocket gossip (lib: `websockets`) with a simple overlay (peer table + heartbeats).
4. Consensus (MVP): **PoA single-author** (devnet) → upgrade to **Compute-Backed Proof (CBP)** after coordinator & miner telemetry are wired.
5. Block content: **ComputeReceipts** = “proofs of delivered AI work” signed by miners, plus standard transfers.
6. Minting: AIToken minted per verified compute unit (e.g., `1 AIT = 1,000 token-ops` — calibrate later).
7. REST RPC: `/rpc/*` for clients & coordinator; `/p2p/*` for peers; `/admin/*` for node ops.
8. Ship a `devnet` script that starts: 1 bootstrap node, 1 coordinator-api mock, 1 miner mock, 1 client demo.
---
## 1) Goal & Scope
- Provide a **minimal, testable blockchain node** that issues AITokens **only** when real compute was delivered (asset-backed).
- Easy to run, easy to reset, deterministic devnet.
- Strong boundaries so **coordinator-api** (job orchestration) and **miner-node** (workers) can integrate quickly.
Out of scope (MVP):
- Smart contracts VM.
- Sharding/advanced networking.
- Custodial wallets. (Use local keypairs for dev.)
---
## 2) Core Concepts
### 2.1 Actors
- **Client**: pays AITokens to request compute jobs.
- **Coordinator**: matches jobs ↔ miners; returns signed receipts.
- **Miner**: executes jobs; produces **ComputeReceipt** signed with miner key.
- **Blockchain Node**: validates receipts, mints AIT for miners, tracks balances, finalizes blocks.
### 2.2 Asset-Backed Minting
- Unit of account: **AIToken (AIT)**.
- A miner earns AIT when a **ComputeReceipt** is included in a block.
- A receipt is valid iff:
1) Its `job_id` exists in coordinator logs,
2) `client_payment_tx` covers the quoted price,
3) `miner_sig` over `(job_id, hash(output_meta), compute_units, price, nonce)` is valid,
4) Not previously claimed (`receipt_id` unique).
---
## 3) Minimal Architecture
```
blockchain-node/
├─ src/
│ ├─ main.py # FastAPI entry
│ ├─ p2p.py # WS gossip, peer table, block relay
│ ├─ consensus.py # PoA/CBP state machine
│ ├─ types.py # dataclasses / pydantic models
│ ├─ state.py # DB access (SQLModel), UTXO/Account
│ ├─ mempool.py # tx pool (transfers + receipts)
│ ├─ crypto.py # ed25519 keys, signatures, hashing
│ ├─ receipts.py # receipt validation (with coordinator)
│ ├─ blocks.py # block build/verify, difficulty stub
│ ├─ rpc.py # REST/RPC routes for clients & ops
│ └─ settings.py # env config
├─ tests/
│ └─ ... # unit & integration tests
├─ scripts/
│ ├─ devnet_up.sh # run bootstrap node + mocks
│ └─ keygen.py # create node/miner/client keys
├─ README.md
└─ requirements.txt
```
---
## 4) Data Model (SQLModel)
### 4.1 Tables
- `blocks(id, parent_id, height, timestamp, proposer, tx_count, hash, state_root, sig)`
- `tx(id, block_id, type, payload_json, sender, nonce, fee, sig, hash, status)`
- `accounts(address, balance, nonce, pubkey)`
- `receipts(receipt_id, job_id, client_addr, miner_addr, compute_units, price, output_hash, miner_sig, status)`
- `peers(node_id, addr, last_seen, score)`
- `params(key, value)` — chain config (mint ratios, fee rate, etc.)
### 4.2 TX Types
- `TRANSFER`: move AIT from A → B
- `RECEIPT_CLAIM`: include a **ComputeReceipt**; mints to miner and settles client escrow
- `STAKE/UNSTAKE` (later)
- `PARAM_UPDATE` (PoA only, gated by admin key for devnet)
---
## 5) Block Format (JSON)
```json
{
"parent": "<block_hash>",
"height": 123,
"timestamp": 1699999999,
"proposer": "<node_address>",
"txs": ["<tx_hash>", "..."],
"stateRoot": "<merkle_root_after_block>",
"sig": "<proposer_signature_over_header>"
}
```
Header sign bytes = `hash(parent|height|timestamp|proposer|stateRoot)`
---
## 6) Consensus
### 6.1 MVP: PoA (Single Author)
- One configured `PROPOSER_KEY` creates blocks at fixed interval (e.g., 2s).
- Honest mode only for devnet; finality by canonical longest/height rule.
### 6.2 Upgrade: **Compute-Backed Proof (CBP)**
- Each block’s **work score** = total `compute_units` in included receipts.
- Proposer election = weighted round-robin by recent work score and stake (later).
- Slashing: submitting invalid receipts reduces score; repeated offenses → temp ban.
---
## 7) Receipt Validation (Coordinator Check)
`receipts.py` performs:
1) **Coordinator attestation** (HTTP call to coordinator-api):
- `/attest/receipt` with `job_id`, `client`, `miner`, `price`, `compute_units`, `output_hash`.
- Returns `{exists: bool, paid: bool, not_double_spent: bool, quote: {...}}`.
2) **Signature check**: verify `miner_sig` with miner’s `pubkey`.
3) **Economic checks**: ensure `client_payment_tx` exists & covers `price + fee`.
> For devnet without live coordinator, ship a **mock** that returns deterministic attestation for known `job_id` ranges.
---
## 8) Fees & Minting
- **Fee model (MVP)**: `fee = base_fee + k * payload_size`.
- **Minting**:
- Miner gets: `mint = compute_units * MINT_PER_UNIT`.
- Coordinator gets: `coord_cut = mint * COORDINATOR_RATIO`.
- Chain treasury (optional): small %, configurable in `params`.
---
## 9) RPC Surface (FastAPI)
### 9.1 Public
- `POST /rpc/sendTx` → `{txHash}`
- `GET /rpc/getTx/{txHash}` → `{status, receipt}`
- `GET /rpc/getBlock/{heightOrHash}`
- `GET /rpc/getHead` → `{height, hash}`
- `GET /rpc/getBalance/{address}` → `{balance, nonce}`
- `POST /rpc/estimateFee` → `{fee}`
### 9.2 Coordinator-facing
- `POST /rpc/submitReceipt` (alias of `sendTx` with type `RECEIPT_CLAIM`)
- `POST /rpc/attest` (devnet mock only)
### 9.3 Admin (devnet)
- `POST /admin/paramSet` (PoA only)
- `POST /admin/peers/add` `{addr}`
- `POST /admin/mintFaucet` `{address, amount}` (devnet)
### 9.4 P2P (WS)
- `GET /p2p/peers` → list
- `WS /p2p/ws` → subscribe to gossip: `{"type":"block"|"tx"|"peer","data":...}`
---
## 10) Keys & Crypto
- **ed25519** for account & node keys.
- Address = `bech32(hrp="ait", sha256(pubkey)[0:20])`.
- Sign bytes:
- TX: `hash(type|sender|nonce|fee|payload_json_canonical)`
- Block: header hash as above.
Ship `scripts/keygen.py` for dev use.
---
## 11) Mempool Rules
- Accept if:
- `sig` valid,
- `nonce == account.nonce + 1`,
- `fee >= minFee`,
- For `RECEIPT_CLAIM`: passes `receipts.validate()` *optimistically* (soft-accept), then **revalidate** at block time.
Replacement: higher-fee replaces same `(sender, nonce)`.
---
## 12) Node Lifecycle
**Start:**
1) Load config, open DB, ensure genesis.
2) Connect to bootstrap peers (if any).
3) Start RPC (FastAPI) + P2P WS server.
4) Start block proposer (if PoA key present).
5) Start peer heartbeats + gossip loops.
**Shutdown:**
- Graceful: flush mempool snapshot, close DB.
---
## 13) Genesis
- `genesis.json`:
- `chain_id`, `timestamp`, `accounts` (faucet), `params` (mint ratios, base fee), `authorities` (PoA keys).
Provide `scripts/make_genesis.py`.
---
## 14) Devnet: End-to-End Demo
### 14.1 Components
- **blockchain-node** (this repo)
- **coordinator-api (mock)**: `/attest/receipt` returns valid for `job_id` in `[1..1_000_000]`
- **miner-node (mock)**: posts `RECEIPT_CLAIM` for synthetic jobs
- **client-web (demo)**: sends `TRANSFER` & displays balances
### 14.2 Flow
1) Client pays `price` to escrow address (coordinator).
2) Miner executes job; coordinator verifies output.
3) Miner submits **ComputeReceipt** → included in next block.
4) Mint AIT to miner; escrow settles; client charged.
---
## 15) Testing Strategy
### 15.1 Unit
- `crypto`: keygen, sign/verify, address derivation
- `state`: balances, nonce, persistence
- `receipts`: signature + coordinator mock
- `blocks`: header hash, stateRoot
### 15.2 Integration
- Single node PoA: produce N blocks; submit transfers/receipts; assert balances.
- Two nodes P2P: block/tx relay; head convergence.
### 15.3 Property tests
- Nonce monotonicity; no double-spend; unique receipts.
---
## 16) Observability
- Structured logs (JSON) with `component`, `event`, `height`, `latency_ms`.
- `/rpc/metrics` (Prometheus format) — block time, mempool size, peers.
---
## 17) Configuration (ENV)
- `CHAIN_ID=ait-devnet`
- `DB_PATH=./data/chain.db`
- `P2P_BIND=127.0.0.2:7070`
- `RPC_BIND=127.0.0.2:8080`
- `BOOTSTRAP_PEERS=ws://host:7070,...`
- `PROPOSER_KEY=...` (optional for non-authors)
- `MINT_PER_UNIT=1000`
- `COORDINATOR_RATIO=0.05`
Provide `.env.example`.
---
## 18) Minimal API Payloads
### 18.1 TRANSFER
```json
{
"type": "TRANSFER",
"sender": "ait1...",
"nonce": 1,
"fee": 10,
"payload": {"to":"ait1...","amount":12345},
"sig": "<ed25519>"
}
```
### 18.2 RECEIPT_CLAIM
```json
{
"type": "RECEIPT_CLAIM",
"sender": "ait1miner...",
"nonce": 7,
"fee": 50,
"payload": {
"receipt_id": "rcpt_7f3a...",
"job_id": "job_42",
"client_addr": "ait1client...",
"miner_addr": "ait1miner...",
"compute_units": 2500,
"price": 50000,
"output_hash": "sha256:abcd...",
"miner_sig": "<sig_over_core_fields>"
},
"sig": "<miner_account_sig>"
}
```
---
## 19) Security Notes (MVP)
- Devnet PoA means trust in proposer; do **not** expose to internet without firewall.
- Enforce coordinator host allowlist for attest calls.
- Rate-limit `/rpc/sendTx`.
---
## 20) Roadmap
1) ✅ PoA devnet with receipts.
2) 🔜 CBP proposer selection from rolling work score.
3) 🔜 Stake & slashing.
4) 🔜 Replace SQLite with PostgreSQL.
5) 🔜 Snapshots & fast-sync.
6) 🔜 Light client (SPV of receipts & balances).
---
## 21) Developer Tasks (Windsurf Order)
1) **Scaffold** project & `requirements.txt`:
- `fastapi`, `uvicorn[standard]`, `sqlmodel`, `pydantic`, `websockets`, `pyyaml`, `python-dotenv`, `ed25519`, `orjson`.
2) **Implement**:
- `crypto.py`, `types.py`, `state.py`.
- `rpc.py` (public routes).
- `mempool.py`.
- `blocks.py` (build/validate).
- `consensus.py` (PoA tick).
- `p2p.py` (WS server + simple gossip).
- `receipts.py` (mock coordinator).
3) **Wire** `main.py`:
- Start RPC, P2P, PoA loops.
4) **Scripts**:
- `scripts/keygen.py`, `scripts/make_genesis.py`, `scripts/devnet_up.sh`.
5) **Tests**:
- Add unit + an integration test that mints on a receipt.
6) **Docs**:
- Update `README.md` with curl examples.
---
## 22) Curl Snippets (Dev)
- Faucet (dev only):
```bash
curl -sX POST localhost:8080/admin/mintFaucet -H 'content-type: application/json' \
-d '{"address":"ait1client...","amount":1000000}'
```
- Transfer:
```bash
curl -sX POST localhost:8080/rpc/sendTx -H 'content-type: application/json' \
-d @transfer.json
```
- Submit Receipt:
```bash
curl -sX POST localhost:8080/rpc/submitReceipt -H 'content-type: application/json' \
-d @receipt_claim.json
```
---
## 23) Definition of Done (MVP)
- Node produces blocks on PoA.
- Can transfer AIT between accounts.
- Can submit a valid **ComputeReceipt** → miner balance increases; escrow decreases.
- Two nodes converge on same head via P2P.
- Basic metrics exposed.
---
## 24) Next Files to Create
- `src/main.py`
- `src/crypto.py`
- `src/types.py`
- `src/state.py`
- `src/mempool.py`
- `src/blocks.py`
- `src/consensus.py`
- `src/p2p.py`
- `src/receipts.py`
- `src/rpc.py`
- `scripts/keygen.py`, `scripts/devnet_up.sh`
- `.env.example`, `README.md`, `requirements.txt`

438
docs/reference/bootstrap/coordinator_api.md Normal file
View File

@ -0,0 +1,438 @@
# coordinator-api.md
Central API that orchestrates **jobs** from clients to **miners**, tracks lifecycle, validates results, and (later) settles AITokens.
**Stage 1 (MVP):** no blockchain, no pool hub — just client ⇄ coordinator ⇄ miner.
## 1) Goals & Non-Goals
**Goals (MVP)**
- Accept computation jobs from clients.
- Match jobs to eligible miners.
- Track job state machine (QUEUED → RUNNING → COMPLETED/FAILED/CANCELED/EXPIRED).
- Stream results back to clients; store minimal metadata.
- Provide a clean, typed API (OpenAPI/Swagger).
- Simple auth (API keys) + idempotency + rate limits.
- Minimal persistence (SQLite/Postgres) with straightforward SQL (no migrations tooling).
**Non-Goals (MVP)**
- Token minting/settlement (stub hooks only).
- Miner marketplace, staking, slashing, reputation (placeholders).
- Pool hub coordination (future stage).
---
## 2) Tech Stack
- **Python 3.12**, **FastAPI**, **Uvicorn**
- **Pydantic** for schemas
- **SQL** via `sqlite3` or Postgres (user can switch later)
- **Redis (optional)** for queueing; MVP can start with in-DB FIFO
- **HTTP + WebSocket** (for miner heartbeats / job streaming)
> Debian 12 target. Run under **systemd** later.
---
## 3) Directory Layout (WindSurf Workspace)
```
coordinator-api/
├─ app/
│ ├─ main.py # FastAPI init, lifespan, routers
│ ├─ config.py # env parsing
│ ├─ deps.py # auth, rate-limit deps
│ ├─ db.py # simple DB layer (sqlite/postgres)
│ ├─ matching.py # job→miner selection
│ ├─ queue.py # enqueue/dequeue logic
│ ├─ settlement.py # stubs for token accounting
│ ├─ models.py # Pydantic request/response schemas
│ ├─ states.py # state machine + transitions
│ ├─ routers/
│ │ ├─ client.py # /v1/jobs (submit/status/result/cancel)
│ │ ├─ miner.py # /v1/miners (register/heartbeat/poll/submit/fail)
│ │ └─ admin.py # /v1/admin (stats)
│ └─ ws/
│ ├─ miner.py # WS for miner heartbeats / job stream (optional)
│ └─ client.py # WS for client result stream (optional)
├─ tests/
│ ├─ test_client_flow.http # REST client flow (HTTP file)
│ └─ test_miner_flow.http # REST miner flow
├─ .env.example
├─ pyproject.toml
└─ README.md
```
---
## 4) Environment (.env)
```
APP_ENV=dev
APP_HOST=127.0.0.1
APP_PORT=8011
DATABASE_URL=sqlite:///./coordinator.db
# or: DATABASE_URL=postgresql://user:pass@localhost:5432/aitbc
# Auth
CLIENT_API_KEYS=client_dev_key_1,client_dev_key_2
MINER_API_KEYS=miner_dev_key_1,miner_dev_key_2
ADMIN_API_KEYS=admin_dev_key_1
# Security
HMAC_SECRET=change_me
ALLOW_ORIGINS=*
# Queue
JOB_TTL_SECONDS=900
HEARTBEAT_INTERVAL_SECONDS=10
HEARTBEAT_TIMEOUT_SECONDS=30
```
---
## 5) Core Data Model (conceptual)
**Job**
- `job_id` (uuid)
- `client_id` (from API key)
- `requested_at`, `expires_at`
- `payload` (opaque JSON / bytes ref)
- `constraints` (gpu, cuda, mem, model, max_price, region)
- `state` (QUEUED|RUNNING|COMPLETED|FAILED|CANCELED|EXPIRED)
- `assigned_miner_id` (nullable)
- `result_ref` (blob path / inline json)
- `error` (nullable)
- `cost_estimate` (optional)
**Miner**
- `miner_id` (from API key)
- `capabilities` (gpu, cuda, vram, models[], region)
- `heartbeat_at`
- `status` (ONLINE|OFFLINE|DRAINING)
- `concurrency` (int), `inflight` (int)
**WorkerSession**
- `session_id`, `miner_id`, `job_id`, `started_at`, `ended_at`, `exit_reason`
---
## 6) State Machine
```
QUEUED
-> RUNNING (assigned to miner)
-> CANCELED (client)
-> EXPIRED (ttl)
RUNNING
-> COMPLETED (miner submit_result)
-> FAILED (miner fail / timeout)
-> CANCELED (client)
```
---
## 7) Matching (MVP)
- Filter ONLINE miners by **capabilities** & **region**
- Prefer lowest `inflight` (simple load)
- Tiebreak by earliest `heartbeat_at` or random
- Lock job row → assign → return to miner
---
## 8) Auth & Rate Limits
- **API keys** via `X-Api-Key` header for `client`, `miner`, `admin`.
- Optional **HMAC** (`X-Signature`) over body with `HMAC_SECRET`.
- **Idempotency**: clients send `Idempotency-Key` on **POST /jobs**.
- **Rate limiting**: naive per-key window (e.g., 60 req / 60 s).
---
## 9) REST API
### Client
- `POST /v1/jobs`
- Create job. Returns `job_id`.
- `GET /v1/jobs/{job_id}`
- Job status & metadata.
- `GET /v1/jobs/{job_id}/result`
- Result (200 when ready, 425 if not ready).
- `POST /v1/jobs/{job_id}/cancel`
- Cancel if QUEUED or RUNNING (best effort).
### Miner
- `POST /v1/miners/register`
- Upsert miner capabilities; set ONLINE.
- `POST /v1/miners/heartbeat`
- Touch `heartbeat_at`, report `inflight`.
- `POST /v1/miners/poll`
- Long-poll for next job → returns a job or 204.
- `POST /v1/miners/{job_id}/start`
- Confirm start (optional if `poll` implies start).
- `POST /v1/miners/{job_id}/result`
- Submit result; transitions to COMPLETED.
- `POST /v1/miners/{job_id}/fail`
- Submit failure; transitions to FAILED.
- `POST /v1/miners/drain`
- Graceful stop accepting new jobs.
### Admin
- `GET /v1/admin/stats`
- Queue depth, miners online, success rates, avg latency.
- `GET /v1/admin/jobs?state=&limit=...`
- `GET /v1/admin/miners`
**Error Shape**
```json
{ "error": { "code": "STRING_CODE", "message": "human readable", "details": {} } }
```
Common codes: `UNAUTHORIZED_KEY`, `RATE_LIMITED`, `INVALID_PAYLOAD`, `NO_ELIGIBLE_MINER`, `JOB_NOT_FOUND`, `JOB_NOT_READY`, `CONFLICT_STATE`.
---
## 10) WebSockets (optional MVP+)
- `WS /v1/ws/miner?api_key=...`
- Server → miner: `job.assigned`
- Miner → server: `heartbeat`, `result`, `fail`
- `WS /v1/ws/client?job_id=...&api_key=...`
- Server → client: `state.changed`, `result.ready`
Fallback remains HTTP long-polling.
---
## 11) Result Storage
- **Inline JSON** if ≤ 1 MB.
- For larger payloads: store to disk path (e.g., `/var/lib/coordinator/results/{job_id}`) and return `result_ref`.
---
## 12) Settlement Hooks (stub)
`settlement.py` exposes:
- `record_usage(job, miner)`
- `quote_cost(job)`
Later wired to **AIToken** mint/transfer when blockchain lands.
---
## 13) Minimal FastAPI Skeleton
```python
# app/main.py
from fastapi import FastAPI
from app.routers import client, miner, admin
def create_app():
app = FastAPI(title="AITBC Coordinator API", version="0.1.0")
app.include_router(client.router, prefix="/v1")
app.include_router(miner.router, prefix="/v1")
app.include_router(admin.router, prefix="/v1")
return app
app = create_app()
```
```python
# app/models.py
from pydantic import BaseModel, Field
from typing import Any, Dict, List, Optional
class Constraints(BaseModel):
gpu: Optional[str] = None
cuda: Optional[str] = None
min_vram_gb: Optional[int] = None
models: Optional[List[str]] = None
region: Optional[str] = None
max_price: Optional[float] = None
class JobCreate(BaseModel):
payload: Dict[str, Any]
constraints: Constraints = Constraints()
ttl_seconds: int = 900
class JobView(BaseModel):
job_id: str
state: str
assigned_miner_id: Optional[str] = None
requested_at: str
expires_at: str
error: Optional[str] = None
class MinerRegister(BaseModel):
capabilities: Dict[str, Any]
concurrency: int = 1
region: Optional[str] = None
class PollRequest(BaseModel):
max_wait_seconds: int = 15
class AssignedJob(BaseModel):
job_id: str
payload: Dict[str, Any]
```
```python
# app/routers/client.py
from fastapi import APIRouter, Depends, HTTPException
from app.models import JobCreate, JobView
from app.deps import require_client_key
router = APIRouter(tags=["client"])
@router.post("/jobs", response_model=JobView)
def submit_job(req: JobCreate, client_id: str = Depends(require_client_key)):
# enqueue + return JobView
...
@router.get("/jobs/{job_id}", response_model=JobView)
def get_job(job_id: str, client_id: str = Depends(require_client_key)):
...
```
```python
# app/routers/miner.py
from fastapi import APIRouter, Depends
from app.models import MinerRegister, PollRequest, AssignedJob
from app.deps import require_miner_key
router = APIRouter(tags=["miner"])
@router.post("/miners/register")
def register(req: MinerRegister, miner_id: str = Depends(require_miner_key)):
...
@router.post("/miners/poll", response_model=AssignedJob, status_code=200)
def poll(req: PollRequest, miner_id: str = Depends(require_miner_key)):
# try dequeue, else 204
...
```
Run:
```bash
uvicorn app.main:app --host 127.0.0.1 --port 8011 --reload
```
OpenAPI: `http://127.0.0.1:8011/docs`
---
## 14) Matching & Queue Pseudocode
```python
def match_next_job(miner):
eligible = db.jobs.filter(
state="QUEUED",
constraints.satisfied_by(miner.capabilities)
).order_by("requested_at").first()
if not eligible:
return None
db.txn(lambda:
db.jobs.assign(eligible.job_id, miner.id) and
db.states.transition(eligible.job_id, "RUNNING")
)
return eligible
```
---
## 15) CURL Examples
**Client creates a job**
```bash
curl -sX POST http://127.0.0.1:8011/v1/jobs \
-H 'X-Api-Key: client_dev_key_1' \
-H 'Idempotency-Key: 7d4a...' \
-H 'Content-Type: application/json' \
-d '{
"payload": {"task":"sum","a":2,"b":3},
"constraints": {"gpu": null, "region": "eu-central"}
}'
```
**Miner registers + polls**
```bash
curl -sX POST http://127.0.0.1:8011/v1/miners/register \
-H 'X-Api-Key: miner_dev_key_1' \
-H 'Content-Type: application/json' \
-d '{"capabilities":{"gpu":"RTX4060Ti","cuda":"12.3","vram_gb":16},"concurrency":2,"region":"eu-central"}'
curl -i -sX POST http://127.0.0.1:8011/v1/miners/poll \
-H 'X-Api-Key: miner_dev_key_1' \
-H 'Content-Type: application/json' \
-d '{"max_wait_seconds":10}'
```
**Miner submits result**
```bash
curl -sX POST http://127.0.0.1:8011/v1/miners/<JOB_ID>/result \
-H 'X-Api-Key: miner_dev_key_1' \
-H 'Content-Type: application/json' \
-d '{"result":{"sum":5},"metrics":{"latency_ms":42}}'
```
**Client fetches result**
```bash
curl -s http://127.0.0.1:8011/v1/jobs/<JOB_ID>/result \
-H 'X-Api-Key: client_dev_key_1'
```
---
## 16) Timeouts & Health
- **Job TTL**: auto-expire QUEUED after `JOB_TTL_SECONDS`.
- **Heartbeat**: miners post every `HEARTBEAT_INTERVAL_SECONDS`.
- **Miner OFFLINE** if no heartbeat for `HEARTBEAT_TIMEOUT_SECONDS`.
- **Requeue**: RUNNING jobs from OFFLINE miners → back to QUEUED.
---
## 17) Security Notes
- Validate `payload` size & type; enforce max 1 MB inline.
- Optional **HMAC** signature for tamper detection.
- Sanitize/validate miner-reported capabilities.
- Log every state transition (append-only).
---
## 18) Admin Metrics (MVP)
- Queue depth, running count
- Miner online/offline, inflight
- P50/P95 job latency
- Success/fail/cancel rates (windowed)
---
## 19) Future Stages
- **Blockchain layer**: mint on verified compute; tie to `record_usage`.
- **Pool hub**: multi-coordinator balancing; marketplace.
- **Reputation**: miner scoring, penalty, slashing.
- **Bidding**: price discovery; client max price.
---
## 20) Checklist (WindSurf)
1. Create repo structure from section **3**.
2. Implement `.env` & `config.py` keys from **4**.
3. Add `models.py`, `states.py`, `deps.py` (auth, rate limit).
4. Implement DB tables for Job, Miner, WorkerSession.
5. Implement `queue.py` and `matching.py`.
6. Wire **client** and **miner** routers (MVP endpoints).
7. Add admin stats (basic counts).
8. Add OpenAPI tags, descriptions.
9. Add curl `.http` test files.
10. Systemd unit + Nginx proxy (later).

133
docs/reference/bootstrap/dirs.md Normal file
View File

@ -0,0 +1,133 @@
# AITBC Monorepo Directory Layout (Windsurf Workspace)
> One workspace for **all** AITBC elements (client · coordinator · miner · blockchain · pool‑hub · marketplace · wallet · docs · ops). No Docker required.
```
aitbc/
├─ .editorconfig
├─ .gitignore
├─ README.md # Top‑level overview, quickstart, workspace tasks
├─ LICENSE
├─ windsurf/ # Windsurf prompts, tasks, run configurations
│ ├─ prompts/ # High‑level task prompts for WS agents
│ ├─ tasks/ # Saved task flows / playbooks
│ └─ settings.json # Editor/workbench preferences for this repo
├─ scripts/ # CLI scripts (bash/python); dev + ops helpers
│ ├─ env/ # venv helpers (create, activate, pin)
│ ├─ dev/ # codegen, lint, format, typecheck wrappers
│ ├─ ops/ # backup, rotate logs, journalctl, users
│ └─ ci/ # sanity checks usable by CI (no runners assumed)
├─ configs/ # Centralized *.conf used by services
│ ├─ nginx/ # (optional) reverse proxy snippets (host‑level)
│ ├─ systemd/ # unit files for host services (no docker)
│ ├─ security/ # fail2ban, firewall/ipset lists, tls policy
│ └─ app/ # app‑level INI/YAML/TOML configs shared across apps
├─ docs/ # Markdown docs (specs, ADRs, guides)
│ ├─ 00-index.md
│ ├─ adr/ # Architecture Decision Records
│ ├─ specs/ # Protocol, API, tokenomics, flows
│ ├─ runbooks/ # Ops runbooks (rotate keys, restore, etc.)
│ └─ diagrams/ # draw.io/mermaid sources + exported PNG/SVG
├─ packages/ # Shared libraries (language‑specific)
│ ├─ py/ # Python packages (FastAPI, utils, protocol)
│ │ ├─ aitbc-core/ # Protocol models, validation, common types
│ │ ├─ aitbc-crypto/ # Key mgmt, signing, wallet primitives
│ │ ├─ aitbc-p2p/ # Node discovery, gossip, transport
│ │ ├─ aitbc-scheduler/ # Task slicing/merging, scoring, QoS
│ │ └─ aitbc-sdk/ # Client SDK for Python integrations
│ └─ js/ # Browser/Node shared libs
│ ├─ aitbc-sdk/ # Client SDK (fetch/ws), typings
│ └─ ui-widgets/ # Reusable UI bits for web apps
├─ apps/ # First‑class runnable services & UIs
│ ├─ client-web/ # Browser UI for users (requests, wallet, status)
│ │ ├─ public/ # static assets
│ │ ├─ src/
│ │ │ ├─ pages/
│ │ │ ├─ components/
│ │ │ ├─ lib/ # uses packages/js/aitbc-sdk
│ │ │ └─ styles/
│ │ └─ README.md
│ ├─ coordinator-api/ # Central API orchestrating jobs ↔ miners
│ │ ├─ src/
│ │ │ ├─ main.py # FastAPI entrypoint
│ │ │ ├─ routes/
│ │ │ ├─ services/ # matchmaking, accounting, rate‑limits
│ │ │ ├─ domain/ # job models, receipts, accounting entities
│ │ │ └─ storage/ # adapters (postgres, files, kv)
│ │ ├─ migrations/ # SQL snippets (no migration framework forced)
│ │ └─ README.md
│ ├─ miner-node/ # Worker node daemon for GPU/CPU tasks
│ │ ├─ src/
│ │ │ ├─ agent/ # job runner, sandbox mgmt, health probes
│ │ │ ├─ gpu/ # CUDA/OpenCL bindings (optional)
│ │ │ ├─ plugins/ # task kinds (LLM, ASR, vision, etc.)
│ │ │ └─ telemetry/ # metrics, logs, heartbeat
│ │ └─ README.md
│ ├─ wallet-daemon/ # Local wallet service (keys, signing, RPC)
│ │ ├─ src/
│ │ └─ README.md
│ ├─ blockchain-node/ # Minimal chain (asset‑backed by compute)
│ │ ├─ src/
│ │ │ ├─ consensus/
│ │ │ ├─ mempool/
│ │ │ ├─ ledger/ # state, balances, receipts linkage
│ │ │ └─ rpc/
│ │ └─ README.md
│ ├─ pool-hub/ # Client↔miners pool + matchmaking gateway
│ │ ├─ src/
│ │ └─ README.md
│ ├─ marketplace-web/ # Web app for offers, bids, stats
│ │ ├─ public/
│ │ ├─ src/
│ │ └─ README.md
│ └─ explorer-web/ # Chain explorer (blocks, tx, receipts)
│ ├─ public/
│ ├─ src/
│ └─ README.md
├─ protocols/ # Canonical protocol definitions
│ ├─ api/ # OpenAPI/JSON‑Schema for REST/WebSocket
│ ├─ receipts/ # Job receipt schema, signing rules
│ ├─ payouts/ # Mint/burn, staking, fees logic (spec)
│ └─ README.md
├─ data/ # Local dev datasets (small, sample only)
│ ├─ fixtures/ # seed users, nodes, jobs
│ └─ samples/
├─ tests/ # Cross‑project test harness
│ ├─ e2e/ # end‑to‑end flows (client→coord→miner→wallet)
│ ├─ load/ # coordinator & miner stress scripts
│ └─ security/ # key rotation, signature verif, replay tests
├─ tools/ # Small CLIs, generators, mermaid->svg, etc.
│ └─ mkdiagram
└─ examples/ # Minimal runnable examples for integrators
├─ quickstart-client-python/
├─ quickstart-client-js/
└─ receipts-sign-verify/
```
## Conventions
- **Languages**: FastAPI/Python for backends; plain JS/TS for web; no Docker.
- **No global venvs**: each `apps/*` and `packages/py/*` can have its own `.venv/` (created by `scripts/env/*`).
- **Systemd over Docker**: unit files live under `configs/systemd/`, with service‑specific overrides documented in `docs/runbooks/`.
- **Static assets** belong to each web app under `public/`. Shared UI in `packages/js/ui-widgets`.
- **SQL**: keep raw SQL snippets in `apps/*/migrations/` (aligned with your “no migration framework” preference). Use `psqln` alias.
- **Security**: central policy under `configs/security/` (fail2ban, ipset lists, TLS ciphers). Keys never committed.
## Minimal READMEs to create next
Create a short `README.md` in each `apps/*` and `packages/*` with:
1. Purpose & scope
2. How to run (dev)
3. Dependencies
4. Configs consumed (from `/configs/app`)
5. Systemd unit name & port (if applicable)
## Suggested first tasks (Way of least resistance)
1. **Bootstrap coordinator-api**: scaffold FastAPI `main.py`, `/health`, `/jobs`, `/miners` routes.
2. **SDKs**: implement `packages/py/aitbc-sdk` & `packages/js/aitbc-sdk` with basic auth + job submit.
3. **miner-node prototype**: heartbeat to coordinator and no‑GPU "echo" job plugin.
4. **client-web**: basic UI to submit a test job and watch status stream.
5. **receipts spec**: draft `protocols/receipts` and a sign/verify example in `examples/`.

235
docs/reference/bootstrap/examples.md Normal file
View File

@ -0,0 +1,235 @@
# examples/ — Minimal runnable examples for integrators
This folder contains three self-contained, copy-pasteable starters that demonstrate how to talk to the coordinator API, submit jobs, poll status, and (optionally) verify signed receipts.
```
examples/
├─ explorer-webexplorer-web/ # (docs live elsewhere; not a runnable example)
├─ quickstart-client-python/ # Minimal Python client
├─ quickstart-client-js/ # Minimal Node/Browser client
└─ receipts-sign-verify/ # Receipt format + sign/verify demos
```
> Conventions: Debian 12/13, zsh, no sudo, run as root if you like. Keep env in a `.env` file. Replace example URLs/tokens with your own.
---
## 1) quickstart-client-python/
### What this shows
- Create a job request
- Submit to `COORDINATOR_URL`
- Poll job status until `succeeded|failed|timeout`
- Fetch the result payload
- (Optional) Save the receipt JSON for later verification
### Files Windsurf should ensure exist
- `main.py` — the tiny client (≈ 80–120 LOC)
- `requirements.txt` — `httpx`, `python-dotenv` (and `pydantic` if you want models)
- `.env.example` — `COORDINATOR_URL`, `API_TOKEN`
- `README.md` — one-screen run guide
### How to run
```sh
cd examples/quickstart-client-python
python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
# Prepare environment
cp .env.example .env
# edit .env → set COORDINATOR_URL=https://api.local.test:8443, API_TOKEN=xyz
# Run
python main.py --prompt "hello compute" --timeout 60
# Outputs:
# - logs to stdout
# - writes ./out/result.json
# - writes ./out/receipt.json (if provided by the coordinator)
```
### Coordinator endpoints the code should touch
- `POST /v1/jobs` → returns `{ job_id }`
- `GET /v1/jobs/{job_id}` → returns `{ status, progress?, result? }`
- `GET /v1/jobs/{job_id}/receipt` → returns `{ receipt }` (optional)
Keep the client resilient: exponential backoff (100ms → 2s), total wall-time cap from `--timeout`.
---
## 2) quickstart-client-js/
### What this shows
- Identical flow to the Python quickstart
- Two variants: Node (fetch via `undici`) and Browser (native `fetch`)
### Files Windsurf should ensure exist
- `node/`
- `package.json` — `undici`, `dotenv`
- `index.js` — Node example client
- `.env.example`
- `README.md`
- `browser/`
- `index.html` — minimal UI with a Prompt box + “Run” button
- `app.js` — client logic (no build step)
- `README.md`
### How to run (Node)
```sh
cd examples/quickstart-client-js/node
npm i
cp .env.example .env
# edit .env → set COORDINATOR_URL, API_TOKEN
node index.js "hello compute"
```
### How to run (Browser)
```sh
cd examples/quickstart-client-js/browser
# Serve statically (choose one)
python3 -m http.server 8080
# or
busybox httpd -f -p 8080
```
Open `http://localhost:8080` and paste your coordinator URL + token in the form.
The app:
- `POST /v1/jobs`
- polls `GET /v1/jobs/{id}` every 1s (with a 60s guard)
- downloads `receipt.json` if available
---
## 3) receipts-sign-verify/
### What this shows
- Receipt JSON structure used by AITBC examples
- Deterministic signing over a canonicalized JSON (RFC 8785-style or stable key order)
- Ed25519 signing & verifying in Python and JS
- CLI snippets to verify receipts offline
> If the project standardizes on another curve, swap the libs accordingly. For Ed25519:
> - Python: `pynacl`
> - JS: `@noble/ed25519`
### Files Windsurf should ensure exist
- `spec.md` — human-readable schema (see below)
- `python/`
- `verify.py` — `python verify.py ./samples/receipt.json ./pubkeys/poolhub_ed25519.pub`
- `requirements.txt` — `pynacl`
- `js/`
- `verify.mjs` — `node js/verify.mjs ./samples/receipt.json ./pubkeys/poolhub_ed25519.pub`
- `package.json` — `@noble/ed25519`
- `samples/receipt.json` — realistic sample
- `pubkeys/poolhub_ed25519.pub` — PEM or raw 32-byte hex
### Minimal receipt schema (for `spec.md`)
```jsonc
{
"version": "1",
"job_id": "string",
"client_id": "string",
"miner_id": "string",
"started_at": "2025-09-26T14:00:00Z",
"completed_at": "2025-09-26T14:00:07Z",
"units_billed": 123, // e.g., “AIToken compute units”
"result_hash": "sha256:…", // hex
"metadata": { "model": "…" }, // optional, stable ordering for signing
"signature": {
"alg": "Ed25519",
"key_id": "poolhub-ed25519-2025-09",
"sig": "base64url…" // signature over canonicalized receipt WITHOUT this signature object
}
}
```
### CLI usage
**Python**
```sh
cd examples/receipts-sign-verify/python
python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
python verify.py ../samples/receipt.json ../pubkeys/poolhub_ed25519.pub
# exit code 0 = valid, non-zero = invalid
```
**Node**
```sh
cd examples/receipts-sign-verify/js
npm i
node verify.mjs ../samples/receipt.json ../pubkeys/poolhub_ed25519.pub
```
**Implementation notes for Windsurf**
- Canonicalize JSON before hashing/signing (stable key order, UTF-8, no trailing spaces).
- Sign bytes of `sha256(canonical_json_without_signature_block)`.
- Reject if `completed_at < started_at`, unknown `alg`, or mismatched `result_hash`.
---
## Shared environment
All quickstarts read the following from `.env` or in-page form fields:
```
COORDINATOR_URL=https://api.local.test:8443
API_TOKEN=replace-me
# Optional: REQUEST_TIMEOUT_SEC=60
```
HTTP headers to include:
```
Authorization: Bearer <API_TOKEN>
Content-Type: application/json
```
---
## Windsurf checklist (do this automatically)
1. **Create folders & files**
- `quickstart-client-python/{main.py,requirements.txt,.env.example,README.md}`
- `quickstart-client-js/node/{index.js,package.json,.env.example,README.md}`
- `quickstart-client-js/browser/{index.html,app.js,README.md}`
- `receipts-sign-verify/{spec.md,samples/receipt.json,pubkeys/poolhub_ed25519.pub}`
- `receipts-sign-verify/python/{verify.py,requirements.txt}`
- `receipts-sign-verify/js/{verify.mjs,package.json}`
2. **Fill templates**
- Implement `POST /v1/jobs`, `GET /v1/jobs/{id}`, `GET /v1/jobs/{id}/receipt` calls.
- Poll with backoff; stop at terminal states; write `out/result.json` & `out/receipt.json`.
3. **Wire Ed25519 libs**
- Python: `pynacl` verify(`public_key`, `message`, `signature`)
- JS: `@noble/ed25519` verifySync
4. **Add DX niceties**
- `.env.example` everywhere
- `README.md` with copy-paste run steps (no global installs)
- Minimal logging and clear non-zero exit on failure
5. **Smoke tests**
- Python quickstart runs end-to-end with a mock coordinator (use our tiny FastAPI mock if available).
- JS Node client runs with `.env`.
- Browser client works via `http://localhost:8080`.
---
## Troubleshooting
- **401 Unauthorized** → check `API_TOKEN`, CORS (browser), or missing `Authorization` header.
- **CORS in browser** → coordinator must set:
- `Access-Control-Allow-Origin: *` (or your host)
- `Access-Control-Allow-Headers: Authorization, Content-Type`
- `Access-Control-Allow-Methods: GET, POST, OPTIONS`
- **Receipt verify fails** → most often due to non-canonical JSON or wrong public key.
---
## License & reuse
Keep examples MIT-licensed. Add a short header to each file:
```
MIT © AITBC Examples — This is demo code; use at your own risk.
```

322
docs/reference/bootstrap/explorer_web.md Normal file
View File

@ -0,0 +1,322 @@
# explorer-web.md
Chain Explorer (blocks · tx · receipts)
## 0) Purpose
A lightweight, fast, dark-themed web UI to browse a minimal AITBC blockchain:
- Latest blocks & block detail
- Transaction list & detail
- Address detail (balance, nonce, tx history)
- Receipt / logs view
- Simple search (block #, hash, tx hash, address)
MVP reads from **blockchain-node** (HTTP/WS API).
No write operations.
---
## 1) Tech & Conventions
- **Pure frontend** (no backend rendering): static HTML + JS modules + CSS.
- **No frameworks** (keep it portable and fast).
- **Files split**: HTML, CSS, JS in separate files (user preference).
- **ES Modules** with strict typing via JSDoc or TypeScript (optional).
- **Dark theme** with orange/ice accents (brand).
- **No animations** unless explicitly requested.
- **Time format**: UTC ISO + relative (e.g., “2m ago”).
---
## 2) Folder Layout (within workspace)
```
explorer-web/
├─ public/
│ ├─ index.html # routes: / (latest blocks)
│ ├─ block.html # /block?hash=... or /block?number=...
│ ├─ tx.html # /tx?hash=...
│ ├─ address.html # /address?addr=...
│ ├─ receipts.html # /receipts?tx=...
│ ├─ 404.html
│ ├─ assets/
│ │ ├─ logo.svg
│ │ └─ icons/*.svg
│ ├─ css/
│ │ ├─ base.css
│ │ ├─ layout.css
│ │ └─ theme-dark.css
│ └─ js/
│ ├─ config.js # API endpoint(s)
│ ├─ api.js # fetch helpers
│ ├─ store.js # simple state cache
│ ├─ utils.js # formatters (hex, time, numbers)
│ ├─ components/
│ │ ├─ header.js
│ │ ├─ footer.js
│ │ ├─ searchbox.js
│ │ ├─ block-table.js
│ │ ├─ tx-table.js
│ │ ├─ pager.js
│ │ └─ keyvalue.js
│ ├─ pages/
│ │ ├─ home.js # latest blocks + mempool/heads
│ │ ├─ block.js
│ │ ├─ tx.js
│ │ ├─ address.js
│ │ └─ receipts.js
│ └─ vendors/ # (empty; we keep it native for now)
├─ docs/
│ └─ explorer-web.md # this file
└─ README.md
```
---
## 3) API Contracts (read-only)
Assume **blockchain-node** exposes:
### 3.1 REST (HTTP)
- `GET /api/chain/head` → `{ number, hash, timestamp }`
- `GET /api/blocks?limit=25&before=<blockNumber>` → `[{number,hash,parentHash,timestamp,txCount,miner,size,gasUsed}]`
- `GET /api/block/by-number/:n` → `{ ...fullBlock }`
- `GET /api/block/by-hash/:h` → `{ ...fullBlock }`
- `GET /api/tx/:hash` → `{ hash, from, to, nonce, value, fee, gas, gasPrice, blockHash, blockNumber, timestamp, input }`
- `GET /api/address/:addr` → `{ address, balance, nonce, txCount }`
- `GET /api/address/:addr/tx?limit=25&before=<blockNumber>` → `[{hash,blockNumber,from,to,value,fee,timestamp}]`
- `GET /api/tx/:hash/receipt` → `{ status, gasUsed, logs: [{address, topics:[...], data, index}], cumulativeGasUsed }`
- `GET /api/search?q=...`
- Accepts block number, block hash, tx hash, or address
- Returns a typed result: `{ type: "block"|"tx"|"address" , key: ... }`
### 3.2 WebSocket (optional, later)
- `ws://.../api/stream/heads` → emits new head `{number,hash,timestamp}`
- `ws://.../api/stream/mempool` → emits tx previews `{hash, from, to, value, timestamp}`
> If the node isn’t ready, create a tiny mock server (FastAPI) consistent with these shapes (already planned in other modules).
---
## 4) Pages & UX
### 4.1 Header (every page)
- Left: logo + “AITBC Explorer”
- Center: search box (accepts block#, block/tx hash, address)
- Right: network tag (e.g., “Local Dev”) + head block# (live)
### 4.2 Home `/`
- **Latest Blocks** (table)
- columns: `#`, `Hash (short)`, `Tx`, `Miner`, `GasUsed`, `Time`
- infinite scroll / “Load older”
- (optional) **Mempool feed** (compact list, toggleable)
- Empty state: helpful instructions + sample query strings
### 4.3 Block Detail `/block?...`
- Top summary (KeyValue component)
- `Number, Hash, Parent, Miner, Timestamp, Size, GasUsed, Difficulty?`
- Transactions table (paginated)
- “Navigate”: Parent ↖, Next ↗, View in raw JSON (debug)
### 4.4 Tx Detail `/tx?hash=...`
- Summary: `Hash, Status, Block, From, To, Value, Fee, Nonce, Gas(gasPrice)`
- Receipt section (logs rendered as topics/data, collapsible)
- Input data: hex preview + decode attempt (if ABI registry exists – later)
### 4.5 Address `/address?addr=...`
- Summary: `Address, Balance, Nonce, TxCount`
- Transactions list (sent/received filter)
- (later) Token balances when chain supports it
### 4.6 Receipts `/receipts?tx=...`
- Focused receipts + logs view with copy buttons
### 4.7 404
- Friendly message + search
---
## 5) Components (JS modules)
- `header.js` : builds header + binds search submit.
- `searchbox.js` : debounced input, detects type (see utils).
- `block-table.js` : render rows, short-hash, time-ago.
- `tx-table.js` : similar render with direction arrows.
- `pager.js` : simple “Load more” with event callback.
- `keyvalue.js` : `<dl>` key/value grid for details.
- `footer.js` : version, links.
---
## 6) Utils
- `formatHexShort(hex, bytes=4)` → `0x1234…abcd`
- `formatNumber(n)` with thin-space groupings
- `formatValueWei(wei)` → AIT units when available (or plain wei)
- `timeAgo(ts)` + `formatUTC(ts)`
- `parseQuery()` helpers for `?hash=...`
- `detectSearchType(q)`:
- `0x` + 66 chars → tx/block hash
- numeric → block number
- `0x` + 42 → address
- fallback → “unknown”
---
## 7) State (store.js)
- `state.head` (number/hash/timestamp)
- `state.cache.blocks[number] = block`
- `state.cache.txs[hash] = tx`
- `state.cache.address[addr] = {balance, nonce, txCount}`
- Simple in-memory LRU eviction (optional).
---
## 8) Styling
- `base.css`: resets, typography, links, buttons, tables.
- `layout.css`: header/footer, grid, content widths (max 960px desktop).
- `theme-dark.css`: colors:
- bg: `#0b0f14`, surface: `#11161c`
- text: `#e6eef7`
- accent-orange: `#ff8a00`
- accent-ice: `#a8d8ff`
- Focus states visible. High contrast table rows on hover.
---
## 9) Error & Loading UX
- Loading spinners (minimal).
- Network errors: inline banner with retry.
- Empty: clear messages & how to search.
---
## 10) Security & Hardening
- Treat inputs as untrusted.
- Only GETs; block any attempt to POST.
- Strict `Content-Security-Policy` sample (for hosting):
- `default-src 'self'; img-src 'self' data:; style-src 'self'; script-src 'self'; connect-src 'self' https://blockchain-node.local;`
- Avoid third-party CDNs.
---
## 11) Test Plan (manual first)
1. Home loads head + 25 latest blocks.
2. Scroll/pager loads older batches.
3. Block search by number + by hash.
4. Tx search → detail + receipt.
5. Address search → tx list.
6. Error states when node is offline.
7. Timezones: display UTC consistently.
---
## 12) Dev Tasks (Windsurf order of work)
1. **Scaffold** folders & empty files.
2. Implement `config.js` with `API_BASE`.
3. Implement `api.js` (fetch JSON + error handling).
4. Build `utils.js` (formatters + search detect).
5. Build `header.js` + `footer.js`.
6. Home page: blocks list + pager.
7. Block detail page.
8. Tx detail + receipts.
9. Address page with tx list.
10. 404 + polish (copy buttons, tiny helpers).
11. CSS pass (dark theme).
12. Final QA.
---
## 13) Mock Data (for offline dev)
Place under `public/js/vendors/mock.js` (opt-in):
- Export functions that resolve Promises with static JSON fixtures in `public/mock/*.json`.
- Toggle via `config.js` flag `USE_MOCK=true`.
---
## 14) Minimal HTML Skeleton (example: index.html)
```html
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width,initial-scale=1" />
<title>AITBC Explorer</title>
<link rel="stylesheet" href="./css/base.css" />
<link rel="stylesheet" href="./css/layout.css" />
<link rel="stylesheet" href="./css/theme-dark.css" />
</head>
<body>
<header id="app-header"></header>
<main id="app"></main>
<footer id="app-footer"></footer>
<script type="module">
import { renderHeader } from './js/components/header.js';
import { renderFooter } from './js/components/footer.js';
import { renderHome } from './js/pages/home.js';
renderHeader(document.getElementById('app-header'));
renderFooter(document.getElementById('app-footer'));
renderHome(document.getElementById('app'));
</script>
</body>
</html>
```
---
## 15) config.js (example)
```js
export const CONFIG = {
API_BASE: 'http://localhost:8545', // adapt to blockchain-node
USE_MOCK: false,
PAGE_SIZE: 25,
NETWORK_NAME: 'Local Dev',
};
```
---
## 16) API Helpers (api.js — sketch)
```js
import { CONFIG } from './config.js';
async function jget(path) {
const res = await fetch(`${CONFIG.API_BASE}${path}`, { method: 'GET' });
if (!res.ok) throw new Error(`HTTP ${res.status}: ${path}`);
return res.json();
}
export const api = {
head: () => jget('/api/chain/head'),
blocks: (limit, before) => jget(`/api/blocks?limit=${limit}&before=${before ?? ''}`),
blockByNo: (n) => jget(`/api/block/by-number/${n}`),
blockByHash: (h) => jget(`/api/block/by-hash/${h}`),
tx: (hash) => jget(`/api/tx/${hash}`),
receipt: (hash) => jget(`/api/tx/${hash}/receipt`),
address: (addr) => jget(`/api/address/${addr}`),
addressTx: (addr, limit, before) => jget(`/api/address/${addr}/tx?limit=${limit}&before=${before ?? ''}`),
search: (q) => jget(`/api/search?q=${encodeURIComponent(q)}`),
};
```
---
## 17) Performance Checklist
- Use pagination/infinite scroll (no huge payloads).
- Cache recent blocks/tx in-memory (store.js).
- Avoid layout thrash: table builds via DocumentFragment.
- Defer non-critical fetches (e.g., mempool).
- Keep CSS small and critical.
---
## 18) Deployment
- Serve `public/` via Nginx under `/explorer/` or own domain.
- Set correct `connect-src` in CSP to point to blockchain-node.
- Ensure CORS on blockchain-node for the explorer origin (read-only).
---
## 19) Roadmap (post-MVP)
- Live head updates via WS.
- Mempool stream view.
- ABI registry + input decoding.
- Token balances (when chain supports).
- Export to CSV / JSON for tables.
- Theming switch (dark/light).
---

468
docs/reference/bootstrap/layout.md Normal file
View File

@ -0,0 +1,468 @@
# Layout & Frontend Guidelines (Windsurf)
Target: **mobile‑first**, dark theme, max content width **960px** on desktop. Reference device: **Nothing Phone 2a**.
---
## 1) Design System
### 1.1 Color (Dark Theme)
- `--bg-0: #0b0f14` (page background)
- `--bg-1: #11161c` (cards/sections)
- `--tx-0: #e6edf3` (primary text)
- `--tx-1: #a7b3be` (muted)
- `--pri: #ff7a1a` (accent orange)
- `--ice: #b9ecff` (ice accent)
- `--ok: #3ddc97`
- `--warn: #ffcc00`
- `--err: #ff4d4d`
### 1.2 Typography
- Base font-size: **16px** (mobile), scale up at desktop.
- Font stack: System UI (`-apple-system, Segoe UI, Roboto, Inter, Arial, sans-serif`).
- Line-height: 1.5 body, 1.2 headings.
### 1.3 Spacing (8‑pt grid)
- `--s-1: 4px`, `--s-2: 8px`, `--s-3: 12px`, `--s-4: 16px`, `--s-5: 24px`, `--s-6: 32px`, `--s-7: 48px`, `--s-8: 64px`.
### 1.4 Radius & Shadow
- Radius: `--r-1: 8px`, `--r-2: 16px`.
- Shadow (subtle): `0 4px 20px rgba(0,0,0,.25)`.
---
## 2) Grid & Layout
### 2.1 Container
- **Mobile‑first**: full‑bleed padding.
- Desktop container: **max‑width: 960px**, centered.
- Side gutters: 16px (mobile), 24px (tablet), 32px (desktop).
**Breakpoint summary**
| Token | Min width | Container behaviour | Notes |
| --- | --- | --- | --- |
| `--bp-sm` | 360px | Fluid | Single-column layouts prioritise readability.
| `--bp-md` | 480px | Fluid | Allow two-up cards or media/text pairings when needed.
| `--bp-lg` | 768px | Max-width 90% (capped at 960px) | Stage tablet/landscape experiences before full desktop.
| `--bp-xl` | 960px | Fixed 960px max width | Full desktop grid, persistent side rails allowed.
Always respect `env(safe-area-inset-*)` for notch devices (use helpers like `.safe-b`).
### 2.2 Columns
- 12‑column grid on screens ≥ **960px**.
- Column gutter: 16px (mobile), 24px (≥960px).
- Utility classes (examples):
- `.row { display:grid; grid-template-columns: repeat(12, 1fr); gap: var(--gutter); }`
- `.col-12, .col-6, .col-4, .col-3` for common spans on desktop.
- Mobile stacks by default; use responsive helpers at breakpoints.
### 2.3 Breakpoints (Nothing Phone 2a aware)
- `--bp-sm: 360px` (small phones)
- `--bp-md: 480px` (Nothing 2a width in portrait ~ 412–480 CSS px)
- `--bp-lg: 768px` (tablets / landscape phones)
- `--bp-xl: 960px` (desktop container)
**Mobile layout rules:**
- Navigation collapses to icon buttons with overflow menu at `--bp-sm`.
- Multi-column sections stack; keep vertical rhythm using `var(--s-6)`.
- Sticky headers take 56px height; ensure content uses `.safe-b` for bottom insets.
**Desktop enhancements:**
- Activate `.row` grid with `.col-*` spans at `--bp-xl`.
- Introduce side rail for filters or secondary nav (span 3 or 4 columns).
- Increase typographic scale by 1 step (`clamp` already handles this).
> **Rule:** Build for < `--bp-lg` first; enhance progressively at `--bp-lg` and `--bp-xl`.
---
## 3) Page Chrome
### 3.1 Header
- Sticky top, height 56–64px.
- Left: brand; Right: primary action or menu.
- Translucent on scroll (backdrop‑filter), solid at top.
### 3.2 Footer
- Thin bar; meta links. Uses ice accent for separators.
### 3.3 Main
- Vertical rhythm: sections spaced by `var(--s-7)` (mobile) / `var(--s-8)` (desktop).
- Cards: background `var(--bg-1)`, radius `var(--r-2)`, padding `var(--s-6)`.
---
## 4) CSS File Strategy (per page)
**Every HTML page ships its own CSS file**, plus shared layers:
- `/css/base.css` — resets, variables, typography, utility helpers.
- `/css/components.css` — buttons, inputs, cards, modals, toast.
- `/css/layout.css` — grid/container/header/footer.
- `/css/pages/<page>.css` — page‑specific rules (one per HTML page).
**Naming** (BEM‑ish): `block__elem--mod`. Avoid nesting >2 levels.
**Example includes:**
```html
<link rel="stylesheet" href="/css/base.css">
<link rel="stylesheet" href="/css/components.css">
<link rel="stylesheet" href="/css/layout.css">
<link rel="stylesheet" href="/css/pages/dashboard.css">
```
---
## 5) Utilities (recommended)
- Spacing: `.mt-4`, `.mb-6`, `.px-4`, `.py-6` (map to spacing scale).
- Flex/Grid: `.flex`, `.grid`, `.ai-c`, `.jc-b`.
- Display: `.hide-sm`, `.hide-lg` using media queries.
---
## 6) Toast Messages (center bottom)
**Position:** centered at bottom above safe‑area insets.
**Behavior:**
- Appear for 3–5s; pause on hover; dismiss on click.
- Max width 90% on mobile, 420px desktop.
- Elevation + subtle slide/fade.
**Structure:**
```html
<div id="toast-root" aria-live="polite" aria-atomic="true"></div>
```
**Styles (concept):**
```css
#toast-root { position: fixed; left: 50%; bottom: max(16px, env(safe-area-inset-bottom)); transform: translateX(-50%); z-index: 9999; }
.toast { background: var(--bg-1); color: var(--tx-0); border: 1px solid rgba(185,236,255,.2); padding: var(--s-5) var(--s-6); border-radius: var(--r-2); box-shadow: 0 10px 30px rgba(0,0,0,.35); margin-bottom: var(--s-4); max-width: min(420px, 90vw); }
.toast--ok { border-color: rgba(61,220,151,.35); }
.toast--warn { border-color: rgba(255,204,0,.35); }
.toast--err { border-color: rgba(255,77,77,.35); }
```
**JS API (minimal):**
```js
function showToast(msg, type = "ok", ms = 3500) {
const root = document.getElementById("toast-root");
const el = document.createElement("div");
el.className = `toast toast--${type}`;
el.role = "status";
el.textContent = msg;
root.appendChild(el);
const t = setTimeout(() => el.remove(), ms);
el.addEventListener("mouseenter", () => clearTimeout(t));
el.addEventListener("click", () => el.remove());
}
```
---
## 7) Browser Notifications (system tray)
**When to use:** Only for important, user‑initiated events (e.g., a new match, message, or scheduled session start). Always provide an in‑app alternative (toast/modal) for users who deny permission.
**Permission flow:**
```js
async function ensureNotifyPermission() {
if (!("Notification" in window)) return false;
if (Notification.permission === "granted") return true;
if (Notification.permission === "denied") return false;
const res = await Notification.requestPermission();
return res === "granted";
}
```
**Send notification:**
```js
function notify(opts) {
if (Notification.permission !== "granted") return;
const n = new Notification(opts.title || "Update", {
body: opts.body || "",
icon: opts.icon || "/icons/notify.png",
tag: opts.tag || "app-event",
requireInteraction: !!opts.sticky
});
if (opts.onclick) n.addEventListener("click", opts.onclick);
}
```
**Pattern:**
```js
const ok = await ensureNotifyPermission();
if (ok) notify({ title: "Match window opens soon", body: "Starts in 10 min" });
else showToast("Enable notifications in settings to get alerts", "warn");
```
---
## 8) Forms & Inputs
- Hit target ≥ 44×44px, labels always visible.
- Focus ring: `outline: 2px solid var(--ice)`.
- Validation: inline text in `--warn/--err`; never only color.
---
## 9) Performance
- CSS: ship **only** what a page uses (per‑page CSS). Avoid giant bundles.
- Images: `loading="lazy"`, responsive sizes; WebP/AVIF first.
- Fonts: use system fonts; if custom, `font-display: swap`.
---
## 10) Accessibility
- Ensure color contrast ratios meet WCAG AA standards (e.g., 4.5:1 for text and 3:1 for large text).
- Use semantic HTML elements (<header>, <nav>, <main>, etc.) and ARIA attributes for dynamic content.
- Support keyboard navigation with logical tab order and visible focus indicators.
- Test for screen readers, providing text alternatives for images and ensuring forms are labeled correctly.
- Adapt layouts for assistive technologies using media queries and flexible components.
## 11) Accessibility Integration
- Apply utility classes for focus states (e.g., :focus-visible with visible outline).
- Use ARIA roles for custom widgets and ensure all content is perceivable, operable, understandable, and robust.
## 12) Breakpoint Examples
```css
/* Mobile-first defaults here */
@media (min-width: 768px) { /* tablets / landscape phones */ }
@media (min-width: 960px) { /* desktop grid & container */ }
```
---
## 13) Checklist (per page)
- [ ] Uses `/css/pages/<page>.css` + shared layers
- [ ] Container max‑width 960px, centered; gutters follow breakpoint summary
- [ ] Mobile‑first; tests on Nothing Phone 2a
- [ ] Toasts render center‑bottom
- [ ] Notifications gated by permission & mirrored with toasts
- [ ] A11y pass (headings order, labels, focus, contrast)
---
## Appendix A — `/css/base.css`
```css
/* Base: variables, reset, typography, utilities */
:root{
--bg-0:#0b0f14; --bg-1:#11161c;
--tx-0:#e6edf3; --tx-1:#a7b3be;
--pri:#ff7a1a; --ice:#b9ecff;
--ok:#3ddc97; --warn:#ffcc00; --err:#ff4d4d;
--s-1:4px; --s-2:8px; --s-3:12px; --s-4:16px;
--s-5:24px; --s-6:32px; --s-7:48px; --s-8:64px;
--r-1:8px; --r-2:16px;
--gutter:16px;
}
@media (min-width:960px){ :root{ --gutter:24px; } }
/* Reset */
*{ box-sizing:border-box; }
html,body{ height:100%; }
html{ color-scheme:dark; }
body{
margin:0; background:var(--bg-0); color:var(--tx-0);
font:16px/1.5 -apple-system, Segoe UI, Roboto, Inter, Arial, sans-serif;
-webkit-font-smoothing:antialiased; -moz-osx-font-smoothing:grayscale;
}
img,svg,video{ max-width:100%; height:auto; display:block; }
button,input,select,textarea{ font:inherit; color:inherit; }
:focus-visible{ outline:2px solid var(--ice); outline-offset:2px; }
/* Typography */
h1{ font-size:clamp(24px, 3.5vw, 36px); line-height:1.2; margin:0 0 var(--s-4); }
h2{ font-size:clamp(20px, 3vw, 28px); line-height:1.25; margin:0 0 var(--s-3); }
h3{ font-size:clamp(18px, 2.5vw, 22px); line-height:1.3; margin:0 0 var(--s-2); }
p{ margin:0 0 var(--s-4); color:var(--tx-1); }
/* Links */
a{ color:var(--ice); text-decoration:none; }
a:hover{ text-decoration:underline; }
/* Utilities */
.container{ width:100%; max-width:960px; margin:0 auto; padding:0 var(--gutter); }
.flex{ display:flex; } .grid{ display:grid; }
.ai-c{ align-items:center; } .jc-b{ justify-content:space-between; }
.mt-4{ margin-top:var(--s-4);} .mb-6{ margin-bottom:var(--s-6);} .px-4{ padding-left:var(--s-4); padding-right:var(--s-4);} .py-6{ padding-top:var(--s-6); padding-bottom:var(--s-6);}
.hide-sm{ display:none; }
@media (min-width:960px){ .hide-lg{ display:none; } .hide-sm{ display:initial; } }
/* Safe area helpers */
.safe-b{ padding-bottom:max(var(--s-4), env(safe-area-inset-bottom)); }
```
---
## Appendix B — `/css/layout.css`
```css
/* Grid, header, footer, sections */
.row{ display:grid; grid-template-columns:repeat(12, 1fr); gap:var(--gutter); }
/* Mobile-first: stack columns */
[class^="col-"]{ grid-column:1/-1; }
@media (min-width:960px){
.col-12{ grid-column:span 12; }
.col-6{ grid-column:span 6; }
.col-4{ grid-column:span 4; }
.col-3{ grid-column:span 3; }
}
header.site{
position:sticky; top:0; z-index:50;
backdrop-filter:saturate(1.2) blur(8px);
background:color-mix(in oklab, var(--bg-0) 85%, transparent);
border-bottom:1px solid rgba(185,236,255,.12);
}
header.site .inner{ height:64px; }
footer.site{
margin-top:var(--s-8);
border-top:1px solid rgba(185,236,255,.12);
background:var(--bg-0);
}
footer.site .inner{ height:56px; font-size:14px; color:var(--tx-1); }
main section{ margin: var(--s-7) 0; }
.card{
background:var(--bg-1);
border:1px solid rgba(185,236,255,.12);
border-radius:var(--r-2);
padding:var(--s-6);
box-shadow:0 4px 20px rgba(0,0,0,.25);
}
```
---
## Appendix C — `/css/components.css`
```css
/* Buttons */
.btn{ display:inline-flex; align-items:center; justify-content:center; gap:8px;
border-radius:var(--r-1); padding:10px 16px; border:1px solid transparent; cursor:pointer;
background:var(--pri); color:#111; font-weight:600; text-align:center; }
.btn:hover{ filter:brightness(1.05); }
.btn--subtle{ background:transparent; color:var(--tx-0); border-color:rgba(185,236,255,.2); }
.btn--ghost{ background:transparent; color:var(--pri); border-color:transparent; }
.btn:disabled{ opacity:.6; cursor:not-allowed; }
/* Inputs */
.input{ width:100%; background:#0e1319; color:var(--tx-0);
border:1px solid rgba(185,236,255,.18); border-radius:var(--r-1);
padding:12px 14px; }
.input::placeholder{ color:#6f7b86; }
/* Badges */
.badge{ display:inline-block; padding:2px 8px; border-radius:999px; font-size:12px; border:1px solid rgba(185,236,255,.2); }
.badge--ok{ color:#0c2; border-color:rgba(61,220,151,.4); }
.badge--warn{ color:#fc0; border-color:rgba(255,204,0,.4); }
.badge--err{ color:#f66; border-color:rgba(255,77,77,.4); }
/* Toasts */
#toast-root{ position:fixed; left:50%; bottom:max(16px, env(safe-area-inset-bottom)); transform:translateX(-50%); z-index:9999; }
.toast{ background:var(--bg-1); color:var(--tx-0);
border:1px solid rgba(185,236,255,.2); padding:var(--s-5) var(--s-6);
border-radius:var(--r-2); box-shadow:0 10px 30px rgba(0,0,0,.35);
margin-bottom:var(--s-4); max-width:min(420px, 90vw);
opacity:0; translate:0 8px; animation:toast-in .24s ease-out forwards; }
.toast--ok{ border-color:rgba(61,220,151,.35); }
.toast--warn{ border-color:rgba(255,204,0,.35); }
.toast--err{ border-color:rgba(255,77,77,.35); }
@keyframes toast-in{ to{ opacity:1; translate:0 0; } }
```
---
## Appendix D — `/js/toast.js`
```js
export function showToast(msg, type = "ok", ms = 3500) {
const root = document.getElementById("toast-root") || (() => {
const r = document.createElement("div");
r.id = "toast-root";
document.body.appendChild(r);
return r;
})();
const el = document.createElement("div");
el.className = `toast toast--${type}`;
el.role = "status";
el.textContent = msg;
root.appendChild(el);
const t = setTimeout(() => el.remove(), ms);
el.addEventListener("mouseenter", () => clearTimeout(t));
el.addEventListener("click", () => el.remove());
}
```
---
## Appendix E — `/js/notify.js`
```js
export async function ensureNotifyPermission() {
if (!("Notification" in window)) return false;
if (Notification.permission === "granted") return true;
if (Notification.permission === "denied") return false;
const res = await Notification.requestPermission();
return res === "granted";
}
export function notify(opts) {
if (Notification.permission !== "granted") return;
const n = new Notification(opts.title || "Update", {
body: opts.body || "",
icon: opts.icon || "/icons/notify.png",
tag: opts.tag || "app-event",
requireInteraction: !!opts.sticky
});
if (opts.onclick) n.addEventListener("click", opts.onclick);
}
```
---
## Appendix F — `/css/pages/dashboard.css`
```css
/* Dashboard specific styles */
.dashboard-header{
margin-bottom:var(--s-6);
display:flex; align-items:center; justify-content:space-between;
}
.dashboard-header h1{ font-size:28px; color:var(--pri); }
.stats-grid{
display:grid;
gap:var(--s-5);
grid-template-columns:repeat(auto-fit, minmax(160px, 1fr));
}
.stat-card{
background:var(--bg-1);
border:1px solid rgba(185,236,255,.12);
border-radius:var(--r-2);
padding:var(--s-5);
text-align:center;
}
.stat-card h2{ margin:0; font-size:20px; color:var(--ice); }
.stat-card p{ margin:var(--s-2) 0 0; color:var(--tx-1); font-size:14px; }
.activity-feed{
margin-top:var(--s-7);
}
.activity-item{
border-bottom:1px solid rgba(185,236,255,.1);
padding:var(--s-4) 0;
display:flex; align-items:center; gap:var(--s-4);
}
.activity-item:last-child{ border-bottom:none; }
.activity-icon{ width:32px; height:32px; border-radius:50%; background:var(--pri); display:flex; align-items:center; justify-content:center; color:#111; font-size:16px; }
.activity-text{ flex:1; font-size:14px; color:var(--tx-0); }
.activity-time{ font-size:12px; color:var(--tx-1); }

237
docs/reference/bootstrap/marketplace_web.md Normal file
View File

@ -0,0 +1,237 @@
# marketplace-web
Web app for listing compute **offers**, placing **bids**, and viewing **market stats** in the AITBC stack.
Stage-aware: works **now** against a mock API, later switches to **coordinator/pool-hub/blockchain** endpoints without touching the UI.
## Goals
1. Browse offers (GPU/CPU, price per token, location, queue, latency).
2. Place/manage bids (instant or scheduled).
3. Watch market stats (price trends, filled volume, miner capacity).
4. Wallet view (balance, recent tx; read-only first).
5. Internationalization (EU langs later), dark theme, 960px layout.
## Tech/Structure (Windsurf-friendly)
- Vanilla **TypeScript + Vite** (no React), separate JS/CSS files.
- File layout (desktop 960px grid, mobile-first CSS, dark theme):
```
marketplace-web/
├─ public/
│ ├─ icons/ # favicons, app icons
│ └─ i18n/ # JSON dictionaries (en/… later)
├─ src/
│ ├─ app.ts # app bootstrap/router
│ ├─ router.ts # hash-router (/, /offer/:id, /bids, /stats, /wallet)
│ ├─ api/
│ │ ├─ http.ts # fetch wrapper + baseURL swap (mock → real)
│ │ └─ marketplace.ts # typed API calls
│ ├─ store/
│ │ ├─ state.ts # global app state (signals or tiny pubsub)
│ │ └─ types.ts # shared types/interfaces
│ ├─ views/
│ │ ├─ HomeView.ts
│ │ ├─ OfferDetailView.ts
│ │ ├─ BidsView.ts
│ │ ├─ StatsView.ts
│ │ └─ WalletView.ts
│ ├─ components/
│ │ ├─ OfferCard.ts
│ │ ├─ BidForm.ts
│ │ ├─ Table.ts
│ │ ├─ Sparkline.ts # minimal chart (no external lib)
│ │ └─ Toast.ts
│ ├─ styles/
│ │ ├─ base.css # reset, variables, dark theme
│ │ ├─ layout.css # 960px grid, sections, header/footer
│ │ └─ components.css
│ └─ util/
│ ├─ format.ts # fmt token, price, time
│ ├─ validate.ts # input validation
│ └─ i18n.ts # simple t() loader
├─ index.html
├─ vite.config.ts
└─ README.md
```
## Routes (Hash router)
- `/` — Offer list + filters
- `/offer/:id` — Offer details + **BidForm**
- `/bids` — User bids (open, filled, cancelled)
- `/stats` — Price/volume/capacity charts
- `/wallet` — Balance + last 10 tx (read-only)
## UI/UX Spec
- **Dark theme**, accent = ice-blue/white outlines (fits OIB style).
- **960px max width** desktop, mobile-first, Nothing Phone 2a as reference.
- **Toast** bottom-center for actions.
- Forms: no animations, clear validation, disable buttons during submit.
## Data Types (minimal)
```ts
type TokenAmount = `${number}`; // keep as string to avoid FP errors
type PricePerToken = `${number}`;
interface Offer {
id: string;
provider: string; // miner or pool label
hw: { gpu: string; vramGB?: number; cpu?: string };
region: string; // e.g., eu-central
queue: number; // jobs waiting
latencyMs: number;
price: PricePerToken; // AIToken per 1k tokens processed
minTokens: number;
maxTokens: number;
updatedAt: string;
}
interface BidInput {
offerId: string;
tokens: number; // requested tokens to process
maxPrice: PricePerToken; // cap
}
interface Bid extends BidInput {
id: string;
status: "open" | "filled" | "cancelled" | "expired";
createdAt: string;
filledTokens?: number;
avgFillPrice?: PricePerToken;
}
interface MarketStats {
ts: string[];
medianPrice: number[]; // per interval
filledVolume: number[]; // tokens
capacity: number[]; // available tokens
}
interface Wallet {
address: string;
balance: TokenAmount;
recent: Array<{ id: string; kind: "mint"|"spend"|"refund"; amount: TokenAmount; at: string }>;
}
```
## Mock API (Stage 0)
Base URL: `/.mock` (served via Vite dev middleware or static JSON)
- `GET /.mock/offers.json` → `Offer[]`
- `GET /.mock/offers/:id.json` → `Offer`
- `POST /.mock/bids` (body `BidInput`) → `Bid`
- `GET /.mock/bids.json` → `Bid[]`
- `GET /.mock/stats.json` → `MarketStats`
- `GET /.mock/wallet.json` → `Wallet`
Switch to real endpoints by changing **`BASE_URL`** in `api/http.ts`.
## Real API (Stage 2/3/4 wiring)
When coordinator/pool-hub/blockchain are ready:
- `GET /api/market/offers` → `Offer[]`
- `GET /api/market/offers/:id` → `Offer`
- `POST /api/market/bids` → create bid, returns `Bid`
- `GET /api/market/bids?owner=<wallet>` → `Bid[]`
- `GET /api/market/stats?range=24h|7d|30d` → `MarketStats`
- `GET /api/wallet/summary?addr=<wallet>` → `Wallet`
Auth header (later): `Authorization: Bearer <session-or-wallet-token>`.
## State & Caching
- In-memory store (`store/state.ts`) with tiny pub/sub.
- Offer list cached 30s; stats cached 60s; bust on route change if stale.
- Optimistic UI for **Bid** create; reconcile on server response.
## Filters (Home)
- Region (multi)
- HW capability (GPU model, min VRAM, CPU present)
- Price range (slider)
- Latency max (ms)
- Queue max
All filters are client-side over fetched offers (server-side later).
## Validation Rules
- `BidInput.tokens` ∈ `[offer.minTokens, offer.maxTokens]`.
- `maxPrice >= offer.price` for instant fill hint; otherwise place as limit.
- Warn if `queue > threshold` or `latencyMs > threshold`.
## Security Notes (Web)
- Input sanitize; never eval.
- CSRF not needed for read-only; for POST use standard token once auth exists.
- Rate-limit POST (server).
- Display wallet **read-only** unless signing is integrated (later via wallet-daemon).
## i18n
- `public/i18n/en.json` as root.
- `util/i18n.ts` provides `t(key, params?)`.
- Keys only (no concatenated sentences). EU languages can be added later via your i18n tool.
## Accessibility
- Semantic HTML, label every input, focus states visible.
- Keyboard: Tab order, Enter submits forms, Esc closes dialogs.
- Color contrast AA in dark theme.
## Minimal Styling Rules
- `styles/base.css`: CSS variables for colors, spacing, radius.
- `styles/layout.css`: header, main container (max-width: 960px), grid for cards.
- `styles/components.css`: OfferCard, Table, Buttons, Toast.
## Testing
- Manual first:
- Offers list loads, filters act.
- Place bid with edge values.
- Stats sparkline renders with missing points.
- Later: Vitest for `util/` + `api/` modules.
## Env/Config
- `VITE_API_BASE` → mock or real.
- `VITE_DEFAULT_REGION` → optional default filter.
- `VITE_FEATURE_WALLET=readonly|disabled`.
## Build/Run
```
# dev
npm i
npm run dev
# build
npm run build
npm run preview
```
## Migration Checklist (Mock → Real)
1. Replace `VITE_API_BASE` with coordinator gateway URL.
2. Enable auth header injection when session is present.
3. Wire `/wallet` to wallet-daemon read endpoint.
4. Swap stats source to real telemetry.
5. Keep the same types; server must honor them.
## Open Tasks
- [ ] Create file skeletons per structure above.
- [ ] Add mock JSON under `public/.mock/`.
- [ ] Implement OfferCard + filters.
- [ ] Implement BidForm with validation + optimistic UI.
- [ ] Implement StatsView with `Sparkline` (no external chart lib).
- [ ] Wire `VITE_API_BASE` switch.
- [ ] Basic a11y pass + dark theme polish.
- [ ] Wallet view (read-only).

423
docs/reference/bootstrap/miner.md Normal file
View File

@ -0,0 +1,423 @@
# AITBC Miner – Windsurf Boot & Ops Guide
A minimal, production‑lean starter for bringing an **AITBC compute‑miner** online on Debian (Bookworm/Trixie). It is optimized for NVIDIA GPUs with CUDA support, yet safe to run CPU‑only. The miner polls jobs from a central **Coordinator API** on behalf of clients, executes AI workloads, generates proofs, and earns tokens. Payments are credited to the configured wallet.
---
## Flow Diagram
```
[ Client ] → submit job → [ Coordinator API ] → dispatch → [ Miner ] → proof → [ Coordinator API ] → credit → [ Wallet ]
```
- **Client**: User or application requesting AI computation.
- **Coordinator API**: Central dispatcher that manages jobs and miners.
- **Miner**: Executes the AI workload, generates proofs, and submits results.
- **Wallet**: Receives token rewards for completed jobs.
---
## Quickstart: Windsurf Fast Boot
The minimal info Windsurf needs to spin everything up quickly:
1. **Minimal Config Values** (edit `/etc/aitbc/miner.conf`):
- `COORD_URL` (use mock for local dev): `http://127.0.0.1:8080`
- `WALLET_ADDR` (any test string for mock): `wallet_demo`
- `API_KEY` (mock ignores, still set one): `CHANGE_ME`
- `MINER_ID`: `$(hostname)-gpu0`
2. **Dependencies**
```bash
apt update
apt install -y python3 python3-venv python3-pip curl jq ca-certificates git pciutils lsb-release
# mock coordinator deps
pip install fastapi uvicorn
```
- **GPU optional**: ensure `nvidia-smi` works for CUDA path.
3. **Boot the Mock Coordinator** (new terminal):
```bash
uvicorn mock_coordinator:app --reload --host 127.0.0.2 --port 8080
```
4. **Install & Start Miner**
```bash
/root/scripts/aitbc-miner/install_miner.sh
systemctl start aitbc-miner.service
```
5. **Verify**
```bash
systemctl status aitbc-miner.service
tail -f /var/log/aitbc-miner.log
curl -s http://127.0.0.1:8080/v1/wallet/balance | jq
```
> With these details, Windsurf can boot both the miner and the mock Coordinator in under a minute without a production backend.
---
## CUDA Support
Yes, the miner supports CUDA GPUs. The installer checks for `nvidia-smi` and, if present, attempts to install PyTorch with CUDA wheels (`cu124`). At runtime, tensors are placed on `'cuda'` if `torch.cuda.is_available()` is true. If no GPU is detected, the miner automatically falls back to CPU mode.
**Prerequisites for CUDA:**
- Install NVIDIA drivers on Debian:
```bash
apt install -y nvidia-driver nvidia-smi
```
- Optional: Install CUDA toolkit if required for advanced workloads:
```bash
apt install -y nvidia-cuda-toolkit
```
- Verify with:
```bash
nvidia-smi
```
Make sure drivers/toolkit are installed before running the miner installer.
---
## 1) Targets & Assumptions
- Host: Debian 12/13, root shell, `zsh` available.
- Optional GPU: NVIDIA (e.g. RTX 4060 Ti) with CUDA toolchain.
- Network egress to Coordinator API (HTTPS). No inbound ports required.
- File paths align with user conventions.
---
## 2) Directory Layout
```
/root/scripts/aitbc-miner/ # scripts
install_miner.sh
miner.sh
/etc/aitbc/
miner.conf # runtime config (env‑like)
/var/log/aitbc-miner.log # log file (rotated by logrotate, optional)
```
---
## 3) Config File: `/etc/aitbc/miner.conf`
Environment‑style key/values. Edit with your Coordinator endpoint and wallet/API key.
```ini
# AITBC Miner config
COORD_URL="https://coordinator.example.net" # Coordinator base URL
WALLET_ADDR="wallet1qxy2kgdygjrsqtzq2n0yrf..." # Your payout address
API_KEY="REPLACE_WITH_WALLET_API_KEY" # Wallet‑issued key for auth
MINER_ID="$(hostname)-gpu0" # Any stable node label
WORK_DIR="/tmp/aitbc-work" # Scratch space
HEARTBEAT_SECS=20 # Health ping interval
JOB_POLL_SECS=3 # Fetch cadence
MAX_CONCURRENCY=1 # Inference job slots
# GPU modes: auto|gpu|cpu
ACCEL_MODE="auto"
```
> **Tip:** Store secrets with `chmod 600 /etc/aitbc/miner.conf`.
---
## 4) Installer Script: `/root/scripts/aitbc-miner/install_miner.sh`
```bash
#!/bin/bash
# Script Version: 01
# Description: Install AITBC miner runtime (deps, folders, service)
set -euo pipefail
LOG_FILE=/var/log/aitbc-miner.log
mkdir -p /root/scripts/aitbc-miner /etc/aitbc
: > "$LOG_FILE"
chmod 600 "$LOG_FILE"
# Base deps
apt update
apt install -y curl jq ca-certificates coreutils procps python3 python3-venv python3-pip git \
pciutils lsb-release
# Optional: NVIDIA CLI utils detection (no failure if absent)
if command -v nvidia-smi >/dev/null 2>&1; then
echo "[INFO] NVIDIA detected" | tee -a "$LOG_FILE"
else
echo "[INFO] NVIDIA not detected, will run CPU mode if configured" | tee -a "$LOG_FILE"
fi
# Python env for exemplar workloads (torch optional)
VENV_DIR=/opt/aitbc-miner/.venv
mkdir -p /opt/aitbc-miner
python3 -m venv "$VENV_DIR"
source "$VENV_DIR/bin/activate"
# Minimal runtime deps
pip install --upgrade pip wheel
# Try torch (GPU if CUDA present; fallback CPU). Best‑effort only.
python - <<'PY'
import os, sys
try:
import subprocess
cuda_ok = subprocess.call(["bash","-lc","nvidia-smi >/dev/null 2>&1"])==0
pkg = "--index-url https://download.pytorch.org/whl/cu124 torch torchvision torchaudio" if cuda_ok else "torch torchvision torchaudio"
os.system(f"pip install -q {pkg}")
print("[INFO] torch installed")
except Exception as e:
print("[WARN] torch install skipped:", e)
PY
# Place default config if missing
CONF=/etc/aitbc/miner.conf
if [ ! -f "$CONF" ]; then
cat >/etc/aitbc/miner.conf <<'CFG'
COORD_URL="https://coordinator.example.net"
WALLET_ADDR="wallet_demo"
API_KEY="CHANGE_ME"
MINER_ID="demo-node"
WORK_DIR="/tmp/aitbc-work"
HEARTBEAT_SECS=20
JOB_POLL_SECS=3
MAX_CONCURRENCY=1
ACCEL_MODE="auto"
CFG
chmod 600 /etc/aitbc/miner.conf
echo "[INFO] Wrote /etc/aitbc/miner.conf" | tee -a "$LOG_FILE"
fi
# Install service unit
cat >/etc/systemd/system/aitbc-miner.service <<'UNIT'
[Unit]
Description=AITBC Compute Miner
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
ExecStart=/root/scripts/aitbc-miner/miner.sh
Restart=always
RestartSec=3
EnvironmentFile=/etc/aitbc/miner.conf
StandardOutput=append:/var/log/aitbc-miner.log
StandardError=append:/var/log/aitbc-miner.log
[Install]
WantedBy=multi-user.target
UNIT
systemctl daemon-reload
systemctl enable --now aitbc-miner.service
echo "[INFO] AITBC miner installed and started" | tee -a "$LOG_FILE"
```
---
## 5) Miner Runtime: `/root/scripts/aitbc-miner/miner.sh`
```bash
#!/bin/bash
# Script Version: 01
# Description: AITBC miner main loop (poll, run, prove, earn)
set -euo pipefail
LOG_FILE=/var/log/aitbc-miner.log
: > "$LOG_FILE"
# ========
# Helpers
# ========
log(){ printf '%s %s\n' "$(date -Is)" "$*" | tee -a "$LOG_FILE"; }
req(){
local method="$1" path="$2" data="${3:-}"
local url="${COORD_URL%/}${path}"
if [ -n "$data" ]; then
curl -fsS -X "$method" "$url" \
-H "Authorization: Bearer $API_KEY" -H 'Content-Type: application/json' \
--data "$data"
else
curl -fsS -X "$method" "$url" -H "Authorization: Bearer $API_KEY"
fi
}
has_gpu(){ command -v nvidia-smi >/dev/null 2>&1 && nvidia-smi >/dev/null 2>&1; }
accel_mode(){
case "$ACCEL_MODE" in
gpu) has_gpu && echo gpu || echo cpu ;;
cpu) echo cpu ;;
auto) has_gpu && echo gpu || echo cpu ;;
*) echo cpu ;;
esac
}
run_job(){
local job_json="$1"
local job_id; job_id=$(echo "$job_json" | jq -r '.id')
local workload; workload=$(echo "$job_json" | jq -r '.workload')
local mode; mode=$(accel_mode)
log "[JOB] start id=$job_id mode=$mode type=$workload"
mkdir -p "$WORK_DIR/$job_id"
# Example workload types: "torch_bench", "sd_infer", "llm_gen"...
case "$workload" in
torch_bench)
/bin/bash -lc "source /opt/aitbc-miner/.venv/bin/activate && python - <<'PY'
import time, torch
start=time.time()
x=torch.randn(1024,1024,device='cuda' if torch.cuda.is_available() else 'cpu')
for _ in range(100): x=x@x
elapsed=time.time()-start
print(f"throughput_ops={100}, seconds={elapsed:.4f}")
PY" | tee -a "$LOG_FILE" >"$WORK_DIR/$job_id/out.txt"
;;
sd_infer)
echo "stub: run stable diffusion pipeline here" | tee -a "$LOG_FILE" >"$WORK_DIR/$job_id/out.txt"
;;
llm_gen)
echo "stub: run text generation here" | tee -a "$LOG_FILE" >"$WORK_DIR/$job_id/out.txt"
;;
*)
echo "unknown workload" >"$WORK_DIR/$job_id/out.txt" ;;
esac
# Build a minimal proof (hash of outputs + metrics placeholder)
local proof; proof=$(jq -n --arg id "$job_id" \
--arg mode "$mode" \
--arg out_sha "$(sha256sum "$WORK_DIR/$job_id/out.txt" | awk '{print $1}')" \
'{id:$id, mode:$mode, output_sha:$out_sha, metrics:{}}')
req POST "/v1/miner/proof" "$proof" >/dev/null
log "[JOB] done id=$job_id proof_submitted"
}
heartbeat(){
local mode; mode=$(accel_mode)
local gpu; gpu=$(has_gpu && echo 1 || echo 0)
req POST "/v1/miner/heartbeat" "$(jq -n \
--arg id "$MINER_ID" --arg w "$WALLET_ADDR" --arg mode "$mode" \
--argjson gpu "$gpu" '{miner_id:$id,wallet:$w,mode:$mode,gpu:$gpu}')" >/dev/null
}
# ========
# Main Process
# ========
log "[BOOT] AITBC miner starting (id=$MINER_ID)"
mkdir -p "$WORK_DIR"
# Prime heartbeat
heartbeat || log "[WARN] initial heartbeat failed"
# Poll/execute loop
while true; do
sleep "$JOB_POLL_SECS"
# Opportunistic heartbeat
heartbeat || true
# Fetch one job
if J=$(req GET "/v1/miner/next?miner=$MINER_ID&slots=$MAX_CONCURRENCY" 2>/dev/null); then
echo "$J" | jq -e '.id' >/dev/null 2>&1 || continue
run_job "$J" || log "[WARN] job failed"
fi
done
```
Make executable:
```
chmod +x /root/scripts/aitbc-miner/install_miner.sh /root/scripts/aitbc-miner/miner.sh
```
---
## 6) Bootstrap
1. Create folders + drop the three files above.
2. Edit `/etc/aitbc/miner.conf` with real values.
3. Run installer:
```
/root/scripts/aitbc-miner/install_miner.sh
```
4. Check status & logs:
```
systemctl status aitbc-miner.service
tail -f /var/log/aitbc-miner.log
```
---
## 7) Health & Debug
- Quick GPU sanity: `nvidia-smi` (optional).
- Liveness: periodic `/v1/miner/heartbeat` pings.
- Job smoke test (coordinator): ensure `/v1/miner/next` returns a JSON job.
---
## 8) Logrotate (optional)
Create `/etc/logrotate.d/aitbc-miner`:
```conf
/var/log/aitbc-miner.log {
rotate 7
daily
missingok
notifempty
compress
copytruncate
}
```
---
## 9) Security Notes
- Keep `API_KEY` scoped to miner ops with revocation.
- No inbound ports required; allow egress HTTPS only.
- Consider systemd `ProtectSystem=full`, `ProtectHome=yes`, `NoNewPrivileges=yes` hardening once stable.
---
## 10) Extending Workloads
- Implement real `sd_infer` (Stable Diffusion) and `llm_gen` via the venv.
- Add job-level resource caps (VRAM/CPU-time) from Coordinator parameters.
- Attach accounting metrics for reward weighting (e.g., `tokens_per_kJ` or `tokens_per_TFLOP_s`).
---
## 11) Common Commands
```
# restart after config edits
systemctl restart aitbc-miner.service
# follow logs
journalctl -u aitbc-miner -f
# disable autostart
systemctl disable --now aitbc-miner.service
```
---
## 12) Coordinator API v1 (Detailed)
This section specifies the **Miner-facing** HTTP API. All endpoints are versioned under `/v1/` and use **JSON** over **HTTPS**. Authentication is via `Authorization: Bearer <API_KEY>` issued by the wallet/coordinator.
### 12.1 Global
- **Base URL**: `${COORD_URL}` (e.g., `https://coordinator.example.net`).
- **Content-Type**: `application/json; charset=utf-8`.
- **Auth**: `Authorization: Bearer <API_KEY>` (scoped for miner ops).
- **Idempotency** *(recommended)*: `Idempotency-Key: <uuid4>` for POSTs.
- **Clock**: All timestamps are ISO‑8601 UTC (e.g., `2025-09-26T13:37:00Z`).
- **Errors**: Non‑2xx responses return a body:
```json
{ "error": { "code": "STRING_CODE", "message": "human readable", "details": {"field": "optional context"} } }
```
- **Common HTTP codes**: `200 OK`, `201 Created`, `204 No Content`, `400 Bad Request`, `401 Unauthorized`, `403 Forbidden`, `404 Not Found`, `409 Conflict`, `422 Unprocessable Entity`, `429 Too Many Requests`, `500/502/503`.
---
### 12.2 Types
#### MinerCapabilities
```json
{
"miner_id": "string",
"mode": "gpu|cpu",
"gpu": true,
"concurrency": 1,
"workloads": ["torch_bench", "sd_infer", "llm_gen"],
"limits": {"vram_mb": 16000, "ram_mb": 32768, "max_runtime_s

412
docs/reference/bootstrap/miner_node.md Normal file
View File

@ -0,0 +1,412 @@
# miner-node/ — Worker Node Daemon for GPU/CPU Tasks
> **Goal:** Implement a Docker‑free worker daemon that connects to the Coordinator API, advertises capabilities (CPU/GPU), fetches jobs, executes them in a sandboxed workspace, and streams results/metrics back.
---
## 1) Scope & MVP
**MVP Features**
- Node registration with Coordinator (auth token + capability descriptor).
- Heartbeat & liveness (interval ± jitter, backoff on failure).
- Job fetch → ack → execute → upload result → finalize.
- Two runner types:
- **CLI runner**: executes a provided command with arguments (whitelist‑based).
- **Python runner**: executes a trusted task module with parameters.
- CPU/GPU capability detection (CUDA, VRAM, driver info) without Docker.
- Sandboxed working dir per job under `/var/lib/aitbc/miner/jobs/<job-id>`.
- Resource controls (nice/ionice/ulimit; optional cgroup v2 if present).
- Structured JSON logging and minimal metrics.
**Post‑MVP**
- Chunked artifact upload; resumable transfers.
- Prometheus `/metrics` endpoint (pull).
- GPU multi‑card scheduling & fractional allocation policy.
- On‑node model cache management (size, eviction, pinning).
- Signed task manifests & attestation of execution.
- Secure TMPFS for secrets; hardware key support (YubiKey).
---
## 2) High‑Level Architecture
```
client → coordinator-api → miner-node(s) → results store → coordinator-api → client
```
Miner components:
- **Agent** (control loop): registration, heartbeat, fetch/dispatch, result reporting.
- **Capability Probe**: CPU/GPU inventory (CUDA, VRAM), free RAM/disk, load.
- **Schedulers**: simple FIFO for MVP; one job per GPU or CPU slot.
- **Runners**: CLI runner & Python runner.
- **Sandbox**: working dirs, resource limits, network I/O gating (optional), file allowlist.
- **Telemetry**: JSON logs, minimal metrics; per‑job timeline.
---
## 3) Directory Layout (on node)
```
/var/lib/aitbc/miner/
├─ jobs/
│ ├─ <job-id>/
│ │ ├─ input/
│ │ ├─ work/
│ │ ├─ output/
│ │ └─ logs/
├─ cache/ # model/assets cache (optional)
└─ tmp/
/etc/aitbc/miner/
├─ config.yaml
└─ allowlist.d/ # allowed CLI programs & argument schema snippets
/var/log/aitbc/miner/
/usr/local/lib/aitbc/miner/ # python package venv install target
```
---
## 4) Config (YAML)
```yaml
node_id: "node-<shortid>"
coordinator:
base_url: "https://coordinator.local/api/v1"
auth_token: "env:MINER_AUTH" # read from env at runtime
tls_verify: true
timeout_s: 20
heartbeat:
interval_s: 15
jitter_pct: 10
backoff:
min_s: 5
max_s: 120
runners:
cli:
enable: true
allowlist_files:
- "/etc/aitbc/miner/allowlist.d/ffmpeg.yaml"
- "/etc/aitbc/miner/allowlist.d/whisper.yaml"
python:
enable: true
task_paths:
- "/usr/local/lib/aitbc/miner/tasks"
venv: "/usr/local/lib/aitbc/miner/.venv"
resources:
max_concurrent_cpu: 2
max_concurrent_gpu: 1
cpu_nice: 10
io_class: "best-effort"
io_level: 6
mem_soft_mb: 16384
workspace:
root: "/var/lib/aitbc/miner/jobs"
keep_success: 24h
keep_failed: 7d
logging:
level: "info"
json: true
path: "/var/log/aitbc/miner/miner.jsonl"
```
---
## 5) Environment & Dependencies
- **OS:** Debian 12/13 (systemd).
- **Python:** 3.11+ in venv under `/usr/local/lib/aitbc/miner/.venv`.
- **Libraries:** `httpx`, `pydantic`, `uvloop` (optional), `pyyaml`, `psutil`.
- **GPU (optional):** NVIDIA driver installed; `nvidia-smi` available; CUDA 12.x runtime on path for GPU tasks.
**Install skeleton**
```
python3 -m venv /usr/local/lib/aitbc/miner/.venv
/usr/local/lib/aitbc/miner/.venv/bin/pip install --upgrade pip
/usr/local/lib/aitbc/miner/.venv/bin/pip install httpx pydantic pyyaml psutil uvloop
install -d /etc/aitbc/miner /var/lib/aitbc/miner/{jobs,cache,tmp} /var/log/aitbc/miner
```
---
## 6) Systemd Service
**/etc/systemd/system/aitbc-miner.service**
```
[Unit]
Description=AITBC Miner Node
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
Environment=MINER_AUTH=***REDACTED***
ExecStart=/usr/local/lib/aitbc/miner/.venv/bin/python -m aitbc_miner --config /etc/aitbc/miner/config.yaml
User=games
Group=games
# Lower CPU/IO priority by default
Nice=10
IOSchedulingClass=best-effort
IOSchedulingPriority=6
Restart=always
RestartSec=5
# Hardening
NoNewPrivileges=true
ProtectSystem=full
ProtectHome=true
PrivateTmp=true
ReadWritePaths=/var/lib/aitbc/miner /var/log/aitbc/miner
[Install]
WantedBy=multi-user.target
```
---
## 7) Capability Probe (sent to Coordinator)
Example payload:
```json
{
"node_id": "node-abc123",
"version": "0.1.0",
"cpu": {"cores": 16, "arch": "x86_64"},
"memory_mb": 64000,
"disk_free_mb": 250000,
"gpu": [
{
"vendor": "nvidia", "name": "RTX 4060 Ti 16GB",
"vram_mb": 16384,
"cuda": {"version": "12.3", "driver": "545.23.06"}
}
],
"runners": ["cli", "python"],
"tags": ["debian", "cuda", "cpu"]
}
```
---
## 8) Coordinator API Contract (MVP)
**Endpoints (HTTPS, JSON):**
- `POST /nodes/register` → returns signed `node_token` (or 401)
- `POST /nodes/heartbeat` → `{node_id, load, free_mb, gpu_free}` → 200
- `POST /jobs/pull` → `{node_id, filters}` → `{job|none}`
- `POST /jobs/ack` → `{job_id, node_id}` → 200
- `POST /jobs/progress` → `{job_id, pct, note}` → 200
- `POST /jobs/result` → multipart (metadata.json + artifacts/*) → 200
- `POST /jobs/fail` → `{job_id, error_code, error_msg, logs_ref}` → 200
**Auth**
- Bearer token in header (Node → Coordinator): `Authorization: Bearer <node_token>`
- Coordinator signs `job.manifest` with HMAC(sha256) or Ed25519 (post‑MVP).
**Job manifest (subset)**
```json
{
"job_id": "j-20250926-001",
"runner": "cli",
"requirements": {"gpu": true, "vram_mb": 12000, "cpu_threads": 4},
"timeout_s": 3600,
"input": {"urls": ["https://.../input1"], "inline": {"text": "..."}},
"command": "ffmpeg",
"args": ["-y", "-i", "input1.mp4", "-c:v", "libx264", "output.mp4"],
"artifacts": [{"path": "output.mp4", "type": "video/mp4", "max_mb": 5000}]
}
```
---
## 9) Runner Design
### CLI Runner
- Validate `command` against allowlist (`/etc/aitbc/miner/allowlist.d/*.yaml`).
- Validate `args` against per‑tool schema (regex & size caps).
- Materialize inputs in job workspace; set `PATH`, `CUDA_VISIBLE_DEVICES`.
- Launch via `subprocess.Popen` with `preexec_fn` applying `nice`, `ionice`, `setrlimit`.
- Live‑tail stdout/stderr to `logs/exec.log`; throttle progress pings.
### Python Runner
- Import trusted module `tasks.<name>:run(**params)` from configured paths.
- Run in same venv; optional `venv per task` later.
- Enforce timeouts; capture logs; write artifacts to `output/`.
---
## 10) Resource Controls (No Docker)
- **CPU:** `nice(10)`; optional cgroups v2 CPU.max if available.
- **IO:** `ionice -c 2 -n 6` (best‑effort) for heavy disk ops.
- **Memory:** `setrlimit(RLIMIT_AS)` soft cap; kill on OOM.
- **GPU:** select by policy (least used VRAM). No hard memory partitioning in MVP.
- **Network:** allowlist outbound hosts; deny by default (optional phase‑2).
---
## 11) Job Lifecycle (State Machine)
`IDLE → PULLING → ACKED → PREP → RUNNING → UPLOADING → DONE | FAILED | RETRY_WAIT`
- Retries: exponential backoff, max N; idempotent uploads.
- On crash: on‑start recovery scans `jobs/*/state.json` and reconciles with Coordinator.
---
## 12) Logging & Metrics
- JSON lines in `/var/log/aitbc/miner/miner.jsonl` with fields: `ts, level, node_id, job_id, event, attrs{}`.
- Optional `/healthz` (HTTP) returning 200 + brief status.
- Future: Prometheus `/metrics` with gauges (queue, running, VRAM free, CPU load).
---
## 13) Security Model
- TLS required; pin CA or enable cert validation per env.
- Node bootstrap token (`MINER_AUTH`) exchanged for `node_token` at registration.
- Strict allowlist for CLI tools + args; size/time caps.
- Secrets never written to disk unencrypted; pass via env vars or in‑memory.
- Wipe workdirs on success (per policy); keep failed for triage.
---
## 14) Windsurf Implementation Plan
**Milestone 1 — Skeleton**
1. `aitbc_miner/` package: `main.py`, `config.py`, `agent.py`, `probe.py`, `runners/{cli.py, python.py}`, `util/{limits.py, fs.py, log.py}`.
2. Load YAML config, bootstrap logs, print probe JSON.
3. Implement `/healthz` (optional FastAPI or bare aiohttp) for local checks.
**Milestone 2 — Control Loop**
1. Register → store `node_token` (in memory only).
2. Heartbeat task (async), backoff on network errors.
3. Pull/ack & single‑slot executor; write state.json.
**Milestone 3 — Runners**
1. CLI allowlist loader + validator; subprocess with limits.
2. Python runner calling `tasks.example:run`.
3. Upload artifacts via multipart; handle large files with chunking stub.
**Milestone 4 — Hardening & Ops**
1. Crash recovery; cleanup policy; TTL sweeper.
2. Metrics counters; structured logging fields.
3. Systemd unit; install scripts; doc.
---
## 15) Minimal Allowlist Example (ffmpeg)
```yaml
# /etc/aitbc/miner/allowlist.d/ffmpeg.yaml
command:
path: "/usr/bin/ffmpeg"
args:
- ["-y"]
- ["-i", ".+\\.(mp4|wav|mkv)$"]
- ["-c:v", "(libx264|copy)"]
- ["-c:a", "(aac|copy)"]
- ["-b:v", "[1-9][0-9]{2,5}k"]
- ["output\\.(mp4|mkv)"]
max_total_args_len: 4096
max_runtime_s: 7200
max_output_mb: 5000
```
---
## 16) Mock Coordinator (for local testing)
> Run a tiny dev server to hand out a single job and accept results.
```python
# mock_coordinator.py (FastAPI)
from fastapi import FastAPI, UploadFile, File, Form
from pydantic import BaseModel
app = FastAPI()
JOB = {
"job_id": "j-local-1",
"runner": "cli",
"requirements": {"gpu": False},
"timeout_s": 120,
"command": "echo",
"args": ["hello", "world"],
"artifacts": [{"path": "output.txt", "type": "text/plain", "max_mb": 1}]
}
class PullReq(BaseModel):
node_id: str
filters: dict | None = None
@app.post("/api/v1/jobs/pull")
def pull(req: PullReq):
return {"job": JOB}
@app.post("/api/v1/jobs/ack")
def ack(job_id: str, node_id: str):
return {"ok": True}
@app.post("/api/v1/jobs/result")
def result(job_id: str = Form(...), metadata: str = Form(...), artifact: UploadFile = File(...)):
return {"ok": True}
```
---
## 17) Developer UX (Make Targets)
```
make venv # create venv + install deps
make run # run miner with local config
make fmt # ruff/black (optional)
make test # unit tests
```
---
## 18) Operational Runbook
- **Start/Stop**: `systemctl enable --now aitbc-miner`
- **Logs**: `journalctl -u aitbc-miner -f` and `/var/log/aitbc/miner/miner.jsonl`
- **Rotate**: logrotate config (size 50M, keep 7)
- **Upgrade**: drain → stop → replace venv → start → verify heartbeat
- **Health**: `/healthz` 200 + JSON `{running, queued, cpu_load, vram_free}`
---
## 19) Failure Modes & Recovery
- **Network errors**: exponential backoff; keep heartbeat local status.
- **Job invalid**: fail fast with reason; do not retry.
- **Runner denied**: allowlist miss → fail with `E_DENY`.
- **OOM**: kill process group; mark `E_OOM`.
- **GPU unavailable**: requeue with reason `E_NOGPU`.
---
## 20) Roadmap Notes
- Binary task bundles with signed SBOM.
- Remote cache warming via Coordinator hints.
- Multi‑Queue scheduling (latency vs throughput).
- MIG/compute‑instance support if hardware allows.
---
## 21) Checklist for Windsurf
1. Create `aitbc_miner/` package skeleton with modules listed in §14.
2. Implement config loader + capability probe output.
3. Implement async agent loop: register → heartbeat → pull/ack.
4. Implement CLI runner with allowlist (§15) and exec log.
5. Implement Python runner stub (`tasks/example.py`).
6. Write result uploader (multipart) and finalize call.
7. Add systemd unit (§6) and basic install script.
8. Test end‑to‑end against `mock_coordinator.py` (§16).
9. Document log fields + troubleshooting card.
10. Add optional `/healthz` endpoint.

314
docs/reference/bootstrap/pool_hub.md Normal file
View File

@ -0,0 +1,314 @@
# pool-hub.md — Client ↔ Miners Pool & Matchmaking Gateway (guide for Windsurf)
> **Role in AITBC**
> The **Pool Hub** is the real-time directory of available miners and a low-latency **matchmaker** between **job requests** (coming from `coordinator-api`) and **worker capacity** (from `miner-node`).
> It tracks miner capabilities, health, and price; computes a score; and returns the **best N candidates** for each job.
---
## 1) MVP Scope (Stage 3 of the boot plan)
- Accept **miner registrations** + **heartbeats** (capabilities, price, queues).
- Maintain an in-memory + persistent **miner registry** with fast lookups.
- Provide a **/match** API for `coordinator-api` to request top candidates.
- Simple **scoring** with pluggable strategy (latency, VRAM, price, trust).
- **No token minting/accounting** in MVP (stub hooks only).
- **No Docker**: Debian 12, `uvicorn` service, optional Nginx reverse proxy.
---
## 2) High-Level Architecture
```
client-web ──> coordinator-api ──> pool-hub ──> miner-node
▲ ▲
│ │
wallet-daemon heartbeat + capability updates
(later) (WebSocket/HTTP)
```
### Responsibilities
- **Registry**: who’s online, what they can do, how much it costs.
- **Health**: heartbeats, timeouts, grace periods, auto-degrade/remove.
- **Matchmaking**: filter → score → return top K miner candidates.
- **Observability**: metrics for availability, match latency, rejection reasons.
---
## 3) Protocols & Endpoints (FastAPI)
> Base URL examples:
> - Public (optional): `https://pool-hub.example/api`
> - Internal (preferred): `http://127.0.0.1:8203` (behind Nginx)
### 3.1 Miner lifecycle (HTTP + WebSocket)
- `POST /v1/miners/register`
- **Body**: `{ miner_id, api_key, addr, proto, gpu_vram, gpu_name, cpu_cores, ram_gb, price_token_per_ksec, max_parallel, tags[], capabilities[] }`
- **Returns**: `{ status, lease_ttl_sec, next_heartbeat_sec }`
- Notes: returns a **session_token** (short-lived) if `api_key` valid.
- `POST /v1/miners/update`
- **Body**: `{ session_token, queue_len, busy, current_jobs[], price_token_per_ksec? }`
- `GET /v1/miners/lease/renew?session_token=...`
- Renews online lease (fallback if WS drops).
- `WS /v1/miners/heartbeat`
- **Auth**: `session_token` in query.
- **Payload (periodic)**: `{ ts, queue_len, avg_latency_ms?, temp_c?, mem_free_gb? }`
- Server may push **commands** (e.g., “update tags”, “set price cap”).
- `POST /v1/miners/logout`
- Clean unregister (otherwise lease expiry removes).
### 3.2 Coordinator matchmaking (HTTP)
- `POST /v1/match`
- **Body**:
```json
{
"job_id": "uuid",
"requirements": {
"task": "image_embedding",
"min_vram_gb": 8,
"min_ram_gb": 8,
"accel": ["cuda"],
"tags_any": ["eu-west","low-latency"],
"capabilities_any": ["sentence-transformers/all-MiniLM-L6-v2"]
},
"hints": { "region": "eu", "max_price": 0.8, "deadline_ms": 5000 },
"top_k": 3
}
```
- **Returns**:
```json
{
"job_id": "uuid",
"candidates": [
{ "miner_id":"...", "addr":"...", "proto":"grpc", "score":0.87, "eta_ms": 320, "price":0.75 },
{ "miner_id":"...", "addr":"...", "proto":"grpc", "score":0.81, "eta_ms": 410, "price":0.65 }
],
"explain": "cap=1.0 • price=0.8 • latency=0.9 • trust=0.7"
}
```
- `POST /v1/feedback`
- **Body**: `{ job_id, miner_id, outcome: "accepted"|"rejected"|"failed"|"completed", latency_ms?, fail_code?, tokens_spent? }`
- Used to adjust **trust score** & calibration.
### 3.3 Observability
- `GET /v1/health` → `{ status:"ok", online_miners, avg_latency_ms }`
- `GET /v1/metrics` → Prometheus-style metrics (text)
- `GET /v1/miners` (admin-guarded) → paginated registry snapshot
---
## 4) Data Model (PostgreSQL minimal)
> Schema name: `poolhub`
**tables**
- `miners`
`miner_id PK, api_key_hash, created_at, last_seen_at, addr, proto, gpu_vram_gb, gpu_name, cpu_cores, ram_gb, max_parallel, base_price, tags jsonb, capabilities jsonb, trust_score float default 0.5, region text`
- `miner_status`
`miner_id FK, queue_len int, busy bool, avg_latency_ms int, temp_c int, mem_free_gb float, updated_at`
- `feedback`
`id PK, job_id, miner_id, outcome, latency_ms, fail_code, tokens_spent, created_at`
- `price_overrides` (optional)
**indexes**
- `idx_miners_region_caps_gte_vram` (GIN on jsonb + partials)
- `idx_status_updated_at`
- `idx_feedback_miner_time`
**in-memory caches**
- Hot registry (Redis or local LRU) for sub-millisecond match filter.
---
## 5) Matching & Scoring
**Filter**:
- Hard constraints: `min_vram_gb`, `min_ram_gb`, `accel`, `capabilities_any`, `region`, `max_price`, `queue_len < max_parallel`.
**Score formula (tunable)**:
```
score = w_cap*cap_fit
+ w_price*price_norm
+ w_latency*latency_norm
+ w_trust*trust_score
+ w_load*load_norm
```
- `cap_fit`: 1 if all required caps present, else <1 proportional to overlap.
- `price_norm`: cheaper = closer to 1 (normalized vs request cap).
- `latency_norm`: 1 for fastest observed in region, decays by percentile.
- `load_norm`: higher if queue_len small and max_parallel large.
- Default weights: `w_cap=0.40, w_price=0.20, w_latency=0.20, w_trust=0.15, w_load=0.05`.
Return top-K with **explain string** for debugging.
---
## 6) Security (MVP → later hardening)
- **Auth**:
- Miners: static `api_key` → exchange for short **session_token**.
- Coordinator: **shared secret** header (MVP), rotate via env.
- **Network**:
- Bind to localhost; expose via Nginx with IP allowlist for coordinator.
- Rate limits on `/match` and `/register`.
- **Later**:
- mTLS between pool-hub ↔ miners.
- Signed capability manifests.
- Attestation (NVIDIA MIG/hash) optional.
---
## 7) Configuration
- `.env` (loaded by FastAPI):
- `POOLHUB_BIND=127.0.0.1`
- `POOLHUB_PORT=8203`
- `DB_DSN=postgresql://poolhub:*****@127.0.0.1:5432/aitbc`
- `REDIS_URL=redis://127.0.0.1:6379/4` (optional)
- `COORDINATOR_SHARED_SECRET=...`
- `SESSION_TTL_SEC=60`
- `HEARTBEAT_GRACE_SEC=120`
- `DEFAULT_WEIGHTS=cap:0.4,price:0.2,latency:0.2,trust:0.15,load:0.05`
---
## 8) Local Dev (Debian 12, no Docker)
```bash
# Python env
apt install -y python3-venv python3-pip
python3 -m venv .venv && source .venv/bin/activate
pip install fastapi uvicorn[standard] pydantic-settings psycopg[binary] orjson redis prometheus-client
# DB (psql one-liners, adjust to your policy)
psql -U postgres -c "CREATE USER poolhub WITH PASSWORD '***';"
psql -U postgres -c "CREATE DATABASE aitbc OWNER poolhub;"
psql -U postgres -d aitbc -c "CREATE SCHEMA poolhub AUTHORIZATION poolhub;"
# Run
uvicorn app.main:app --host 127.0.0.1 --port 8203 --workers 1 --proxy-headers
```
---
## 9) Systemd service (uvicorn)
`/etc/systemd/system/pool-hub.service`
```
[Unit]
Description=AITBC Pool Hub (FastAPI)
After=network-online.target postgresql.service
Wants=network-online.target
[Service]
User=games
Group=games
WorkingDirectory=/var/www/aitbc/pool-hub
Environment=PYTHONUNBUFFERED=1
EnvironmentFile=/var/www/aitbc/pool-hub/.env
ExecStart=/var/www/aitbc/pool-hub/.venv/bin/uvicorn app.main:app --host 127.0.0.1 --port 8203 --workers=1 --proxy-headers
Restart=always
RestartSec=2
[Install]
WantedBy=multi-user.target
```
---
## 10) Optional Nginx (host as reverse proxy)
```
server {
listen 443 ssl http2;
server_name pool-hub.example;
# ssl_certificate ...; ssl_certificate_key ...;
# Only allow coordinator’s IPs
allow 10.0.3.32;
allow 127.0.0.1;
deny all;
location / {
proxy_pass http://127.0.0.1:8203;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header X-Forwarded-Proto https;
}
}
```
---
## 11) FastAPI App Skeleton (files to create)
```
pool-hub/
app/
__init__.py
main.py # FastAPI init, routers include, metrics
deps.py # auth, db sessions, rate limits
models.py # SQLAlchemy/SQLModel tables (see schema)
schemas.py # Pydantic I/O models
registry.py # in-memory index + Redis bridge
scoring.py # score strategies
routers/
miners.py # register/update/ws/logout
match.py # /match and /feedback
admin.py # /miners snapshot (guarded)
health.py # /health, /metrics
.env.example
README.md
```
---
## 12) Testing & Validation Checklist
1. Register 3 miners (mixed VRAM, price, region).
2. Heartbeats arrive; stale miners drop after `HEARTBEAT_GRACE_SEC`.
3. `/match` with constraints returns consistent top-K and explain string.
4. Rate limits kick in (flood `/match` with 50 rps).
5. Coordinator feedback adjusts trust score; observe score shifts.
6. Systemd restarts on crash; Nginx denies non-coordinator IPs.
7. Metrics expose `poolhub_miners_online`, `poolhub_match_latency_ms`.
---
## 13) Roadmap (post-MVP)
- Weighted **multi-dispatch** (split jobs across K miners, merge).
- **Price discovery**: spot vs reserved capacity.
- **mTLS**, signed manifests, hardware attestation.
- **AIToken hooks**: settle/pay via `wallet-daemon` + `blockchain-node`.
- Global **region routing** + latency probes.
---
## 14) Dev Notes for Windsurf
- Start with **routers/miners.py** and **routers/match.py**.
- Implement `registry.py` with a simple in-proc dict + RWLock; add Redis later.
- Keep scoring in **scoring.py** with clear weight constants and unit tests.
- Provide a tiny CLI seed to simulate miners for local testing.
---
### Open Questions
- Trust model inputs (what counts as failure vs transient)?
- Minimal capability schema vs free-form tags?
- Region awareness from IP vs miner-provided claim?
---
**Want me to generate the initial FastAPI file stubs for this layout now (yes/no)?**

335
docs/reference/bootstrap/wallet_daemon.md Normal file
View File

@ -0,0 +1,335 @@
# wallet-daemon.md
> **Role:** Local wallet service (keys, signing, RPC) for the 🤖 AITBC 🤑 stack
> **Audience:** Windsurf (programming assistant) + developers
> **Stage:** Bootable without blockchain; later pluggable to chain node
---
## 1) What this daemon is (and isn’t)
**Goals**
1. Generate/import encrypted wallets (seed or raw keys).
2. Derive accounts/addresses (HD, multiple curves).
3. Hold keys **locally** and sign messages/transactions/receipts.
4. Expose a minimal **RPC** for client/coordinator/miner.
5. Provide a mock “ledger view” (balance cache + test transfers) until the chain is ready.
**Non-Goals (for now)**
- No P2P networking.
- No full node / block validation.
- No remote key export (never leaves box unencrypted).
---
## 2) Architecture (minimal, extensible)
- **Process:** Python FastAPI app (`uvicorn`)
- **RPC:** HTTP+JSON (REST) and JSON-RPC (both; same server)
- **KeyStore:** on-disk, encrypted with **Argon2id + XChaCha20-Poly1305**
- **Curves:** `ed25519` (default), `secp256k1` (optional flag per-account)
- **HD Derivation:** BIP-39 seed → SLIP-10 (ed25519), BIP-32 (secp256k1)
- **Coin type (provisional):** `AITBC = 12345` (placeholder; replace once registered)
- **Auth:** Local-only by default (bind `127.0.0.1`), optional token header for remote.
- **Events:** Webhooks + local FIFO/Unix-socket stream for “signed” notifications.
- **Mock Ledger (stage-1):** sqlite table for balances & transfers; sync adapter later.
**Directory layout**
```
wallet-daemon/
├─ app/ # FastAPI service
│ ├─ main.py # entrypoint
│ ├─ api_rest.py # REST routes
│ ├─ api_jsonrpc.py # JSON-RPC methods
│ ├─ crypto/ # key, derivation, sign, addr
│ ├─ keystore/ # encrypted store backend
│ ├─ models/ # pydantic schemas
│ ├─ ledger_mock/ # sqlite-backed balances
│ └─ settings.py # config
├─ data/
│ ├─ keystore/ # *.kdb (per wallet)
│ └─ ledger.db
├─ tests/
└─ run.sh
```
---
## 3) Security model (Windsurf implementation notes)
- **Passwords:** Argon2id (tuned to machine; env overrides)
- `ARGON_TIME=4`, `ARGON_MEMORY=256MB`, `ARGON_PARALLELISM=2` (defaults)
- **Encryption:** libsodium `crypto_aead_xchacha20poly1305_ietf`
- **At-Rest Format (per wallet file)**
```
magic=v1-kdb
salt=32B
argon_params={t,m,p}
nonce=24B
ciphertext=… # includes seed or master private key; plus metadata JSON
```
- **In-memory:** zeroize sensitive bytes after use; use `memoryview`/`ctypes` scrubbing where possible.
- **API hardening:**
- Bind to `127.0.0.1` by default.
- Optional `X-Auth-Token: <TOKEN>` for REST/JSON-RPC.
- Rate limits on sign endpoints.
---
## 4) Key & address strategy
- **Wallet types**
1. **HD (preferred):** BIP-39 mnemonic (12/24 words) → master seed.
2. **Single-key:** import raw private key (ed25519/secp256k1).
- **Derivation paths**
- **ed25519 (SLIP-10):** `m / 44' / 12345' / account' / change / index`
- **secp256k1 (BIP-44):** `m / 44' / 12345' / account' / change / index`
- **Address format (temporary)**
- **ed25519:** `base32(bech32_hrp="ait")` of `blake2b-20(pubkey)`
- **secp256k1:** same hash → bech32; flag curve in metadata
> Replace HRP and hash rules if the canonical AITBC chain spec differs.
---
## 5) REST API (for Windsurf to scaffold)
**Headers**
- Optional: `X-Auth-Token: <token>`
- `Content-Type: application/json`
### 5.1 Wallet lifecycle
- `POST /v1/wallet/create`
- body: `{ "name": "main", "password": "…", "mnemonic_len": 24 }`
- returns: `{ "wallet_id": "…" }`
- `POST /v1/wallet/import-mnemonic`
- `{ "name":"…","password":"…","mnemonic":"…","passphrase":"" }`
- `POST /v1/wallet/import-key`
- `{ "name":"…","password":"…","curve":"ed25519|secp256k1","private_key_hex":"…" }`
- `POST /v1/wallet/unlock`
- `{ "wallet_id":"…","password":"…" }` → unlocks into memory for N seconds (config TTL)
- `POST /v1/wallet/lock` → no body
- `GET /v1/wallets` → list minimal metadata (never secrets)
### 5.2 Accounts & addresses
- `POST /v1/account/derive`
- `{ "wallet_id":"…","curve":"ed25519","path":"m/44'/12345'/0'/0/0" }`
- `GET /v1/accounts?wallet_id=…`
- `GET /v1/address/{account_id}` → `{ "address":"ait1…", "curve":"ed25519" }`
### 5.3 Signing
- `POST /v1/sign/message`
- `{ "account_id":"…","message_base64":"…" }` → `{ "signature_base64":"…" }`
- `POST /v1/sign/tx`
- `{ "account_id":"…","tx_bytes_base64":"…","type":"aitbc_v0" }` → signature + (optionally) signed blob
- `POST /v1/sign/receipt`
- `{ "account_id":"…","payload":"…"}`
- **Used by coordinator/miner** to sign job receipts & payouts.
### 5.4 Mock ledger (stage-1 only)
- `GET /v1/ledger/balance/{address}`
- `POST /v1/ledger/transfer`
- `{ "from":"…","to":"…","amount":"123.456","memo":"test" }`
- `GET /v1/ledger/tx/{txid}`
### 5.5 Webhooks
- `POST /v1/webhooks`
- `{ "url":"https://…/callback", "events":["signed","transfer"] }`
**Error model**
```json
{ "error": { "code": "WALLET_LOCKED|NOT_FOUND|BAD_PASSWORD|RATE_LIMIT", "detail": "…" } }
```
---
## 6) JSON-RPC mirror (method → params)
- `wallet_create(name, password, mnemonic_len)`
- `wallet_import_mnemonic(name, password, mnemonic, passphrase)`
- `wallet_import_key(name, password, curve, private_key_hex)`
- `wallet_unlock(wallet_id, password)` / `wallet_lock()`
- `account_derive(wallet_id, curve, path)`
- `sign_message(account_id, message_base64)`
- `sign_tx(account_id, tx_bytes_base64, type)`
- `ledger_getBalance(address)` / `ledger_transfer(from, to, amount, memo)`
Same auth header; endpoint `/rpc`.
---
## 7) Data schemas (Pydantic hints)
```py
class WalletMeta(BaseModel):
wallet_id: str
name: str
curves: list[str] # ["ed25519", "secp256k1"]
created_at: datetime
class AccountMeta(BaseModel):
account_id: str
wallet_id: str
curve: Literal["ed25519","secp256k1"]
path: str # HD path or "imported"
address: str # bech32 ait1...
class SignedResult(BaseModel):
signature_base64: str
public_key_hex: str
algo: str # e.g., ed25519
```
---
## 8) Configuration (ENV)
```
WALLET_BIND=127.0.0.1
WALLET_PORT=8555
WALLET_TOKEN= # optional
KEYSTORE_DIR=./data/keystore
LEDGER_DB=./data/ledger.db
UNLOCK_TTL_SEC=120
ARGON_TIME=4
ARGON_MEMORY_MB=256
ARGON_PARALLELISM=2
```
---
## 9) Minimal boot script (dev)
```bash
# run.sh
export WALLET_BIND=127.0.0.1
export WALLET_PORT=8555
export KEYSTORE_DIR=./data/keystore
mkdir -p "$KEYSTORE_DIR" data
exec uvicorn app.main:app --host ${WALLET_BIND} --port ${WALLET_PORT}
```
---
## 10) Systemd unit (prod, template)
```
[Unit]
Description=AITBC Wallet Daemon
After=network.target
[Service]
User=root
WorkingDirectory=/opt/aitbc/wallet-daemon
Environment=WALLET_BIND=127.0.0.1
Environment=WALLET_PORT=8555
Environment=KEYSTORE_DIR=/opt/aitbc/wallet-daemon/data/keystore
ExecStart=/usr/bin/uvicorn app.main:app --host 127.0.0.1 --port 8555
Restart=always
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=full
[Install]
WantedBy=multi-user.target
```
---
## 11) Curl smoke tests
```bash
# 1) Create + unlock
curl -s localhost:8555/v1/wallet/create -X POST \
-H 'Content-Type: application/json' \
-d '{"name":"main","password":"pw","mnemonic_len":24}'
curl -s localhost:8555/v1/wallets
# 2) Derive first account
curl -s localhost:8555/v1/account/derive -X POST \
-H 'Content-Type: application/json' \
-d '{"wallet_id":"W1","curve":"ed25519","path":"m/44'/12345'/0'/0/0"}'
# 3) Sign message
curl -s localhost:8555/v1/sign/message -X POST \
-H 'Content-Type: application/json' \
-d '{"account_id":"A1","message_base64":"aGVsbG8="}'
```
---
## 12) Coordinator/miner integration (stage-1)
**Coordinator needs:**
- `GET /v1/address/{account_id}` for payout address lookup.
- `POST /v1/sign/receipt` to sign `{job_id, miner_id, result_hash, ts}`.
- Optional webhook on `signed` to chain/queue the payout request.
**Miner needs:**
- Local message signing for **proof-of-work-done** receipts.
- Optional “ephemeral sub-accounts” derived at `change=1` for per-job audit.
---
## 13) Migration path to real chain
1. Introduce **ChainAdapter** interface:
- `get_balance(address)`, `broadcast_tx(signed_tx)`, `fetch_utxos|nonce`, `estimate_fee`.
2. Implement `MockAdapter` (current), then `AitbcNodeAdapter` (RPC to real node).
3. Swap via `WALLET_CHAIN=mock|aitbc`.
---
## 14) Windsurf tasks checklist
1. Create repo structure (see **Directory layout**).
2. Add dependencies: `fastapi`, `uvicorn`, `pydantic`, `argon2-cffi`, `pynacl` (or `libsodium` bindings), `bech32`, `sqlalchemy` (for mock ledger), `aiosqlite`.
3. Implement **keystore** (encrypt/decrypt, file IO, metadata).
4. Implement **HD derivation** (SLIP-10 for ed25519, BIP-32 for secp256k1).
5. Implement **address** helper (hash → bech32 with `ait`).
6. Implement **REST** routes, then JSON-RPC shim.
7. Implement **mock ledger** with sqlite + simple transfers.
8. Add **webhook** delivery (async task + retry).
9. Add **rate limits** on signing; add unlock TTL.
10. Write unit tests for keystore, derivation, signing.
11. Provide `run.sh` + systemd unit.
12. Add curl examples to `README`.
---
## 15) Open questions (defaults proposed)
1. Keep both curves or start **ed25519-only**? → **Start ed25519-only**, gate secp256k1 by flag.
2. HRP `ait` and hash `blake2b-20` acceptable as interim? → **Yes (interim)**.
3. Store **per-account tags** (e.g., “payout”, “ops”)? → **Yes**, simple string map.
---
## 16) Threat model notes (MVP)
- Local compromise = keys at risk ⇒ encourage **passphrase on mnemonic** + **short unlock TTL**.
- Add **IPC allowlist** if binding to non-localhost.
- Consider **YubiKey/PKCS#11** module later (signing via hardware).
---
## 17) Logging
- Default: INFO without sensitive fields.
- Redact: passwords, mnemonics, private keys, nonces.
- Correlate by `req_id` header (generate if missing).
---
## 18) License / Compliance
- Keep crypto libs permissive (MIT/ISC/BSD).
- Record algorithm choices & versions in `/about` endpoint for audits.
---
**Proceed to generate the scaffold (FastAPI app + keystore + ed25519 HD + REST) now?** `y` / `n`

185
docs/reference/confidential-implementation-summary.md Normal file
View File

@ -0,0 +1,185 @@
# Confidential Transactions Implementation Summary
## Overview
Successfully implemented a comprehensive confidential transaction system for AITBC with opt-in encryption, selective disclosure, and full audit compliance. The implementation provides privacy for sensitive transaction data while maintaining regulatory compliance.
## Completed Components
### 1. Encryption Service ✅
- **Hybrid Encryption**: AES-256-GCM for data encryption, X25519 for key exchange
- **Envelope Pattern**: Random DEK per transaction, encrypted for each participant
- **Audit Escrow**: Separate encryption key for regulatory access
- **Performance**: Efficient batch operations, key caching
### 2. Key Management ✅
- **Per-Participant Keys**: X25519 key pairs for each participant
- **Key Rotation**: Automated rotation with re-encryption of active data
- **Secure Storage**: File-based storage (development), HSM-ready interface
- **Access Control**: Role-based permissions for key operations
### 3. Access Control ✅
- **Role-Based Policies**: Client, Miner, Coordinator, Auditor, Regulator roles
- **Time Restrictions**: Business hours, retention periods
- **Purpose-Based Access**: Settlement, Audit, Compliance, Dispute, Support
- **Dynamic Policies**: Custom policy creation and management
### 4. Audit Logging ✅
- **Tamper-Evident**: Chain of hashes for integrity verification
- **Comprehensive**: All access, key operations, policy changes
- **Export Capabilities**: JSON, CSV formats for regulators
- **Retention**: Configurable retention periods by role
### 5. API Endpoints ✅
- **/confidential/transactions**: Create and manage confidential transactions
- **/confidential/access**: Request access to encrypted data
- **/confidential/audit**: Regulatory access with authorization
- **/confidential/keys**: Key registration and rotation
- **Rate Limiting**: Protection against abuse
### 6. Data Models ✅
- **ConfidentialTransaction**: Opt-in privacy flags
- **Access Control Models**: Requests, responses, logs
- **Key Management Models**: Registration, rotation, audit
## Security Features
### Encryption
- AES-256-GCM provides confidentiality + integrity
- X25519 ECDH for secure key exchange
- Per-transaction DEKs for forward secrecy
- Random IVs per encryption
### Access Control
- Multi-factor authentication ready
- Time-bound access permissions
- Business hour restrictions for auditors
- Retention period enforcement
### Audit Compliance
- GDPR right to encryption
- SEC Rule 17a-4 compliance
- Immutable audit trails
- Regulatory access with court orders
## Current Limitations
### 1. Database Persistence ❌
- Current implementation uses mock storage
- Needs SQLModel/SQLAlchemy integration
- Transaction storage and querying
- Encrypted data BLOB handling
### 2. Private Key Security ❌
- File storage writes keys unencrypted
- Needs HSM or KMS integration
- Key escrow for recovery
- Hardware security module support
### 3. Async Issues ❌
- AuditLogger uses threading in async context
- Needs asyncio task conversion
- Background writer refactoring
- Proper async/await patterns
### 4. Rate Limiting ⚠️
- slowapi not properly integrated
- Needs FastAPI app state setup
- Distributed rate limiting for production
- Redis backend for scalability
## Production Readiness Checklist
### Critical (Must Fix)
- [ ] Database persistence layer
- [ ] HSM/KMS integration for private keys
- [ ] Fix async issues in audit logging
- [ ] Proper rate limiting setup
### Important (Should Fix)
- [ ] Performance optimization for high volume
- [ ] Distributed key management
- [ ] Backup and recovery procedures
- [ ] Monitoring and alerting
### Nice to Have (Future)
- [ ] Multi-party computation
- [ ] Zero-knowledge proofs integration
- [ ] Advanced privacy features
- [ ] Cross-chain confidential settlements
## Testing Coverage
### Unit Tests ✅
- Encryption/decryption correctness
- Key management operations
- Access control logic
- Audit logging functionality
### Integration Tests ✅
- End-to-end transaction flow
- Cross-service integration
- API endpoint testing
- Error handling scenarios
### Performance Tests ⚠️
- Basic benchmarks included
- Needs load testing
- Scalability assessment
- Resource usage profiling
## Migration Strategy
### Phase 1: Infrastructure (Week 1-2)
1. Implement database persistence
2. Integrate HSM for key storage
3. Fix async issues
4. Set up proper rate limiting
### Phase 2: Security Hardening (Week 3-4)
1. Security audit and penetration testing
2. Implement additional monitoring
3. Create backup procedures
4. Document security controls
### Phase 3: Production Rollout (Month 2)
1. Gradual rollout with feature flags
2. Performance monitoring
3. User training and documentation
4. Compliance validation
## Compliance Status
### GDPR ✅
- Right to encryption implemented
- Data minimization by design
- Privacy by default
### Financial Regulations ✅
- SEC Rule 17a-4 audit logs
- MiFID II transaction reporting
- AML/KYC integration points
### Industry Standards ✅
- ISO 27001 alignment
- NIST Cybersecurity Framework
- PCI DSS considerations
## Next Steps
1. **Immediate**: Fix database persistence and HSM integration
2. **Short-term**: Complete security hardening and testing
3. **Long-term**: Production deployment and monitoring
## Documentation
- [Architecture Design](confidential-transactions.md)
- [API Documentation](../docs/api/coordinator/endpoints.md)
- [Security Guide](security-guidelines.md)
- [Compliance Matrix](compliance-matrix.md)
## Conclusion
The confidential transaction system provides a solid foundation for privacy-preserving transactions in AITBC. While the core functionality is complete and tested, several production readiness items need to be addressed before deployment.
The modular design allows for incremental improvements and ensures the system can evolve with changing requirements and regulations.

354
docs/reference/confidential-transactions.md Normal file
View File

@ -0,0 +1,354 @@
# Confidential Transactions Architecture
## Overview
Design for opt-in confidential transaction support in AITBC, enabling participants to encrypt sensitive transaction data while maintaining selective disclosure and audit capabilities.
## Architecture
### Encryption Model
**Hybrid Encryption with Envelope Pattern**:
1. **Data Encryption**: AES-256-GCM for transaction data
2. **Key Exchange**: X25519 ECDH for per-recipient key distribution
3. **Envelope Pattern**: Random DEK per transaction, encrypted for each authorized party
### Key Components
```
┌─────────────────┐ ┌──────────────────┐ ┌─────────────────┐
│ Transaction │───▶│ Encryption │───▶│ Storage │
│ Service │ │ Service │ │ Layer │
└─────────────────┘ └──────────────────┘ └─────────────────┘
│ │ │
▼ ▼ ▼
┌─────────────────┐ ┌──────────────────┐ ┌─────────────────┐
│ Key Manager │ │ Access Control │ │ Audit Log │
└─────────────────┘ └──────────────────┘ └─────────────────┘
```
## Data Flow
### 1. Transaction Creation (Opt-in)
```python
# Client requests confidential transaction
transaction = {
"job_id": "job-123",
"amount": "1000",
"confidential": True,
"participants": ["client-456", "miner-789", "auditor-001"]
}
# Coordinator encrypts sensitive fields
encrypted = encryption_service.encrypt(
data={"amount": "1000", "pricing": "details"},
participants=transaction["participants"]
)
# Store with encrypted payload
stored_transaction = {
"job_id": "job-123",
"public_data": {"job_id": "job-123"},
"encrypted_data": encrypted.ciphertext,
"encrypted_keys": encrypted.encrypted_keys,
"confidential": True
}
```
### 2. Data Access (Authorized Party)
```python
# Miner requests access to transaction data
access_request = {
"transaction_id": "tx-456",
"requester": "miner-789",
"purpose": "settlement"
}
# Verify access rights
if access_control.verify(access_request):
# Decrypt using recipient's private key
decrypted = encryption_service.decrypt(
ciphertext=stored_transaction.encrypted_data,
encrypted_key=stored_transaction.encrypted_keys["miner-789"],
private_key=miner_private_key
)
```
### 3. Audit Access (Regulatory)
```python
# Auditor with court order requests access
audit_request = {
"transaction_id": "tx-456",
"requester": "auditor-001",
"authorization": "court-order-123"
}
# Special audit key escrow
audit_key = key_manager.get_audit_key(audit_request.authorization)
decrypted = encryption_service.audit_decrypt(
ciphertext=stored_transaction.encrypted_data,
audit_key=audit_key
)
```
## Implementation Details
### Encryption Service
```python
class ConfidentialTransactionService:
"""Service for handling confidential transactions"""
def __init__(self, key_manager: KeyManager):
self.key_manager = key_manager
self.cipher = AES256GCM()
def encrypt(self, data: Dict, participants: List[str]) -> EncryptedData:
"""Encrypt data for multiple participants"""
# Generate random DEK
dek = os.urandom(32)
# Encrypt data with DEK
ciphertext = self.cipher.encrypt(dek, json.dumps(data))
# Encrypt DEK for each participant
encrypted_keys = {}
for participant in participants:
public_key = self.key_manager.get_public_key(participant)
encrypted_keys[participant] = self._encrypt_dek(dek, public_key)
# Add audit escrow
audit_public_key = self.key_manager.get_audit_key()
encrypted_keys["audit"] = self._encrypt_dek(dek, audit_public_key)
return EncryptedData(
ciphertext=ciphertext,
encrypted_keys=encrypted_keys,
algorithm="AES-256-GCM+X25519"
)
def decrypt(self, ciphertext: bytes, encrypted_key: bytes,
private_key: bytes) -> Dict:
"""Decrypt data for specific participant"""
# Decrypt DEK
dek = self._decrypt_dek(encrypted_key, private_key)
# Decrypt data
plaintext = self.cipher.decrypt(dek, ciphertext)
return json.loads(plaintext)
```
### Key Management
```python
class KeyManager:
"""Manages encryption keys for participants"""
def __init__(self, storage: KeyStorage):
self.storage = storage
self.key_pairs = {}
def generate_key_pair(self, participant_id: str) -> KeyPair:
"""Generate X25519 key pair for participant"""
private_key = X25519.generate_private_key()
public_key = private_key.public_key()
key_pair = KeyPair(
participant_id=participant_id,
private_key=private_key,
public_key=public_key
)
self.storage.store(key_pair)
return key_pair
def rotate_keys(self, participant_id: str):
"""Rotate encryption keys"""
# Generate new key pair
new_key_pair = self.generate_key_pair(participant_id)
# Re-encrypt active transactions
self._reencrypt_transactions(participant_id, new_key_pair)
```
### Access Control
```python
class AccessController:
"""Controls access to confidential transaction data"""
def __init__(self, policy_store: PolicyStore):
self.policy_store = policy_store
def verify_access(self, request: AccessRequest) -> bool:
"""Verify if requester has access rights"""
# Check participant status
if not self._is_authorized_participant(request.requester):
return False
# Check purpose-based access
if not self._check_purpose(request.purpose, request.requester):
return False
# Check time-based restrictions
if not self._check_time_restrictions(request):
return False
return True
def _is_authorized_participant(self, participant_id: str) -> bool:
"""Check if participant is authorized for confidential transactions"""
# Verify KYC/KYB status
# Check compliance flags
# Validate regulatory approval
return True
```
## Data Models
### Confidential Transaction
```python
class ConfidentialTransaction(BaseModel):
"""Transaction with optional confidential fields"""
# Public fields (always visible)
transaction_id: str
job_id: str
timestamp: datetime
status: str
# Confidential fields (encrypted when opt-in)
amount: Optional[str] = None
pricing: Optional[Dict] = None
settlement_details: Optional[Dict] = None
# Encryption metadata
confidential: bool = False
encrypted_data: Optional[bytes] = None
encrypted_keys: Optional[Dict[str, bytes]] = None
algorithm: Optional[str] = None
# Access control
participants: List[str] = []
access_policies: Dict[str, Any] = {}
```
### Access Log
```python
class ConfidentialAccessLog(BaseModel):
"""Audit log for confidential data access"""
transaction_id: str
requester: str
purpose: str
timestamp: datetime
authorized_by: str
data_accessed: List[str]
ip_address: str
user_agent: str
```
## Security Considerations
### 1. Key Security
- Private keys stored in HSM or secure enclave
- Key rotation every 90 days
- Zero-knowledge proof of key possession
### 2. Data Protection
- AES-256-GCM provides confidentiality + integrity
- Random IV per encryption
- Forward secrecy with per-transaction DEKs
### 3. Access Control
- Multi-factor authentication for decryption
- Role-based access control
- Time-bound access permissions
### 4. Audit Compliance
- Immutable audit logs
- Regulatory access with court orders
- Privacy-preserving audit proofs
## Performance Optimization
### 1. Lazy Encryption
- Only encrypt fields marked as confidential
- Cache encrypted data for frequent access
- Batch encryption for bulk operations
### 2. Key Management
- Pre-compute shared secrets for regular participants
- Use key derivation for multiple access levels
- Implement key caching with secure eviction
### 3. Storage Optimization
- Compress encrypted data
- Deduplicate common encrypted patterns
- Use column-level encryption for databases
## Migration Strategy
### Phase 1: Opt-in Support
- Add confidential flags to existing models
- Deploy encryption service
- Update transaction endpoints
### Phase 2: Participant Onboarding
- Generate key pairs for all participants
- Implement key distribution
- Train users on privacy features
### Phase 3: Full Rollout
- Enable confidential transactions by default for sensitive data
- Implement advanced access controls
- Add privacy analytics and reporting
## Testing Strategy
### 1. Unit Tests
- Encryption/decryption correctness
- Key management operations
- Access control logic
### 2. Integration Tests
- End-to-end confidential transaction flow
- Cross-system key exchange
- Audit trail verification
### 3. Security Tests
- Penetration testing
- Cryptographic validation
- Side-channel resistance
## Compliance
### 1. GDPR
- Right to encryption
- Data minimization
- Privacy by design
### 2. Financial Regulations
- SEC Rule 17a-4
- MiFID II transaction reporting
- AML/KYC requirements
### 3. Industry Standards
- ISO 27001
- NIST Cybersecurity Framework
- PCI DSS for payment data
## Next Steps
1. Implement core encryption service
2. Create key management infrastructure
3. Update transaction models and APIs
4. Deploy access control system
5. Implement audit logging
6. Conduct security testing
7. Gradual rollout with monitoring

192
docs/reference/docs-gaps.md Normal file
View File

@ -0,0 +1,192 @@
# AITBC Documentation Gaps Report
This document identifies missing documentation for completed features based on the `done.md` file and current documentation state.
## Critical Missing Documentation
### 1. Zero-Knowledge Proof Receipt Attestation
**Status**: ✅ Completed (Implementation in Stage 7)
**Missing Documentation**:
- [ ] User guide: How to use ZK proofs for receipt attestation
- [ ] Developer guide: Integrating ZK proofs into applications
- [ ] Operator guide: Setting up ZK proof generation service
- [ ] API reference: ZK proof endpoints and parameters
- [ ] Tutorial: End-to-end ZK proof workflow
**Priority**: High - Complex feature requiring user education
### 2. Confidential Transactions
**Status**: ✅ Completed (Implementation in Stage 7)
**Existing**: Technical implementation docs
**Missing Documentation**:
- [ ] User guide: How to create confidential transactions
- [ ] Developer guide: Building privacy-preserving applications
- [ ] Migration guide: Moving from regular to confidential transactions
- [ ] Security considerations: Best practices for confidential transactions
**Priority**: High - Security-sensitive feature
### 3. HSM Key Management
**Status**: ✅ Completed (Implementation in Stage 7)
**Missing Documentation**:
- [ ] Operator guide: HSM setup and configuration
- [ ] Integration guide: Azure Key Vault integration
- [ ] Integration guide: AWS KMS integration
- [ ] Security guide: HSM best practices
- [ ] Troubleshooting: Common HSM issues
**Priority**: High - Enterprise feature
### 4. Multi-tenant Coordinator Infrastructure
**Status**: ✅ Completed (Implementation in Stage 7)
**Missing Documentation**:
- [ ] Architecture guide: Multi-tenant architecture overview
- [ ] Operator guide: Setting up multi-tenant infrastructure
- [ ] Tenant management: Creating and managing tenants
- [ ] Billing guide: Understanding billing and quotas
- [ ] Migration guide: Moving to multi-tenant setup
**Priority**: High - Major architectural change
### 5. Enterprise Connectors (Python SDK)
**Status**: ✅ Completed (Implementation in Stage 7)
**Existing**: Technical implementation
**Missing Documentation**:
- [ ] Quick start: Getting started with enterprise connectors
- [ ] Connector guide: Stripe connector usage
- [ ] Connector guide: ERP connector usage
- [ ] Development guide: Building custom connectors
- [ ] Reference: Complete API documentation
**Priority**: Medium - Developer-facing feature
### 6. Ecosystem Certification Program
**Status**: ✅ Completed (Implementation in Stage 7)
**Existing**: Program documentation
**Missing Documentation**:
- [ ] Participant guide: How to get certified
- [ ] Self-service portal: Using the certification portal
- [ ] Badge guide: Displaying certification badges
- [ ] Maintenance guide: Maintaining certification status
**Priority**: Medium - Program adoption
## Moderate Priority Gaps
### 7. Cross-Chain Settlement
**Status**: ✅ Completed (Implementation in Stage 6)
**Existing**: Design documentation
**Missing Documentation**:
- [ ] Integration guide: Setting up cross-chain bridges
- [ ] Tutorial: Cross-chain transaction walkthrough
- [ ] Reference: Bridge API documentation
### 8. GPU Service Registry (30+ Services)
**Status**: ✅ Completed (Implementation in Stage 7)
**Missing Documentation**:
- [ ] Provider guide: Registering GPU services
- [ ] Service catalog: Available service types
- [ ] Pricing guide: Setting service prices
- [ ] Integration guide: Using GPU services
### 9. Advanced Cryptography Features
**Status**: ✅ Completed (Implementation in Stage 7)
**Missing Documentation**:
- [ ] Hybrid encryption guide: Using AES-256-GCM + X25519
- [ ] Role-based access control: Setting up RBAC
- [ ] Audit logging: Configuring tamper-evident logging
## Low Priority Gaps
### 10. Community & Governance
**Status**: ✅ Completed (Implementation in Stage 7)
**Existing**: Framework documentation
**Missing Documentation**:
- [ ] Governance website: User guide for governance site
- [ ] RFC templates: Detailed RFC writing guide
- [ ] Community metrics: Understanding KPIs
### 11. Ecosystem Growth Initiatives
**Status**: ✅ Completed (Implementation in Stage 7)
**Existing**: Program documentation
**Missing Documentation**:
- [ ] Hackathon platform: Using the submission platform
- [ ] Grant tracking: Monitoring grant progress
- [ ] Extension marketplace: Publishing extensions
## Documentation Structure Improvements
### Missing Sections
1. **Migration Guides** - No migration documentation for major changes
2. **Troubleshooting** - Limited troubleshooting guides
3. **Best Practices** - Few best practice documents
4. **Performance Guides** - No performance optimization guides
5. **Security Guides** - Limited security documentation beyond threat modeling
### Outdated Documentation
1. **API References** - May not reflect latest endpoints
2. **Installation Guides** - May not include all components
3. **Configuration** - Missing new configuration options
## Recommended Actions
### Immediate (Next Sprint)
1. Create ZK proof user guide and developer tutorial
2. Document HSM integration for Azure Key Vault and AWS KMS
3. Write multi-tenant setup guide for operators
4. Create confidential transaction quick start
### Short Term (Next Month)
1. Complete enterprise connector documentation
2. Add cross-chain settlement integration guides
3. Document GPU service provider workflow
4. Create migration guides for major features
### Medium Term (Next Quarter)
1. Expand troubleshooting section
2. Add performance optimization guides
3. Create security best practices documentation
4. Build interactive tutorials for complex features
### Long Term (Next 6 Months)
1. Create video tutorials for key workflows
2. Build interactive API documentation
3. Add regional deployment guides
4. Create compliance documentation for regulated markets
## Documentation Metrics
### Current State
- Total markdown files: 65+
- Organized into: 5 main categories
- Missing critical docs: 11 major features
- Coverage estimate: 60% of completed features documented
### Target State
- Critical features: 100% documented
- User guides: All major features
- Developer resources: Complete API coverage
- Operator guides: All deployment scenarios
## Resources Needed
### Writers
- Technical writer: 1 FTE for 3 months
- Developer advocates: 2 FTE for tutorials
- Security specialist: For security documentation
### Tools
- Documentation platform: GitBook or Docusaurus
- API documentation: Swagger/OpenAPI tools
- Interactive tutorials: CodeSandbox or similar
### Process
- Documentation review workflow
- Translation process for internationalization
- Community contribution process for docs
---
**Last Updated**: 2024-01-15
**Next Review**: 2024-02-15
**Owner**: Documentation Team

205
docs/reference/done.md Normal file
View File

@ -0,0 +1,205 @@
# Completed Bootstrap Tasks
## Repository Initialization
- Scaffolded core monorepo directories reflected in `docs/bootstrap/dirs.md`.
- Added top-level config files: `.editorconfig`, `.gitignore`, `LICENSE`, and root `README.md`.
- Created Windsurf workspace metadata under `windsurf/`.
## Documentation
- Authored `docs/roadmap.md` capturing staged development targets.
- Added README placeholders for primary apps under `apps/` to outline purpose and setup notes.
## Coordinator API
- Implemented SQLModel-backed job persistence and service layer in `apps/coordinator-api/src/app/`.
- Wired client, miner, and admin routers to coordinator services (job lifecycle, scheduling, stats).
- Added initial pytest coverage under `apps/coordinator-api/tests/test_jobs.py`.
- Added signed receipt generation, persistence (`Job.receipt`, `JobReceipt` history table), retrieval endpoints, telemetry metrics, and optional coordinator attestations.
- Persisted historical receipts via `JobReceipt`; exposed `/v1/jobs/{job_id}/receipts` endpoint and integrated canonical serialization.
- Documented receipt attestation configuration (`RECEIPT_ATTESTATION_KEY_HEX`) in `docs/run.md` and coordinator README.
## Miner Node
- Created coordinator client, control loop, and capability/backoff utilities in `apps/miner-node/src/aitbc_miner/`.
- Implemented CLI/Python runners and execution pipeline with result reporting.
- Added starter tests for runners in `apps/miner-node/tests/test_runners.py`.
## Blockchain Node
- Added websocket fan-out, disconnect cleanup, and load-test coverage in `apps/blockchain-node/tests/test_websocket.py`, ensuring gossip topics deliver reliably to multiple subscribers.
## Directory Preparation
- Established scaffolds for Python and JavaScript packages in `packages/py/` and `packages/js/`.
- Seeded example project directories under `examples/` for quickstart clients and receipt verification.
- Added `examples/receipts-sign-verify/fetch_and_verify.py` demonstrating coordinator receipt fetching + verification using Python SDK.
## Python SDK
- Created `packages/py/aitbc-sdk/` with coordinator receipt client and verification helpers consuming `aitbc_crypto` utilities.
- Added pytest coverage under `packages/py/aitbc-sdk/tests/test_receipts.py` validating miner/coordinator signature checks and client behavior.
## Wallet Daemon
- Added `apps/wallet-daemon/src/app/receipts/service.py` providing `ReceiptVerifierService` that fetches and validates receipts via `aitbc_sdk`.
- Created unit tests under `apps/wallet-daemon/tests/test_receipts.py` verifying service behavior.
- Implemented wallet SDK receipt ingestion + attestation surfacing in `packages/py/aitbc-sdk/src/receipts.py`, including pagination client, signature verification, and failure diagnostics with full pytest coverage.
- Hardened REST API by wiring dependency overrides in `apps/wallet-daemon/tests/test_wallet_api.py`, expanding workflow coverage (create/list/unlock/sign) and enforcing structured password policy errors consumed in CI.
## Explorer Web
- Initialized a Vite + TypeScript scaffold in `apps/explorer-web/` with `vite.config.ts`, `tsconfig.json`, and placeholder `src/main.ts` content.
- Installed frontend dependencies locally to unblock editor tooling and TypeScript type resolution.
- Implemented `overview` page stats rendering backed by mock block/transaction/receipt fetchers, including robust empty-state handling and TypeScript type fixes.
## Pool Hub
- Implemented FastAPI service scaffolding with Redis/PostgreSQL-backed repositories, match/health/metrics endpoints, and Prometheus instrumentation (`apps/pool-hub/src/poolhub/`).
- Added Alembic migrations (`apps/pool-hub/migrations/`) and async integration tests covering repositories and endpoints (`apps/pool-hub/tests/`).
## Solidity Token
- Implemented attested minting logic in `packages/solidity/aitbc-token/contracts/AIToken.sol` using `AccessControl` role gates and ECDSA signature recovery.
- Added Hardhat unit tests in `packages/solidity/aitbc-token/test/aitoken.test.ts` covering successful minting, replay prevention, and invalid attestor signatures.
- Configured project TypeScript settings via `packages/solidity/aitbc-token/tsconfig.json` to align Hardhat, Node, and Mocha typings for the contract test suite.
## JavaScript SDK
- Delivered fetch-based client wrapper with TypeScript definitions and Vitest coverage under `packages/js/aitbc-sdk/`.
## Blockchain Node Enhancements
- Added comprehensive WebSocket tests for blocks and transactions streams including multi-subscriber and high-volume scenarios.
- Extended PoA consensus with per-proposer block metrics and rotation tracking.
- Added latest block interval gauge and RPC error spike alerting.
- Enhanced observability with Grafana dashboards for blockchain node and coordinator overview.
- Implemented marketplace endpoints in coordinator API with explorer and marketplace routers.
- Added mock coordinator integration with enhanced telemetry capabilities.
- Created comprehensive observability documentation and alerting rules.
## Explorer Web Production Readiness
- Implemented Playwright end-to-end tests for live mode functionality.
- Enhanced responsive design with improved CSS layout system.
- Added comprehensive error handling and fallback mechanisms for live API responses.
- Integrated live coordinator endpoints with proper data reconciliation.
## Marketplace Web Launch
- Completed auth/session scaffolding for marketplace actions.
- Implemented API abstraction layer with mock/live mode toggle.
- Connected mock listings and bids to coordinator data sources.
- Added feature flags for controlled live mode rollout.
## Cross-Chain Settlement
- Implemented cross-chain settlement hooks with external bridges.
- Created BridgeAdapter interface for LayerZero integration.
- Implemented BridgeManager for orchestration and retry logic.
- Added settlement storage and API endpoints.
- Created cross-chain settlement documentation.
## Python SDK Transport Abstraction
- Designed pluggable transport abstraction layer for multi-network support.
- Implemented base Transport interface with HTTP/WebSocket transports.
- Created MultiNetworkClient for managing multiple blockchain networks.
- Updated AITBCClient to use transport abstraction with backward compatibility.
- Added transport documentation and examples.
## GPU Service Provider Configuration
- Extended Miner model to include service configurations.
- Created service configuration API endpoints in pool-hub.
- Built HTML/JS UI for service provider configuration.
- Added service pricing configuration and capability validation.
- Implemented service selection for GPU providers.
## GPU Service Expansion
- Implemented dynamic service registry framework for 30+ GPU services.
- Created service definitions for 6 categories: AI/ML, Media Processing, Scientific Computing, Data Analytics, Gaming, Development Tools.
- Built comprehensive service registry API with validation and discovery.
- Added hardware requirement checking and pricing models.
- Updated roadmap with service expansion phase documentation.
## Stage 7 - GPU Service Expansion & Privacy Features
### GPU Service Infrastructure
- Create dynamic service registry with JSON schema validation
- Implement service provider configuration UI with dynamic service selection
- Create service definitions for AI/ML (LLM inference, image/video generation, speech recognition, computer vision, recommendation systems)
- Create service definitions for Media Processing (video transcoding, streaming, 3D rendering, image/audio processing)
- Create service definitions for Scientific Computing (molecular dynamics, weather modeling, financial modeling, physics simulation, bioinformatics)
- Create service definitions for Data Analytics (big data processing, real-time analytics, graph analytics, time series analysis)
- Create service definitions for Gaming & Entertainment (cloud gaming, asset baking, physics simulation, VR/AR rendering)
- Create service definitions for Development Tools (GPU compilation, model training, data processing, simulation testing, code generation)
- Implement service-specific validation and hardware requirement checking
### Privacy & Cryptography Features
- ✅ Research zk-proof-based receipt attestation and prototype a privacy-preserving settlement flow
- ✅ Implement Groth16 ZK circuit for receipt hash preimage proofs
- ✅ Create ZK proof generation service in coordinator API
- ✅ Implement on-chain verification contract (ZKReceiptVerifier.sol)
- ✅ Add confidential transaction support with opt-in ciphertext storage
- ✅ Implement HSM-backed key management (Azure Key Vault, AWS KMS, Software)
- ✅ Create hybrid encryption system (AES-256-GCM + X25519)
- ✅ Implement role-based access control with time restrictions
- ✅ Create tamper-evident audit logging with chain of hashes
- ✅ Publish comprehensive threat modeling with STRIDE analysis
- ✅ Update cross-chain settlement hooks for ZK proofs and privacy levels
### Enterprise Integration Features
- ✅ Deliver reference connectors for ERP/payment systems with Python SDK
- ✅ Implement Stripe payment connector with full charge/refund/subscription support
- ✅ Create enterprise-grade Python SDK with async support, dependency injection, metrics
- ✅ Build ERP connector base classes with plugin architecture for protocols
- ✅ Document comprehensive SLAs with uptime guarantees and support commitments
- ✅ Stand up multi-tenant coordinator infrastructure with per-tenant isolation
- ✅ Implement tenant management service with lifecycle operations
- ✅ Create tenant context middleware for automatic tenant identification
- ✅ Build resource quota enforcement with Redis-backed caching
- ✅ Create usage tracking and billing metrics with tiered pricing
- ✅ Launch ecosystem certification program with SDK conformance testing
- ✅ Define Bronze/Silver/Gold certification tiers with clear requirements
- ✅ Build language-agnostic test suite with OpenAPI contract validation
- ✅ Implement security validation framework with dependency scanning
- ✅ Design public registry API for partner/SDK discovery
- ✅ Validate certification system with Stripe connector certification
### Community & Governance Features
- ✅ Establish open RFC process with clear stages and review criteria
- ✅ Create governance website with documentation and navigation
- ✅ Set up community call schedule with multiple call types
- ✅ Design RFC template and GitHub PR template for submissions
- ✅ Implement benevolent dictator model with sunset clause
- ✅ Create hybrid governance structure (GitHub + Discord + Website)
- ✅ Document participation guidelines and code of conduct
- ✅ Establish transparency and accountability processes
### Ecosystem Growth Initiatives
- ✅ Create hackathon organization framework with quarterly themes and bounty board
- ✅ Design grant program with hybrid approach (micro-grants + strategic grants)
- ✅ Build marketplace extension SDK with cookiecutter templates
- ✅ Create analytics tooling for ecosystem metrics and KPI tracking
- ✅ Track ecosystem KPIs (active marketplaces, cross-chain volume) and feed them into quarterly strategy reviews
- ✅ Establish judging criteria with ecosystem impact weighting
- ✅ Create sponsor partnership framework with tiered benefits
- ✅ Design retroactive grants for proven projects
- ✅ Implement milestone-based disbursement for accountability
### Stage 8 - Frontier R&D & Global Expansion
- ✅ Launch research consortium framework with governance model and membership tiers
- ✅ Develop hybrid PoA/PoS consensus research plan with 12-month implementation timeline
- ✅ Create scaling research plan for sharding and rollups (100K+ TPS target)
- ✅ Design ZK applications research plan for privacy-preserving AI
- ✅ Create governance research plan with liquid democracy and AI assistance
- ✅ Develop economic models research plan with sustainable tokenomics
- ✅ Implement hybrid consensus prototype demonstrating dynamic mode switching
- ✅ Create executive summary for consortium recruitment
- ✅ Prototype sharding architecture with beacon chain coordination
- ✅ Implement ZK-rollup prototype for transaction batching
- ⏳ Set up consortium legal structure and operational infrastructure
- ⏳ Recruit founding members from industry and academia

230
docs/reference/enterprise-sla.md Normal file
View File

@ -0,0 +1,230 @@
# AITBC Enterprise Integration SLA
## Overview
This document outlines the Service Level Agreement (SLA) for enterprise integrations with the AITBC network, including uptime guarantees, performance expectations, and support commitments.
## Document Version
- Version: 1.0
- Date: December 2024
- Effective Date: January 1, 2025
## Service Availability
### Coordinator API
- **Uptime Guarantee**: 99.9% monthly (excluding scheduled maintenance)
- **Scheduled Maintenance**: Maximum 4 hours per month, announced 72 hours in advance
- **Emergency Maintenance**: Maximum 2 hours per month, announced 2 hours in advance
### Mining Pool Network
- **Network Uptime**: 99.5% monthly
- **Minimum Active Miners**: 1000 miners globally distributed
- **Geographic Distribution**: Minimum 3 continents, 5 countries
### Settlement Layer
- **Confirmation Time**: 95% of transactions confirmed within 30 seconds
- **Cross-Chain Bridge**: 99% availability for supported chains
- **Finality**: 99.9% of transactions final after 2 confirmations
## Performance Metrics
### API Response Times
| Endpoint | 50th Percentile | 95th Percentile | 99th Percentile |
|----------|-----------------|-----------------|-----------------|
| Job Submission | 50ms | 100ms | 200ms |
| Job Status | 25ms | 50ms | 100ms |
| Receipt Verification | 100ms | 200ms | 500ms |
| Settlement Initiation | 150ms | 300ms | 1000ms |
### Throughput Limits
| Service | Rate Limit | Burst Limit |
|---------|------------|------------|
| Job Submission | 1000/minute | 100/minute |
| API Calls | 10,000/minute | 1000/minute |
| Webhook Events | 5000/minute | 500/minute |
### Data Processing
- **Proof Generation**: Average 2 seconds, 95% under 5 seconds
- **ZK Verification**: Average 100ms, 95% under 200ms
- **Encryption/Decryption**: Average 50ms, 95% under 100ms
## Support Services
### Support Tiers
| Tier | Response Time | Availability | Escalation |
|------|---------------|--------------|------------|
| Enterprise | 1 hour (P1), 4 hours (P2), 24 hours (P3) | 24x7x365 | Direct to engineering |
| Business | 4 hours (P1), 24 hours (P2), 48 hours (P3) | Business hours | Technical lead |
| Developer | 24 hours (P1), 72 hours (P2), 5 days (P3) | Business hours | Support team |
### Incident Management
- **P1 - Critical**: System down, data loss, security breach
- **P2 - High**: Significant feature degradation, performance impact
- **P3 - Medium**: Feature not working, documentation issues
- **P4 - Low**: General questions, enhancement requests
### Maintenance Windows
- **Regular Maintenance**: Every Sunday 02:00-04:00 UTC
- **Security Updates**: As needed, minimum 24 hours notice
- **Major Upgrades**: Quarterly, minimum 30 days notice
## Data Management
### Data Retention
| Data Type | Retention Period | Archival |
|-----------|------------------|----------|
| Transaction Records | 7 years | Yes |
| Audit Logs | 7 years | Yes |
| Performance Metrics | 2 years | Yes |
| Error Logs | 90 days | No |
| Debug Logs | 30 days | No |
### Data Availability
- **Backup Frequency**: Every 15 minutes
- **Recovery Point Objective (RPO)**: 15 minutes
- **Recovery Time Objective (RTO)**: 4 hours
- **Geographic Redundancy**: 3 regions, cross-replicated
### Privacy and Compliance
- **GDPR Compliant**: Yes
- **Data Processing Agreement**: Available
- **Privacy Impact Assessment**: Completed
- **Certifications**: ISO 27001, SOC 2 Type II
## Integration SLAs
### ERP Connectors
| Metric | Target |
|--------|--------|
| Sync Latency | < 5 minutes |
| Data Accuracy | 99.99% |
| Error Rate | < 0.1% |
| Retry Success Rate | > 99% |
### Payment Processors
| Metric | Target |
|--------|--------|
| Settlement Time | < 2 minutes |
| Success Rate | 99.9% |
| Fraud Detection | < 0.01% false positive |
| Chargeback Handling | 24 hours |
### Webhook Delivery
- **Delivery Guarantee**: 99.5% successful delivery
- **Retry Policy**: Exponential backoff, max 10 attempts
- **Timeout**: 30 seconds per attempt
- **Verification**: HMAC-SHA256 signatures
## Security Commitments
### Availability
- **DDoS Protection**: 99.9% mitigation success
- **Incident Response**: < 1 hour detection, < 4 hours containment
- **Vulnerability Patching**: Critical patches within 24 hours
### Encryption Standards
- **In Transit**: TLS 1.3 minimum
- **At Rest**: AES-256 encryption
- **Key Management**: HSM-backed, regular rotation
- **Compliance**: FIPS 140-2 Level 3
## Penalties and Credits
### Service Credits
| Downtime | Credit Percentage |
|----------|------------------|
| < 99.9% uptime | 10% |
| < 99.5% uptime | 25% |
| < 99.0% uptime | 50% |
| < 98.0% uptime | 100% |
### Performance Credits
| Metric Miss | Credit |
|-------------|--------|
| Response time > 95th percentile | 5% |
| Throughput limit exceeded | 10% |
| Data loss > RPO | 100% |
### Claim Process
1. Submit ticket within 30 days of incident
2. Provide evidence of SLA breach
3. Review within 5 business days
4. Credit applied to next invoice
## Exclusions
### Force Majeure
- Natural disasters
- War, terrorism, civil unrest
- Government actions
- Internet outages beyond control
### Customer Responsibilities
- Proper API implementation
- Adequate error handling
- Rate limit compliance
- Security best practices
### Third-Party Dependencies
- External payment processors
- Cloud provider outages
- Blockchain network congestion
- DNS issues
## Monitoring and Reporting
### Available Metrics
- Real-time dashboard
- Historical reports (24 months)
- API usage analytics
- Performance benchmarks
### Custom Reports
- Monthly SLA reports
- Quarterly business reviews
- Annual security assessments
- Custom KPI tracking
### Alerting
- Email notifications
- SMS for critical issues
- Webhook callbacks
- Slack integration
## Contact Information
### Support
- **Enterprise Support**: enterprise@aitbc.io
- **Technical Support**: support@aitbc.io
- **Security Issues**: security@aitbc.io
- **Emergency Hotline**: +1-555-SECURITY
### Account Management
- **Enterprise Customers**: account@aitbc.io
- **Partners**: partners@aitbc.io
- **Billing**: billing@aitbc.io
## Definitions
### Terms
- **Uptime**: Percentage of time services are available and functional
- **Response Time**: Time from request receipt to first byte of response
- **Throughput**: Number of requests processed per time unit
- **Error Rate**: Percentage of requests resulting in errors
### Calculations
- Monthly uptime calculated as (total minutes - downtime) / total minutes
- Percentiles measured over trailing 30-day period
- Credits calculated on monthly service fees
## Amendments
This SLA may be amended with:
- 30 days written notice for non-material changes
- 90 days written notice for material changes
- Mutual agreement for custom terms
- Immediate notice for security updates
---
*This SLA is part of the Enterprise Integration Agreement and is subject to the terms and conditions therein.*

45
docs/reference/index.md Normal file
View File

@ -0,0 +1,45 @@
# AITBC Reference Documentation
Welcome to the AITBC reference documentation. This section contains technical specifications, architecture details, and historical documentation.
## Architecture & Design
- [Architecture Overview](architecture/) - System architecture documentation
- [Cross-Chain Settlement](architecture/cross-chain-settlement-design.md) - Cross-chain settlement design
- [Python SDK Transport](architecture/python-sdk-transport-design.md) - Transport abstraction design
## Bootstrap Specifications
- [Bootstrap Directory](bootstrap/dirs.md) - Original directory structure
- [Technical Plan](bootstrap/aitbc_tech_plan.md) - Original technical specification
- [Component Specs](bootstrap/) - Individual component specifications
## Cryptography & Privacy
- [ZK Receipt Attestation](zk-receipt-attestation.md) - Zero-knowledge proof implementation
- [ZK Implementation Summary](zk-implementation-summary.md) - ZK implementation overview
- [ZK Technology Comparison](zk-technology-comparison.md) - ZK technology comparison
- [Confidential Transactions](confidential-transactions.md) - Confidential transaction implementation
- [Confidential Implementation Summary](confidential-implementation-summary.md) - Implementation summary
- [Threat Modeling](threat-modeling.md) - Security threat modeling
## Enterprise Features
- [Enterprise SLA](enterprise-sla.md) - Service level agreements
- [Multi-tenancy](multi-tenancy.md) - Multi-tenant infrastructure
- [HSM Integration](hsm-integration.md) - Hardware security module integration
## Project Documentation
- [Roadmap](roadmap.md) - Development roadmap
- [Completed Tasks](done.md) - List of completed features
- [Beta Release Plan](beta-release-plan.md) - Beta release planning
## Historical
- [Component Documentation](../coordinator_api.md) - Historical component docs
- [Bootstrap Archive](bootstrap/) - Original bootstrap documentation
## Glossary
- [Terms](glossary.md) - AITBC terminology and definitions

236
docs/reference/roadmap.md Normal file
View File

@ -0,0 +1,236 @@
# AITBC Development Roadmap
This roadmap aggregates high-priority tasks derived from the bootstrap specifications in `docs/bootstrap/` and tracks progress across the monorepo. Update this document as milestones evolve.
## Stage 1 — Upcoming Focus Areas
- **Blockchain Node Foundations**
- ✅ Bootstrap module layout in `apps/blockchain-node/src/`.
- ✅ Implement SQLModel schemas and RPC stubs aligned with historical/attested receipts.
- **Explorer Web Enablement**
- ✅ Finish mock integration across all pages and polish styling + mock/live toggle.
- ✅ Begin wiring coordinator endpoints (e.g., `/v1/jobs/{job_id}/receipts`).
- **Marketplace Web Scaffolding**
- ✅ Scaffold Vite/vanilla frontends consuming coordinator receipt history endpoints and SDK examples.
- **Pool Hub Services**
- ✅ Initialize FastAPI project, scoring registry, and telemetry ingestion hooks leveraging coordinator/miner metrics.
- **CI Enhancements**
- ✅ Add blockchain-node tests once available and frontend build/lint checks to `.github/workflows/python-tests.yml` or follow-on workflows.
- ✅ Provide systemd unit + installer scripts under `scripts/` for streamlined deployment.
## Stage 2 — Core Services (MVP)
- **Coordinator API**
- ✅ Scaffold FastAPI project (`apps/coordinator-api/src/app/`).
- ✅ Implement job submission, status, result endpoints.
- ✅ Add miner registration, heartbeat, poll, result routes.
- ✅ Wire SQLite persistence for jobs, miners, receipts (historical `JobReceipt` table).
- ✅ Provide `.env.example`, `pyproject.toml`, and run scripts.
- **Miner Node**
- ✅ Implement capability probe and control loop (register → heartbeat → fetch jobs).
- ✅ Build CLI and Python runners with sandboxed work dirs (result reporting stubbed to coordinator).
- **Blockchain Node**
- ✅ Define SQLModel schema for blocks, transactions, accounts, receipts (`apps/blockchain-node/src/aitbc_chain/models.py`).
- ✅ Harden schema parity across runtime + storage:
- Alembic baseline + follow-on migrations in `apps/blockchain-node/migrations/` now track the SQLModel schema (blocks, transactions, receipts, accounts).
- Added `Relationship` + `ForeignKey` wiring in `apps/blockchain-node/src/aitbc_chain/models.py` for block ↔ transaction ↔ receipt joins.
- Introduced hex/enum validation hooks via Pydantic validators to ensure hash integrity and safe persistence.
- ✅ Implement PoA proposer loop with block assembly (`apps/blockchain-node/src/aitbc_chain/consensus/poa.py`).
- ✅ Expose REST RPC endpoints for tx submission, balances, receipts (`apps/blockchain-node/src/aitbc_chain/rpc/router.py`).
- ✅ Deliver WebSocket RPC + P2P gossip layer:
- ✅ Stand up WebSocket subscription endpoints (`apps/blockchain-node/src/aitbc_chain/rpc/websocket.py`) mirroring REST payloads.
- ✅ Implement pub/sub transport for block + transaction gossip backed by an in-memory broker (Starlette `Broadcast` or Redis) with configurable fan-out.
- ✅ Add integration tests and load-test harness ensuring gossip convergence and back-pressure handling.
- ✅ Ship devnet scripts (`apps/blockchain-node/scripts/`).
- ✅ Add observability hooks (JSON logging, Prometheus metrics) and integrate coordinator mock into devnet tooling.
- ✅ Expand observability dashboards + miner mock integration:
- Build Grafana dashboards for consensus health (block intervals, proposer rotation) and RPC latency (`apps/blockchain-node/observability/`).
- Expose miner mock telemetry (job throughput, error rates) via shared Prometheus registry and ingest into blockchain-node dashboards.
- Add alerting rules (Prometheus `Alertmanager`) for stalled proposers, queue saturation, and miner mock disconnects.
- Wire coordinator mock into devnet tooling to simulate real-world load and validate observability hooks.
- **Receipt Schema**
- ✅ Finalize canonical JSON receipt format under `protocols/receipts/` (includes sample signed receipts).
- ✅ Implement signing/verification helpers in `packages/py/aitbc-crypto` (JS SDK pending).
- ✅ Translate `docs/bootstrap/aitbc_tech_plan.md` contract skeleton into Solidity project (`packages/solidity/aitbc-token/`).
- ✅ Add deployment/test scripts and document minting flow (`packages/solidity/aitbc-token/scripts/` and `docs/run.md`).
- **Wallet Daemon**
- ✅ Implement encrypted keystore (Argon2id + XChaCha20-Poly1305) via `KeystoreService`.
- ✅ Provide REST and JSON-RPC endpoints for wallet management and signing (`api_rest.py`, `api_jsonrpc.py`).
- ✅ Add mock ledger adapter with SQLite backend powering event history (`ledger_mock/`).
- ✅ Integrate Python receipt verification helpers (`aitbc_sdk`) and expose API/service utilities validating miner + coordinator signatures.
- ✅ Harden REST API workflows (create/list/unlock/sign) with structured password policy enforcement and deterministic pytest coverage in `apps/wallet-daemon/tests/test_wallet_api.py`.
- ✅ Implement Wallet SDK receipt ingestion + attestation surfacing:
- Added `/v1/jobs/{job_id}/receipts` client helpers with cursor pagination, retry/backoff, and summary reporting (`packages/py/aitbc-sdk/src/receipts.py`).
- Reused crypto helpers to validate miner and coordinator signatures, capturing per-key failure reasons for downstream UX.
- Surfaced aggregated attestation status (`ReceiptStatus`) and failure diagnostics for SDK + UI consumers; JS helper parity still planned.
## Stage 3 — Pool Hub & Marketplace
- **Pool Hub**
- ✅ Implement miner registry, scoring engine, and `/v1/match` API with Redis/PostgreSQL backing stores.
- ✅ Add observability endpoints (`/v1/health`, `/v1/metrics`) plus Prometheus instrumentation and integration tests.
- **Marketplace Web**
- ✅ Initialize Vite project with vanilla TypeScript (`apps/marketplace-web/`).
- ✅ Build offer list, bid form, and stats cards powered by mock data fixtures (`public/mock/`).
- ✅ Provide API abstraction toggling mock/live mode (`src/lib/api.ts`) and wire coordinator endpoints.
- ✅ Validate live mode against coordinator `/v1/marketplace/*` responses and add auth feature flags for rollout.
- **Explorer Web**
- ✅ Initialize Vite + TypeScript project scaffold (`apps/explorer-web/`).
- ✅ Add routed pages for overview, blocks, transactions, addresses, receipts.
- ✅ Seed mock datasets (`public/mock/`) and fetch helpers powering overview + blocks tables.
- ✅ Extend mock integrations to transactions, addresses, and receipts pages.
- ✅ Implement styling system, mock/live data toggle, and coordinator API wiring scaffold.
- ✅ Render overview stats from mock block/transaction/receipt summaries with graceful empty-state fallbacks.
- ✅ Validate live mode + responsive polish:
- Hit live coordinator endpoints (`/v1/blocks`, `/v1/transactions`, `/v1/addresses`, `/v1/receipts`) via `getDataMode() === "live"` and reconcile payloads with UI models.
- Add fallbacks + error surfacing for partial/failed live responses (toast + console diagnostics).
- Audit responsive breakpoints (`public/css/layout.css`) and adjust grid/typography for tablet + mobile; add regression checks in Percy/Playwright snapshots.
## Stage 4 — Observability & Production Polish
- **Observability & Telemetry**
- ✅ Build Grafana dashboards for PoA consensus health (block intervals, proposer rotation cadence) leveraging `poa_last_block_interval_seconds`, `poa_proposer_rotations_total`, and per-proposer counters.
- ✅ Surface RPC latency histograms/summaries for critical endpoints (`rpc_get_head`, `rpc_send_tx`, `rpc_submit_receipt`) and add Grafana panels with SLO thresholds.
- ✅ Ingest miner mock telemetry (job throughput, failure rate) into the shared Prometheus registry and wire panels/alerts that correlate miner health with consensus metrics.
- **Explorer Web (Live Mode)**
- ✅ Finalize live `getDataMode() === "live"` workflow: align API payload contracts, render loading/error states, and persist mock/live toggle preference.
- ✅ Expand responsive testing (tablet/mobile) and add automated visual regression snapshots prior to launch.
- ✅ Integrate Playwright smoke tests covering overview, blocks, and transactions pages in live mode.
- **Marketplace Web (Launch Readiness)**
- ✅ Connect mock listings/bids to coordinator data sources and provide feature flags for live mode rollout.
- ✅ Implement auth/session scaffolding for marketplace actions and document API assumptions in `apps/marketplace-web/README.md`.
- ✅ Add Grafana panels monitoring marketplace API throughput and error rates once endpoints are live.
- **Operational Hardening**
- ✅ Extend Alertmanager rules to cover RPC error spikes, proposer stalls, and miner disconnects using the new metrics.
- ✅ Document dashboard import + alert deployment steps in `docs/run.md` for operators.
- ✅ Prepare Stage 3 release checklist linking dashboards, alerts, and smoke tests prior to production cutover.
## Stage 5 — Scaling & Release Readiness
- **Infrastructure Scaling**
- ✅ Benchmark blockchain node throughput under sustained load; capture CPU/memory targets and suggest horizontal scaling thresholds.
- ✅ Build Terraform/Helm templates for dev/staging/prod environments, including Prometheus/Grafana bundles.
- ✅ Implement autoscaling policies for coordinator, miners, and marketplace services with synthetic traffic tests.
- **Reliability & Compliance**
- ✅ Formalize backup/restore procedures for PostgreSQL, Redis, and ledger storage with scheduled jobs.
- ✅ Complete security hardening review (TLS termination, API auth, secrets management) and document mitigations in `docs/security.md`.
- ✅ Add chaos testing scripts (network partition, coordinator outage) and track mean-time-to-recovery metrics.
- **Product Launch Checklist**
- ✅ Finalize public documentation (API references, onboarding guides) and publish to the docs portal.
- ✅ Coordinate beta release timeline, including user acceptance testing of explorer/marketplace live modes.
- ✅ Establish post-launch monitoring playbooks and on-call rotations.
## Stage 6 — Ecosystem Expansion
- **Cross-Chain & Interop**
- ✅ Prototype cross-chain settlement hooks leveraging external bridges; document integration patterns.
- ✅ Extend SDKs (Python/JS) with pluggable transport abstractions for multi-network support.
- ⏳ Evaluate third-party explorer/analytics integrations and publish partner onboarding guides.
- **Marketplace Growth**
- ⏳ Launch incentive programs (staking, liquidity mining) and expose telemetry dashboards tracking campaign performance.
- ⏳ Implement governance module (proposal voting, parameter changes) and add API/UX flows to explorer/marketplace.
- ⏳ Provide SLA-backed coordinator/pool hubs with capacity planning and billing instrumentation.
- **Developer Experience**
- ⏳ Publish advanced tutorials (custom proposers, marketplace extensions) and maintain versioned API docs.
- ⏳ Integrate CI/CD pipelines with canary deployments and blue/green release automation.
- ⏳ Host quarterly architecture reviews capturing lessons learned and feeding into roadmap revisions.
## Stage 7 — Innovation & Ecosystem Services
- **GPU Service Expansion**
- ✅ Implement dynamic service registry framework for 30+ GPU-accelerated services
- ✅ Create service definitions for AI/ML (LLM inference, image/video generation, speech recognition, computer vision, recommendation systems)
- ✅ Create service definitions for Media Processing (video transcoding, streaming, 3D rendering, image/audio processing)
- ✅ Create service definitions for Scientific Computing (molecular dynamics, weather modeling, financial modeling, physics simulation, bioinformatics)
- ✅ Create service definitions for Data Analytics (big data processing, real-time analytics, graph analytics, time series analysis)
- ✅ Create service definitions for Gaming & Entertainment (cloud gaming, asset baking, physics simulation, VR/AR rendering)
- ✅ Create service definitions for Development Tools (GPU compilation, model training, data processing, simulation testing, code generation)
- ✅ Deploy service provider configuration UI with dynamic service selection
- ✅ Implement service-specific validation and hardware requirement checking
- **Advanced Cryptography & Privacy**
- ✅ Research zk-proof-based receipt attestation and prototype a privacy-preserving settlement flow.
- ✅ Add confidential transaction support with opt-in ciphertext storage and HSM-backed key management.
- ✅ Publish threat modeling updates and share mitigations with ecosystem partners.
- **Enterprise Integrations**
- ✅ Deliver reference connectors for ERP/payment systems and document SLA expectations.
- ✅ Stand up multi-tenant coordinator infrastructure with per-tenant isolation and billing metrics.
- ✅ Launch ecosystem certification program (SDK conformance, security best practices) with public registry.
- **Community & Governance**
- ✅ Establish open RFC process, publish governance website, and schedule regular community calls.
- ✅ Sponsor hackathons/accelerators and provide grants for marketplace extensions and analytics tooling.
- ✅ Track ecosystem KPIs (active marketplaces, cross-chain volume) and feed them into quarterly strategy reviews.
## Stage 8 — Frontier R&D & Global Expansion
- **Protocol Evolution**
- ✅ Launch research consortium exploring next-gen consensus (hybrid PoA/PoS) and finalize whitepapers.
- ⏳ Prototype sharding or rollup architectures to scale throughput beyond current limits.
- ⏳ Standardize interoperability specs with industry bodies and submit proposals for adoption.
- **Global Rollout**
- ⏳ Establish regional infrastructure hubs (multi-cloud) with localized compliance and data residency guarantees.
- ⏳ Partner with regulators/enterprises to pilot regulated marketplaces and publish compliance playbooks.
- ⏳ Expand localization (UI, documentation, support) covering top target markets.
- **Long-Term Sustainability**
- ⏳ Create sustainability fund for ecosystem maintenance, bug bounties, and community stewardship.
- ⏳ Define succession planning for core teams, including training programs and contributor pathways.
- ⏳ Publish bi-annual roadmap retrospectives assessing KPI alignment and revising long-term goals.
## Stage 9 — Moonshot Initiatives
- **Decentralized Infrastructure**
- ⏳ Transition coordinator/miner roles toward community-governed validator sets with incentive alignment.
- ⏳ Explore decentralized storage/backbone options (IPFS/Filecoin) for ledger and marketplace artifacts.
- ⏳ Prototype fully trustless marketplace settlement leveraging zero-knowledge rollups.
- **AI & Automation**
- ⏳ Integrate AI-driven monitoring/anomaly detection for proposer health, market liquidity, and fraud detection.
- ⏳ Automate incident response playbooks with ChatOps and policy engines.
- ⏳ Launch research into autonomous agent participation (AI agents bidding/offering in the marketplace) and governance implications.
- **Global Standards Leadership**
- ⏳ chair industry working groups defining receipt/marketplace interoperability standards.
- ⏳ Publish annual transparency reports and sustainability metrics for stakeholders.
- ⏳ Engage with academia and open-source foundations to steward long-term protocol evolution.
### Stage 10 — Stewardship & Legacy Planning
- **Open Governance Maturity**
- ⏳ Transition roadmap ownership to community-elected councils with transparent voting and treasury controls.
- ⏳ Codify constitutional documents (mission, values, conflict resolution) and publish public charters.
- ⏳ Implement on-chain governance modules for protocol upgrades and ecosystem-wide decisions.
- **Educational & Outreach Programs**
- ⏳ Fund university partnerships, research chairs, and developer fellowships focused on decentralized marketplace tech.
- ⏳ Create certification tracks and mentorship programs for new validator/operators.
- ⏳ Launch annual global summit and publish proceedings to share best practices across partners.
- **Long-Term Preservation**
- ⏳ Archive protocol specs, governance records, and cultural artifacts in decentralized storage with redundancy.
- ⏳ Establish legal/organizational frameworks to ensure continuity across jurisdictions.
- ⏳ Develop end-of-life/transition plans for legacy components, documenting deprecation strategies and migration tooling.
## Shared Libraries & Examples
the canonical checklist during implementation. Mark completed tasks with ✅ and add dates or links to relevant PRs as development progresses.

286
docs/reference/threat-modeling.md Normal file
View File

@ -0,0 +1,286 @@
# AITBC Threat Modeling: Privacy Features
## Overview
This document provides a comprehensive threat model for AITBC's privacy-preserving features, focusing on zero-knowledge receipt attestation and confidential transactions. The analysis uses the STRIDE methodology to systematically identify threats and their mitigations.
## Document Version
- Version: 1.0
- Date: December 2024
- Status: Published - Shared with Ecosystem Partners
## Scope
### In-Scope Components
1. **ZK Receipt Attestation System**
- Groth16 circuit implementation
- Proof generation service
- Verification contract
- Trusted setup ceremony
2. **Confidential Transaction System**
- Hybrid encryption (AES-256-GCM + X25519)
- HSM-backed key management
- Access control system
- Audit logging infrastructure
### Out-of-Scope Components
- Core blockchain consensus
- Basic transaction processing
- Non-confidential marketplace operations
- Network layer security
## Threat Actors
| Actor | Motivation | Capability | Impact |
|-------|------------|------------|--------|
| Malicious Miner | Financial gain, sabotage | Access to mining software, limited compute | High |
| Compromised Coordinator | Data theft, market manipulation | System access, private keys | Critical |
| External Attacker | Financial theft, privacy breach | Public network, potential exploits | High |
| Regulator | Compliance investigation | Legal authority, subpoenas | Medium |
| Insider Threat | Data exfiltration | Internal access, knowledge | High |
| Quantum Computer | Break cryptography | Future quantum capability | Future |
## STRIDE Analysis
### 1. Spoofing
#### ZK Receipt Attestation
| Threat | Description | Likelihood | Impact | Mitigations |
|--------|-------------|------------|--------|-------------|
| Proof Forgery | Attacker creates fake ZK proofs | Medium | High | ✅ Groth16 soundness property<br>✅ Verification on-chain<br>⚠️ Trusted setup security |
| Identity Spoofing | Miner impersonates another | Low | Medium | ✅ Miner registration with KYC<br>✅ Cryptographic signatures |
| Coordinator Impersonation | Fake coordinator services | Low | High | ✅ TLS certificates<br>⚠️ DNSSEC recommended |
#### Confidential Transactions
| Threat | Description | Likelihood | Impact | Mitigations |
|--------|-------------|------------|--------|-------------|
| Key Spoofing | Fake public keys for participants | Medium | High | ✅ HSM-protected keys<br>✅ Certificate validation |
| Authorization Forgery | Fake audit authorization | Low | High | ✅ Signed tokens<br>✅ Short expiration times |
### 2. Tampering
#### ZK Receipt Attestation
| Threat | Description | Likelihood | Impact | Mitigations |
|--------|-------------|------------|--------|-------------|
| Circuit Modification | Malicious changes to circom circuit | Low | Critical | ✅ Open-source circuits<br>✅ Circuit hash verification |
| Proof Manipulation | Altering proofs during transmission | Medium | High | ✅ End-to-end encryption<br>✅ On-chain verification |
| Setup Parameter Poisoning | Compromise trusted setup | Low | Critical | ⚠️ Multi-party ceremony needed<br>⚠️ Secure destruction of toxic waste |
#### Confidential Transactions
| Threat | Description | Likelihood | Impact | Mitigations |
|--------|-------------|------------|--------|-------------|
| Data Tampering | Modify encrypted transaction data | Medium | High | ✅ AES-GCM authenticity<br>✅ Immutable audit logs |
| Key Substitution | Swap public keys in transit | Low | High | ✅ Certificate pinning<br>✅ HSM key validation |
| Access Control Bypass | Override authorization checks | Low | High | ✅ Role-based access control<br>✅ Audit logging of all changes |
### 3. Repudiation
#### ZK Receipt Attestation
| Threat | Description | Likelihood | Impact | Mitigations |
|--------|-------------|------------|--------|-------------|
| Denial of Proof Generation | Miner denies creating proof | Low | Medium | ✅ On-chain proof records<br>✅ Signed proof metadata |
| Receipt Denial | Party denies transaction occurred | Medium | Medium | ✅ Immutable blockchain ledger<br>✅ Cryptographic receipts |
#### Confidential Transactions
| Threat | Description | Likelihood | Impact | Mitigations |
|--------|-------------|------------|--------|-------------|
| Access Denial | User denies accessing data | Low | Medium | ✅ Comprehensive audit logs<br>✅ Non-repudiation signatures |
| Key Generation Denial | Deny creating encryption keys | Low | Medium | ✅ HSM audit trails<br>✅ Key rotation logs |
### 4. Information Disclosure
#### ZK Receipt Attestation
| Threat | Description | Likelihood | Impact | Mitigations |
|--------|-------------|------------|--------|-------------|
| Witness Extraction | Extract private inputs from proof | Low | Critical | ✅ Zero-knowledge property<br>✅ No knowledge of witness |
| Setup Parameter Leak | Expose toxic waste from trusted setup | Low | Critical | ⚠️ Secure multi-party setup<br>⚠️ Parameter destruction |
| Side-Channel Attacks | Timing/power analysis | Low | Medium | ✅ Constant-time implementations<br>⚠️ Needs hardware security review |
#### Confidential Transactions
| Threat | Description | Likelihood | Impact | Mitigations |
|--------|-------------|------------|--------|-------------|
| Private Key Extraction | Steal keys from HSM | Low | Critical | ✅ HSM security controls<br>✅ Hardware tamper resistance |
| Decryption Key Leak | Expose DEKs | Medium | High | ✅ Per-transaction DEKs<br>✅ Encrypted key storage |
| Metadata Analysis | Infer data from access patterns | Medium | Medium | ✅ Access logging<br>⚠️ Differential privacy needed |
### 5. Denial of Service
#### ZK Receipt Attestation
| Threat | Description | Likelihood | Impact | Mitigations |
|--------|-------------|------------|--------|-------------|
| Proof Generation DoS | Overwhelm proof service | High | Medium | ✅ Rate limiting<br>✅ Queue management<br>⚠️ Need monitoring |
| Verification Spam | Flood verification contract | High | High | ✅ Gas costs limit spam<br>⚠️ Need circuit optimization |
#### Confidential Transactions
| Threat | Description | Likelihood | Impact | Mitigations |
|--------|-------------|------------|--------|-------------|
| Key Exhaustion | Deplete HSM key slots | Medium | Medium | ✅ Key rotation<br>✅ Resource monitoring |
| Database Overload | Saturate with encrypted data | High | Medium | ✅ Connection pooling<br>✅ Query optimization |
| Audit Log Flooding | Fill audit storage | Medium | Medium | ✅ Log rotation<br>✅ Storage monitoring |
### 6. Elevation of Privilege
#### ZK Receipt Attestation
| Threat | Description | Likelihood | Impact | Mitigations |
|--------|-------------|------------|--------|-------------|
| Setup Privilege | Gain trusted setup access | Low | Critical | ⚠️ Multi-party ceremony<br>⚠️ Independent audits |
| Coordinator Compromise | Full system control | Medium | Critical | ✅ Multi-sig controls<br>✅ Regular security audits |
#### Confidential Transactions
| Threat | Description | Likelihood | Impact | Mitigations |
|--------|-------------|------------|--------|-------------|
| HSM Takeover | Gain HSM admin access | Low | Critical | ✅ HSM access controls<br>✅ Dual authorization |
| Access Control Escalation | Bypass role restrictions | Medium | High | ✅ Principle of least privilege<br>✅ Regular access reviews |
## Risk Matrix
| Threat | Likelihood | Impact | Risk Level | Priority |
|--------|------------|--------|------------|----------|
| Trusted Setup Compromise | Low | Critical | HIGH | 1 |
| HSM Compromise | Low | Critical | HIGH | 1 |
| Proof Forgery | Medium | High | HIGH | 2 |
| Private Key Extraction | Low | Critical | HIGH | 2 |
| Information Disclosure | Medium | High | MEDIUM | 3 |
| DoS Attacks | High | Medium | MEDIUM | 3 |
| Side-Channel Attacks | Low | Medium | LOW | 4 |
| Repudiation | Low | Medium | LOW | 4 |
## Implemented Mitigations
### ZK Receipt Attestation
- ✅ Groth16 soundness and zero-knowledge properties
- ✅ On-chain verification prevents tampering
- ✅ Open-source circuit code for transparency
- ✅ Rate limiting on proof generation
- ✅ Comprehensive audit logging
### Confidential Transactions
- ✅ AES-256-GCM provides confidentiality and authenticity
- ✅ HSM-backed key management prevents key extraction
- ✅ Role-based access control with time restrictions
- ✅ Per-transaction DEKs for forward secrecy
- ✅ Immutable audit trails with chain of hashes
- ✅ Multi-factor authentication for sensitive operations
## Recommended Future Improvements
### Short Term (1-3 months)
1. **Trusted Setup Ceremony**
- Implement multi-party computation (MPC) setup
- Engage independent auditors
- Publicly document process
2. **Enhanced Monitoring**
- Real-time threat detection
- Anomaly detection for access patterns
- Automated alerting for security events
3. **Security Testing**
- Penetration testing by third party
- Side-channel resistance evaluation
- Fuzzing of circuit implementations
### Medium Term (3-6 months)
1. **Advanced Privacy**
- Differential privacy for metadata
- Secure multi-party computation
- Homomorphic encryption support
2. **Quantum Resistance**
- Evaluate post-quantum schemes
- Migration planning for quantum threats
- Hybrid cryptography implementations
3. **Compliance Automation**
- Automated compliance reporting
- Privacy impact assessments
- Regulatory audit tools
### Long Term (6-12 months)
1. **Formal Verification**
- Formal proofs of circuit correctness
- Verified smart contract deployments
- Mathematical security proofs
2. **Decentralized Trust**
- Distributed key generation
- Threshold cryptography
- Community governance of security
## Security Controls Summary
### Preventive Controls
- Cryptographic guarantees (ZK proofs, encryption)
- Access control mechanisms
- Secure key management
- Network security (TLS, certificates)
### Detective Controls
- Comprehensive audit logging
- Real-time monitoring
- Anomaly detection
- Security incident response
### Corrective Controls
- Key rotation procedures
- Incident response playbooks
- Backup and recovery
- System patching processes
### Compensating Controls
- Insurance for cryptographic risks
- Legal protections
- Community oversight
- Bug bounty programs
## Compliance Mapping
| Regulation | Requirement | Implementation |
|------------|-------------|----------------|
| GDPR | Right to encryption | ✅ Opt-in confidential transactions |
| GDPR | Data minimization | ✅ Selective disclosure |
| SEC 17a-4 | Audit trail | ✅ Immutable logs |
| MiFID II | Transaction reporting | ✅ ZK proof verification |
| PCI DSS | Key management | ✅ HSM-backed keys |
## Incident Response
### Security Event Classification
1. **Critical** - HSM compromise, trusted setup breach
2. **High** - Large-scale data breach, proof forgery
3. **Medium** - Single key compromise, access violation
4. **Low** - Failed authentication, minor DoS
### Response Procedures
1. Immediate containment
2. Evidence preservation
3. Stakeholder notification
4. Root cause analysis
5. Remediation actions
6. Post-incident review
## Review Schedule
- **Monthly**: Security monitoring review
- **Quarterly**: Threat model update
- **Semi-annually**: Penetration testing
- **Annually**: Full security audit
## Contact Information
- Security Team: security@aitbc.io
- Bug Reports: security-bugs@aitbc.io
- Security Researchers: research@aitbc.io
## Acknowledgments
This threat model was developed with input from:
- AITBC Security Team
- External Security Consultants
- Community Security Researchers
- Cryptography Experts
---
*This document is living and will be updated as new threats emerge and mitigations are implemented.*

166
docs/reference/zk-implementation-summary.md Normal file
View File

@ -0,0 +1,166 @@
# ZK Receipt Attestation Implementation Summary
## Overview
Successfully implemented a zero-knowledge proof system for privacy-preserving receipt attestation in AITBC, enabling confidential settlements while maintaining verifiability.
## Components Implemented
### 1. ZK Circuits (`apps/zk-circuits/`)
- **Basic Circuit**: Receipt hash preimage proof in circom
- **Advanced Circuit**: Full receipt validation with pricing (WIP)
- **Build System**: npm scripts for compilation, setup, and proving
- **Testing**: Proof generation and verification tests
- **Benchmarking**: Performance measurement tools
### 2. Proof Service (`apps/coordinator-api/src/app/services/zk_proofs.py`)
- **ZKProofService**: Handles proof generation and verification
- **Privacy Levels**: Basic (hide computation) and Enhanced (hide amounts)
- **Integration**: Works with existing receipt signing system
- **Error Handling**: Graceful fallback when ZK unavailable
### 3. Receipt Integration (`apps/coordinator-api/src/app/services/receipts.py`)
- **Async Support**: Updated create_receipt to support async ZK generation
- **Optional Privacy**: ZK proofs generated only when requested
- **Backward Compatibility**: Existing receipts work unchanged
### 4. Verification Contract (`contracts/ZKReceiptVerifier.sol`)
- **On-Chain Verification**: Groth16 proof verification
- **Security Features**: Double-spend prevention, timestamp validation
- **Authorization**: Controlled access to verification functions
- **Batch Support**: Efficient batch verification
### 5. Settlement Integration (`apps/coordinator-api/aitbc/settlement/hooks.py`)
- **Privacy Options**: Settlement requests can specify privacy level
- **Proof Inclusion**: ZK proofs included in settlement messages
- **Bridge Support**: Works with existing cross-chain bridges
## Key Features
### Privacy Levels
1. **Basic**: Hide computation details, reveal settlement amount
2. **Enhanced**: Hide all amounts, prove correctness mathematically
### Performance Metrics
- **Proof Size**: ~200 bytes (Groth16)
- **Generation Time**: 5-15 seconds
- **Verification Time**: <5ms on-chain
- **Gas Cost**: ~200k gas
### Security Measures
- Trusted setup requirements documented
- Circuit audit procedures defined
- Gradual rollout strategy
- Emergency pause capabilities
## Testing Coverage
### Unit Tests
- Proof generation with various inputs
- Verification success/failure scenarios
- Privacy level validation
- Error handling
### Integration Tests
- Receipt creation with ZK proofs
- Settlement flow with privacy
- Cross-chain bridge integration
### Benchmarks
- Proof generation time measurement
- Verification performance
- Memory usage tracking
- Gas cost estimation
## Usage Examples
### Creating Private Receipt
```python
receipt = await receipt_service.create_receipt(
job=job,
miner_id=miner_id,
job_result=result,
result_metrics=metrics,
privacy_level="basic" # Enable ZK proof
)
```
### Cross-Chain Settlement with Privacy
```python
settlement = await settlement_hook.initiate_manual_settlement(
job_id="job-123",
target_chain_id=2,
use_zk_proof=True,
privacy_level="enhanced"
)
```
### On-Chain Verification
```solidity
bool verified = verifier.verifyAndRecord(
proof.a,
proof.b,
proof.c,
proof.publicSignals
);
```
## Current Status
### Completed ✅
1. Research and technology selection (Groth16)
2. Development environment setup
3. Basic circuit implementation
4. Proof generation service
5. Verification contract
6. Settlement integration
7. Comprehensive testing
8. Performance benchmarking
### Pending ⏳
1. Trusted setup ceremony (production requirement)
2. Circuit security audit
3. Full receipt validation circuit
4. Production deployment
## Next Steps for Production
### Immediate (Week 1-2)
1. Run end-to-end tests with real data
2. Performance optimization based on benchmarks
3. Security review of implementation
### Short Term (Month 1)
1. Plan and execute trusted setup ceremony
2. Complete advanced circuit with signature verification
3. Third-party security audit
### Long Term (Month 2-3)
1. Production deployment with gradual rollout
2. Monitor performance and gas costs
3. Consider PLONK for universal setup
## Risks and Mitigations
### Technical Risks
- **Trusted Setup**: Mitigate with multi-party ceremony
- **Performance**: Optimize circuits and use batch verification
- **Complexity**: Maintain clear documentation and examples
### Operational Risks
- **User Adoption**: Provide clear UI indicators for privacy
- **Gas Costs**: Optimize proof size and verification
- **Regulatory**: Ensure compliance with privacy regulations
## Documentation
- [ZK Technology Comparison](zk-technology-comparison.md)
- [Circuit Design](zk-receipt-attestation.md)
- [Development Guide](../apps/zk-circuits/README.md)
- [API Documentation](../docs/api/coordinator/endpoints.md)
## Conclusion
The ZK receipt attestation system provides a solid foundation for privacy-preserving settlements in AITBC. The implementation balances privacy, performance, and usability while maintaining backward compatibility with existing systems.
The modular design allows for gradual adoption and future enhancements, making it suitable for both testing and production deployment.

260
docs/reference/zk-receipt-attestation.md Normal file
View File

@ -0,0 +1,260 @@
# Zero-Knowledge Receipt Attestation Design
## Overview
This document outlines the design for adding zero-knowledge proof capabilities to the AITBC receipt attestation system, enabling privacy-preserving settlement flows while maintaining verifiability.
## Goals
1. **Privacy**: Hide sensitive transaction details (amounts, parties, specific computations)
2. **Verifiability**: Prove receipts are valid and correctly signed without revealing contents
3. **Compatibility**: Work with existing receipt signing and settlement systems
4. **Efficiency**: Minimize proof generation and verification overhead
## Architecture
### Current Receipt System
The existing system has:
- Receipt signing with coordinator private key
- Optional coordinator attestations
- History retrieval endpoints
- Cross-chain settlement hooks
Receipt structure includes:
- Job ID and metadata
- Computation results
- Pricing information
- Miner and coordinator signatures
### Privacy-Preserving Flow
```
1. Job Execution
↓
2. Receipt Generation (clear text)
↓
3. ZK Circuit Input Preparation
↓
4. ZK Proof Generation
↓
5. On-Chain Settlement (with proof)
↓
6. Verification (without revealing data)
```
## ZK Circuit Design
### What to Prove
1. **Receipt Validity**
- Receipt was signed by coordinator
- Computation was performed correctly
- Pricing follows agreed rules
2. **Settlement Conditions**
- Amount owed is correctly calculated
- Parties have sufficient funds/balance
- Cross-chain transfer conditions met
### What to Hide
1. **Sensitive Data**
- Actual computation amounts
- Specific job details
- Pricing rates
- Participant identities
### Circuit Components
```circom
// High-level circuit structure
template ReceiptAttestation() {
// Public inputs
signal input receiptHash;
signal input settlementAmount;
signal input timestamp;
// Private inputs
signal input receipt;
signal input computationResult;
signal input pricingRate;
signal input minerReward;
// Verify receipt signature
component signatureVerifier = ECDSAVerify();
// ... signature verification logic
// Verify computation correctness
component computationChecker = ComputationVerify();
// ... computation verification logic
// Verify pricing calculation
component pricingVerifier = PricingVerify();
// ... pricing verification logic
// Output settlement proof
settlementAmount <== minerReward + coordinatorFee;
}
```
## Implementation Plan
### Phase 1: Research & Prototyping
1. **Library Selection**
- snarkjs for development (JavaScript/TypeScript)
- circomlib2 for standard circuits
- Web3.js for blockchain integration
2. **Basic Circuit**
- Simple receipt hash preimage proof
- ECDSA signature verification
- Basic arithmetic operations
### Phase 2: Integration
1. **Coordinator API Updates**
- Add ZK proof generation endpoint
- Integrate with existing receipt signing
- Add proof verification utilities
2. **Settlement Flow**
- Modify cross-chain hooks to accept proofs
- Update verification logic
- Maintain backward compatibility
### Phase 3: Optimization
1. **Performance**
- Trusted setup for Groth16
- Batch proof generation
- Recursive proofs for complex receipts
2. **Security**
- Audit circuits
- Formal verification
- Side-channel resistance
## Data Flow
### Proof Generation (Coordinator)
```python
async def generate_receipt_proof(receipt: Receipt) -> ZKProof:
# 1. Prepare circuit inputs
public_inputs = {
"receiptHash": hash_receipt(receipt),
"settlementAmount": calculate_settlement(receipt),
"timestamp": receipt.timestamp
}
private_inputs = {
"receipt": receipt,
"computationResult": receipt.result,
"pricingRate": receipt.pricing.rate,
"minerReward": receipt.pricing.miner_reward
}
# 2. Generate witness
witness = generate_witness(public_inputs, private_inputs)
# 3. Generate proof
proof = groth16.prove(witness, proving_key)
return {
"proof": proof,
"publicSignals": public_inputs
}
```
### Proof Verification (On-Chain/Settlement Layer)
```solidity
contract SettlementVerifier {
// Groth16 verifier
function verifySettlement(
uint256[2] memory a,
uint256[2][2] memory b,
uint256[2] memory c,
uint256[] memory input
) public pure returns (bool) {
return verifyProof(a, b, c, input);
}
function settleWithProof(
address recipient,
uint256 amount,
ZKProof memory proof
) public {
require(verifySettlement(proof.a, proof.b, proof.c, proof.inputs));
// Execute settlement
_transfer(recipient, amount);
}
}
```
## Privacy Levels
### Level 1: Basic Privacy
- Hide computation amounts
- Prove pricing correctness
- Reveal participant identities
### Level 2: Enhanced Privacy
- Hide all amounts
- Zero-knowledge participant proofs
- Anonymous settlement
### Level 3: Full Privacy
- Complete transaction privacy
- Ring signatures or similar
- Confidential transfers
## Security Considerations
1. **Trusted Setup**
- Multi-party ceremony for Groth16
- Documentation of setup process
- Toxic waste destruction proof
2. **Circuit Security**
- Constant-time operations
- No side-channel leaks
- Formal verification where possible
3. **Integration Security**
- Maintain existing security guarantees
- Fail-safe verification
- Gradual rollout with monitoring
## Migration Strategy
1. **Parallel Operation**
- Run both clear and ZK receipts
- Gradual opt-in adoption
- Performance monitoring
2. **Backward Compatibility**
- Existing receipts remain valid
- Optional ZK proofs
- Graceful degradation
3. **Network Upgrade**
- Coordinate with all participants
- Clear communication
- Rollback capability
## Next Steps
1. **Research Task**
- Evaluate zk-SNARKs vs zk-STARKs trade-offs
- Benchmark proof generation times
- Assess gas costs for on-chain verification
2. **Prototype Development**
- Implement basic circuit in circom
- Create proof generation service
- Build verification contract
3. **Integration Planning**
- Design API changes
- Plan data migration
- Prepare rollout strategy

181
docs/reference/zk-technology-comparison.md Normal file
View File

@ -0,0 +1,181 @@
# ZK Technology Comparison for Receipt Attestation
## Overview
Analysis of zero-knowledge proof systems for AITBC receipt attestation, focusing on practical considerations for integration with existing infrastructure.
## Technology Options
### 1. zk-SNARKs (Zero-Knowledge Succinct Non-Interactive Argument of Knowledge)
**Examples**: Groth16, PLONK, Halo2
**Pros**:
- **Small proof size**: ~200 bytes for Groth16
- **Fast verification**: Constant time, ~3ms on-chain
- **Mature ecosystem**: circom, snarkjs, bellman, arkworks
- **Low gas costs**: ~200k gas for verification on Ethereum
- **Industry adoption**: Used by Aztec, Tornado Cash, Zcash
**Cons**:
- **Trusted setup**: Required for Groth16 (toxic waste problem)
- **Longer proof generation**: 10-30 seconds depending on circuit size
- **Complex setup**: Ceremony needs multiple participants
- **Quantum vulnerability**: Not post-quantum secure
### 2. zk-STARKs (Zero-Knowledge Scalable Transparent Argument of Knowledge)
**Examples**: STARKEx, Winterfell, gnark
**Pros**:
- **No trusted setup**: Transparent setup process
- **Post-quantum secure**: Resistant to quantum attacks
- **Faster proving**: Often faster than SNARKs for large circuits
- **Transparent**: No toxic waste, fully verifiable setup
**Cons**:
- **Larger proofs**: ~45KB for typical circuits
- **Higher verification cost**: ~500k-1M gas on-chain
- **Newer ecosystem**: Fewer tools and libraries
- **Less adoption**: Limited production deployments
## Use Case Analysis
### Receipt Attestation Requirements
1. **Proof Size**: Important for on-chain storage costs
2. **Verification Speed**: Critical for settlement latency
3. **Setup Complexity**: Affects deployment timeline
4. **Ecosystem Maturity**: Impacts development speed
5. **Privacy Needs**: Moderate (hiding amounts, not full anonymity)
### Quantitative Comparison
| Metric | Groth16 (SNARK) | PLONK (SNARK) | STARK |
|--------|----------------|---------------|-------|
| Proof Size | 200 bytes | 400-500 bytes | 45KB |
| Prover Time | 10-30s | 5-15s | 2-10s |
| Verifier Time | 3ms | 5ms | 50ms |
| Gas Cost | 200k | 300k | 800k |
| Trusted Setup | Yes | Universal | No |
| Library Support | Excellent | Good | Limited |
## Recommendation
### Phase 1: Groth16 for MVP
**Rationale**:
1. **Proven technology**: Battle-tested in production
2. **Small proofs**: Essential for cost-effective on-chain verification
3. **Fast verification**: Critical for settlement performance
4. **Tool maturity**: circom + snarkjs ecosystem
5. **Community knowledge**: Extensive documentation and examples
**Mitigations for trusted setup**:
- Multi-party ceremony with >100 participants
- Public documentation of process
- Consider PLONK for Phase 2 if setup becomes bottleneck
### Phase 2: Evaluate PLONK
**Rationale**:
- Universal trusted setup (one-time for all circuits)
- Slightly larger proofs but acceptable
- More flexible for circuit updates
- Growing ecosystem support
### Phase 3: Consider STARKs
**Rationale**:
- If quantum resistance becomes priority
- If proof size optimizations improve
- If gas costs become less critical
## Implementation Strategy
### Circuit Complexity Analysis
**Basic Receipt Circuit**:
- Hash verification: ~50 constraints
- Signature verification: ~10,000 constraints
- Arithmetic operations: ~100 constraints
- Total: ~10,150 constraints
**With Privacy Features**:
- Range proofs: ~1,000 constraints
- Merkle proofs: ~1,000 constraints
- Additional checks: ~500 constraints
- Total: ~12,650 constraints
### Performance Estimates
**Groth16**:
- Setup time: 2-5 hours
- Proving time: 5-15 seconds
- Verification: 3ms
- Proof size: 200 bytes
**Infrastructure Impact**:
- Coordinator: Additional 5-15s per receipt
- Settlement layer: Minimal impact (fast verification)
- Storage: Negligible increase
## Security Considerations
### Trusted Setup Risks
1. **Toxic Waste**: If compromised, can forge proofs
2. **Setup Integrity**: Requires honest participants
3. **Documentation**: Must be publicly verifiable
### Mitigation Strategies
1. **Multi-party Ceremony**:
- Minimum 100 participants
- Geographically distributed
- Public livestream
2. **Circuit Audits**:
- Formal verification where possible
- Third-party security review
- Public disclosure of circuits
3. **Gradual Rollout**:
- Start with low-value transactions
- Monitor for anomalies
- Emergency pause capability
## Development Plan
### Week 1-2: Environment Setup
- Install circom and snarkjs
- Create basic test circuit
- Benchmark proof generation
### Week 3-4: Basic Circuit
- Implement receipt hash verification
- Add signature verification
- Test with sample receipts
### Week 5-6: Integration
- Add to coordinator API
- Create verification contract
- Test settlement flow
### Week 7-8: Trusted Setup
- Plan ceremony logistics
- Prepare ceremony software
- Execute multi-party setup
### Week 9-10: Testing & Audit
- End-to-end testing
- Security review
- Performance optimization
## Next Steps
1. **Immediate**: Set up development environment
2. **Research**: Deep dive into circom best practices
3. **Prototype**: Build minimal viable circuit
4. **Evaluate**: Performance with real receipt data
5. **Decide**: Final technology choice based on testing