docs: add structure.md, update files.md, rewrite README for GitHub, add favicon, replace debug prints with logging, remove stale src/ copy and empty dirs
This commit is contained in:
oib
11
.gitignore
vendored
11
.gitignore
vendored
@@ -82,6 +82,12 @@ pids/
|
||||
.classpath
|
||||
.settings/
|
||||
|
||||
# ===================
|
||||
# Runtime / PID files
|
||||
# ===================
|
||||
*.pid
|
||||
apps/.service_pids
|
||||
|
||||
# ===================
|
||||
# OS Files
|
||||
# ===================
|
||||
@@ -143,6 +149,11 @@ home/client/client_wallet.json
|
||||
home/genesis_wallet.json
|
||||
home/miner/miner_wallet.json
|
||||
|
||||
# ===================
|
||||
# Stale source copies
|
||||
# ===================
|
||||
src/aitbc_chain/
|
||||
|
||||
# ===================
|
||||
# Project Specific
|
||||
# ===================
|
||||
|
||||
157
README.md
157
README.md
@@ -1,36 +1,143 @@
|
||||
# AITBC Monorepo
|
||||
# AITBC — AI Token Blockchain
|
||||
|
||||
This repository houses all components of the Artificial Intelligence Token Blockchain (AITBC) stack, including coordinator services, blockchain node, miner daemon, client-facing web apps, SDKs, and documentation.
|
||||
Decentralized GPU compute marketplace with blockchain-based job coordination, Ollama inference, ZK receipt verification, and token payments.
|
||||
|
||||
[](LICENSE)
|
||||
|
||||
## Overview
|
||||
|
||||
AITBC is a full-stack blockchain platform that connects GPU compute providers (miners) with AI workload consumers (clients) through a decentralized marketplace. The system handles job submission, miner matching, GPU inference execution, cryptographic receipt generation, and on-chain payment settlement.
|
||||
|
||||
**Key capabilities:**
|
||||
- **Blockchain nodes** — PoA consensus, gossip relay, WebSocket RPC
|
||||
- **Coordinator API** — Job lifecycle, miner registry, marketplace, multi-tenancy
|
||||
- **GPU mining** — Ollama-based LLM inference with host GPU passthrough
|
||||
- **Wallet daemon** — Balance tracking, receipt verification, ledger management
|
||||
- **Trade exchange** — Bitcoin/AITBC trading with order book and price ticker
|
||||
- **ZK circuits** — Zero-knowledge receipt verification (Circom/snarkjs)
|
||||
- **Browser wallet** — Firefox extension for AITBC transactions
|
||||
- **Explorer** — Real-time blockchain explorer (blocks, transactions, addresses, receipts)
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
Client ──► Coordinator API ──► Pool Hub ──► Miner (GPU/Ollama)
|
||||
│ │
|
||||
▼ ▼
|
||||
Marketplace ZK Receipt
|
||||
│ │
|
||||
▼ ▼
|
||||
Wallet Daemon ◄──── Blockchain Nodes ◄─── Settlement
|
||||
│
|
||||
▼
|
||||
Trade Exchange
|
||||
```
|
||||
|
||||
## Directory Structure
|
||||
|
||||
```
|
||||
aitbc/
|
||||
├── apps/ # Main applications (coordinator-api, blockchain-node, etc.)
|
||||
├── packages/ # Shared packages and SDKs
|
||||
├── scripts/ # Utility scripts
|
||||
│ └── testing/ # Test scripts and utilities
|
||||
├── tests/ # Pytest test suites
|
||||
├── docs/ # Documentation
|
||||
│ ├── guides/ # Development guides
|
||||
│ └── reports/ # Generated reports and summaries
|
||||
├── infra/ # Infrastructure and deployment configs
|
||||
├── dev-utils/ # Development utilities
|
||||
└── .windsurf/ # Windsurf IDE configuration
|
||||
├── apps/ # Core microservices
|
||||
│ ├── blockchain-node/ # PoA blockchain node (FastAPI + gossip)
|
||||
│ ├── coordinator-api/ # Job coordination API (FastAPI)
|
||||
│ ├── explorer-web/ # Blockchain explorer (TypeScript + Vite)
|
||||
│ ├── marketplace-web/ # GPU marketplace frontend (TypeScript + Vite)
|
||||
│ ├── pool-hub/ # Mining pool management (FastAPI + Redis)
|
||||
│ ├── trade-exchange/ # BTC/AITBC exchange (FastAPI + WebSocket)
|
||||
│ ├── wallet-daemon/ # Wallet service (FastAPI)
|
||||
│ └── zk-circuits/ # ZK proof circuits (Circom)
|
||||
├── cli/ # CLI tools (client, miner, wallet)
|
||||
├── contracts/ # Solidity smart contracts
|
||||
├── docs/ # Documentation (structure, guides, reference, reports)
|
||||
├── extensions/ # Browser extensions (Firefox wallet)
|
||||
├── home/ # Local simulation scripts
|
||||
├── infra/ # Infrastructure (nginx, k8s, helm, terraform)
|
||||
├── packages/ # Shared libraries
|
||||
│ ├── py/aitbc-crypto/ # Cryptographic primitives
|
||||
│ ├── py/aitbc-sdk/ # Python SDK
|
||||
│ └── solidity/aitbc-token/# ERC-20 token contract
|
||||
├── plugins/ollama/ # Ollama LLM integration
|
||||
├── scripts/ # Deployment, GPU, service, and test scripts
|
||||
├── systemd/ # Systemd service units
|
||||
├── tests/ # Test suites (unit, integration, e2e, security, load)
|
||||
└── website/ # Public website and HTML documentation
|
||||
```
|
||||
|
||||
## Getting Started
|
||||
See [docs/structure.md](docs/structure.md) for detailed file-level documentation.
|
||||
|
||||
1. Review the bootstrap documents under `docs/bootstrap/` to understand stage-specific goals.
|
||||
2. Fill in service-specific READMEs located under `apps/` and `packages/` as the implementations progress.
|
||||
3. Use the provided directory scaffold as the starting point for coding each subsystem.
|
||||
4. Explore the new Python receipt SDK under `packages/py/aitbc-sdk/` for helpers to fetch and verify coordinator receipts (see `docs/run.md` for examples).
|
||||
5. Run `scripts/ci/run_python_tests.sh` (via Poetry) to execute coordinator, SDK, miner-node, and wallet-daemon test suites before submitting changes.
|
||||
6. GitHub Actions (`.github/workflows/python-tests.yml`) automatically runs the same script on pushes and pull requests targeting `main`.
|
||||
## Quick Start
|
||||
|
||||
## Testing
|
||||
### Prerequisites
|
||||
|
||||
- Test scripts are located in `scripts/testing/`
|
||||
- Run individual tests: `python3 scripts/testing/test_block_import.py`
|
||||
- Run pytest suite: `pytest tests/`
|
||||
- See `docs/guides/WINDSURF_TESTING_GUIDE.md` for detailed testing instructions
|
||||
- Python 3.11+
|
||||
- Node.js 18+ (for web apps and ZK circuits)
|
||||
- PostgreSQL (optional, SQLite for development)
|
||||
- Ollama (for GPU inference)
|
||||
|
||||
### Run Services Locally
|
||||
|
||||
```bash
|
||||
# Start all services
|
||||
./scripts/dev_services.sh
|
||||
|
||||
# Or start individually
|
||||
cd apps/coordinator-api && uvicorn app.main:app --port 8000
|
||||
cd apps/blockchain-node && python -m aitbc_chain.main
|
||||
cd apps/wallet-daemon && uvicorn app.main:app --port 8002
|
||||
```
|
||||
|
||||
### Run Tests
|
||||
|
||||
```bash
|
||||
# Full test suite
|
||||
pytest tests/
|
||||
|
||||
# Unit tests only
|
||||
pytest tests/unit/
|
||||
|
||||
# Integration tests
|
||||
pytest tests/integration/
|
||||
|
||||
# CI script (all apps)
|
||||
./scripts/ci/run_python_tests.sh
|
||||
```
|
||||
|
||||
### CLI Usage
|
||||
|
||||
```bash
|
||||
# Submit a job as a client
|
||||
python cli/client.py submit --model llama3 --prompt "Hello world"
|
||||
|
||||
# Start mining
|
||||
python cli/miner.py start --gpu 0
|
||||
|
||||
# Check wallet balance
|
||||
python cli/wallet.py balance
|
||||
```
|
||||
|
||||
## Deployment
|
||||
|
||||
Services run in an Incus container with systemd units. See `systemd/` for service definitions and `scripts/deploy/` for deployment automation.
|
||||
|
||||
```bash
|
||||
# Deploy to container
|
||||
./scripts/deploy/deploy-to-container.sh
|
||||
|
||||
# Deploy blockchain nodes
|
||||
./scripts/deploy/deploy-blockchain.sh
|
||||
```
|
||||
|
||||
## Documentation
|
||||
|
||||
| Document | Description |
|
||||
|----------|-------------|
|
||||
| [docs/structure.md](docs/structure.md) | Codebase structure and file layout |
|
||||
| [docs/files.md](docs/files.md) | File audit and status tracking |
|
||||
| [docs/roadmap.md](docs/roadmap.md) | Development roadmap |
|
||||
| [docs/components.md](docs/components.md) | Component overview |
|
||||
| [docs/infrastructure.md](docs/infrastructure.md) | Infrastructure guide |
|
||||
| [docs/full-documentation.md](docs/full-documentation.md) | Complete technical documentation |
|
||||
|
||||
## License
|
||||
|
||||
[MIT](LICENSE) — Copyright (c) 2025 AITBC
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
1529925 1529926 1529927 1529928
|
||||
@@ -8,6 +8,9 @@ import uuid
|
||||
import time
|
||||
import json
|
||||
import os
|
||||
import logging
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
from ..schemas import ExchangePaymentRequest, ExchangePaymentResponse
|
||||
from ..services.bitcoin_wallet import get_wallet_balance, get_wallet_info
|
||||
@@ -109,7 +112,7 @@ async def confirm_payment(
|
||||
from ..services.blockchain import mint_tokens
|
||||
mint_tokens(payment['user_id'], payment['aitbc_amount'])
|
||||
except Exception as e:
|
||||
print(f"Error minting tokens: {e}")
|
||||
logger.error("Error minting tokens: %s", e)
|
||||
# In production, handle this error properly
|
||||
|
||||
return {
|
||||
|
||||
@@ -7,6 +7,9 @@ from pydantic import BaseModel, Field
|
||||
from typing import Optional, Dict, Any, List
|
||||
from datetime import datetime, timedelta
|
||||
import json
|
||||
import logging
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
from ..schemas import UserProfile
|
||||
from ..storage import SessionDep
|
||||
@@ -349,7 +352,7 @@ async def execute_parameter_change(target: Dict[str, Any], background_tasks):
|
||||
"""Execute a parameter change proposal"""
|
||||
|
||||
# This would update system parameters
|
||||
print(f"Executing parameter change: {target}")
|
||||
logger.info("Executing parameter change: %s", target)
|
||||
# Implementation would depend on the specific parameters
|
||||
|
||||
|
||||
@@ -357,7 +360,7 @@ async def execute_protocol_upgrade(target: Dict[str, Any], background_tasks):
|
||||
"""Execute a protocol upgrade proposal"""
|
||||
|
||||
# This would trigger a protocol upgrade
|
||||
print(f"Executing protocol upgrade: {target}")
|
||||
logger.info("Executing protocol upgrade: %s", target)
|
||||
# Implementation would involve coordinating with nodes
|
||||
|
||||
|
||||
@@ -365,7 +368,7 @@ async def execute_fund_allocation(target: Dict[str, Any], background_tasks):
|
||||
"""Execute a fund allocation proposal"""
|
||||
|
||||
# This would transfer funds from treasury
|
||||
print(f"Executing fund allocation: {target}")
|
||||
logger.info("Executing fund allocation: %s", target)
|
||||
# Implementation would involve treasury management
|
||||
|
||||
|
||||
@@ -373,7 +376,7 @@ async def execute_policy_change(target: Dict[str, Any], background_tasks):
|
||||
"""Execute a policy change proposal"""
|
||||
|
||||
# This would update system policies
|
||||
print(f"Executing policy change: {target}")
|
||||
logger.info("Executing policy change: %s", target)
|
||||
# Implementation would depend on the specific policy
|
||||
|
||||
|
||||
|
||||
@@ -6,9 +6,12 @@ Uses RPC to connect to Bitcoin Core (or alternative like Block.io)
|
||||
|
||||
import os
|
||||
import json
|
||||
import logging
|
||||
import requests
|
||||
from typing import Dict, Optional
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Bitcoin wallet configuration
|
||||
WALLET_CONFIG = {
|
||||
# For development, we'll use testnet
|
||||
@@ -31,11 +34,11 @@ class BitcoinWallet:
|
||||
try:
|
||||
result = self._rpc_call('getbalance', ["*", 0, False])
|
||||
if result.get('error') is not None:
|
||||
print(f"Bitcoin RPC error: {result['error']}")
|
||||
logger.error("Bitcoin RPC error: %s", result['error'])
|
||||
return 0.0
|
||||
return result.get('result', 0.0)
|
||||
except Exception as e:
|
||||
print(f"Failed to get balance: {e}")
|
||||
logger.error("Failed to get balance: %s", e)
|
||||
return 0.0
|
||||
|
||||
def get_new_address(self) -> str:
|
||||
@@ -43,11 +46,11 @@ class BitcoinWallet:
|
||||
try:
|
||||
result = self._rpc_call('getnewaddress', ["", "bech32"])
|
||||
if result.get('error') is not None:
|
||||
print(f"Bitcoin RPC error: {result['error']}")
|
||||
logger.error("Bitcoin RPC error: %s", result['error'])
|
||||
return self.config['fallback_address']
|
||||
return result.get('result', self.config['fallback_address'])
|
||||
except Exception as e:
|
||||
print(f"Failed to get new address: {e}")
|
||||
logger.error("Failed to get new address: %s", e)
|
||||
return self.config['fallback_address']
|
||||
|
||||
def list_transactions(self, count: int = 10) -> list:
|
||||
@@ -55,11 +58,11 @@ class BitcoinWallet:
|
||||
try:
|
||||
result = self._rpc_call('listtransactions', ["*", count, 0, True])
|
||||
if result.get('error') is not None:
|
||||
print(f"Bitcoin RPC error: {result['error']}")
|
||||
logger.error("Bitcoin RPC error: %s", result['error'])
|
||||
return []
|
||||
return result.get('result', [])
|
||||
except Exception as e:
|
||||
print(f"Failed to list transactions: {e}")
|
||||
logger.error("Failed to list transactions: %s", e)
|
||||
return []
|
||||
|
||||
def _rpc_call(self, method: str, params: list = None) -> Dict:
|
||||
@@ -83,7 +86,7 @@ class BitcoinWallet:
|
||||
response.raise_for_status()
|
||||
return response.json()
|
||||
except Exception as e:
|
||||
print(f"RPC call failed: {e}")
|
||||
logger.error("RPC call failed: %s", e)
|
||||
return {"error": str(e)}
|
||||
|
||||
# Create a wallet instance
|
||||
@@ -117,7 +120,7 @@ def get_wallet_info() -> Dict[str, any]:
|
||||
"blocks": blockchain_info.get('result', {}).get('blocks', 0) if is_connected else 0
|
||||
}
|
||||
except Exception as e:
|
||||
print(f"Error getting wallet info: {e}")
|
||||
logger.error("Error getting wallet info: %s", e)
|
||||
return {
|
||||
"balance": 0.0,
|
||||
"address": "tb1qxy2kgdygjrsqtzq2n0yrf2493p83kkfjhx0wlh",
|
||||
|
||||
@@ -4,10 +4,13 @@ Blockchain service for AITBC token operations
|
||||
|
||||
import httpx
|
||||
import asyncio
|
||||
import logging
|
||||
from typing import Optional
|
||||
|
||||
from ..config import settings
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
BLOCKCHAIN_RPC = f"http://127.0.0.1:9080/rpc"
|
||||
|
||||
async def mint_tokens(address: str, amount: float) -> dict:
|
||||
@@ -44,6 +47,6 @@ def get_balance(address: str) -> Optional[float]:
|
||||
return float(data.get("balance", 0))
|
||||
|
||||
except Exception as e:
|
||||
print(f"Error getting balance: {e}")
|
||||
logger.error("Error getting balance: %s", e)
|
||||
|
||||
return None
|
||||
|
||||
@@ -1,9 +1,12 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from typing import Any, Dict, Optional
|
||||
from secrets import token_hex
|
||||
from datetime import datetime
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
from aitbc_crypto.signing import ReceiptSigner
|
||||
|
||||
from sqlmodel import Session
|
||||
@@ -129,7 +132,7 @@ class ReceiptService:
|
||||
|
||||
except Exception as e:
|
||||
# Log error but don't fail receipt creation
|
||||
print(f"Failed to generate ZK proof: {e}")
|
||||
logger.warning("Failed to generate ZK proof: %s", e)
|
||||
|
||||
receipt_row = JobReceipt(job_id=job.id, receipt_id=payload["receipt_id"], payload=payload)
|
||||
self.session.add(receipt_row)
|
||||
|
||||
@@ -8,9 +8,12 @@ import psycopg2
|
||||
from psycopg2.extras import RealDictCursor
|
||||
from typing import Generator, Optional, Dict, Any, List
|
||||
import json
|
||||
import logging
|
||||
from datetime import datetime
|
||||
from decimal import Decimal
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
from .config_pg import settings
|
||||
|
||||
# SQLAlchemy setup for complex queries
|
||||
@@ -203,7 +206,7 @@ def init_db():
|
||||
# Create all tables
|
||||
Base.metadata.create_all(bind=engine)
|
||||
|
||||
print("✅ PostgreSQL database initialized successfully!")
|
||||
logger.info("PostgreSQL database initialized successfully")
|
||||
|
||||
# Health check
|
||||
def check_db_health() -> Dict[str, Any]:
|
||||
|
||||
@@ -5,7 +5,7 @@ This document categorizes all files and folders in the repository by their statu
|
||||
- **Greylist (⚠️)**: Uncertain status, may need review
|
||||
- **Blacklist (❌)**: Legacy, unused, outdated, candidates for removal
|
||||
|
||||
Last updated: 2026-01-29
|
||||
Last updated: 2026-02-11
|
||||
|
||||
---
|
||||
|
||||
@@ -65,14 +65,6 @@ Last updated: 2026-01-29
|
||||
| `docs/guides/` | ✅ Active | Development guides (moved from root) |
|
||||
| `docs/reports/` | ✅ Active | Generated reports (moved from root) |
|
||||
|
||||
### Cascade Skills (`.windsurf/`)
|
||||
|
||||
| Path | Status | Notes |
|
||||
|------|--------|-------|
|
||||
| `.windsurf/skills/blockchain-operations/` | ✅ Active | Node management skill |
|
||||
| `.windsurf/skills/deploy-production/` | ✅ Active | Deployment skill |
|
||||
| `.windsurf/workflows/` | ✅ Active | Workflow definitions |
|
||||
|
||||
### CLI Tools (`cli/`)
|
||||
|
||||
| Path | Status | Notes |
|
||||
@@ -264,14 +256,82 @@ These empty folders are intentional scaffolding for planned future work per the
|
||||
|
||||
---
|
||||
|
||||
## Issues Found (2026-02-11)
|
||||
|
||||
### Empty Directories (Delete)
|
||||
|
||||
| Path | Action |
|
||||
|------|--------|
|
||||
| `apps/blockchain-node/src/aitbc_chain/ledger/` | Delete — empty placeholder, never implemented |
|
||||
| `apps/blockchain-node/src/aitbc_chain/mempool/` | Delete — empty dir, mempool logic is in `mempool.py` |
|
||||
| `apps/coordinator-api/src/app/ws/` | Delete — empty WebSocket placeholder, never implemented |
|
||||
| `apps/explorer-web/public/js/components/` | Delete — empty, TS components are in `src/components/` |
|
||||
| `apps/explorer-web/public/js/pages/` | Delete — empty, TS pages are in `src/pages/` |
|
||||
| `apps/explorer-web/public/js/vendors/` | Delete — empty vendor dir |
|
||||
| `apps/explorer-web/public/assets/` | Delete — empty assets dir |
|
||||
| `packages/py/aitbc-crypto/build/bdist.linux-x86_64/` | Delete — build artifact |
|
||||
|
||||
### Files in Wrong Location (Move)
|
||||
|
||||
| Current Path | Correct Path | Reason |
|
||||
|-------------|-------------|--------|
|
||||
| `apps/coordinator-api/coordinator.db` | gitignored / `data/` | SQLite database should not be in git |
|
||||
| `apps/coordinator-api/.env` | gitignored | Environment file with secrets, should not be in git |
|
||||
| `apps/.service_pids` | gitignored | Runtime PID file, should not be in git |
|
||||
| `src/aitbc_chain/` | `apps/blockchain-node/src/aitbc_chain/` | Duplicate/stale copy of blockchain node source |
|
||||
| `website/docs-clients.html` | `website/docs/docs-clients.html` | Inconsistent location, duplicate of file in `docs/` |
|
||||
| `website/docs-developers.html` | `website/docs/docs-developers.html` | Inconsistent location, duplicate of file in `docs/` |
|
||||
| `website/docs-miners.html` | `website/docs/docs-miners.html` | Inconsistent location, duplicate of file in `docs/` |
|
||||
| `website/docs-index.html` | `website/docs/index.html` | Inconsistent location, duplicate of file in `docs/` |
|
||||
|
||||
### Legacy Files (Delete)
|
||||
|
||||
| Path | Reason |
|
||||
|------|--------|
|
||||
| `SECURITY_CLEANUP_GUIDE.md` | One-time cleanup guide, already completed |
|
||||
| `apps/trade-exchange/index_working.html` | Backup copy of `index.html` |
|
||||
| `apps/trade-exchange/index.prod.html` | Superseded by `build.py` production build |
|
||||
| `apps/trade-exchange/index.real.html` | Superseded by `build.py` production build |
|
||||
| `tests/conftest_fixtures.py` | Unused alternate conftest |
|
||||
| `tests/conftest_full.py` | Unused alternate conftest |
|
||||
| `tests/conftest_path.py` | Unused alternate conftest |
|
||||
| `tests/pytest_simple.ini` | Duplicate of root `pytest.ini` |
|
||||
| `tests/test_blockchain_simple.py` | Superseded by `test_blockchain_nodes.py` |
|
||||
| `tests/test_blockchain_final.py` | Superseded by `test_blockchain_nodes.py` |
|
||||
| `tests/test_discovery.py` | One-time discovery script |
|
||||
| `tests/test_windsurf_integration.py` | IDE-specific test, not for GitHub |
|
||||
| `scripts/exchange-router-fixed.py` | One-time fix script |
|
||||
| `scripts/start_mock_blockchain.sh` | Superseded by `tests/mock_blockchain_node.py` |
|
||||
| `apps/marketplace-web/src/counter.ts` | Vite template boilerplate, unused |
|
||||
| `apps/marketplace-web/src/typescript.svg` | Vite template boilerplate, unused |
|
||||
| `apps/marketplace-web/public/vite.svg` | Vite template boilerplate, unused |
|
||||
| `.vscode/` | IDE-specific, should be gitignored |
|
||||
|
||||
### Debug Print Statements (Replace with logging)
|
||||
|
||||
| File | Lines | Statement |
|
||||
|------|-------|-----------|
|
||||
| `apps/coordinator-api/src/app/routers/exchange.py` | 112 | `print(f"Error minting tokens: {e}")` |
|
||||
| `apps/coordinator-api/src/app/routers/governance.py` | 352-376 | 4x `print(f"Executing ...")` |
|
||||
| `apps/coordinator-api/src/app/services/receipts.py` | 132 | `print(f"Failed to generate ZK proof: {e}")` |
|
||||
| `apps/coordinator-api/src/app/services/blockchain.py` | 47 | `print(f"Error getting balance: {e}")` |
|
||||
| `apps/coordinator-api/src/app/services/bitcoin_wallet.py` | 34-134 | 8x `print(...)` debug statements |
|
||||
| `apps/coordinator-api/src/app/storage/db_pg.py` | 206 | `print("✅ PostgreSQL database initialized successfully!")` |
|
||||
|
||||
---
|
||||
|
||||
## Summary Statistics
|
||||
|
||||
| Category | Count | Action |
|
||||
|----------|-------|--------|
|
||||
| **Whitelist ✅** | ~60 items | Keep and maintain |
|
||||
| **Greylist ⚠️** | 0 items | All resolved! |
|
||||
| **Placeholders 📋** | 12 folders | Fill per roadmap Stage 19 |
|
||||
| **Greylist ⚠️** | 0 items | All resolved |
|
||||
| **Placeholders 📋** | 12 folders | Fill per roadmap |
|
||||
| **Removed ❌** | 35 items | Cleaned up 2026-01-24 |
|
||||
| **Empty dirs** | 8 dirs | Delete |
|
||||
| **Misplaced files** | 8 files | Move or gitignore |
|
||||
| **Legacy files** | 18 files | Delete |
|
||||
| **Debug prints** | 17 statements | Replace with logger |
|
||||
|
||||
### Completed Actions (2026-01-24)
|
||||
|
||||
|
||||
265
docs/structure.md
Normal file
265
docs/structure.md
Normal file
@@ -0,0 +1,265 @@
|
||||
# AITBC Codebase Structure
|
||||
|
||||
> Monorepo layout for the AI Token Blockchain platform.
|
||||
|
||||
## Top-Level Overview
|
||||
|
||||
```
|
||||
aitbc/
|
||||
├── apps/ # Core microservices and web applications
|
||||
├── assets/ # Shared frontend assets (CSS, JS, fonts)
|
||||
├── cli/ # Command-line interface tools
|
||||
├── contracts/ # Solidity smart contracts (standalone)
|
||||
├── dev-utils/ # Developer utilities and path configs
|
||||
├── docs/ # Markdown documentation
|
||||
├── examples/ # Usage examples and demos
|
||||
├── extensions/ # Browser extensions (Firefox wallet)
|
||||
├── home/ # Local workflow scripts (client/miner simulation)
|
||||
├── infra/ # Infrastructure configs (nginx, k8s, terraform, helm)
|
||||
├── packages/ # Shared libraries and SDKs
|
||||
├── plugins/ # Plugin integrations (Ollama)
|
||||
├── protocols/ # Protocol specs and sample data
|
||||
├── scripts/ # Operational and deployment scripts
|
||||
├── src/ # Shared Python source (cross-site sync, RPC)
|
||||
├── systemd/ # Systemd service unit files
|
||||
├── tests/ # Pytest test suites (unit, integration, e2e, security, load)
|
||||
├── website/ # Public-facing website and HTML documentation
|
||||
├── .gitignore
|
||||
├── .editorconfig
|
||||
├── LICENSE # MIT License
|
||||
├── pyproject.toml # Python project configuration
|
||||
├── pytest.ini # Pytest settings and markers
|
||||
└── README.md
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## apps/ — Core Applications
|
||||
|
||||
### blockchain-node
|
||||
Full blockchain node implementation with PoA consensus, gossip relay, mempool, RPC API, WebSocket support, and observability dashboards.
|
||||
|
||||
```
|
||||
apps/blockchain-node/
|
||||
├── src/aitbc_chain/
|
||||
│ ├── app.py # FastAPI application
|
||||
│ ├── main.py # Entry point
|
||||
│ ├── config.py # Node configuration
|
||||
│ ├── database.py # Chain storage
|
||||
│ ├── models.py # Block/Transaction models
|
||||
│ ├── mempool.py # Transaction mempool
|
||||
│ ├── metrics.py # Prometheus metrics
|
||||
│ ├── logger.py # Structured logging
|
||||
│ ├── consensus/poa.py # Proof-of-Authority consensus
|
||||
│ ├── gossip/ # P2P gossip protocol (broker, relay)
|
||||
│ ├── observability/ # Dashboards and exporters
|
||||
│ └── rpc/ # JSON-RPC router and WebSocket
|
||||
├── scripts/ # Genesis creation, key generation, benchmarks
|
||||
├── tests/ # Unit tests (models, gossip, WebSocket, observability)
|
||||
└── pyproject.toml
|
||||
```
|
||||
|
||||
### coordinator-api
|
||||
Central job coordination API with marketplace, payments, ZK proofs, multi-tenancy, and governance.
|
||||
|
||||
```
|
||||
apps/coordinator-api/
|
||||
├── src/app/
|
||||
│ ├── main.py # FastAPI entry point
|
||||
│ ├── config.py # Configuration
|
||||
│ ├── database.py # Database setup
|
||||
│ ├── deps.py # Dependency injection
|
||||
│ ├── exceptions.py # Custom exceptions
|
||||
│ ├── logging.py # Logging config
|
||||
│ ├── metrics.py # Prometheus metrics
|
||||
│ ├── domain/ # Domain models (job, miner, payment, user, marketplace)
|
||||
│ ├── models/ # DB models (registry, confidential, multitenant, services)
|
||||
│ ├── routers/ # API endpoints (admin, client, miner, marketplace, payments, governance, exchange, explorer, ZK)
|
||||
│ ├── services/ # Business logic (jobs, miners, payments, receipts, ZK proofs, encryption, HSM, blockchain, bitcoin wallet)
|
||||
│ ├── storage/ # Database adapters (SQLite, PostgreSQL)
|
||||
│ ├── middleware/ # Tenant context middleware
|
||||
│ ├── repositories/ # Data access layer
|
||||
│ └── schemas/ # Pydantic schemas
|
||||
├── aitbc/settlement/ # Cross-chain settlement (LayerZero bridge)
|
||||
├── migrations/ # SQL migrations (schema, indexes, data, payments)
|
||||
├── scripts/ # PostgreSQL migration scripts
|
||||
├── tests/ # API tests (jobs, marketplace, ZK, receipts, miners)
|
||||
└── pyproject.toml
|
||||
```
|
||||
|
||||
### explorer-web
|
||||
Blockchain explorer SPA built with TypeScript and Vite.
|
||||
|
||||
```
|
||||
apps/explorer-web/
|
||||
├── src/
|
||||
│ ├── main.ts # Application entry
|
||||
│ ├── config.ts # API configuration
|
||||
│ ├── components/ # UI components (header, footer, data mode toggle, notifications)
|
||||
│ ├── lib/ # Data models and mock data
|
||||
│ └── pages/ # Page views (overview, blocks, transactions, addresses, receipts)
|
||||
├── public/ # Static assets (CSS themes, mock JSON data)
|
||||
├── tests/e2e/ # Playwright end-to-end tests
|
||||
├── vite.config.ts
|
||||
└── tsconfig.json
|
||||
```
|
||||
|
||||
### marketplace-web
|
||||
GPU compute marketplace frontend built with TypeScript and Vite.
|
||||
|
||||
```
|
||||
apps/marketplace-web/
|
||||
├── src/
|
||||
│ ├── main.ts # Application entry
|
||||
│ ├── lib/ # API client and auth
|
||||
│ └── style.css # Styles
|
||||
├── public/ # Mock data (offers, stats)
|
||||
├── vite.config.ts
|
||||
└── tsconfig.json
|
||||
```
|
||||
|
||||
### wallet-daemon
|
||||
Wallet service with receipt verification and ledger management.
|
||||
|
||||
```
|
||||
apps/wallet-daemon/
|
||||
├── src/app/
|
||||
│ ├── main.py # FastAPI entry point
|
||||
│ ├── settings.py # Configuration
|
||||
│ ├── ledger_mock/ # Mock ledger with PostgreSQL adapter
|
||||
│ └── receipts/ # Receipt verification service
|
||||
├── scripts/ # PostgreSQL migration
|
||||
├── tests/ # Wallet API and receipt tests
|
||||
└── pyproject.toml
|
||||
```
|
||||
|
||||
### trade-exchange
|
||||
Bitcoin/AITBC trading exchange with order book, price ticker, and admin panel.
|
||||
|
||||
```
|
||||
apps/trade-exchange/
|
||||
├── server.py # WebSocket price server
|
||||
├── simple_exchange_api.py # Exchange REST API (SQLite)
|
||||
├── simple_exchange_api_pg.py # Exchange REST API (PostgreSQL)
|
||||
├── exchange_api.py # Full exchange API
|
||||
├── bitcoin-wallet.py # Bitcoin wallet integration
|
||||
├── database.py # Database layer
|
||||
├── build.py # Production build script
|
||||
├── index.html # Exchange frontend
|
||||
├── admin.html # Admin panel
|
||||
└── scripts/ # PostgreSQL migration
|
||||
```
|
||||
|
||||
### pool-hub
|
||||
Mining pool management with job matching, miner scoring, and Redis caching.
|
||||
|
||||
```
|
||||
apps/pool-hub/
|
||||
├── src/
|
||||
│ ├── app/ # Legacy app structure (routers, registry, scoring)
|
||||
│ └── poolhub/ # Current app (routers, models, repositories, services, Redis)
|
||||
├── migrations/ # Alembic migrations
|
||||
└── tests/ # API and repository tests
|
||||
```
|
||||
|
||||
### zk-circuits
|
||||
Zero-knowledge proof circuits for receipt verification.
|
||||
|
||||
```
|
||||
apps/zk-circuits/
|
||||
├── circuits/receipt.circom # Circom circuit definition
|
||||
├── generate_proof.js # Proof generation
|
||||
├── test.js # Circuit tests
|
||||
└── benchmark.js # Performance benchmarks
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## packages/ — Shared Libraries
|
||||
|
||||
```
|
||||
packages/
|
||||
├── py/
|
||||
│ ├── aitbc-crypto/ # Cryptographic primitives (signing, hashing, key derivation)
|
||||
│ └── aitbc-sdk/ # Python SDK for coordinator API (receipt fetching/verification)
|
||||
└── solidity/
|
||||
└── aitbc-token/ # ERC-20 AITBC token contract with Hardhat tooling
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## scripts/ — Operations
|
||||
|
||||
```
|
||||
scripts/
|
||||
├── aitbc-cli.sh # Main CLI entry point
|
||||
├── deploy/ # Deployment scripts (container, remote, blockchain, explorer, exchange, nginx)
|
||||
├── gpu/ # GPU miner management (host miner, registry, exchange integration)
|
||||
├── service/ # Service lifecycle (start, stop, diagnose, fix)
|
||||
├── testing/ # Test runners and verification scripts
|
||||
├── test/ # Individual test scripts (coordinator, GPU, explorer)
|
||||
├── ci/ # CI pipeline scripts
|
||||
├── ops/ # Operational scripts (systemd install)
|
||||
└── dev/ # Development tools (WebSocket load test)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## infra/ — Infrastructure
|
||||
|
||||
```
|
||||
infra/
|
||||
├── nginx/ # Nginx configs (reverse proxy, local, production)
|
||||
├── k8s/ # Kubernetes manifests (backup, cert-manager, network policies, sealed secrets)
|
||||
├── helm/ # Helm charts (coordinator deployment, values per environment)
|
||||
├── terraform/ # Terraform modules (Kubernetes cluster, environments: dev/staging/prod)
|
||||
└── scripts/ # Infra scripts (backup, restore, chaos testing)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## tests/ — Test Suites
|
||||
|
||||
```
|
||||
tests/
|
||||
├── unit/ # Unit tests (blockchain node, coordinator API, wallet daemon)
|
||||
├── integration/ # Integration tests (blockchain node, full workflow)
|
||||
├── e2e/ # End-to-end tests (user scenarios, wallet daemon)
|
||||
├── security/ # Security tests (confidential transactions, comprehensive audit)
|
||||
├── load/ # Load tests (Locust)
|
||||
├── conftest.py # Shared pytest fixtures
|
||||
└── test_blockchain_nodes.py # Live node connectivity tests
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## website/ — Public Website
|
||||
|
||||
```
|
||||
website/
|
||||
├── index.html # Landing page
|
||||
├── 404.html # Error page
|
||||
├── docs/ # HTML documentation (per-component pages, CSS, JS)
|
||||
├── dashboards/ # Admin and miner dashboards
|
||||
├── BrowserWallet/ # Browser wallet interface
|
||||
├── extensions/ # Packaged browser extensions (.zip, .xpi)
|
||||
└── aitbc-proxy.conf # Nginx proxy config for website
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Other Directories
|
||||
|
||||
| Directory | Purpose |
|
||||
|-----------|---------|
|
||||
| `cli/` | CLI tools for client, miner, wallet operations and GPU testing |
|
||||
| `plugins/ollama/` | Ollama LLM integration (client plugin, miner plugin, service layer) |
|
||||
| `home/` | Local simulation scripts for client/miner workflows |
|
||||
| `extensions/` | Firefox wallet extension source code |
|
||||
| `contracts/` | Standalone Solidity contracts (ZKReceiptVerifier) |
|
||||
| `protocols/` | Protocol sample data (signed receipts) |
|
||||
| `src/` | Shared Python modules (cross-site sync, RPC router) |
|
||||
| `systemd/` | Systemd unit files for all AITBC services |
|
||||
| `dev-utils/` | Developer utilities (Python path config) |
|
||||
| `docs/` | Markdown documentation (guides, reports, reference, tutorials, operator docs) |
|
||||
| `assets/` | Shared frontend assets (Tailwind CSS, FontAwesome, Lucide icons, Axios) |
|
||||
@@ -1,457 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import hashlib
|
||||
import json
|
||||
import time
|
||||
from datetime import datetime
|
||||
from typing import Any, Dict, Optional, List
|
||||
|
||||
from fastapi import APIRouter, HTTPException, status
|
||||
from pydantic import BaseModel, Field, model_validator
|
||||
from sqlmodel import select
|
||||
|
||||
from ..config import settings
|
||||
from ..database import session_scope
|
||||
from ..gossip import gossip_broker
|
||||
from ..mempool import get_mempool
|
||||
from ..metrics import metrics_registry
|
||||
from ..models import Account, Block, Receipt, Transaction
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
def _serialize_receipt(receipt: Receipt) -> Dict[str, Any]:
|
||||
return {
|
||||
"receipt_id": receipt.receipt_id,
|
||||
"job_id": receipt.job_id,
|
||||
"payload": receipt.payload,
|
||||
"miner_signature": receipt.miner_signature,
|
||||
"coordinator_attestations": receipt.coordinator_attestations,
|
||||
"minted_amount": receipt.minted_amount,
|
||||
"recorded_at": receipt.recorded_at.isoformat(),
|
||||
}
|
||||
|
||||
|
||||
class TransactionRequest(BaseModel):
|
||||
type: str = Field(description="Transaction type, e.g. TRANSFER or RECEIPT_CLAIM")
|
||||
sender: str
|
||||
nonce: int
|
||||
fee: int = Field(ge=0)
|
||||
payload: Dict[str, Any]
|
||||
sig: Optional[str] = Field(default=None, description="Signature payload")
|
||||
|
||||
@model_validator(mode="after")
|
||||
def normalize_type(self) -> "TransactionRequest": # type: ignore[override]
|
||||
normalized = self.type.upper()
|
||||
if normalized not in {"TRANSFER", "RECEIPT_CLAIM"}:
|
||||
raise ValueError(f"unsupported transaction type: {self.type}")
|
||||
self.type = normalized
|
||||
return self
|
||||
|
||||
|
||||
class ReceiptSubmissionRequest(BaseModel):
|
||||
sender: str
|
||||
nonce: int
|
||||
fee: int = Field(ge=0)
|
||||
payload: Dict[str, Any]
|
||||
sig: Optional[str] = None
|
||||
|
||||
|
||||
class EstimateFeeRequest(BaseModel):
|
||||
type: Optional[str] = None
|
||||
payload: Dict[str, Any] = Field(default_factory=dict)
|
||||
|
||||
|
||||
class MintFaucetRequest(BaseModel):
|
||||
address: str
|
||||
amount: int = Field(gt=0)
|
||||
|
||||
|
||||
@router.get("/head", summary="Get current chain head")
|
||||
async def get_head() -> Dict[str, Any]:
|
||||
metrics_registry.increment("rpc_get_head_total")
|
||||
start = time.perf_counter()
|
||||
with session_scope() as session:
|
||||
result = session.exec(select(Block).order_by(Block.height.desc()).limit(1)).first()
|
||||
if result is None:
|
||||
metrics_registry.increment("rpc_get_head_not_found_total")
|
||||
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="no blocks yet")
|
||||
metrics_registry.increment("rpc_get_head_success_total")
|
||||
metrics_registry.observe("rpc_get_head_duration_seconds", time.perf_counter() - start)
|
||||
return {
|
||||
"height": result.height,
|
||||
"hash": result.hash,
|
||||
"timestamp": result.timestamp.isoformat(),
|
||||
"tx_count": result.tx_count,
|
||||
}
|
||||
|
||||
|
||||
@router.get("/blocks/{height}", summary="Get block by height")
|
||||
async def get_block(height: int) -> Dict[str, Any]:
|
||||
metrics_registry.increment("rpc_get_block_total")
|
||||
start = time.perf_counter()
|
||||
with session_scope() as session:
|
||||
block = session.exec(select(Block).where(Block.height == height)).first()
|
||||
if block is None:
|
||||
metrics_registry.increment("rpc_get_block_not_found_total")
|
||||
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="block not found")
|
||||
metrics_registry.increment("rpc_get_block_success_total")
|
||||
metrics_registry.observe("rpc_get_block_duration_seconds", time.perf_counter() - start)
|
||||
return {
|
||||
"proposer": block.proposer,
|
||||
"proposer": block.proposer,
|
||||
"height": block.height,
|
||||
"hash": block.hash,
|
||||
"parent_hash": block.parent_hash,
|
||||
"timestamp": block.timestamp.isoformat(),
|
||||
"tx_count": block.tx_count,
|
||||
"state_root": block.state_root,
|
||||
}
|
||||
|
||||
|
||||
@router.get("/blocks-range", summary="Get blocks in range")
|
||||
async def get_blocks_range(start_height: int = 0, end_height: int = 100, limit: int = 1000) -> List[Dict[str, Any]]:
|
||||
metrics_registry.increment("rpc_get_blocks_range_total")
|
||||
start = time.perf_counter()
|
||||
|
||||
# Validate parameters
|
||||
if limit > 10000:
|
||||
limit = 10000
|
||||
if end_height - start_height > limit:
|
||||
end_height = start_height + limit
|
||||
|
||||
with session_scope() as session:
|
||||
stmt = (
|
||||
select(Block)
|
||||
.where(Block.height >= start_height)
|
||||
.where(Block.height <= end_height)
|
||||
.order_by(Block.height)
|
||||
)
|
||||
blocks = session.exec(stmt).all()
|
||||
|
||||
metrics_registry.observe("rpc_get_blocks_range_duration_seconds", time.perf_counter() - start)
|
||||
return [
|
||||
{
|
||||
"proposer": block.proposer,
|
||||
"proposer": block.proposer,
|
||||
"height": block.height,
|
||||
"hash": block.hash,
|
||||
"parent_hash": block.parent_hash,
|
||||
"timestamp": block.timestamp.isoformat(),
|
||||
"tx_count": block.tx_count,
|
||||
"state_root": block.state_root,
|
||||
}
|
||||
for block in blocks
|
||||
]
|
||||
|
||||
@router.get("/tx/{tx_hash}", summary="Get transaction by hash")
|
||||
async def get_transaction(tx_hash: str) -> Dict[str, Any]:
|
||||
metrics_registry.increment("rpc_get_transaction_total")
|
||||
start = time.perf_counter()
|
||||
with session_scope() as session:
|
||||
tx = session.exec(select(Transaction).where(Transaction.tx_hash == tx_hash)).first()
|
||||
if tx is None:
|
||||
metrics_registry.increment("rpc_get_transaction_not_found_total")
|
||||
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="transaction not found")
|
||||
metrics_registry.increment("rpc_get_transaction_success_total")
|
||||
metrics_registry.observe("rpc_get_transaction_duration_seconds", time.perf_counter() - start)
|
||||
return {
|
||||
"tx_hash": tx.tx_hash,
|
||||
"block_height": tx.block_height,
|
||||
"sender": tx.sender,
|
||||
"recipient": tx.recipient,
|
||||
"payload": tx.payload,
|
||||
"created_at": tx.created_at.isoformat(),
|
||||
}
|
||||
|
||||
|
||||
@router.get("/receipts/{receipt_id}", summary="Get receipt by ID")
|
||||
async def get_receipt(receipt_id: str) -> Dict[str, Any]:
|
||||
metrics_registry.increment("rpc_get_receipt_total")
|
||||
start = time.perf_counter()
|
||||
with session_scope() as session:
|
||||
receipt = session.exec(select(Receipt).where(Receipt.receipt_id == receipt_id)).first()
|
||||
if receipt is None:
|
||||
metrics_registry.increment("rpc_get_receipt_not_found_total")
|
||||
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="receipt not found")
|
||||
metrics_registry.increment("rpc_get_receipt_success_total")
|
||||
metrics_registry.observe("rpc_get_receipt_duration_seconds", time.perf_counter() - start)
|
||||
return _serialize_receipt(receipt)
|
||||
|
||||
|
||||
@router.get("/getBalance/{address}", summary="Get account balance")
|
||||
async def get_balance(address: str) -> Dict[str, Any]:
|
||||
metrics_registry.increment("rpc_get_balance_total")
|
||||
start = time.perf_counter()
|
||||
with session_scope() as session:
|
||||
account = session.get(Account, address)
|
||||
if account is None:
|
||||
metrics_registry.increment("rpc_get_balance_empty_total")
|
||||
metrics_registry.observe("rpc_get_balance_duration_seconds", time.perf_counter() - start)
|
||||
return {"address": address, "balance": 0, "nonce": 0}
|
||||
metrics_registry.increment("rpc_get_balance_success_total")
|
||||
metrics_registry.observe("rpc_get_balance_duration_seconds", time.perf_counter() - start)
|
||||
return {
|
||||
"address": account.address,
|
||||
"balance": account.balance,
|
||||
"nonce": account.nonce,
|
||||
"updated_at": account.updated_at.isoformat(),
|
||||
}
|
||||
|
||||
|
||||
@router.post("/sendTx", summary="Submit a new transaction")
|
||||
async def send_transaction(request: TransactionRequest) -> Dict[str, Any]:
|
||||
metrics_registry.increment("rpc_send_tx_total")
|
||||
start = time.perf_counter()
|
||||
mempool = get_mempool()
|
||||
tx_dict = request.model_dump()
|
||||
tx_hash = mempool.add(tx_dict)
|
||||
try:
|
||||
asyncio.create_task(
|
||||
gossip_broker.publish(
|
||||
"transactions",
|
||||
{
|
||||
"tx_hash": tx_hash,
|
||||
"sender": request.sender,
|
||||
"payload": request.payload,
|
||||
"nonce": request.nonce,
|
||||
"fee": request.fee,
|
||||
"type": request.type,
|
||||
},
|
||||
)
|
||||
)
|
||||
metrics_registry.increment("rpc_send_tx_success_total")
|
||||
return {"tx_hash": tx_hash}
|
||||
except Exception:
|
||||
metrics_registry.increment("rpc_send_tx_failed_total")
|
||||
raise
|
||||
finally:
|
||||
metrics_registry.observe("rpc_send_tx_duration_seconds", time.perf_counter() - start)
|
||||
|
||||
|
||||
@router.post("/submitReceipt", summary="Submit receipt claim transaction")
|
||||
async def submit_receipt(request: ReceiptSubmissionRequest) -> Dict[str, Any]:
|
||||
metrics_registry.increment("rpc_submit_receipt_total")
|
||||
start = time.perf_counter()
|
||||
tx_payload = {
|
||||
"type": "RECEIPT_CLAIM",
|
||||
"sender": request.sender,
|
||||
"nonce": request.nonce,
|
||||
"fee": request.fee,
|
||||
"payload": request.payload,
|
||||
"sig": request.sig,
|
||||
}
|
||||
tx_request = TransactionRequest.model_validate(tx_payload)
|
||||
try:
|
||||
response = await send_transaction(tx_request)
|
||||
metrics_registry.increment("rpc_submit_receipt_success_total")
|
||||
return response
|
||||
except HTTPException:
|
||||
metrics_registry.increment("rpc_submit_receipt_failed_total")
|
||||
raise
|
||||
except Exception:
|
||||
metrics_registry.increment("rpc_submit_receipt_failed_total")
|
||||
raise
|
||||
finally:
|
||||
metrics_registry.observe("rpc_submit_receipt_duration_seconds", time.perf_counter() - start)
|
||||
|
||||
|
||||
@router.post("/estimateFee", summary="Estimate transaction fee")
|
||||
async def estimate_fee(request: EstimateFeeRequest) -> Dict[str, Any]:
|
||||
metrics_registry.increment("rpc_estimate_fee_total")
|
||||
start = time.perf_counter()
|
||||
base_fee = 10
|
||||
per_byte = 1
|
||||
payload_bytes = len(json.dumps(request.payload, sort_keys=True, separators=(",", ":")).encode())
|
||||
estimated_fee = base_fee + per_byte * payload_bytes
|
||||
tx_type = (request.type or "TRANSFER").upper()
|
||||
metrics_registry.increment("rpc_estimate_fee_success_total")
|
||||
metrics_registry.observe("rpc_estimate_fee_duration_seconds", time.perf_counter() - start)
|
||||
return {
|
||||
"type": tx_type,
|
||||
"base_fee": base_fee,
|
||||
"payload_bytes": payload_bytes,
|
||||
"estimated_fee": estimated_fee,
|
||||
}
|
||||
|
||||
|
||||
@router.post("/admin/mintFaucet", summary="Mint devnet funds to an address")
|
||||
async def mint_faucet(request: MintFaucetRequest) -> Dict[str, Any]:
|
||||
metrics_registry.increment("rpc_mint_faucet_total")
|
||||
start = time.perf_counter()
|
||||
with session_scope() as session:
|
||||
account = session.get(Account, request.address)
|
||||
if account is None:
|
||||
account = Account(address=request.address, balance=request.amount)
|
||||
session.add(account)
|
||||
else:
|
||||
account.balance += request.amount
|
||||
session.commit()
|
||||
updated_balance = account.balance
|
||||
metrics_registry.increment("rpc_mint_faucet_success_total")
|
||||
metrics_registry.observe("rpc_mint_faucet_duration_seconds", time.perf_counter() - start)
|
||||
return {"address": request.address, "balance": updated_balance}
|
||||
|
||||
|
||||
class TransactionData(BaseModel):
|
||||
tx_hash: str
|
||||
sender: str
|
||||
recipient: str
|
||||
payload: Dict[str, Any] = Field(default_factory=dict)
|
||||
|
||||
class ReceiptData(BaseModel):
|
||||
receipt_id: str
|
||||
job_id: str
|
||||
payload: Dict[str, Any] = Field(default_factory=dict)
|
||||
miner_signature: Optional[str] = None
|
||||
coordinator_attestations: List[str] = Field(default_factory=list)
|
||||
minted_amount: int = 0
|
||||
recorded_at: str
|
||||
|
||||
class BlockImportRequest(BaseModel):
|
||||
height: int = Field(gt=0)
|
||||
hash: str
|
||||
parent_hash: str
|
||||
proposer: str
|
||||
timestamp: str
|
||||
tx_count: int = Field(ge=0)
|
||||
state_root: Optional[str] = None
|
||||
transactions: List[TransactionData] = Field(default_factory=list)
|
||||
receipts: List[ReceiptData] = Field(default_factory=list)
|
||||
|
||||
|
||||
@router.get("/test-endpoint", summary="Test endpoint")
|
||||
async def test_endpoint() -> Dict[str, str]:
|
||||
"""Test if new code is deployed"""
|
||||
return {"status": "updated_code_running"}
|
||||
|
||||
@router.post("/blocks/import", summary="Import block from remote node")
|
||||
async def import_block(request: BlockImportRequest) -> Dict[str, Any]:
|
||||
"""Import a block from a remote node after validation."""
|
||||
import logging
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
metrics_registry.increment("rpc_import_block_total")
|
||||
start = time.perf_counter()
|
||||
|
||||
try:
|
||||
logger.info(f"Received block import request: height={request.height}, hash={request.hash}")
|
||||
logger.info(f"Transactions count: {len(request.transactions)}")
|
||||
if request.transactions:
|
||||
logger.info(f"First transaction: {request.transactions[0]}")
|
||||
|
||||
with session_scope() as session:
|
||||
# Check if block already exists
|
||||
existing = session.exec(select(Block).where(Block.height == request.height)).first()
|
||||
if existing:
|
||||
if existing.hash == request.hash:
|
||||
metrics_registry.increment("rpc_import_block_exists_total")
|
||||
return {"status": "exists", "height": request.height, "hash": request.hash}
|
||||
else:
|
||||
metrics_registry.increment("rpc_import_block_conflict_total")
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_409_CONFLICT,
|
||||
detail=f"Block at height {request.height} already exists with different hash"
|
||||
)
|
||||
|
||||
# Check if parent block exists
|
||||
if request.height > 0:
|
||||
parent = session.exec(select(Block).where(Block.hash == request.parent_hash)).first()
|
||||
if not parent:
|
||||
metrics_registry.increment("rpc_import_block_orphan_total")
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_400_BAD_REQUEST,
|
||||
detail="Parent block not found"
|
||||
)
|
||||
|
||||
# Validate block hash using the same algorithm as PoA proposer
|
||||
payload = f"{settings.chain_id}|{request.height}|{request.parent_hash}|{request.timestamp}".encode()
|
||||
expected_hash = "0x" + hashlib.sha256(payload).hexdigest()
|
||||
|
||||
if request.hash != expected_hash:
|
||||
metrics_registry.increment("rpc_import_block_invalid_hash_total")
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_400_BAD_REQUEST,
|
||||
detail=f"Invalid block hash. Expected: {expected_hash}, Got: {request.hash}"
|
||||
)
|
||||
|
||||
# Create and save block
|
||||
block_timestamp = datetime.fromisoformat(request.timestamp.replace('Z', '+00:00'))
|
||||
|
||||
block = Block(
|
||||
height=request.height,
|
||||
hash=request.hash,
|
||||
parent_hash=request.parent_hash,
|
||||
proposer=request.proposer,
|
||||
timestamp=block_timestamp,
|
||||
tx_count=request.tx_count,
|
||||
state_root=request.state_root
|
||||
)
|
||||
|
||||
session.add(block)
|
||||
session.flush() # Get block ID
|
||||
|
||||
# Add transactions if provided
|
||||
for tx_data in request.transactions:
|
||||
# Create transaction using constructor with all fields
|
||||
tx = Transaction(
|
||||
tx_hash=str(tx_data.tx_hash),
|
||||
block_height=block.id, # Use block.id instead of block.height for foreign key
|
||||
sender=str(tx_data.sender),
|
||||
recipient=str(tx_data.recipient),
|
||||
payload=tx_data.payload if tx_data.payload else {},
|
||||
created_at=datetime.utcnow()
|
||||
)
|
||||
|
||||
session.add(tx)
|
||||
|
||||
# Add receipts if provided
|
||||
for receipt_data in request.receipts:
|
||||
receipt = Receipt(
|
||||
block_height=block.id, # Use block.id instead of block.height for foreign key
|
||||
receipt_id=receipt_data.receipt_id,
|
||||
job_id=receipt_data.job_id,
|
||||
payload=receipt_data.payload,
|
||||
miner_signature=receipt_data.miner_signature,
|
||||
coordinator_attestations=receipt_data.coordinator_attestations,
|
||||
minted_amount=receipt_data.minted_amount,
|
||||
recorded_at=datetime.fromisoformat(receipt_data.recorded_at.replace('Z', '+00:00'))
|
||||
)
|
||||
session.add(receipt)
|
||||
|
||||
session.commit()
|
||||
|
||||
# Broadcast block via gossip
|
||||
try:
|
||||
gossip_broker.broadcast("blocks", {
|
||||
"type": "block_imported",
|
||||
"height": block.height,
|
||||
"hash": block.hash,
|
||||
"proposer": block.proposer
|
||||
})
|
||||
except Exception:
|
||||
pass # Gossip broadcast is optional
|
||||
|
||||
metrics_registry.increment("rpc_import_block_success_total")
|
||||
metrics_registry.observe("rpc_import_block_duration_seconds", time.perf_counter() - start)
|
||||
|
||||
logger.info(f"Successfully imported block {request.height}")
|
||||
|
||||
return {
|
||||
"status": "imported",
|
||||
"height": block.height,
|
||||
"hash": block.hash,
|
||||
"tx_count": block.tx_count
|
||||
}
|
||||
|
||||
except HTTPException:
|
||||
# Re-raise HTTP exceptions as-is
|
||||
raise
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to import block {request.height}: {str(e)}", exc_info=True)
|
||||
metrics_registry.increment("rpc_import_block_failed_total")
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
|
||||
detail=f"Internal server error: {str(e)}"
|
||||
)
|
||||
@@ -1,7 +0,0 @@
|
||||
"""
|
||||
Cross-site synchronization module for AITBC blockchain.
|
||||
"""
|
||||
|
||||
from .cross_site import CrossSiteSync
|
||||
|
||||
__all__ = ['CrossSiteSync']
|
||||
@@ -1,222 +0,0 @@
|
||||
"""
|
||||
Cross-site RPC synchronization module for AITBC blockchain nodes.
|
||||
Enables block and transaction synchronization across different sites via HTTP RPC endpoints.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
from typing import List, Dict, Optional, Any
|
||||
import httpx
|
||||
from datetime import datetime, timedelta
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class CrossSiteSync:
|
||||
"""Handles synchronization with remote blockchain nodes via RPC."""
|
||||
|
||||
def __init__(self, local_rpc_url: str, remote_endpoints: List[str], poll_interval: int = 10):
|
||||
"""
|
||||
Initialize cross-site synchronization.
|
||||
|
||||
Args:
|
||||
local_rpc_url: URL of local RPC endpoint (e.g., "http://localhost:8082")
|
||||
remote_endpoints: List of remote RPC URLs to sync with
|
||||
poll_interval: Seconds between sync checks
|
||||
"""
|
||||
self.local_rpc_url = local_rpc_url.rstrip('/')
|
||||
self.remote_endpoints = remote_endpoints
|
||||
self.poll_interval = poll_interval
|
||||
self.last_sync = {}
|
||||
self.sync_task = None
|
||||
self.client = httpx.AsyncClient(timeout=5.0)
|
||||
|
||||
async def get_remote_head(self, endpoint: str) -> Optional[Dict[str, Any]]:
|
||||
"""Get the head block from a remote node."""
|
||||
try:
|
||||
response = await self.client.get(f"{endpoint.rstrip('/')}/head")
|
||||
if response.status_code == 200:
|
||||
return response.json()
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to get head from {endpoint}: {e}")
|
||||
return None
|
||||
|
||||
async def get_remote_block(self, endpoint: str, height: int) -> Optional[Dict[str, Any]]:
|
||||
"""Get a specific block from a remote node."""
|
||||
try:
|
||||
response = await self.client.get(f"{endpoint.rstrip('/')}/blocks/{height}")
|
||||
if response.status_code == 200:
|
||||
return response.json()
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to get block {height} from {endpoint}: {e}")
|
||||
return None
|
||||
|
||||
async def get_local_head(self) -> Optional[Dict[str, Any]]:
|
||||
"""Get the local head block."""
|
||||
try:
|
||||
response = await self.client.get(f"{self.local_rpc_url}/rpc/head")
|
||||
if response.status_code == 200:
|
||||
return response.json()
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to get local head: {e}")
|
||||
return None
|
||||
|
||||
async def import_block(self, block_data: Dict[str, Any]) -> bool:
|
||||
"""Import a block from a remote node."""
|
||||
try:
|
||||
response = await self.client.post(
|
||||
f"{self.local_rpc_url}/rpc/blocks/import",
|
||||
json=block_data
|
||||
)
|
||||
if response.status_code == 200:
|
||||
result = response.json()
|
||||
if result.get("status") in ["imported", "exists"]:
|
||||
logger.info(f"Successfully imported block {block_data.get('height')}")
|
||||
return True
|
||||
else:
|
||||
logger.error(f"Block import failed: {result}")
|
||||
return False
|
||||
else:
|
||||
logger.error(f"Block import request failed: {response.status_code} {response.text}")
|
||||
return False
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to import block: {e}")
|
||||
return False
|
||||
|
||||
async def submit_block(self, block_data: Dict[str, Any]) -> bool:
|
||||
"""Submit a block to the local node."""
|
||||
try:
|
||||
response = await self.client.post(
|
||||
f"{self.local_rpc_url}/rpc/block",
|
||||
json=block_data
|
||||
)
|
||||
return response.status_code == 200
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to submit block: {e}")
|
||||
return False
|
||||
|
||||
async def sync_with_remotes(self) -> None:
|
||||
"""Check and sync with all remote endpoints."""
|
||||
local_head = await self.get_local_head()
|
||||
if not local_head:
|
||||
return
|
||||
|
||||
local_height = local_head.get('height', 0)
|
||||
|
||||
for endpoint in self.remote_endpoints:
|
||||
remote_head = await self.get_remote_head(endpoint)
|
||||
if not remote_head:
|
||||
continue
|
||||
|
||||
remote_height = remote_head.get('height', 0)
|
||||
|
||||
# If remote is ahead, fetch missing blocks
|
||||
if remote_height > local_height:
|
||||
logger.info(f"Remote {endpoint} is ahead (height {remote_height} vs {local_height})")
|
||||
|
||||
# Fetch missing blocks one by one
|
||||
for height in range(local_height + 1, remote_height + 1):
|
||||
block = await self.get_remote_block(endpoint, height)
|
||||
if block:
|
||||
# Format block data for import
|
||||
import_data = {
|
||||
"height": block.get("height"),
|
||||
"hash": block.get("hash"),
|
||||
"parent_hash": block.get("parent_hash"),
|
||||
"proposer": block.get("proposer"),
|
||||
"timestamp": block.get("timestamp"),
|
||||
"tx_count": block.get("tx_count", 0),
|
||||
"state_root": block.get("state_root"),
|
||||
"transactions": block.get("transactions", []),
|
||||
"receipts": block.get("receipts", [])
|
||||
}
|
||||
success = await self.import_block(import_data)
|
||||
if success:
|
||||
logger.info(f"Imported block {height} from {endpoint}")
|
||||
local_height = height
|
||||
else:
|
||||
logger.error(f"Failed to import block {height}")
|
||||
break
|
||||
else:
|
||||
logger.error(f"Failed to fetch block {height} from {endpoint}")
|
||||
break
|
||||
|
||||
async def get_remote_mempool(self, endpoint: str) -> List[Dict[str, Any]]:
|
||||
"""Get mempool transactions from a remote node."""
|
||||
try:
|
||||
response = await self.client.get(f"{endpoint.rstrip('/')}/mempool")
|
||||
if response.status_code == 200:
|
||||
return response.json()
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to get mempool from {endpoint}: {e}")
|
||||
return []
|
||||
|
||||
async def get_local_mempool(self) -> List[Dict[str, Any]]:
|
||||
"""Get local mempool transactions."""
|
||||
try:
|
||||
response = await self.client.get(f"{self.local_rpc_url}/rpc/mempool")
|
||||
if response.status_code == 200:
|
||||
return response.json()
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to get local mempool: {e}")
|
||||
return []
|
||||
|
||||
async def submit_transaction(self, tx_data: Dict[str, Any]) -> bool:
|
||||
"""Submit a transaction to the local node."""
|
||||
try:
|
||||
response = await self.client.post(
|
||||
f"{self.local_rpc_url}/rpc/transaction",
|
||||
json=tx_data
|
||||
)
|
||||
return response.status_code == 200
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to submit transaction: {e}")
|
||||
return False
|
||||
|
||||
async def sync_transactions(self) -> None:
|
||||
"""Sync transactions from remote mempools."""
|
||||
local_mempool = await self.get_local_mempool()
|
||||
local_tx_hashes = {tx.get('hash') for tx in local_mempool}
|
||||
|
||||
for endpoint in self.remote_endpoints:
|
||||
remote_mempool = await self.get_remote_mempool(endpoint)
|
||||
for tx in remote_mempool:
|
||||
tx_hash = tx.get('hash')
|
||||
if tx_hash and tx_hash not in local_tx_hashes:
|
||||
success = await self.submit_transaction(tx)
|
||||
if success:
|
||||
logger.info(f"Imported transaction {tx_hash[:8]}... from {endpoint}")
|
||||
|
||||
async def sync_loop(self) -> None:
|
||||
"""Main synchronization loop."""
|
||||
logger.info("Starting cross-site sync loop")
|
||||
|
||||
while True:
|
||||
try:
|
||||
# Sync blocks
|
||||
await self.sync_with_remotes()
|
||||
|
||||
# Sync transactions
|
||||
await self.sync_transactions()
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error in sync loop: {e}")
|
||||
|
||||
await asyncio.sleep(self.poll_interval)
|
||||
|
||||
async def start(self) -> None:
|
||||
"""Start the synchronization task."""
|
||||
if self.sync_task is None:
|
||||
self.sync_task = asyncio.create_task(self.sync_loop())
|
||||
|
||||
async def stop(self) -> None:
|
||||
"""Stop the synchronization task."""
|
||||
if self.sync_task:
|
||||
self.sync_task.cancel()
|
||||
try:
|
||||
await self.sync_task
|
||||
except asyncio.CancelledError:
|
||||
pass
|
||||
self.sync_task = None
|
||||
|
||||
await self.client.aclose()
|
||||
10
website/favicon.svg
Normal file
10
website/favicon.svg
Normal file
@@ -0,0 +1,10 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 64 64">
|
||||
<defs>
|
||||
<linearGradient id="g" x1="0%" y1="0%" x2="100%" y2="100%">
|
||||
<stop offset="0%" style="stop-color:#6366f1"/>
|
||||
<stop offset="100%" style="stop-color:#8b5cf6"/>
|
||||
</linearGradient>
|
||||
</defs>
|
||||
<rect width="64" height="64" rx="14" fill="url(#g)"/>
|
||||
<text x="32" y="44" font-family="system-ui,sans-serif" font-size="32" font-weight="700" fill="white" text-anchor="middle">AI</text>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 471 B |
Reference in New Issue
Block a user