chore: initialize monorepo with project scaffolding, configs, and CI setup
This commit is contained in:
oib
600
docs/bootstrap/aitbc_tech_plan.md
Normal file
600
docs/bootstrap/aitbc_tech_plan.md
Normal file
@ -0,0 +1,600 @@
|
||||
# AITBC – Artificial Intelligence Token Blockchain
|
||||
|
||||
## Overview (Recovered)
|
||||
|
||||
- AITBC couples decentralized blockchain control with asset‑backed value derived from AI computation.
|
||||
- No pre‑mint: tokens are minted by providers **only after** serving compute; prices are set by providers (can be free at bootstrap).
|
||||
|
||||
## Staged Development Roadmap
|
||||
|
||||
### Stage 1: Client–Server Prototype (no blockchain, no hub)
|
||||
|
||||
- Direct client → server API.
|
||||
- API‑key auth; local job logging.
|
||||
- Goal: validate AI service loop and throughput.
|
||||
|
||||
### Stage 2: Blockchain Integration
|
||||
|
||||
- Introduce **AIToken** and minimal smart contracts for minting + accounting.
|
||||
- Mint = amount of compute successfully served; no premint.
|
||||
|
||||
### Stage 3: AI Pool Hub
|
||||
|
||||
- Hub matches requests to multiple servers (sharding/parallelization), verifies outputs, and accounts contributions.
|
||||
- Distributes payments/minted tokens proportionally to work.
|
||||
|
||||
### Stage 4: Marketplace
|
||||
|
||||
- Web DEX/market to buy/sell AITokens; price discovery; reputation and SLAs.
|
||||
|
||||
## System Architecture: Actors
|
||||
|
||||
- **Client** – requests AI jobs (e.g., image/video generation).
|
||||
- **Server/Provider** – runs models (Stable Diffusion, PyTorch, etc.).
|
||||
- **Blockchain Node** – ledger + minting rules.
|
||||
- **AI Pool Hub** – orchestration, metering, payouts.
|
||||
|
||||
## Token Minting Logic (Genesis‑less)
|
||||
|
||||
- No tokens at boot.
|
||||
- Provider advertises price/unit (e.g., 1 AIToken per image or per N GPU‑seconds).
|
||||
- After successful job → provider mints that amount. Free jobs mint 0.
|
||||
|
||||
---
|
||||
|
||||
# Stage 1 – Technischer Implementierungsplan (Detail)
|
||||
|
||||
## Ziele
|
||||
|
||||
- Funktionierender End‑to‑End‑Pfad: Prompt → Inferenz → Ergebnis.
|
||||
- Authentifizierung, Rate‑Limit, Logging.
|
||||
|
||||
## Architektur
|
||||
|
||||
```
|
||||
[ Client ] ⇄ HTTP/JSON ⇄ [ FastAPI AI‑Server (GPU) ]
|
||||
```
|
||||
|
||||
- Server hostet Inferenz-Endpunkte; Client sendet Aufträge.
|
||||
- Optional: WebSocket für Streaming‑Logs/Progress.
|
||||
|
||||
## Technologie‑Stack
|
||||
|
||||
- **Server**: Python 3.10+, FastAPI, Uvicorn, PyTorch, diffusers (Stable Diffusion), PIL.
|
||||
- **Client**: Python CLI (requests / httpx) oder schlankes Web‑UI.
|
||||
- **Persistenz**: SQLite oder JSON Log; Artefakte auf Disk/S3‑ähnlich.
|
||||
- **Sicherheit**: API‑Key (env/secret file), CORS policy, Rate‑Limit (slowapi), timeouts.
|
||||
|
||||
## API‑Spezifikation (v0)
|
||||
|
||||
### POST `/v1/generate-image`
|
||||
|
||||
Request JSON:
|
||||
|
||||
```json
|
||||
{
|
||||
"api_key": "<KEY>",
|
||||
"prompt": "a futuristic city skyline at night",
|
||||
"steps": 30,
|
||||
"guidance": 7.5,
|
||||
"width": 512,
|
||||
"height": 512,
|
||||
"seed": 12345
|
||||
}
|
||||
```
|
||||
|
||||
Response JSON:
|
||||
|
||||
```json
|
||||
{
|
||||
"status": "ok",
|
||||
"job_id": "2025-09-26-000123",
|
||||
"image_base64": "data:image/png;base64,....",
|
||||
"duration_ms": 2180,
|
||||
"gpu_seconds": 1.9
|
||||
}
|
||||
```
|
||||
|
||||
### GET `/v1/health`
|
||||
|
||||
- Rückgabe von `{ "status": "ok", "gpu": "RTX 2060", "model": "SD1.5" }`.
|
||||
|
||||
## Server‑Ablauf (Pseudocode)
|
||||
|
||||
```python
|
||||
@app.post("/v1/generate-image")
|
||||
def gen(req: Request):
|
||||
assert check_api_key(req.api_key)
|
||||
rate_limit(req.key)
|
||||
t0 = now()
|
||||
img = stable_diffusion.generate(prompt=req.prompt, ...)
|
||||
log_job(user=req.key, gpu_seconds=measure_gpu(), ok=True)
|
||||
return {"status":"ok", "image_base64": b64(img), "duration_ms": ms_since(t0)}
|
||||
```
|
||||
|
||||
## Betriebliche Aspekte
|
||||
|
||||
- **Logging**: strukturierte Logs (JSON) inkl. Prompt‑Hash, Laufzeit, GPU‑Sekunden, Exit‑Code.
|
||||
- **Observability**: Prometheus‑/OpenTelemetry‑Metriken (req/sec, p95 Latenz, VRAM‑Nutzung).
|
||||
- **Fehler**: Retry‑Policy (idempotent), Graceful shutdown, Max batch/queue size.
|
||||
- **Sicherheit**: Input‑Sanitization, Upload‑Limits, tmp‑Verzeichnis säubern.
|
||||
|
||||
## Setup‑Schritte (Linux, NVIDIA RTX 2060)
|
||||
|
||||
```bash
|
||||
sudo apt update && sudo apt install -y python3-venv git
|
||||
python3 -m venv venv && source venv/bin/activate
|
||||
pip install fastapi uvicorn[standard] torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
|
||||
pip install diffusers transformers accelerate pillow safetensors xformers slowapi httpx
|
||||
# Start
|
||||
uvicorn app:app --host 0.0.0.0 --port 8000 --workers 1
|
||||
```
|
||||
|
||||
## Akzeptanzkriterien
|
||||
|
||||
- `GET /v1/health` liefert GPU/Model‑Infos.
|
||||
- `POST /v1/generate-image` liefert innerhalb < 5s ein 512×512 PNG (bei RTX 2060, SD‑1.5, \~30 steps).
|
||||
- Logs enthalten pro Job mindestens: job\_id, duration\_ms, gpu\_seconds, bytes\_out.
|
||||
|
||||
## Nächste Schritte zu Stage 2
|
||||
|
||||
- Job‑Quittungsschema definieren (hashbare Receipt für On‑Chain‑Mint später).
|
||||
- Einheit „Compute‑Einheit“ festlegen (z. B. GPU‑Sekunden, Token/Prompt).
|
||||
- Nonce/Signatur im Request zur späteren On‑Chain‑Verifikation.
|
||||
|
||||
---
|
||||
|
||||
## Kurze Stage‑2/3/4‑Vorschau (Implementierungsnotizen)
|
||||
|
||||
- **Stage 2 (Blockchain)**: Smart Contract mit `mint(provider, units, receipt_hash)`, Off‑Chain‑Orakel/Attester.
|
||||
- **Stage 3 (Hub)**: Scheduler (priority, price, reputation), Sharding großer Jobs, Konsistenz‑Checks, Reward‑Split.
|
||||
- **Stage 4 (Marketplace)**: Orderbook, KYC/Compliance Layer (jurisdictions), Custody‑freie Wallet‑Anbindung.
|
||||
|
||||
## Quellen (Auszug)
|
||||
|
||||
- Ethereum Smart Contracts: [https://ethereum.org/en/smart-contracts/](https://ethereum.org/en/smart-contracts/)
|
||||
- PoS Überblick: [https://ethereum.org/en/developers/docs/consensus-mechanisms/pos/](https://ethereum.org/en/developers/docs/consensus-mechanisms/pos/)
|
||||
- PyTorch Deploy: [https://pytorch.org/tutorials/](https://pytorch.org/tutorials/)
|
||||
- FastAPI Docs: [https://fastapi.tiangolo.com/](https://fastapi.tiangolo.com/)
|
||||
|
||||
---
|
||||
|
||||
# Stage 1 – Referenz‑Implementierung (Code)
|
||||
|
||||
## `.env`
|
||||
|
||||
```
|
||||
API_KEY=CHANGE_ME_SUPERSECRET
|
||||
MODEL_ID=runwayml/stable-diffusion-v1-5
|
||||
BIND_HOST=0.0.0.0
|
||||
BIND_PORT=8000
|
||||
```
|
||||
|
||||
## `requirements.txt`
|
||||
|
||||
```
|
||||
fastapi
|
||||
uvicorn[standard]
|
||||
httpx
|
||||
pydantic
|
||||
python-dotenv
|
||||
slowapi
|
||||
pillow
|
||||
torch
|
||||
torchvision
|
||||
torchaudio
|
||||
transformers
|
||||
diffusers
|
||||
accelerate
|
||||
safetensors
|
||||
xformers
|
||||
```
|
||||
|
||||
## `server.py`
|
||||
|
||||
```python
|
||||
import base64, io, os, time, hashlib
|
||||
from functools import lru_cache
|
||||
from typing import Optional
|
||||
from dotenv import load_dotenv
|
||||
from fastapi import FastAPI, HTTPException, Request
|
||||
from pydantic import BaseModel, Field
|
||||
from slowapi import Limiter
|
||||
from slowapi.util import get_remote_address
|
||||
from PIL import Image
|
||||
|
||||
load_dotenv()
|
||||
API_KEY = os.getenv("API_KEY", "CHANGE_ME_SUPERSECRET")
|
||||
MODEL_ID = os.getenv("MODEL_ID", "runwayml/stable-diffusion-v1-5")
|
||||
|
||||
app = FastAPI(title="AITBC Stage1 Server", version="0.1.0")
|
||||
limiter = Limiter(key_func=get_remote_address)
|
||||
|
||||
class GenRequest(BaseModel):
|
||||
api_key: str
|
||||
prompt: str
|
||||
steps: int = Field(30, ge=5, le=100)
|
||||
guidance: float = Field(7.5, ge=0, le=25)
|
||||
width: int = Field(512, ge=256, le=1024)
|
||||
height: int = Field(512, ge=256, le=1024)
|
||||
seed: Optional[int] = None
|
||||
|
||||
@lru_cache(maxsize=1)
|
||||
def load_pipeline():
|
||||
from diffusers import StableDiffusionPipeline
|
||||
import torch
|
||||
pipe = StableDiffusionPipeline.from_pretrained(MODEL_ID, torch_dtype=torch.float16, safety_checker=None)
|
||||
pipe = pipe.to("cuda" if torch.cuda.is_available() else "cpu")
|
||||
pipe.enable_attention_slicing()
|
||||
return pipe
|
||||
|
||||
@app.get("/v1/health")
|
||||
def health():
|
||||
gpu = os.getenv("NVIDIA_VISIBLE_DEVICES", "auto")
|
||||
return {"status": "ok", "gpu": gpu, "model": MODEL_ID}
|
||||
|
||||
@app.post("/v1/generate-image")
|
||||
@limiter.limit("10/minute")
|
||||
def generate(req: GenRequest, request: Request):
|
||||
if req.api_key != API_KEY:
|
||||
raise HTTPException(status_code=401, detail="invalid api_key")
|
||||
t0 = time.time()
|
||||
pipe = load_pipeline()
|
||||
generator = None
|
||||
if req.seed is not None:
|
||||
import torch
|
||||
generator = torch.Generator(device=pipe.device).manual_seed(int(req.seed))
|
||||
result = pipe(req.prompt, num_inference_steps=req.steps, guidance_scale=req.guidance, width=req.width, height=req.height, generator=generator)
|
||||
img: Image.Image = result.images[0]
|
||||
buf = io.BytesIO()
|
||||
img.save(buf, format="PNG")
|
||||
b64 = base64.b64encode(buf.getvalue()).decode("ascii")
|
||||
dur_ms = int((time.time() - t0) * 1000)
|
||||
job_id = hashlib.sha256(f"{t0}-{req.prompt[:64]}".encode()).hexdigest()[:16]
|
||||
log_line = {"job_id": job_id, "duration_ms": dur_ms, "bytes_out": len(b64), "prompt_hash": hashlib.sha256(req.prompt.encode()).hexdigest()}
|
||||
print(log_line, flush=True)
|
||||
return {"status": "ok", "job_id": job_id, "image_base64": f"data:image/png;base64,{b64}", "duration_ms": dur_ms}
|
||||
|
||||
if __name__ == "__main__":
|
||||
import uvicorn, os
|
||||
uvicorn.run("server:app", host=os.getenv("BIND_HOST", "0.0.0.0"), port=int(os.getenv("BIND_PORT", "8000")), reload=False)
|
||||
```
|
||||
|
||||
## `client.py`
|
||||
|
||||
```python
|
||||
import base64, json, os
|
||||
import httpx
|
||||
|
||||
API = os.getenv("API", "http://localhost:8000")
|
||||
API_KEY = os.getenv("API_KEY", "CHANGE_ME_SUPERSECRET")
|
||||
|
||||
payload = {
|
||||
"api_key": API_KEY,
|
||||
"prompt": "a futuristic city skyline at night, ultra detailed, neon",
|
||||
"steps": 30,
|
||||
"guidance": 7.5,
|
||||
"width": 512,
|
||||
"height": 512,
|
||||
}
|
||||
|
||||
r = httpx.post(f"{API}/v1/generate-image", json=payload, timeout=120)
|
||||
r.raise_for_status()
|
||||
resp = r.json()
|
||||
print("job:", resp.get("job_id"), "duration_ms:", resp.get("duration_ms"))
|
||||
img_b64 = resp["image_base64"].split(",",1)[1]
|
||||
open("out.png","wb").write(base64.b64decode(img_b64))
|
||||
print("saved out.png")
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
# OpenAPI 3.1 Spezifikation (Stage 1)
|
||||
|
||||
```yaml
|
||||
openapi: 3.1.0
|
||||
info:
|
||||
title: AITBC Stage1 Server
|
||||
version: 0.1.0
|
||||
servers:
|
||||
- url: http://localhost:8000
|
||||
paths:
|
||||
/v1/health:
|
||||
get:
|
||||
summary: Health check
|
||||
responses:
|
||||
'200':
|
||||
description: OK
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
properties:
|
||||
status: { type: string }
|
||||
gpu: { type: string }
|
||||
model: { type: string }
|
||||
/v1/generate-image:
|
||||
post:
|
||||
summary: Generate image from text prompt
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
required: [api_key, prompt]
|
||||
properties:
|
||||
api_key: { type: string }
|
||||
prompt: { type: string }
|
||||
steps: { type: integer, minimum: 5, maximum: 100, default: 30 }
|
||||
guidance: { type: number, minimum: 0, maximum: 25, default: 7.5 }
|
||||
width: { type: integer, minimum: 256, maximum: 1024, default: 512 }
|
||||
height: { type: integer, minimum: 256, maximum: 1024, default: 512 }
|
||||
seed: { type: integer, nullable: true }
|
||||
responses:
|
||||
'200':
|
||||
description: Image generated
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
properties:
|
||||
status: { type: string }
|
||||
job_id: { type: string }
|
||||
image_base64: { type: string }
|
||||
duration_ms: { type: integer }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
# Stage 2 – Receipt‑/Quittungs‑Schema & Hashing
|
||||
|
||||
## JSON Receipt (off‑chain, signierbar)
|
||||
|
||||
```json
|
||||
{
|
||||
"job_id": "2025-09-26-000123",
|
||||
"provider": "0xProviderAddress",
|
||||
"client": "client_public_key_or_id",
|
||||
"units": 1.90,
|
||||
"unit_type": "gpu_seconds",
|
||||
"model": "runwayml/stable-diffusion-v1-5",
|
||||
"prompt_hash": "sha256:...",
|
||||
"started_at": 1695720000,
|
||||
"finished_at": 1695720002,
|
||||
"artifact_sha256": "...",
|
||||
"nonce": "b7f3...",
|
||||
"hub_id": "optional-hub",
|
||||
"chain_id": 11155111
|
||||
}
|
||||
```
|
||||
|
||||
## Hashing
|
||||
|
||||
- Kanonische Serialisierung (minified JSON, Felder in alphabetischer Reihenfolge).
|
||||
- `receipt_hash = keccak256(bytes(serialized))` (für EVM‑Kompatibilität) **oder** `sha256` falls kettenagnostisch.
|
||||
|
||||
## Signatur
|
||||
|
||||
- Signatur über `receipt_hash`:
|
||||
- **secp256k1/ECDSA** (Ethereum‑kompatibel, EIP‑191/EIP‑712) **oder** Ed25519 (falls Off‑Chain‑Attester bevorzugt).
|
||||
- Felder zur Verifikation on‑chain: `provider`, `units`, `receipt_hash`, `signature`.
|
||||
|
||||
## Double‑Mint‑Prevention
|
||||
|
||||
- Smart Contract speichert `used[receipt_hash] = true` nach erfolgreichem Mint.
|
||||
|
||||
---
|
||||
|
||||
# Stage 2 – Smart‑Contract‑Skeleton (Solidity)
|
||||
|
||||
```solidity
|
||||
// SPDX-License-Identifier: MIT
|
||||
pragma solidity ^0.8.24;
|
||||
|
||||
interface IERC20Mint {
|
||||
function mint(address to, uint256 amount) external;
|
||||
}
|
||||
|
||||
contract AITokenMinter {
|
||||
IERC20Mint public token;
|
||||
address public attester; // Off‑chain Hub/Oracle, darf Quittungen bescheinigen
|
||||
mapping(bytes32 => bool) public usedReceipt; // receipt_hash → consumed
|
||||
|
||||
event Minted(address indexed provider, uint256 units, bytes32 receiptHash);
|
||||
event AttesterChanged(address indexed oldA, address indexed newA);
|
||||
|
||||
constructor(address _token, address _attester) {
|
||||
token = IERC20Mint(_token);
|
||||
attester = _attester;
|
||||
}
|
||||
|
||||
function setAttester(address _attester) external /* add access control */ {
|
||||
emit AttesterChanged(attester, _attester);
|
||||
attester = _attester;
|
||||
}
|
||||
|
||||
function mintWithReceipt(
|
||||
address provider,
|
||||
uint256 units,
|
||||
bytes32 receiptHash,
|
||||
bytes calldata attesterSig
|
||||
) external {
|
||||
require(!usedReceipt[receiptHash], "receipt used");
|
||||
// Verify attester signature over EIP‑191 style message: keccak256(abi.encode(provider, units, receiptHash))
|
||||
bytes32 msgHash = keccak256(abi.encode(provider, units, receiptHash));
|
||||
require(_recover(msgHash, attesterSig) == attester, "bad sig");
|
||||
usedReceipt[receiptHash] = true;
|
||||
token.mint(provider, units);
|
||||
emit Minted(provider, units, receiptHash);
|
||||
}
|
||||
|
||||
function _recover(bytes32 msgHash, bytes memory sig) internal pure returns (address) {
|
||||
bytes32 ethHash = keccak256(abi.encodePacked("\x19Ethereum Signed Message:\n32", msgHash));
|
||||
(bytes32 r, bytes32 s, uint8 v) = _split(sig);
|
||||
return ecrecover(ethHash, v, r, s);
|
||||
}
|
||||
|
||||
function _split(bytes memory sig) internal pure returns (bytes32 r, bytes32 s, uint8 v) {
|
||||
require(sig.length == 65, "sig len");
|
||||
assembly {
|
||||
r := mload(add(sig, 32))
|
||||
s := mload(add(sig, 64))
|
||||
v := byte(0, mload(add(sig, 96)))
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
> Hinweis: In Produktion Access‑Control (Ownable/Role‑based), Pausable, Reentrancy‑Guard und EIP‑712‑Typed‑Data einführen.
|
||||
|
||||
---
|
||||
|
||||
# Stage 3 – Hub‑Spezifikation (Kurz)
|
||||
|
||||
- **Scheduler**: Round‑robin + Preis/VRAM‑Filter; optional Reputation.
|
||||
- **Split**: Große Jobs shard‑en; `units` aus Subjobs aggregieren.
|
||||
- **Verification**: Stichprobenhafte Re‑Auswertung / Konsistenz‑Hashes.
|
||||
- **Payout**: Proportionale Verteilung; ein Receipt je Gesamtjob.
|
||||
|
||||
# Stage 4 – Marketplace (Kurz)
|
||||
|
||||
- **Orderbook** (limit/market), **Wallet‑Connect**, Non‑custodial.
|
||||
- **KYC/Compliance** optional je Jurisdiktion.
|
||||
- **Reputation/SLAs** on‑/off‑chain verknüpfbar.
|
||||
|
||||
|
||||
|
||||
---
|
||||
|
||||
# Deployment ohne Docker (Bare‑Metal / VM)
|
||||
|
||||
## Voraussetzungen
|
||||
|
||||
- Ubuntu/Debian mit NVIDIA Treiber (535+) und CUDA/CuDNN passend zur PyTorch‑Version.
|
||||
- Python 3.10+ und `python3-venv`.
|
||||
- Öffentliche Ports: **8000/tcp** (API) – optional Reverse‑Proxy auf 80/443.
|
||||
|
||||
## Treiber & CUDA (Kurz)
|
||||
|
||||
```bash
|
||||
# NVIDIA Treiber (Beispiel Ubuntu)
|
||||
sudo apt update && sudo apt install -y nvidia-driver-535
|
||||
# Nach Reboot: nvidia-smi prüfen
|
||||
# PyTorch bringt eigenes CUDA-Toolkit über Wheels (empfohlen). Kein System-CUDA zwingend nötig.
|
||||
```
|
||||
|
||||
## Benutzer & Verzeichnisstruktur
|
||||
|
||||
```bash
|
||||
sudo useradd -m -r -s /bin/bash aitbc
|
||||
sudo -u aitbc mkdir -p /opt/aitbc/app /opt/aitbc/logs
|
||||
# Code nach /opt/aitbc/app kopieren
|
||||
```
|
||||
|
||||
## Virtualenv & Abhängigkeiten
|
||||
|
||||
```bash
|
||||
sudo -u aitbc bash -lc '
|
||||
cd /opt/aitbc/app && python3 -m venv venv && source venv/bin/activate && \
|
||||
pip install --upgrade pip && pip install -r requirements.txt
|
||||
'
|
||||
```
|
||||
|
||||
## Konfiguration (.env)
|
||||
|
||||
```
|
||||
API_KEY=<GEHEIM>
|
||||
MODEL_ID=runwayml/stable-diffusion-v1-5
|
||||
BIND_HOST=127.0.0.1 # hinter Reverse Proxy
|
||||
BIND_PORT=8000
|
||||
```
|
||||
|
||||
## Systemd‑Unit (Uvicorn)
|
||||
|
||||
`/etc/systemd/system/aitbc.service`
|
||||
|
||||
```ini
|
||||
[Unit]
|
||||
Description=AITBC Stage1 FastAPI Server
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
User=aitbc
|
||||
Group=aitbc
|
||||
WorkingDirectory=/opt/aitbc/app
|
||||
EnvironmentFile=/opt/aitbc/app/.env
|
||||
ExecStart=/opt/aitbc/app/venv/bin/python -m uvicorn server:app --host ${BIND_HOST} --port ${BIND_PORT} --workers 1
|
||||
Restart=always
|
||||
RestartSec=3
|
||||
# GPU/VRAM limits optional per nvidia-visible-devices
|
||||
StandardOutput=append:/opt/aitbc/logs/stdout.log
|
||||
StandardError=append:/opt/aitbc/logs/stderr.log
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
Aktivieren & Starten:
|
||||
|
||||
```bash
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable --now aitbc.service
|
||||
sudo systemctl status aitbc.service
|
||||
```
|
||||
|
||||
## Reverse Proxy (optional, ohne Docker)
|
||||
|
||||
### Nginx (TLS via Certbot)
|
||||
|
||||
```bash
|
||||
sudo apt install -y nginx certbot python3-certbot-nginx
|
||||
sudo tee /etc/nginx/sites-available/aitbc <<'NG'
|
||||
server {
|
||||
listen 80; server_name example.com;
|
||||
location / {
|
||||
proxy_pass http://127.0.0.1:8000;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
}
|
||||
}
|
||||
NG
|
||||
sudo ln -s /etc/nginx/sites-available/aitbc /etc/nginx/sites-enabled/aitbc
|
||||
sudo nginx -t && sudo systemctl reload nginx
|
||||
sudo certbot --nginx -d example.com
|
||||
```
|
||||
|
||||
## Firewall/Netzwerk
|
||||
|
||||
```bash
|
||||
sudo ufw allow OpenSSH
|
||||
sudo ufw allow 80/tcp
|
||||
sudo ufw allow 443/tcp
|
||||
sudo ufw enable
|
||||
```
|
||||
|
||||
## Monitoring ohne Docker
|
||||
|
||||
- **systemd**: `journalctl -u aitbc -f`
|
||||
- **Metriken**: Prometheus Node Exporter, `nvtop`/`nvidia-smi dmon` für GPU.
|
||||
- **Alerts**: systemd `Restart=always`, optional Monit.
|
||||
|
||||
## Zero‑Downtime Update (Rolling ohne Container)
|
||||
|
||||
```bash
|
||||
sudo systemctl stop aitbc
|
||||
sudo -u aitbc bash -lc 'cd /opt/aitbc/app && git pull && source venv/bin/activate && pip install -r requirements.txt'
|
||||
sudo systemctl start aitbc
|
||||
```
|
||||
|
||||
## Härtung & Best Practices
|
||||
|
||||
- Starker API‑Key, IP‑basierte Allow‑List am Reverse‑Proxy.
|
||||
- Rate‑Limit (slowapi) aktivieren; Request‑Body‑Limits setzen (`client_max_body_size`).
|
||||
- Temporäre Dateien regelmäßig bereinigen (systemd tmpfiles).
|
||||
- Separate GPU‑Workstation vs. Edge‑Expose (API hinter Proxy).
|
||||
|
||||
> Hinweis: Diese Anleitung vermeidet bewusst jeglichen Docker‑Einsatz und nutzt **systemd + venv** für einen reproduzierbaren, schlanken Betrieb.
|
||||
|
||||
392
docs/bootstrap/blockchain_node.md
Normal file
392
docs/bootstrap/blockchain_node.md
Normal file
@ -0,0 +1,392 @@
|
||||
# blockchain-node/ — Minimal Chain (asset-backed by compute)
|
||||
|
||||
## 0) TL;DR boot path for Windsurf
|
||||
1. Create the service: `apps/blockchain-node` (Python, FastAPI, asyncio, uvicorn).
|
||||
2. Data layer: `sqlite` via `SQLModel` (later: PostgreSQL).
|
||||
3. P2P: WebSocket gossip (lib: `websockets`) with a simple overlay (peer table + heartbeats).
|
||||
4. Consensus (MVP): **PoA single-author** (devnet) → upgrade to **Compute-Backed Proof (CBP)** after coordinator & miner telemetry are wired.
|
||||
5. Block content: **ComputeReceipts** = “proofs of delivered AI work” signed by miners, plus standard transfers.
|
||||
6. Minting: AIToken minted per verified compute unit (e.g., `1 AIT = 1,000 token-ops` — calibrate later).
|
||||
7. REST RPC: `/rpc/*` for clients & coordinator; `/p2p/*` for peers; `/admin/*` for node ops.
|
||||
8. Ship a `devnet` script that starts: 1 bootstrap node, 1 coordinator-api mock, 1 miner mock, 1 client demo.
|
||||
|
||||
---
|
||||
|
||||
## 1) Goal & Scope
|
||||
- Provide a **minimal, testable blockchain node** that issues AITokens **only** when real compute was delivered (asset-backed).
|
||||
- Easy to run, easy to reset, deterministic devnet.
|
||||
- Strong boundaries so **coordinator-api** (job orchestration) and **miner-node** (workers) can integrate quickly.
|
||||
|
||||
Out of scope (MVP):
|
||||
- Smart contracts VM.
|
||||
- Sharding/advanced networking.
|
||||
- Custodial wallets. (Use local keypairs for dev.)
|
||||
|
||||
---
|
||||
|
||||
## 2) Core Concepts
|
||||
|
||||
### 2.1 Actors
|
||||
- **Client**: pays AITokens to request compute jobs.
|
||||
- **Coordinator**: matches jobs ↔ miners; returns signed receipts.
|
||||
- **Miner**: executes jobs; produces **ComputeReceipt** signed with miner key.
|
||||
- **Blockchain Node**: validates receipts, mints AIT for miners, tracks balances, finalizes blocks.
|
||||
|
||||
### 2.2 Asset-Backed Minting
|
||||
- Unit of account: **AIToken (AIT)**.
|
||||
- A miner earns AIT when a **ComputeReceipt** is included in a block.
|
||||
- A receipt is valid iff:
|
||||
1) Its `job_id` exists in coordinator logs,
|
||||
2) `client_payment_tx` covers the quoted price,
|
||||
3) `miner_sig` over `(job_id, hash(output_meta), compute_units, price, nonce)` is valid,
|
||||
4) Not previously claimed (`receipt_id` unique).
|
||||
|
||||
---
|
||||
|
||||
## 3) Minimal Architecture
|
||||
|
||||
```
|
||||
blockchain-node/
|
||||
├─ src/
|
||||
│ ├─ main.py # FastAPI entry
|
||||
│ ├─ p2p.py # WS gossip, peer table, block relay
|
||||
│ ├─ consensus.py # PoA/CBP state machine
|
||||
│ ├─ types.py # dataclasses / pydantic models
|
||||
│ ├─ state.py # DB access (SQLModel), UTXO/Account
|
||||
│ ├─ mempool.py # tx pool (transfers + receipts)
|
||||
│ ├─ crypto.py # ed25519 keys, signatures, hashing
|
||||
│ ├─ receipts.py # receipt validation (with coordinator)
|
||||
│ ├─ blocks.py # block build/verify, difficulty stub
|
||||
│ ├─ rpc.py # REST/RPC routes for clients & ops
|
||||
│ └─ settings.py # env config
|
||||
├─ tests/
|
||||
│ └─ ... # unit & integration tests
|
||||
├─ scripts/
|
||||
│ ├─ devnet_up.sh # run bootstrap node + mocks
|
||||
│ └─ keygen.py # create node/miner/client keys
|
||||
├─ README.md
|
||||
└─ requirements.txt
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4) Data Model (SQLModel)
|
||||
|
||||
### 4.1 Tables
|
||||
- `blocks(id, parent_id, height, timestamp, proposer, tx_count, hash, state_root, sig)`
|
||||
- `tx(id, block_id, type, payload_json, sender, nonce, fee, sig, hash, status)`
|
||||
- `accounts(address, balance, nonce, pubkey)`
|
||||
- `receipts(receipt_id, job_id, client_addr, miner_addr, compute_units, price, output_hash, miner_sig, status)`
|
||||
- `peers(node_id, addr, last_seen, score)`
|
||||
- `params(key, value)` — chain config (mint ratios, fee rate, etc.)
|
||||
|
||||
### 4.2 TX Types
|
||||
- `TRANSFER`: move AIT from A → B
|
||||
- `RECEIPT_CLAIM`: include a **ComputeReceipt**; mints to miner and settles client escrow
|
||||
- `STAKE/UNSTAKE` (later)
|
||||
- `PARAM_UPDATE` (PoA only, gated by admin key for devnet)
|
||||
|
||||
---
|
||||
|
||||
## 5) Block Format (JSON)
|
||||
```json
|
||||
{
|
||||
"parent": "<block_hash>",
|
||||
"height": 123,
|
||||
"timestamp": 1699999999,
|
||||
"proposer": "<node_address>",
|
||||
"txs": ["<tx_hash>", "..."],
|
||||
"stateRoot": "<merkle_root_after_block>",
|
||||
"sig": "<proposer_signature_over_header>"
|
||||
}
|
||||
```
|
||||
|
||||
Header sign bytes = `hash(parent|height|timestamp|proposer|stateRoot)`
|
||||
|
||||
---
|
||||
|
||||
## 6) Consensus
|
||||
|
||||
### 6.1 MVP: PoA (Single Author)
|
||||
- One configured `PROPOSER_KEY` creates blocks at fixed interval (e.g., 2s).
|
||||
- Honest mode only for devnet; finality by canonical longest/height rule.
|
||||
|
||||
### 6.2 Upgrade: **Compute-Backed Proof (CBP)**
|
||||
- Each block’s **work score** = total `compute_units` in included receipts.
|
||||
- Proposer election = weighted round-robin by recent work score and stake (later).
|
||||
- Slashing: submitting invalid receipts reduces score; repeated offenses → temp ban.
|
||||
|
||||
---
|
||||
|
||||
## 7) Receipt Validation (Coordinator Check)
|
||||
|
||||
`receipts.py` performs:
|
||||
1) **Coordinator attestation** (HTTP call to coordinator-api):
|
||||
- `/attest/receipt` with `job_id`, `client`, `miner`, `price`, `compute_units`, `output_hash`.
|
||||
- Returns `{exists: bool, paid: bool, not_double_spent: bool, quote: {...}}`.
|
||||
2) **Signature check**: verify `miner_sig` with miner’s `pubkey`.
|
||||
3) **Economic checks**: ensure `client_payment_tx` exists & covers `price + fee`.
|
||||
|
||||
> For devnet without live coordinator, ship a **mock** that returns deterministic attestation for known `job_id` ranges.
|
||||
|
||||
---
|
||||
|
||||
## 8) Fees & Minting
|
||||
|
||||
- **Fee model (MVP)**: `fee = base_fee + k * payload_size`.
|
||||
- **Minting**:
|
||||
- Miner gets: `mint = compute_units * MINT_PER_UNIT`.
|
||||
- Coordinator gets: `coord_cut = mint * COORDINATOR_RATIO`.
|
||||
- Chain treasury (optional): small %, configurable in `params`.
|
||||
|
||||
---
|
||||
|
||||
## 9) RPC Surface (FastAPI)
|
||||
|
||||
### 9.1 Public
|
||||
- `POST /rpc/sendTx` → `{txHash}`
|
||||
- `GET /rpc/getTx/{txHash}` → `{status, receipt}`
|
||||
- `GET /rpc/getBlock/{heightOrHash}`
|
||||
- `GET /rpc/getHead` → `{height, hash}`
|
||||
- `GET /rpc/getBalance/{address}` → `{balance, nonce}`
|
||||
- `POST /rpc/estimateFee` → `{fee}`
|
||||
|
||||
### 9.2 Coordinator-facing
|
||||
- `POST /rpc/submitReceipt` (alias of `sendTx` with type `RECEIPT_CLAIM`)
|
||||
- `POST /rpc/attest` (devnet mock only)
|
||||
|
||||
### 9.3 Admin (devnet)
|
||||
- `POST /admin/paramSet` (PoA only)
|
||||
- `POST /admin/peers/add` `{addr}`
|
||||
- `POST /admin/mintFaucet` `{address, amount}` (devnet)
|
||||
|
||||
### 9.4 P2P (WS)
|
||||
- `GET /p2p/peers` → list
|
||||
- `WS /p2p/ws` → subscribe to gossip: `{"type":"block"|"tx"|"peer","data":...}`
|
||||
|
||||
---
|
||||
|
||||
## 10) Keys & Crypto
|
||||
- **ed25519** for account & node keys.
|
||||
- Address = `bech32(hrp="ait", sha256(pubkey)[0:20])`.
|
||||
- Sign bytes:
|
||||
- TX: `hash(type|sender|nonce|fee|payload_json_canonical)`
|
||||
- Block: header hash as above.
|
||||
|
||||
Ship `scripts/keygen.py` for dev use.
|
||||
|
||||
---
|
||||
|
||||
## 11) Mempool Rules
|
||||
- Accept if:
|
||||
- `sig` valid,
|
||||
- `nonce == account.nonce + 1`,
|
||||
- `fee >= minFee`,
|
||||
- For `RECEIPT_CLAIM`: passes `receipts.validate()` *optimistically* (soft-accept), then **revalidate** at block time.
|
||||
|
||||
Replacement: higher-fee replaces same `(sender, nonce)`.
|
||||
|
||||
---
|
||||
|
||||
## 12) Node Lifecycle
|
||||
|
||||
**Start:**
|
||||
1) Load config, open DB, ensure genesis.
|
||||
2) Connect to bootstrap peers (if any).
|
||||
3) Start RPC (FastAPI) + P2P WS server.
|
||||
4) Start block proposer (if PoA key present).
|
||||
5) Start peer heartbeats + gossip loops.
|
||||
|
||||
**Shutdown:**
|
||||
- Graceful: flush mempool snapshot, close DB.
|
||||
|
||||
---
|
||||
|
||||
## 13) Genesis
|
||||
- `genesis.json`:
|
||||
- `chain_id`, `timestamp`, `accounts` (faucet), `params` (mint ratios, base fee), `authorities` (PoA keys).
|
||||
|
||||
Provide `scripts/make_genesis.py`.
|
||||
|
||||
---
|
||||
|
||||
## 14) Devnet: End-to-End Demo
|
||||
|
||||
### 14.1 Components
|
||||
- **blockchain-node** (this repo)
|
||||
- **coordinator-api (mock)**: `/attest/receipt` returns valid for `job_id` in `[1..1_000_000]`
|
||||
- **miner-node (mock)**: posts `RECEIPT_CLAIM` for synthetic jobs
|
||||
- **client-web (demo)**: sends `TRANSFER` & displays balances
|
||||
|
||||
### 14.2 Flow
|
||||
1) Client pays `price` to escrow address (coordinator).
|
||||
2) Miner executes job; coordinator verifies output.
|
||||
3) Miner submits **ComputeReceipt** → included in next block.
|
||||
4) Mint AIT to miner; escrow settles; client charged.
|
||||
|
||||
---
|
||||
|
||||
## 15) Testing Strategy
|
||||
|
||||
### 15.1 Unit
|
||||
- `crypto`: keygen, sign/verify, address derivation
|
||||
- `state`: balances, nonce, persistence
|
||||
- `receipts`: signature + coordinator mock
|
||||
- `blocks`: header hash, stateRoot
|
||||
|
||||
### 15.2 Integration
|
||||
- Single node PoA: produce N blocks; submit transfers/receipts; assert balances.
|
||||
- Two nodes P2P: block/tx relay; head convergence.
|
||||
|
||||
### 15.3 Property tests
|
||||
- Nonce monotonicity; no double-spend; unique receipts.
|
||||
|
||||
---
|
||||
|
||||
## 16) Observability
|
||||
- Structured logs (JSON) with `component`, `event`, `height`, `latency_ms`.
|
||||
- `/rpc/metrics` (Prometheus format) — block time, mempool size, peers.
|
||||
|
||||
---
|
||||
|
||||
## 17) Configuration (ENV)
|
||||
- `CHAIN_ID=ait-devnet`
|
||||
- `DB_PATH=./data/chain.db`
|
||||
- `P2P_BIND=0.0.0.0:7070`
|
||||
- `RPC_BIND=0.0.0.0:8080`
|
||||
- `BOOTSTRAP_PEERS=ws://host:7070,...`
|
||||
- `PROPOSER_KEY=...` (optional for non-authors)
|
||||
- `MINT_PER_UNIT=1000`
|
||||
- `COORDINATOR_RATIO=0.05`
|
||||
|
||||
Provide `.env.example`.
|
||||
|
||||
---
|
||||
|
||||
## 18) Minimal API Payloads
|
||||
|
||||
### 18.1 TRANSFER
|
||||
```json
|
||||
{
|
||||
"type": "TRANSFER",
|
||||
"sender": "ait1...",
|
||||
"nonce": 1,
|
||||
"fee": 10,
|
||||
"payload": {"to":"ait1...","amount":12345},
|
||||
"sig": "<ed25519>"
|
||||
}
|
||||
```
|
||||
|
||||
### 18.2 RECEIPT_CLAIM
|
||||
```json
|
||||
{
|
||||
"type": "RECEIPT_CLAIM",
|
||||
"sender": "ait1miner...",
|
||||
"nonce": 7,
|
||||
"fee": 50,
|
||||
"payload": {
|
||||
"receipt_id": "rcpt_7f3a...",
|
||||
"job_id": "job_42",
|
||||
"client_addr": "ait1client...",
|
||||
"miner_addr": "ait1miner...",
|
||||
"compute_units": 2500,
|
||||
"price": 50000,
|
||||
"output_hash": "sha256:abcd...",
|
||||
"miner_sig": "<sig_over_core_fields>"
|
||||
},
|
||||
"sig": "<miner_account_sig>"
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 19) Security Notes (MVP)
|
||||
- Devnet PoA means trust in proposer; do **not** expose to internet without firewall.
|
||||
- Enforce coordinator host allowlist for attest calls.
|
||||
- Rate-limit `/rpc/sendTx`.
|
||||
|
||||
---
|
||||
|
||||
## 20) Roadmap
|
||||
1) ✅ PoA devnet with receipts.
|
||||
2) 🔜 CBP proposer selection from rolling work score.
|
||||
3) 🔜 Stake & slashing.
|
||||
4) 🔜 Replace SQLite with PostgreSQL.
|
||||
5) 🔜 Snapshots & fast-sync.
|
||||
6) 🔜 Light client (SPV of receipts & balances).
|
||||
|
||||
---
|
||||
|
||||
## 21) Developer Tasks (Windsurf Order)
|
||||
|
||||
1) **Scaffold** project & `requirements.txt`:
|
||||
- `fastapi`, `uvicorn[standard]`, `sqlmodel`, `pydantic`, `websockets`, `pyyaml`, `python-dotenv`, `ed25519`, `orjson`.
|
||||
|
||||
2) **Implement**:
|
||||
- `crypto.py`, `types.py`, `state.py`.
|
||||
- `rpc.py` (public routes).
|
||||
- `mempool.py`.
|
||||
- `blocks.py` (build/validate).
|
||||
- `consensus.py` (PoA tick).
|
||||
- `p2p.py` (WS server + simple gossip).
|
||||
- `receipts.py` (mock coordinator).
|
||||
|
||||
3) **Wire** `main.py`:
|
||||
- Start RPC, P2P, PoA loops.
|
||||
|
||||
4) **Scripts**:
|
||||
- `scripts/keygen.py`, `scripts/make_genesis.py`, `scripts/devnet_up.sh`.
|
||||
|
||||
5) **Tests**:
|
||||
- Add unit + an integration test that mints on a receipt.
|
||||
|
||||
6) **Docs**:
|
||||
- Update `README.md` with curl examples.
|
||||
|
||||
---
|
||||
|
||||
## 22) Curl Snippets (Dev)
|
||||
|
||||
- Faucet (dev only):
|
||||
```bash
|
||||
curl -sX POST localhost:8080/admin/mintFaucet -H 'content-type: application/json' \
|
||||
-d '{"address":"ait1client...","amount":1000000}'
|
||||
```
|
||||
|
||||
- Transfer:
|
||||
```bash
|
||||
curl -sX POST localhost:8080/rpc/sendTx -H 'content-type: application/json' \
|
||||
-d @transfer.json
|
||||
```
|
||||
|
||||
- Submit Receipt:
|
||||
```bash
|
||||
curl -sX POST localhost:8080/rpc/submitReceipt -H 'content-type: application/json' \
|
||||
-d @receipt_claim.json
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 23) Definition of Done (MVP)
|
||||
- Node produces blocks on PoA.
|
||||
- Can transfer AIT between accounts.
|
||||
- Can submit a valid **ComputeReceipt** → miner balance increases; escrow decreases.
|
||||
- Two nodes converge on same head via P2P.
|
||||
- Basic metrics exposed.
|
||||
|
||||
---
|
||||
|
||||
## 24) Next Files to Create
|
||||
- `src/main.py`
|
||||
- `src/crypto.py`
|
||||
- `src/types.py`
|
||||
- `src/state.py`
|
||||
- `src/mempool.py`
|
||||
- `src/blocks.py`
|
||||
- `src/consensus.py`
|
||||
- `src/p2p.py`
|
||||
- `src/receipts.py`
|
||||
- `src/rpc.py`
|
||||
- `scripts/keygen.py`, `scripts/devnet_up.sh`
|
||||
- `.env.example`, `README.md`, `requirements.txt`
|
||||
|
||||
438
docs/bootstrap/coordinator_api.md
Normal file
438
docs/bootstrap/coordinator_api.md
Normal file
@ -0,0 +1,438 @@
|
||||
# coordinator-api.md
|
||||
|
||||
Central API that orchestrates **jobs** from clients to **miners**, tracks lifecycle, validates results, and (later) settles AITokens.
|
||||
**Stage 1 (MVP):** no blockchain, no pool hub — just client ⇄ coordinator ⇄ miner.
|
||||
|
||||
## 1) Goals & Non-Goals
|
||||
|
||||
**Goals (MVP)**
|
||||
- Accept computation jobs from clients.
|
||||
- Match jobs to eligible miners.
|
||||
- Track job state machine (QUEUED → RUNNING → COMPLETED/FAILED/CANCELED/EXPIRED).
|
||||
- Stream results back to clients; store minimal metadata.
|
||||
- Provide a clean, typed API (OpenAPI/Swagger).
|
||||
- Simple auth (API keys) + idempotency + rate limits.
|
||||
- Minimal persistence (SQLite/Postgres) with straightforward SQL (no migrations tooling).
|
||||
|
||||
**Non-Goals (MVP)**
|
||||
- Token minting/settlement (stub hooks only).
|
||||
- Miner marketplace, staking, slashing, reputation (placeholders).
|
||||
- Pool hub coordination (future stage).
|
||||
|
||||
---
|
||||
|
||||
## 2) Tech Stack
|
||||
|
||||
- **Python 3.12**, **FastAPI**, **Uvicorn**
|
||||
- **Pydantic** for schemas
|
||||
- **SQL** via `sqlite3` or Postgres (user can switch later)
|
||||
- **Redis (optional)** for queueing; MVP can start with in-DB FIFO
|
||||
- **HTTP + WebSocket** (for miner heartbeats / job streaming)
|
||||
|
||||
> Debian 12 target. Run under **systemd** later.
|
||||
|
||||
---
|
||||
|
||||
## 3) Directory Layout (WindSurf Workspace)
|
||||
|
||||
```
|
||||
coordinator-api/
|
||||
├─ app/
|
||||
│ ├─ main.py # FastAPI init, lifespan, routers
|
||||
│ ├─ config.py # env parsing
|
||||
│ ├─ deps.py # auth, rate-limit deps
|
||||
│ ├─ db.py # simple DB layer (sqlite/postgres)
|
||||
│ ├─ matching.py # job→miner selection
|
||||
│ ├─ queue.py # enqueue/dequeue logic
|
||||
│ ├─ settlement.py # stubs for token accounting
|
||||
│ ├─ models.py # Pydantic request/response schemas
|
||||
│ ├─ states.py # state machine + transitions
|
||||
│ ├─ routers/
|
||||
│ │ ├─ client.py # /v1/jobs (submit/status/result/cancel)
|
||||
│ │ ├─ miner.py # /v1/miners (register/heartbeat/poll/submit/fail)
|
||||
│ │ └─ admin.py # /v1/admin (stats)
|
||||
│ └─ ws/
|
||||
│ ├─ miner.py # WS for miner heartbeats / job stream (optional)
|
||||
│ └─ client.py # WS for client result stream (optional)
|
||||
├─ tests/
|
||||
│ ├─ test_client_flow.http # REST client flow (HTTP file)
|
||||
│ └─ test_miner_flow.http # REST miner flow
|
||||
├─ .env.example
|
||||
├─ pyproject.toml
|
||||
└─ README.md
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4) Environment (.env)
|
||||
|
||||
```
|
||||
APP_ENV=dev
|
||||
APP_HOST=127.0.0.1
|
||||
APP_PORT=8011
|
||||
DATABASE_URL=sqlite:///./coordinator.db
|
||||
# or: DATABASE_URL=postgresql://user:pass@localhost:5432/aitbc
|
||||
|
||||
# Auth
|
||||
CLIENT_API_KEYS=client_dev_key_1,client_dev_key_2
|
||||
MINER_API_KEYS=miner_dev_key_1,miner_dev_key_2
|
||||
ADMIN_API_KEYS=admin_dev_key_1
|
||||
|
||||
# Security
|
||||
HMAC_SECRET=change_me
|
||||
ALLOW_ORIGINS=*
|
||||
|
||||
# Queue
|
||||
JOB_TTL_SECONDS=900
|
||||
HEARTBEAT_INTERVAL_SECONDS=10
|
||||
HEARTBEAT_TIMEOUT_SECONDS=30
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5) Core Data Model (conceptual)
|
||||
|
||||
**Job**
|
||||
- `job_id` (uuid)
|
||||
- `client_id` (from API key)
|
||||
- `requested_at`, `expires_at`
|
||||
- `payload` (opaque JSON / bytes ref)
|
||||
- `constraints` (gpu, cuda, mem, model, max_price, region)
|
||||
- `state` (QUEUED|RUNNING|COMPLETED|FAILED|CANCELED|EXPIRED)
|
||||
- `assigned_miner_id` (nullable)
|
||||
- `result_ref` (blob path / inline json)
|
||||
- `error` (nullable)
|
||||
- `cost_estimate` (optional)
|
||||
|
||||
**Miner**
|
||||
- `miner_id` (from API key)
|
||||
- `capabilities` (gpu, cuda, vram, models[], region)
|
||||
- `heartbeat_at`
|
||||
- `status` (ONLINE|OFFLINE|DRAINING)
|
||||
- `concurrency` (int), `inflight` (int)
|
||||
|
||||
**WorkerSession**
|
||||
- `session_id`, `miner_id`, `job_id`, `started_at`, `ended_at`, `exit_reason`
|
||||
|
||||
---
|
||||
|
||||
## 6) State Machine
|
||||
|
||||
```
|
||||
QUEUED
|
||||
-> RUNNING (assigned to miner)
|
||||
-> CANCELED (client)
|
||||
-> EXPIRED (ttl)
|
||||
|
||||
RUNNING
|
||||
-> COMPLETED (miner submit_result)
|
||||
-> FAILED (miner fail / timeout)
|
||||
-> CANCELED (client)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7) Matching (MVP)
|
||||
|
||||
- Filter ONLINE miners by **capabilities** & **region**
|
||||
- Prefer lowest `inflight` (simple load)
|
||||
- Tiebreak by earliest `heartbeat_at` or random
|
||||
- Lock job row → assign → return to miner
|
||||
|
||||
---
|
||||
|
||||
## 8) Auth & Rate Limits
|
||||
|
||||
- **API keys** via `X-Api-Key` header for `client`, `miner`, `admin`.
|
||||
- Optional **HMAC** (`X-Signature`) over body with `HMAC_SECRET`.
|
||||
- **Idempotency**: clients send `Idempotency-Key` on **POST /jobs**.
|
||||
- **Rate limiting**: naive per-key window (e.g., 60 req / 60 s).
|
||||
|
||||
---
|
||||
|
||||
## 9) REST API
|
||||
|
||||
### Client
|
||||
|
||||
- `POST /v1/jobs`
|
||||
- Create job. Returns `job_id`.
|
||||
- `GET /v1/jobs/{job_id}`
|
||||
- Job status & metadata.
|
||||
- `GET /v1/jobs/{job_id}/result`
|
||||
- Result (200 when ready, 425 if not ready).
|
||||
- `POST /v1/jobs/{job_id}/cancel`
|
||||
- Cancel if QUEUED or RUNNING (best effort).
|
||||
|
||||
### Miner
|
||||
|
||||
- `POST /v1/miners/register`
|
||||
- Upsert miner capabilities; set ONLINE.
|
||||
- `POST /v1/miners/heartbeat`
|
||||
- Touch `heartbeat_at`, report `inflight`.
|
||||
- `POST /v1/miners/poll`
|
||||
- Long-poll for next job → returns a job or 204.
|
||||
- `POST /v1/miners/{job_id}/start`
|
||||
- Confirm start (optional if `poll` implies start).
|
||||
- `POST /v1/miners/{job_id}/result`
|
||||
- Submit result; transitions to COMPLETED.
|
||||
- `POST /v1/miners/{job_id}/fail`
|
||||
- Submit failure; transitions to FAILED.
|
||||
- `POST /v1/miners/drain`
|
||||
- Graceful stop accepting new jobs.
|
||||
|
||||
### Admin
|
||||
|
||||
- `GET /v1/admin/stats`
|
||||
- Queue depth, miners online, success rates, avg latency.
|
||||
- `GET /v1/admin/jobs?state=&limit=...`
|
||||
- `GET /v1/admin/miners`
|
||||
|
||||
**Error Shape**
|
||||
```json
|
||||
{ "error": { "code": "STRING_CODE", "message": "human readable", "details": {} } }
|
||||
```
|
||||
|
||||
Common codes: `UNAUTHORIZED_KEY`, `RATE_LIMITED`, `INVALID_PAYLOAD`, `NO_ELIGIBLE_MINER`, `JOB_NOT_FOUND`, `JOB_NOT_READY`, `CONFLICT_STATE`.
|
||||
|
||||
---
|
||||
|
||||
## 10) WebSockets (optional MVP+)
|
||||
|
||||
- `WS /v1/ws/miner?api_key=...`
|
||||
- Server → miner: `job.assigned`
|
||||
- Miner → server: `heartbeat`, `result`, `fail`
|
||||
- `WS /v1/ws/client?job_id=...&api_key=...`
|
||||
- Server → client: `state.changed`, `result.ready`
|
||||
|
||||
Fallback remains HTTP long-polling.
|
||||
|
||||
---
|
||||
|
||||
## 11) Result Storage
|
||||
|
||||
- **Inline JSON** if ≤ 1 MB.
|
||||
- For larger payloads: store to disk path (e.g., `/var/lib/coordinator/results/{job_id}`) and return `result_ref`.
|
||||
|
||||
---
|
||||
|
||||
## 12) Settlement Hooks (stub)
|
||||
|
||||
`settlement.py` exposes:
|
||||
- `record_usage(job, miner)`
|
||||
- `quote_cost(job)`
|
||||
Later wired to **AIToken** mint/transfer when blockchain lands.
|
||||
|
||||
---
|
||||
|
||||
## 13) Minimal FastAPI Skeleton
|
||||
|
||||
```python
|
||||
# app/main.py
|
||||
from fastapi import FastAPI
|
||||
from app.routers import client, miner, admin
|
||||
|
||||
def create_app():
|
||||
app = FastAPI(title="AITBC Coordinator API", version="0.1.0")
|
||||
app.include_router(client.router, prefix="/v1")
|
||||
app.include_router(miner.router, prefix="/v1")
|
||||
app.include_router(admin.router, prefix="/v1")
|
||||
return app
|
||||
|
||||
app = create_app()
|
||||
```
|
||||
|
||||
```python
|
||||
# app/models.py
|
||||
from pydantic import BaseModel, Field
|
||||
from typing import Any, Dict, List, Optional
|
||||
|
||||
class Constraints(BaseModel):
|
||||
gpu: Optional[str] = None
|
||||
cuda: Optional[str] = None
|
||||
min_vram_gb: Optional[int] = None
|
||||
models: Optional[List[str]] = None
|
||||
region: Optional[str] = None
|
||||
max_price: Optional[float] = None
|
||||
|
||||
class JobCreate(BaseModel):
|
||||
payload: Dict[str, Any]
|
||||
constraints: Constraints = Constraints()
|
||||
ttl_seconds: int = 900
|
||||
|
||||
class JobView(BaseModel):
|
||||
job_id: str
|
||||
state: str
|
||||
assigned_miner_id: Optional[str] = None
|
||||
requested_at: str
|
||||
expires_at: str
|
||||
error: Optional[str] = None
|
||||
|
||||
class MinerRegister(BaseModel):
|
||||
capabilities: Dict[str, Any]
|
||||
concurrency: int = 1
|
||||
region: Optional[str] = None
|
||||
|
||||
class PollRequest(BaseModel):
|
||||
max_wait_seconds: int = 15
|
||||
|
||||
class AssignedJob(BaseModel):
|
||||
job_id: str
|
||||
payload: Dict[str, Any]
|
||||
```
|
||||
|
||||
```python
|
||||
# app/routers/client.py
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from app.models import JobCreate, JobView
|
||||
from app.deps import require_client_key
|
||||
|
||||
router = APIRouter(tags=["client"])
|
||||
|
||||
@router.post("/jobs", response_model=JobView)
|
||||
def submit_job(req: JobCreate, client_id: str = Depends(require_client_key)):
|
||||
# enqueue + return JobView
|
||||
...
|
||||
|
||||
@router.get("/jobs/{job_id}", response_model=JobView)
|
||||
def get_job(job_id: str, client_id: str = Depends(require_client_key)):
|
||||
...
|
||||
```
|
||||
|
||||
```python
|
||||
# app/routers/miner.py
|
||||
from fastapi import APIRouter, Depends
|
||||
from app.models import MinerRegister, PollRequest, AssignedJob
|
||||
from app.deps import require_miner_key
|
||||
|
||||
router = APIRouter(tags=["miner"])
|
||||
|
||||
@router.post("/miners/register")
|
||||
def register(req: MinerRegister, miner_id: str = Depends(require_miner_key)):
|
||||
...
|
||||
|
||||
@router.post("/miners/poll", response_model=AssignedJob, status_code=200)
|
||||
def poll(req: PollRequest, miner_id: str = Depends(require_miner_key)):
|
||||
# try dequeue, else 204
|
||||
...
|
||||
```
|
||||
|
||||
Run:
|
||||
```bash
|
||||
uvicorn app.main:app --host 127.0.0.1 --port 8011 --reload
|
||||
```
|
||||
|
||||
OpenAPI: `http://127.0.0.1:8011/docs`
|
||||
|
||||
---
|
||||
|
||||
## 14) Matching & Queue Pseudocode
|
||||
|
||||
```python
|
||||
def match_next_job(miner):
|
||||
eligible = db.jobs.filter(
|
||||
state="QUEUED",
|
||||
constraints.satisfied_by(miner.capabilities)
|
||||
).order_by("requested_at").first()
|
||||
if not eligible:
|
||||
return None
|
||||
db.txn(lambda:
|
||||
db.jobs.assign(eligible.job_id, miner.id) and
|
||||
db.states.transition(eligible.job_id, "RUNNING")
|
||||
)
|
||||
return eligible
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 15) CURL Examples
|
||||
|
||||
**Client creates a job**
|
||||
```bash
|
||||
curl -sX POST http://127.0.0.1:8011/v1/jobs \
|
||||
-H 'X-Api-Key: client_dev_key_1' \
|
||||
-H 'Idempotency-Key: 7d4a...' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"payload": {"task":"sum","a":2,"b":3},
|
||||
"constraints": {"gpu": null, "region": "eu-central"}
|
||||
}'
|
||||
```
|
||||
|
||||
**Miner registers + polls**
|
||||
```bash
|
||||
curl -sX POST http://127.0.0.1:8011/v1/miners/register \
|
||||
-H 'X-Api-Key: miner_dev_key_1' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"capabilities":{"gpu":"RTX4060Ti","cuda":"12.3","vram_gb":16},"concurrency":2,"region":"eu-central"}'
|
||||
|
||||
curl -i -sX POST http://127.0.0.1:8011/v1/miners/poll \
|
||||
-H 'X-Api-Key: miner_dev_key_1' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"max_wait_seconds":10}'
|
||||
```
|
||||
|
||||
**Miner submits result**
|
||||
```bash
|
||||
curl -sX POST http://127.0.0.1:8011/v1/miners/<JOB_ID>/result \
|
||||
-H 'X-Api-Key: miner_dev_key_1' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"result":{"sum":5},"metrics":{"latency_ms":42}}'
|
||||
```
|
||||
|
||||
**Client fetches result**
|
||||
```bash
|
||||
curl -s http://127.0.0.1:8011/v1/jobs/<JOB_ID>/result \
|
||||
-H 'X-Api-Key: client_dev_key_1'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 16) Timeouts & Health
|
||||
|
||||
- **Job TTL**: auto-expire QUEUED after `JOB_TTL_SECONDS`.
|
||||
- **Heartbeat**: miners post every `HEARTBEAT_INTERVAL_SECONDS`.
|
||||
- **Miner OFFLINE** if no heartbeat for `HEARTBEAT_TIMEOUT_SECONDS`.
|
||||
- **Requeue**: RUNNING jobs from OFFLINE miners → back to QUEUED.
|
||||
|
||||
---
|
||||
|
||||
## 17) Security Notes
|
||||
|
||||
- Validate `payload` size & type; enforce max 1 MB inline.
|
||||
- Optional **HMAC** signature for tamper detection.
|
||||
- Sanitize/validate miner-reported capabilities.
|
||||
- Log every state transition (append-only).
|
||||
|
||||
---
|
||||
|
||||
## 18) Admin Metrics (MVP)
|
||||
|
||||
- Queue depth, running count
|
||||
- Miner online/offline, inflight
|
||||
- P50/P95 job latency
|
||||
- Success/fail/cancel rates (windowed)
|
||||
|
||||
---
|
||||
|
||||
## 19) Future Stages
|
||||
|
||||
- **Blockchain layer**: mint on verified compute; tie to `record_usage`.
|
||||
- **Pool hub**: multi-coordinator balancing; marketplace.
|
||||
- **Reputation**: miner scoring, penalty, slashing.
|
||||
- **Bidding**: price discovery; client max price.
|
||||
|
||||
---
|
||||
|
||||
## 20) Checklist (WindSurf)
|
||||
|
||||
1. Create repo structure from section **3**.
|
||||
2. Implement `.env` & `config.py` keys from **4**.
|
||||
3. Add `models.py`, `states.py`, `deps.py` (auth, rate limit).
|
||||
4. Implement DB tables for Job, Miner, WorkerSession.
|
||||
5. Implement `queue.py` and `matching.py`.
|
||||
6. Wire **client** and **miner** routers (MVP endpoints).
|
||||
7. Add admin stats (basic counts).
|
||||
8. Add OpenAPI tags, descriptions.
|
||||
9. Add curl `.http` test files.
|
||||
10. Systemd unit + Nginx proxy (later).
|
||||
|
||||
133
docs/bootstrap/dirs.md
Normal file
133
docs/bootstrap/dirs.md
Normal file
@ -0,0 +1,133 @@
|
||||
# AITBC Monorepo Directory Layout (Windsurf Workspace)
|
||||
|
||||
> One workspace for **all** AITBC elements (client · coordinator · miner · blockchain · pool‑hub · marketplace · wallet · docs · ops). No Docker required.
|
||||
|
||||
```
|
||||
aitbc/
|
||||
├─ .editorconfig
|
||||
├─ .gitignore
|
||||
├─ README.md # Top‑level overview, quickstart, workspace tasks
|
||||
├─ LICENSE
|
||||
├─ windsurf/ # Windsurf prompts, tasks, run configurations
|
||||
│ ├─ prompts/ # High‑level task prompts for WS agents
|
||||
│ ├─ tasks/ # Saved task flows / playbooks
|
||||
│ └─ settings.json # Editor/workbench preferences for this repo
|
||||
├─ scripts/ # CLI scripts (bash/python); dev + ops helpers
|
||||
│ ├─ env/ # venv helpers (create, activate, pin)
|
||||
│ ├─ dev/ # codegen, lint, format, typecheck wrappers
|
||||
│ ├─ ops/ # backup, rotate logs, journalctl, users
|
||||
│ └─ ci/ # sanity checks usable by CI (no runners assumed)
|
||||
├─ configs/ # Centralized *.conf used by services
|
||||
│ ├─ nginx/ # (optional) reverse proxy snippets (host‑level)
|
||||
│ ├─ systemd/ # unit files for host services (no docker)
|
||||
│ ├─ security/ # fail2ban, firewall/ipset lists, tls policy
|
||||
│ └─ app/ # app‑level INI/YAML/TOML configs shared across apps
|
||||
├─ docs/ # Markdown docs (specs, ADRs, guides)
|
||||
│ ├─ 00-index.md
|
||||
│ ├─ adr/ # Architecture Decision Records
|
||||
│ ├─ specs/ # Protocol, API, tokenomics, flows
|
||||
│ ├─ runbooks/ # Ops runbooks (rotate keys, restore, etc.)
|
||||
│ └─ diagrams/ # draw.io/mermaid sources + exported PNG/SVG
|
||||
├─ packages/ # Shared libraries (language‑specific)
|
||||
│ ├─ py/ # Python packages (FastAPI, utils, protocol)
|
||||
│ │ ├─ aitbc-core/ # Protocol models, validation, common types
|
||||
│ │ ├─ aitbc-crypto/ # Key mgmt, signing, wallet primitives
|
||||
│ │ ├─ aitbc-p2p/ # Node discovery, gossip, transport
|
||||
│ │ ├─ aitbc-scheduler/ # Task slicing/merging, scoring, QoS
|
||||
│ │ └─ aitbc-sdk/ # Client SDK for Python integrations
|
||||
│ └─ js/ # Browser/Node shared libs
|
||||
│ ├─ aitbc-sdk/ # Client SDK (fetch/ws), typings
|
||||
│ └─ ui-widgets/ # Reusable UI bits for web apps
|
||||
├─ apps/ # First‑class runnable services & UIs
|
||||
│ ├─ client-web/ # Browser UI for users (requests, wallet, status)
|
||||
│ │ ├─ public/ # static assets
|
||||
│ │ ├─ src/
|
||||
│ │ │ ├─ pages/
|
||||
│ │ │ ├─ components/
|
||||
│ │ │ ├─ lib/ # uses packages/js/aitbc-sdk
|
||||
│ │ │ └─ styles/
|
||||
│ │ └─ README.md
|
||||
│ ├─ coordinator-api/ # Central API orchestrating jobs ↔ miners
|
||||
│ │ ├─ src/
|
||||
│ │ │ ├─ main.py # FastAPI entrypoint
|
||||
│ │ │ ├─ routes/
|
||||
│ │ │ ├─ services/ # matchmaking, accounting, rate‑limits
|
||||
│ │ │ ├─ domain/ # job models, receipts, accounting entities
|
||||
│ │ │ └─ storage/ # adapters (postgres, files, kv)
|
||||
│ │ ├─ migrations/ # SQL snippets (no migration framework forced)
|
||||
│ │ └─ README.md
|
||||
│ ├─ miner-node/ # Worker node daemon for GPU/CPU tasks
|
||||
│ │ ├─ src/
|
||||
│ │ │ ├─ agent/ # job runner, sandbox mgmt, health probes
|
||||
│ │ │ ├─ gpu/ # CUDA/OpenCL bindings (optional)
|
||||
│ │ │ ├─ plugins/ # task kinds (LLM, ASR, vision, etc.)
|
||||
│ │ │ └─ telemetry/ # metrics, logs, heartbeat
|
||||
│ │ └─ README.md
|
||||
│ ├─ wallet-daemon/ # Local wallet service (keys, signing, RPC)
|
||||
│ │ ├─ src/
|
||||
│ │ └─ README.md
|
||||
│ ├─ blockchain-node/ # Minimal chain (asset‑backed by compute)
|
||||
│ │ ├─ src/
|
||||
│ │ │ ├─ consensus/
|
||||
│ │ │ ├─ mempool/
|
||||
│ │ │ ├─ ledger/ # state, balances, receipts linkage
|
||||
│ │ │ └─ rpc/
|
||||
│ │ └─ README.md
|
||||
│ ├─ pool-hub/ # Client↔miners pool + matchmaking gateway
|
||||
│ │ ├─ src/
|
||||
│ │ └─ README.md
|
||||
│ ├─ marketplace-web/ # Web app for offers, bids, stats
|
||||
│ │ ├─ public/
|
||||
│ │ ├─ src/
|
||||
│ │ └─ README.md
|
||||
│ └─ explorer-web/ # Chain explorer (blocks, tx, receipts)
|
||||
│ ├─ public/
|
||||
│ ├─ src/
|
||||
│ └─ README.md
|
||||
├─ protocols/ # Canonical protocol definitions
|
||||
│ ├─ api/ # OpenAPI/JSON‑Schema for REST/WebSocket
|
||||
│ ├─ receipts/ # Job receipt schema, signing rules
|
||||
│ ├─ payouts/ # Mint/burn, staking, fees logic (spec)
|
||||
│ └─ README.md
|
||||
├─ data/ # Local dev datasets (small, sample only)
|
||||
│ ├─ fixtures/ # seed users, nodes, jobs
|
||||
│ └─ samples/
|
||||
├─ tests/ # Cross‑project test harness
|
||||
│ ├─ e2e/ # end‑to‑end flows (client→coord→miner→wallet)
|
||||
│ ├─ load/ # coordinator & miner stress scripts
|
||||
│ └─ security/ # key rotation, signature verif, replay tests
|
||||
├─ tools/ # Small CLIs, generators, mermaid->svg, etc.
|
||||
│ └─ mkdiagram
|
||||
└─ examples/ # Minimal runnable examples for integrators
|
||||
├─ quickstart-client-python/
|
||||
├─ quickstart-client-js/
|
||||
└─ receipts-sign-verify/
|
||||
```
|
||||
|
||||
## Conventions
|
||||
|
||||
- **Languages**: FastAPI/Python for backends; plain JS/TS for web; no Docker.
|
||||
- **No global venvs**: each `apps/*` and `packages/py/*` can have its own `.venv/` (created by `scripts/env/*`).
|
||||
- **Systemd over Docker**: unit files live under `configs/systemd/`, with service‑specific overrides documented in `docs/runbooks/`.
|
||||
- **Static assets** belong to each web app under `public/`. Shared UI in `packages/js/ui-widgets`.
|
||||
- **SQL**: keep raw SQL snippets in `apps/*/migrations/` (aligned with your “no migration framework” preference). Use `psqln` alias.
|
||||
- **Security**: central policy under `configs/security/` (fail2ban, ipset lists, TLS ciphers). Keys never committed.
|
||||
|
||||
## Minimal READMEs to create next
|
||||
|
||||
Create a short `README.md` in each `apps/*` and `packages/*` with:
|
||||
|
||||
1. Purpose & scope
|
||||
2. How to run (dev)
|
||||
3. Dependencies
|
||||
4. Configs consumed (from `/configs/app`)
|
||||
5. Systemd unit name & port (if applicable)
|
||||
|
||||
## Suggested first tasks (Way of least resistance)
|
||||
|
||||
1. **Bootstrap coordinator-api**: scaffold FastAPI `main.py`, `/health`, `/jobs`, `/miners` routes.
|
||||
2. **SDKs**: implement `packages/py/aitbc-sdk` & `packages/js/aitbc-sdk` with basic auth + job submit.
|
||||
3. **miner-node prototype**: heartbeat to coordinator and no‑GPU "echo" job plugin.
|
||||
4. **client-web**: basic UI to submit a test job and watch status stream.
|
||||
5. **receipts spec**: draft `protocols/receipts` and a sign/verify example in `examples/`.
|
||||
|
||||
235
docs/bootstrap/examples.md
Normal file
235
docs/bootstrap/examples.md
Normal file
@ -0,0 +1,235 @@
|
||||
# examples/ — Minimal runnable examples for integrators
|
||||
|
||||
This folder contains three self-contained, copy-pasteable starters that demonstrate how to talk to the coordinator API, submit jobs, poll status, and (optionally) verify signed receipts.
|
||||
|
||||
```
|
||||
examples/
|
||||
├─ explorer-webexplorer-web/ # (docs live elsewhere; not a runnable example)
|
||||
├─ quickstart-client-python/ # Minimal Python client
|
||||
├─ quickstart-client-js/ # Minimal Node/Browser client
|
||||
└─ receipts-sign-verify/ # Receipt format + sign/verify demos
|
||||
```
|
||||
|
||||
> Conventions: Debian 12/13, zsh, no sudo, run as root if you like. Keep env in a `.env` file. Replace example URLs/tokens with your own.
|
||||
|
||||
---
|
||||
|
||||
## 1) quickstart-client-python/
|
||||
|
||||
### What this shows
|
||||
- Create a job request
|
||||
- Submit to `COORDINATOR_URL`
|
||||
- Poll job status until `succeeded|failed|timeout`
|
||||
- Fetch the result payload
|
||||
- (Optional) Save the receipt JSON for later verification
|
||||
|
||||
### Files Windsurf should ensure exist
|
||||
- `main.py` — the tiny client (≈ 80–120 LOC)
|
||||
- `requirements.txt` — `httpx`, `python-dotenv` (and `pydantic` if you want models)
|
||||
- `.env.example` — `COORDINATOR_URL`, `API_TOKEN`
|
||||
- `README.md` — one-screen run guide
|
||||
|
||||
### How to run
|
||||
```sh
|
||||
cd examples/quickstart-client-python
|
||||
python3 -m venv .venv && source .venv/bin/activate
|
||||
pip install -r requirements.txt
|
||||
|
||||
# Prepare environment
|
||||
cp .env.example .env
|
||||
# edit .env → set COORDINATOR_URL=https://api.local.test:8443, API_TOKEN=xyz
|
||||
|
||||
# Run
|
||||
python main.py --prompt "hello compute" --timeout 60
|
||||
|
||||
# Outputs:
|
||||
# - logs to stdout
|
||||
# - writes ./out/result.json
|
||||
# - writes ./out/receipt.json (if provided by the coordinator)
|
||||
```
|
||||
|
||||
### Coordinator endpoints the code should touch
|
||||
- `POST /v1/jobs` → returns `{ job_id }`
|
||||
- `GET /v1/jobs/{job_id}` → returns `{ status, progress?, result? }`
|
||||
- `GET /v1/jobs/{job_id}/receipt` → returns `{ receipt }` (optional)
|
||||
|
||||
Keep the client resilient: exponential backoff (100ms → 2s), total wall-time cap from `--timeout`.
|
||||
|
||||
---
|
||||
|
||||
## 2) quickstart-client-js/
|
||||
|
||||
### What this shows
|
||||
- Identical flow to the Python quickstart
|
||||
- Two variants: Node (fetch via `undici`) and Browser (native `fetch`)
|
||||
|
||||
### Files Windsurf should ensure exist
|
||||
- `node/`
|
||||
- `package.json` — `undici`, `dotenv`
|
||||
- `index.js` — Node example client
|
||||
- `.env.example`
|
||||
- `README.md`
|
||||
- `browser/`
|
||||
- `index.html` — minimal UI with a Prompt box + “Run” button
|
||||
- `app.js` — client logic (no build step)
|
||||
- `README.md`
|
||||
|
||||
### How to run (Node)
|
||||
```sh
|
||||
cd examples/quickstart-client-js/node
|
||||
npm i
|
||||
cp .env.example .env
|
||||
# edit .env → set COORDINATOR_URL, API_TOKEN
|
||||
node index.js "hello compute"
|
||||
```
|
||||
|
||||
### How to run (Browser)
|
||||
```sh
|
||||
cd examples/quickstart-client-js/browser
|
||||
# Serve statically (choose one)
|
||||
python3 -m http.server 8080
|
||||
# or
|
||||
busybox httpd -f -p 8080
|
||||
```
|
||||
Open `http://localhost:8080` and paste your coordinator URL + token in the form.
|
||||
The app:
|
||||
- `POST /v1/jobs`
|
||||
- polls `GET /v1/jobs/{id}` every 1s (with a 60s guard)
|
||||
- downloads `receipt.json` if available
|
||||
|
||||
---
|
||||
|
||||
## 3) receipts-sign-verify/
|
||||
|
||||
### What this shows
|
||||
- Receipt JSON structure used by AITBC examples
|
||||
- Deterministic signing over a canonicalized JSON (RFC 8785-style or stable key order)
|
||||
- Ed25519 signing & verifying in Python and JS
|
||||
- CLI snippets to verify receipts offline
|
||||
|
||||
> If the project standardizes on another curve, swap the libs accordingly. For Ed25519:
|
||||
> - Python: `pynacl`
|
||||
> - JS: `@noble/ed25519`
|
||||
|
||||
### Files Windsurf should ensure exist
|
||||
- `spec.md` — human-readable schema (see below)
|
||||
- `python/`
|
||||
- `verify.py` — `python verify.py ./samples/receipt.json ./pubkeys/poolhub_ed25519.pub`
|
||||
- `requirements.txt` — `pynacl`
|
||||
- `js/`
|
||||
- `verify.mjs` — `node js/verify.mjs ./samples/receipt.json ./pubkeys/poolhub_ed25519.pub`
|
||||
- `package.json` — `@noble/ed25519`
|
||||
- `samples/receipt.json` — realistic sample
|
||||
- `pubkeys/poolhub_ed25519.pub` — PEM or raw 32-byte hex
|
||||
|
||||
### Minimal receipt schema (for `spec.md`)
|
||||
```jsonc
|
||||
{
|
||||
"version": "1",
|
||||
"job_id": "string",
|
||||
"client_id": "string",
|
||||
"miner_id": "string",
|
||||
"started_at": "2025-09-26T14:00:00Z",
|
||||
"completed_at": "2025-09-26T14:00:07Z",
|
||||
"units_billed": 123, // e.g., “AIToken compute units”
|
||||
"result_hash": "sha256:…", // hex
|
||||
"metadata": { "model": "…" }, // optional, stable ordering for signing
|
||||
"signature": {
|
||||
"alg": "Ed25519",
|
||||
"key_id": "poolhub-ed25519-2025-09",
|
||||
"sig": "base64url…" // signature over canonicalized receipt WITHOUT this signature object
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### CLI usage
|
||||
|
||||
**Python**
|
||||
```sh
|
||||
cd examples/receipts-sign-verify/python
|
||||
python3 -m venv .venv && source .venv/bin/activate
|
||||
pip install -r requirements.txt
|
||||
python verify.py ../samples/receipt.json ../pubkeys/poolhub_ed25519.pub
|
||||
# exit code 0 = valid, non-zero = invalid
|
||||
```
|
||||
|
||||
**Node**
|
||||
```sh
|
||||
cd examples/receipts-sign-verify/js
|
||||
npm i
|
||||
node verify.mjs ../samples/receipt.json ../pubkeys/poolhub_ed25519.pub
|
||||
```
|
||||
|
||||
**Implementation notes for Windsurf**
|
||||
- Canonicalize JSON before hashing/signing (stable key order, UTF-8, no trailing spaces).
|
||||
- Sign bytes of `sha256(canonical_json_without_signature_block)`.
|
||||
- Reject if `completed_at < started_at`, unknown `alg`, or mismatched `result_hash`.
|
||||
|
||||
---
|
||||
|
||||
## Shared environment
|
||||
|
||||
All quickstarts read the following from `.env` or in-page form fields:
|
||||
|
||||
```
|
||||
COORDINATOR_URL=https://api.local.test:8443
|
||||
API_TOKEN=replace-me
|
||||
# Optional: REQUEST_TIMEOUT_SEC=60
|
||||
```
|
||||
|
||||
HTTP headers to include:
|
||||
```
|
||||
Authorization: Bearer <API_TOKEN>
|
||||
Content-Type: application/json
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Windsurf checklist (do this automatically)
|
||||
|
||||
1. **Create folders & files**
|
||||
- `quickstart-client-python/{main.py,requirements.txt,.env.example,README.md}`
|
||||
- `quickstart-client-js/node/{index.js,package.json,.env.example,README.md}`
|
||||
- `quickstart-client-js/browser/{index.html,app.js,README.md}`
|
||||
- `receipts-sign-verify/{spec.md,samples/receipt.json,pubkeys/poolhub_ed25519.pub}`
|
||||
- `receipts-sign-verify/python/{verify.py,requirements.txt}`
|
||||
- `receipts-sign-verify/js/{verify.mjs,package.json}`
|
||||
|
||||
2. **Fill templates**
|
||||
- Implement `POST /v1/jobs`, `GET /v1/jobs/{id}`, `GET /v1/jobs/{id}/receipt` calls.
|
||||
- Poll with backoff; stop at terminal states; write `out/result.json` & `out/receipt.json`.
|
||||
|
||||
3. **Wire Ed25519 libs**
|
||||
- Python: `pynacl` verify(`public_key`, `message`, `signature`)
|
||||
- JS: `@noble/ed25519` verifySync
|
||||
|
||||
4. **Add DX niceties**
|
||||
- `.env.example` everywhere
|
||||
- `README.md` with copy-paste run steps (no global installs)
|
||||
- Minimal logging and clear non-zero exit on failure
|
||||
|
||||
5. **Smoke tests**
|
||||
- Python quickstart runs end-to-end with a mock coordinator (use our tiny FastAPI mock if available).
|
||||
- JS Node client runs with `.env`.
|
||||
- Browser client works via `http://localhost:8080`.
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
- **401 Unauthorized** → check `API_TOKEN`, CORS (browser), or missing `Authorization` header.
|
||||
- **CORS in browser** → coordinator must set:
|
||||
- `Access-Control-Allow-Origin: *` (or your host)
|
||||
- `Access-Control-Allow-Headers: Authorization, Content-Type`
|
||||
- `Access-Control-Allow-Methods: GET, POST, OPTIONS`
|
||||
- **Receipt verify fails** → most often due to non-canonical JSON or wrong public key.
|
||||
|
||||
---
|
||||
|
||||
## License & reuse
|
||||
|
||||
Keep examples MIT-licensed. Add a short header to each file:
|
||||
```
|
||||
MIT © AITBC Examples — This is demo code; use at your own risk.
|
||||
```
|
||||
|
||||
322
docs/bootstrap/explorer_web.md
Normal file
322
docs/bootstrap/explorer_web.md
Normal file
@ -0,0 +1,322 @@
|
||||
# explorer-web.md
|
||||
Chain Explorer (blocks · tx · receipts)
|
||||
|
||||
## 0) Purpose
|
||||
A lightweight, fast, dark-themed web UI to browse a minimal AITBC blockchain:
|
||||
- Latest blocks & block detail
|
||||
- Transaction list & detail
|
||||
- Address detail (balance, nonce, tx history)
|
||||
- Receipt / logs view
|
||||
- Simple search (block #, hash, tx hash, address)
|
||||
|
||||
MVP reads from **blockchain-node** (HTTP/WS API).
|
||||
No write operations.
|
||||
|
||||
---
|
||||
|
||||
## 1) Tech & Conventions
|
||||
- **Pure frontend** (no backend rendering): static HTML + JS modules + CSS.
|
||||
- **No frameworks** (keep it portable and fast).
|
||||
- **Files split**: HTML, CSS, JS in separate files (user preference).
|
||||
- **ES Modules** with strict typing via JSDoc or TypeScript (optional).
|
||||
- **Dark theme** with orange/ice accents (brand).
|
||||
- **No animations** unless explicitly requested.
|
||||
- **Time format**: UTC ISO + relative (e.g., “2m ago”).
|
||||
|
||||
---
|
||||
|
||||
## 2) Folder Layout (within workspace)
|
||||
```
|
||||
explorer-web/
|
||||
├─ public/
|
||||
│ ├─ index.html # routes: / (latest blocks)
|
||||
│ ├─ block.html # /block?hash=... or /block?number=...
|
||||
│ ├─ tx.html # /tx?hash=...
|
||||
│ ├─ address.html # /address?addr=...
|
||||
│ ├─ receipts.html # /receipts?tx=...
|
||||
│ ├─ 404.html
|
||||
│ ├─ assets/
|
||||
│ │ ├─ logo.svg
|
||||
│ │ └─ icons/*.svg
|
||||
│ ├─ css/
|
||||
│ │ ├─ base.css
|
||||
│ │ ├─ layout.css
|
||||
│ │ └─ theme-dark.css
|
||||
│ └─ js/
|
||||
│ ├─ config.js # API endpoint(s)
|
||||
│ ├─ api.js # fetch helpers
|
||||
│ ├─ store.js # simple state cache
|
||||
│ ├─ utils.js # formatters (hex, time, numbers)
|
||||
│ ├─ components/
|
||||
│ │ ├─ header.js
|
||||
│ │ ├─ footer.js
|
||||
│ │ ├─ searchbox.js
|
||||
│ │ ├─ block-table.js
|
||||
│ │ ├─ tx-table.js
|
||||
│ │ ├─ pager.js
|
||||
│ │ └─ keyvalue.js
|
||||
│ ├─ pages/
|
||||
│ │ ├─ home.js # latest blocks + mempool/heads
|
||||
│ │ ├─ block.js
|
||||
│ │ ├─ tx.js
|
||||
│ │ ├─ address.js
|
||||
│ │ └─ receipts.js
|
||||
│ └─ vendors/ # (empty; we keep it native for now)
|
||||
├─ docs/
|
||||
│ └─ explorer-web.md # this file
|
||||
└─ README.md
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3) API Contracts (read-only)
|
||||
Assume **blockchain-node** exposes:
|
||||
|
||||
### 3.1 REST (HTTP)
|
||||
- `GET /api/chain/head` → `{ number, hash, timestamp }`
|
||||
- `GET /api/blocks?limit=25&before=<blockNumber>` → `[{number,hash,parentHash,timestamp,txCount,miner,size,gasUsed}]`
|
||||
- `GET /api/block/by-number/:n` → `{ ...fullBlock }`
|
||||
- `GET /api/block/by-hash/:h` → `{ ...fullBlock }`
|
||||
- `GET /api/tx/:hash` → `{ hash, from, to, nonce, value, fee, gas, gasPrice, blockHash, blockNumber, timestamp, input }`
|
||||
- `GET /api/address/:addr` → `{ address, balance, nonce, txCount }`
|
||||
- `GET /api/address/:addr/tx?limit=25&before=<blockNumber>` → `[{hash,blockNumber,from,to,value,fee,timestamp}]`
|
||||
- `GET /api/tx/:hash/receipt` → `{ status, gasUsed, logs: [{address, topics:[...], data, index}], cumulativeGasUsed }`
|
||||
- `GET /api/search?q=...`
|
||||
- Accepts block number, block hash, tx hash, or address
|
||||
- Returns a typed result: `{ type: "block"|"tx"|"address" , key: ... }`
|
||||
|
||||
### 3.2 WebSocket (optional, later)
|
||||
- `ws://.../api/stream/heads` → emits new head `{number,hash,timestamp}`
|
||||
- `ws://.../api/stream/mempool` → emits tx previews `{hash, from, to, value, timestamp}`
|
||||
|
||||
> If the node isn’t ready, create a tiny mock server (FastAPI) consistent with these shapes (already planned in other modules).
|
||||
|
||||
---
|
||||
|
||||
## 4) Pages & UX
|
||||
|
||||
### 4.1 Header (every page)
|
||||
- Left: logo + “AITBC Explorer”
|
||||
- Center: search box (accepts block#, block/tx hash, address)
|
||||
- Right: network tag (e.g., “Local Dev”) + head block# (live)
|
||||
|
||||
### 4.2 Home `/`
|
||||
- **Latest Blocks** (table)
|
||||
- columns: `#`, `Hash (short)`, `Tx`, `Miner`, `GasUsed`, `Time`
|
||||
- infinite scroll / “Load older”
|
||||
- (optional) **Mempool feed** (compact list, toggleable)
|
||||
- Empty state: helpful instructions + sample query strings
|
||||
|
||||
### 4.3 Block Detail `/block?...`
|
||||
- Top summary (KeyValue component)
|
||||
- `Number, Hash, Parent, Miner, Timestamp, Size, GasUsed, Difficulty?`
|
||||
- Transactions table (paginated)
|
||||
- “Navigate”: Parent ↖, Next ↗, View in raw JSON (debug)
|
||||
|
||||
### 4.4 Tx Detail `/tx?hash=...`
|
||||
- Summary: `Hash, Status, Block, From, To, Value, Fee, Nonce, Gas(gasPrice)`
|
||||
- Receipt section (logs rendered as topics/data, collapsible)
|
||||
- Input data: hex preview + decode attempt (if ABI registry exists – later)
|
||||
|
||||
### 4.5 Address `/address?addr=...`
|
||||
- Summary: `Address, Balance, Nonce, TxCount`
|
||||
- Transactions list (sent/received filter)
|
||||
- (later) Token balances when chain supports it
|
||||
|
||||
### 4.6 Receipts `/receipts?tx=...`
|
||||
- Focused receipts + logs view with copy buttons
|
||||
|
||||
### 4.7 404
|
||||
- Friendly message + search
|
||||
|
||||
---
|
||||
|
||||
## 5) Components (JS modules)
|
||||
- `header.js` : builds header + binds search submit.
|
||||
- `searchbox.js` : debounced input, detects type (see utils).
|
||||
- `block-table.js` : render rows, short-hash, time-ago.
|
||||
- `tx-table.js` : similar render with direction arrows.
|
||||
- `pager.js` : simple “Load more” with event callback.
|
||||
- `keyvalue.js` : `<dl>` key/value grid for details.
|
||||
- `footer.js` : version, links.
|
||||
|
||||
---
|
||||
|
||||
## 6) Utils
|
||||
- `formatHexShort(hex, bytes=4)` → `0x1234…abcd`
|
||||
- `formatNumber(n)` with thin-space groupings
|
||||
- `formatValueWei(wei)` → AIT units when available (or plain wei)
|
||||
- `timeAgo(ts)` + `formatUTC(ts)`
|
||||
- `parseQuery()` helpers for `?hash=...`
|
||||
- `detectSearchType(q)`:
|
||||
- `0x` + 66 chars → tx/block hash
|
||||
- numeric → block number
|
||||
- `0x` + 42 → address
|
||||
- fallback → “unknown”
|
||||
|
||||
---
|
||||
|
||||
## 7) State (store.js)
|
||||
- `state.head` (number/hash/timestamp)
|
||||
- `state.cache.blocks[number] = block`
|
||||
- `state.cache.txs[hash] = tx`
|
||||
- `state.cache.address[addr] = {balance, nonce, txCount}`
|
||||
- Simple in-memory LRU eviction (optional).
|
||||
|
||||
---
|
||||
|
||||
## 8) Styling
|
||||
- `base.css`: resets, typography, links, buttons, tables.
|
||||
- `layout.css`: header/footer, grid, content widths (max 960px desktop).
|
||||
- `theme-dark.css`: colors:
|
||||
- bg: `#0b0f14`, surface: `#11161c`
|
||||
- text: `#e6eef7`
|
||||
- accent-orange: `#ff8a00`
|
||||
- accent-ice: `#a8d8ff`
|
||||
- Focus states visible. High contrast table rows on hover.
|
||||
|
||||
---
|
||||
|
||||
## 9) Error & Loading UX
|
||||
- Loading spinners (minimal).
|
||||
- Network errors: inline banner with retry.
|
||||
- Empty: clear messages & how to search.
|
||||
|
||||
---
|
||||
|
||||
## 10) Security & Hardening
|
||||
- Treat inputs as untrusted.
|
||||
- Only GETs; block any attempt to POST.
|
||||
- Strict `Content-Security-Policy` sample (for hosting):
|
||||
- `default-src 'self'; img-src 'self' data:; style-src 'self'; script-src 'self'; connect-src 'self' https://blockchain-node.local;`
|
||||
- Avoid third-party CDNs.
|
||||
|
||||
---
|
||||
|
||||
## 11) Test Plan (manual first)
|
||||
1. Home loads head + 25 latest blocks.
|
||||
2. Scroll/pager loads older batches.
|
||||
3. Block search by number + by hash.
|
||||
4. Tx search → detail + receipt.
|
||||
5. Address search → tx list.
|
||||
6. Error states when node is offline.
|
||||
7. Timezones: display UTC consistently.
|
||||
|
||||
---
|
||||
|
||||
## 12) Dev Tasks (Windsurf order of work)
|
||||
1. **Scaffold** folders & empty files.
|
||||
2. Implement `config.js` with `API_BASE`.
|
||||
3. Implement `api.js` (fetch JSON + error handling).
|
||||
4. Build `utils.js` (formatters + search detect).
|
||||
5. Build `header.js` + `footer.js`.
|
||||
6. Home page: blocks list + pager.
|
||||
7. Block detail page.
|
||||
8. Tx detail + receipts.
|
||||
9. Address page with tx list.
|
||||
10. 404 + polish (copy buttons, tiny helpers).
|
||||
11. CSS pass (dark theme).
|
||||
12. Final QA.
|
||||
|
||||
---
|
||||
|
||||
## 13) Mock Data (for offline dev)
|
||||
Place under `public/js/vendors/mock.js` (opt-in):
|
||||
- Export functions that resolve Promises with static JSON fixtures in `public/mock/*.json`.
|
||||
- Toggle via `config.js` flag `USE_MOCK=true`.
|
||||
|
||||
---
|
||||
|
||||
## 14) Minimal HTML Skeleton (example: index.html)
|
||||
```html
|
||||
<!doctype html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8" />
|
||||
<meta name="viewport" content="width=device-width,initial-scale=1" />
|
||||
<title>AITBC Explorer</title>
|
||||
<link rel="stylesheet" href="./css/base.css" />
|
||||
<link rel="stylesheet" href="./css/layout.css" />
|
||||
<link rel="stylesheet" href="./css/theme-dark.css" />
|
||||
</head>
|
||||
<body>
|
||||
<header id="app-header"></header>
|
||||
<main id="app"></main>
|
||||
<footer id="app-footer"></footer>
|
||||
<script type="module">
|
||||
import { renderHeader } from './js/components/header.js';
|
||||
import { renderFooter } from './js/components/footer.js';
|
||||
import { renderHome } from './js/pages/home.js';
|
||||
renderHeader(document.getElementById('app-header'));
|
||||
renderFooter(document.getElementById('app-footer'));
|
||||
renderHome(document.getElementById('app'));
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 15) config.js (example)
|
||||
```js
|
||||
export const CONFIG = {
|
||||
API_BASE: 'http://localhost:8545', // adapt to blockchain-node
|
||||
USE_MOCK: false,
|
||||
PAGE_SIZE: 25,
|
||||
NETWORK_NAME: 'Local Dev',
|
||||
};
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 16) API Helpers (api.js — sketch)
|
||||
```js
|
||||
import { CONFIG } from './config.js';
|
||||
|
||||
async function jget(path) {
|
||||
const res = await fetch(`${CONFIG.API_BASE}${path}`, { method: 'GET' });
|
||||
if (!res.ok) throw new Error(`HTTP ${res.status}: ${path}`);
|
||||
return res.json();
|
||||
}
|
||||
|
||||
export const api = {
|
||||
head: () => jget('/api/chain/head'),
|
||||
blocks: (limit, before) => jget(`/api/blocks?limit=${limit}&before=${before ?? ''}`),
|
||||
blockByNo: (n) => jget(`/api/block/by-number/${n}`),
|
||||
blockByHash: (h) => jget(`/api/block/by-hash/${h}`),
|
||||
tx: (hash) => jget(`/api/tx/${hash}`),
|
||||
receipt: (hash) => jget(`/api/tx/${hash}/receipt`),
|
||||
address: (addr) => jget(`/api/address/${addr}`),
|
||||
addressTx: (addr, limit, before) => jget(`/api/address/${addr}/tx?limit=${limit}&before=${before ?? ''}`),
|
||||
search: (q) => jget(`/api/search?q=${encodeURIComponent(q)}`),
|
||||
};
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 17) Performance Checklist
|
||||
- Use pagination/infinite scroll (no huge payloads).
|
||||
- Cache recent blocks/tx in-memory (store.js).
|
||||
- Avoid layout thrash: table builds via DocumentFragment.
|
||||
- Defer non-critical fetches (e.g., mempool).
|
||||
- Keep CSS small and critical.
|
||||
|
||||
---
|
||||
|
||||
## 18) Deployment
|
||||
- Serve `public/` via Nginx under `/explorer/` or own domain.
|
||||
- Set correct `connect-src` in CSP to point to blockchain-node.
|
||||
- Ensure CORS on blockchain-node for the explorer origin (read-only).
|
||||
|
||||
---
|
||||
|
||||
## 19) Roadmap (post-MVP)
|
||||
- Live head updates via WS.
|
||||
- Mempool stream view.
|
||||
- ABI registry + input decoding.
|
||||
- Token balances (when chain supports).
|
||||
- Export to CSV / JSON for tables.
|
||||
- Theming switch (dark/light).
|
||||
|
||||
---
|
||||
|
||||
468
docs/bootstrap/layout.md
Normal file
468
docs/bootstrap/layout.md
Normal file
@ -0,0 +1,468 @@
|
||||
# Layout & Frontend Guidelines (Windsurf)
|
||||
|
||||
Target: **mobile‑first**, dark theme, max content width **960px** on desktop. Reference device: **Nothing Phone 2a**.
|
||||
|
||||
---
|
||||
|
||||
## 1) Design System
|
||||
|
||||
### 1.1 Color (Dark Theme)
|
||||
- `--bg-0: #0b0f14` (page background)
|
||||
- `--bg-1: #11161c` (cards/sections)
|
||||
- `--tx-0: #e6edf3` (primary text)
|
||||
- `--tx-1: #a7b3be` (muted)
|
||||
- `--pri: #ff7a1a` (accent orange)
|
||||
- `--ice: #b9ecff` (ice accent)
|
||||
- `--ok: #3ddc97`
|
||||
- `--warn: #ffcc00`
|
||||
- `--err: #ff4d4d`
|
||||
|
||||
### 1.2 Typography
|
||||
- Base font-size: **16px** (mobile), scale up at desktop.
|
||||
- Font stack: System UI (`-apple-system, Segoe UI, Roboto, Inter, Arial, sans-serif`).
|
||||
- Line-height: 1.5 body, 1.2 headings.
|
||||
|
||||
### 1.3 Spacing (8‑pt grid)
|
||||
- `--s-1: 4px`, `--s-2: 8px`, `--s-3: 12px`, `--s-4: 16px`, `--s-5: 24px`, `--s-6: 32px`, `--s-7: 48px`, `--s-8: 64px`.
|
||||
|
||||
### 1.4 Radius & Shadow
|
||||
- Radius: `--r-1: 8px`, `--r-2: 16px`.
|
||||
- Shadow (subtle): `0 4px 20px rgba(0,0,0,.25)`.
|
||||
|
||||
---
|
||||
|
||||
## 2) Grid & Layout
|
||||
|
||||
### 2.1 Container
|
||||
- **Mobile‑first**: full‑bleed padding.
|
||||
- Desktop container: **max‑width: 960px**, centered.
|
||||
- Side gutters: 16px (mobile), 24px (tablet), 32px (desktop).
|
||||
|
||||
**Breakpoint summary**
|
||||
| Token | Min width | Container behaviour | Notes |
|
||||
| --- | --- | --- | --- |
|
||||
| `--bp-sm` | 360px | Fluid | Single-column layouts prioritise readability.
|
||||
| `--bp-md` | 480px | Fluid | Allow two-up cards or media/text pairings when needed.
|
||||
| `--bp-lg` | 768px | Max-width 90% (capped at 960px) | Stage tablet/landscape experiences before full desktop.
|
||||
| `--bp-xl` | 960px | Fixed 960px max width | Full desktop grid, persistent side rails allowed.
|
||||
|
||||
Always respect `env(safe-area-inset-*)` for notch devices (use helpers like `.safe-b`).
|
||||
|
||||
### 2.2 Columns
|
||||
- 12‑column grid on screens ≥ **960px**.
|
||||
- Column gutter: 16px (mobile), 24px (≥960px).
|
||||
- Utility classes (examples):
|
||||
- `.row { display:grid; grid-template-columns: repeat(12, 1fr); gap: var(--gutter); }`
|
||||
- `.col-12, .col-6, .col-4, .col-3` for common spans on desktop.
|
||||
- Mobile stacks by default; use responsive helpers at breakpoints.
|
||||
|
||||
### 2.3 Breakpoints (Nothing Phone 2a aware)
|
||||
- `--bp-sm: 360px` (small phones)
|
||||
- `--bp-md: 480px` (Nothing 2a width in portrait ~ 412–480 CSS px)
|
||||
- `--bp-lg: 768px` (tablets / landscape phones)
|
||||
- `--bp-xl: 960px` (desktop container)
|
||||
|
||||
**Mobile layout rules:**
|
||||
- Navigation collapses to icon buttons with overflow menu at `--bp-sm`.
|
||||
- Multi-column sections stack; keep vertical rhythm using `var(--s-6)`.
|
||||
- Sticky headers take 56px height; ensure content uses `.safe-b` for bottom insets.
|
||||
|
||||
**Desktop enhancements:**
|
||||
- Activate `.row` grid with `.col-*` spans at `--bp-xl`.
|
||||
- Introduce side rail for filters or secondary nav (span 3 or 4 columns).
|
||||
- Increase typographic scale by 1 step (`clamp` already handles this).
|
||||
|
||||
> **Rule:** Build for < `--bp-lg` first; enhance progressively at `--bp-lg` and `--bp-xl`.
|
||||
|
||||
---
|
||||
|
||||
## 3) Page Chrome
|
||||
|
||||
### 3.1 Header
|
||||
- Sticky top, height 56–64px.
|
||||
- Left: brand; Right: primary action or menu.
|
||||
- Translucent on scroll (backdrop‑filter), solid at top.
|
||||
|
||||
### 3.2 Footer
|
||||
- Thin bar; meta links. Uses ice accent for separators.
|
||||
|
||||
### 3.3 Main
|
||||
- Vertical rhythm: sections spaced by `var(--s-7)` (mobile) / `var(--s-8)` (desktop).
|
||||
- Cards: background `var(--bg-1)`, radius `var(--r-2)`, padding `var(--s-6)`.
|
||||
|
||||
---
|
||||
|
||||
## 4) CSS File Strategy (per page)
|
||||
|
||||
**Every HTML page ships its own CSS file**, plus shared layers:
|
||||
|
||||
- `/css/base.css` — resets, variables, typography, utility helpers.
|
||||
- `/css/components.css` — buttons, inputs, cards, modals, toast.
|
||||
- `/css/layout.css` — grid/container/header/footer.
|
||||
- `/css/pages/<page>.css` — page‑specific rules (one per HTML page).
|
||||
|
||||
**Naming** (BEM‑ish): `block__elem--mod`. Avoid nesting >2 levels.
|
||||
|
||||
**Example includes:**
|
||||
```html
|
||||
<link rel="stylesheet" href="/css/base.css">
|
||||
<link rel="stylesheet" href="/css/components.css">
|
||||
<link rel="stylesheet" href="/css/layout.css">
|
||||
<link rel="stylesheet" href="/css/pages/dashboard.css">
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5) Utilities (recommended)
|
||||
- Spacing: `.mt-4`, `.mb-6`, `.px-4`, `.py-6` (map to spacing scale).
|
||||
- Flex/Grid: `.flex`, `.grid`, `.ai-c`, `.jc-b`.
|
||||
- Display: `.hide-sm`, `.hide-lg` using media queries.
|
||||
|
||||
---
|
||||
|
||||
## 6) Toast Messages (center bottom)
|
||||
|
||||
**Position:** centered at bottom above safe‑area insets.
|
||||
|
||||
**Behavior:**
|
||||
- Appear for 3–5s; pause on hover; dismiss on click.
|
||||
- Max width 90% on mobile, 420px desktop.
|
||||
- Elevation + subtle slide/fade.
|
||||
|
||||
**Structure:**
|
||||
```html
|
||||
<div id="toast-root" aria-live="polite" aria-atomic="true"></div>
|
||||
```
|
||||
|
||||
**Styles (concept):**
|
||||
```css
|
||||
#toast-root { position: fixed; left: 50%; bottom: max(16px, env(safe-area-inset-bottom)); transform: translateX(-50%); z-index: 9999; }
|
||||
.toast { background: var(--bg-1); color: var(--tx-0); border: 1px solid rgba(185,236,255,.2); padding: var(--s-5) var(--s-6); border-radius: var(--r-2); box-shadow: 0 10px 30px rgba(0,0,0,.35); margin-bottom: var(--s-4); max-width: min(420px, 90vw); }
|
||||
.toast--ok { border-color: rgba(61,220,151,.35); }
|
||||
.toast--warn { border-color: rgba(255,204,0,.35); }
|
||||
.toast--err { border-color: rgba(255,77,77,.35); }
|
||||
```
|
||||
|
||||
**JS API (minimal):**
|
||||
```js
|
||||
function showToast(msg, type = "ok", ms = 3500) {
|
||||
const root = document.getElementById("toast-root");
|
||||
const el = document.createElement("div");
|
||||
el.className = `toast toast--${type}`;
|
||||
el.role = "status";
|
||||
el.textContent = msg;
|
||||
root.appendChild(el);
|
||||
const t = setTimeout(() => el.remove(), ms);
|
||||
el.addEventListener("mouseenter", () => clearTimeout(t));
|
||||
el.addEventListener("click", () => el.remove());
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7) Browser Notifications (system tray)
|
||||
|
||||
**When to use:** Only for important, user‑initiated events (e.g., a new match, message, or scheduled session start). Always provide an in‑app alternative (toast/modal) for users who deny permission.
|
||||
|
||||
**Permission flow:**
|
||||
```js
|
||||
async function ensureNotifyPermission() {
|
||||
if (!("Notification" in window)) return false;
|
||||
if (Notification.permission === "granted") return true;
|
||||
if (Notification.permission === "denied") return false;
|
||||
const res = await Notification.requestPermission();
|
||||
return res === "granted";
|
||||
}
|
||||
```
|
||||
|
||||
**Send notification:**
|
||||
```js
|
||||
function notify(opts) {
|
||||
if (Notification.permission !== "granted") return;
|
||||
const n = new Notification(opts.title || "Update", {
|
||||
body: opts.body || "",
|
||||
icon: opts.icon || "/icons/notify.png",
|
||||
tag: opts.tag || "app-event",
|
||||
requireInteraction: !!opts.sticky
|
||||
});
|
||||
if (opts.onclick) n.addEventListener("click", opts.onclick);
|
||||
}
|
||||
```
|
||||
|
||||
**Pattern:**
|
||||
```js
|
||||
const ok = await ensureNotifyPermission();
|
||||
if (ok) notify({ title: "Match window opens soon", body: "Starts in 10 min" });
|
||||
else showToast("Enable notifications in settings to get alerts", "warn");
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 8) Forms & Inputs
|
||||
- Hit target ≥ 44×44px, labels always visible.
|
||||
- Focus ring: `outline: 2px solid var(--ice)`.
|
||||
- Validation: inline text in `--warn/--err`; never only color.
|
||||
|
||||
---
|
||||
|
||||
## 9) Performance
|
||||
- CSS: ship **only** what a page uses (per‑page CSS). Avoid giant bundles.
|
||||
- Images: `loading="lazy"`, responsive sizes; WebP/AVIF first.
|
||||
- Fonts: use system fonts; if custom, `font-display: swap`.
|
||||
|
||||
---
|
||||
|
||||
## 10) Accessibility
|
||||
- Ensure color contrast ratios meet WCAG AA standards (e.g., 4.5:1 for text and 3:1 for large text).
|
||||
- Use semantic HTML elements (<header>, <nav>, <main>, etc.) and ARIA attributes for dynamic content.
|
||||
- Support keyboard navigation with logical tab order and visible focus indicators.
|
||||
- Test for screen readers, providing text alternatives for images and ensuring forms are labeled correctly.
|
||||
- Adapt layouts for assistive technologies using media queries and flexible components.
|
||||
|
||||
## 11) Accessibility Integration
|
||||
- Apply utility classes for focus states (e.g., :focus-visible with visible outline).
|
||||
- Use ARIA roles for custom widgets and ensure all content is perceivable, operable, understandable, and robust.
|
||||
|
||||
## 12) Breakpoint Examples
|
||||
```css
|
||||
/* Mobile-first defaults here */
|
||||
@media (min-width: 768px) { /* tablets / landscape phones */ }
|
||||
@media (min-width: 960px) { /* desktop grid & container */ }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 13) Checklist (per page)
|
||||
- [ ] Uses `/css/pages/<page>.css` + shared layers
|
||||
- [ ] Container max‑width 960px, centered; gutters follow breakpoint summary
|
||||
- [ ] Mobile‑first; tests on Nothing Phone 2a
|
||||
- [ ] Toasts render center‑bottom
|
||||
- [ ] Notifications gated by permission & mirrored with toasts
|
||||
- [ ] A11y pass (headings order, labels, focus, contrast)
|
||||
|
||||
|
||||
|
||||
---
|
||||
|
||||
## Appendix A — `/css/base.css`
|
||||
```css
|
||||
/* Base: variables, reset, typography, utilities */
|
||||
:root{
|
||||
--bg-0:#0b0f14; --bg-1:#11161c;
|
||||
--tx-0:#e6edf3; --tx-1:#a7b3be;
|
||||
--pri:#ff7a1a; --ice:#b9ecff;
|
||||
--ok:#3ddc97; --warn:#ffcc00; --err:#ff4d4d;
|
||||
|
||||
--s-1:4px; --s-2:8px; --s-3:12px; --s-4:16px;
|
||||
--s-5:24px; --s-6:32px; --s-7:48px; --s-8:64px;
|
||||
--r-1:8px; --r-2:16px;
|
||||
--gutter:16px;
|
||||
}
|
||||
|
||||
@media (min-width:960px){ :root{ --gutter:24px; } }
|
||||
|
||||
/* Reset */
|
||||
*{ box-sizing:border-box; }
|
||||
html,body{ height:100%; }
|
||||
html{ color-scheme:dark; }
|
||||
body{
|
||||
margin:0; background:var(--bg-0); color:var(--tx-0);
|
||||
font:16px/1.5 -apple-system, Segoe UI, Roboto, Inter, Arial, sans-serif;
|
||||
-webkit-font-smoothing:antialiased; -moz-osx-font-smoothing:grayscale;
|
||||
}
|
||||
img,svg,video{ max-width:100%; height:auto; display:block; }
|
||||
button,input,select,textarea{ font:inherit; color:inherit; }
|
||||
:focus-visible{ outline:2px solid var(--ice); outline-offset:2px; }
|
||||
|
||||
/* Typography */
|
||||
h1{ font-size:clamp(24px, 3.5vw, 36px); line-height:1.2; margin:0 0 var(--s-4); }
|
||||
h2{ font-size:clamp(20px, 3vw, 28px); line-height:1.25; margin:0 0 var(--s-3); }
|
||||
h3{ font-size:clamp(18px, 2.5vw, 22px); line-height:1.3; margin:0 0 var(--s-2); }
|
||||
p{ margin:0 0 var(--s-4); color:var(--tx-1); }
|
||||
|
||||
/* Links */
|
||||
a{ color:var(--ice); text-decoration:none; }
|
||||
a:hover{ text-decoration:underline; }
|
||||
|
||||
/* Utilities */
|
||||
.container{ width:100%; max-width:960px; margin:0 auto; padding:0 var(--gutter); }
|
||||
.flex{ display:flex; } .grid{ display:grid; }
|
||||
.ai-c{ align-items:center; } .jc-b{ justify-content:space-between; }
|
||||
.mt-4{ margin-top:var(--s-4);} .mb-6{ margin-bottom:var(--s-6);} .px-4{ padding-left:var(--s-4); padding-right:var(--s-4);} .py-6{ padding-top:var(--s-6); padding-bottom:var(--s-6);}
|
||||
.hide-sm{ display:none; }
|
||||
@media (min-width:960px){ .hide-lg{ display:none; } .hide-sm{ display:initial; } }
|
||||
|
||||
/* Safe area helpers */
|
||||
.safe-b{ padding-bottom:max(var(--s-4), env(safe-area-inset-bottom)); }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Appendix B — `/css/layout.css`
|
||||
```css
|
||||
/* Grid, header, footer, sections */
|
||||
.row{ display:grid; grid-template-columns:repeat(12, 1fr); gap:var(--gutter); }
|
||||
|
||||
/* Mobile-first: stack columns */
|
||||
[class^="col-"]{ grid-column:1/-1; }
|
||||
|
||||
@media (min-width:960px){
|
||||
.col-12{ grid-column:span 12; }
|
||||
.col-6{ grid-column:span 6; }
|
||||
.col-4{ grid-column:span 4; }
|
||||
.col-3{ grid-column:span 3; }
|
||||
}
|
||||
|
||||
header.site{
|
||||
position:sticky; top:0; z-index:50;
|
||||
backdrop-filter:saturate(1.2) blur(8px);
|
||||
background:color-mix(in oklab, var(--bg-0) 85%, transparent);
|
||||
border-bottom:1px solid rgba(185,236,255,.12);
|
||||
}
|
||||
header.site .inner{ height:64px; }
|
||||
|
||||
footer.site{
|
||||
margin-top:var(--s-8);
|
||||
border-top:1px solid rgba(185,236,255,.12);
|
||||
background:var(--bg-0);
|
||||
}
|
||||
footer.site .inner{ height:56px; font-size:14px; color:var(--tx-1); }
|
||||
|
||||
main section{ margin: var(--s-7) 0; }
|
||||
.card{
|
||||
background:var(--bg-1);
|
||||
border:1px solid rgba(185,236,255,.12);
|
||||
border-radius:var(--r-2);
|
||||
padding:var(--s-6);
|
||||
box-shadow:0 4px 20px rgba(0,0,0,.25);
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Appendix C — `/css/components.css`
|
||||
```css
|
||||
/* Buttons */
|
||||
.btn{ display:inline-flex; align-items:center; justify-content:center; gap:8px;
|
||||
border-radius:var(--r-1); padding:10px 16px; border:1px solid transparent; cursor:pointer;
|
||||
background:var(--pri); color:#111; font-weight:600; text-align:center; }
|
||||
.btn:hover{ filter:brightness(1.05); }
|
||||
.btn--subtle{ background:transparent; color:var(--tx-0); border-color:rgba(185,236,255,.2); }
|
||||
.btn--ghost{ background:transparent; color:var(--pri); border-color:transparent; }
|
||||
.btn:disabled{ opacity:.6; cursor:not-allowed; }
|
||||
|
||||
/* Inputs */
|
||||
.input{ width:100%; background:#0e1319; color:var(--tx-0);
|
||||
border:1px solid rgba(185,236,255,.18); border-radius:var(--r-1);
|
||||
padding:12px 14px; }
|
||||
.input::placeholder{ color:#6f7b86; }
|
||||
|
||||
/* Badges */
|
||||
.badge{ display:inline-block; padding:2px 8px; border-radius:999px; font-size:12px; border:1px solid rgba(185,236,255,.2); }
|
||||
.badge--ok{ color:#0c2; border-color:rgba(61,220,151,.4); }
|
||||
.badge--warn{ color:#fc0; border-color:rgba(255,204,0,.4); }
|
||||
.badge--err{ color:#f66; border-color:rgba(255,77,77,.4); }
|
||||
|
||||
/* Toasts */
|
||||
#toast-root{ position:fixed; left:50%; bottom:max(16px, env(safe-area-inset-bottom)); transform:translateX(-50%); z-index:9999; }
|
||||
.toast{ background:var(--bg-1); color:var(--tx-0);
|
||||
border:1px solid rgba(185,236,255,.2); padding:var(--s-5) var(--s-6);
|
||||
border-radius:var(--r-2); box-shadow:0 10px 30px rgba(0,0,0,.35);
|
||||
margin-bottom:var(--s-4); max-width:min(420px, 90vw);
|
||||
opacity:0; translate:0 8px; animation:toast-in .24s ease-out forwards; }
|
||||
.toast--ok{ border-color:rgba(61,220,151,.35); }
|
||||
.toast--warn{ border-color:rgba(255,204,0,.35); }
|
||||
.toast--err{ border-color:rgba(255,77,77,.35); }
|
||||
@keyframes toast-in{ to{ opacity:1; translate:0 0; } }
|
||||
```
|
||||
|
||||
|
||||
|
||||
---
|
||||
|
||||
## Appendix D — `/js/toast.js`
|
||||
```js
|
||||
export function showToast(msg, type = "ok", ms = 3500) {
|
||||
const root = document.getElementById("toast-root") || (() => {
|
||||
const r = document.createElement("div");
|
||||
r.id = "toast-root";
|
||||
document.body.appendChild(r);
|
||||
return r;
|
||||
})();
|
||||
|
||||
const el = document.createElement("div");
|
||||
el.className = `toast toast--${type}`;
|
||||
el.role = "status";
|
||||
el.textContent = msg;
|
||||
root.appendChild(el);
|
||||
|
||||
const t = setTimeout(() => el.remove(), ms);
|
||||
el.addEventListener("mouseenter", () => clearTimeout(t));
|
||||
el.addEventListener("click", () => el.remove());
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Appendix E — `/js/notify.js`
|
||||
```js
|
||||
export async function ensureNotifyPermission() {
|
||||
if (!("Notification" in window)) return false;
|
||||
if (Notification.permission === "granted") return true;
|
||||
if (Notification.permission === "denied") return false;
|
||||
const res = await Notification.requestPermission();
|
||||
return res === "granted";
|
||||
}
|
||||
|
||||
export function notify(opts) {
|
||||
if (Notification.permission !== "granted") return;
|
||||
const n = new Notification(opts.title || "Update", {
|
||||
body: opts.body || "",
|
||||
icon: opts.icon || "/icons/notify.png",
|
||||
tag: opts.tag || "app-event",
|
||||
requireInteraction: !!opts.sticky
|
||||
});
|
||||
if (opts.onclick) n.addEventListener("click", opts.onclick);
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
|
||||
---
|
||||
|
||||
## Appendix F — `/css/pages/dashboard.css`
|
||||
```css
|
||||
/* Dashboard specific styles */
|
||||
.dashboard-header{
|
||||
margin-bottom:var(--s-6);
|
||||
display:flex; align-items:center; justify-content:space-between;
|
||||
}
|
||||
.dashboard-header h1{ font-size:28px; color:var(--pri); }
|
||||
|
||||
.stats-grid{
|
||||
display:grid;
|
||||
gap:var(--s-5);
|
||||
grid-template-columns:repeat(auto-fit, minmax(160px, 1fr));
|
||||
}
|
||||
.stat-card{
|
||||
background:var(--bg-1);
|
||||
border:1px solid rgba(185,236,255,.12);
|
||||
border-radius:var(--r-2);
|
||||
padding:var(--s-5);
|
||||
text-align:center;
|
||||
}
|
||||
.stat-card h2{ margin:0; font-size:20px; color:var(--ice); }
|
||||
.stat-card p{ margin:var(--s-2) 0 0; color:var(--tx-1); font-size:14px; }
|
||||
|
||||
.activity-feed{
|
||||
margin-top:var(--s-7);
|
||||
}
|
||||
.activity-item{
|
||||
border-bottom:1px solid rgba(185,236,255,.1);
|
||||
padding:var(--s-4) 0;
|
||||
display:flex; align-items:center; gap:var(--s-4);
|
||||
}
|
||||
.activity-item:last-child{ border-bottom:none; }
|
||||
.activity-icon{ width:32px; height:32px; border-radius:50%; background:var(--pri); display:flex; align-items:center; justify-content:center; color:#111; font-size:16px; }
|
||||
.activity-text{ flex:1; font-size:14px; color:var(--tx-0); }
|
||||
.activity-time{ font-size:12px; color:var(--tx-1); }
|
||||
237
docs/bootstrap/marketplace_web.md
Normal file
237
docs/bootstrap/marketplace_web.md
Normal file
@ -0,0 +1,237 @@
|
||||
# marketplace-web
|
||||
|
||||
Web app for listing compute **offers**, placing **bids**, and viewing **market stats** in the AITBC stack.
|
||||
Stage-aware: works **now** against a mock API, later switches to **coordinator/pool-hub/blockchain** endpoints without touching the UI.
|
||||
|
||||
## Goals
|
||||
|
||||
1. Browse offers (GPU/CPU, price per token, location, queue, latency).
|
||||
2. Place/manage bids (instant or scheduled).
|
||||
3. Watch market stats (price trends, filled volume, miner capacity).
|
||||
4. Wallet view (balance, recent tx; read-only first).
|
||||
5. Internationalization (EU langs later), dark theme, 960px layout.
|
||||
|
||||
## Tech/Structure (Windsurf-friendly)
|
||||
|
||||
- Vanilla **TypeScript + Vite** (no React), separate JS/CSS files.
|
||||
- File layout (desktop 960px grid, mobile-first CSS, dark theme):
|
||||
|
||||
```
|
||||
marketplace-web/
|
||||
├─ public/
|
||||
│ ├─ icons/ # favicons, app icons
|
||||
│ └─ i18n/ # JSON dictionaries (en/… later)
|
||||
├─ src/
|
||||
│ ├─ app.ts # app bootstrap/router
|
||||
│ ├─ router.ts # hash-router (/, /offer/:id, /bids, /stats, /wallet)
|
||||
│ ├─ api/
|
||||
│ │ ├─ http.ts # fetch wrapper + baseURL swap (mock → real)
|
||||
│ │ └─ marketplace.ts # typed API calls
|
||||
│ ├─ store/
|
||||
│ │ ├─ state.ts # global app state (signals or tiny pubsub)
|
||||
│ │ └─ types.ts # shared types/interfaces
|
||||
│ ├─ views/
|
||||
│ │ ├─ HomeView.ts
|
||||
│ │ ├─ OfferDetailView.ts
|
||||
│ │ ├─ BidsView.ts
|
||||
│ │ ├─ StatsView.ts
|
||||
│ │ └─ WalletView.ts
|
||||
│ ├─ components/
|
||||
│ │ ├─ OfferCard.ts
|
||||
│ │ ├─ BidForm.ts
|
||||
│ │ ├─ Table.ts
|
||||
│ │ ├─ Sparkline.ts # minimal chart (no external lib)
|
||||
│ │ └─ Toast.ts
|
||||
│ ├─ styles/
|
||||
│ │ ├─ base.css # reset, variables, dark theme
|
||||
│ │ ├─ layout.css # 960px grid, sections, header/footer
|
||||
│ │ └─ components.css
|
||||
│ └─ util/
|
||||
│ ├─ format.ts # fmt token, price, time
|
||||
│ ├─ validate.ts # input validation
|
||||
│ └─ i18n.ts # simple t() loader
|
||||
├─ index.html
|
||||
├─ vite.config.ts
|
||||
└─ README.md
|
||||
```
|
||||
|
||||
## Routes (Hash router)
|
||||
|
||||
- `/` — Offer list + filters
|
||||
- `/offer/:id` — Offer details + **BidForm**
|
||||
- `/bids` — User bids (open, filled, cancelled)
|
||||
- `/stats` — Price/volume/capacity charts
|
||||
- `/wallet` — Balance + last 10 tx (read-only)
|
||||
|
||||
## UI/UX Spec
|
||||
|
||||
- **Dark theme**, accent = ice-blue/white outlines (fits OIB style).
|
||||
- **960px max width** desktop, mobile-first, Nothing Phone 2a as reference.
|
||||
- **Toast** bottom-center for actions.
|
||||
- Forms: no animations, clear validation, disable buttons during submit.
|
||||
|
||||
## Data Types (minimal)
|
||||
|
||||
```ts
|
||||
type TokenAmount = `${number}`; // keep as string to avoid FP errors
|
||||
type PricePerToken = `${number}`;
|
||||
|
||||
interface Offer {
|
||||
id: string;
|
||||
provider: string; // miner or pool label
|
||||
hw: { gpu: string; vramGB?: number; cpu?: string };
|
||||
region: string; // e.g., eu-central
|
||||
queue: number; // jobs waiting
|
||||
latencyMs: number;
|
||||
price: PricePerToken; // AIToken per 1k tokens processed
|
||||
minTokens: number;
|
||||
maxTokens: number;
|
||||
updatedAt: string;
|
||||
}
|
||||
|
||||
interface BidInput {
|
||||
offerId: string;
|
||||
tokens: number; // requested tokens to process
|
||||
maxPrice: PricePerToken; // cap
|
||||
}
|
||||
|
||||
interface Bid extends BidInput {
|
||||
id: string;
|
||||
status: "open" | "filled" | "cancelled" | "expired";
|
||||
createdAt: string;
|
||||
filledTokens?: number;
|
||||
avgFillPrice?: PricePerToken;
|
||||
}
|
||||
|
||||
interface MarketStats {
|
||||
ts: string[];
|
||||
medianPrice: number[]; // per interval
|
||||
filledVolume: number[]; // tokens
|
||||
capacity: number[]; // available tokens
|
||||
}
|
||||
|
||||
interface Wallet {
|
||||
address: string;
|
||||
balance: TokenAmount;
|
||||
recent: Array<{ id: string; kind: "mint"|"spend"|"refund"; amount: TokenAmount; at: string }>;
|
||||
}
|
||||
```
|
||||
|
||||
## Mock API (Stage 0)
|
||||
|
||||
Base URL: `/.mock` (served via Vite dev middleware or static JSON)
|
||||
|
||||
- `GET /.mock/offers.json` → `Offer[]`
|
||||
- `GET /.mock/offers/:id.json` → `Offer`
|
||||
- `POST /.mock/bids` (body `BidInput`) → `Bid`
|
||||
- `GET /.mock/bids.json` → `Bid[]`
|
||||
- `GET /.mock/stats.json` → `MarketStats`
|
||||
- `GET /.mock/wallet.json` → `Wallet`
|
||||
|
||||
Switch to real endpoints by changing **`BASE_URL`** in `api/http.ts`.
|
||||
|
||||
## Real API (Stage 2/3/4 wiring)
|
||||
|
||||
When coordinator/pool-hub/blockchain are ready:
|
||||
|
||||
- `GET /api/market/offers` → `Offer[]`
|
||||
- `GET /api/market/offers/:id` → `Offer`
|
||||
- `POST /api/market/bids` → create bid, returns `Bid`
|
||||
- `GET /api/market/bids?owner=<wallet>` → `Bid[]`
|
||||
- `GET /api/market/stats?range=24h|7d|30d` → `MarketStats`
|
||||
- `GET /api/wallet/summary?addr=<wallet>` → `Wallet`
|
||||
|
||||
Auth header (later): `Authorization: Bearer <session-or-wallet-token>`.
|
||||
|
||||
## State & Caching
|
||||
|
||||
- In-memory store (`store/state.ts`) with tiny pub/sub.
|
||||
- Offer list cached 30s; stats cached 60s; bust on route change if stale.
|
||||
- Optimistic UI for **Bid** create; reconcile on server response.
|
||||
|
||||
## Filters (Home)
|
||||
|
||||
- Region (multi)
|
||||
- HW capability (GPU model, min VRAM, CPU present)
|
||||
- Price range (slider)
|
||||
- Latency max (ms)
|
||||
- Queue max
|
||||
|
||||
All filters are client-side over fetched offers (server-side later).
|
||||
|
||||
## Validation Rules
|
||||
|
||||
- `BidInput.tokens` ∈ `[offer.minTokens, offer.maxTokens]`.
|
||||
- `maxPrice >= offer.price` for instant fill hint; otherwise place as limit.
|
||||
- Warn if `queue > threshold` or `latencyMs > threshold`.
|
||||
|
||||
## Security Notes (Web)
|
||||
|
||||
- Input sanitize; never eval.
|
||||
- CSRF not needed for read-only; for POST use standard token once auth exists.
|
||||
- Rate-limit POST (server).
|
||||
- Display wallet **read-only** unless signing is integrated (later via wallet-daemon).
|
||||
|
||||
## i18n
|
||||
|
||||
- `public/i18n/en.json` as root.
|
||||
- `util/i18n.ts` provides `t(key, params?)`.
|
||||
- Keys only (no concatenated sentences). EU languages can be added later via your i18n tool.
|
||||
|
||||
## Accessibility
|
||||
|
||||
- Semantic HTML, label every input, focus states visible.
|
||||
- Keyboard: Tab order, Enter submits forms, Esc closes dialogs.
|
||||
- Color contrast AA in dark theme.
|
||||
|
||||
## Minimal Styling Rules
|
||||
|
||||
- `styles/base.css`: CSS variables for colors, spacing, radius.
|
||||
- `styles/layout.css`: header, main container (max-width: 960px), grid for cards.
|
||||
- `styles/components.css`: OfferCard, Table, Buttons, Toast.
|
||||
|
||||
## Testing
|
||||
|
||||
- Manual first:
|
||||
- Offers list loads, filters act.
|
||||
- Place bid with edge values.
|
||||
- Stats sparkline renders with missing points.
|
||||
- Later: Vitest for `util/` + `api/` modules.
|
||||
|
||||
## Env/Config
|
||||
|
||||
- `VITE_API_BASE` → mock or real.
|
||||
- `VITE_DEFAULT_REGION` → optional default filter.
|
||||
- `VITE_FEATURE_WALLET=readonly|disabled`.
|
||||
|
||||
## Build/Run
|
||||
|
||||
```
|
||||
# dev
|
||||
npm i
|
||||
npm run dev
|
||||
|
||||
# build
|
||||
npm run build
|
||||
npm run preview
|
||||
```
|
||||
|
||||
## Migration Checklist (Mock → Real)
|
||||
|
||||
1. Replace `VITE_API_BASE` with coordinator gateway URL.
|
||||
2. Enable auth header injection when session is present.
|
||||
3. Wire `/wallet` to wallet-daemon read endpoint.
|
||||
4. Swap stats source to real telemetry.
|
||||
5. Keep the same types; server must honor them.
|
||||
|
||||
## Open Tasks
|
||||
|
||||
- [ ] Create file skeletons per structure above.
|
||||
- [ ] Add mock JSON under `public/.mock/`.
|
||||
- [ ] Implement OfferCard + filters.
|
||||
- [ ] Implement BidForm with validation + optimistic UI.
|
||||
- [ ] Implement StatsView with `Sparkline` (no external chart lib).
|
||||
- [ ] Wire `VITE_API_BASE` switch.
|
||||
- [ ] Basic a11y pass + dark theme polish.
|
||||
- [ ] Wallet view (read-only).
|
||||
|
||||
423
docs/bootstrap/miner.md
Normal file
423
docs/bootstrap/miner.md
Normal file
@ -0,0 +1,423 @@
|
||||
# AITBC Miner – Windsurf Boot & Ops Guide
|
||||
|
||||
A minimal, production‑lean starter for bringing an **AITBC compute‑miner** online on Debian (Bookworm/Trixie). It is optimized for NVIDIA GPUs with CUDA support, yet safe to run CPU‑only. The miner polls jobs from a central **Coordinator API** on behalf of clients, executes AI workloads, generates proofs, and earns tokens. Payments are credited to the configured wallet.
|
||||
|
||||
---
|
||||
|
||||
## Flow Diagram
|
||||
```
|
||||
[ Client ] → submit job → [ Coordinator API ] → dispatch → [ Miner ] → proof → [ Coordinator API ] → credit → [ Wallet ]
|
||||
```
|
||||
- **Client**: User or application requesting AI computation.
|
||||
- **Coordinator API**: Central dispatcher that manages jobs and miners.
|
||||
- **Miner**: Executes the AI workload, generates proofs, and submits results.
|
||||
- **Wallet**: Receives token rewards for completed jobs.
|
||||
|
||||
---
|
||||
|
||||
## Quickstart: Windsurf Fast Boot
|
||||
The minimal info Windsurf needs to spin everything up quickly:
|
||||
|
||||
1. **Minimal Config Values** (edit `/etc/aitbc/miner.conf`):
|
||||
- `COORD_URL` (use mock for local dev): `http://127.0.0.1:8080`
|
||||
- `WALLET_ADDR` (any test string for mock): `wallet_demo`
|
||||
- `API_KEY` (mock ignores, still set one): `CHANGE_ME`
|
||||
- `MINER_ID`: `$(hostname)-gpu0`
|
||||
2. **Dependencies**
|
||||
```bash
|
||||
apt update
|
||||
apt install -y python3 python3-venv python3-pip curl jq ca-certificates git pciutils lsb-release
|
||||
# mock coordinator deps
|
||||
pip install fastapi uvicorn
|
||||
```
|
||||
- **GPU optional**: ensure `nvidia-smi` works for CUDA path.
|
||||
3. **Boot the Mock Coordinator** (new terminal):
|
||||
```bash
|
||||
uvicorn mock_coordinator:app --reload --host 0.0.0.0 --port 8080
|
||||
```
|
||||
4. **Install & Start Miner**
|
||||
```bash
|
||||
/root/scripts/aitbc-miner/install_miner.sh
|
||||
systemctl start aitbc-miner.service
|
||||
```
|
||||
5. **Verify**
|
||||
```bash
|
||||
systemctl status aitbc-miner.service
|
||||
tail -f /var/log/aitbc-miner.log
|
||||
curl -s http://127.0.0.1:8080/v1/wallet/balance | jq
|
||||
```
|
||||
|
||||
> With these details, Windsurf can boot both the miner and the mock Coordinator in under a minute without a production backend.
|
||||
|
||||
---
|
||||
|
||||
## CUDA Support
|
||||
Yes, the miner supports CUDA GPUs. The installer checks for `nvidia-smi` and, if present, attempts to install PyTorch with CUDA wheels (`cu124`). At runtime, tensors are placed on `'cuda'` if `torch.cuda.is_available()` is true. If no GPU is detected, the miner automatically falls back to CPU mode.
|
||||
|
||||
**Prerequisites for CUDA:**
|
||||
- Install NVIDIA drivers on Debian:
|
||||
```bash
|
||||
apt install -y nvidia-driver nvidia-smi
|
||||
```
|
||||
- Optional: Install CUDA toolkit if required for advanced workloads:
|
||||
```bash
|
||||
apt install -y nvidia-cuda-toolkit
|
||||
```
|
||||
- Verify with:
|
||||
```bash
|
||||
nvidia-smi
|
||||
```
|
||||
|
||||
Make sure drivers/toolkit are installed before running the miner installer.
|
||||
|
||||
---
|
||||
|
||||
## 1) Targets & Assumptions
|
||||
- Host: Debian 12/13, root shell, `zsh` available.
|
||||
- Optional GPU: NVIDIA (e.g. RTX 4060 Ti) with CUDA toolchain.
|
||||
- Network egress to Coordinator API (HTTPS). No inbound ports required.
|
||||
- File paths align with user conventions.
|
||||
|
||||
---
|
||||
|
||||
## 2) Directory Layout
|
||||
```
|
||||
/root/scripts/aitbc-miner/ # scripts
|
||||
install_miner.sh
|
||||
miner.sh
|
||||
/etc/aitbc/
|
||||
miner.conf # runtime config (env‑like)
|
||||
/var/log/aitbc-miner.log # log file (rotated by logrotate, optional)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3) Config File: `/etc/aitbc/miner.conf`
|
||||
Environment‑style key/values. Edit with your Coordinator endpoint and wallet/API key.
|
||||
|
||||
```ini
|
||||
# AITBC Miner config
|
||||
COORD_URL="https://coordinator.example.net" # Coordinator base URL
|
||||
WALLET_ADDR="wallet1qxy2kgdygjrsqtzq2n0yrf..." # Your payout address
|
||||
API_KEY="REPLACE_WITH_WALLET_API_KEY" # Wallet‑issued key for auth
|
||||
MINER_ID="$(hostname)-gpu0" # Any stable node label
|
||||
WORK_DIR="/tmp/aitbc-work" # Scratch space
|
||||
HEARTBEAT_SECS=20 # Health ping interval
|
||||
JOB_POLL_SECS=3 # Fetch cadence
|
||||
MAX_CONCURRENCY=1 # Inference job slots
|
||||
# GPU modes: auto|gpu|cpu
|
||||
ACCEL_MODE="auto"
|
||||
```
|
||||
|
||||
> **Tip:** Store secrets with `chmod 600 /etc/aitbc/miner.conf`.
|
||||
|
||||
---
|
||||
|
||||
## 4) Installer Script: `/root/scripts/aitbc-miner/install_miner.sh`
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# Script Version: 01
|
||||
# Description: Install AITBC miner runtime (deps, folders, service)
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
LOG_FILE=/var/log/aitbc-miner.log
|
||||
|
||||
mkdir -p /root/scripts/aitbc-miner /etc/aitbc
|
||||
: > "$LOG_FILE"
|
||||
chmod 600 "$LOG_FILE"
|
||||
|
||||
# Base deps
|
||||
apt update
|
||||
apt install -y curl jq ca-certificates coreutils procps python3 python3-venv python3-pip git \
|
||||
pciutils lsb-release
|
||||
|
||||
# Optional: NVIDIA CLI utils detection (no failure if absent)
|
||||
if command -v nvidia-smi >/dev/null 2>&1; then
|
||||
echo "[INFO] NVIDIA detected" | tee -a "$LOG_FILE"
|
||||
else
|
||||
echo "[INFO] NVIDIA not detected, will run CPU mode if configured" | tee -a "$LOG_FILE"
|
||||
fi
|
||||
|
||||
# Python env for exemplar workloads (torch optional)
|
||||
VENV_DIR=/opt/aitbc-miner/.venv
|
||||
mkdir -p /opt/aitbc-miner
|
||||
python3 -m venv "$VENV_DIR"
|
||||
source "$VENV_DIR/bin/activate"
|
||||
|
||||
# Minimal runtime deps
|
||||
pip install --upgrade pip wheel
|
||||
# Try torch (GPU if CUDA present; fallback CPU). Best‑effort only.
|
||||
python - <<'PY'
|
||||
import os, sys
|
||||
try:
|
||||
import subprocess
|
||||
cuda_ok = subprocess.call(["bash","-lc","nvidia-smi >/dev/null 2>&1"])==0
|
||||
pkg = "--index-url https://download.pytorch.org/whl/cu124 torch torchvision torchaudio" if cuda_ok else "torch torchvision torchaudio"
|
||||
os.system(f"pip install -q {pkg}")
|
||||
print("[INFO] torch installed")
|
||||
except Exception as e:
|
||||
print("[WARN] torch install skipped:", e)
|
||||
PY
|
||||
|
||||
# Place default config if missing
|
||||
CONF=/etc/aitbc/miner.conf
|
||||
if [ ! -f "$CONF" ]; then
|
||||
cat >/etc/aitbc/miner.conf <<'CFG'
|
||||
COORD_URL="https://coordinator.example.net"
|
||||
WALLET_ADDR="wallet_demo"
|
||||
API_KEY="CHANGE_ME"
|
||||
MINER_ID="demo-node"
|
||||
WORK_DIR="/tmp/aitbc-work"
|
||||
HEARTBEAT_SECS=20
|
||||
JOB_POLL_SECS=3
|
||||
MAX_CONCURRENCY=1
|
||||
ACCEL_MODE="auto"
|
||||
CFG
|
||||
chmod 600 /etc/aitbc/miner.conf
|
||||
echo "[INFO] Wrote /etc/aitbc/miner.conf" | tee -a "$LOG_FILE"
|
||||
fi
|
||||
|
||||
# Install service unit
|
||||
cat >/etc/systemd/system/aitbc-miner.service <<'UNIT'
|
||||
[Unit]
|
||||
Description=AITBC Compute Miner
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
ExecStart=/root/scripts/aitbc-miner/miner.sh
|
||||
Restart=always
|
||||
RestartSec=3
|
||||
EnvironmentFile=/etc/aitbc/miner.conf
|
||||
StandardOutput=append:/var/log/aitbc-miner.log
|
||||
StandardError=append:/var/log/aitbc-miner.log
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
UNIT
|
||||
|
||||
systemctl daemon-reload
|
||||
systemctl enable --now aitbc-miner.service
|
||||
|
||||
echo "[INFO] AITBC miner installed and started" | tee -a "$LOG_FILE"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5) Miner Runtime: `/root/scripts/aitbc-miner/miner.sh`
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# Script Version: 01
|
||||
# Description: AITBC miner main loop (poll, run, prove, earn)
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
LOG_FILE=/var/log/aitbc-miner.log
|
||||
: > "$LOG_FILE"
|
||||
|
||||
# ========
|
||||
# Helpers
|
||||
# ========
|
||||
log(){ printf '%s %s\n' "$(date -Is)" "$*" | tee -a "$LOG_FILE"; }
|
||||
req(){
|
||||
local method="$1" path="$2" data="${3:-}"
|
||||
local url="${COORD_URL%/}${path}"
|
||||
if [ -n "$data" ]; then
|
||||
curl -fsS -X "$method" "$url" \
|
||||
-H "Authorization: Bearer $API_KEY" -H 'Content-Type: application/json' \
|
||||
--data "$data"
|
||||
else
|
||||
curl -fsS -X "$method" "$url" -H "Authorization: Bearer $API_KEY"
|
||||
fi
|
||||
}
|
||||
|
||||
has_gpu(){ command -v nvidia-smi >/dev/null 2>&1 && nvidia-smi >/dev/null 2>&1; }
|
||||
|
||||
accel_mode(){
|
||||
case "$ACCEL_MODE" in
|
||||
gpu) has_gpu && echo gpu || echo cpu ;;
|
||||
cpu) echo cpu ;;
|
||||
auto) has_gpu && echo gpu || echo cpu ;;
|
||||
*) echo cpu ;;
|
||||
esac
|
||||
}
|
||||
|
||||
run_job(){
|
||||
local job_json="$1"
|
||||
local job_id; job_id=$(echo "$job_json" | jq -r '.id')
|
||||
local workload; workload=$(echo "$job_json" | jq -r '.workload')
|
||||
local mode; mode=$(accel_mode)
|
||||
|
||||
log "[JOB] start id=$job_id mode=$mode type=$workload"
|
||||
mkdir -p "$WORK_DIR/$job_id"
|
||||
|
||||
# Example workload types: "torch_bench", "sd_infer", "llm_gen"...
|
||||
case "$workload" in
|
||||
torch_bench)
|
||||
/bin/bash -lc "source /opt/aitbc-miner/.venv/bin/activate && python - <<'PY'
|
||||
import time, torch
|
||||
start=time.time()
|
||||
x=torch.randn(1024,1024,device='cuda' if torch.cuda.is_available() else 'cpu')
|
||||
for _ in range(100): x=x@x
|
||||
elapsed=time.time()-start
|
||||
print(f"throughput_ops={100}, seconds={elapsed:.4f}")
|
||||
PY" | tee -a "$LOG_FILE" >"$WORK_DIR/$job_id/out.txt"
|
||||
;;
|
||||
sd_infer)
|
||||
echo "stub: run stable diffusion pipeline here" | tee -a "$LOG_FILE" >"$WORK_DIR/$job_id/out.txt"
|
||||
;;
|
||||
llm_gen)
|
||||
echo "stub: run text generation here" | tee -a "$LOG_FILE" >"$WORK_DIR/$job_id/out.txt"
|
||||
;;
|
||||
*)
|
||||
echo "unknown workload" >"$WORK_DIR/$job_id/out.txt" ;;
|
||||
esac
|
||||
|
||||
# Build a minimal proof (hash of outputs + metrics placeholder)
|
||||
local proof; proof=$(jq -n --arg id "$job_id" \
|
||||
--arg mode "$mode" \
|
||||
--arg out_sha "$(sha256sum "$WORK_DIR/$job_id/out.txt" | awk '{print $1}')" \
|
||||
'{id:$id, mode:$mode, output_sha:$out_sha, metrics:{}}')
|
||||
|
||||
req POST "/v1/miner/proof" "$proof" >/dev/null
|
||||
log "[JOB] done id=$job_id proof_submitted"
|
||||
}
|
||||
|
||||
heartbeat(){
|
||||
local mode; mode=$(accel_mode)
|
||||
local gpu; gpu=$(has_gpu && echo 1 || echo 0)
|
||||
req POST "/v1/miner/heartbeat" "$(jq -n \
|
||||
--arg id "$MINER_ID" --arg w "$WALLET_ADDR" --arg mode "$mode" \
|
||||
--argjson gpu "$gpu" '{miner_id:$id,wallet:$w,mode:$mode,gpu:$gpu}')" >/dev/null
|
||||
}
|
||||
|
||||
# ========
|
||||
# Main Process
|
||||
# ========
|
||||
log "[BOOT] AITBC miner starting (id=$MINER_ID)"
|
||||
mkdir -p "$WORK_DIR"
|
||||
|
||||
# Prime heartbeat
|
||||
heartbeat || log "[WARN] initial heartbeat failed"
|
||||
|
||||
# Poll/execute loop
|
||||
while true; do
|
||||
sleep "$JOB_POLL_SECS"
|
||||
# Opportunistic heartbeat
|
||||
heartbeat || true
|
||||
|
||||
# Fetch one job
|
||||
if J=$(req GET "/v1/miner/next?miner=$MINER_ID&slots=$MAX_CONCURRENCY" 2>/dev/null); then
|
||||
echo "$J" | jq -e '.id' >/dev/null 2>&1 || continue
|
||||
run_job "$J" || log "[WARN] job failed"
|
||||
fi
|
||||
|
||||
done
|
||||
```
|
||||
|
||||
Make executable:
|
||||
```
|
||||
chmod +x /root/scripts/aitbc-miner/install_miner.sh /root/scripts/aitbc-miner/miner.sh
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6) Bootstrap
|
||||
1. Create folders + drop the three files above.
|
||||
2. Edit `/etc/aitbc/miner.conf` with real values.
|
||||
3. Run installer:
|
||||
```
|
||||
/root/scripts/aitbc-miner/install_miner.sh
|
||||
```
|
||||
4. Check status & logs:
|
||||
```
|
||||
systemctl status aitbc-miner.service
|
||||
tail -f /var/log/aitbc-miner.log
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7) Health & Debug
|
||||
- Quick GPU sanity: `nvidia-smi` (optional).
|
||||
- Liveness: periodic `/v1/miner/heartbeat` pings.
|
||||
- Job smoke test (coordinator): ensure `/v1/miner/next` returns a JSON job.
|
||||
|
||||
---
|
||||
|
||||
## 8) Logrotate (optional)
|
||||
Create `/etc/logrotate.d/aitbc-miner`:
|
||||
```conf
|
||||
/var/log/aitbc-miner.log {
|
||||
rotate 7
|
||||
daily
|
||||
missingok
|
||||
notifempty
|
||||
compress
|
||||
copytruncate
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 9) Security Notes
|
||||
- Keep `API_KEY` scoped to miner ops with revocation.
|
||||
- No inbound ports required; allow egress HTTPS only.
|
||||
- Consider systemd `ProtectSystem=full`, `ProtectHome=yes`, `NoNewPrivileges=yes` hardening once stable.
|
||||
|
||||
---
|
||||
|
||||
## 10) Extending Workloads
|
||||
- Implement real `sd_infer` (Stable Diffusion) and `llm_gen` via the venv.
|
||||
- Add job-level resource caps (VRAM/CPU-time) from Coordinator parameters.
|
||||
- Attach accounting metrics for reward weighting (e.g., `tokens_per_kJ` or `tokens_per_TFLOP_s`).
|
||||
|
||||
---
|
||||
|
||||
## 11) Common Commands
|
||||
```
|
||||
# restart after config edits
|
||||
systemctl restart aitbc-miner.service
|
||||
|
||||
# follow logs
|
||||
journalctl -u aitbc-miner -f
|
||||
|
||||
# disable autostart
|
||||
systemctl disable --now aitbc-miner.service
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 12) Coordinator API v1 (Detailed)
|
||||
|
||||
This section specifies the **Miner-facing** HTTP API. All endpoints are versioned under `/v1/` and use **JSON** over **HTTPS**. Authentication is via `Authorization: Bearer <API_KEY>` issued by the wallet/coordinator.
|
||||
|
||||
### 12.1 Global
|
||||
- **Base URL**: `${COORD_URL}` (e.g., `https://coordinator.example.net`).
|
||||
- **Content-Type**: `application/json; charset=utf-8`.
|
||||
- **Auth**: `Authorization: Bearer <API_KEY>` (scoped for miner ops).
|
||||
- **Idempotency** *(recommended)*: `Idempotency-Key: <uuid4>` for POSTs.
|
||||
- **Clock**: All timestamps are ISO‑8601 UTC (e.g., `2025-09-26T13:37:00Z`).
|
||||
- **Errors**: Non‑2xx responses return a body:
|
||||
```json
|
||||
{ "error": { "code": "STRING_CODE", "message": "human readable", "details": {"field": "optional context"} } }
|
||||
```
|
||||
- **Common HTTP codes**: `200 OK`, `201 Created`, `204 No Content`, `400 Bad Request`, `401 Unauthorized`, `403 Forbidden`, `404 Not Found`, `409 Conflict`, `422 Unprocessable Entity`, `429 Too Many Requests`, `500/502/503`.
|
||||
|
||||
---
|
||||
|
||||
### 12.2 Types
|
||||
|
||||
#### MinerCapabilities
|
||||
```json
|
||||
{
|
||||
"miner_id": "string",
|
||||
"mode": "gpu|cpu",
|
||||
"gpu": true,
|
||||
"concurrency": 1,
|
||||
"workloads": ["torch_bench", "sd_infer", "llm_gen"],
|
||||
"limits": {"vram_mb": 16000, "ram_mb": 32768, "max_runtime_s
|
||||
|
||||
412
docs/bootstrap/miner_node.md
Normal file
412
docs/bootstrap/miner_node.md
Normal file
@ -0,0 +1,412 @@
|
||||
# miner-node/ — Worker Node Daemon for GPU/CPU Tasks
|
||||
|
||||
> **Goal:** Implement a Docker‑free worker daemon that connects to the Coordinator API, advertises capabilities (CPU/GPU), fetches jobs, executes them in a sandboxed workspace, and streams results/metrics back.
|
||||
|
||||
---
|
||||
|
||||
## 1) Scope & MVP
|
||||
|
||||
**MVP Features**
|
||||
- Node registration with Coordinator (auth token + capability descriptor).
|
||||
- Heartbeat & liveness (interval ± jitter, backoff on failure).
|
||||
- Job fetch → ack → execute → upload result → finalize.
|
||||
- Two runner types:
|
||||
- **CLI runner**: executes a provided command with arguments (whitelist‑based).
|
||||
- **Python runner**: executes a trusted task module with parameters.
|
||||
- CPU/GPU capability detection (CUDA, VRAM, driver info) without Docker.
|
||||
- Sandboxed working dir per job under `/var/lib/aitbc/miner/jobs/<job-id>`.
|
||||
- Resource controls (nice/ionice/ulimit; optional cgroup v2 if present).
|
||||
- Structured JSON logging and minimal metrics.
|
||||
|
||||
**Post‑MVP**
|
||||
- Chunked artifact upload; resumable transfers.
|
||||
- Prometheus `/metrics` endpoint (pull).
|
||||
- GPU multi‑card scheduling & fractional allocation policy.
|
||||
- On‑node model cache management (size, eviction, pinning).
|
||||
- Signed task manifests & attestation of execution.
|
||||
- Secure TMPFS for secrets; hardware key support (YubiKey).
|
||||
|
||||
---
|
||||
|
||||
## 2) High‑Level Architecture
|
||||
|
||||
```
|
||||
client → coordinator-api → miner-node(s) → results store → coordinator-api → client
|
||||
```
|
||||
|
||||
Miner components:
|
||||
- **Agent** (control loop): registration, heartbeat, fetch/dispatch, result reporting.
|
||||
- **Capability Probe**: CPU/GPU inventory (CUDA, VRAM), free RAM/disk, load.
|
||||
- **Schedulers**: simple FIFO for MVP; one job per GPU or CPU slot.
|
||||
- **Runners**: CLI runner & Python runner.
|
||||
- **Sandbox**: working dirs, resource limits, network I/O gating (optional), file allowlist.
|
||||
- **Telemetry**: JSON logs, minimal metrics; per‑job timeline.
|
||||
|
||||
---
|
||||
|
||||
## 3) Directory Layout (on node)
|
||||
|
||||
```
|
||||
/var/lib/aitbc/miner/
|
||||
├─ jobs/
|
||||
│ ├─ <job-id>/
|
||||
│ │ ├─ input/
|
||||
│ │ ├─ work/
|
||||
│ │ ├─ output/
|
||||
│ │ └─ logs/
|
||||
├─ cache/ # model/assets cache (optional)
|
||||
└─ tmp/
|
||||
/etc/aitbc/miner/
|
||||
├─ config.yaml
|
||||
└─ allowlist.d/ # allowed CLI programs & argument schema snippets
|
||||
/var/log/aitbc/miner/
|
||||
/usr/local/lib/aitbc/miner/ # python package venv install target
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4) Config (YAML)
|
||||
|
||||
```yaml
|
||||
node_id: "node-<shortid>"
|
||||
coordinator:
|
||||
base_url: "https://coordinator.local/api/v1"
|
||||
auth_token: "env:MINER_AUTH" # read from env at runtime
|
||||
tls_verify: true
|
||||
timeout_s: 20
|
||||
|
||||
heartbeat:
|
||||
interval_s: 15
|
||||
jitter_pct: 10
|
||||
backoff:
|
||||
min_s: 5
|
||||
max_s: 120
|
||||
|
||||
runners:
|
||||
cli:
|
||||
enable: true
|
||||
allowlist_files:
|
||||
- "/etc/aitbc/miner/allowlist.d/ffmpeg.yaml"
|
||||
- "/etc/aitbc/miner/allowlist.d/whisper.yaml"
|
||||
python:
|
||||
enable: true
|
||||
task_paths:
|
||||
- "/usr/local/lib/aitbc/miner/tasks"
|
||||
venv: "/usr/local/lib/aitbc/miner/.venv"
|
||||
|
||||
resources:
|
||||
max_concurrent_cpu: 2
|
||||
max_concurrent_gpu: 1
|
||||
cpu_nice: 10
|
||||
io_class: "best-effort"
|
||||
io_level: 6
|
||||
mem_soft_mb: 16384
|
||||
|
||||
workspace:
|
||||
root: "/var/lib/aitbc/miner/jobs"
|
||||
keep_success: 24h
|
||||
keep_failed: 7d
|
||||
|
||||
logging:
|
||||
level: "info"
|
||||
json: true
|
||||
path: "/var/log/aitbc/miner/miner.jsonl"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5) Environment & Dependencies
|
||||
|
||||
- **OS:** Debian 12/13 (systemd).
|
||||
- **Python:** 3.11+ in venv under `/usr/local/lib/aitbc/miner/.venv`.
|
||||
- **Libraries:** `httpx`, `pydantic`, `uvloop` (optional), `pyyaml`, `psutil`.
|
||||
- **GPU (optional):** NVIDIA driver installed; `nvidia-smi` available; CUDA 12.x runtime on path for GPU tasks.
|
||||
|
||||
**Install skeleton**
|
||||
```
|
||||
python3 -m venv /usr/local/lib/aitbc/miner/.venv
|
||||
/usr/local/lib/aitbc/miner/.venv/bin/pip install --upgrade pip
|
||||
/usr/local/lib/aitbc/miner/.venv/bin/pip install httpx pydantic pyyaml psutil uvloop
|
||||
install -d /etc/aitbc/miner /var/lib/aitbc/miner/{jobs,cache,tmp} /var/log/aitbc/miner
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6) Systemd Service
|
||||
|
||||
**/etc/systemd/system/aitbc-miner.service**
|
||||
```
|
||||
[Unit]
|
||||
Description=AITBC Miner Node
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
Environment=MINER_AUTH=***REDACTED***
|
||||
ExecStart=/usr/local/lib/aitbc/miner/.venv/bin/python -m aitbc_miner --config /etc/aitbc/miner/config.yaml
|
||||
User=games
|
||||
Group=games
|
||||
# Lower CPU/IO priority by default
|
||||
Nice=10
|
||||
IOSchedulingClass=best-effort
|
||||
IOSchedulingPriority=6
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
# Hardening
|
||||
NoNewPrivileges=true
|
||||
ProtectSystem=full
|
||||
ProtectHome=true
|
||||
PrivateTmp=true
|
||||
ReadWritePaths=/var/lib/aitbc/miner /var/log/aitbc/miner
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7) Capability Probe (sent to Coordinator)
|
||||
|
||||
Example payload:
|
||||
```json
|
||||
{
|
||||
"node_id": "node-abc123",
|
||||
"version": "0.1.0",
|
||||
"cpu": {"cores": 16, "arch": "x86_64"},
|
||||
"memory_mb": 64000,
|
||||
"disk_free_mb": 250000,
|
||||
"gpu": [
|
||||
{
|
||||
"vendor": "nvidia", "name": "RTX 4060 Ti 16GB",
|
||||
"vram_mb": 16384,
|
||||
"cuda": {"version": "12.3", "driver": "545.23.06"}
|
||||
}
|
||||
],
|
||||
"runners": ["cli", "python"],
|
||||
"tags": ["debian", "cuda", "cpu"]
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 8) Coordinator API Contract (MVP)
|
||||
|
||||
**Endpoints (HTTPS, JSON):**
|
||||
- `POST /nodes/register` → returns signed `node_token` (or 401)
|
||||
- `POST /nodes/heartbeat` → `{node_id, load, free_mb, gpu_free}` → 200
|
||||
- `POST /jobs/pull` → `{node_id, filters}` → `{job|none}`
|
||||
- `POST /jobs/ack` → `{job_id, node_id}` → 200
|
||||
- `POST /jobs/progress` → `{job_id, pct, note}` → 200
|
||||
- `POST /jobs/result` → multipart (metadata.json + artifacts/*) → 200
|
||||
- `POST /jobs/fail` → `{job_id, error_code, error_msg, logs_ref}` → 200
|
||||
|
||||
**Auth**
|
||||
- Bearer token in header (Node → Coordinator): `Authorization: Bearer <node_token>`
|
||||
- Coordinator signs `job.manifest` with HMAC(sha256) or Ed25519 (post‑MVP).
|
||||
|
||||
**Job manifest (subset)**
|
||||
```json
|
||||
{
|
||||
"job_id": "j-20250926-001",
|
||||
"runner": "cli",
|
||||
"requirements": {"gpu": true, "vram_mb": 12000, "cpu_threads": 4},
|
||||
"timeout_s": 3600,
|
||||
"input": {"urls": ["https://.../input1"], "inline": {"text": "..."}},
|
||||
"command": "ffmpeg",
|
||||
"args": ["-y", "-i", "input1.mp4", "-c:v", "libx264", "output.mp4"],
|
||||
"artifacts": [{"path": "output.mp4", "type": "video/mp4", "max_mb": 5000}]
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 9) Runner Design
|
||||
|
||||
### CLI Runner
|
||||
- Validate `command` against allowlist (`/etc/aitbc/miner/allowlist.d/*.yaml`).
|
||||
- Validate `args` against per‑tool schema (regex & size caps).
|
||||
- Materialize inputs in job workspace; set `PATH`, `CUDA_VISIBLE_DEVICES`.
|
||||
- Launch via `subprocess.Popen` with `preexec_fn` applying `nice`, `ionice`, `setrlimit`.
|
||||
- Live‑tail stdout/stderr to `logs/exec.log`; throttle progress pings.
|
||||
|
||||
### Python Runner
|
||||
- Import trusted module `tasks.<name>:run(**params)` from configured paths.
|
||||
- Run in same venv; optional `venv per task` later.
|
||||
- Enforce timeouts; capture logs; write artifacts to `output/`.
|
||||
|
||||
---
|
||||
|
||||
## 10) Resource Controls (No Docker)
|
||||
|
||||
- **CPU:** `nice(10)`; optional cgroups v2 CPU.max if available.
|
||||
- **IO:** `ionice -c 2 -n 6` (best‑effort) for heavy disk ops.
|
||||
- **Memory:** `setrlimit(RLIMIT_AS)` soft cap; kill on OOM.
|
||||
- **GPU:** select by policy (least used VRAM). No hard memory partitioning in MVP.
|
||||
- **Network:** allowlist outbound hosts; deny by default (optional phase‑2).
|
||||
|
||||
---
|
||||
|
||||
## 11) Job Lifecycle (State Machine)
|
||||
|
||||
`IDLE → PULLING → ACKED → PREP → RUNNING → UPLOADING → DONE | FAILED | RETRY_WAIT`
|
||||
|
||||
- Retries: exponential backoff, max N; idempotent uploads.
|
||||
- On crash: on‑start recovery scans `jobs/*/state.json` and reconciles with Coordinator.
|
||||
|
||||
---
|
||||
|
||||
## 12) Logging & Metrics
|
||||
|
||||
- JSON lines in `/var/log/aitbc/miner/miner.jsonl` with fields: `ts, level, node_id, job_id, event, attrs{}`.
|
||||
- Optional `/healthz` (HTTP) returning 200 + brief status.
|
||||
- Future: Prometheus `/metrics` with gauges (queue, running, VRAM free, CPU load).
|
||||
|
||||
---
|
||||
|
||||
## 13) Security Model
|
||||
|
||||
- TLS required; pin CA or enable cert validation per env.
|
||||
- Node bootstrap token (`MINER_AUTH`) exchanged for `node_token` at registration.
|
||||
- Strict allowlist for CLI tools + args; size/time caps.
|
||||
- Secrets never written to disk unencrypted; pass via env vars or in‑memory.
|
||||
- Wipe workdirs on success (per policy); keep failed for triage.
|
||||
|
||||
---
|
||||
|
||||
## 14) Windsurf Implementation Plan
|
||||
|
||||
**Milestone 1 — Skeleton**
|
||||
1. `aitbc_miner/` package: `main.py`, `config.py`, `agent.py`, `probe.py`, `runners/{cli.py, python.py}`, `util/{limits.py, fs.py, log.py}`.
|
||||
2. Load YAML config, bootstrap logs, print probe JSON.
|
||||
3. Implement `/healthz` (optional FastAPI or bare aiohttp) for local checks.
|
||||
|
||||
**Milestone 2 — Control Loop**
|
||||
1. Register → store `node_token` (in memory only).
|
||||
2. Heartbeat task (async), backoff on network errors.
|
||||
3. Pull/ack & single‑slot executor; write state.json.
|
||||
|
||||
**Milestone 3 — Runners**
|
||||
1. CLI allowlist loader + validator; subprocess with limits.
|
||||
2. Python runner calling `tasks.example:run`.
|
||||
3. Upload artifacts via multipart; handle large files with chunking stub.
|
||||
|
||||
**Milestone 4 — Hardening & Ops**
|
||||
1. Crash recovery; cleanup policy; TTL sweeper.
|
||||
2. Metrics counters; structured logging fields.
|
||||
3. Systemd unit; install scripts; doc.
|
||||
|
||||
---
|
||||
|
||||
## 15) Minimal Allowlist Example (ffmpeg)
|
||||
|
||||
```yaml
|
||||
# /etc/aitbc/miner/allowlist.d/ffmpeg.yaml
|
||||
command:
|
||||
path: "/usr/bin/ffmpeg"
|
||||
args:
|
||||
- ["-y"]
|
||||
- ["-i", ".+\\.(mp4|wav|mkv)$"]
|
||||
- ["-c:v", "(libx264|copy)"]
|
||||
- ["-c:a", "(aac|copy)"]
|
||||
- ["-b:v", "[1-9][0-9]{2,5}k"]
|
||||
- ["output\\.(mp4|mkv)"]
|
||||
max_total_args_len: 4096
|
||||
max_runtime_s: 7200
|
||||
max_output_mb: 5000
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 16) Mock Coordinator (for local testing)
|
||||
|
||||
> Run a tiny dev server to hand out a single job and accept results.
|
||||
|
||||
```python
|
||||
# mock_coordinator.py (FastAPI)
|
||||
from fastapi import FastAPI, UploadFile, File, Form
|
||||
from pydantic import BaseModel
|
||||
|
||||
app = FastAPI()
|
||||
JOB = {
|
||||
"job_id": "j-local-1",
|
||||
"runner": "cli",
|
||||
"requirements": {"gpu": False},
|
||||
"timeout_s": 120,
|
||||
"command": "echo",
|
||||
"args": ["hello", "world"],
|
||||
"artifacts": [{"path": "output.txt", "type": "text/plain", "max_mb": 1}]
|
||||
}
|
||||
|
||||
class PullReq(BaseModel):
|
||||
node_id: str
|
||||
filters: dict | None = None
|
||||
|
||||
@app.post("/api/v1/jobs/pull")
|
||||
def pull(req: PullReq):
|
||||
return {"job": JOB}
|
||||
|
||||
@app.post("/api/v1/jobs/ack")
|
||||
def ack(job_id: str, node_id: str):
|
||||
return {"ok": True}
|
||||
|
||||
@app.post("/api/v1/jobs/result")
|
||||
def result(job_id: str = Form(...), metadata: str = Form(...), artifact: UploadFile = File(...)):
|
||||
return {"ok": True}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 17) Developer UX (Make Targets)
|
||||
|
||||
```
|
||||
make venv # create venv + install deps
|
||||
make run # run miner with local config
|
||||
make fmt # ruff/black (optional)
|
||||
make test # unit tests
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 18) Operational Runbook
|
||||
|
||||
- **Start/Stop**: `systemctl enable --now aitbc-miner`
|
||||
- **Logs**: `journalctl -u aitbc-miner -f` and `/var/log/aitbc/miner/miner.jsonl`
|
||||
- **Rotate**: logrotate config (size 50M, keep 7)
|
||||
- **Upgrade**: drain → stop → replace venv → start → verify heartbeat
|
||||
- **Health**: `/healthz` 200 + JSON `{running, queued, cpu_load, vram_free}`
|
||||
|
||||
---
|
||||
|
||||
## 19) Failure Modes & Recovery
|
||||
|
||||
- **Network errors**: exponential backoff; keep heartbeat local status.
|
||||
- **Job invalid**: fail fast with reason; do not retry.
|
||||
- **Runner denied**: allowlist miss → fail with `E_DENY`.
|
||||
- **OOM**: kill process group; mark `E_OOM`.
|
||||
- **GPU unavailable**: requeue with reason `E_NOGPU`.
|
||||
|
||||
---
|
||||
|
||||
## 20) Roadmap Notes
|
||||
|
||||
- Binary task bundles with signed SBOM.
|
||||
- Remote cache warming via Coordinator hints.
|
||||
- Multi‑Queue scheduling (latency vs throughput).
|
||||
- MIG/compute‑instance support if hardware allows.
|
||||
|
||||
---
|
||||
|
||||
## 21) Checklist for Windsurf
|
||||
|
||||
1. Create `aitbc_miner/` package skeleton with modules listed in §14.
|
||||
2. Implement config loader + capability probe output.
|
||||
3. Implement async agent loop: register → heartbeat → pull/ack.
|
||||
4. Implement CLI runner with allowlist (§15) and exec log.
|
||||
5. Implement Python runner stub (`tasks/example.py`).
|
||||
6. Write result uploader (multipart) and finalize call.
|
||||
7. Add systemd unit (§6) and basic install script.
|
||||
8. Test end‑to‑end against `mock_coordinator.py` (§16).
|
||||
9. Document log fields + troubleshooting card.
|
||||
10. Add optional `/healthz` endpoint.
|
||||
|
||||
314
docs/bootstrap/pool_hub.md
Normal file
314
docs/bootstrap/pool_hub.md
Normal file
@ -0,0 +1,314 @@
|
||||
# pool-hub.md — Client ↔ Miners Pool & Matchmaking Gateway (guide for Windsurf)
|
||||
|
||||
> **Role in AITBC**
|
||||
> The **Pool Hub** is the real-time directory of available miners and a low-latency **matchmaker** between **job requests** (coming from `coordinator-api`) and **worker capacity** (from `miner-node`).
|
||||
> It tracks miner capabilities, health, and price; computes a score; and returns the **best N candidates** for each job.
|
||||
|
||||
---
|
||||
|
||||
## 1) MVP Scope (Stage 3 of the boot plan)
|
||||
|
||||
- Accept **miner registrations** + **heartbeats** (capabilities, price, queues).
|
||||
- Maintain an in-memory + persistent **miner registry** with fast lookups.
|
||||
- Provide a **/match** API for `coordinator-api` to request top candidates.
|
||||
- Simple **scoring** with pluggable strategy (latency, VRAM, price, trust).
|
||||
- **No token minting/accounting** in MVP (stub hooks only).
|
||||
- **No Docker**: Debian 12, `uvicorn` service, optional Nginx reverse proxy.
|
||||
|
||||
---
|
||||
|
||||
## 2) High-Level Architecture
|
||||
|
||||
```
|
||||
client-web ──> coordinator-api ──> pool-hub ──> miner-node
|
||||
▲ ▲
|
||||
│ │
|
||||
wallet-daemon heartbeat + capability updates
|
||||
(later) (WebSocket/HTTP)
|
||||
```
|
||||
|
||||
### Responsibilities
|
||||
- **Registry**: who’s online, what they can do, how much it costs.
|
||||
- **Health**: heartbeats, timeouts, grace periods, auto-degrade/remove.
|
||||
- **Matchmaking**: filter → score → return top K miner candidates.
|
||||
- **Observability**: metrics for availability, match latency, rejection reasons.
|
||||
|
||||
---
|
||||
|
||||
## 3) Protocols & Endpoints (FastAPI)
|
||||
|
||||
> Base URL examples:
|
||||
> - Public (optional): `https://pool-hub.example/api`
|
||||
> - Internal (preferred): `http://127.0.0.1:8203` (behind Nginx)
|
||||
|
||||
### 3.1 Miner lifecycle (HTTP + WebSocket)
|
||||
|
||||
- `POST /v1/miners/register`
|
||||
- **Body**: `{ miner_id, api_key, addr, proto, gpu_vram, gpu_name, cpu_cores, ram_gb, price_token_per_ksec, max_parallel, tags[], capabilities[] }`
|
||||
- **Returns**: `{ status, lease_ttl_sec, next_heartbeat_sec }`
|
||||
- Notes: returns a **session_token** (short-lived) if `api_key` valid.
|
||||
|
||||
- `POST /v1/miners/update`
|
||||
- **Body**: `{ session_token, queue_len, busy, current_jobs[], price_token_per_ksec? }`
|
||||
|
||||
- `GET /v1/miners/lease/renew?session_token=...`
|
||||
- Renews online lease (fallback if WS drops).
|
||||
|
||||
- `WS /v1/miners/heartbeat`
|
||||
- **Auth**: `session_token` in query.
|
||||
- **Payload (periodic)**: `{ ts, queue_len, avg_latency_ms?, temp_c?, mem_free_gb? }`
|
||||
- Server may push **commands** (e.g., “update tags”, “set price cap”).
|
||||
|
||||
- `POST /v1/miners/logout`
|
||||
- Clean unregister (otherwise lease expiry removes).
|
||||
|
||||
### 3.2 Coordinator matchmaking (HTTP)
|
||||
|
||||
- `POST /v1/match`
|
||||
- **Body**:
|
||||
```json
|
||||
{
|
||||
"job_id": "uuid",
|
||||
"requirements": {
|
||||
"task": "image_embedding",
|
||||
"min_vram_gb": 8,
|
||||
"min_ram_gb": 8,
|
||||
"accel": ["cuda"],
|
||||
"tags_any": ["eu-west","low-latency"],
|
||||
"capabilities_any": ["sentence-transformers/all-MiniLM-L6-v2"]
|
||||
},
|
||||
"hints": { "region": "eu", "max_price": 0.8, "deadline_ms": 5000 },
|
||||
"top_k": 3
|
||||
}
|
||||
```
|
||||
- **Returns**:
|
||||
```json
|
||||
{
|
||||
"job_id": "uuid",
|
||||
"candidates": [
|
||||
{ "miner_id":"...", "addr":"...", "proto":"grpc", "score":0.87, "eta_ms": 320, "price":0.75 },
|
||||
{ "miner_id":"...", "addr":"...", "proto":"grpc", "score":0.81, "eta_ms": 410, "price":0.65 }
|
||||
],
|
||||
"explain": "cap=1.0 • price=0.8 • latency=0.9 • trust=0.7"
|
||||
}
|
||||
```
|
||||
|
||||
- `POST /v1/feedback`
|
||||
- **Body**: `{ job_id, miner_id, outcome: "accepted"|"rejected"|"failed"|"completed", latency_ms?, fail_code?, tokens_spent? }`
|
||||
- Used to adjust **trust score** & calibration.
|
||||
|
||||
### 3.3 Observability
|
||||
|
||||
- `GET /v1/health` → `{ status:"ok", online_miners, avg_latency_ms }`
|
||||
- `GET /v1/metrics` → Prometheus-style metrics (text)
|
||||
- `GET /v1/miners` (admin-guarded) → paginated registry snapshot
|
||||
|
||||
---
|
||||
|
||||
## 4) Data Model (PostgreSQL minimal)
|
||||
|
||||
> Schema name: `poolhub`
|
||||
|
||||
**tables**
|
||||
- `miners`
|
||||
`miner_id PK, api_key_hash, created_at, last_seen_at, addr, proto, gpu_vram_gb, gpu_name, cpu_cores, ram_gb, max_parallel, base_price, tags jsonb, capabilities jsonb, trust_score float default 0.5, region text`
|
||||
- `miner_status`
|
||||
`miner_id FK, queue_len int, busy bool, avg_latency_ms int, temp_c int, mem_free_gb float, updated_at`
|
||||
- `feedback`
|
||||
`id PK, job_id, miner_id, outcome, latency_ms, fail_code, tokens_spent, created_at`
|
||||
- `price_overrides` (optional)
|
||||
|
||||
**indexes**
|
||||
- `idx_miners_region_caps_gte_vram` (GIN on jsonb + partials)
|
||||
- `idx_status_updated_at`
|
||||
- `idx_feedback_miner_time`
|
||||
|
||||
**in-memory caches**
|
||||
- Hot registry (Redis or local LRU) for sub-millisecond match filter.
|
||||
|
||||
---
|
||||
|
||||
## 5) Matching & Scoring
|
||||
|
||||
**Filter**:
|
||||
- Hard constraints: `min_vram_gb`, `min_ram_gb`, `accel`, `capabilities_any`, `region`, `max_price`, `queue_len < max_parallel`.
|
||||
|
||||
**Score formula (tunable)**:
|
||||
```
|
||||
score = w_cap*cap_fit
|
||||
+ w_price*price_norm
|
||||
+ w_latency*latency_norm
|
||||
+ w_trust*trust_score
|
||||
+ w_load*load_norm
|
||||
```
|
||||
- `cap_fit`: 1 if all required caps present, else <1 proportional to overlap.
|
||||
- `price_norm`: cheaper = closer to 1 (normalized vs request cap).
|
||||
- `latency_norm`: 1 for fastest observed in region, decays by percentile.
|
||||
- `load_norm`: higher if queue_len small and max_parallel large.
|
||||
- Default weights: `w_cap=0.40, w_price=0.20, w_latency=0.20, w_trust=0.15, w_load=0.05`.
|
||||
|
||||
Return top-K with **explain string** for debugging.
|
||||
|
||||
---
|
||||
|
||||
## 6) Security (MVP → later hardening)
|
||||
|
||||
- **Auth**:
|
||||
- Miners: static `api_key` → exchange for short **session_token**.
|
||||
- Coordinator: **shared secret** header (MVP), rotate via env.
|
||||
- **Network**:
|
||||
- Bind to localhost; expose via Nginx with IP allowlist for coordinator.
|
||||
- Rate limits on `/match` and `/register`.
|
||||
- **Later**:
|
||||
- mTLS between pool-hub ↔ miners.
|
||||
- Signed capability manifests.
|
||||
- Attestation (NVIDIA MIG/hash) optional.
|
||||
|
||||
---
|
||||
|
||||
## 7) Configuration
|
||||
|
||||
- `.env` (loaded by FastAPI):
|
||||
- `POOLHUB_BIND=127.0.0.1`
|
||||
- `POOLHUB_PORT=8203`
|
||||
- `DB_DSN=postgresql://poolhub:*****@127.0.0.1:5432/aitbc`
|
||||
- `REDIS_URL=redis://127.0.0.1:6379/4` (optional)
|
||||
- `COORDINATOR_SHARED_SECRET=...`
|
||||
- `SESSION_TTL_SEC=60`
|
||||
- `HEARTBEAT_GRACE_SEC=120`
|
||||
- `DEFAULT_WEIGHTS=cap:0.4,price:0.2,latency:0.2,trust:0.15,load:0.05`
|
||||
|
||||
---
|
||||
|
||||
## 8) Local Dev (Debian 12, no Docker)
|
||||
|
||||
```bash
|
||||
# Python env
|
||||
apt install -y python3-venv python3-pip
|
||||
python3 -m venv .venv && source .venv/bin/activate
|
||||
pip install fastapi uvicorn[standard] pydantic-settings psycopg[binary] orjson redis prometheus-client
|
||||
|
||||
# DB (psql one-liners, adjust to your policy)
|
||||
psql -U postgres -c "CREATE USER poolhub WITH PASSWORD '***';"
|
||||
psql -U postgres -c "CREATE DATABASE aitbc OWNER poolhub;"
|
||||
psql -U postgres -d aitbc -c "CREATE SCHEMA poolhub AUTHORIZATION poolhub;"
|
||||
|
||||
# Run
|
||||
uvicorn app.main:app --host 127.0.0.1 --port 8203 --workers 1 --proxy-headers
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 9) Systemd service (uvicorn)
|
||||
|
||||
`/etc/systemd/system/pool-hub.service`
|
||||
```
|
||||
[Unit]
|
||||
Description=AITBC Pool Hub (FastAPI)
|
||||
After=network-online.target postgresql.service
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
User=games
|
||||
Group=games
|
||||
WorkingDirectory=/var/www/aitbc/pool-hub
|
||||
Environment=PYTHONUNBUFFERED=1
|
||||
EnvironmentFile=/var/www/aitbc/pool-hub/.env
|
||||
ExecStart=/var/www/aitbc/pool-hub/.venv/bin/uvicorn app.main:app --host 127.0.0.1 --port 8203 --workers=1 --proxy-headers
|
||||
Restart=always
|
||||
RestartSec=2
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 10) Optional Nginx (host as reverse proxy)
|
||||
|
||||
```
|
||||
server {
|
||||
listen 443 ssl http2;
|
||||
server_name pool-hub.example;
|
||||
# ssl_certificate ...; ssl_certificate_key ...;
|
||||
|
||||
# Only allow coordinator’s IPs
|
||||
allow 10.0.3.32;
|
||||
allow 127.0.0.1;
|
||||
deny all;
|
||||
|
||||
location / {
|
||||
proxy_pass http://127.0.0.1:8203;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Forwarded-For $remote_addr;
|
||||
proxy_set_header X-Forwarded-Proto https;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 11) FastAPI App Skeleton (files to create)
|
||||
|
||||
```
|
||||
pool-hub/
|
||||
app/
|
||||
__init__.py
|
||||
main.py # FastAPI init, routers include, metrics
|
||||
deps.py # auth, db sessions, rate limits
|
||||
models.py # SQLAlchemy/SQLModel tables (see schema)
|
||||
schemas.py # Pydantic I/O models
|
||||
registry.py # in-memory index + Redis bridge
|
||||
scoring.py # score strategies
|
||||
routers/
|
||||
miners.py # register/update/ws/logout
|
||||
match.py # /match and /feedback
|
||||
admin.py # /miners snapshot (guarded)
|
||||
health.py # /health, /metrics
|
||||
.env.example
|
||||
README.md
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 12) Testing & Validation Checklist
|
||||
|
||||
1. Register 3 miners (mixed VRAM, price, region).
|
||||
2. Heartbeats arrive; stale miners drop after `HEARTBEAT_GRACE_SEC`.
|
||||
3. `/match` with constraints returns consistent top-K and explain string.
|
||||
4. Rate limits kick in (flood `/match` with 50 rps).
|
||||
5. Coordinator feedback adjusts trust score; observe score shifts.
|
||||
6. Systemd restarts on crash; Nginx denies non-coordinator IPs.
|
||||
7. Metrics expose `poolhub_miners_online`, `poolhub_match_latency_ms`.
|
||||
|
||||
---
|
||||
|
||||
## 13) Roadmap (post-MVP)
|
||||
|
||||
- Weighted **multi-dispatch** (split jobs across K miners, merge).
|
||||
- **Price discovery**: spot vs reserved capacity.
|
||||
- **mTLS**, signed manifests, hardware attestation.
|
||||
- **AIToken hooks**: settle/pay via `wallet-daemon` + `blockchain-node`.
|
||||
- Global **region routing** + latency probes.
|
||||
|
||||
---
|
||||
|
||||
## 14) Dev Notes for Windsurf
|
||||
|
||||
- Start with **routers/miners.py** and **routers/match.py**.
|
||||
- Implement `registry.py` with a simple in-proc dict + RWLock; add Redis later.
|
||||
- Keep scoring in **scoring.py** with clear weight constants and unit tests.
|
||||
- Provide a tiny CLI seed to simulate miners for local testing.
|
||||
|
||||
---
|
||||
|
||||
### Open Questions
|
||||
|
||||
- Trust model inputs (what counts as failure vs transient)?
|
||||
- Minimal capability schema vs free-form tags?
|
||||
- Region awareness from IP vs miner-provided claim?
|
||||
|
||||
---
|
||||
|
||||
**Want me to generate the initial FastAPI file stubs for this layout now (yes/no)?**
|
||||
|
||||
335
docs/bootstrap/wallet_daemon.md
Normal file
335
docs/bootstrap/wallet_daemon.md
Normal file
@ -0,0 +1,335 @@
|
||||
# wallet-daemon.md
|
||||
|
||||
> **Role:** Local wallet service (keys, signing, RPC) for the 🤖 AITBC 🤑 stack
|
||||
> **Audience:** Windsurf (programming assistant) + developers
|
||||
> **Stage:** Bootable without blockchain; later pluggable to chain node
|
||||
|
||||
---
|
||||
|
||||
## 1) What this daemon is (and isn’t)
|
||||
|
||||
**Goals**
|
||||
1. Generate/import encrypted wallets (seed or raw keys).
|
||||
2. Derive accounts/addresses (HD, multiple curves).
|
||||
3. Hold keys **locally** and sign messages/transactions/receipts.
|
||||
4. Expose a minimal **RPC** for client/coordinator/miner.
|
||||
5. Provide a mock “ledger view” (balance cache + test transfers) until the chain is ready.
|
||||
|
||||
**Non-Goals (for now)**
|
||||
- No P2P networking.
|
||||
- No full node / block validation.
|
||||
- No remote key export (never leaves box unencrypted).
|
||||
|
||||
---
|
||||
|
||||
## 2) Architecture (minimal, extensible)
|
||||
|
||||
- **Process:** Python FastAPI app (`uvicorn`)
|
||||
- **RPC:** HTTP+JSON (REST) and JSON-RPC (both; same server)
|
||||
- **KeyStore:** on-disk, encrypted with **Argon2id + XChaCha20-Poly1305**
|
||||
- **Curves:** `ed25519` (default), `secp256k1` (optional flag per-account)
|
||||
- **HD Derivation:** BIP-39 seed → SLIP-10 (ed25519), BIP-32 (secp256k1)
|
||||
- **Coin type (provisional):** `AITBC = 12345` (placeholder; replace once registered)
|
||||
- **Auth:** Local-only by default (bind `127.0.0.1`), optional token header for remote.
|
||||
- **Events:** Webhooks + local FIFO/Unix-socket stream for “signed” notifications.
|
||||
- **Mock Ledger (stage-1):** sqlite table for balances & transfers; sync adapter later.
|
||||
|
||||
**Directory layout**
|
||||
```
|
||||
wallet-daemon/
|
||||
├─ app/ # FastAPI service
|
||||
│ ├─ main.py # entrypoint
|
||||
│ ├─ api_rest.py # REST routes
|
||||
│ ├─ api_jsonrpc.py # JSON-RPC methods
|
||||
│ ├─ crypto/ # key, derivation, sign, addr
|
||||
│ ├─ keystore/ # encrypted store backend
|
||||
│ ├─ models/ # pydantic schemas
|
||||
│ ├─ ledger_mock/ # sqlite-backed balances
|
||||
│ └─ settings.py # config
|
||||
├─ data/
|
||||
│ ├─ keystore/ # *.kdb (per wallet)
|
||||
│ └─ ledger.db
|
||||
├─ tests/
|
||||
└─ run.sh
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3) Security model (Windsurf implementation notes)
|
||||
|
||||
- **Passwords:** Argon2id (tuned to machine; env overrides)
|
||||
- `ARGON_TIME=4`, `ARGON_MEMORY=256MB`, `ARGON_PARALLELISM=2` (defaults)
|
||||
- **Encryption:** libsodium `crypto_aead_xchacha20poly1305_ietf`
|
||||
- **At-Rest Format (per wallet file)**
|
||||
```
|
||||
magic=v1-kdb
|
||||
salt=32B
|
||||
argon_params={t,m,p}
|
||||
nonce=24B
|
||||
ciphertext=… # includes seed or master private key; plus metadata JSON
|
||||
```
|
||||
- **In-memory:** zeroize sensitive bytes after use; use `memoryview`/`ctypes` scrubbing where possible.
|
||||
- **API hardening:**
|
||||
- Bind to `127.0.0.1` by default.
|
||||
- Optional `X-Auth-Token: <TOKEN>` for REST/JSON-RPC.
|
||||
- Rate limits on sign endpoints.
|
||||
|
||||
---
|
||||
|
||||
## 4) Key & address strategy
|
||||
|
||||
- **Wallet types**
|
||||
1. **HD (preferred):** BIP-39 mnemonic (12/24 words) → master seed.
|
||||
2. **Single-key:** import raw private key (ed25519/secp256k1).
|
||||
|
||||
- **Derivation paths**
|
||||
- **ed25519 (SLIP-10):** `m / 44' / 12345' / account' / change / index`
|
||||
- **secp256k1 (BIP-44):** `m / 44' / 12345' / account' / change / index`
|
||||
|
||||
- **Address format (temporary)**
|
||||
- **ed25519:** `base32(bech32_hrp="ait")` of `blake2b-20(pubkey)`
|
||||
- **secp256k1:** same hash → bech32; flag curve in metadata
|
||||
|
||||
> Replace HRP and hash rules if the canonical AITBC chain spec differs.
|
||||
|
||||
---
|
||||
|
||||
## 5) REST API (for Windsurf to scaffold)
|
||||
|
||||
**Headers**
|
||||
- Optional: `X-Auth-Token: <token>`
|
||||
- `Content-Type: application/json`
|
||||
|
||||
### 5.1 Wallet lifecycle
|
||||
- `POST /v1/wallet/create`
|
||||
- body: `{ "name": "main", "password": "…", "mnemonic_len": 24 }`
|
||||
- returns: `{ "wallet_id": "…" }`
|
||||
- `POST /v1/wallet/import-mnemonic`
|
||||
- `{ "name":"…","password":"…","mnemonic":"…","passphrase":"" }`
|
||||
- `POST /v1/wallet/import-key`
|
||||
- `{ "name":"…","password":"…","curve":"ed25519|secp256k1","private_key_hex":"…" }`
|
||||
- `POST /v1/wallet/unlock`
|
||||
- `{ "wallet_id":"…","password":"…" }` → unlocks into memory for N seconds (config TTL)
|
||||
- `POST /v1/wallet/lock` → no body
|
||||
- `GET /v1/wallets` → list minimal metadata (never secrets)
|
||||
|
||||
### 5.2 Accounts & addresses
|
||||
- `POST /v1/account/derive`
|
||||
- `{ "wallet_id":"…","curve":"ed25519","path":"m/44'/12345'/0'/0/0" }`
|
||||
- `GET /v1/accounts?wallet_id=…`
|
||||
- `GET /v1/address/{account_id}` → `{ "address":"ait1…", "curve":"ed25519" }`
|
||||
|
||||
### 5.3 Signing
|
||||
- `POST /v1/sign/message`
|
||||
- `{ "account_id":"…","message_base64":"…" }` → `{ "signature_base64":"…" }`
|
||||
- `POST /v1/sign/tx`
|
||||
- `{ "account_id":"…","tx_bytes_base64":"…","type":"aitbc_v0" }` → signature + (optionally) signed blob
|
||||
- `POST /v1/sign/receipt`
|
||||
- `{ "account_id":"…","payload":"…"}`
|
||||
- **Used by coordinator/miner** to sign job receipts & payouts.
|
||||
|
||||
### 5.4 Mock ledger (stage-1 only)
|
||||
- `GET /v1/ledger/balance/{address}`
|
||||
- `POST /v1/ledger/transfer`
|
||||
- `{ "from":"…","to":"…","amount":"123.456","memo":"test" }`
|
||||
- `GET /v1/ledger/tx/{txid}`
|
||||
|
||||
### 5.5 Webhooks
|
||||
- `POST /v1/webhooks`
|
||||
- `{ "url":"https://…/callback", "events":["signed","transfer"] }`
|
||||
|
||||
**Error model**
|
||||
```json
|
||||
{ "error": { "code": "WALLET_LOCKED|NOT_FOUND|BAD_PASSWORD|RATE_LIMIT", "detail": "…" } }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6) JSON-RPC mirror (method → params)
|
||||
|
||||
- `wallet_create(name, password, mnemonic_len)`
|
||||
- `wallet_import_mnemonic(name, password, mnemonic, passphrase)`
|
||||
- `wallet_import_key(name, password, curve, private_key_hex)`
|
||||
- `wallet_unlock(wallet_id, password)` / `wallet_lock()`
|
||||
- `account_derive(wallet_id, curve, path)`
|
||||
- `sign_message(account_id, message_base64)`
|
||||
- `sign_tx(account_id, tx_bytes_base64, type)`
|
||||
- `ledger_getBalance(address)` / `ledger_transfer(from, to, amount, memo)`
|
||||
|
||||
Same auth header; endpoint `/rpc`.
|
||||
|
||||
---
|
||||
|
||||
## 7) Data schemas (Pydantic hints)
|
||||
|
||||
```py
|
||||
class WalletMeta(BaseModel):
|
||||
wallet_id: str
|
||||
name: str
|
||||
curves: list[str] # ["ed25519", "secp256k1"]
|
||||
created_at: datetime
|
||||
|
||||
class AccountMeta(BaseModel):
|
||||
account_id: str
|
||||
wallet_id: str
|
||||
curve: Literal["ed25519","secp256k1"]
|
||||
path: str # HD path or "imported"
|
||||
address: str # bech32 ait1...
|
||||
|
||||
class SignedResult(BaseModel):
|
||||
signature_base64: str
|
||||
public_key_hex: str
|
||||
algo: str # e.g., ed25519
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 8) Configuration (ENV)
|
||||
|
||||
```
|
||||
WALLET_BIND=127.0.0.1
|
||||
WALLET_PORT=8555
|
||||
WALLET_TOKEN= # optional
|
||||
KEYSTORE_DIR=./data/keystore
|
||||
LEDGER_DB=./data/ledger.db
|
||||
UNLOCK_TTL_SEC=120
|
||||
ARGON_TIME=4
|
||||
ARGON_MEMORY_MB=256
|
||||
ARGON_PARALLELISM=2
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 9) Minimal boot script (dev)
|
||||
|
||||
```bash
|
||||
# run.sh
|
||||
export WALLET_BIND=127.0.0.1
|
||||
export WALLET_PORT=8555
|
||||
export KEYSTORE_DIR=./data/keystore
|
||||
mkdir -p "$KEYSTORE_DIR" data
|
||||
exec uvicorn app.main:app --host ${WALLET_BIND} --port ${WALLET_PORT}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 10) Systemd unit (prod, template)
|
||||
|
||||
```
|
||||
[Unit]
|
||||
Description=AITBC Wallet Daemon
|
||||
After=network.target
|
||||
|
||||
[Service]
|
||||
User=root
|
||||
WorkingDirectory=/opt/aitbc/wallet-daemon
|
||||
Environment=WALLET_BIND=127.0.0.1
|
||||
Environment=WALLET_PORT=8555
|
||||
Environment=KEYSTORE_DIR=/opt/aitbc/wallet-daemon/data/keystore
|
||||
ExecStart=/usr/bin/uvicorn app.main:app --host 127.0.0.1 --port 8555
|
||||
Restart=always
|
||||
NoNewPrivileges=true
|
||||
PrivateTmp=true
|
||||
ProtectSystem=full
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 11) Curl smoke tests
|
||||
|
||||
```bash
|
||||
# 1) Create + unlock
|
||||
curl -s localhost:8555/v1/wallet/create -X POST \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"name":"main","password":"pw","mnemonic_len":24}'
|
||||
|
||||
curl -s localhost:8555/v1/wallets
|
||||
|
||||
# 2) Derive first account
|
||||
curl -s localhost:8555/v1/account/derive -X POST \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"wallet_id":"W1","curve":"ed25519","path":"m/44'/12345'/0'/0/0"}'
|
||||
|
||||
# 3) Sign message
|
||||
curl -s localhost:8555/v1/sign/message -X POST \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"account_id":"A1","message_base64":"aGVsbG8="}'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 12) Coordinator/miner integration (stage-1)
|
||||
|
||||
**Coordinator needs:**
|
||||
- `GET /v1/address/{account_id}` for payout address lookup.
|
||||
- `POST /v1/sign/receipt` to sign `{job_id, miner_id, result_hash, ts}`.
|
||||
- Optional webhook on `signed` to chain/queue the payout request.
|
||||
|
||||
**Miner needs:**
|
||||
- Local message signing for **proof-of-work-done** receipts.
|
||||
- Optional “ephemeral sub-accounts” derived at `change=1` for per-job audit.
|
||||
|
||||
---
|
||||
|
||||
## 13) Migration path to real chain
|
||||
|
||||
1. Introduce **ChainAdapter** interface:
|
||||
- `get_balance(address)`, `broadcast_tx(signed_tx)`, `fetch_utxos|nonce`, `estimate_fee`.
|
||||
2. Implement `MockAdapter` (current), then `AitbcNodeAdapter` (RPC to real node).
|
||||
3. Swap via `WALLET_CHAIN=mock|aitbc`.
|
||||
|
||||
---
|
||||
|
||||
## 14) Windsurf tasks checklist
|
||||
|
||||
1. Create repo structure (see **Directory layout**).
|
||||
2. Add dependencies: `fastapi`, `uvicorn`, `pydantic`, `argon2-cffi`, `pynacl` (or `libsodium` bindings), `bech32`, `sqlalchemy` (for mock ledger), `aiosqlite`.
|
||||
3. Implement **keystore** (encrypt/decrypt, file IO, metadata).
|
||||
4. Implement **HD derivation** (SLIP-10 for ed25519, BIP-32 for secp256k1).
|
||||
5. Implement **address** helper (hash → bech32 with `ait`).
|
||||
6. Implement **REST** routes, then JSON-RPC shim.
|
||||
7. Implement **mock ledger** with sqlite + simple transfers.
|
||||
8. Add **webhook** delivery (async task + retry).
|
||||
9. Add **rate limits** on signing; add unlock TTL.
|
||||
10. Write unit tests for keystore, derivation, signing.
|
||||
11. Provide `run.sh` + systemd unit.
|
||||
12. Add curl examples to `README`.
|
||||
|
||||
---
|
||||
|
||||
## 15) Open questions (defaults proposed)
|
||||
|
||||
1. Keep both curves or start **ed25519-only**? → **Start ed25519-only**, gate secp256k1 by flag.
|
||||
2. HRP `ait` and hash `blake2b-20` acceptable as interim? → **Yes (interim)**.
|
||||
3. Store **per-account tags** (e.g., “payout”, “ops”)? → **Yes**, simple string map.
|
||||
|
||||
---
|
||||
|
||||
## 16) Threat model notes (MVP)
|
||||
|
||||
- Local compromise = keys at risk ⇒ encourage **passphrase on mnemonic** + **short unlock TTL**.
|
||||
- Add **IPC allowlist** if binding to non-localhost.
|
||||
- Consider **YubiKey/PKCS#11** module later (signing via hardware).
|
||||
|
||||
---
|
||||
|
||||
## 17) Logging
|
||||
|
||||
- Default: INFO without sensitive fields.
|
||||
- Redact: passwords, mnemonics, private keys, nonces.
|
||||
- Correlate by `req_id` header (generate if missing).
|
||||
|
||||
---
|
||||
|
||||
## 18) License / Compliance
|
||||
|
||||
- Keep crypto libs permissive (MIT/ISC/BSD).
|
||||
- Record algorithm choices & versions in `/about` endpoint for audits.
|
||||
|
||||
---
|
||||
|
||||
**Proceed to generate the scaffold (FastAPI app + keystore + ed25519 HD + REST) now?** `y` / `n`
|
||||
|
||||
Reference in New Issue
Block a user