This commit is contained in:
oib
2026-01-26 19:58:21 +01:00
parent 329b3beeba
commit 5c99c92ffb
54 changed files with 6790 additions and 654 deletions

View File

@@ -9,16 +9,24 @@ All AITBC API endpoints require authentication using API keys.
## Getting API Keys
### Production
1. Visit the [AITBC Dashboard](https://dashboard.aitbc.io)
2. Create an account or sign in
3. Navigate to API Keys section
4. Generate a new API key
### Testing/Development
For integration tests and development, these test keys are available:
- `REDACTED_CLIENT_KEY` - For client API access
- `REDACTED_MINER_KEY` - For miner registration
- `test-tenant` - Default tenant ID for testing
## Using API Keys
### HTTP Header
```http
X-API-Key: your_api_key_here
X-Tenant-ID: your_tenant_id # Optional for multi-tenant
```
### Environment Variable

View File

@@ -4,6 +4,42 @@
This document outlines a comprehensive testing scenario for customers and service providers interacting on the AITBC platform. This scenario enables end-to-end testing of the complete marketplace workflow using the publicly accessible deployment at https://aitbc.bubuit.net/.
## Integration Tests
### Test Suite Status (Updated 2026-01-26)
The integration test suite has been updated to use real implemented features:
#### ✅ Passing Tests (6)
1. **End-to-End Job Execution** - Tests complete job workflow
2. **Multi-Tenant Isolation** - Verifies tenant data separation
3. **Block Propagation** - Tests P2P network block sync
4. **Transaction Propagation** - Tests P2P transaction sync
5. **Marketplace Integration** - Connects to live marketplace
6. **Security Integration** - Uses real ZK proof features
#### ⏸️ Skipped Tests (1)
1. **Wallet Payment Flow** - Awaiting wallet-coordinator integration
#### Running Tests
```bash
# Run all integration tests
python -m pytest tests/integration/test_full_workflow.py -v
# Run specific test class
python -m pytest tests/integration/test_full_workflow.py::TestSecurityIntegration -v
# Run with real client (not mocks)
export USE_REAL_CLIENT=1
python -m pytest tests/integration/ -v
```
#### Test Features
- Tests work with both real client and mock fallback
- Security tests use actual ZK proof requirements
- Marketplace tests connect to https://aitbc.bubuit.net/marketplace
- All tests pass in CLI and Windsorf environments
## Prerequisites
### System Requirements

View File

@@ -20,6 +20,7 @@ This document tracks components that have been successfully deployed and are ope
- Vite + TypeScript frontend
- Offer list, bid form, stats cards
- Mock data fixtures with API abstraction
- Integration tests now connect to live marketplace
-**Coordinator API** - Deployed in container
- FastAPI service running on port 8000
@@ -28,6 +29,7 @@ This document tracks components that have been successfully deployed and are ope
- Explorer API (nginx): `/api/explorer/*` → backend `/v1/explorer/*`
- Users API: `/api/v1/users/*` (compat: `/api/users/*` for Exchange)
- ZK Applications API: /api/zk/ endpoints for privacy-preserving features
- Integration tests use real ZK proof features
-**Wallet Daemon** - Deployed in container
- FastAPI service with encrypted keystore (Argon2id + XChaCha20-Poly1305)
@@ -35,6 +37,7 @@ This document tracks components that have been successfully deployed and are ope
- Mock ledger adapter with SQLite backend
- Running on port 8002, nginx proxy: /wallet/
- Dependencies: aitbc-sdk, aitbc-crypto, fastapi, uvicorn
- Bitcoin payment gateway implemented
-**Documentation** - Deployed at https://aitbc.bubuit.net/docs/
- Split documentation for different audiences
@@ -49,6 +52,15 @@ This document tracks components that have been successfully deployed and are ope
- Session-based authentication
- Exchange rate: 1 BTC = 100,000 AITBC
## Integration Tests
-**Test Suite Updates** - Completed 2026-01-26
- Security tests now use real ZK proof features
- Marketplace tests connect to live service
- Performance tests removed (too early)
- Wallet-coordinator integration added to roadmap
- 6 tests passing, 1 skipped (wallet integration)
-**ZK Applications** - Privacy-preserving features deployed
- Circom compiler v2.2.3 installed
- ZK circuits compiled (receipt_simple with 300 constraints)

View File

@@ -5,7 +5,7 @@ This document categorizes all files and folders in the repository by their statu
- **Greylist (⚠️)**: Uncertain status, may need review
- **Blacklist (❌)**: Legacy, unused, outdated, candidates for removal
Last updated: 2026-01-24
Last updated: 2026-01-26
---
@@ -103,6 +103,10 @@ Last updated: 2026-01-24
| `.gitignore` | ✅ Active | Recently updated (145 lines) |
| `pyproject.toml` | ✅ Active | Python project config |
| `.editorconfig` | ✅ Active | Editor config |
| `INTEGRATION_TEST_FIXES.md` | ✅ Active | Integration test fixes documentation |
| `INTEGRATION_TEST_UPDATES.md` | ✅ Active | Integration test real features implementation |
| `SKIPPED_TESTS_ROADMAP.md` | ✅ Active | Skipped tests roadmap status |
| `TEST_FIXES_COMPLETE.md` | ✅ Active | Complete test fixes summary |
---

View File

@@ -1,6 +1,19 @@
# AITBC Incident Runbooks
This document contains specific runbooks for common incident scenarios, based on our chaos testing validation.
This document contains specific runbooks for common incident scenarios, based on our chaos testing validation and integration test suite.
## Integration Test Status (Updated 2026-01-26)
### Current Test Coverage
- ✅ 6 integration tests passing
- ✅ Security tests using real ZK proof features
- ✅ Marketplace tests connecting to live service
- ⏸️ 1 test skipped (wallet payment flow)
### Test Environment
- Tests run against both real and mock clients
- CI/CD pipeline runs full test suite
- Local development: `python -m pytest tests/integration/ -v`
## Runbook: Coordinator API Outage

View File

@@ -1,133 +0,0 @@
# AITBC Monorepo Directory Layout (Windsurf Workspace)
> One workspace for **all** AITBC elements (client · coordinator · miner · blockchain · poolhub · marketplace · wallet · docs · ops). No Docker required.
```
aitbc/
├─ .editorconfig
├─ .gitignore
├─ README.md # Toplevel overview, quickstart, workspace tasks
├─ LICENSE
├─ windsurf/ # Windsurf prompts, tasks, run configurations
│ ├─ prompts/ # Highlevel task prompts for WS agents
│ ├─ tasks/ # Saved task flows / playbooks
│ └─ settings.json # Editor/workbench preferences for this repo
├─ scripts/ # CLI scripts (bash/python); dev + ops helpers
│ ├─ env/ # venv helpers (create, activate, pin)
│ ├─ dev/ # codegen, lint, format, typecheck wrappers
│ ├─ ops/ # backup, rotate logs, journalctl, users
│ └─ ci/ # sanity checks usable by CI (no runners assumed)
├─ configs/ # Centralized *.conf used by services
│ ├─ nginx/ # (optional) reverse proxy snippets (hostlevel)
│ ├─ systemd/ # unit files for host services (no docker)
│ ├─ security/ # fail2ban, firewall/ipset lists, tls policy
│ └─ app/ # applevel INI/YAML/TOML configs shared across apps
├─ docs/ # Markdown docs (specs, ADRs, guides)
│ ├─ 00-index.md
│ ├─ adr/ # Architecture Decision Records
│ ├─ specs/ # Protocol, API, tokenomics, flows
│ ├─ runbooks/ # Ops runbooks (rotate keys, restore, etc.)
│ └─ diagrams/ # draw.io/mermaid sources + exported PNG/SVG
├─ packages/ # Shared libraries (languagespecific)
│ ├─ py/ # Python packages (FastAPI, utils, protocol)
│ │ ├─ aitbc-core/ # Protocol models, validation, common types
│ │ ├─ aitbc-crypto/ # Key mgmt, signing, wallet primitives
│ │ ├─ aitbc-p2p/ # Node discovery, gossip, transport
│ │ ├─ aitbc-scheduler/ # Task slicing/merging, scoring, QoS
│ │ └─ aitbc-sdk/ # Client SDK for Python integrations
│ └─ js/ # Browser/Node shared libs
│ ├─ aitbc-sdk/ # Client SDK (fetch/ws), typings
│ └─ ui-widgets/ # Reusable UI bits for web apps
├─ apps/ # Firstclass runnable services & UIs
│ ├─ client-web/ # Browser UI for users (requests, wallet, status)
│ │ ├─ public/ # static assets
│ │ ├─ src/
│ │ │ ├─ pages/
│ │ │ ├─ components/
│ │ │ ├─ lib/ # uses packages/js/aitbc-sdk
│ │ │ └─ styles/
│ │ └─ README.md
│ ├─ coordinator-api/ # Central API orchestrating jobs ↔ miners
│ │ ├─ src/
│ │ │ ├─ main.py # FastAPI entrypoint
│ │ │ ├─ routes/
│ │ │ ├─ services/ # matchmaking, accounting, ratelimits
│ │ │ ├─ domain/ # job models, receipts, accounting entities
│ │ │ └─ storage/ # adapters (postgres, files, kv)
│ │ ├─ migrations/ # SQL snippets (no migration framework forced)
│ │ └─ README.md
│ ├─ miner-node/ # Worker node daemon for GPU/CPU tasks
│ │ ├─ src/
│ │ │ ├─ agent/ # job runner, sandbox mgmt, health probes
│ │ │ ├─ gpu/ # CUDA/OpenCL bindings (optional)
│ │ │ ├─ plugins/ # task kinds (LLM, ASR, vision, etc.)
│ │ │ └─ telemetry/ # metrics, logs, heartbeat
│ │ └─ README.md
│ ├─ wallet-daemon/ # Local wallet service (keys, signing, RPC)
│ │ ├─ src/
│ │ └─ README.md
│ ├─ blockchain-node/ # Minimal chain (assetbacked by compute)
│ │ ├─ src/
│ │ │ ├─ consensus/
│ │ │ ├─ mempool/
│ │ │ ├─ ledger/ # state, balances, receipts linkage
│ │ │ └─ rpc/
│ │ └─ README.md
│ ├─ pool-hub/ # Client↔miners pool + matchmaking gateway
│ │ ├─ src/
│ │ └─ README.md
│ ├─ marketplace-web/ # Web app for offers, bids, stats
│ │ ├─ public/
│ │ ├─ src/
│ │ └─ README.md
│ └─ explorer-web/ # Chain explorer (blocks, tx, receipts)
│ ├─ public/
│ ├─ src/
│ └─ README.md
├─ protocols/ # Canonical protocol definitions
│ ├─ api/ # OpenAPI/JSONSchema for REST/WebSocket
│ ├─ receipts/ # Job receipt schema, signing rules
│ ├─ payouts/ # Mint/burn, staking, fees logic (spec)
│ └─ README.md
├─ data/ # Local dev datasets (small, sample only)
│ ├─ fixtures/ # seed users, nodes, jobs
│ └─ samples/
├─ tests/ # Crossproject test harness
│ ├─ e2e/ # endtoend flows (client→coord→miner→wallet)
│ ├─ load/ # coordinator & miner stress scripts
│ └─ security/ # key rotation, signature verif, replay tests
├─ tools/ # Small CLIs, generators, mermaid->svg, etc.
│ └─ mkdiagram
└─ examples/ # Minimal runnable examples for integrators
├─ quickstart-client-python/
├─ quickstart-client-js/
└─ receipts-sign-verify/
```
## Conventions
- **Languages**: FastAPI/Python for backends; plain JS/TS for web; no Docker.
- **No global venvs**: each `apps/*` and `packages/py/*` can have its own `.venv/` (created by `scripts/env/*`).
- **Systemd over Docker**: unit files live under `configs/systemd/`, with servicespecific overrides documented in `docs/runbooks/`.
- **Static assets** belong to each web app under `public/`. Shared UI in `packages/js/ui-widgets`.
- **SQL**: keep raw SQL snippets in `apps/*/migrations/` (aligned with your “no migration framework” preference). Use `psqln` alias.
- **Security**: central policy under `configs/security/` (fail2ban, ipset lists, TLS ciphers). Keys never committed.
## Minimal READMEs to create next
Create a short `README.md` in each `apps/*` and `packages/*` with:
1. Purpose & scope
2. How to run (dev)
3. Dependencies
4. Configs consumed (from `/configs/app`)
5. Systemd unit name & port (if applicable)
## Suggested first tasks (Way of least resistance)
1. **Bootstrap coordinator-api**: scaffold FastAPI `main.py`, `/health`, `/jobs`, `/miners` routes.
2. **SDKs**: implement `packages/py/aitbc-sdk` & `packages/js/aitbc-sdk` with basic auth + job submit.
3. **miner-node prototype**: heartbeat to coordinator and noGPU "echo" job plugin.
4. **client-web**: basic UI to submit a test job and watch status stream.
5. **receipts spec**: draft `protocols/receipts` and a sign/verify example in `examples/`.

View File

@@ -497,9 +497,42 @@ Fill the intentional placeholder folders with actual content. Priority order bas
- [x] `backend.tf` - State backend configuration (S3 + DynamoDB)
- **Helm Chart Values** (`infra/helm/values/`)
- [x] `dev/values.yaml` - Development values
- [x] `staging/values.yaml` - Staging values
- [x] `prod/values.yaml` - Production values with HA, autoscaling, security
- [x] `coordinator.yaml` - Coordinator service configuration
- [x] `blockchain.yaml` - Blockchain node configuration
- [x] `wallet.yaml` - Wallet daemon configuration
- [x] `marketplace.yaml` - Marketplace service configuration
### Phase 3: Missing Integrations (High Priority)
- **Wallet-Coordinator Integration** [NEW]
- [ ] Add payment endpoints to coordinator API for job payments
- [ ] Implement escrow service for holding payments during job execution
- [ ] Integrate wallet daemon with coordinator for payment processing
- [ ] Add payment status tracking to job lifecycle
- [ ] Implement refund mechanism for failed jobs
- [ ] Add payment receipt generation and verification
- [ ] Update integration tests to use real payment flow
### Phase 4: Integration Test Improvements ✅ COMPLETE 2026-01-26
- **Security Integration Tests** ✅ COMPLETE
- [x] Updated to use real ZK proof features instead of mocks
- [x] Test confidential job creation with `require_zk_proof: True`
- [x] Verify secure job retrieval with tenant isolation
- **Marketplace Integration Tests** ✅ COMPLETE
- [x] Updated to connect to live marketplace at https://aitbc.bubuit.net/marketplace
- [x] Test marketplace accessibility and service integration
- [x] Flexible API endpoint handling
- **Performance Tests** ❌ REMOVED
- [x] Removed high throughput and load tests (too early for implementation)
- [ ] Can be added back when performance thresholds are defined
- **Test Infrastructure** ✅ COMPLETE
- [x] All tests work with both real client and mock fallback
- [x] Fixed termination issues in Windsorf environment
- [x] Current status: 6 tests passing, 1 skipped (wallet integration)
### Phase 3: Application Components (Lower Priority) ✅ COMPLETE