chore: standardize configuration, logging, and error handling across blockchain node and coordinator API
- Add infrastructure.md and workflow files to .gitignore to prevent sensitive info leaks - Change blockchain node mempool backend default from memory to database for persistence - Refactor blockchain node logger with StructuredLogFormatter and AuditLogger (consistent with coordinator) - Add structured logging fields: service, module, function, line number - Unify coordinator config with Database
This commit is contained in:
oib
441
docs/1_project/1_files.md
Normal file
441
docs/1_project/1_files.md
Normal file
@@ -0,0 +1,441 @@
|
||||
# AITBC Repository File Audit
|
||||
|
||||
This document categorizes all files and folders in the repository by their status:
|
||||
- **Whitelist (✅)**: Active, up-to-date, essential
|
||||
- **Greylist (⚠️)**: Uncertain status, may need review
|
||||
- **Blacklist (❌)**: Legacy, unused, outdated, candidates for removal
|
||||
|
||||
Last updated: 2026-02-12 (evening)
|
||||
|
||||
---
|
||||
|
||||
## Whitelist ✅ (Active & Essential)
|
||||
|
||||
### Core Applications (`apps/`)
|
||||
|
||||
| Path | Status | Notes |
|
||||
|------|--------|-------|
|
||||
| `apps/coordinator-api/` | ✅ Active | Main API service, recently updated (Feb 2026) |
|
||||
| `apps/explorer-web/` | ✅ Active | Blockchain explorer, recently updated |
|
||||
| `apps/wallet-daemon/` | ✅ Active | Wallet service, deployed in production |
|
||||
| `apps/trade-exchange/` | ✅ Active | Bitcoin exchange, deployed |
|
||||
| `apps/zk-circuits/` | ✅ Active | ZK proof circuits, deployed |
|
||||
| `apps/marketplace-web/` | ✅ Active | Marketplace frontend, deployed |
|
||||
| `apps/coordinator-api/src/app/domain/gpu_marketplace.py` | ✅ Active | GPURegistry, GPUBooking, GPUReview SQLModel tables (Feb 2026) |
|
||||
| `apps/coordinator-api/tests/test_gpu_marketplace.py` | ✅ Active | 22 GPU marketplace tests (Feb 2026) |
|
||||
| `apps/coordinator-api/tests/test_billing.py` | ✅ Active | 21 billing/usage-tracking tests (Feb 2026) |
|
||||
| `apps/coordinator-api/tests/conftest.py` | ✅ Active | App namespace isolation for coordinator tests |
|
||||
| `tests/cli/test_cli_integration.py` | ✅ Active | 24 CLI → live coordinator integration tests (Feb 2026) |
|
||||
|
||||
### Scripts (`scripts/`)
|
||||
|
||||
| Path | Status | Notes |
|
||||
|------|--------|-------|
|
||||
| `scripts/aitbc-cli.sh` | ✅ Active | Main CLI tool, heavily used |
|
||||
| `scripts/gpu/gpu_miner_host.py` | ✅ Active | Production GPU miner |
|
||||
| `scripts/gpu/gpu_miner_host_wrapper.sh` | ✅ Active | Systemd wrapper |
|
||||
| `scripts/deploy/` | ✅ Active | Deployment scripts |
|
||||
| `scripts/service/` | ✅ Active | Service management |
|
||||
| `scripts/dev_services.sh` | ✅ Active | Local development |
|
||||
| `scripts/testing/` | ✅ Active | Test scripts (moved from root, 13 files) |
|
||||
|
||||
### Infrastructure (`infra/`, `systemd/`)
|
||||
|
||||
| Path | Status | Notes |
|
||||
|------|--------|-------|
|
||||
| `infra/nginx/` | ✅ Active | Production nginx configs |
|
||||
| `systemd/aitbc-host-gpu-miner.service` | ✅ Active | Production service |
|
||||
| `systemd/coordinator-api.service` | ✅ Active | Production service |
|
||||
|
||||
### Website (`website/`)
|
||||
|
||||
| Path | Status | Notes |
|
||||
|------|--------|-------|
|
||||
| `website/docs/` | ✅ Active | HTML documentation, recently refactored |
|
||||
| `website/docs/css/docs.css` | ✅ Active | Shared CSS (1232 lines) |
|
||||
| `website/docs/js/theme.js` | ✅ Active | Theme toggle |
|
||||
| `website/index.html` | ✅ Active | Main website |
|
||||
| `website/dashboards/` | ✅ Active | Admin/miner dashboards |
|
||||
|
||||
### Documentation (`docs/`)
|
||||
|
||||
| Path | Status | Notes |
|
||||
|------|--------|-------|
|
||||
| `docs/done.md` | ✅ Active | Completion tracking |
|
||||
| `docs/roadmap.md` | ✅ Active | Development roadmap |
|
||||
| `docs/developer/testing/localhost-testing-scenario.md` | ✅ Active | Testing guide |
|
||||
| `docs/reference/components/miner_node.md` | ✅ Active | Miner documentation |
|
||||
| `docs/reference/components/coordinator_api.md` | ✅ Active | API documentation |
|
||||
| `docs/developer/integration/skills-framework.md` | ✅ Active | Skills documentation |
|
||||
| `docs/guides/` | ✅ Active | Development guides (moved from root) |
|
||||
| `docs/guides/block-production-runbook.md` | ✅ Active | Block production operational runbook |
|
||||
| `docs/guides/blockchain-node-deployment.md` | ✅ Active | Blockchain node deployment guide |
|
||||
| `docs/reports/` | ✅ Active | Generated reports (moved from root) |
|
||||
| `docs/reference/specs/receipt-spec.md` | ✅ Active | Receipt spec v1.1 (multi-sig, ZK, Merkle) |
|
||||
|
||||
### CLI Tools (`cli/`)
|
||||
|
||||
| Path | Status | Notes |
|
||||
|------|--------|-------|
|
||||
| `cli/aitbc_cli/commands/client.py` | ✅ Active | Client CLI (submit, batch-submit, templates, history) |
|
||||
| `cli/aitbc_cli/commands/miner.py` | ✅ Active | Miner CLI (register, earnings, capabilities, concurrent) |
|
||||
| `cli/aitbc_cli/commands/wallet.py` | ✅ Active | Wallet CLI (balance, staking, multisig, backup/restore) |
|
||||
| `cli/aitbc_cli/commands/auth.py` | ✅ Active | Auth CLI (login, tokens, API keys) |
|
||||
| `cli/aitbc_cli/commands/blockchain.py` | ✅ Active | Blockchain queries |
|
||||
| `cli/aitbc_cli/commands/marketplace.py` | ✅ Active | GPU marketplace operations |
|
||||
| `cli/aitbc_cli/commands/admin.py` | ✅ Active | System administration, audit logging |
|
||||
| `cli/aitbc_cli/commands/config.py` | ✅ Active | Configuration, profiles, encrypted secrets |
|
||||
| `cli/aitbc_cli/commands/monitor.py` | ✅ Active | Dashboard, metrics, alerts, webhooks |
|
||||
| `cli/aitbc_cli/commands/simulate.py` | ✅ Active | Test simulation framework |
|
||||
| `cli/aitbc_cli/plugins.py` | ✅ Active | Plugin system for custom commands |
|
||||
| `cli/aitbc_cli/main.py` | ✅ Active | CLI entry point (12 command groups) |
|
||||
| `cli/man/aitbc.1` | ✅ Active | Man page |
|
||||
| `cli/aitbc_shell_completion.sh` | ✅ Active | Shell completion script |
|
||||
| `cli/test_ollama_gpu_provider.py` | ✅ Active | GPU testing |
|
||||
| `.github/workflows/cli-tests.yml` | ✅ Active | CI/CD for CLI tests (Python 3.10/3.11/3.12) |
|
||||
|
||||
### Home Scripts (`home/`)
|
||||
|
||||
| Path | Status | Notes |
|
||||
|------|--------|-------|
|
||||
| `home/client/` | ✅ Active | Client test scripts |
|
||||
| `home/miner/` | ✅ Active | Miner test scripts |
|
||||
| `home/quick_job.py` | ✅ Active | Quick job submission |
|
||||
| `home/simple_job_flow.py` | ✅ Active | Job flow testing |
|
||||
|
||||
### Plugins (`plugins/`)
|
||||
|
||||
| Path | Status | Notes |
|
||||
|------|--------|-------|
|
||||
| `plugins/ollama/` | ✅ Active | Ollama integration |
|
||||
|
||||
### Development Utilities (`dev-utils/`)
|
||||
|
||||
| Path | Status | Notes |
|
||||
|------|--------|-------|
|
||||
| `dev-utils/` | ✅ Active | Development utilities (newly created) |
|
||||
| `dev-utils/aitbc-pythonpath.pth` | ✅ Active | Python path configuration |
|
||||
|
||||
### Data Directory (`data/`)
|
||||
|
||||
| Path | Status | Notes |
|
||||
|------|--------|-------|
|
||||
| `data/` | ✅ Active | Runtime data directory (gitignored) |
|
||||
| `data/coordinator.db` | ⚠️ Runtime | SQLite database, moved from root |
|
||||
|
||||
### Root Files
|
||||
|
||||
| Path | Status | Notes |
|
||||
|------|--------|-------|
|
||||
| `README.md` | ✅ Active | Project readme, updated with new structure |
|
||||
| `LICENSE` | ✅ Active | License file |
|
||||
| `.gitignore` | ✅ Active | Recently updated (145 lines) |
|
||||
| `pyproject.toml` | ✅ Active | Python project config |
|
||||
| `.editorconfig` | ✅ Active | Editor config |
|
||||
| `pytest.ini` | ✅ Active | Pytest configuration with custom markers |
|
||||
| `CLEANUP_SUMMARY.md` | ✅ Active | Documentation of directory cleanup |
|
||||
| `test_block_import.py` | ⚠️ Duplicate | Recreated in root (exists in scripts/testing/) |
|
||||
|
||||
---
|
||||
|
||||
### Blockchain Node (`apps/blockchain-node/`)
|
||||
|
||||
| Path | Status | Notes |
|
||||
|------|--------|-------|
|
||||
| `apps/blockchain-node/` | ✅ Active | Blockchain node with PoA, mempool, sync (Stage 20/21/22 complete) |
|
||||
| `apps/blockchain-node/src/aitbc_chain/mempool.py` | ✅ Active | Dual-backend mempool (memory + SQLite) |
|
||||
| `apps/blockchain-node/src/aitbc_chain/sync.py` | ✅ Active | Chain sync with conflict resolution |
|
||||
| `apps/blockchain-node/src/aitbc_chain/consensus/poa.py` | ✅ Active | PoA proposer with circuit breaker |
|
||||
| `apps/blockchain-node/src/aitbc_chain/app.py` | ✅ Active | FastAPI app with rate limiting middleware |
|
||||
| `apps/blockchain-node/tests/test_mempool.py` | ✅ Active | 27 mempool tests |
|
||||
| `apps/blockchain-node/tests/test_sync.py` | ✅ Active | 23 sync tests |
|
||||
|
||||
### Smart Contracts (`contracts/`)
|
||||
|
||||
| Path | Status | Notes |
|
||||
|------|--------|-------|
|
||||
| `contracts/ZKReceiptVerifier.sol` | ✅ Active | ZK receipt verifier contract |
|
||||
| `contracts/Groth16Verifier.sol` | ✅ Active | Groth16 verifier stub (snarkjs-replaceable) |
|
||||
| `contracts/scripts/security-analysis.sh` | ✅ Active | Slither + Mythril analysis script |
|
||||
| `contracts/scripts/deploy-testnet.sh` | ✅ Active | Testnet deployment script |
|
||||
|
||||
---
|
||||
|
||||
## Greylist ⚠️ (Needs Review)
|
||||
|
||||
### Packages
|
||||
|
||||
| Path | Status | Notes |
|
||||
|------|--------|-------|
|
||||
| `packages/solidity/aitbc-token/` | 📋 Planned | Smart contracts, deployment planned - see roadmap Stage 20 |
|
||||
|
||||
### Scripts
|
||||
|
||||
| Path | Status | Notes |
|
||||
|------|--------|-------|
|
||||
| `scripts/test/` | ✅ Keep | 7 test scripts, all current (Jan 2026) |
|
||||
|
||||
### Documentation
|
||||
|
||||
| Path | Status | Notes |
|
||||
|------|--------|-------|
|
||||
| `docs/developer/` | ✅ Keep | 6 markdown files |
|
||||
| `docs/operator/` | ✅ Keep | 5 markdown files |
|
||||
| `docs/user/` | ✅ Keep | 1 markdown file |
|
||||
| `docs/tutorials/` | ✅ Keep | 3 markdown files |
|
||||
|
||||
### Infrastructure
|
||||
|
||||
| Path | Status | Notes |
|
||||
|------|--------|-------|
|
||||
| `infra/k8s/` | ✅ Keep | 5 yaml files (backup, cert-manager, netpol, sealed-secrets) |
|
||||
|
||||
### Extensions
|
||||
|
||||
| Path | Status | Notes |
|
||||
|------|--------|-------|
|
||||
| `extensions/aitbc-wallet-firefox/` | ✅ Keep | Firefox extension source (7 files) |
|
||||
| `extensions/aitbc-wallet-firefox-v1.0.5.xpi` | ✅ Keep | Built extension package |
|
||||
|
||||
---
|
||||
|
||||
## Future Placeholders 📋 (Keep - Will Be Populated)
|
||||
|
||||
These empty folders are intentional scaffolding for planned future work per the roadmap.
|
||||
|
||||
| Path | Status | Roadmap Stage |
|
||||
|------|--------|---------------|
|
||||
| `docs/user/guides/` | 📋 Placeholder | Stage 5 - Documentation |
|
||||
| `docs/developer/tutorials/` | 📋 Placeholder | Stage 5 - Documentation |
|
||||
| `docs/reference/specs/` | 📋 Placeholder | Stage 5 - Documentation |
|
||||
| `infra/terraform/environments/staging/` | 📋 Placeholder | Stage 5 - Infrastructure |
|
||||
| `infra/terraform/environments/prod/` | 📋 Placeholder | Stage 5 - Infrastructure |
|
||||
| `infra/helm/values/dev/` | 📋 Placeholder | Stage 5 - Infrastructure |
|
||||
| `infra/helm/values/staging/` | 📋 Placeholder | Stage 5 - Infrastructure |
|
||||
| `infra/helm/values/prod/` | 📋 Placeholder | Stage 5 - Infrastructure |
|
||||
| `apps/coordinator-api/migrations/` | 📋 Placeholder | Alembic migrations |
|
||||
| `apps/pool-hub/src/app/routers/` | 📋 Placeholder | Stage 3 - Pool Hub |
|
||||
| `apps/pool-hub/src/app/registry/` | 📋 Placeholder | Stage 3 - Pool Hub |
|
||||
| `apps/pool-hub/src/app/scoring/` | 📋 Placeholder | Stage 3 - Pool Hub |
|
||||
|
||||
---
|
||||
|
||||
## Blacklist ❌ (Abandoned - Remove)
|
||||
|
||||
### Abandoned Empty Folders (Created but never used)
|
||||
|
||||
| Path | Status | Notes |
|
||||
|------|--------|-------|
|
||||
| `apps/client-web/src/` | ❌ Remove | Created Sep 2025, never implemented |
|
||||
| `apps/client-web/public/` | ❌ Remove | Created Sep 2025, never implemented |
|
||||
| `apps/marketplace-ui/` | ❌ Remove | Superseded by `marketplace-web` |
|
||||
| `apps/wallet-cli/` | ❌ Remove | Superseded by `cli/wallet.py` |
|
||||
| `packages/py/aitbc-core/src/` | ❌ Remove | Created Sep 2025, never implemented |
|
||||
| `packages/py/aitbc-p2p/src/` | ❌ Remove | Created Sep 2025, never implemented |
|
||||
| `packages/py/aitbc-scheduler/src/` | ❌ Remove | Created Sep 2025, never implemented |
|
||||
| `packages/js/ui-widgets/src/` | ❌ Remove | Created Sep 2025, never implemented |
|
||||
| `protocols/api/` | ❌ Remove | Never implemented |
|
||||
| `protocols/payouts/` | ❌ Remove | Never implemented |
|
||||
| `data/fixtures/` | ❌ Remove | Never populated |
|
||||
| `data/samples/` | ❌ Remove | Never populated |
|
||||
| `tools/mkdiagram/` | ❌ Remove | Never implemented |
|
||||
| `examples/quickstart-client-python/` | ❌ Remove | Never implemented |
|
||||
| `examples/quickstart-client-js/node/` | ❌ Remove | Never implemented |
|
||||
| `examples/quickstart-client-js/browser/` | ❌ Remove | Never implemented |
|
||||
| `examples/receipts-sign-verify/python/` | ❌ Remove | Never implemented |
|
||||
| `examples/receipts-sign-verify/js/` | ❌ Remove | Never implemented |
|
||||
| `scripts/env/` | ❌ Remove | Never populated |
|
||||
| `windsurf/prompts/` | ❌ Remove | Superseded by `.windsurf/` |
|
||||
| `windsurf/tasks/` | ❌ Remove | Superseded by `.windsurf/` |
|
||||
|
||||
### Duplicate/Redundant Folders
|
||||
|
||||
| Path | Status | Notes |
|
||||
|------|--------|-------|
|
||||
| `python-sdk/` | ❌ Duplicate | Duplicates `packages/py/aitbc-sdk/` |
|
||||
| `windsurf/` | ❌ Duplicate | Superseded by `.windsurf/` |
|
||||
| `configs/` | ❌ Duplicate | Empty subfolders, duplicates `infra/` and `systemd/` |
|
||||
| `docs/user-guide/` | ❌ Duplicate | Duplicates `docs/user/` |
|
||||
|
||||
### Ecosystem Folders (Scaffolded but Unused)
|
||||
|
||||
| Path | Status | Notes |
|
||||
|------|--------|-------|
|
||||
| `ecosystem/` | ❌ Unused | Only has empty `academic/` subfolder |
|
||||
| `ecosystem-analytics/` | ❌ Unused | Scaffolded Dec 2025, never used |
|
||||
| `ecosystem-certification/` | ❌ Unused | Scaffolded Dec 2025, never used |
|
||||
| `ecosystem-extensions/` | ❌ Unused | Only has template folder |
|
||||
| `enterprise-connectors/` | ❌ Unused | Scaffolded Dec 2025, never used |
|
||||
|
||||
### Research Folders (Scaffolded but Unused)
|
||||
|
||||
| Path | Status | Notes |
|
||||
|------|--------|-------|
|
||||
| `research/autonomous-agents/` | ❌ Unused | Scaffolded, no active work |
|
||||
| `research/consortium/` | ❌ Unused | Scaffolded, no active work |
|
||||
| `research/prototypes/` | ❌ Unused | Scaffolded, no active work |
|
||||
| `research/standards/` | ❌ Unused | Scaffolded, no active work |
|
||||
|
||||
### Generated/Build Artifacts (Should be in .gitignore)
|
||||
|
||||
| Path | Status | Notes |
|
||||
|------|--------|-------|
|
||||
| `packages/solidity/aitbc-token/typechain-types/` | ❌ Generated | Build artifact |
|
||||
| `apps/explorer-web/dist/` | ❌ Generated | Build artifact |
|
||||
| `logs/` | ❌ Generated | Runtime logs |
|
||||
|
||||
---
|
||||
|
||||
## Issues Found (2026-02-11)
|
||||
|
||||
### Empty Directories (Delete)
|
||||
|
||||
| Path | Action |
|
||||
|------|--------|
|
||||
| `apps/blockchain-node/src/aitbc_chain/ledger/` | Delete — empty placeholder, never implemented |
|
||||
| `apps/blockchain-node/src/aitbc_chain/mempool/` | Delete — empty dir, mempool logic is in `mempool.py` |
|
||||
| `apps/coordinator-api/src/app/ws/` | Delete — empty WebSocket placeholder, never implemented |
|
||||
| `apps/explorer-web/public/js/components/` | Delete — empty, TS components are in `src/components/` |
|
||||
| `apps/explorer-web/public/js/pages/` | Delete — empty, TS pages are in `src/pages/` |
|
||||
| `apps/explorer-web/public/js/vendors/` | Delete — empty vendor dir |
|
||||
| `apps/explorer-web/public/assets/` | Delete — empty assets dir |
|
||||
| `packages/py/aitbc-crypto/build/bdist.linux-x86_64/` | Delete — build artifact |
|
||||
|
||||
### Files in Wrong Location (Move)
|
||||
|
||||
| Current Path | Correct Path | Reason |
|
||||
|-------------|-------------|--------|
|
||||
| `apps/coordinator-api/coordinator.db` | gitignored / `data/` | SQLite database should not be in git |
|
||||
| `apps/coordinator-api/.env` | gitignored | Environment file with secrets, should not be in git |
|
||||
| `apps/.service_pids` | gitignored | Runtime PID file, should not be in git |
|
||||
| `src/aitbc_chain/` | `apps/blockchain-node/src/aitbc_chain/` | Duplicate/stale copy of blockchain node source |
|
||||
| `website/docs-clients.html` | `website/docs/docs-clients.html` | Inconsistent location, duplicate of file in `docs/` |
|
||||
| `website/docs-developers.html` | `website/docs/docs-developers.html` | Inconsistent location, duplicate of file in `docs/` |
|
||||
| `website/docs-miners.html` | `website/docs/docs-miners.html` | Inconsistent location, duplicate of file in `docs/` |
|
||||
| `website/docs-index.html` | `website/docs/index.html` | Inconsistent location, duplicate of file in `docs/` |
|
||||
|
||||
### Legacy Files (Delete)
|
||||
|
||||
| Path | Reason |
|
||||
|------|--------|
|
||||
| `SECURITY_CLEANUP_GUIDE.md` | One-time cleanup guide, already completed |
|
||||
| `apps/trade-exchange/index_working.html` | Backup copy of `index.html` |
|
||||
| `apps/trade-exchange/index.prod.html` | Superseded by `build.py` production build |
|
||||
| `apps/trade-exchange/index.real.html` | Superseded by `build.py` production build |
|
||||
| `tests/conftest_fixtures.py` | Unused alternate conftest |
|
||||
| `tests/conftest_full.py` | Unused alternate conftest |
|
||||
| `tests/conftest_path.py` | Unused alternate conftest |
|
||||
| `tests/pytest_simple.ini` | Duplicate of root `pytest.ini` |
|
||||
| `tests/test_blockchain_simple.py` | Superseded by `test_blockchain_nodes.py` |
|
||||
| `tests/test_blockchain_final.py` | Superseded by `test_blockchain_nodes.py` |
|
||||
| `tests/test_discovery.py` | One-time discovery script |
|
||||
| `tests/test_windsurf_integration.py` | IDE-specific test, not for GitHub |
|
||||
| `scripts/exchange-router-fixed.py` | One-time fix script |
|
||||
| `scripts/start_mock_blockchain.sh` | Superseded by `tests/mock_blockchain_node.py` |
|
||||
| `apps/marketplace-web/src/counter.ts` | Vite template boilerplate, unused |
|
||||
| `apps/marketplace-web/src/typescript.svg` | Vite template boilerplate, unused |
|
||||
| `apps/marketplace-web/public/vite.svg` | Vite template boilerplate, unused |
|
||||
| `.vscode/` | IDE-specific, should be gitignored |
|
||||
|
||||
### Debug Print Statements (Replace with logging)
|
||||
|
||||
| File | Lines | Statement |
|
||||
|------|-------|-----------|
|
||||
| `apps/coordinator-api/src/app/routers/exchange.py` | 112 | `print(f"Error minting tokens: {e}")` |
|
||||
| `apps/coordinator-api/src/app/routers/governance.py` | 352-376 | 4x `print(f"Executing ...")` |
|
||||
| `apps/coordinator-api/src/app/services/receipts.py` | 132 | `print(f"Failed to generate ZK proof: {e}")` |
|
||||
| `apps/coordinator-api/src/app/services/blockchain.py` | 47 | `print(f"Error getting balance: {e}")` |
|
||||
| `apps/coordinator-api/src/app/services/bitcoin_wallet.py` | 34-134 | 8x `print(...)` debug statements |
|
||||
| `apps/coordinator-api/src/app/storage/db_pg.py` | 206 | `print("✅ PostgreSQL database initialized successfully!")` |
|
||||
|
||||
---
|
||||
|
||||
## Summary Statistics
|
||||
|
||||
| Category | Count | Action |
|
||||
|----------|-------|--------|
|
||||
| **Whitelist ✅** | ~60 items | Keep and maintain |
|
||||
| **Greylist ⚠️** | 0 items | All resolved |
|
||||
| **Placeholders 📋** | 12 folders | Fill per roadmap |
|
||||
| **Removed ❌** | 35 items | Cleaned up 2026-01-24 |
|
||||
| **Empty dirs** | 8 dirs | Delete |
|
||||
| **Misplaced files** | 8 files | Move or gitignore |
|
||||
| **Legacy files** | 18 files | Delete |
|
||||
| **Debug prints** | 17 statements | Replace with logger |
|
||||
|
||||
### Completed Actions (2026-01-24)
|
||||
|
||||
1. **Cleanup Done**:
|
||||
- ✅ Removed 21 abandoned/duplicate folders
|
||||
- ✅ Updated `.gitignore` with comprehensive rules
|
||||
- ✅ Created this audit document
|
||||
|
||||
2. **Additional Cleanup (2026-01-24)**:
|
||||
- ✅ Removed `apps/miner-node/` (superseded by `scripts/gpu/`)
|
||||
- ✅ Removed `apps/miner-dashboard/` (superseded by `website/dashboards/`)
|
||||
- ✅ Removed `docs/bootstrap/` (empty)
|
||||
- ✅ Removed 5 GPU miner variants (kept only `gpu_miner_host.py`)
|
||||
- ✅ Removed 3 extension variants (kept only `aitbc-wallet-firefox/`)
|
||||
|
||||
3. **Final Cleanup (2026-01-24)**:
|
||||
- ✅ Removed `api/` folder (mock no longer needed - using live production)
|
||||
- ✅ Removed `governance/` folder (too far in future)
|
||||
- ✅ Removed `protocols/` folder (spec moved to docs/reference/specs/)
|
||||
- ✅ Moved `protocols/receipts/spec.md` → `docs/reference/specs/receipt-spec.md`
|
||||
- ✅ Added ZKReceiptVerifier and receipt spec to roadmap Stage 20
|
||||
|
||||
4. **Placeholder Plan** (see `roadmap.md` Stage 19):
|
||||
- Q1 2026: Documentation folders (`docs/user/guides/`, `docs/developer/tutorials/`, `docs/reference/specs/`)
|
||||
- Q2 2026: Infrastructure (`infra/terraform/`, `infra/helm/`)
|
||||
- Q2 2026: Pool Hub components
|
||||
|
||||
5. **Directory Organization (2026-01-29)**:
|
||||
- ✅ Created `docs/guides/` and moved 2 guide files from root
|
||||
- ✅ Created `docs/reports/` and moved 10 report files from root
|
||||
- ✅ Created `scripts/testing/` and moved 13 test scripts from root
|
||||
- ✅ Created `dev-utils/` and moved `aitbc-pythonpath.pth`
|
||||
- ✅ Moved `coordinator.db` to `data/` directory
|
||||
- ✅ Updated README.md with new structure
|
||||
- ✅ Created index README files for new directories
|
||||
|
||||
---
|
||||
|
||||
## Folder Structure Recommendation
|
||||
|
||||
```
|
||||
aitbc/
|
||||
├── apps/ # Core applications
|
||||
│ ├── coordinator-api/ # ✅ Keep
|
||||
│ ├── explorer-web/ # ✅ Keep
|
||||
│ ├── marketplace-web/ # ✅ Keep
|
||||
│ ├── wallet-daemon/ # ✅ Keep
|
||||
│ └── zk-circuits/ # ✅ Keep
|
||||
├── cli/ # ✅ CLI tools
|
||||
├── docs/ # ✅ Markdown documentation
|
||||
│ ├── guides/ # Development guides
|
||||
│ └── reports/ # Generated reports
|
||||
├── infra/ # ✅ Infrastructure configs
|
||||
├── packages/ # ✅ Keep (aitbc-crypto, aitbc-sdk, aitbc-token)
|
||||
├── plugins/ # ✅ Keep (ollama)
|
||||
├── scripts/ # ✅ Keep - organized
|
||||
│ └── testing/ # Test scripts
|
||||
├── systemd/ # ✅ Keep
|
||||
├── tests/ # ✅ Keep (e2e, integration, unit, security, load)
|
||||
├── website/ # ✅ Keep
|
||||
├── dev-utils/ # ✅ Development utilities
|
||||
├── data/ # ✅ Runtime data (gitignored)
|
||||
└── .windsurf/ # ✅ Keep
|
||||
```
|
||||
|
||||
**Folders Removed (2026-01-24)**:
|
||||
- ✅ `ecosystem*/` (all 4 folders) - removed
|
||||
- ✅ `enterprise-connectors/` - removed
|
||||
- ✅ `research/` - removed
|
||||
- ✅ `python-sdk/` - removed (duplicate)
|
||||
- ✅ `windsurf/` - removed (duplicate of `.windsurf/`)
|
||||
- ✅ `configs/` - removed (duplicated `infra/`)
|
||||
- ✅ Empty `apps/` subfolders - removed (client-web, marketplace-ui, wallet-cli)
|
||||
- ✅ Empty `packages/` subfolders - removed (aitbc-core, aitbc-p2p, aitbc-scheduler, ui-widgets)
|
||||
- ✅ Empty `examples/` subfolders - removed
|
||||
- ✅ `tools/` - removed (empty)
|
||||
- ✅ `docs/user-guide/` - removed (duplicate)
|
||||
972
docs/1_project/2_roadmap.md
Normal file
972
docs/1_project/2_roadmap.md
Normal file
@@ -0,0 +1,972 @@
|
||||
# AITBC Development Roadmap
|
||||
|
||||
This roadmap aggregates high-priority tasks derived from the bootstrap specifications in `docs/bootstrap/` and tracks progress across the monorepo. Update this document as milestones evolve.
|
||||
|
||||
## Stage 1 — Upcoming Focus Areas [COMPLETED: 2025-12-22]
|
||||
|
||||
- **Blockchain Node Foundations**
|
||||
- ✅ Bootstrap module layout in `apps/blockchain-node/src/`.
|
||||
- ✅ Implement SQLModel schemas and RPC stubs aligned with historical/attested receipts.
|
||||
|
||||
- **Explorer Web Enablement**
|
||||
- ✅ Finish mock integration across all pages and polish styling + mock/live toggle.
|
||||
- ✅ Begin wiring coordinator endpoints (e.g., `/v1/jobs/{job_id}/receipts`).
|
||||
|
||||
- **Marketplace Web Scaffolding**
|
||||
- ✅ Scaffold Vite/vanilla frontends consuming coordinator receipt history endpoints and SDK examples.
|
||||
|
||||
- **Pool Hub Services**
|
||||
- ✅ Initialize FastAPI project, scoring registry, and telemetry ingestion hooks leveraging coordinator/miner metrics.
|
||||
|
||||
- **CI Enhancements**
|
||||
- ✅ Add blockchain-node tests once available and frontend build/lint checks to `.github/workflows/python-tests.yml` or follow-on workflows.
|
||||
- ✅ Provide systemd unit + installer scripts under `scripts/` for streamlined deployment.
|
||||
|
||||
## Stage 2 — Core Services (MVP) [COMPLETED: 2025-12-22]
|
||||
|
||||
- **Coordinator API**
|
||||
- ✅ Scaffold FastAPI project (`apps/coordinator-api/src/app/`).
|
||||
- ✅ Implement job submission, status, result endpoints.
|
||||
- ✅ Add miner registration, heartbeat, poll, result routes.
|
||||
- ✅ Wire SQLite persistence for jobs, miners, receipts (historical `JobReceipt` table).
|
||||
- ✅ Provide `.env.example`, `pyproject.toml`, and run scripts.
|
||||
- ✅ Deploy minimal version in container with nginx proxy
|
||||
|
||||
- **Miner Node**
|
||||
- ✅ Implement capability probe and control loop (register → heartbeat → fetch jobs).
|
||||
- ✅ Build CLI and Python runners with sandboxed work dirs (result reporting stubbed to coordinator).
|
||||
|
||||
- **Blockchain Node**
|
||||
- ✅ Define SQLModel schema for blocks, transactions, accounts, receipts (`apps/blockchain-node/src/aitbc_chain/models.py`).
|
||||
- ✅ Harden schema parity across runtime + storage:
|
||||
- Alembic baseline + follow-on migrations in `apps/blockchain-node/migrations/` now track the SQLModel schema (blocks, transactions, receipts, accounts).
|
||||
- Added `Relationship` + `ForeignKey` wiring in `apps/blockchain-node/src/aitbc_chain/models.py` for block ↔ transaction ↔ receipt joins.
|
||||
- Introduced hex/enum validation hooks via Pydantic validators to ensure hash integrity and safe persistence.
|
||||
- ✅ Implement PoA proposer loop with block assembly (`apps/blockchain-node/src/aitbc_chain/consensus/poa.py`).
|
||||
- ✅ Expose REST RPC endpoints for tx submission, balances, receipts (`apps/blockchain-node/src/aitbc_chain/rpc/router.py`).
|
||||
- ✅ Deliver WebSocket RPC + P2P gossip layer:
|
||||
- ✅ Stand up WebSocket subscription endpoints (`apps/blockchain-node/src/aitbc_chain/rpc/websocket.py`) mirroring REST payloads.
|
||||
- ✅ Implement pub/sub transport for block + transaction gossip backed by an in-memory broker (Starlette `Broadcast` or Redis) with configurable fan-out.
|
||||
- ✅ Add integration tests and load-test harness ensuring gossip convergence and back-pressure handling.
|
||||
- ✅ Ship devnet scripts (`apps/blockchain-node/scripts/`).
|
||||
- ✅ Add observability hooks (JSON logging, Prometheus metrics) and integrate coordinator mock into devnet tooling.
|
||||
- ✅ Expand observability dashboards + miner mock integration:
|
||||
- Build Grafana dashboards for consensus health (block intervals, proposer rotation) and RPC latency (`apps/blockchain-node/observability/`).
|
||||
- Expose miner mock telemetry (job throughput, error rates) via shared Prometheus registry and ingest into blockchain-node dashboards.
|
||||
- Add alerting rules (Prometheus `Alertmanager`) for stalled proposers, queue saturation, and miner mock disconnects.
|
||||
- Wire coordinator mock into devnet tooling to simulate real-world load and validate observability hooks.
|
||||
|
||||
- **Receipt Schema**
|
||||
- ✅ Finalize canonical JSON receipt format under `protocols/receipts/` (includes sample signed receipts).
|
||||
- ✅ Implement signing/verification helpers in `packages/py/aitbc-crypto` (JS SDK pending).
|
||||
- ✅ Translate `docs/bootstrap/aitbc_tech_plan.md` contract skeleton into Solidity project (`packages/solidity/aitbc-token/`).
|
||||
- ✅ Add deployment/test scripts and document minting flow (`packages/solidity/aitbc-token/scripts/` and `docs/run.md`).
|
||||
|
||||
- **Wallet Daemon**
|
||||
- ✅ Implement encrypted keystore (Argon2id + XChaCha20-Poly1305) via `KeystoreService`.
|
||||
- ✅ Provide REST and JSON-RPC endpoints for wallet management and signing (`api_rest.py`, `api_jsonrpc.py`).
|
||||
- ✅ Add mock ledger adapter with SQLite backend powering event history (`ledger_mock/`).
|
||||
- ✅ Integrate Python receipt verification helpers (`aitbc_sdk`) and expose API/service utilities validating miner + coordinator signatures.
|
||||
- ✅ Harden REST API workflows (create/list/unlock/sign) with structured password policy enforcement and deterministic pytest coverage in `apps/wallet-daemon/tests/test_wallet_api.py`.
|
||||
- ✅ Implement Wallet SDK receipt ingestion + attestation surfacing:
|
||||
- Added `/v1/jobs/{job_id}/receipts` client helpers with cursor pagination, retry/backoff, and summary reporting (`packages/py/aitbc-sdk/src/receipts.py`).
|
||||
- Reused crypto helpers to validate miner and coordinator signatures, capturing per-key failure reasons for downstream UX.
|
||||
- Surfaced aggregated attestation status (`ReceiptStatus`) and failure diagnostics for SDK + UI consumers; JS helper parity still planned.
|
||||
|
||||
## Stage 3 — Pool Hub & Marketplace [COMPLETED: 2025-12-22]
|
||||
|
||||
- **Pool Hub**
|
||||
- ✅ Implement miner registry, scoring engine, and `/v1/match` API with Redis/PostgreSQL backing stores.
|
||||
- ✅ Add observability endpoints (`/v1/health`, `/v1/metrics`) plus Prometheus instrumentation and integration tests.
|
||||
|
||||
- **Marketplace Web**
|
||||
- ✅ Initialize Vite project with vanilla TypeScript (`apps/marketplace-web/`).
|
||||
- ✅ Build offer list, bid form, and stats cards powered by mock data fixtures (`public/mock/`).
|
||||
- ✅ Provide API abstraction toggling mock/live mode (`src/lib/api.ts`) and wire coordinator endpoints.
|
||||
- ✅ Validate live mode against coordinator `/v1/marketplace/*` responses and add auth feature flags for rollout.
|
||||
- ✅ Deploy to production at https://aitbc.bubuit.net/marketplace/
|
||||
|
||||
- **Explorer Web**
|
||||
- ✅ Initialize Vite + TypeScript project scaffold (`apps/explorer-web/`).
|
||||
- ✅ Add routed pages for overview, blocks, transactions, addresses, receipts.
|
||||
- ✅ Seed mock datasets (`public/mock/`) and fetch helpers powering overview + blocks tables.
|
||||
- ✅ Extend mock integrations to transactions, addresses, and receipts pages.
|
||||
- ✅ Implement styling system, mock/live data toggle, and coordinator API wiring scaffold.
|
||||
- ✅ Render overview stats from mock block/transaction/receipt summaries with graceful empty-state fallbacks.
|
||||
- ✅ Validate live mode + responsive polish:
|
||||
- Hit live coordinator endpoints via nginx (`/api/explorer/blocks`, `/api/explorer/transactions`, `/api/explorer/addresses`, `/api/explorer/receipts`) via `getDataMode() === "live"` and reconcile payloads with UI models.
|
||||
- Add fallbacks + error surfacing for partial/failed live responses (toast + console diagnostics).
|
||||
- Audit responsive breakpoints (`public/css/layout.css`) and adjust grid/typography for tablet + mobile; add regression checks in Percy/Playwright snapshots.
|
||||
- ✅ Deploy to production at https://aitbc.bubuit.net/explorer/ with genesis block display
|
||||
|
||||
## Stage 4 — Observability & Production Polish
|
||||
|
||||
- **Observability & Telemetry**
|
||||
- ✅ Build Grafana dashboards for PoA consensus health (block intervals, proposer rotation cadence) leveraging `poa_last_block_interval_seconds`, `poa_proposer_rotations_total`, and per-proposer counters.
|
||||
- ✅ Surface RPC latency histograms/summaries for critical endpoints (`rpc_get_head`, `rpc_send_tx`, `rpc_submit_receipt`) and add Grafana panels with SLO thresholds.
|
||||
- ✅ Ingest miner mock telemetry (job throughput, failure rate) into the shared Prometheus registry and wire panels/alerts that correlate miner health with consensus metrics.
|
||||
|
||||
- **Explorer Web (Live Mode)**
|
||||
- ✅ Finalize live `getDataMode() === "live"` workflow: align API payload contracts, render loading/error states, and persist mock/live toggle preference.
|
||||
- ✅ Expand responsive testing (tablet/mobile) and add automated visual regression snapshots prior to launch.
|
||||
- ✅ Integrate Playwright smoke tests covering overview, blocks, and transactions pages in live mode.
|
||||
|
||||
- **Marketplace Web (Launch Readiness)**
|
||||
- ✅ Connect mock listings/bids to coordinator data sources and provide feature flags for live mode rollout.
|
||||
- ✅ Implement auth/session scaffolding for marketplace actions and document API assumptions in `apps/marketplace-web/README.md`.
|
||||
- ✅ Add Grafana panels monitoring marketplace API throughput and error rates once endpoints are live.
|
||||
|
||||
- **Operational Hardening**
|
||||
- ✅ Extend Alertmanager rules to cover RPC error spikes, proposer stalls, and miner disconnects using the new metrics.
|
||||
- ✅ Document dashboard import + alert deployment steps in `docs/run.md` for operators.
|
||||
- ✅ Prepare Stage 3 release checklist linking dashboards, alerts, and smoke tests prior to production cutover.
|
||||
- ✅ Enable host GPU miner with coordinator proxy routing and systemd-backed coordinator service; add proxy health timer.
|
||||
|
||||
## Stage 5 — Scaling & Release Readiness
|
||||
|
||||
- **Infrastructure Scaling**
|
||||
- ✅ Benchmark blockchain node throughput under sustained load; capture CPU/memory targets and suggest horizontal scaling thresholds.
|
||||
- ✅ Build Terraform/Helm templates for dev/staging/prod environments, including Prometheus/Grafana bundles.
|
||||
- ✅ Implement autoscaling policies for coordinator, miners, and marketplace services with synthetic traffic tests.
|
||||
|
||||
- **Reliability & Compliance**
|
||||
- ✅ Formalize backup/restore procedures for PostgreSQL, Redis, and ledger storage with scheduled jobs.
|
||||
- ✅ Complete security hardening review (TLS termination, API auth, secrets management) and document mitigations in `docs/security.md`.
|
||||
- ✅ Add chaos testing scripts (network partition, coordinator outage) and track mean-time-to-recovery metrics.
|
||||
|
||||
- **Product Launch Checklist**
|
||||
- ✅ Finalize public documentation (API references, onboarding guides) and publish to the docs portal.
|
||||
- ✅ Coordinate beta release timeline, including user acceptance testing of explorer/marketplace live modes.
|
||||
- ✅ Establish post-launch monitoring playbooks and on-call rotations.
|
||||
|
||||
## Stage 6 — Ecosystem Expansion
|
||||
|
||||
- **Cross-Chain & Interop**
|
||||
- ✅ Prototype cross-chain settlement hooks leveraging external bridges; document integration patterns.
|
||||
- ✅ Extend SDKs (Python/JS) with pluggable transport abstractions for multi-network support.
|
||||
- 🔄 Evaluate third-party explorer/analytics integrations and publish partner onboarding guides.
|
||||
|
||||
- **Marketplace Growth**
|
||||
- ✅ Launch incentive programs (staking, liquidity mining) and expose telemetry dashboards tracking campaign performance.
|
||||
- ✅ Implement governance module (proposal voting, parameter changes) and add API/UX flows to explorer/marketplace.
|
||||
- 🔄 Provide SLA-backed coordinator/pool hubs with capacity planning and billing instrumentation.
|
||||
|
||||
- **Developer Experience**
|
||||
- ✅ Publish advanced tutorials (custom proposers, marketplace extensions) and maintain versioned API docs.
|
||||
- 🔄 Integrate CI/CD pipelines with canary deployments and blue/green release automation.
|
||||
- 🔄 Host quarterly architecture reviews capturing lessons learned and feeding into roadmap revisions.
|
||||
|
||||
## Stage 7 — Innovation & Ecosystem Services
|
||||
|
||||
- **GPU Service Expansion**
|
||||
- ✅ Implement dynamic service registry framework for 30+ GPU-accelerated services
|
||||
- ✅ Create service definitions for AI/ML (LLM inference, image/video generation, speech recognition, computer vision, recommendation systems)
|
||||
- ✅ Create service definitions for Media Processing (video transcoding, streaming, 3D rendering, image/audio processing)
|
||||
- ✅ Create service definitions for Scientific Computing (molecular dynamics, weather modeling, financial modeling, physics simulation, bioinformatics)
|
||||
- ✅ Create service definitions for Data Analytics (big data processing, real-time analytics, graph analytics, time series analysis)
|
||||
- ✅ Create service definitions for Gaming & Entertainment (cloud gaming, asset baking, physics simulation, VR/AR rendering)
|
||||
- ✅ Create service definitions for Development Tools (GPU compilation, model training, data processing, simulation testing, code generation)
|
||||
- ✅ Deploy service provider configuration UI with dynamic service selection
|
||||
- ✅ Implement service-specific validation and hardware requirement checking
|
||||
|
||||
- **Advanced Cryptography & Privacy**
|
||||
- ✅ Research zk-proof-based receipt attestation and prototype a privacy-preserving settlement flow.
|
||||
- ✅ Add confidential transaction support with opt-in ciphertext storage and HSM-backed key management.
|
||||
- ✅ Publish threat modeling updates and share mitigations with ecosystem partners.
|
||||
|
||||
- **Enterprise Integrations**
|
||||
- ✅ Deliver reference connectors for ERP/payment systems and document SLA expectations.
|
||||
- ✅ Stand up multi-tenant coordinator infrastructure with per-tenant isolation and billing metrics.
|
||||
- ✅ Launch ecosystem certification program (SDK conformance, security best practices) with public registry.
|
||||
|
||||
- **Community & Governance**
|
||||
- ✅ Establish open RFC process, publish governance website, and schedule regular community calls.
|
||||
- ✅ Sponsor hackathons/accelerators and provide grants for marketplace extensions and analytics tooling.
|
||||
- ✅ Track ecosystem KPIs (active marketplaces, cross-chain volume) and feed them into quarterly strategy reviews.
|
||||
|
||||
## Stage 8 — Frontier R&D & Global Expansion [COMPLETED: 2025-12-28]
|
||||
|
||||
- **Protocol Evolution**
|
||||
- ✅ Launch research consortium exploring next-gen consensus (hybrid PoA/PoS) and finalize whitepapers.
|
||||
- 🔄 Prototype sharding or rollup architectures to scale throughput beyond current limits.
|
||||
- 🔄 Standardize interoperability specs with industry bodies and submit proposals for adoption.
|
||||
|
||||
- **Global Rollout**
|
||||
- 🔄 Establish regional infrastructure hubs (multi-cloud) with localized compliance and data residency guarantees.
|
||||
- 🔄 Partner with regulators/enterprises to pilot regulated marketplaces and publish compliance playbooks.
|
||||
- 🔄 Expand localization (UI, documentation, support) covering top target markets.
|
||||
|
||||
- **Long-Term Sustainability**
|
||||
- 🔄 Create sustainability fund for ecosystem maintenance, bug bounties, and community stewardship.
|
||||
- 🔄 Define succession planning for core teams, including training programs and contributor pathways.
|
||||
- 🔄 Publish bi-annual roadmap retrospectives assessing KPI alignment and revising long-term goals.
|
||||
|
||||
## Stage 9 — Moonshot Initiatives [COMPLETED: 2025-12-28]
|
||||
|
||||
- **Decentralized Infrastructure**
|
||||
- 🔄 Transition coordinator/miner roles toward community-governed validator sets with incentive alignment.
|
||||
- 🔄 Explore decentralized storage/backbone options (IPFS/Filecoin) for ledger and marketplace artifacts.
|
||||
- 🔄 Prototype fully trustless marketplace settlement leveraging zero-knowledge rollups.
|
||||
|
||||
- **AI & Automation**
|
||||
- 🔄 Integrate AI-driven monitoring/anomaly detection for proposer health, market liquidity, and fraud detection.
|
||||
- 🔄 Automate incident response playbooks with ChatOps and policy engines.
|
||||
- 🔄 Launch research into autonomous agent participation (AI agents bidding/offering in the marketplace) and governance implications.
|
||||
- **Global Standards Leadership**
|
||||
- 🔄 Chair industry working groups defining receipt/marketplace interoperability standards.
|
||||
- 🔄 Publish annual transparency reports and sustainability metrics for stakeholders.
|
||||
- 🔄 Engage with academia and open-source foundations to steward long-term protocol evolution.
|
||||
|
||||
### Stage 10 — Stewardship & Legacy Planning [COMPLETED: 2025-12-28]
|
||||
|
||||
- **Open Governance Maturity**
|
||||
- 🔄 Transition roadmap ownership to community-elected councils with transparent voting and treasury controls.
|
||||
- 🔄 Codify constitutional documents (mission, values, conflict resolution) and publish public charters.
|
||||
- 🔄 Implement on-chain governance modules for protocol upgrades and ecosystem-wide decisions.
|
||||
|
||||
- **Educational & Outreach Programs**
|
||||
- 🔄 Fund university partnerships, research chairs, and developer fellowships focused on decentralized marketplace tech.
|
||||
- 🔄 Create certification tracks and mentorship programs for new validator/operators.
|
||||
- 🔄 Launch annual global summit and publish proceedings to share best practices across partners.
|
||||
|
||||
- **Long-Term Preservation**
|
||||
- 🔄 Archive protocol specs, governance records, and cultural artifacts in decentralized storage with redundancy.
|
||||
- 🔄 Establish legal/organizational frameworks to ensure continuity across jurisdictions.
|
||||
- 🔄 Develop end-of-life/transition plans for legacy components, documenting deprecation strategies and migration tooling.
|
||||
|
||||
|
||||
## Shared Libraries & Examples
|
||||
|
||||
## Stage 11 — Trade Exchange & Token Economy [COMPLETED: 2025-12-28]
|
||||
|
||||
- **Bitcoin Wallet Integration**
|
||||
- ✅ Implement Bitcoin payment gateway for AITBC token purchases
|
||||
- ✅ Create payment request API with unique payment addresses
|
||||
- ✅ Add QR code generation for mobile payments
|
||||
- ✅ Implement real-time payment monitoring with blockchain API
|
||||
- ✅ Configure exchange rate: 1 BTC = 100,000 AITBC
|
||||
|
||||
- **User Management System**
|
||||
- ✅ Implement wallet-based authentication with session management
|
||||
- ✅ Create individual user accounts with unique wallets
|
||||
- ✅ Add user profile pages with transaction history
|
||||
- ✅ Implement secure session tokens with 24-hour expiry
|
||||
- ✅ Add login/logout functionality across all pages
|
||||
|
||||
- **Trade Exchange Platform**
|
||||
- ✅ Build responsive trading interface with real-time price updates
|
||||
- ✅ Integrate Bitcoin payment flow with QR code display
|
||||
- ✅ Add payment status monitoring and confirmation handling
|
||||
- ✅ Implement AITBC token minting upon payment confirmation
|
||||
- ✅ Deploy to production at https://aitbc.bubuit.net/Exchange/
|
||||
|
||||
- **API Infrastructure**
|
||||
- ✅ Add user management endpoints (/api/users/*)
|
||||
- ✅ Implement exchange payment endpoints (/api/exchange/*)
|
||||
- ✅ Add session-based authentication for protected routes
|
||||
- ✅ Create transaction history and balance tracking APIs
|
||||
- ✅ Fix all import and syntax errors in coordinator API
|
||||
|
||||
## Stage 13 — Explorer Live API & Reverse Proxy Fixes [COMPLETED: 2025-12-28]
|
||||
|
||||
- **Explorer Live API**
|
||||
- ✅ Enable coordinator explorer routes at `/v1/explorer/*`.
|
||||
- ✅ Expose nginx explorer proxy at `/api/explorer/*` (maps to backend `/v1/explorer/*`).
|
||||
- ✅ Fix response schema mismatches (e.g., receipts response uses `jobId`).
|
||||
|
||||
- **Coordinator API Users/Login**
|
||||
- ✅ Ensure `/v1/users/login` is registered and working.
|
||||
- ✅ Fix missing SQLModel tables by initializing DB on startup (wallet/user tables created).
|
||||
|
||||
- **nginx Reverse Proxy Hardening**
|
||||
- ✅ Fix `/api/v1/*` routing to avoid double `/v1` prefix.
|
||||
- ✅ Add compatibility proxy for Exchange: `/api/users/*` → backend `/v1/users/*`.
|
||||
|
||||
## Stage 12 — Zero-Knowledge Proof Implementation [COMPLETED: 2025-12-28]
|
||||
|
||||
- **Circom Compiler Setup**
|
||||
- ✅ Install Circom compiler v2.2.3 on production server
|
||||
- ✅ Configure Node.js environment for ZK circuit compilation
|
||||
- ✅ Install circomlib and required dependencies
|
||||
|
||||
- **ZK Circuit Development**
|
||||
- ✅ Create receipt attestation circuit (receipt_simple.circom)
|
||||
- ✅ Implement membership proof circuit template
|
||||
- ✅ Implement bid range proof circuit template
|
||||
- ✅ Compile circuits to R1CS, WASM, and symbolic files
|
||||
|
||||
- **Trusted Setup Ceremony**
|
||||
- ✅ Perform Powers of Tau setup ceremony (2^12)
|
||||
- ✅ Generate proving keys (zkey) for Groth16
|
||||
- ✅ Export verification keys for on-chain verification
|
||||
- ✅ Complete phase 2 preparation with contributions
|
||||
|
||||
- **ZK Applications API**
|
||||
- ✅ Implement identity commitment endpoints
|
||||
- ✅ Create stealth address generation service
|
||||
- ✅ Add private receipt attestation API
|
||||
- ✅ Implement group membership proof verification
|
||||
- ✅ Add private bidding functionality
|
||||
- ✅ Create computation proof verification
|
||||
- ✅ Deploy to production at /api/zk/ endpoints
|
||||
|
||||
- **Integration & Deployment**
|
||||
- ✅ Integrate ZK proof service with coordinator API
|
||||
- ✅ Configure circuit files in production environment
|
||||
- ✅ Enable ZK proof generation in coordinator service
|
||||
- ✅ Update documentation with ZK capabilities
|
||||
|
||||
## Stage 14 — Explorer JavaScript Error Fixes [COMPLETED: 2025-12-30]
|
||||
|
||||
- **JavaScript Error Resolution**
|
||||
- ✅ Fixed "can't access property 'length', t is undefined" error on Explorer page load
|
||||
- ✅ Updated fetchMock function in mockData.ts to return correct structure with 'items' property
|
||||
- ✅ Added defensive null checks in all page init functions (overview, blocks, transactions, addresses, receipts)
|
||||
- ✅ Fixed TypeScript errors for null checks and missing properties
|
||||
- ✅ Deployed fixes to production server (/var/www/aitbc.bubuit.net/explorer/)
|
||||
- ✅ Configured mock data serving from correct path (/explorer/mock/)
|
||||
|
||||
## Stage 15 — Cascade Skills Framework [COMPLETED: 2025-01-19]
|
||||
|
||||
- **Skills Infrastructure**
|
||||
- ✅ Implement Cascade skills framework for complex workflow automation
|
||||
- ✅ Create skills directory structure at `.windsurf/skills/`
|
||||
- ✅ Define skill metadata format with YAML frontmatter
|
||||
- ✅ Add progressive disclosure for intelligent skill invocation
|
||||
|
||||
- **Deploy-Production Skill**
|
||||
- ✅ Create comprehensive deployment workflow skill
|
||||
- ✅ Implement pre-deployment validation script (disk, memory, services, SSL)
|
||||
- ✅ Add environment template with all production variables
|
||||
- ✅ Create rollback procedures with emergency steps
|
||||
- ✅ Build health check script for post-deployment verification
|
||||
|
||||
- **Blockchain-Operations Skill**
|
||||
- ✅ Create node health monitoring with peer analysis and sync status
|
||||
- ✅ Implement transaction tracer for debugging and gas optimization
|
||||
- ✅ Build GPU mining optimization script for NVIDIA/AMD cards
|
||||
- ✅ Add real-time sync monitor with visual progress bar
|
||||
- ✅ Create network diagnostics tool with connectivity analysis
|
||||
|
||||
- **Skills Integration**
|
||||
- ✅ Enable automatic skill invocation based on context
|
||||
- ✅ Add manual skill triggering with keyword detection
|
||||
- ✅ Implement error handling and logging in all skills
|
||||
- ✅ Create comprehensive documentation and usage examples
|
||||
|
||||
## Stage 16 — Service Maintenance & Optimization [COMPLETED: 2026-01-21]
|
||||
|
||||
- **Service Recovery**
|
||||
- ✅ Diagnose and fix all failing AITBC container services
|
||||
- ✅ Resolve duplicate service conflicts causing port binding errors
|
||||
- ✅ Fix marketplace service implementation (missing server.py)
|
||||
- ✅ Disable redundant services to prevent resource conflicts
|
||||
|
||||
- **System Administration**
|
||||
- ✅ Configure passwordless SSH access for automation
|
||||
- ✅ Create dedicated SSH keys for secure service management
|
||||
- ✅ Document service dependencies and port mappings
|
||||
- ✅ Establish service monitoring procedures
|
||||
|
||||
- **Service Status Verification**
|
||||
- ✅ Verify all 7 core services running correctly
|
||||
- ✅ Confirm proper nginx reverse proxy configuration
|
||||
- ✅ Validate API endpoints accessibility
|
||||
- ✅ Test service recovery procedures
|
||||
|
||||
## Stage 17 — Ollama GPU Inference & CLI Tooling [COMPLETED: 2026-01-24]
|
||||
|
||||
- **End-to-End Ollama Testing**
|
||||
- ✅ Verify complete GPU inference workflow from job submission to receipt generation
|
||||
- ✅ Test Ollama integration with multiple models (llama3.2, mistral, deepseek, etc.)
|
||||
- ✅ Validate job lifecycle: QUEUED → RUNNING → COMPLETED
|
||||
- ✅ Confirm receipt generation with accurate payment calculations
|
||||
- ✅ Record transactions on blockchain with proper metadata
|
||||
|
||||
- **Coordinator API Bug Fixes**
|
||||
- ✅ Fix missing `_coerce_float()` helper function causing 500 errors
|
||||
- ✅ Deploy fix to production incus container via SSH
|
||||
- ✅ Verify result submission returns 200 OK with valid receipts
|
||||
- ✅ Validate receipt payload structure and signature generation
|
||||
|
||||
- **Miner Configuration & Optimization**
|
||||
- ✅ Fix miner ID mismatch (host-gpu-miner → ${MINER_API_KEY})
|
||||
- ✅ Enhance logging with explicit flush handlers for systemd journal
|
||||
- ✅ Configure unbuffered Python logging environment variables
|
||||
- ✅ Create systemd service unit with proper environment configuration
|
||||
|
||||
- **CLI Tooling Development**
|
||||
- ✅ Create unified bash CLI wrapper (`scripts/aitbc-cli.sh`)
|
||||
- ✅ Implement commands: submit, status, browser, blocks, receipts, cancel
|
||||
- ✅ Add admin commands: admin-miners, admin-jobs, admin-stats
|
||||
- ✅ Support environment variable overrides for URL and API keys
|
||||
- ✅ Make script executable and document usage patterns
|
||||
|
||||
- **Blockchain-Operations Skill Enhancement**
|
||||
- ✅ Add comprehensive Ollama testing scenarios to skill
|
||||
- ✅ Create detailed test documentation (`ollama-test-scenario.md`)
|
||||
- ✅ Document common issues and troubleshooting procedures
|
||||
- ✅ Add performance metrics and expected results
|
||||
- ✅ Include end-to-end automation script template
|
||||
|
||||
- **Documentation Updates**
|
||||
- ✅ Update localhost testing scenario with CLI wrapper usage
|
||||
- ✅ Convert examples to use localhost URLs (127.0.0.1)
|
||||
- ✅ Add host user paths and quick start commands
|
||||
- ✅ Document complete workflow from setup to verification
|
||||
- ✅ Update skill documentation with testing scenarios
|
||||
|
||||
## Stage 18 — Repository Reorganization & CSS Consolidation [COMPLETED: 2026-01-24]
|
||||
|
||||
- **Root Level Cleanup**
|
||||
- ✅ Move 60+ loose files from root to proper directories
|
||||
- ✅ Organize deployment scripts into `scripts/deploy/`
|
||||
- ✅ Organize GPU miner files into `scripts/gpu/`
|
||||
- ✅ Organize test/verify files into `scripts/test/`
|
||||
- ✅ Organize service management scripts into `scripts/service/`
|
||||
- ✅ Move systemd services to `systemd/`
|
||||
- ✅ Move nginx configs to `infra/nginx/`
|
||||
- ✅ Move dashboards to `website/dashboards/`
|
||||
|
||||
- **Website/Docs Folder Structure**
|
||||
- ✅ Establish `/website/docs/` as source for HTML documentation
|
||||
- ✅ Create shared CSS file (`css/docs.css`) with 1232 lines
|
||||
- ✅ Create theme toggle JavaScript (`js/theme.js`)
|
||||
- ✅ Migrate all HTML files to use external CSS (45-66% size reduction)
|
||||
- ✅ Clean `/docs/` folder to only contain mkdocs markdown files
|
||||
|
||||
- **Documentation Styling Fixes**
|
||||
- ✅ Fix dark theme background consistency across all docs pages
|
||||
- ✅ Add dark theme support to `full-documentation.html`
|
||||
- ✅ Fix Quick Start section cascade styling in docs-miners.html
|
||||
- ✅ Fix SDK Examples cascade indentation in docs-clients.html
|
||||
- ✅ Fix malformed `</code-block>` tags across all docs
|
||||
- ✅ Update API endpoint example to use Python/FastAPI
|
||||
|
||||
- **Path Reference Updates**
|
||||
- ✅ Update systemd service file with new `scripts/gpu/gpu_miner_host.py` path
|
||||
- ✅ Update skill documentation with new file locations
|
||||
- ✅ Update localhost-testing-scenario.md with correct paths
|
||||
- ✅ Update gpu_miner_host_wrapper.sh with new path
|
||||
|
||||
- **Repository Maintenance**
|
||||
- ✅ Expand .gitignore from 39 to 145 lines with organized sections
|
||||
- ✅ Add project-specific ignore rules for coordinator, explorer, GPU miner
|
||||
- ✅ Document final folder structure in done.md
|
||||
- ✅ Create `docs/files.md` file audit with whitelist/greylist/blacklist
|
||||
- ✅ Remove 35 abandoned/duplicate folders and files
|
||||
- ✅ Reorganize `docs/` folder - root contains only done.md, files.md, roadmap.md
|
||||
- ✅ Move 25 doc files to appropriate subfolders (components, deployment, migration, etc.)
|
||||
|
||||
## Stage 19 — Placeholder Content Development [PLANNED]
|
||||
|
||||
Fill the intentional placeholder folders with actual content. Priority order based on user impact.
|
||||
|
||||
### Phase 1: Documentation (High Priority)
|
||||
|
||||
- **User Guides** (`docs/user/guides/`) ✅ COMPLETE
|
||||
- [x] Bitcoin wallet setup (`BITCOIN-WALLET-SETUP.md`)
|
||||
- [x] User interface guide (`USER-INTERFACE-GUIDE.md`)
|
||||
- [x] User management setup (`USER-MANAGEMENT-SETUP.md`)
|
||||
- [x] Local assets summary (`LOCAL_ASSETS_SUMMARY.md`)
|
||||
- [x] Getting started guide (`getting-started.md`)
|
||||
- [x] Job submission workflow (`job-submission.md`)
|
||||
- [x] Payment and receipt understanding (`payments-receipts.md`)
|
||||
- [x] Troubleshooting common issues (`troubleshooting.md`)
|
||||
|
||||
- **Developer Tutorials** (`docs/developer/tutorials/`) ✅ COMPLETE
|
||||
- [x] Building a custom miner (`building-custom-miner.md`)
|
||||
- [x] Integrating with Coordinator API (`coordinator-api-integration.md`)
|
||||
- [x] Creating marketplace extensions (`marketplace-extensions.md`)
|
||||
- [x] Working with ZK proofs (`zk-proofs.md`)
|
||||
- [x] SDK usage examples (`sdk-examples.md`)
|
||||
|
||||
- **Reference Specs** (`docs/reference/specs/`) ✅ COMPLETE
|
||||
- [x] Receipt JSON schema specification (`receipt-spec.md`)
|
||||
- [x] API endpoint reference (`api-reference.md`)
|
||||
- [x] Protocol message formats (`protocol-messages.md`)
|
||||
- [x] Error codes and handling (`error-codes.md`)
|
||||
|
||||
### Phase 2: Infrastructure (Medium Priority) ✅ COMPLETE
|
||||
|
||||
- **Terraform Environments** (`infra/terraform/environments/`)
|
||||
- [x] `staging/main.tf` - Staging environment config
|
||||
- [x] `prod/main.tf` - Production environment config
|
||||
- [x] `variables.tf` - Shared variables
|
||||
- [x] `secrets.tf` - Secrets management (AWS Secrets Manager)
|
||||
- [x] `backend.tf` - State backend configuration (S3 + DynamoDB)
|
||||
|
||||
- **Helm Chart Values** (`infra/helm/values/`)
|
||||
- [x] `coordinator.yaml` - Coordinator service configuration
|
||||
- [x] `blockchain.yaml` - Blockchain node configuration
|
||||
- [x] `wallet.yaml` - Wallet daemon configuration
|
||||
- [x] `marketplace.yaml` - Marketplace service configuration
|
||||
|
||||
### Phase 3: Missing Integrations (High Priority)
|
||||
|
||||
- **Wallet-Coordinator Integration** ✅ COMPLETE
|
||||
- [x] Add payment endpoints to coordinator API for job payments (`routers/payments.py`)
|
||||
- [x] Implement escrow service for holding payments during job execution (`services/payments.py`)
|
||||
- [x] Integrate wallet daemon with coordinator for payment processing
|
||||
- [x] Add payment status tracking to job lifecycle (`domain/job.py` payment_id/payment_status)
|
||||
- [x] Implement refund mechanism for failed jobs (auto-refund on failure in `routers/miner.py`)
|
||||
- [x] Add payment receipt generation and verification (`/payments/{id}/receipt`)
|
||||
- [x] CLI payment commands: `client pay/payment-status/payment-receipt/refund` (7 tests)
|
||||
|
||||
### Phase 4: Integration Test Improvements ✅ COMPLETE 2026-01-26
|
||||
|
||||
- **Security Integration Tests** ✅ COMPLETE
|
||||
- [x] Updated to use real ZK proof features instead of mocks
|
||||
- [x] Test confidential job creation with `require_zk_proof: True`
|
||||
- [x] Verify secure job retrieval with tenant isolation
|
||||
|
||||
- **Marketplace Integration Tests** ✅ COMPLETE
|
||||
- [x] Updated to connect to live marketplace at https://aitbc.bubuit.net/marketplace
|
||||
- [x] Test marketplace accessibility and service integration
|
||||
- [x] Flexible API endpoint handling
|
||||
|
||||
- **Performance Tests** ❌ REMOVED
|
||||
- [x] Removed high throughput and load tests (too early for implementation)
|
||||
- [ ] Can be added back when performance thresholds are defined
|
||||
|
||||
- **Test Infrastructure** ✅ COMPLETE
|
||||
- [x] All tests work with both real client and mock fallback
|
||||
- [x] Fixed termination issues in Windsorf environment
|
||||
- [x] Current status: 6 tests passing, 1 skipped (wallet integration)
|
||||
|
||||
### Phase 3: Application Components (Lower Priority) ✅ COMPLETE
|
||||
|
||||
- **Pool Hub Service** (`apps/pool-hub/src/app/`)
|
||||
- [x] `routers/` - API route handlers (miners.py, pools.py, jobs.py, health.py)
|
||||
- [x] `registry/` - Miner registry implementation (miner_registry.py)
|
||||
- [x] `scoring/` - Scoring engine logic (scoring_engine.py)
|
||||
|
||||
- **Coordinator Migrations** (`apps/coordinator-api/migrations/`)
|
||||
- [x] `001_initial_schema.sql` - Initial schema migration
|
||||
- [x] `002_indexes.sql` - Index optimizations
|
||||
- [x] `003_data_migration.py` - Data migration scripts
|
||||
- [x] `README.md` - Migration documentation
|
||||
|
||||
### Placeholder Filling Schedule
|
||||
|
||||
| Folder | Target Date | Owner | Status |
|
||||
|--------|-------------|-------|--------|
|
||||
| `docs/user/guides/` | Q1 2026 | Documentation | ✅ Complete (2026-01-24) |
|
||||
| `docs/developer/tutorials/` | Q1 2026 | Documentation | ✅ Complete (2026-01-24) |
|
||||
| `docs/reference/specs/` | Q1 2026 | Documentation | ✅ Complete (2026-01-24) |
|
||||
| `infra/terraform/environments/` | Q2 2026 | DevOps | ✅ Complete (2026-01-24) |
|
||||
| `infra/helm/values/` | Q2 2026 | DevOps | ✅ Complete (2026-01-24) |
|
||||
| `apps/pool-hub/src/app/` | Q2 2026 | Backend | ✅ Complete (2026-01-24) |
|
||||
| `apps/coordinator-api/migrations/` | As needed | Backend | ✅ Complete (2026-01-24) |
|
||||
|
||||
## Stage 21 — Transaction-Dependent Block Creation [COMPLETED: 2026-01-28]
|
||||
|
||||
- **PoA Consensus Enhancement**
|
||||
- ✅ Modify PoA proposer to only create blocks when mempool has pending transactions
|
||||
- ✅ Implement HTTP polling mechanism to check RPC mempool size
|
||||
- ✅ Add transaction storage in block data with tx_count field
|
||||
- ✅ Remove processed transactions from mempool after block creation
|
||||
- ✅ Fix syntax errors and import issues in consensus/poa.py
|
||||
|
||||
- **Architecture Implementation**
|
||||
- ✅ RPC Service: Receives transactions and maintains in-memory mempool
|
||||
- ✅ Metrics Endpoint: Exposes mempool_size for node polling
|
||||
- ✅ Node Process: Polls metrics every 2 seconds, creates blocks only when needed
|
||||
- ✅ Eliminates empty blocks from blockchain
|
||||
- ✅ Maintains block integrity with proper transaction inclusion
|
||||
|
||||
- **Testing and Validation**
|
||||
- ✅ Deploy changes to both Node 1 and Node 2
|
||||
- ✅ Verify proposer skips block creation when no transactions
|
||||
- ✅ Confirm blocks are created when transactions are submitted
|
||||
- ✅ Fix gossip broker integration issues
|
||||
- ✅ Implement message passing solution for transaction synchronization
|
||||
|
||||
## Stage 22 — Future Enhancements ✅ COMPLETE
|
||||
|
||||
- **Shared Mempool Implementation** ✅
|
||||
- [x] Implement database-backed mempool for true sharing between services (`DatabaseMempool` with SQLite)
|
||||
- [x] Add gossip-based pub/sub for real-time transaction propagation (gossip broker on `/sendTx`)
|
||||
- [x] Optimize polling with fee-based prioritization and drain API
|
||||
|
||||
- **Advanced Block Production** ✅
|
||||
- [x] Implement block size limits and gas optimization (`max_block_size_bytes`, `max_txs_per_block`)
|
||||
- [x] Add transaction prioritization based on fees (highest-fee-first drain)
|
||||
- [x] Implement batch transaction processing (proposer drains + batch-inserts into block)
|
||||
- [x] Add block production metrics and monitoring (build duration, tx count, fees, interval)
|
||||
|
||||
- **Production Hardening** ✅
|
||||
- [x] Add comprehensive error handling for network failures (RPC 400/503, mempool ValueError)
|
||||
- [x] Implement graceful degradation when RPC service unavailable (circuit breaker skip)
|
||||
- [x] Add circuit breaker pattern for mempool polling (`CircuitBreaker` class with threshold/timeout)
|
||||
- [x] Create operational runbooks for block production issues (`docs/guides/block-production-runbook.md`)
|
||||
|
||||
## Stage 21 — Cross-Site Synchronization [COMPLETED: 2026-01-29]
|
||||
|
||||
Enable blockchain nodes to synchronize across different sites via RPC.
|
||||
|
||||
### Multi-Site Architecture
|
||||
- **Site A (localhost)**: 2 nodes (ports 8081, 8082)
|
||||
- **Site B (remote host)**: ns3 server (95.216.198.140)
|
||||
- **Site C (remote container)**: 1 node (port 8082)
|
||||
- **Network**: Cross-site RPC synchronization enabled
|
||||
|
||||
### Implementation
|
||||
- **Synchronization Module** ✅ COMPLETE
|
||||
- [x] Create `/src/aitbc_chain/cross_site.py` module
|
||||
- [x] Implement remote endpoint polling (10-second interval)
|
||||
- [x] Add transaction propagation between sites
|
||||
- [x] Detect height differences between nodes
|
||||
- [x] Integrate into node lifecycle (start/stop)
|
||||
|
||||
- **Configuration** ✅ COMPLETE
|
||||
- [x] Add `cross_site_sync_enabled` to ChainSettings
|
||||
- [x] Add `cross_site_remote_endpoints` list
|
||||
- [x] Add `cross_site_poll_interval` setting
|
||||
- [x] Configure endpoints for all 3 nodes
|
||||
|
||||
- **Deployment** ✅ COMPLETE
|
||||
- [x] Deploy to all 3 nodes
|
||||
- [x] Fix Python compatibility issues
|
||||
- [x] Fix RPC endpoint URL paths
|
||||
- [x] Verify network connectivity
|
||||
|
||||
### Current Status
|
||||
- All nodes running with cross-site sync enabled
|
||||
- Transaction propagation working
|
||||
- ✅ Block sync fully implemented with transaction support
|
||||
- ✅ Transaction data properly saved during block import
|
||||
- Nodes maintain independent chains (PoA design)
|
||||
- Nginx routing fixed to port 8081 for blockchain-rpc-2
|
||||
|
||||
### Future Enhancements ✅ COMPLETE
|
||||
- [x] ✅ Block import endpoint fully implemented with transactions
|
||||
- [x] Implement conflict resolution for divergent chains (`ChainSync._resolve_fork` with longest-chain rule)
|
||||
- [x] Add sync metrics and monitoring (15 sync metrics: received, accepted, rejected, forks, reorgs, duration)
|
||||
- [x] Add proposer signature validation for imported blocks (`ProposerSignatureValidator` with trusted proposer set)
|
||||
|
||||
## Stage 20 — Technical Debt Remediation [PLANNED]
|
||||
|
||||
Address known issues in existing components that are blocking production use.
|
||||
|
||||
### Blockchain Node (`apps/blockchain-node/`)
|
||||
|
||||
Current Status: SQLModel schema fixed, relationships working, tests passing.
|
||||
|
||||
- **SQLModel Compatibility** ✅ COMPLETE
|
||||
- [x] Audit current SQLModel schema definitions in `models.py`
|
||||
- [x] Fix relationship and foreign key wiring issues
|
||||
- [x] Add explicit `__tablename__` to all models
|
||||
- [x] Add `sa_relationship_kwargs` for lazy loading
|
||||
- [x] Document SQLModel validator limitation (table=True bypasses validators)
|
||||
- [x] Integration tests passing (2 passed, 1 skipped)
|
||||
- [x] Schema documentation (`docs/SCHEMA.md`)
|
||||
|
||||
- **Production Readiness** ✅ COMPLETE
|
||||
- [x] Fix PoA consensus loop stability (retry logic in `_fetch_chain_head`, circuit breaker, health tracking)
|
||||
- [x] Harden RPC endpoints for production load (rate limiting middleware, CORS, `/health` endpoint)
|
||||
- [x] Add proper error handling and logging (`RequestLoggingMiddleware`, unhandled error catch, structured logging)
|
||||
- [x] Create deployment documentation (`docs/guides/blockchain-node-deployment.md`)
|
||||
|
||||
### Solidity Token (`packages/solidity/aitbc-token/`)
|
||||
|
||||
Current Status: Contracts reviewed, tests expanded, deployment documented.
|
||||
|
||||
- **Contract Audit** ✅ COMPLETE
|
||||
- [x] Review AIToken.sol and AITokenRegistry.sol
|
||||
- [x] Add comprehensive test coverage (17 tests passing)
|
||||
- [x] Test edge cases: zero address, zero units, non-coordinator, replay
|
||||
- [x] Run security analysis (Slither, Mythril) — `contracts/scripts/security-analysis.sh`
|
||||
- [ ] External audit - Future
|
||||
|
||||
- **Deployment Preparation** ✅ COMPLETE
|
||||
- [x] Deployment script exists (`scripts/deploy.ts`)
|
||||
- [x] Mint script exists (`scripts/mintWithReceipt.ts`)
|
||||
- [x] Deployment documentation (`docs/DEPLOYMENT.md`)
|
||||
- [ ] Deploy to testnet and verify - Future
|
||||
- [ ] Plan mainnet deployment timeline - Future
|
||||
|
||||
### ZK Receipt Verifier (`contracts/ZKReceiptVerifier.sol`)
|
||||
|
||||
Current Status: Contract updated to match circuit, documentation complete.
|
||||
|
||||
- **Integration with ZK Circuits** ✅ COMPLETE
|
||||
- [x] Verify compatibility with `receipt_simple` circuit (1 public signal)
|
||||
- [x] Fix contract to use `uint[1]` for publicSignals
|
||||
- [x] Fix authorization checks (`require(authorizedVerifiers[msg.sender])`)
|
||||
- [x] Add `verifyReceiptProof()` for view-only verification
|
||||
- [x] Update `verifyAndRecord()` with separate settlementAmount param
|
||||
|
||||
- **Documentation** ✅ COMPLETE
|
||||
- [x] On-chain verification flow (`contracts/docs/ZK-VERIFICATION.md`)
|
||||
- [x] Proof generation examples (JavaScript, Python)
|
||||
- [x] Coordinator API integration guide
|
||||
- [x] Deployment instructions
|
||||
|
||||
- **Deployment** ✅ COMPLETE
|
||||
- [x] Generate Groth16Verifier.sol from circuit (`contracts/Groth16Verifier.sol` stub + snarkjs generation instructions)
|
||||
- [x] Deploy to testnet with ZK circuits (`contracts/scripts/deploy-testnet.sh`)
|
||||
- [x] Integration test with Coordinator API (`tests/test_zk_integration.py` — 8 tests)
|
||||
|
||||
### Receipt Specification (`docs/reference/specs/receipt-spec.md`)
|
||||
|
||||
Current Status: Canonical receipt schema specification moved from `protocols/receipts/`.
|
||||
|
||||
- **Specification Finalization** ✅ COMPLETE
|
||||
- [x] Core schema defined (version 1.0)
|
||||
- [x] Signature format specified (Ed25519)
|
||||
- [x] Validation rules documented
|
||||
- [x] Add multi-signature receipt format (`signatures` array, threshold, quorum policy)
|
||||
- [x] Document ZK-proof metadata extension (`metadata.zk_proof` with Groth16/PLONK/STARK)
|
||||
- [x] Add Merkle proof anchoring spec (`metadata.merkle_anchor` with verification algorithm)
|
||||
|
||||
### Technical Debt Schedule
|
||||
|
||||
| Component | Priority | Target | Status |
|
||||
|-----------|----------|--------|--------|
|
||||
| `apps/blockchain-node/` SQLModel fixes | Medium | Q2 2026 | ✅ Complete (2026-01-24) |
|
||||
| `packages/solidity/aitbc-token/` audit | Low | Q3 2026 | ✅ Complete (2026-01-24) |
|
||||
| `packages/solidity/aitbc-token/` testnet | Low | Q3 2026 | 🔄 Pending deployment |
|
||||
| `contracts/ZKReceiptVerifier.sol` deploy | Low | Q3 2026 | ✅ Code ready (2026-01-24) |
|
||||
| `docs/reference/specs/receipt-spec.md` finalize | Low | Q2 2026 | ✅ Complete (2026-02-12) |
|
||||
| Cross-site synchronization | High | Q1 2026 | ✅ Complete (2026-01-29) |
|
||||
|
||||
## Recent Progress (2026-02-12)
|
||||
|
||||
### Persistent GPU Marketplace ✅
|
||||
- Replaced in-memory mock with SQLModel-backed tables (`GPURegistry`, `GPUBooking`, `GPUReview`)
|
||||
- Rewrote `routers/marketplace_gpu.py` — all 10 endpoints use DB sessions
|
||||
- **22/22 GPU marketplace tests** (`apps/coordinator-api/tests/test_gpu_marketplace.py`)
|
||||
|
||||
### CLI Integration Tests ✅
|
||||
- End-to-end tests: real coordinator app (in-memory SQLite) + CLI commands via `_ProxyClient` shim
|
||||
- Covers all command groups: client, miner, admin, marketplace GPU, explorer, payments, end-to-end lifecycle
|
||||
- **24/24 CLI integration tests** (`tests/cli/test_cli_integration.py`)
|
||||
- **208/208 total** when run with billing + GPU marketplace + CLI unit tests
|
||||
|
||||
### Coordinator Billing Stubs ✅
|
||||
- Usage tracking: `_apply_credit`, `_apply_charge`, `_adjust_quota`, `_reset_daily_quotas`, `_process_pending_events`, `_generate_monthly_invoices`
|
||||
- Tenant context: `_extract_from_token` (HS256 JWT)
|
||||
- **21/21 billing tests** (`apps/coordinator-api/tests/test_billing.py`)
|
||||
|
||||
### CLI Enhancement — All Phases Complete ✅
|
||||
- **141/141 CLI unit tests** (0 failures) across 9 test files
|
||||
- **12 command groups**: client, miner, wallet, auth, config, blockchain, marketplace, simulate, admin, monitor, governance, plugin
|
||||
- CI/CD: `.github/workflows/cli-tests.yml` (Python 3.10/3.11/3.12)
|
||||
|
||||
- **Phase 1–2**: Core enhancements + new CLI tools (client retry, miner earnings/capabilities/deregister, wallet staking/multi-wallet/backup, auth, blockchain, marketplace, admin, config, simulate)
|
||||
- **Phase 3**: 116→141 tests, CLI reference docs (560+ lines), shell completion, man page
|
||||
- **Phase 4**: MarketplaceOffer GPU fields, booking system, review system
|
||||
- **Phase 5**: Batch CSV/JSON ops, job templates, webhooks, plugin system, real-time dashboard, metrics/alerts, multi-sig wallets, encrypted config, audit logging, progress bars
|
||||
|
||||
## Recent Progress (2026-02-13)
|
||||
|
||||
### Critical Security Fixes ✅ COMPLETE
|
||||
- **Fixed Hardcoded Secrets**
|
||||
- JWT secret now required from environment (no longer hardcoded)
|
||||
- PostgreSQL credentials parsed from DATABASE_URL
|
||||
- Added fail-fast validation for missing secrets
|
||||
|
||||
- **Unified Database Sessions**
|
||||
- Migrated all routers to use `storage.SessionDep`
|
||||
- Removed legacy session dependencies
|
||||
- Consistent database session management across services
|
||||
|
||||
- **Closed Authentication Gaps**
|
||||
- Implemented session-based authentication in exchange API
|
||||
- Fixed hardcoded user IDs - now uses authenticated context
|
||||
- Added login/logout endpoints with wallet authentication
|
||||
|
||||
- **Tightened CORS Defaults**
|
||||
- Replaced wildcard origins with specific localhost URLs
|
||||
- Restricted HTTP methods to only those needed
|
||||
- Applied across all services (Coordinator, Exchange, Blockchain, Gossip)
|
||||
|
||||
- **Enhanced Wallet Encryption**
|
||||
- Replaced weak XOR with Fernet (AES-128 CBC)
|
||||
- Added secure key derivation (PBKDF2 with SHA-256)
|
||||
- Integrated keyring for password management
|
||||
|
||||
- **CI Import Error Fix**
|
||||
- Replaced `requests` with `httpx` (already a dependency)
|
||||
- Fixed build pipeline failures
|
||||
- Added graceful fallback for missing dependencies
|
||||
|
||||
### Deployment Status
|
||||
- ✅ Site A (aitbc.bubuit.net): All fixes deployed and active
|
||||
- ✅ Site B (ns3): No action needed (blockchain node only)
|
||||
- ✅ Commit: `26edd70` - Changes committed and deployed
|
||||
|
||||
## Recent Progress (2026-02-13) - Code Quality & Observability ✅ COMPLETE
|
||||
|
||||
### Structured Logging Implementation
|
||||
- ✅ Added JSON structured logging to Coordinator API
|
||||
- `StructuredLogFormatter` class for consistent log output
|
||||
- Added `AuditLogger` class for tracking sensitive operations
|
||||
- Configurable JSON/text format via settings
|
||||
- ✅ Added JSON structured logging to Blockchain Node
|
||||
- Consistent log format with Coordinator API
|
||||
- Added `service` field for log parsing
|
||||
- Added `get_audit_logger()` function
|
||||
|
||||
### Structured Error Responses
|
||||
- ✅ Implemented standardized error responses across all APIs
|
||||
- Added `ErrorResponse` and `ErrorDetail` Pydantic models
|
||||
- All exceptions now have `error_code`, `status_code`, and `to_response()` method
|
||||
- Added new exception types: `AuthorizationError`, `NotFoundError`, `ConflictError`
|
||||
- Added exception handlers in main.py for consistent error formatting
|
||||
|
||||
### OpenAPI Documentation
|
||||
- ✅ Enabled OpenAPI documentation with ReDoc
|
||||
- Added `docs_url="/docs"`, `redoc_url="/redoc"`, `openapi_url="/openapi.json"`
|
||||
- Added OpenAPI tags for all router groups (health, client, miner, admin, marketplace, exchange, governance, zk)
|
||||
- Structured endpoint organization for better API discoverability
|
||||
|
||||
### Health Check Endpoints
|
||||
- ✅ Added liveness and readiness probes
|
||||
- `/health/live` - Simple alive check
|
||||
- `/health/ready` - Database connectivity check
|
||||
- Used by orchestrators for service health monitoring
|
||||
|
||||
### Unified Configuration
|
||||
- ✅ Consolidated configuration with environment-based adapter selection
|
||||
- Added `DatabaseConfig` class with adapter selection (sqlite/postgresql)
|
||||
- Added connection pooling settings (`pool_size`, `max_overflow`, `pool_pre_ping`)
|
||||
- Added `validate_secrets()` method for production environments
|
||||
- Added `mempool_backend` configuration for persistence
|
||||
- Backward compatible `database_url` property
|
||||
|
||||
### Connection Pooling
|
||||
- ✅ Added database connection pooling
|
||||
- `QueuePool` for PostgreSQL with configurable pool settings
|
||||
- `pool_size=10`, `max_overflow=20`, `pool_pre_ping=True`
|
||||
- Improved session scope with proper commit/rollback handling
|
||||
- Better resource management under load
|
||||
|
||||
### Unified SessionDep
|
||||
- ✅ Completed migration to unified `storage.SessionDep`
|
||||
- All routers now use `SessionDep` dependency injection
|
||||
- Removed legacy session code paths
|
||||
- Consistent database session management across services
|
||||
|
||||
### DatabaseMempool Default
|
||||
- ✅ Changed mempool backend to use database persistence by default
|
||||
- `mempool_backend: str = "database"` (was "memory")
|
||||
- Transaction persistence across restarts
|
||||
- Better reliability for production deployments
|
||||
|
||||
### Systemd Service Standardization
|
||||
- ✅ Standardized all service paths to `/opt/<service-name>` convention
|
||||
- Updated 10 systemd service files:
|
||||
- aitbc-coordinator-api.service
|
||||
- aitbc-exchange-api.service
|
||||
- aitbc-exchange-frontend.service
|
||||
- aitbc-wallet.service
|
||||
- aitbc-node.service
|
||||
- aitbc-gpu-miner.service
|
||||
- aitbc-gpu-miner-root.service
|
||||
- aitbc-host-gpu-miner.service
|
||||
- aitbc-gpu-registry.service
|
||||
- aitbc-coordinator-proxy-health.service
|
||||
- Consistent deployment paths across all services
|
||||
|
||||
## Upcoming Improvements (2026-02-14+)
|
||||
|
||||
### High Priority - Security & Stability
|
||||
|
||||
- **Redis-backed Rate Limiting**
|
||||
- Replace in-memory rate limiter with Redis-backed implementation
|
||||
- Support for distributed rate limiting across multiple instances
|
||||
- Configurable limits per endpoint
|
||||
- Status: Pending implementation
|
||||
|
||||
- **Request Validation Middleware**
|
||||
- Add request size limits for all endpoints
|
||||
- Input sanitization for all user inputs
|
||||
- SQL injection and XSS prevention
|
||||
- Status: Pending implementation
|
||||
|
||||
- **Audit Logging**
|
||||
- Comprehensive audit logging for sensitive operations
|
||||
- Track: API key usage, admin actions, configuration changes
|
||||
- Integration with existing `AuditLogger` class
|
||||
- Status: Pending implementation
|
||||
|
||||
### Medium Priority - Performance & Quality
|
||||
|
||||
- **Redis-backed Mempool (Production)**
|
||||
- Add Redis adapter for mempool in production
|
||||
- Support for distributed mempool across nodes
|
||||
- Better persistence and recovery
|
||||
- Status: Pending implementation
|
||||
|
||||
- **Async I/O Conversion**
|
||||
- Convert blocking I/O operations to async where possible
|
||||
- Use `aiohttp` or `httpx` async clients for external API calls
|
||||
- Async database operations with SQLModel
|
||||
- Status: Pending implementation
|
||||
|
||||
- **Custom Business Metrics**
|
||||
- Add Prometheus metrics for business logic
|
||||
- Track: jobs created, miners registered, payments processed
|
||||
- Custom dashboards for operational visibility
|
||||
- Status: Pending implementation
|
||||
|
||||
### Low Priority - Polish & Documentation
|
||||
|
||||
- **API Documentation Enhancement**
|
||||
- Add detailed endpoint descriptions
|
||||
- Include request/response examples
|
||||
- Add code samples for common operations
|
||||
- Status: Pending implementation
|
||||
|
||||
- **Architecture Diagrams**
|
||||
- Create architecture diagrams for `docs/`
|
||||
- Include data flow diagrams
|
||||
- Service interaction diagrams
|
||||
- Deployment architecture diagrams
|
||||
- Status: Pending implementation
|
||||
|
||||
- **Operational Runbook**
|
||||
- Create operational runbook for production
|
||||
- Include: deployment procedures, troubleshooting guides
|
||||
- Escalation procedures and contact information
|
||||
- Status: Pending implementation
|
||||
|
||||
- **Chaos Engineering Tests**
|
||||
- Add tests for service failures
|
||||
- Test network partitions and recovery
|
||||
- Simulate database outages
|
||||
- Status: Pending implementation
|
||||
|
||||
### Git & Repository Hygiene ✅ COMPLETE
|
||||
- Renamed local `master` branch to `main` and set tracking to `github/main`
|
||||
- Deleted remote `master` branch from GitHub (was recreated on each push)
|
||||
- Removed stale `origin` remote (Gitea — repo not found)
|
||||
- Set `git config --global init.defaultBranch main`
|
||||
- Removed `.github/` directory (legacy RFC PR template, no active workflows)
|
||||
- Single remote: `github` → `https://github.com/oib/AITBC.git`, branch: `main`
|
||||
|
||||
## Recent Progress (2026-01-29)
|
||||
|
||||
### Testing Infrastructure
|
||||
- **Ollama GPU Provider Test Workflow** ✅ COMPLETE
|
||||
- End-to-end test from client submission to blockchain recording
|
||||
- Payment processing verified (0.05206 AITBC for inference job)
|
||||
- Created comprehensive test script and workflow documentation
|
||||
|
||||
### Code Quality
|
||||
- **Pytest Warning Fixes** ✅ COMPLETE
|
||||
- Fixed all pytest warnings (`PytestReturnNotNoneWarning`, `PydanticDeprecatedSince20`, `PytestUnknownMarkWarning`)
|
||||
- Migrated Pydantic validators to V2 style
|
||||
- Moved `pytest.ini` to project root with proper marker configuration
|
||||
|
||||
### Project Organization
|
||||
- **Directory Cleanup** ✅ COMPLETE
|
||||
- Reorganized root files into logical directories
|
||||
- Created `docs/guides/`, `docs/reports/`, `scripts/testing/`, `dev-utils/`
|
||||
- Updated documentation to reflect new structure
|
||||
- Fixed GPU miner systemd service path
|
||||
|
||||
the canonical checklist during implementation. Mark completed tasks with ✅ and add dates or links to relevant PRs as development progresses.
|
||||
|
||||
640
docs/1_project/5_done.md
Normal file
640
docs/1_project/5_done.md
Normal file
@@ -0,0 +1,640 @@
|
||||
# Completed Deployments
|
||||
|
||||
This document tracks components that have been successfully deployed and are operational.
|
||||
|
||||
## Container Services (aitbc.bubuit.net)
|
||||
|
||||
- ✅ **Main Website** - Deployed at https://aitbc.bubuit.net/
|
||||
- Static HTML/CSS with responsive design
|
||||
- Features overview, architecture, roadmap, platform status
|
||||
- Documentation portal integrated
|
||||
|
||||
- ✅ **Explorer Web** - Deployed at https://aitbc.bubuit.net/explorer/
|
||||
- Full-featured blockchain explorer
|
||||
- Mock data with genesis block (height 0) displayed
|
||||
- Blocks, transactions, addresses, receipts tracking
|
||||
- Mock/live data toggle functionality (live mode backed by Coordinator API)
|
||||
- Live API (nginx): `/api/explorer/*`
|
||||
|
||||
- ✅ **Marketplace Web** - Deployed at https://aitbc.bubuit.net/marketplace/
|
||||
- Vite + TypeScript frontend
|
||||
- Offer list, bid form, stats cards
|
||||
- Mock data fixtures with API abstraction
|
||||
- Integration tests now connect to live marketplace
|
||||
|
||||
- ✅ **Coordinator API** - Deployed in container
|
||||
- FastAPI service running on port 8000
|
||||
- Health endpoint: `/api/v1/health` returns `{"status":"ok","env":"dev"}`
|
||||
- nginx proxy: `/api/` routes to container service (so `/api/v1/*` works)
|
||||
- Explorer API (nginx): `/api/explorer/*` → backend `/v1/explorer/*`
|
||||
- Users API: `/api/v1/users/*` (compat: `/api/users/*` for Exchange)
|
||||
- ZK Applications API: /api/zk/ endpoints for privacy-preserving features
|
||||
- Integration tests use real ZK proof features
|
||||
|
||||
- ✅ **Wallet Daemon** - Deployed in container
|
||||
- FastAPI service with encrypted keystore (Argon2id + XChaCha20-Poly1305)
|
||||
- REST and JSON-RPC endpoints for wallet management
|
||||
- Mock ledger adapter with SQLite backend
|
||||
- Running on port 8002, nginx proxy: /wallet/
|
||||
- Dependencies: aitbc-sdk, aitbc-crypto, fastapi, uvicorn
|
||||
- Bitcoin payment gateway implemented
|
||||
|
||||
- ✅ **Documentation** - Deployed at https://aitbc.bubuit.net/docs/
|
||||
- Split documentation for different audiences
|
||||
- Miner, client, developer guides
|
||||
- API references and technical specs
|
||||
|
||||
- ✅ **Trade Exchange** - Deployed at https://aitbc.bubuit.net/Exchange/
|
||||
- Bitcoin wallet integration for AITBC purchases
|
||||
- User management system with individual wallets
|
||||
- QR code generation for payments
|
||||
- Real-time payment monitoring
|
||||
- Session-based authentication
|
||||
- Exchange rate: 1 BTC = 100,000 AITBC
|
||||
|
||||
## Integration Tests
|
||||
|
||||
- ✅ **Test Suite Updates** - Completed 2026-01-26
|
||||
- Security tests now use real ZK proof features
|
||||
- Marketplace tests connect to live service
|
||||
- Performance tests removed (too early)
|
||||
- Wallet-coordinator integration added to roadmap
|
||||
- 6 tests passing, 1 skipped (wallet integration)
|
||||
|
||||
- ✅ **ZK Applications** - Privacy-preserving features deployed
|
||||
- Circom compiler v2.2.3 installed
|
||||
- ZK circuits compiled (receipt_simple with 300 constraints)
|
||||
- Trusted setup ceremony completed (Powers of Tau)
|
||||
- Available features:
|
||||
- Identity commitments
|
||||
- Stealth addresses
|
||||
- Private receipt attestation
|
||||
- Group membership proofs
|
||||
- Private bidding
|
||||
- Computation proofs
|
||||
- API endpoints: /api/zk/
|
||||
|
||||
## Host Services (GPU Access)
|
||||
|
||||
- ✅ **Blockchain Node** - Running on host
|
||||
- SQLModel-based blockchain with PoA consensus
|
||||
- RPC API on ports 8081/8082 (proxied via /rpc/ and /rpc2/)
|
||||
- Mock coordinator on port 8090 (proxied via /v1/)
|
||||
- Devnet scripts and observability hooks
|
||||
- Cross-site RPC synchronization enabled
|
||||
- Transaction propagation between sites
|
||||
- ✅ **Host GPU Miner** - Running on host (RTX 4060 Ti)
|
||||
- Real GPU inference via Ollama
|
||||
- Connects to container coordinator through Incus proxy on `127.0.0.1:18000`
|
||||
- Receives jobs, submits results, and completes successfully
|
||||
|
||||
## Infrastructure
|
||||
|
||||
- ✅ **Incus Container** - 'aitbc' container deployed
|
||||
- RAID1 configuration for data redundancy
|
||||
- nginx reverse proxy for all web services
|
||||
- Bridge networking (10.1.223.1 gateway)
|
||||
|
||||
- ✅ **nginx Configuration** - All routes configured
|
||||
- /explorer/ → Explorer Web
|
||||
- /marketplace/ → Marketplace Web
|
||||
- /api/ → Coordinator API (container)
|
||||
- /api/v1/ → Coordinator API (container)
|
||||
- /api/explorer/ → Explorer API (container)
|
||||
- /api/users/ → Users API (container, Exchange compatibility)
|
||||
- /api/zk/ → ZK Applications API (container)
|
||||
- /rpc/ → Blockchain RPC (host)
|
||||
- /v1/ → Mock Coordinator (host)
|
||||
- /wallet/ → Wallet Daemon (container)
|
||||
- /docs/ → Documentation portal
|
||||
|
||||
- ✅ **SSL/HTTPS** - Configured and working
|
||||
- All services accessible via https://aitbc.bubuit.net/
|
||||
- Proper security headers implemented
|
||||
|
||||
- ✅ **DNS Resolution** - Fully operational
|
||||
- All endpoints accessible via domain name
|
||||
- SSL certificates properly configured
|
||||
|
||||
## Deployment Architecture
|
||||
|
||||
- **Container Services**: Public web access, no GPU required
|
||||
- Website, Explorer, Marketplace, Coordinator API, Wallet Daemon, Docs, ZK Apps
|
||||
- **Host Services**: GPU access required, private network
|
||||
- Blockchain Node, Mining operations
|
||||
- **nginx Proxy**: Routes requests between container and host
|
||||
- Seamless user experience across all services
|
||||
|
||||
## Current Status
|
||||
|
||||
**Production Ready**: All core services deployed and operational
|
||||
- ✅ 9 container services running (including ZK Applications and Trade Exchange)
|
||||
- ✅ 2 host services running (blockchain node + GPU miner)
|
||||
- ✅ Complete nginx proxy configuration
|
||||
- ✅ SSL/HTTPS fully configured
|
||||
- ✅ DNS resolution working
|
||||
- ✅ Trade Exchange with Bitcoin integration
|
||||
- ✅ Zero-Knowledge proof capabilities enabled
|
||||
- ✅ Explorer live API integration complete
|
||||
|
||||
## Remaining Tasks
|
||||
|
||||
- Fix full Coordinator API codebase import issues (low priority)
|
||||
- Fix Blockchain Node SQLModel/SQLAlchemy compatibility issues (low priority)
|
||||
- Configure additional monitoring and observability
|
||||
- Set up automated backup procedures
|
||||
|
||||
## Recent Updates (2026-02-11)
|
||||
|
||||
### Git & Repository Hygiene
|
||||
- ✅ **Branch Cleanup** - Purged all `master` branches from GitHub
|
||||
- Renamed local `master` branch to `main`
|
||||
- Set tracking to `github/main`
|
||||
- Deleted remote `master` branch from GitHub
|
||||
- Set `git config --global init.defaultBranch main` to prevent future `master` branches
|
||||
- ✅ **Remote Cleanup** - Removed stale `origin` remote (Gitea)
|
||||
- Only `github` remote remains (https://github.com/oib/AITBC.git)
|
||||
- ✅ **Legacy Cleanup** - Removed `.github/` directory
|
||||
- Contained only a legacy RFC pull request template (unused)
|
||||
- No active CI workflows or GitHub Actions
|
||||
|
||||
## Recent Updates (2026-01-29)
|
||||
|
||||
### Cross-Site Synchronization Implementation
|
||||
- ✅ **Multi-site Deployment**: Successfully deployed cross-site synchronization across 3 nodes
|
||||
- ✅ **Technical Implementation**:
|
||||
- Created `/src/aitbc_chain/cross_site.py` module
|
||||
- Integrated into node lifecycle in `main.py`
|
||||
- Added configuration in `config.py`
|
||||
- Added `/blocks/import` POST endpoint in `router.py`
|
||||
- ✅ **Network Configuration**:
|
||||
- Local nodes: https://aitbc.bubuit.net/rpc/, /rpc2/
|
||||
- Remote node: http://aitbc.keisanki.net/rpc/
|
||||
- ✅ **Current Status**:
|
||||
- Transaction sync working
|
||||
- ✅ Block import endpoint fully functional with transaction support
|
||||
- ✅ Transaction data properly saved to database during block import
|
||||
- Endpoint validates blocks and handles imports correctly
|
||||
- Node heights: Local (771153), Remote (40324)
|
||||
- Nginx routing fixed to port 8081 for blockchain-rpc-2
|
||||
|
||||
- ✅ **Technical Fixes Applied**
|
||||
- Fixed URL paths for correct RPC endpoint access
|
||||
- Integrated sync lifecycle into main node process
|
||||
- Resolved Python compatibility issues (removed AbstractAsyncContextManager)
|
||||
|
||||
- ✅ **Network Configuration**
|
||||
- Site A (localhost): https://aitbc.bubuit.net/rpc/ and /rpc2/
|
||||
- Site C (remote): http://aitbc.keisanki.net/rpc/
|
||||
- All nodes maintain independent chains (PoA design)
|
||||
- Cross-site sync enabled with 10-second polling interval
|
||||
|
||||
## Recent Updates (2026-01-28)
|
||||
|
||||
### Transaction-Dependent Block Creation
|
||||
- ✅ **PoA Proposer Enhancement** - Modified blockchain nodes to only create blocks when transactions are pending
|
||||
- Updated PoA proposer to check RPC mempool before creating blocks
|
||||
- Implemented HTTP polling mechanism to check mempool size every 2 seconds
|
||||
- Added transaction storage in blocks with proper tx_count field
|
||||
- Fixed syntax errors and import issues in poa.py
|
||||
- Node 1 now active and operational with new block creation logic
|
||||
- Eliminates empty blocks from the blockchain
|
||||
|
||||
- ✅ **Architecture Implementation**
|
||||
- RPC Service (port 8082): Receives and stores transactions in in-memory mempool
|
||||
- Node Process: Checks RPC metrics endpoint for mempool_size
|
||||
- If mempool_size > 0: Creates block with transactions
|
||||
- If mempool_size == 0: Skips block creation, logs "No pending transactions"
|
||||
- Removes processed transactions from mempool after block creation
|
||||
|
||||
## Recent Updates (2026-01-21)
|
||||
|
||||
### Service Maintenance and Fixes
|
||||
|
||||
- ✅ **Container Service Recovery** - Fixed all failing AITBC services
|
||||
- Resolved duplicate service conflicts (aitbc-coordinator-api, aitbc-exchange-frontend)
|
||||
- Fixed marketplace service by creating proper server.py file
|
||||
- Identified and disabled redundant services to prevent port conflicts
|
||||
- All essential services now running correctly
|
||||
|
||||
- ✅ **Service Status Summary**:
|
||||
- aitbc-blockchain.service - Running ✅
|
||||
- aitbc-exchange-api.service - Running ✅
|
||||
- aitbc-exchange.service - Running ✅
|
||||
- aitbc-marketplace.service - Running ✅ (Fixed)
|
||||
- aitbc-miner-dashboard.service - Running ✅
|
||||
- coordinator-api.service - Running ✅
|
||||
- wallet-daemon.service - Running ✅
|
||||
|
||||
- ✅ **SSH Access Configuration** - Set up passwordless SSH access
|
||||
- Created dedicated SSH key for Cascade automation
|
||||
- Configured SSH alias 'aitbc-cascade' for seamless access
|
||||
- Enabled secure service management and monitoring
|
||||
|
||||
### Skills Framework Implementation (2025-01-19)
|
||||
|
||||
- ✅ **Deploy-Production Skill** - Created comprehensive deployment workflow skill
|
||||
- Location: `.windsurf/skills/deploy-production/`
|
||||
- Features: Pre-deployment checks, environment templates, rollback procedures
|
||||
- Scripts: `pre-deploy-checks.sh`, `health-check.py`
|
||||
use cases: Automated production deployments with safety checks
|
||||
|
||||
- ✅ **Blockchain-Operations Skill** - Created blockchain operations management skill
|
||||
- Location: `.windsurf/skills/blockchain-operations/`
|
||||
- Features: Node health monitoring, transaction debugging, mining optimization
|
||||
- Scripts: `node-health.sh`, `tx-tracer.py`, `mining-optimize.sh`, `sync-monitor.py`, `network-diag.py`
|
||||
- Use cases: Node management, mining optimization, network diagnostics
|
||||
|
||||
### Skills Benefits
|
||||
|
||||
- Standardized workflows for complex operations
|
||||
- Automated safety checks and validation
|
||||
- Comprehensive documentation and error handling
|
||||
- Integration with Cascade for intelligent execution
|
||||
|
||||
## Recent Updates (2026-01-23)
|
||||
|
||||
- ✅ **Host GPU Miner (Real GPU)**
|
||||
- Host miner runs on RTX 4060 Ti with Ollama inference.
|
||||
- Uses Incus proxy on `127.0.0.1:18000` to reach the container coordinator.
|
||||
- Result submission fixed and jobs complete successfully.
|
||||
- ✅ **Coordinator Systemd Alignment**
|
||||
- `coordinator-api.service` enabled in container for startup on boot.
|
||||
- Legacy `aitbc-coordinator-api.service` removed to avoid conflicts.
|
||||
- ✅ **Proxy Health Check (Host)**
|
||||
- Added systemd timer `aitbc-coordinator-proxy-health.timer` to monitor proxy availability.
|
||||
|
||||
## Recent Updates (2026-01-24)
|
||||
|
||||
### Ollama GPU Inference End-to-End Testing
|
||||
- ✅ **Complete Workflow Verification**
|
||||
- Job submission via CLI → Coordinator API → Miner polling → Ollama inference → Result submission → Receipt generation → Blockchain recording
|
||||
- Successfully processed test job in 11.12 seconds with 218 tokens
|
||||
- Receipt generated with proper payment amounts: 11.846 gpu_seconds @ 0.02 AITBC = 0.23692 AITBC
|
||||
|
||||
- ✅ **Bash CLI Wrapper Script**
|
||||
- Created unified CLI tool at `/home/oib/windsurf/aitbc/scripts/aitbc-cli.sh`
|
||||
- Commands: submit, status, browser, blocks, receipts, cancel, admin-miners, admin-jobs, admin-stats, health
|
||||
- Environment variable overrides for URL and API keys
|
||||
- Made executable and documented in localhost testing scenario
|
||||
|
||||
- ✅ **Coordinator API Bug Fix**
|
||||
- Fixed `NameError: name '_coerce_float' is not defined` in receipt service
|
||||
- Added missing helper function to `/opt/coordinator-api/src/app/services/receipts.py`
|
||||
- Deployed fix to incus container via SSH
|
||||
- Result submission now returns 200 OK instead of 500 Internal Server Error
|
||||
|
||||
- ✅ **Miner Configuration Fix**
|
||||
- Updated miner ID from `host-gpu-miner` to `${MINER_API_KEY}` for proper job assignment
|
||||
- Added explicit flush logging handler for better systemd journal visibility
|
||||
- Enhanced systemd unit with unbuffered logging environment variables
|
||||
|
||||
- ✅ **Blockchain-Operations Skill Enhancement**
|
||||
- Updated skill with comprehensive Ollama testing scenarios
|
||||
- Created detailed test documentation in `ollama-test-scenario.md`
|
||||
- Added end-to-end test automation script template
|
||||
- Documented common issues, troubleshooting, and performance metrics
|
||||
|
||||
- ✅ **Documentation Updates**
|
||||
- Updated `docs/developer/testing/localhost-testing-scenario.md` with CLI wrapper usage
|
||||
- Converted all examples to use localhost URLs (127.0.0.1) instead of production
|
||||
- Added host user paths and quick start commands
|
||||
- Documented complete testing workflow from setup to verification
|
||||
|
||||
### Explorer Live Data Integration
|
||||
- ✅ **Explorer API Integration**
|
||||
- Switched explorer from mock data to live Coordinator API
|
||||
- Fixed receipt display: jobId, miner, payment amounts now shown correctly
|
||||
- Fixed address balances: calculated from actual job receipts
|
||||
- Updated all page text to indicate "Live data from AITBC coordinator API"
|
||||
|
||||
- ✅ **CLI Tool Enhancement**
|
||||
- Added `admin-cancel-running` command to cancel all hanging jobs at once
|
||||
- Useful for cleaning up stuck jobs from dev/test sessions
|
||||
|
||||
### Repository Reorganization
|
||||
- ✅ **Root Level Cleanup** - Moved 60+ loose files to proper directories
|
||||
- `scripts/deploy/` - 9 deployment scripts
|
||||
- `scripts/gpu/` - 13 GPU miner files
|
||||
- `scripts/test/` - 7 test/verify scripts
|
||||
- `scripts/service/` - 7 service management scripts
|
||||
- `systemd/` - 4 systemd service files
|
||||
- `infra/nginx/` - 5 nginx config files
|
||||
- `website/dashboards/` - 2 dashboard HTML files
|
||||
- `docs/` - 8 documentation MD files
|
||||
|
||||
- ✅ **Website/Docs Folder Structure**
|
||||
- Moved HTML documentation to `/website/docs/`
|
||||
- Created shared CSS: `/website/docs/css/docs.css` (1232 lines)
|
||||
- Created theme toggle JS: `/website/docs/js/theme.js`
|
||||
- Migrated all HTML files to use external CSS (reduced file sizes 45-66%)
|
||||
- Cleaned `/docs/` folder to only contain mkdocs markdown files
|
||||
|
||||
- ✅ **Dark Theme Fixes**
|
||||
- Fixed background color consistency across all docs pages
|
||||
- Added dark theme support to `full-documentation.html`
|
||||
- Fixed Quick Start section cascade styling in docs-miners.html
|
||||
- Fixed SDK Examples cascade indentation in docs-clients.html
|
||||
- Updated API endpoint example to use Python/FastAPI (matches actual codebase)
|
||||
|
||||
- ✅ **Path References Updated**
|
||||
- Updated systemd service file with new `scripts/gpu/gpu_miner_host.py` path
|
||||
- Updated skill documentation with new file locations
|
||||
- Updated localhost-testing-scenario.md with correct paths
|
||||
|
||||
- ✅ **Comprehensive .gitignore**
|
||||
- Expanded from 39 to 145 lines with organized sections
|
||||
- Added project-specific rules for coordinator, explorer, GPU miner
|
||||
|
||||
### Repository File Audit & Cleanup
|
||||
- ✅ **File Audit Document** (`docs/files.md`)
|
||||
- Created comprehensive audit of all 849 repository files
|
||||
- Categorized into Whitelist (60), Greylist (0), Placeholders (12), Removed (35)
|
||||
- All greylist items resolved - no pending reviews
|
||||
|
||||
- ✅ **Abandoned Folders Removed** (35 items total)
|
||||
- `ecosystem*/` (4 folders), `enterprise-connectors/`, `research/`
|
||||
- `apps/client-web/`, `apps/marketplace-ui/`, `apps/wallet-cli/`
|
||||
- `apps/miner-node/`, `apps/miner-dashboard/`
|
||||
- `packages/py/aitbc-core/`, `aitbc-p2p/`, `aitbc-scheduler/`
|
||||
- `packages/js/ui-widgets/`
|
||||
- `python-sdk/`, `windsurf/`, `configs/`, `docs/user-guide/`, `docs/bootstrap/`
|
||||
- `api/`, `governance/`, `protocols/`
|
||||
- 5 GPU miner variants, 3 extension variants
|
||||
|
||||
- ✅ **Docs Folder Reorganization**
|
||||
- Root now contains only: `done.md`, `files.md`, `roadmap.md`
|
||||
- Created new subfolders: `_config/`, `reference/components/`, `reference/governance/`
|
||||
- Created: `operator/deployment/`, `operator/migration/`
|
||||
- Created: `developer/testing/`, `developer/integration/`
|
||||
- Moved 25 files to appropriate subfolders
|
||||
- Moved receipt spec: `protocols/receipts/spec.md` → `docs/reference/specs/receipt-spec.md`
|
||||
|
||||
- ✅ **Roadmap Updates**
|
||||
- Added Stage 19: Placeholder Content Development
|
||||
- Added Stage 20: Technical Debt Remediation (blockchain-node, solidity-token, ZKReceiptVerifier)
|
||||
|
||||
### Stage 19: Placeholder Content Development (2026-01-24)
|
||||
|
||||
- ✅ **Phase 1: Documentation** (17 files created)
|
||||
- User Guides (`docs/user/guides/`): 8 files
|
||||
- `getting-started.md`, `job-submission.md`, `payments-receipts.md`, `troubleshooting.md`
|
||||
- Developer Tutorials (`docs/developer/tutorials/`): 5 files
|
||||
- `building-custom-miner.md`, `coordinator-api-integration.md`
|
||||
- `marketplace-extensions.md`, `zk-proofs.md`, `sdk-examples.md`
|
||||
- Reference Specs (`docs/reference/specs/`): 4 files
|
||||
- `api-reference.md` (OpenAPI 3.0), `protocol-messages.md`, `error-codes.md`
|
||||
|
||||
- ✅ **Phase 2: Infrastructure** (8 files created)
|
||||
- Terraform Environments (`infra/terraform/environments/`):
|
||||
- `staging/main.tf`, `prod/main.tf`, `variables.tf`, `secrets.tf`, `backend.tf`
|
||||
- Helm Chart Values (`infra/helm/values/`):
|
||||
- `dev/values.yaml`, `staging/values.yaml`, `prod/values.yaml`
|
||||
|
||||
- ✅ **Phase 3: Application Components** (13 files created)
|
||||
- Pool Hub Service (`apps/pool-hub/src/app/`):
|
||||
- `routers/`: miners.py, pools.py, jobs.py, health.py, __init__.py
|
||||
- `registry/`: miner_registry.py, __init__.py
|
||||
- `scoring/`: scoring_engine.py, __init__.py
|
||||
- Coordinator Migrations (`apps/coordinator-api/migrations/`):
|
||||
- `001_initial_schema.sql`, `002_indexes.sql`, `003_data_migration.py`, `README.md`
|
||||
|
||||
### Stage 20: Technical Debt Remediation (2026-01-24)
|
||||
|
||||
- ✅ **Blockchain Node SQLModel Fixes**
|
||||
- Fixed `models.py`: Added `__tablename__`, proper `Relationship` definitions
|
||||
- Fixed type hints: `List["Transaction"]` instead of `list["Transaction"]`
|
||||
- Added `sa_relationship_kwargs={"lazy": "selectin"}` for efficient loading
|
||||
- Updated tests: 2 passing, 1 skipped (SQLModel validator limitation documented)
|
||||
- Created `docs/SCHEMA.md` with ERD and usage examples
|
||||
|
||||
- ✅ **Solidity Token Audit**
|
||||
- Reviewed `AIToken.sol` and `AITokenRegistry.sol`
|
||||
- Added comprehensive tests: 17 tests passing
|
||||
- AIToken: 8 tests (minting, replay, zero address, zero units, non-coordinator)
|
||||
- AITokenRegistry: 9 tests (registration, updates, access control)
|
||||
- Created `docs/DEPLOYMENT.md` with full deployment guide
|
||||
|
||||
- ✅ **ZK Receipt Verifier Integration**
|
||||
- Fixed `ZKReceiptVerifier.sol` to match `receipt_simple` circuit
|
||||
- Updated `publicSignals` to `uint[1]` (1 public signal: receiptHash)
|
||||
- Fixed authorization checks: `require(authorizedVerifiers[msg.sender])`
|
||||
- Created `contracts/docs/ZK-VERIFICATION.md` with integration guide
|
||||
|
||||
### Recent Updates (2026-01-29)
|
||||
|
||||
- ✅ **Cross-Site Synchronization Issue Resolved**
|
||||
- Fixed database foreign key constraint in transaction/receipt tables
|
||||
- Updated import code to use block.id instead of block.height
|
||||
- Applied database migration to all nodes
|
||||
- Full details in: `docs/issues/2026-01-29_cross-site-sync-resolved.md`
|
||||
|
||||
- ✅ **Ollama GPU Provider Test Workflow**
|
||||
- Complete end-to-end test from client submission to blockchain recording
|
||||
- Created `/home/oib/windsurf/aitbc/home/test_ollama_blockchain.py`
|
||||
- Updated skill: `.windsurf/skills/ollama-gpu-provider/SKILL.md` (v2.0)
|
||||
- Created workflow: `.windsurf/workflows/ollama-gpu-test.md`
|
||||
- Verified payment flow: Client → Miner (0.05206 AITBC for inference)
|
||||
|
||||
- ✅ **Issue Management Workflow**
|
||||
- Created `.windsurf/workflows/issue-management.md`
|
||||
- Established process for tracking and archiving resolved issues
|
||||
- Moved resolved cross-site sync issue to `docs/issues/`
|
||||
|
||||
- ✅ **Pytest Warning Fixes**
|
||||
- Fixed `PytestReturnNotNoneWarning` in `test_blockchain_nodes.py`
|
||||
- Fixed `PydanticDeprecatedSince20` by migrating to V2 style validators
|
||||
- Fixed `PytestUnknownMarkWarning` by moving `pytest.ini` to project root
|
||||
|
||||
- ✅ **Directory Organization**
|
||||
- Created `docs/guides/` and moved 2 guide files from root
|
||||
- Created `docs/reports/` and moved 10 report files from root
|
||||
- Created `scripts/testing/` and moved 13 test scripts from root
|
||||
- Created `dev-utils/` and moved `aitbc-pythonpath.pth`
|
||||
- Updated `docs/files.md` with new structure
|
||||
- Fixed systemd service path for GPU miner
|
||||
|
||||
## Recent Updates (2026-02-12)
|
||||
|
||||
### Persistent GPU Marketplace ✅
|
||||
|
||||
- ✅ **SQLModel-backed GPU Marketplace** — replaced in-memory mock with persistent tables
|
||||
- `GPURegistry`, `GPUBooking`, `GPUReview` models in `apps/coordinator-api/src/app/domain/gpu_marketplace.py`
|
||||
- Registered in `domain/__init__.py` and `storage/db.py` (auto-created on `init_db()`)
|
||||
- Rewrote `routers/marketplace_gpu.py` — all 10 endpoints now use DB sessions
|
||||
- Fixed review count bug (auto-flush double-count in `add_gpu_review`)
|
||||
- 22/22 GPU marketplace tests (`apps/coordinator-api/tests/test_gpu_marketplace.py`)
|
||||
|
||||
### CLI Integration Tests ✅
|
||||
|
||||
- ✅ **End-to-end CLI → Coordinator tests** — 24 tests in `tests/cli/test_cli_integration.py`
|
||||
- `_ProxyClient` shim routes sync `httpx.Client` calls through Starlette TestClient
|
||||
- `APIKeyValidator` monkey-patch bypasses stale key sets from cross-suite `sys.modules` flushes
|
||||
- Covers: client (submit/status/cancel), miner (register/heartbeat/poll), admin (stats/jobs/miners), marketplace GPU (9 tests), explorer, payments, end-to-end lifecycle
|
||||
- 208/208 tests pass when run together with billing + GPU marketplace + CLI unit tests
|
||||
|
||||
### Coordinator Billing Stubs ✅
|
||||
|
||||
- ✅ **Usage tracking & tenant context** — 21 tests in `apps/coordinator-api/tests/test_billing.py`
|
||||
- `_apply_credit`, `_apply_charge`, `_adjust_quota`, `_reset_daily_quotas`
|
||||
- `_process_pending_events`, `_generate_monthly_invoices`
|
||||
- `_extract_from_token` (HS256 JWT verification)
|
||||
|
||||
### Blockchain Node — Stage 20/21/22 Enhancements ✅ (Milestone 3)
|
||||
|
||||
- ✅ **Shared Mempool Implementation**
|
||||
- `InMemoryMempool` rewritten with fee-based prioritization, size limits, eviction
|
||||
- `DatabaseMempool` — new SQLite-backed mempool for persistence and cross-service sharing
|
||||
- `init_mempool()` factory function configurable via `MEMPOOL_BACKEND` env var
|
||||
|
||||
- ✅ **Advanced Block Production**
|
||||
- Block size limits: `max_block_size_bytes` (1MB), `max_txs_per_block` (500)
|
||||
- Fee prioritization: highest-fee transactions drained first into blocks
|
||||
- Batch processing: proposer drains mempool and batch-inserts `Transaction` records
|
||||
- Metrics: `block_build_duration_seconds`, `last_block_tx_count`, `last_block_total_fees`
|
||||
|
||||
- ✅ **Production Hardening**
|
||||
- Circuit breaker pattern (`CircuitBreaker` class with threshold/timeout)
|
||||
- RPC error handling: 400 for fee rejection, 503 for mempool unavailable
|
||||
- PoA stability: retry logic in `_fetch_chain_head`, `poa_proposer_running` gauge
|
||||
- RPC hardening: `RateLimitMiddleware` (200 req/min), `RequestLoggingMiddleware`, CORS, `/health`
|
||||
- Operational runbook: `docs/guides/block-production-runbook.md`
|
||||
- Deployment guide: `docs/guides/blockchain-node-deployment.md`
|
||||
|
||||
- ✅ **Cross-Site Sync Enhancements (Stage 21)**
|
||||
- Conflict resolution: `ChainSync._resolve_fork` with longest-chain rule, max reorg depth
|
||||
- Proposer signature validation: `ProposerSignatureValidator` with trusted proposer set
|
||||
- Sync metrics: 15 metrics (received, accepted, rejected, forks, reorgs, duration)
|
||||
- RPC endpoints: `POST /importBlock`, `GET /syncStatus`
|
||||
|
||||
- ✅ **Smart Contract & ZK Deployment (Stage 20)**
|
||||
- `contracts/Groth16Verifier.sol` — functional stub with snarkjs regeneration instructions
|
||||
- `contracts/scripts/security-analysis.sh` — Slither + Mythril analysis
|
||||
- `contracts/scripts/deploy-testnet.sh` — testnet deployment workflow
|
||||
- ZK integration test: `tests/test_zk_integration.py` (8 tests)
|
||||
|
||||
- ✅ **Receipt Specification v1.1**
|
||||
- Multi-signature receipt format (`signatures` array, threshold, quorum policy)
|
||||
- ZK-proof metadata extension (`metadata.zk_proof` with Groth16/PLONK/STARK)
|
||||
- Merkle proof anchoring spec (`metadata.merkle_anchor` with verification algorithm)
|
||||
|
||||
- ✅ **Test Results**
|
||||
- 50/50 blockchain node tests (27 mempool + 23 sync)
|
||||
- 8/8 ZK integration tests
|
||||
- 141/141 CLI tests (unchanged)
|
||||
|
||||
### Governance & Incentive Programs ✅ (Milestone 2)
|
||||
- ✅ **Governance CLI** (`governance.py`) — propose, vote, list, result commands
|
||||
- Parameter change, feature toggle, funding, and general proposal types
|
||||
- Weighted voting with duplicate prevention and auto-close
|
||||
- 13 governance tests passing
|
||||
- ✅ **Liquidity Mining** — wallet liquidity-stake/unstake/rewards
|
||||
- APY tiers: bronze (3%), silver (5%), gold (8%), platinum (12%)
|
||||
- Lock period support with reward calculation
|
||||
- 7 new wallet tests (24 total wallet tests)
|
||||
- ✅ **Campaign Telemetry** — monitor campaigns/campaign-stats
|
||||
- TVL, participants, rewards distributed, progress tracking
|
||||
- Auto-seeded default campaigns
|
||||
- ✅ **134/134 tests passing** (0 failures) across 9 test files
|
||||
- Roadmap Stage 6 items checked off (governance + incentive programs)
|
||||
|
||||
### CLI Enhancement — All Phases Complete ✅ (Milestone 1)
|
||||
- ✅ **Enhanced CLI Tool** - 141/141 unit tests + 24 integration tests passing (0 failures)
|
||||
- Location: `/home/oib/windsurf/aitbc/cli/aitbc_cli/`
|
||||
- 12 command groups: client, miner, wallet, auth, config, blockchain, marketplace, simulate, admin, monitor, governance, plugin
|
||||
- CI/CD: `.github/workflows/cli-tests.yml` (Python 3.10/3.11/3.12 matrix)
|
||||
|
||||
- ✅ **Phase 1: Core Enhancements**
|
||||
- Client: retry with exponential backoff, job history/filtering, batch submit from CSV/JSON, job templates
|
||||
- Miner: earnings tracking, capability management, deregistration, job filtering, concurrent processing
|
||||
- Wallet: multi-wallet, backup/restore, staking (stake/unstake/staking-info), `--wallet-path` option
|
||||
- Auth: login/logout, token management, multi-environment, API key rotation
|
||||
|
||||
- ✅ **Phase 2: New CLI Tools**
|
||||
- blockchain.py, marketplace.py, admin.py, config.py, simulate.py
|
||||
|
||||
- ✅ **Phase 3: Testing & Documentation**
|
||||
- 141/141 CLI unit tests across 9 test files + 24 integration tests
|
||||
- CLI reference docs (`docs/cli-reference.md` — 560+ lines)
|
||||
- Shell completion script, man page (`cli/man/aitbc.1`)
|
||||
|
||||
- ✅ **Phase 4: Backend Integration**
|
||||
- MarketplaceOffer model extended with GPU-specific fields (gpu_model, gpu_memory_gb, gpu_count, cuda_version, price_per_hour, region)
|
||||
- GPU booking system, review system, sync-offers endpoint
|
||||
|
||||
- ✅ **Phase 5: Advanced Features**
|
||||
- Scripting: batch CSV/JSON ops, job templates, webhook notifications, plugin system
|
||||
- Monitoring: real-time dashboard, metrics collection/export, alert configuration, historical analysis
|
||||
- Security: multi-signature wallets (create/propose/sign), encrypted config (set-secret/get-secret), audit logging
|
||||
- UX: Rich progress bars, colored output, interactive prompts, auto-completion, man pages
|
||||
|
||||
- ✅ **Documentation Updates**
|
||||
- Updated `.windsurf/workflows/ollama-gpu-test.md` with CLI commands
|
||||
- Updated `.windsurf/workflows/test.md` with CLI testing guide
|
||||
- Updated `.windsurf/skills/blockchain-operations/` and `ollama-gpu-provider/`
|
||||
- System requirements updated to Debian Trixie (Linux)
|
||||
- All currentTask.md checkboxes complete (0 unchecked items)
|
||||
|
||||
## Recent Updates (2026-02-13)
|
||||
|
||||
### Critical Security Fixes ✅
|
||||
|
||||
- ✅ **Fixed Hardcoded Secrets** - Removed security vulnerabilities
|
||||
- JWT secret no longer hardcoded in `config_pg.py` - required from environment
|
||||
- PostgreSQL credentials removed from `db_pg.py` - parsed from DATABASE_URL
|
||||
- Added validation to fail-fast if secrets aren't provided
|
||||
- Made PostgreSQL adapter instantiation lazy to avoid import-time issues
|
||||
|
||||
- ✅ **Unified Database Sessions** - Consolidated session management
|
||||
- Migrated all routers from `deps.get_session` to `storage.SessionDep`
|
||||
- Removed legacy session code from `deps.py` and `database.py`
|
||||
- Updated `main.py` to use `storage.init_db`
|
||||
- All routers now use unified session dependency
|
||||
|
||||
- ✅ **Closed Authentication Gaps** - Secured exchange API
|
||||
- Added session token management with in-memory store
|
||||
- Implemented login/logout endpoints with wallet address authentication
|
||||
- Fixed hardcoded `user_id=1` - now uses authenticated user context
|
||||
- Added user-specific order endpoints (`/api/my/orders`)
|
||||
- Implemented optional authentication for public endpoints
|
||||
|
||||
- ✅ **Tightened CORS Defaults** - Restricted cross-origin access
|
||||
- Replaced wildcard origins with specific localhost URLs
|
||||
- Updated all services: Coordinator API, Exchange API, Blockchain Node, Gossip Relay
|
||||
- Restricted methods to only those needed (GET, POST, PUT, DELETE, OPTIONS)
|
||||
- Unauthorized origins now receive 400 Bad Request
|
||||
|
||||
- ✅ **Wallet Encryption Enhancement** - Private keys protected at rest
|
||||
- Replaced weak XOR encryption with Fernet (AES-128 in CBC mode)
|
||||
- Added password management with keyring support
|
||||
- Implemented secure key derivation (PBKDF2 with SHA-256)
|
||||
- All wallet private keys now encrypted by default
|
||||
|
||||
- ✅ **CI Import Error Fix** - Resolved build issues
|
||||
- Replaced `requests` with `httpx` in `bitcoin_wallet.py` and `blockchain.py`
|
||||
- Added graceful fallback for when httpx is not available
|
||||
- Fixed CI pipeline that was failing due to missing requests dependency
|
||||
|
||||
### Deployment Status
|
||||
- ✅ **Site A** (aitbc.bubuit.net): All security fixes deployed and active
|
||||
- ✅ **Site B** (ns3): No action needed - only blockchain node running
|
||||
- ✅ **Commit**: `26edd70` - All changes committed and deployed
|
||||
|
||||
### Legacy Service Cleanup (2026-02-13)
|
||||
- ✅ Removed legacy `aitbc-blockchain.service` running on port 9080
|
||||
- ✅ Confirmed only 2 blockchain nodes running (ports 8081 and 8082)
|
||||
- ✅ Both active nodes responding correctly to RPC requests
|
||||
|
||||
### Systemd Service Naming Standardization (2026-02-13)
|
||||
- ✅ Renamed all services to use `aitbc-` prefix for consistency
|
||||
- ✅ Site A updates:
|
||||
- `blockchain-node.service` → `aitbc-blockchain-node-1.service`
|
||||
- `blockchain-node-2.service` → `aitbc-blockchain-node-2.service`
|
||||
- `blockchain-rpc.service` → `aitbc-blockchain-rpc-1.service`
|
||||
- `blockchain-rpc-2.service` → `aitbc-blockchain-rpc-2.service`
|
||||
- `coordinator-api.service` → `aitbc-coordinator-api.service`
|
||||
- `exchange-mock-api.service` → `aitbc-exchange-mock-api.service`
|
||||
- ✅ Site B updates:
|
||||
- `blockchain-node.service` → `aitbc-blockchain-node-3.service`
|
||||
- `blockchain-rpc.service` → `aitbc-blockchain-rpc-3.service`
|
||||
- ✅ All services restarted and verified operational
|
||||
108
docs/1_project/6_cross-site-sync-resolved.md
Normal file
108
docs/1_project/6_cross-site-sync-resolved.md
Normal file
@@ -0,0 +1,108 @@
|
||||
# Current Issues
|
||||
|
||||
## Cross-Site Synchronization - ✅ RESOLVED
|
||||
|
||||
### Date
|
||||
2026-01-29
|
||||
|
||||
### Status
|
||||
**FULLY IMPLEMENTED** - Cross-site sync is running on all nodes. Transaction propagation works. Block import endpoint works with transactions after database foreign key fix.
|
||||
|
||||
### Description
|
||||
Cross-site synchronization has been integrated into all blockchain nodes. The sync module detects height differences between nodes and can propagate transactions via RPC.
|
||||
|
||||
### Components Affected
|
||||
- `/src/aitbc_chain/main.py` - Main blockchain node process
|
||||
- `/src/aitbc_chain/cross_site.py` - Cross-site sync module (implemented but not integrated)
|
||||
- All three blockchain nodes (localhost Node 1 & 2, remote Node 3)
|
||||
|
||||
### What Was Fixed
|
||||
1. **main.py integration**: Removed problematic `AbstractAsyncContextManager` type annotation and simplified the code structure
|
||||
2. **Cross-site sync module**: Integrated into all three nodes and now starts automatically
|
||||
3. **Config settings**: Added `cross_site_sync_enabled`, `cross_site_remote_endpoints`, `cross_site_poll_interval` inside the `ChainSettings` class
|
||||
4. **URL paths**: Fixed RPC endpoint paths (e.g., `/head` instead of `/rpc/head` for remote endpoints that already include `/rpc`)
|
||||
|
||||
### Current Status
|
||||
- **All nodes**: Running with cross-site sync enabled
|
||||
- **Transaction sync**: Working - mempool transactions can propagate between sites
|
||||
- **Block sync**: ✅ FULLY IMPLEMENTED - `/blocks/import` endpoint works with transactions
|
||||
- **Height difference**: Nodes maintain independent chains (local: 771153, remote: 40324)
|
||||
- **Status**: ✅ RESOLVED - Fixed database foreign key constraint issue (2026-01-29)
|
||||
|
||||
### Database Fix Applied (2026-01-29)
|
||||
- **Issue**: Transaction and receipt tables had foreign key to `block.height` instead of `block.id`
|
||||
- **Solution**:
|
||||
1. Updated database schema to reference `block.id`
|
||||
2. Fixed import code in `/src/aitbc_chain/rpc/router.py` to use `block.id`
|
||||
3. Applied migration to existing databases
|
||||
- **Result**: Block import with transactions now works correctly
|
||||
|
||||
### Resolved Issues
|
||||
Block synchronization transaction import issue has been **FIXED**:
|
||||
- `/blocks/import` POST endpoint is functional and deployed on all nodes
|
||||
- Endpoint validates block hashes, parent blocks, and prevents conflicts
|
||||
- ✅ Can import blocks with and without transactions
|
||||
- ✅ Transaction data properly saved to database
|
||||
- Root cause: nginx was routing to wrong port (8082 instead of 8081)
|
||||
- Fix: Updated nginx config to route to correct blockchain-rpc-2 service
|
||||
|
||||
### Block Sync Implementation Progress
|
||||
|
||||
1. **✅ Block Import Endpoint Created** - `/src/aitbc_chain/rpc/router.py`:
|
||||
- Added `@router.post("/blocks/import")` endpoint
|
||||
- Implemented block validation (hash, parent, existence checks)
|
||||
- Added transaction and receipt import logic
|
||||
- Returns status: "imported", "exists", or error details
|
||||
|
||||
2. **✅ Cross-Site Sync Updated** - `/src/aitbc_chain/sync/cross_site.py`:
|
||||
- Modified `import_block()` to call `/rpc/blocks/import`
|
||||
- Formats block data correctly for import
|
||||
- Handles import success/failure responses
|
||||
|
||||
3. **✅ Runtime Error Fixed**:
|
||||
- Moved inline imports (hashlib, datetime, config) to top of file
|
||||
- Added proper error logging and exception handling
|
||||
- Fixed indentation issues in the function
|
||||
- Endpoint now returns proper validation responses
|
||||
|
||||
4. **✅ Transaction Import Fixed**:
|
||||
- Root cause was nginx routing to wrong port (8082 instead of 8081)
|
||||
- Updated transaction creation to use constructor with all fields
|
||||
- Server rebooted to clear all caches
|
||||
- Nginx config fixed to route to blockchain-rpc-2 on port 8081
|
||||
- Verified transaction is saved correctly with all fields
|
||||
|
||||
5. **⏳ Future Enhancements**:
|
||||
- Add proposer signature validation
|
||||
- Implement fork resolution for conflicting chains
|
||||
- Add authorized node list configuration
|
||||
|
||||
### What Works Now
|
||||
- Cross-site sync loop runs every 10 seconds
|
||||
- Remote endpoint polling detects height differences
|
||||
- Transaction propagation between sites via mempool sync
|
||||
- ✅ Block import endpoint functional with validation
|
||||
- ✅ Blocks with and without transactions can be imported between sites via RPC
|
||||
- ✅ Transaction data properly saved to database
|
||||
- Logging shows sync activity in journalctl
|
||||
|
||||
### Files Modified
|
||||
- `/src/aitbc_chain/main.py` - Added cross-site sync integration
|
||||
- `/src/aitbc_chain/cross_site.py` - Fixed URL paths, updated to use /blocks/import endpoint
|
||||
- `/src/aitbc_chain/config.py` - Added sync settings inside ChainSettings class (all nodes)
|
||||
- `/src/aitbc_chain/rpc/router.py` - Added /blocks/import POST endpoint with validation
|
||||
|
||||
### Next Steps
|
||||
1. **Monitor Block Synchronization**:
|
||||
- Watch logs for successful block imports with transactions
|
||||
- Verify cross-site sync is actively syncing block heights
|
||||
- Monitor for any validation errors or conflicts
|
||||
|
||||
2. **Future Enhancements**:
|
||||
- Add proposer signature validation for security
|
||||
- Implement fork resolution for conflicting chains
|
||||
- Add sync metrics and monitoring dashboard
|
||||
|
||||
**Status**: ✅ COMPLETE - Block import with transactions working
|
||||
**Impact**: Full cross-site block synchronization now available
|
||||
**Resolution**: Server rebooted, nginx routing fixed to port 8081
|
||||
Reference in New Issue
Block a user