diff --git a/COMPLETE_TEST_PLAN.md b/COMPLETE_TEST_PLAN.md deleted file mode 100644 index d4ed399c..00000000 --- a/COMPLETE_TEST_PLAN.md +++ /dev/null @@ -1,262 +0,0 @@ -# AITBC Complete Test Plan - Genesis to Full Operations -# Using OpenClaw Skills and Workflow Scripts - -## ๐ŸŽฏ Test Plan Overview -Sequential testing from genesis block generation through full AI operations using OpenClaw agents and skills. - -## ๐Ÿ“‹ Prerequisites Check -```bash -# Verify OpenClaw is running -openclaw status - -# Verify all AITBC services are running -systemctl list-units --type=service --state=running | grep aitbc - -# Check wallet access -ls -la /var/lib/aitbc/keystore/ -``` - -## ๐Ÿš€ Phase 1: Genesis Block Generation (OpenClaw) - -### Step 1.1: Pre-flight Setup -**Skill**: `openclaw-agent-testing-skill` -**Script**: `01_preflight_setup_openclaw.sh` - -```bash -# Create OpenClaw session -SESSION_ID="genesis-test-$(date +%s)" - -# Test OpenClaw agents first -openclaw agent --agent main --message "Execute openclaw-agent-testing-skill with operation: comprehensive, thinking_level: medium" --thinking medium - -# Run pre-flight setup -/opt/aitbc/scripts/workflow-openclaw/01_preflight_setup_openclaw.sh -``` - -### Step 1.2: Genesis Authority Setup -**Skill**: `aitbc-basic-operations-skill` -**Script**: `02_genesis_authority_setup_openclaw.sh` - -```bash -# Setup genesis node using OpenClaw -openclaw agent --agent main --message "Execute aitbc-basic-operations-skill to setup genesis authority, create genesis block, and initialize blockchain services" --thinking medium - -# Run genesis setup script -/opt/aitbc/scripts/workflow-openclaw/02_genesis_authority_setup_openclaw.sh -``` - -### Step 1.3: Verify Genesis Block -**Skill**: `aitbc-transaction-processor` - -```bash -# Verify genesis block creation -openclaw agent --agent main --message "Execute aitbc-transaction-processor to verify genesis block, check block height 0, and validate chain state" --thinking medium - -# Manual verification -curl -s http://localhost:8006/rpc/head | jq '.height' -``` - -## ๐Ÿ”— Phase 2: Follower Node Setup - -### Step 2.1: Follower Node Configuration -**Skill**: `aitbc-basic-operations-skill` -**Script**: `03_follower_node_setup_openclaw.sh` - -```bash -# Setup follower node (aitbc1) -openclaw agent --agent main --message "Execute aitbc-basic-operations-skill to setup follower node, connect to genesis, and establish sync" --thinking medium - -# Run follower setup (from aitbc, targets aitbc1) -/opt/aitbc/scripts/workflow-openclaw/03_follower_node_setup_openclaw.sh -``` - -### Step 2.2: Verify Cross-Node Sync -**Skill**: `openclaw-agent-communicator` - -```bash -# Test cross-node communication -openclaw agent --agent main --message "Execute openclaw-agent-communicator to verify aitbc1 sync with genesis node" --thinking medium - -# Check sync status -ssh aitbc1 'curl -s http://localhost:8006/rpc/head | jq ".height"' -``` - -## ๐Ÿ’ฐ Phase 3: Wallet Operations - -### Step 3.1: Cross-Node Wallet Creation -**Skill**: `aitbc-wallet-manager` -**Script**: `04_wallet_operations_openclaw.sh` - -```bash -# Create wallets on both nodes -openclaw agent --agent main --message "Execute aitbc-wallet-manager to create cross-node wallets and establish wallet infrastructure" --thinking medium - -# Run wallet operations -/opt/aitbc/scripts/workflow-openclaw/04_wallet_operations_openclaw.sh -``` - -### Step 3.2: Fund Wallets & Initial Transactions -**Skill**: `aitbc-transaction-processor` - -```bash -# Fund wallets from genesis -openclaw agent --agent main --message "Execute aitbc-transaction-processor to fund wallets and execute initial cross-node transactions" --thinking medium - -# Verify transactions -curl -s http://localhost:8006/rpc/balance/ -``` - -## ๐Ÿค– Phase 4: AI Operations Setup - -### Step 4.1: Coordinator API Testing -**Skill**: `aitbc-ai-operator` - -```bash -# Test AI coordinator functionality -openclaw agent --agent main --message "Execute aitbc-ai-operator to test coordinator API, job submission, and AI service integration" --thinking medium - -# Test API endpoints -curl -s http://localhost:8000/health -curl -s http://localhost:8000/docs -``` - -### Step 4.2: GPU Marketplace Setup -**Skill**: `aitbc-marketplace-participant` - -```bash -# Initialize GPU marketplace -openclaw agent --agent main --message "Execute aitbc-marketplace-participant to setup GPU marketplace, register providers, and prepare for AI jobs" --thinking medium - -# Verify marketplace status -curl -s http://localhost:8000/api/marketplace/stats -``` - -## ๐Ÿงช Phase 5: Complete AI Workflow Testing - -### Step 5.1: Ollama GPU Testing -**Skill**: `ollama-gpu-testing-skill` -**Script**: Reference `ollama-gpu-test-openclaw.md` - -```bash -# Execute complete Ollama GPU test -openclaw agent --agent main --message "Execute ollama-gpu-testing-skill with complete end-to-end test: client submission โ†’ GPU processing โ†’ blockchain recording" --thinking high - -# Monitor job progress -curl -s http://localhost:8000/api/jobs -``` - -### Step 5.2: Advanced AI Operations -**Skill**: `aitbc-ai-operations-skill` -**Script**: `06_advanced_ai_workflow_openclaw.sh` - -```bash -# Run advanced AI workflow -openclaw agent --agent main --message "Execute aitbc-ai-operations-skill with advanced AI job processing, multi-modal RL, and agent coordination" --thinking high - -# Execute advanced workflow script -/opt/aitbc/scripts/workflow-openclaw/06_advanced_ai_workflow_openclaw.sh -``` - -## ๐Ÿ”„ Phase 6: Agent Coordination Testing - -### Step 6.1: Multi-Agent Coordination -**Skill**: `openclaw-agent-communicator` -**Script**: `07_enhanced_agent_coordination.sh` - -```bash -# Test agent coordination -openclaw agent --agent main --message "Execute openclaw-agent-communicator to establish multi-agent coordination and cross-node agent messaging" --thinking high - -# Run coordination script -/opt/aitbc/scripts/workflow-openclaw/07_enhanced_agent_coordination.sh -``` - -### Step 6.2: AI Economics Testing -**Skill**: `aitbc-marketplace-participant` -**Script**: `08_ai_economics_masters.sh` - -```bash -# Test AI economics and marketplace dynamics -openclaw agent --agent main --message "Execute aitbc-marketplace-participant to test AI economics, pricing models, and marketplace dynamics" --thinking high - -# Run economics test -/opt/aitbc/scripts/workflow-openclaw/08_ai_economics_masters.sh -``` - -## ๐Ÿ“Š Phase 7: Complete Integration Test - -### Step 7.1: End-to-End Workflow -**Script**: `05_complete_workflow_openclaw.sh` - -```bash -# Execute complete workflow -openclaw agent --agent main --message "Execute complete end-to-end AITBC workflow: genesis โ†’ nodes โ†’ wallets โ†’ AI operations โ†’ marketplace โ†’ economics" --thinking high - -# Run complete workflow -/opt/aitbc/scripts/workflow-openclaw/05_complete_workflow_openclaw.sh -``` - -### Step 7.2: Performance & Stress Testing -**Skill**: `openclaw-agent-testing-skill` - -```bash -# Stress test the system -openclaw agent --agent main --message "Execute openclaw-agent-testing-skill with operation: comprehensive, test_duration: 300, concurrent_agents: 3" --thinking high -``` - -## โœ… Verification Checklist - -### After Each Phase: -- [ ] Services running: `systemctl status aitbc-*` -- [ ] Blockchain syncing: Check block heights -- [ ] API responding: Health endpoints -- [ ] Wallets funded: Balance checks -- [ ] Agent communication: OpenClaw logs - -### Final Verification: -- [ ] Genesis block height > 0 -- [ ] Follower node synced -- [ ] Cross-node transactions successful -- [ ] AI jobs processing -- [ ] Marketplace active -- [ ] All agents communicating - -## ๐Ÿšจ Troubleshooting - -### Common Issues: -1. **OpenClaw not responding**: Check gateway status -2. **Services not starting**: Check logs with `journalctl -u aitbc-*` -3. **Sync issues**: Verify network connectivity between nodes -4. **Wallet problems**: Check keystore permissions -5. **AI jobs failing**: Verify GPU availability and Ollama status - -### Recovery Commands: -```bash -# Reset OpenClaw session -SESSION_ID="recovery-$(date +%s)" - -# Restart all services -systemctl restart aitbc-* - -# Reset blockchain (if needed) -rm -rf /var/lib/aitbc/data/ait-mainnet/* -# Then re-run Phase 1 -``` - -## ๐Ÿ“ˆ Success Metrics - -### Expected Results: -- Genesis block created and validated -- 2+ nodes syncing properly -- Cross-node transactions working -- AI jobs submitting and completing -- Marketplace active with providers -- Agent coordination established -- End-to-end workflow successful - -### Performance Targets: -- Block production: Every 10 seconds -- Transaction confirmation: < 30 seconds -- AI job completion: < 2 minutes -- Agent response time: < 5 seconds -- Cross-node sync: < 1 minute diff --git a/aitbc-blockchain-rpc-code-map.md b/aitbc-blockchain-rpc-code-map.md deleted file mode 100644 index 53b3a272..00000000 --- a/aitbc-blockchain-rpc-code-map.md +++ /dev/null @@ -1,142 +0,0 @@ -# AITBC Blockchain RPC Service Code Map - -## Service Configuration -**File**: `/etc/systemd/system/aitbc-blockchain-rpc.service` -**Entry Point**: `python3 -m uvicorn aitbc_chain.app:app --host ${rpc_bind_host} --port ${rpc_bind_port}` -**Working Directory**: `/opt/aitbc/apps/blockchain-node` -**Environment File**: `/etc/aitbc/blockchain.env` - -## Application Structure - -### 1. Main Entry Point: `app.py` -**Location**: `/opt/aitbc/apps/blockchain-node/src/aitbc_chain/app.py` - -#### Key Components: -- **FastAPI App**: `create_app()` function -- **Lifespan Manager**: `async def lifespan(app: FastAPI)` -- **Middleware**: RateLimitMiddleware, RequestLoggingMiddleware -- **Routers**: rpc_router, websocket_router, metrics_router - -#### Startup Sequence (lifespan function): -1. `init_db()` - Initialize database -2. `init_mempool()` - Initialize mempool -3. `create_backend()` - Create gossip backend -4. `await gossip_broker.set_backend(backend)` - Set up gossip broker -5. **PoA Proposer** (if enabled): - - Check `settings.enable_block_production and settings.proposer_id` - - Create `PoAProposer` instance - - Call `asyncio.create_task(proposer.start())` - -### 2. RPC Router: `rpc/router.py` -**Location**: `/opt/aitbc/apps/blockchain-node/src/aitbc_chain/rpc/router.py` - -#### Key Endpoints: -- `GET /rpc/head` - Returns current chain head (404 when no blocks exist) -- `GET /rpc/mempool` - Returns pending transactions (200 OK) -- `GET /rpc/blocks/{height}` - Returns block by height -- `POST /rpc/transaction` - Submit transaction -- `GET /rpc/blocks-range` - Get blocks in height range - -### 3. Gossip System: `gossip/broker.py` -**Location**: `/opt/aitbc/apps/blockchain-node/src/aitbc_chain/gossip/broker.py` - -#### Backend Types: -- `InMemoryGossipBackend` - Local memory backend (currently used) -- `BroadcastGossipBackend` - Network broadcast backend - -#### Key Functions: -- `create_backend(backend_type, broadcast_url)` - Creates backend instance -- `gossip_broker.set_backend(backend)` - Sets active backend - -### 4. Chain Sync System: `chain_sync.py` -**Location**: `/opt/aitbc/apps/blockchain-node/src/aitbc_chain/chain_sync.py` - -#### ChainSyncService Class: -- **Purpose**: Synchronizes blocks between nodes -- **Key Methods**: - - `async def start()` - Starts sync service - - `async def _broadcast_blocks()` - **MONITORING SOURCE** - - `async def _receive_blocks()` - Receives blocks from Redis - -#### Monitoring Code (_broadcast_blocks method): -```python -async def _broadcast_blocks(self): - """Broadcast local blocks to other nodes""" - import aiohttp - - last_broadcast_height = 0 - retry_count = 0 - max_retries = 5 - base_delay = 2 - - while not self._stop_event.is_set(): - try: - # Get current head from local RPC - async with aiohttp.ClientSession() as session: - async with session.get(f"http://{self.source_host}:{self.source_port}/rpc/head") as resp: - if resp.status == 200: - head_data = await resp.json() - current_height = head_data.get('height', 0) - - # Reset retry count on successful connection - retry_count = 0 -``` - -### 5. PoA Consensus: `consensus/poa.py` -**Location**: `/opt/aitbc/apps/blockchain-node/src/aitbc_chain/consensus/poa.py` - -#### PoAProposer Class: -- **Purpose**: Proposes blocks in Proof-of-Authority system -- **Key Methods**: - - `async def start()` - Starts proposer loop - - `async def _run_loop()` - Main proposer loop - - `def _fetch_chain_head()` - Fetches chain head from database - -### 6. Configuration: `blockchain.env` -**Location**: `/etc/aitbc/blockchain.env` - -#### Key Settings: -- `rpc_bind_host=0.0.0.0` -- `rpc_bind_port=8006` -- `gossip_backend=memory` (currently set to memory backend) -- `enable_block_production=false` (currently disabled) -- `proposer_id=` (currently empty) - -## Monitoring Source Analysis - -### Current Configuration: -- **PoA Proposer**: DISABLED (`enable_block_production=false`) -- **Gossip Backend**: MEMORY (no network sync) -- **ChainSyncService**: NOT EXPLICITLY STARTED - -### Mystery Monitoring: -Despite all monitoring sources being disabled, the service still makes requests to: -- `GET /rpc/head` (404 Not Found) -- `GET /rpc/mempool` (200 OK) - -### Possible Hidden Sources: -1. **Built-in Health Check**: The service might have an internal health check mechanism -2. **Background Task**: There might be a hidden background task making these requests -3. **External Process**: Another process might be making these requests -4. **Gossip Backend**: Even the memory backend might have monitoring - -### Network Behavior: -- **Source IP**: `10.1.223.1` (LXC gateway) -- **Destination**: `localhost:8006` (blockchain RPC) -- **Pattern**: Every 10 seconds -- **Requests**: `/rpc/head` + `/rpc/mempool` - -## Conclusion - -The monitoring is coming from **within the blockchain RPC service itself**, but the exact source remains unclear after examining all obvious candidates. The most likely explanations are: - -1. **Hidden Health Check**: A built-in health check mechanism not visible in the main code paths -2. **Memory Backend Monitoring**: Even the memory backend might have monitoring capabilities -3. **Internal Process**: A subprocess or thread within the main process making these requests - -### Recommendations: -1. **Accept the monitoring** - It appears to be harmless internal health checking -2. **Add authentication** to require API keys for RPC endpoints -3. **Modify source code** to remove the hidden monitoring if needed - -**The monitoring is confirmed to be internal to the blockchain RPC service, not external surveillance.**