aitbc/docs/agent-training/IMPROVEMENT_SUGGESTIONS.md

Agent Training Improvement Suggestions

This document provides actionable suggestions for improving the AITBC agent training curriculum based on analysis of current gaps and missing operations.

Executive Summary

Current State: 9 training stages with partial operation coverage Identified Gaps: 50+ missing operations across stages Scenario Coverage: 46 scenarios mapped but not fully integrated Priority Actions: Add high-priority missing operations, integrate scenarios, enhance validation

---

Priority 1: Critical Missing Operations (Immediate Action)

Stage 1: Foundation

Missing Operations:

• genesis_init - Genesis block initialization
• genesis_verify - Genesis verification
• genesis_info - Genesis information retrieval
• wallet_list - List all wallets
• wallet_transactions - Query wallet transaction history

Impact: Foundation stage lacks complete wallet and genesis operations Suggestion: Add these operations to stage1_foundation.json with proper validation

Stage 2: Operations Mastery

Missing Operations:

• wallet_send - Send transactions (critical for testing)
• wallet_delete - Wallet cleanup
• wallet_rename - Wallet management
• wallet_backup - Backup creation
• wallet_sync - Wallet synchronization
• wallet_batch - Batch operations
• mining_stop - Stop mining operations
• mining_rewards - Query mining rewards
• network_status - Network status check
• network_ping - Network connectivity test
• network_propagate - Transaction propagation
• network_force_sync - Force blockchain sync

Impact: Core wallet and mining operations incomplete Suggestion: Prioritize wallet_send, wallet_list, mining_stop, network_status as these are fundamental

Stage 5: Expert Operations

Missing Operations:

• workflow_run - Execute workflows (critical)
• resource_optimize - Resource optimization
• resource_benchmark - Performance benchmarking
• resource_monitor - Resource monitoring
• analytics_blocks - Blockchain analytics
• analytics_report - Generate reports
• analytics_export - Export analytics data
• analytics_predict - Predictive analytics
• analytics_optimize - Analytics optimization
• performance_benchmark - Performance testing
• performance_optimize - Performance tuning
• performance_tune - Fine-tuning
• security_scan - Security scanning
• security_patch - Security patching
• compliance_check - Compliance verification
• compliance_report - Compliance reporting

Impact: Expert operations lack workflow execution and advanced analytics Suggestion: Add workflow_run immediately, then prioritize analytics and performance operations

Stage 6: Agent Identity & SDK

Missing Operations:

• agent_create - Create agent instance
• agent_list - List agents
• agent_status - Query agent status
• agent_capabilities - Query agent capabilities
• agent_message - Send message to agent
• agent_messages - Query agent messages

Impact: Agent operations incomplete despite SDK coverage Suggestion: Add core agent operations (agent_create, agent_list, agent_status) first

---

Priority 2: Scenario Integration (High Impact)

Immediate Scenario Additions

Stage 1: Add scenarios 1-6 from docs/scenarios

• 01_wallet_basics.md - Wallet creation, import, balance checks
• 02_transaction_sending.md - Send transactions, batch transfers
• 03_genesis_deployment.md - Create and deploy genesis blocks
• 04_messaging_basics.md - Send/receive messages via gossip
• 05_island_creation.md - Create and join islands
• 06_basic_trading.md - Simple AIT coin trading

Stage 2: Add scenarios 10-14

• 10_plugin_development.md - Create simple Ollama plugin
• 11_ipfs_storage.md - Store/retrieve data via IPFS
• 12_database_operations.md - Basic database hosting
• 13_mining_setup.md - Start mining operations
• 14_staking_basics.md - Stake tokens and earn rewards

Stage 3: Add scenario 22

• 22_ai_training_agent.md - AI job submission + GPU marketplace + payment

Stage 4: Add scenarios 8-9, 21, 25

• 08_marketplace_bidding.md - Place bids on marketplace
• 09_gpu_listing.md - List GPU resources on marketplace
• 21_compute_provider_agent.md - GPU listing + marketplace bidding + wallet management
• 25_marketplace_arbitrage.md - Trading + GPU marketplace + analytics

Stage 5: Add scenarios 15-19, 23, 28-31, 34-35

• 15_blockchain_monitoring.md - Monitor blockchain status
• 16_agent_registration.md - Register agent on network
• 17_governance_voting.md - Participate in governance
• 18_analytics_collection.md - Collect analytics data
• 19_security_setup.md - JWT authentication setup
• 23_data_oracle_agent.md - IPFS storage + messaging + transaction sending
• 28_monitoring_agent.md - Blockchain monitoring + analytics + alerting
• 29_plugin_marketplace_agent.md - Plugin development + marketplace + IPFS
• 30_database_service_agent.md - Database hosting + marketplace + security
• 31_federation_bridge_agent.md - Island operations + cross-chain bridge + messaging
• 34_compliance_agent.md - Governance + security + analytics
• 35_edge_compute_agent.md - GPU marketplace + island operations + database

Suggestion: Create scenario-specific training operations mapped to existing CLI commands

---

Priority 3: Structural Improvements

3.1 Add Stage Dependencies

Current Issue: No explicit dependency tracking between stages Suggestion: Add depends_on field to training schema:

{
  "stage": "stage2_operations_mastery",
  "depends_on": ["stage1_foundation"],
  "required_completion": true
}

3.2 Add Prerequisite Validation

Current Issue: Prerequisites listed but not automatically validated Suggestion: Add automated prerequisite checks:

{
  "prerequisites": {
    "auto_validate": true,
    "checks": [
      {"type": "service", "name": "blockchain-node", "state": "running"},
      {"type": "wallet", "name": "genesis", "balance": ">1000"},
      {"type": "network", "endpoint": "rpc", "reachable": true}
    ]
  }
}

3.3 Add Rollback Mechanisms

Current Issue: No way to undo failed stage operations Suggestion: Add rollback operations for each stage:

{
  "operations": [
    {
      "operation": "wallet_create",
      "rollback": "wallet_delete"
    }
  ]
}

3.4 Add State Persistence

Current Issue: Training state not persisted between stages Suggestion: Add state checkpointing:

{
  "checkpoint": {
    "save_after": ["wallet_create", "transaction_send"],
    "restore_on_failure": true,
    "checkpoint_dir": "/var/lib/aitbc/training-checkpoints"
  }
}

---

Priority 4: Validation Enhancements

4.1 Add Performance Metrics

Current Issue: Only success/failure validation Suggestion: Add performance thresholds:

{
  "success_criteria": {
    "status": "success",
    "response_fields": ["wallet_id"],
    "performance": {
      "max_duration_ms": 5000,
      "min_success_rate": 0.95,
      "max_memory_mb": 512
    }
  }
}

4.2 Add Idempotency Checks

Current Issue: Operations may fail on re-run Suggestion: Mark idempotent operations:

{
  "operation": "wallet_create",
  "idempotent": false,
  "check_exists": "wallet_list"
}

4.3 Add Transaction Validation

Current Issue: Transaction success not verified on-chain Suggestion: Add on-chain verification:

{
  "operation": "transaction_send",
  "verify_on_chain": true,
  "confirmations_required": 1,
  "timeout_seconds": 60
}

4.4 Add Error Recovery

Current Issue: No retry logic for transient failures Suggestion: Add retry configuration:

{
  "operation": "network_sync",
  "retry": {
    "max_attempts": 3,
    "backoff_seconds": 5,
    "retryable_errors": ["timeout", "network_error"]
  }
}

---

Priority 5: Documentation Improvements

5.1 Add Operation Examples

Current Issue: Operations lack concrete examples Suggestion: Add example CLI commands:

{
  "operation": "wallet_create",
  "examples": [
    {
      "command": "aitbc-cli wallet create my-wallet --password mypass",
      "description": "Create a new wallet with password"
    }
  ]
}

5.2 Add Error Scenarios

Current Issue: No documentation of expected errors Suggestion: Add error handling documentation:

{
  "operation": "wallet_create",
  "errors": [
    {
      "error": "wallet_exists",
      "message": "Wallet already exists",
      "resolution": "Use wallet_import or different name"
    }
  ]
}

5.3 Add Success Indicators

Current Issue: Unclear how to verify operation success Suggestion: Add verification commands:

{
  "operation": "wallet_create",
  "verify": {
    "command": "aitbc-cli wallet list",
    "expected_pattern": "my-wallet"
  }
}

5.4 Add Learning Resources

Current Issue: No links to deeper documentation Suggestion: Add resource links:

{
  "operation": "wallet_create",
  "resources": [
    {"type": "docs", "url": "/docs/wallet/README.md"},
    {"type": "api", "url": "/api/wallet#create"},
    {"type": "scenario", "url": "/docs/scenarios/01_wallet_basics.md"}
  ]
}

---

Priority 6: Testing Improvements

6.1 Add Integration Tests

Current Issue: Only unit exam tests Suggestion: Add integration test suite:

{
  "validation": {
    "exam_tests": [...],
    "integration_tests": [
      {
        "name": "end_to_end_wallet_flow",
        "operations": ["wallet_create", "wallet_fund", "wallet_send"],
        "cleanup": true
      }
    ]
  }
}

6.2 Add Mock Data

Current Issue: Tests require real blockchain Suggestion: Add mock data generators:

{
  "mock_data": {
    "wallets": ["test-wallet-1", "test-wallet-2"],
    "addresses": ["0x123...", "0x456..."],
    "transactions": ["tx_1", "tx_2"]
  }
}

6.3 Add Coverage Metrics

Current Issue: No test coverage tracking Suggestion: Add coverage reporting:

{
  "validation": {
    "coverage_target": 0.8,
    "coverage_report": true
  }
}

---

Priority 7: Curriculum Enhancements

7.1 Add Progressive Difficulty

Current Issue: Difficulty not clearly graduated Suggestion: Add difficulty ratings:

{
  "stage": "stage1_foundation",
  "difficulty": "beginner",
  "estimated_time_minutes": 30,
  "prerequisite_stages": []
}

7.2 Add Skill Tags

Current Issue: No skill categorization Suggestion: Add skill taxonomy:

{
  "stage": "stage3_ai_operations",
  "skills": ["ai", "job_submission", "task_management", "gpu_marketplace"],
  "skill_level": "intermediate"
}

7.3 Add Learning Objectives

Current Issue: No explicit learning goals Suggestion: Add objective statements:

{
  "stage": "stage1_foundation",
  "objectives": [
    "Create and manage wallets",
    "Send and receive transactions",
    "Query blockchain state",
    "Verify service status"
  ]
}

7.4 Add Certification Paths

Current Issue: No credential system Suggestion: Add certification tracks:

{
  "certifications": [
    {
      "name": "Foundation Certified",
      "required_stages": ["stage1_foundation"],
      "exam_score": 90
    },
    {
      "name": "Operations Master",
      "required_stages": ["stage1_foundation", "stage2_operations_mastery"],
      "exam_score": 85
    }
  ]
}

---

Implementation Roadmap

Phase 1 (Week 1-2): Critical Operations

1. Add missing Stage 1 operations (genesis, wallet_list, wallet_transactions)
2. Add missing Stage 2 operations (wallet_send, wallet_list, mining_stop, network_status)
3. Add missing Stage 5 operations (workflow_run)
4. Add missing Stage 6 operations (agent_create, agent_list, agent_status)

Phase 2 (Week 3-4): Scenario Integration

1. Integrate Stage 1 scenarios (1-6)
2. Integrate Stage 2 scenarios (10-14)
3. Integrate Stage 3 scenario (22)
4. Integrate Stage 4 scenarios (8-9, 21, 25)
5. Integrate Stage 5 scenarios (15-19, 23, 28-31, 34-35)

Phase 3 (Week 5-6): Structural Improvements

1. Add stage dependencies to schema
2. Implement prerequisite validation
3. Add rollback mechanisms
4. Add state persistence

Phase 4 (Week 7-8): Validation Enhancements

1. Add performance metrics
2. Add idempotency checks
3. Add transaction validation
4. Add error recovery

Phase 5 (Week 9-10): Documentation & Testing

1. Add operation examples
2. Add error scenarios
3. Add integration tests
4. Add coverage metrics

Phase 6 (Week 11-12): Curriculum Enhancements

1. Add difficulty ratings
2. Add skill tags
3. Add learning objectives
4. Add certification paths

---

Success Metrics

Operation Coverage: Increase from ~40% to 90% Scenario Integration: Increase from 0% to 80% of high-priority scenarios Validation Depth: Add performance, idempotency, and transaction validation Test Coverage: Achieve 80% test coverage across all stages Documentation Quality: 100% of operations have examples and error documentation

---

Next Steps

1. Review and prioritize this improvement list with the team
2. Create implementation tasks for Phase 1 (critical operations)
3. Update training schema to support new features (dependencies, rollback, state)
4. Begin adding missing operations starting with Stage 1
5. Integrate first batch of scenarios into training stages
6. Establish validation metrics to track improvement progress

---

Last Updated: 2026-05-07 Status: Ready for review and implementation Priority: High - Critical operations missing from foundation stages