aitbc/docs/scenarios/VALIDATION_GUIDE.md

AITBC Scenario Validation Guide

Last Updated: 2026-05-09
Version: 1.0
Purpose: Automated validation procedures and checks for AITBC scenarios

---

Overview

This guide provides validation procedures for verifying successful completion of AITBC scenarios. Each scenario includes specific validation criteria to ensure operations were executed correctly.

Validation Framework

Validation Command

# Validate a specific scenario
aitbc validate scenario <scenario-id>

# Validate all scenarios in a stage
aitbc validate stage <stage-number>

# Validate with detailed output
aitbc validate scenario <scenario-id> --verbose

# Validate with auto-fix (where applicable)
aitbc validate scenario <scenario-id> --auto-fix

Validation Output Format

{
  "scenario_id": "01_wallet_basics",
  "status": "passed",
  "checks": [
    {
      "name": "wallet_created",
      "status": "passed",
      "message": "Wallet test-wallet-001 created successfully"
    },
    {
      "name": "wallet_balance",
      "status": "passed",
      "message": "Wallet balance: 1000 AIT"
    }
  ],
  "timestamp": "2026-05-09T12:00:00Z"
}

Scenario Validation Criteria

Scenario 01: Wallet Basics

Validation Checks:

1. Wallet creation successful
2. Wallet keystore file exists
3. Wallet can be unlocked
4. Wallet balance is retrievable
5. Wallet address is valid

Validation Commands:

# Check wallet exists
test -f /var/lib/aitbc/keystore/<wallet-name>.json

# Verify wallet can be unlocked
aitbc wallet unlock <wallet-name> --password <password>

# Check wallet balance
aitbc wallet balance <wallet-name> | grep -q "^[0-9]"

# Validate wallet address format
aitbc wallet address <wallet-name> | grep -q "^0x[a-fA-F0-9]\{40\}$"

Expected Results:

• Wallet file exists in keystore directory
• Balance is non-negative number
• Address is valid Ethereum-style address
• Unlock operation succeeds without error

Scenario 02: Transaction Sending

Validation Checks:

1. Transaction submitted successfully
2. Transaction hash is valid
3. Transaction confirms on blockchain
4. Sender balance decreases by amount + fee
5. Recipient balance increases by amount

Validation Commands:

# Submit test transaction
TX_HASH=$(aitbc transaction send <sender> <recipient> 10)

# Validate transaction hash format
echo $TX_HASH | grep -q "^0x[a-fA-F0-9]\{64\}$"

# Check transaction status
aitbc transaction status $TX_HASH | grep -q "confirmed"

# Verify balance changes
SENDER_BEFORE=$(aitbc wallet balance <sender>)
aitbc transaction send <sender> <recipient> 10
SENDER_AFTER=$(aitbc wallet balance <sender>)
# Verify: SENDER_BEFORE > SENDER_AFTER

Expected Results:

• Transaction hash is 64-character hex string
• Transaction status shows "confirmed"
• Balance changes reflect transaction amount and fees
• Transaction appears in blockchain explorer

Scenario 03: Genesis Deployment

Validation Checks:

1. Genesis wallet configured
2. Genesis block deployed
3. Network initialized
4. Genesis block hash is valid
5. Network is syncable

Validation Commands:

# Check genesis wallet
test -f /var/lib/aitbc/keystore/genesis.json

# Verify genesis block
GENESIS_HASH=$(aitbc genesis hash)
echo $GENESIS_HASH | grep -q "^0x[a-fA-F0-9]\{64\}$"

# Check network status
aitbc blockchain status | grep -q "initialized"

# Verify block number is 0 for new network
aitbc blockchain block number | grep -q "^0$"

Expected Results:

• Genesis wallet exists with funds
• Genesis block hash is valid
• Network shows as initialized
• Block number starts at 0
• Peers can connect to genesis node

Scenario 07: AI Job Submission

Validation Checks:

1. Job submitted successfully
2. Job ID is valid
3. Job parameters validated
4. Job enters queue
5. Job executes successfully

Validation Commands:

# Submit AI job
JOB_ID=$(aitbc ai job submit job-config.json)

# Validate job ID format
echo $JOB_ID | grep -q "^job-[a-fA-F0-9]\{32\}$"

# Check job status
aitbc ai job status $JOB_ID | grep -q "queued\|running\|completed"

# Monitor job completion
aitbc ai job wait $JOB_ID --timeout 300

# Verify results
aitbc ai job results $JOB_ID | test -s

Expected Results:

• Job ID follows expected format
• Job transitions through queue → running → completed
• Results are generated and retrievable
• No errors in job execution logs

Scenario 08: Marketplace Bidding

Validation Checks:

1. Bid submitted successfully
2. Bid ID is valid
3. Bid appears in marketplace
4. Bid status is active
5. Bid can be accepted if winning

Validation Commands:

# Submit bid
BID_ID=$(aitbc marketplace bid <listing-id> <amount> <wallet>)

# Validate bid ID format
echo $BID_ID | grep -q "^bid-[a-fA-F0-9]\{32\}$"

# Check bid status
aitbc marketplace bid status $BID_ID | grep -q "active"

# Verify bid in marketplace
aitbc marketplace bids <listing-id> | grep -q $BID_ID

# Check wallet balance after bid (if bid accepted)
aitbc wallet balance <wallet>

Expected Results:

• Bid ID follows expected format
• Bid appears in marketplace listing
• Bid status shows as active/pending
• Wallet balance reflects bid if accepted
• Marketplace updates bid status correctly

Scenario 13: Mining Setup

Validation Checks:

1. Mining service started
2. Mining node registered
3. Mining rewards received
4. Block production active
5. Mining statistics available

Validation Commands:

# Check mining service status
systemctl status aitbc-mining.service | grep -q "active (running)"

# Verify mining registration
aitbc mining status | grep -q "registered"

# Check mining rewards
aitbc mining rewards | grep -q "^[0-9]"

# Monitor block production
aitbc mining blocks | tail -1

# Check mining statistics
aitbc mining stats

Expected Results:

• Mining service is running
• Node is registered as miner
• Rewards are being earned
• Blocks are being produced
• Statistics show mining activity

Scenario 14: Staking Basics

Validation Checks:

1. Staking operation successful
2. Staked amount locked
3. Staking rewards accruing
4. Unstaking works correctly
5. Validator status updated

Validation Commands:

# Stake tokens
aitbc staking stake <wallet> 1000

# Verify staked amount
aitbc staking balance <wallet> | grep -q "1000"

# Check rewards
aitbc staking rewards <wallet> | grep -q "^[0-9]"

# Unstake tokens
aitbc staking unstake <wallet> 500

# Verify unstaking
aitbc staking balance <wallet> | grep -q "500"

Expected Results:

• Staked tokens are locked
• Rewards accrue over time
• Unstaking releases tokens after unbonding period
• Balance reflects staking operations
• Validator status updates correctly

Scenario 20: Cross Chain Transfer

Validation Checks:

1. Bridge initialized
2. Transfer submitted successfully
3. Transfer status trackable
4. Transfer completes on destination chain
5. Balances updated on both chains

Validation Commands:

# Check bridge status
aitbc cross-chain bridge status | grep -q "available"

# Submit cross-chain transfer
TRANSFER_ID=$(aitbc cross-chain transfer <source> <dest> 100)

# Track transfer status
aitbc cross-chain status $TRANSFER_ID | grep -q "pending\|completed"

# Verify destination chain balance
aitbc wallet balance <wallet> --chain <dest>

# Verify source chain balance decreased
aitbc wallet balance <wallet> --chain <source>

Expected Results:

• Bridge is operational
• Transfer ID is valid and trackable
• Transfer completes successfully
• Balances updated correctly on both chains
• No tokens lost in transfer

Automated Validation Scripts

Batch Validation Script

#!/bin/bash
# validate_scenarios.sh - Batch validate multiple scenarios

SCENARIOS=("01_wallet_basics" "02_transaction_sending" "03_genesis_deployment")
PASSED=0
FAILED=0

for scenario in "${SCENARIOS[@]}"; do
    echo "Validating $scenario..."
    if aitbc validate scenario $scenario; then
        ((PASSED++))
        echo "✓ $scenario passed"
    else
        ((FAILED++))
        echo "✗ $scenario failed"
    fi
done

echo ""
echo "Validation Summary:"
echo "Passed: $PASSED"
echo "Failed: $FAILED"
echo "Total: $((PASSED + FAILED))"

exit $FAILED

Stage Validation Script

#!/bin/bash
# validate_stage.sh - Validate all scenarios in a stage

STAGE=$1

if [ -z "$STAGE" ]; then
    echo "Usage: $0 <stage-number>"
    exit 1
fi

echo "Validating Stage $STAGE scenarios..."
aitbc validate stage $STAGE --verbose

if [ $? -eq 0 ]; then
    echo "✓ Stage $STAGE validation passed"
else
    echo "✗ Stage $STAGE validation failed"
    exit 1
fi

Continuous Validation

#!/bin/bash
# continuous_validation.sh - Run validation continuously

INTERVAL=300  # 5 minutes

while true; do
    echo "Running validation at $(date)"
    aitbc validate stage 1
    aitbc validate stage 2
    aitbc validate stage 3
    
    echo "Sleeping for $INTERVAL seconds..."
    sleep $INTERVAL
done

Validation Best Practices

Pre-Validation Checklist

• Ensure blockchain node is running and synced
• Verify wallet has sufficient funds
• Check network connectivity
• Confirm required services are active
• Review scenario prerequisites

Validation Frequency

• Development: Run validation after each scenario completion
• Testing: Run full validation before deployment
• Production: Run validation daily or on schedule
• CI/CD: Integrate validation into pipeline

Handling Validation Failures

1. Review validation output for specific failure
2. Check logs for error messages
3. Verify prerequisites are met
4. Re-run scenario if needed
5. Consult troubleshooting guide if persistent

Validation Reporting

# Generate validation report
aitbc validate stage 1 --report > validation-report-$(date +%Y%m%d).json

# View validation history
aitbc validate history

# Compare validation results
aitbc validate compare report-20260508.json report-20260509.json

Integration with CI/CD

GitHub Actions Example

name: Scenario Validation
on: [push, pull_request]

jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Setup AITBC
        run: |
          pip install -e .
          aitbc setup environment
      - name: Validate Stage 1
        run: aitbc validate stage 1
      - name: Validate Stage 2
        run: aitbc validate stage 2
      - name: Validate Stage 3
        run: aitbc validate stage 3

Gitea Actions Example

name: Scenario Validation
on: [push]

jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v2
      - name: Setup Python
        uses: actions/setup-python@v2
        with:
          python-version: '3.13'
      - name: Install AITBC
        run: |
          pip install -e .
          aitbc setup environment
      - name: Run Validation
        run: |
          for stage in {1..3}; do
            aitbc validate stage $stage
          done

Validation Metrics

Key Metrics to Track

• Scenario pass rate
• Average validation time
• Common failure patterns
• Validation trend over time
• Resource utilization during validation

Metrics Collection

# Collect validation metrics
aitbc validate metrics --period 7d

# Export metrics to CSV
aitbc validate metrics --format csv > metrics.csv

# Generate validation report
aitbc validate report --period 30d > report.html

---

Related Documentation:

• Troubleshooting Guide
• Agent Training README
• Interactive Learning Paths