aitbc/docs/deployment/comprehensive-guide.md

Comprehensive Deployment Guide

This guide provides detailed instructions for deploying the AITBC platform in various scenarios.

Table of Contents

• Prerequisites
• System Requirements
• Deployment Scenarios
• Local Development Setup
• Single-Server Production Deployment
• Multi-Server Deployment
• Cloud Deployment
• Docker Containerized Deployment
• Configuration
• SSL/TLS Configuration
• Health Checks
• Troubleshooting

Prerequisites

Software Requirements

• Operating System: Debian 12 (bookworm) or Ubuntu 22.04 LTS
• Python: 3.13 or higher
• Node.js: 24.14.0 or higher (for JavaScript SDK)
• CUDA Toolkit: 12.4 (for GPU support)
• Docker: 24.0 or higher (for containerized deployment)
• Docker Compose: 2.20 or higher

Hardware Requirements

Minimum (Development)

• CPU: 4 cores
• RAM: 8 GB
• Storage: 100 GB SSD
• GPU: Not required for development

Recommended (Production)

• CPU: 8+ cores
• RAM: 16+ GB
• Storage: 500 GB NVMe SSD
• GPU: NVIDIA RTX 3090 or better (for mining)

Multi-Node

• Each node: 8+ cores, 16+ GB RAM, 100+ GB SSD
• GPU nodes: NVIDIA RTX 3090 or better
• Network: 10 Gbps interconnect

Network Requirements

• Public IP address (for blockchain node)
• Open ports: 8080 (blockchain), 8011 (coordinator), 8071 (wallet), 8102 (marketplace)
• DNS configuration (optional but recommended)
• Firewall rules configured

System Requirements

Operating System

Supported:

• Debian 12 (bookworm)
• Ubuntu 22.04 LTS

Recommended:

• Debian 12 (bookworm) for production

Dependencies

# System dependencies
sudo apt update
sudo apt install -y \
    build-essential \
    python3-dev \
    python3-venv \
    python3-pip \
    git \
    curl \
    wget \
    gnupg \
    lsb-release \
    software-properties-common \
    apt-transport-https \
    ca-certificates \
    gnupg \
    lsb-release

# CUDA dependencies (for GPU support)
sudo apt install -y \
    nvidia-cuda-toolkit \
    nvidia-cudnn \
    libnvidia-common

Python Environment

# Create virtual environment
python3 -m venv /opt/aitbc/venv
source /opt/aitbc/venv/bin/activate

# Upgrade pip
pip install --upgrade pip

Deployment Scenarios

Scenario Comparison

Scenario	Complexity	Scalability	Cost	Use Case
Local Development	Low	None	Low	Development, testing
Single-Server	Medium	Low	Low	Small deployments, POC
Multi-Server	High	High	High	Production, HA
Cloud	Medium	High	Variable	Flexible scaling
Docker	Medium	High	Variable	Container orchestration

Local Development Setup

Quick Start

# Clone repository
git clone https://github.com/oib/AITBC.git /opt/aitbc
cd /opt/aitbc

# Create virtual environment
python3 -m venv venv
source venv/bin/activate

# Install dependencies
pip install -r requirements.txt

# Install local packages
pip install -e packages/py/aitbc-crypto
pip install -e packages/py/aitbc-sdk

# Start services
./scripts/setup.sh

Service Configuration

# Configure environment
cp .env.example .env
# Edit .env with your settings

# Start blockchain node
python -m apps.blockchain_node.main

# Start coordinator API
python -m apps.coordinator_api.main

# Start marketplace service
python -m apps.marketplace_service.main

Verification

# Check service health
curl http://localhost:8080/health  # Blockchain
curl http://localhost:8011/health  # Coordinator
curl http://localhost:8102/health  # Marketplace

Single-Server Production Deployment

Installation Steps

1. Prepare Server

# Update system
sudo apt update && sudo apt upgrade -y

# Create user
sudo useradd -m -s /bin/bash aitbc
sudo usermod -aG docker aitbc

2. Install Dependencies

# Install system dependencies
sudo apt install -y \
    build-essential \
    python3-dev \
    python3-venv \
    git \
    curl \
    nginx \
    postgresql \
    redis-server \
    docker.io \
    docker-compose

3. Deploy Application

# Clone repository
sudo -u aitbc git clone https://github.com/oib/AITBC.git /opt/aitbc
cd /opt/aitbc

# Setup virtual environment
sudo -u aitbc python3 -m venv /opt/aitbc/venv
sudo -u aitbc /opt/aitbc/venv/bin/pip install -r requirements.txt

# Setup database
sudo -u postgres psql -c "CREATE DATABASE aitbc;"
sudo -u postgres psql -c "CREATE USER aitbc WITH PASSWORD 'secure-password';"
sudo -u postgres psql -c "GRANT ALL PRIVILEGES ON DATABASE aitbc TO aitbc;"

4. Configure Systemd Services

# Setup services
sudo ./scripts/setup.sh

# Enable services
sudo systemctl enable aitbc-blockchain
sudo systemctl enable aitbc-coordinator-api
sudo systemctl enable aitbc-marketplace

# Start services
sudo systemctl start aitbc-blockchain
sudo systemctl start aitbc-coordinator-api
sudo systemctl start aitbc-marketplace

5. Configure Nginx

# /etc/nginx/sites-available/aitbc
upstream coordinator {
    server 127.0.0.1:8011;
}

upstream blockchain {
    server 127.0.0.1:8080;
}

upstream marketplace {
    server 127.0.0.1:8102;
}

server {
    listen 80;
    server_name your-domain.com;

    location /api/ {
        proxy_pass http://coordinator;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }

    location /blockchain/ {
        proxy_pass http://blockchain;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }

    location /marketplace/ {
        proxy_pass http://marketplace;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }
}

Multi-Server Deployment

Architecture

                    Load Balancer
                         |
        +----------------+----------------+
        |                |                |
   Blockchain Node   Coordinator API   Marketplace
        |                |                |
        +----------------+----------------+
                         |
                   PostgreSQL Cluster
                         |
                   Redis Cluster

Node Types

1. Blockchain Node

   • Runs blockchain consensus
   • Maintains ledger
   • Requires public IP

2. Coordinator API

   • Job submission and management
   • Payment processing
   • API gateway

3. Marketplace Service

   • GPU offer management
   • Matching engine
   • Price discovery

4. Database Node

   • PostgreSQL cluster
   • Redis cache
   • Data persistence

Setup Steps

1. Configure Network

# On each node, configure network
sudo apt install -y etcd
sudo systemctl enable etcd
sudo systemctl start etcd

2. Deploy Blockchain Node

# On blockchain node
sudo apt install -y nvidia-cuda-toolkit
git clone https://github.com/oib/AITBC.git /opt/aitbc
cd /opt/aitbc
./scripts/setup/blockchain.sh

3. Deploy Coordinator API

# On coordinator node
git clone https://github.com/oib/AITBC.git /opt/aitbc
cd /opt/aitbc
./scripts/setup/coordinator.sh

4. Deploy Marketplace Service

# On marketplace node
git clone https://github.com/oib/AITBC.git /opt/aitbc
cd /opt/aitbc
./scripts/setup/marketplace.sh

5. Configure Database Cluster

# On database node
sudo apt install -y postgresql redis-server
sudo -u postgres psql -c "CREATE DATABASE aitbc;"

Cloud Deployment

AWS Deployment

EC2 Setup

# Launch EC2 instances
- Blockchain: t3.xlarge or g4dn.xlarge (GPU)
- Coordinator: t3.large
- Marketplace: t3.large
- Database: RDS PostgreSQL

# Security groups
- Allow ports 8080, 8011, 8071, 8102
- Configure VPC and subnets

EKS Deployment

# kubernetes/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: coordinator-api
spec:
  replicas: 3
  selector:
    matchLabels:
      app: coordinator-api
  template:
    metadata:
      labels:
        app: coordinator-api
    spec:
      containers:
      - name: coordinator-api
        image: aitbc/coordinator-api:latest
        ports:
        - containerPort: 8011
        env:
        - name: DATABASE_URL
          valueFrom:
            secretKeyRef:
              name: database-secret
              key: url

GCP Deployment

GKE Setup

# Create GKE cluster
gcloud container clusters create aitbc-cluster \
    --num-nodes=3 \
    --machine-type=n1-standard-4 \
    --zone=us-central1-a

# Deploy services
kubectl apply -f kubernetes/

Docker Containerized Deployment

Docker Compose

# docker-compose.yml
version: '3.8'

services:
  blockchain:
    build: ./apps/blockchain_node
    ports:
      - "8080:8080"
    volumes:
      - blockchain-data:/data
    environment:
      - DATABASE_URL=postgresql://user:pass@postgres:5432/aitbc

  coordinator:
    build: ./apps/coordinator-api
    ports:
      - "8011:8011"
    depends_on:
      - blockchain
      - postgres
    environment:
      - DATABASE_URL=postgresql://user:pass@postgres:5432/aitbc

  marketplace:
    build: ./apps/marketplace_service
    ports:
      - "8102:8102"
    depends_on:
      - postgres
    environment:
      - DATABASE_URL=postgresql://user:pass@postgres:5432/aitbc

  postgres:
    image: postgres:15
    volumes:
      - postgres-data:/var/lib/postgresql/data
    environment:
      - POSTGRES_DB=aitbc
      - POSTGRES_USER=aitbc
      - POSTGRES_PASSWORD=secure-password

  redis:
    image: redis:7
    ports:
      - "6379:6379"

volumes:
  blockchain-data:
  postgres-data:

Build and Run

# Build images
docker-compose build

# Start services
docker-compose up -d

# Check status
docker-compose ps

# View logs
docker-compose logs -f

Configuration

Environment Variables

# /etc/aitbc/blockchain.env
BLOCKCHAIN_NETWORK_ID=1
BLOCKCHAIN_GENESIS_BLOCK_HASH=0x...
BLOCKCHAIN_CONSENSUS_ALGORITHM=proof_of_stake
BLOCKCHAIN_VALIDATOR_PRIVATE_KEY=0x...

# /etc/aitbc/coordinator.env
COORDINATOR_API_KEY=your-api-key
COORDINATOR_DATABASE_URL=postgresql://user:pass@localhost:5432/aitbc
COORDINATOR_REDIS_URL=redis://localhost:6379
COORDINATOR_JWT_SECRET=your-jwt-secret

# /etc/aitbc/marketplace.env
MARKETPLACE_DATABASE_URL=postgresql://user:pass@localhost:5432/aitbc
MARKETPLACE_REDIS_URL=redis://localhost:6379
MARKETPLACE_API_KEY=your-api-key

Configuration Files

# /etc/aitbc/config.yaml
services:
  blockchain:
    port: 8080
    host: 0.0.0.0
    database:
      host: localhost
      port: 5432
      name: aitbc
  
  coordinator:
    port: 8011
    host: 0.0.0.0
    database:
      host: localhost
      port: 5432
      name: aitbc
    cache:
      host: localhost
      port: 6379
  
  marketplace:
    port: 8102
    host: 0.0.0.0
    database:
      host: localhost
      port: 5432
      name: aitbc

SSL/TLS Configuration

Let's Encrypt

# Install certbot
sudo apt install -y certbot python3-certbot-nginx

# Obtain certificate
sudo certbot --nginx -d your-domain.com

# Auto-renewal
sudo certbot renew --dry-run

Manual Certificate

# Generate self-signed certificate
sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
  -keyout /etc/ssl/private/aitbc.key \
  -out /etc/ssl/certs/aitbc.crt

# Configure Nginx
sudo nano /etc/nginx/sites-available/aitbc

Nginx SSL Configuration

server {
    listen 443 ssl http2;
    server_name your-domain.com;

    ssl_certificate /etc/ssl/certs/aitbc.crt;
    ssl_certificate_key /etc/ssl/private/aitbc.key;
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers HIGH:!aNULL:!MD5;

    location / {
        proxy_pass http://localhost:8011;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-Proto https;
    }
}

server {
    listen 80;
    server_name your-domain.com;
    return 301 https://$server_name$request_uri;
}

Health Checks

Service Health Endpoints

# Blockchain health
curl http://localhost:8080/health

# Coordinator health
curl http://localhost:8011/health

# Marketplace health
curl http://localhost:8102/health

Monitoring Script

#!/bin/bash
# health-check.sh

services=("blockchain:8080" "coordinator:8011" "marketplace:8102")

for service in "${services[@]}"; do
    name="${service%%:*}"
    port="${service##*:}"
    
    if curl -f "http://localhost:$port/health" > /dev/null 2>&1; then
        echo "✓ $name is healthy"
    else
        echo "✗ $name is unhealthy"
        # Send alert
    fi
done

Systemd Health Monitoring

# /etc/systemd/system/aitbc-health-check.service
[Unit]
Description=AITBC Health Check
After=network.target

[Service]
Type=oneshot
ExecStart=/opt/aitbc/scripts/health-check.sh

[Install]
WantedBy=multi-user.target

Troubleshooting

Common Issues

Service Won't Start

# Check logs
sudo journalctl -u aitbc-coordinator-api -n 50

# Check port conflicts
sudo netstat -tulpn | grep -E '8080|8011|8102'

# Check permissions
sudo -u aitbc ls -la /opt/aitbc

Database Connection Failed

# Check PostgreSQL status
sudo systemctl status postgresql

# Check connection
psql -h localhost -U aitbc -d aitbc

# Check firewall
sudo ufw status

GPU Not Detected

# Check GPU
nvidia-smi

# Check CUDA
nvcc --version

# Check driver
sudo dmesg | grep -i nvidia

Performance Issues

High CPU Usage

# Check process CPU
top -p $(pgrep -f coordinator-api)

# Profile with cProfile
python -m cProfile -o profile.stats apps/coordinator_api/main.py

High Memory Usage

# Check memory
free -h

# Check process memory
ps aux | grep coordinator-api

# Check for memory leaks
valgrind --leak-check=full python apps/coordinator_api/main.py

Network Issues

Connection Refused

# Check service status
sudo systemctl status aitbc-coordinator-api

# Check firewall
sudo iptables -L -n

# Check network
ping localhost
telnet localhost 8011

Slow Performance

# Check network latency
ping -c 10 localhost

# Check bandwidth
iperf3 -s
iperf3 -c localhost

# Check DNS
nslookup your-domain.com

Maintenance

Backup

# Database backup
sudo -u postgres pg_dump aitbc > backup-$(date +%Y%m%d).sql

# Blockchain data backup
tar -czf blockchain-backup-$(date +%Y%m%d).tar.gz /var/lib/aitbc/blockchain

# Configuration backup
tar -czf config-backup-$(date +%Y%m%d).tar.gz /etc/aitbc

Updates

# Update application
cd /opt/aitbc
git pull origin main
source venv/bin/activate
pip install -r requirements.txt

# Restart services
sudo systemctl restart aitbc-coordinator-api
sudo systemctl restart aitbc-blockchain
sudo systemctl restart aitbc-marketplace

Monitoring

# Check service logs
sudo journalctl -u aitbc-coordinator-api -f

# Check system metrics
htop

# Check network
iftop