aitbc/docs/test-infrastructure.md

Test Infrastructure Documentation

Overview

The AITBC project uses pytest-based testing with comprehensive coverage across all applications. The test infrastructure is organized by application complexity phases, with each app having unit, integration, and edge case tests.

Test Structure

Directory Organization

/opt/aitbc/
├── tests/
│   └── conftest.py                 # Global pytest configuration
└── apps/
    ├── <app-name>/
    │   └── tests/
    │       ├── __init__.py         # Test package marker
    │       ├── test_unit_<app>.py  # Unit tests (app-specific naming)
    │       ├── test_integration_<app>.py  # Integration tests (app-specific naming)
    │       └── test_edge_cases_<app>.py  # Edge case tests (app-specific naming)

Test Types

1. Unit Tests (test_unit_<app>.py)

   • Test Pydantic models and data validation
   • Test app initialization
   • Test individual functions in isolation
   • Mock external dependencies

2. Integration Tests (test_integration_<app>.py)

   • Test API endpoints using FastAPI TestClient
   • Test database operations
   • Test component interactions
   • Use fixtures for state management

3. Edge Case Tests (test_edge_cases_<app>.py)

   • Test unusual inputs and boundary conditions
   • Test error handling
   • Test empty/invalid data scenarios
   • Test negative values and special characters

Configuration

Global Configuration (tests/conftest.py)

The global conftest.py manages:

• Import paths: Adds app source directories to sys.path for test discovery
• Environment variables: Sets TEST_MODE=true, AUDIT_LOG_DIR, TEST_DATABASE_URL
• Mock dependencies: Mocks optional dependencies like slowapi

# Example import path configuration
sys.path.insert(0, str(project_root / "apps" / "app-name"))

Per-App Fixtures

Each app can define fixtures in its test files:

• Database reset: For apps with databases (SQLite, PostgreSQL)
• State cleanup: For apps with in-memory state
• Mock setup: For external service dependencies

Running Tests

Run All Tests

python3 -m pytest apps/ -v

Run Specific App Tests

python3 -m pytest apps/<app-name>/tests/ -v

Run Specific Test File

python3 -m pytest apps/<app-name>/tests/test_unit_<app>.py -v

Run Specific Test

python3 -m pytest apps/<app-name>/tests/test_unit_<app>.py::test_function_name -v

Test Patterns

Unit Test Pattern

@pytest.mark.unit
def test_model_validation():
    """Test Pydantic model with valid data"""
    model = Model(field1="value", field2=123)
    assert model.field1 == "value"
    assert model.field2 == 123

Integration Test Pattern

@pytest.mark.integration
def test_api_endpoint():
    """Test API endpoint with TestClient"""
    from app import app
    client = TestClient(app)
    response = client.get("/api/endpoint")
    assert response.status_code == 200
    data = response.json()
    assert data["field"] == "expected_value"

Edge Case Test Pattern

@pytest.mark.unit
def test_model_empty_field():
    """Test model with empty field"""
    model = Model(field1="", field2=123)
    assert model.field1 == ""

Mocking External Dependencies

HTTP Requests

from unittest.mock import patch, Mock

@pytest.mark.integration
@patch('app.httpx.get')
def test_external_api_call(mock_get):
    """Test with mocked HTTP request"""
    mock_get.return_value = Mock(status_code=200, json=lambda: {"data": "value"})
    result = function_that_calls_http()
    assert result is not None

Subprocess Calls

@patch('app.subprocess.run')
def test_subprocess_command(mock_run):
    """Test with mocked subprocess"""
    mock_run.return_value = Mock(stdout="output", returncode=0)
    result = function_that_calls_subprocess()
    assert result is not None

Time Delays

@patch('app.time.sleep')
def test_with_delay(mock_sleep):
    """Test without actual delay"""
    mock_sleep.return_value = None
    result = function_with_delay()
    assert result is not None

Database Handling

SQLite Apps

For apps using SQLite:

• Use in-memory databases for tests
• Delete database file before/after tests
• Use fixtures to reset state

@pytest.fixture(autouse=True)
def reset_db():
    """Reset database before each test"""
    db_path = Path("database.db")
    if db_path.exists():
        db_path.unlink()
    
    init_db()
    yield
    
    if db_path.exists():
        db_path.unlink()

PostgreSQL Apps

For apps using PostgreSQL:

• Set TEST_DATABASE_URL to use test database
• Use transactions and rollback
• Clean up test data

Coverage Summary

Phase 1: Simple Apps (7 apps, 201 tests)

• monitor, ai-engine, simple-explorer, zk-circuits
• exchange-integration, compliance-service, plugin-registry
• Test files renamed with app-specific suffixes (e.g., test_unit_monitor.py)

Phase 2: Medium Apps (7 apps, 260 tests)

• trading-engine, plugin-security, plugin-analytics
• global-infrastructure, plugin-marketplace
• multi-region-load-balancer, global-ai-agents
• Test files renamed with app-specific suffixes (e.g., test_unit_trading_engine.py)

Phase 3: Complex Apps (4 apps)

• miner (44 tests) - GPU miner with coordinator communication
• marketplace (49 tests) - Agent-first GPU marketplace
• agent-services (22 tests) - Agent registry and coordination
• blockchain-explorer (46 tests) - Blockchain exploration UI
• Test files renamed with app-specific suffixes (e.g., test_unit_miner.py)

Phase 4: Most Complex App (1 app, 27 tests)

• exchange - Full trading exchange with database
• Test files renamed with app-specific suffixes (e.g., test_unit_exchange.py)

Best Practices

1. Use descriptive test names: test_function_scenario_expected_result
2. Group related tests: Use pytest markers (@pytest.mark.unit, @pytest.mark.integration)
3. Mock external dependencies: Never call external services in tests
4. Clean up state: Use fixtures to reset state between tests
5. Test error cases: Test both success and failure scenarios
6. Keep tests isolated: Each test should be independent
7. Use type hints: Improve test readability and IDE support
8. Document edge cases: Explain why a particular edge case is being tested

Common Issues and Solutions

Import Errors

Problem: Module not found when running tests Solution: Add app path to sys.path in tests/conftest.py

sys.path.insert(0, str(project_root / "apps" / "app-name"))

Import File Conflicts

Problem: Pytest import conflicts when running all apps together due to identical test file names Solution: Test files renamed with app-specific suffixes (e.g., test_unit_marketplace.py) to avoid module naming collisions

Database Lock Issues

Problem: Tests fail due to database locks Solution: Use in-memory databases or delete database files in fixtures

Async Function Errors

Problem: Tests fail when calling async functions Solution: Use TestClient for FastAPI apps, or mark tests with @pytest.mark.asyncio

Stuck Tests

Problem: Test hangs indefinitely Solution: Mock time.sleep or reduce retry delays in tests

@patch('app.time.sleep')
def test_with_delay(mock_sleep):
    mock_sleep.return_value = None
    # test code

Pydantic v2 Compatibility

For apps using Pydantic v2:

• Replace .dict() with .model_dump()
• Use from_attributes = True in model Config
• Update validation patterns as needed

Continuous Integration

Tests are integrated into CI workflows:

• python-tests.yml - Generic Python test runner
• api-endpoint-tests.yml - API endpoint testing
• Tests run on every pull request
• Coverage reports are generated

Future Enhancements

• Add performance benchmarking tests
• Add load testing for API endpoints
• Add contract testing for external service integrations
• Increase code coverage targets
• Add property-based testing with Hypothesis