aitbc/docs/testing/USAGE_GUIDE.md

Test Configuration Refactoring - Usage Guide

🚀 Quick Start

The AITBC test suite has been refactored to eliminate the shell script smell and use proper pytest configuration in pyproject.toml. We standardize on Python 3.13 for all testing and use SQLite exclusively for database testing.

Basic Usage

# Run all fast tests (default)
pytest

# Run with the convenient test runner
python tests/test_runner.py

# Run all tests including slow ones
python tests/test_runner.py --all

# Run with coverage
python tests/test_runner.py --coverage

Test Categories

# Unit tests only
pytest -m "unit"
python tests/test_runner.py --unit

# Integration tests only
pytest -m "integration"
python tests/test_runner.py --integration

# CLI tests only
pytest -m "cli"
python tests/test_runner.py --cli

# API tests only
pytest -m "api"
python tests/test_runner.py --api

# Blockchain tests only
pytest -m "blockchain"
python tests/test_runner.py --blockchain

# Slow tests only
pytest -m "slow"
python tests/test_runner.py --slow

# Performance tests only
pytest -m "performance"
python tests/test_runner.py --performance

# Security tests only
pytest -m "security"
python tests/test_runner.py --security

Advanced Usage

# Run specific test files
pytest tests/cli/test_agent_commands.py
pytest apps/coordinator-api/tests/test_api.py

# Run with verbose output
pytest -v
python tests/test_runner.py --verbose

# Run with coverage
pytest --cov=aitbc_cli --cov-report=term-missing
python tests/test_runner.py --coverage

# List available tests
pytest --collect-only
python tests/test_runner.py --list

# Show available markers
pytest --markers
python tests/test_runner.py --markers

# Run with specific Python path
pytest --pythonpath=cli

# Run with custom options
pytest -v --tb=short --disable-warnings

📋 Test Markers

The test suite uses the following markers to categorize tests:

Marker	Description	Usage
unit	Unit tests (fast, isolated)	pytest -m unit
integration	Integration tests (may require external services)	pytest -m integration
cli	CLI command tests	pytest -m cli
api	API endpoint tests	pytest -m api
blockchain	Blockchain-related tests	pytest -m blockchain
crypto	Cryptography tests	pytest -m crypto
contracts	Smart contract tests	pytest -m contracts
slow	Slow running tests	pytest -m slow
performance	Performance tests	pytest -m performance
security	Security tests	pytest -m security
gpu	Tests requiring GPU resources	pytest -m gpu
e2e	End-to-end tests	pytest -m e2e

🗂️ Test Discovery

The test suite automatically discovers tests in these directories:

• tests/ - Main test directory
• apps/*/tests/ - Application tests
• cli/tests/ - CLI tests
• contracts/test/ - Smart contract tests
• packages/*/tests/ - Package tests
• scripts/test/ - Script tests

🔧 Configuration

All test configuration is now in pyproject.toml with SQLite as the default database:

[tool.pytest.ini_options]
python_files = ["test_*.py", "*_test.py"]
testpaths = ["tests", "apps/*/tests", "cli/tests", ...]
addopts = ["--verbose", "--tb=short", "--strict-markers", "--disable-warnings", "-ra"]
env = [
    "DATABASE_URL=sqlite:///./test_coordinator.db",
    "SQLITE_DATABASE=sqlite:///./test_coordinator.db"
]
markers = [
    "unit: Unit tests (fast, isolated)",
    "integration: Integration tests (may require external services)",
    # ... more markers
]

🚦 CI/CD Integration

The CI workflows now call pytest directly:

- name: Run tests
  run: pytest --cov=aitbc_cli --cov-report=xml

📊 Coverage

# Run with coverage
pytest --cov=aitbc_cli --cov-report=term-missing

# Generate HTML coverage report
pytest --cov=aitbc_cli --cov-report=html

# Coverage for specific module
pytest --cov=aitbc_cli.commands.agent --cov-report=term-missing

🐛 Troubleshooting

Common Issues

Import errors:

# Check python path
python -c "import sys; print(sys.path)"

# Run with explicit python path
PYTHONPATH=cli pytest

Test discovery issues:

# Check what tests are discovered
pytest --collect-only

# Check configuration
python -c "import pytest; print(pytest.config.getini('testpaths'))"

Coverage issues:

# Check coverage configuration
pytest --cov=aitbc_cli --cov-report=term-missing --debug

# Verify coverage source
python -c "import coverage; print(coverage.Coverage().source)"

Migration from Shell Script

Before:

./tests/run_all_tests.sh

After:

pytest
# or
python tests/test_runner.py

🎯 Best Practices

For Developers

1. Use appropriate markers: Mark your tests with the correct category
2. Keep unit tests fast: Unit tests should not depend on external services
3. Use fixtures: Leverage pytest fixtures for setup/teardown
4. Write descriptive tests: Use clear test names and descriptions

Test Writing Example

import pytest

@pytest.mark.unit
def test_cli_command_help():
    """Test CLI help command."""
    # Test implementation
    
@pytest.mark.integration
@pytest.mark.slow
def test_blockchain_sync():
    """Test blockchain synchronization."""
    # Test implementation
    
@pytest.mark.cli
def test_agent_create_command():
    """Test agent creation CLI command."""
    # Test implementation

Running Tests During Development

# Quick feedback during development
pytest -m "unit" -v

# Run tests for specific module
pytest tests/cli/test_agent_commands.py -v

# Run tests with coverage for your changes
pytest --cov=aitbc_cli --cov-report=term-missing

# Run tests before committing
python tests/test_runner.py --coverage

📈 Performance Tips

Fast Test Execution

# Run only unit tests for quick feedback
pytest -m "unit" -v

# Use parallel execution (if pytest-xdist is installed)
pytest -n auto -m "unit"

# Skip slow tests during development
pytest -m "not slow"

Memory Usage

# Run tests with minimal output
pytest -q

# Use specific test paths to reduce discovery overhead
pytest tests/cli/

🔍 Debugging

Debug Mode

# Run with debug output
pytest --debug

# Run with pdb on failure
pytest --pdb

# Run with verbose output
pytest -v -s

Test Selection

# Run specific test
pytest tests/cli/test_agent_commands.py::test_agent_create

# Run tests matching pattern
pytest -k "agent_create"

# Run failed tests only
pytest --lf

📚 Additional Resources

• pytest documentation: https://docs.pytest.org/
• pytest-cov documentation: https://pytest-cov.readthedocs.io/
• pytest-mock documentation: https://pytest-mock.readthedocs.io/
• AITBC Development Guidelines: See docs/DEVELOPMENT_GUIDELINES.md

---

Migration completed: ✅ All test configuration moved to pyproject.toml
Shell script eliminated: ✅ No more run_all_tests.sh dependency
CI/CD updated: ✅ Direct pytest integration in workflows
Developer experience improved: ✅ Simplified test execution