aitbc/.windsurf/workflows/code-quality.md

description

description
Comprehensive code quality workflow with pre-commit hooks, formatting, linting, type checking, and security scanning

Code Quality Workflow

🎯 Overview

Comprehensive code quality assurance workflow that ensures high standards across the AITBC codebase through automated pre-commit hooks, formatting, linting, type checking, and security scanning.

---

📋 Workflow Steps

Step 1: Setup Pre-commit Environment

# Install pre-commit hooks
./venv/bin/pre-commit install

# Verify installation
./venv/bin/pre-commit --version

Step 2: Run All Quality Checks

# Run all hooks on all files
./venv/bin/pre-commit run --all-files

# Run on staged files (git commit)
./venv/bin/pre-commit run

Step 3: Individual Quality Categories

🧹 Code Formatting

# Black code formatting
./venv/bin/black --line-length=127 --check .

# Auto-fix formatting issues
./venv/bin/black --line-length=127 .

# Import sorting with isort
./venv/bin/isort --profile=black --line-length=127 .

🔍 Linting & Code Analysis

# Flake8 linting
./venv/bin/flake8 --max-line-length=127 --extend-ignore=E203,W503 .

# Pydocstyle documentation checking
./venv/bin/pydocstyle --convention=google .

# Python version upgrade checking
./venv/bin/pyupgrade --py311-plus .

🔍 Type Checking

# Core domain models type checking
./venv/bin/mypy --ignore-missing-imports --show-error-codes apps/coordinator-api/src/app/domain/job.py apps/coordinator-api/src/app/domain/miner.py apps/coordinator-api/src/app/domain/agent_portfolio.py

# Type checking coverage analysis
./scripts/type-checking/check-coverage.sh

# Full mypy checking
./venv/bin/mypy --ignore-missing-imports apps/coordinator-api/src/app/

🛡️ Security Scanning

# Bandit security scanning
./venv/bin/bandit -r . -f json -o bandit-report.json

# Safety dependency vulnerability check
./venv/bin/safety check --json --output safety-report.json

# Safety dependency check for requirements files
./venv/bin/safety check requirements.txt

🧪 Testing

# Unit tests
pytest tests/unit/ --tb=short -q

# Security tests
pytest tests/security/ --tb=short -q

# Performance tests
pytest tests/performance/test_performance_lightweight.py::TestPerformance::test_cli_performance --tb=short -q

---

🔧 Pre-commit Configuration

Repository Structure

repos:
  # Basic file checks
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v5.0.0
    hooks:
      - id: trailing-whitespace
      - id: end-of-file-fixer
      - id: check-yaml
      - id: check-added-large-files
      - id: check-json
      - id: check-merge-conflict
      - id: debug-statements
      - id: check-docstring-first
      - id: check-executables-have-shebangs
      - id: check-toml
      - id: check-xml
      - id: check-case-conflict
      - id: check-ast

  # Code formatting
  - repo: https://github.com/psf/black
    rev: 26.3.1
    hooks:
      - id: black
        language_version: python3
        args: [--line-length=127]

  # Import sorting
  - repo: https://github.com/pycqa/isort
    rev: 8.0.1
    hooks:
      - id: isort
        args: [--profile=black, --line-length=127]

  # Linting
  - repo: https://github.com/pycqa/flake8
    rev: 7.3.0
    hooks:
      - id: flake8
        args: [--max-line-length=127, --extend-ignore=E203,W503]

  # Type checking
  - repo: https://github.com/pre-commit/mirrors-mypy
    rev: v1.19.1
    hooks:
      - id: mypy
        additional_dependencies: [types-requests, types-python-dateutil]
        args: [--ignore-missing-imports]

  # Security scanning
  - repo: https://github.com/PyCQA/bandit
    rev: 1.9.4
    hooks:
      - id: bandit
        args: [-r, ., -f, json, -o, bandit-report.json]
        pass_filenames: false

  # Documentation checking
  - repo: https://github.com/pycqa/pydocstyle
    rev: 6.3.0
    hooks:
      - id: pydocstyle
        args: [--convention=google]

  # Python version upgrade
  - repo: https://github.com/asottile/pyupgrade
    rev: v3.21.2
    hooks:
      - id: pyupgrade
        args: [--py311-plus]

  # Dependency security
  - repo: https://github.com/Lucas-C/pre-commit-hooks-safety
    rev: v1.4.2
    hooks:
      - id: python-safety-dependencies-check
        files: requirements.*\.txt$

  - repo: https://github.com/Lucas-C/pre-commit-hooks-safety
    rev: v1.3.2
    hooks:
      - id: python-safety-check
        args: [--json, --output, safety-report.json]

  # Local hooks
  - repo: local
    hooks:
      - id: pytest-check
        name: pytest-check
        entry: pytest
        language: system
        args: [tests/unit/, --tb=short, -q]
        pass_filenames: false
        always_run: true

      - id: security-check
        name: security-check
        entry: pytest
        language: system
        args: [tests/security/, --tb=short, -q]
        pass_filenames: false
        always_run: true

      - id: performance-check
        name: performance-check
        entry: pytest
        language: system
        args: [tests/performance/test_performance_lightweight.py::TestPerformance::test_cli_performance, --tb=short, -q]
        pass_filenames: false
        always_run: true

      - id: mypy-domain-core
        name: mypy-domain-core
        entry: ./venv/bin/mypy
        language: system
        args: [--ignore-missing-imports, --show-error-codes]
        files: ^apps/coordinator-api/src/app/domain/(job|miner|agent_portfolio)\.py$
        pass_filenames: false

      - id: type-check-coverage
        name: type-check-coverage
        entry: ./scripts/type-checking/check-coverage.sh
        language: script
        files: ^apps/coordinator-api/src/app/
        pass_filenames: false

---

📊 Quality Metrics & Reporting

Coverage Reports

# Type checking coverage
./scripts/type-checking/check-coverage.sh

# Security scan reports
cat bandit-report.json | jq '.results | length'
cat safety-report.json | jq '.vulnerabilities | length'

# Test coverage
pytest --cov=apps --cov-report=html tests/

Quality Score Calculation

# Quality score components:
# - Code formatting: 20%
# - Linting compliance: 20%
# - Type coverage: 25%
# - Test coverage: 20%
# - Security compliance: 15%

# Overall quality score >= 80% required

Automated Reporting

# Generate comprehensive quality report
./scripts/quality/generate-quality-report.sh

# Quality dashboard metrics
curl http://localhost:8000/metrics/quality

---

🚀 Integration with Development Workflow

Before Commit

# 1. Stage your changes
git add .

# 2. Pre-commit hooks run automatically
git commit -m "Your commit message"

# 3. If any hook fails, fix the issues and try again

Manual Quality Checks

# Run all quality checks manually
./venv/bin/pre-commit run --all-files

# Check specific category
./venv/bin/black --check .
./venv/bin/flake8 .
./venv/bin/mypy apps/coordinator-api/src/app/

CI/CD Integration

# GitHub Actions workflow
name: Code Quality
on: [push, pull_request]
jobs:
  quality:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Setup Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.13'
      - name: Install dependencies
        run: pip install -r requirements.txt
      - name: Run pre-commit
        run: ./venv/bin/pre-commit run --all-files

---

🎯 Quality Standards

Code Formatting Standards

• Black: Line length 127 characters
• isort: Black profile compatibility
• Python 3.13+: Modern Python syntax

Linting Standards

• Flake8: Line length 127, ignore E203, W503
• Pydocstyle: Google convention
• No debug statements: Production code only

Type Safety Standards

• MyPy: Strict mode for new code
• Coverage: 90% minimum for core domain
• Error handling: Proper exception types

Security Standards

• Bandit: Zero high-severity issues
• Safety: No known vulnerabilities
• Dependencies: Regular security updates

Testing Standards

• Coverage: 80% minimum test coverage
• Unit tests: All business logic tested
• Security tests: Authentication and authorization
• Performance tests: Critical paths validated

---

📈 Quality Improvement Workflow

1. Initial Setup

# Install pre-commit hooks
./venv/bin/pre-commit install

# Run initial quality check
./venv/bin/pre-commit run --all-files

# Fix any issues found
./venv/bin/black .
./venv/bin/isort .
# Fix other issues manually

2. Daily Development

# Make changes
vim your_file.py

# Stage and commit (pre-commit runs automatically)
git add your_file.py
git commit -m "Add new feature"

# If pre-commit fails, fix issues and retry
git commit -m "Add new feature"

3. Quality Monitoring

# Check quality metrics
./scripts/quality/check-quality-metrics.sh

# Generate quality report
./scripts/quality/generate-quality-report.sh

# Review quality trends
./scripts/quality/quality-trends.sh

---

🔧 Troubleshooting

Common Issues

Black Formatting Issues

# Check formatting issues
./venv/bin/black --check .

# Auto-fix formatting
./venv/bin/black .

# Specific file
./venv/bin/black --check path/to/file.py

Import Sorting Issues

# Check import sorting
./venv/bin/isort --check-only .

# Auto-fix imports
./venv/bin/isort .

# Specific file
./venv/bin/isort path/to/file.py

Type Checking Issues

# Check type errors
./venv/bin/mypy apps/coordinator-api/src/app/

# Ignore specific errors
./venv/bin/mypy --ignore-missing-imports apps/coordinator-api/src/app/

# Show error codes
./venv/bin/mypy --show-error-codes apps/coordinator-api/src/app/

Security Issues

# Check security issues
./venv/bin/bandit -r .

# Generate security report
./venv/bin/bandit -r . -f json -o security-report.json

# Check dependencies
./venv/bin/safety check

Performance Optimization

Pre-commit Performance

# Run hooks in parallel
./venv/bin/pre-commit run --all-files --parallel

# Skip slow hooks during development
./venv/bin/pre-commit run --all-files --hook-stage manual

# Cache dependencies
./venv/bin/pre-commit run --all-files --cache

Selective Hook Running

# Run specific hooks
./venv/bin/pre-commit run black flake8 mypy

# Run on specific files
./venv/bin/pre-commit run --files apps/coordinator-api/src/app/

# Skip hooks
./venv/bin/pre-commit run --all-files --skip mypy

---

📋 Quality Checklist

Before Commit

• Code formatted with Black
• Imports sorted with isort
• Linting passes with Flake8
• Type checking passes with MyPy
• Documentation follows Pydocstyle
• No security vulnerabilities
• All tests pass
• Performance tests pass

Before Merge

• Code review completed
• Quality score >= 80%
• Test coverage >= 80%
• Type coverage >= 90% (core domain)
• Security scan clean
• Documentation updated
• Performance benchmarks met

Before Release

• Full quality suite passes
• Integration tests pass
• Security audit complete
• Performance validation
• Documentation complete
• Release notes prepared

---

🎉 Benefits

Immediate Benefits

• Consistent Code: Uniform formatting and style
• Bug Prevention: Type checking and linting catch issues early
• Security: Automated vulnerability scanning
• Quality Assurance: Comprehensive test coverage

Long-term Benefits

• Maintainability: Clean, well-documented code
• Developer Experience: Automated quality gates
• Team Consistency: Shared quality standards
• Production Readiness: Enterprise-grade code quality

---

Last Updated: March 31, 2026
Workflow Version: 1.0
Next Review: April 30, 2026