Some checks failed
AITBC CI/CD Pipeline / lint-and-test (3.11) (push) Has been cancelled
AITBC CI/CD Pipeline / lint-and-test (3.12) (push) Has been cancelled
AITBC CI/CD Pipeline / lint-and-test (3.13) (push) Has been cancelled
AITBC CI/CD Pipeline / test-cli (push) Has been cancelled
AITBC CI/CD Pipeline / test-services (push) Has been cancelled
AITBC CI/CD Pipeline / test-production-services (push) Has been cancelled
AITBC CI/CD Pipeline / security-scan (push) Has been cancelled
AITBC CI/CD Pipeline / build (push) Has been cancelled
AITBC CI/CD Pipeline / deploy-staging (push) Has been cancelled
AITBC CI/CD Pipeline / deploy-production (push) Has been cancelled
AITBC CI/CD Pipeline / performance-test (push) Has been cancelled
AITBC CI/CD Pipeline / docs (push) Has been cancelled
AITBC CI/CD Pipeline / release (push) Has been cancelled
AITBC CI/CD Pipeline / notify (push) Has been cancelled
Security Scanning / Bandit Security Scan (apps/coordinator-api/src) (push) Has been cancelled
Security Scanning / Bandit Security Scan (cli/aitbc_cli) (push) Has been cancelled
Security Scanning / Bandit Security Scan (packages/py/aitbc-core/src) (push) Has been cancelled
Security Scanning / Bandit Security Scan (packages/py/aitbc-crypto/src) (push) Has been cancelled
Security Scanning / Bandit Security Scan (packages/py/aitbc-sdk/src) (push) Has been cancelled
Security Scanning / Bandit Security Scan (tests) (push) Has been cancelled
Security Scanning / CodeQL Security Analysis (javascript) (push) Has been cancelled
Security Scanning / CodeQL Security Analysis (python) (push) Has been cancelled
Security Scanning / Dependency Security Scan (push) Has been cancelled
Security Scanning / Container Security Scan (push) Has been cancelled
Security Scanning / OSSF Scorecard (push) Has been cancelled
Security Scanning / Security Summary Report (push) Has been cancelled
AITBC CLI Level 1 Commands Test / test-cli-level1 (3.11) (push) Has been cancelled
AITBC CLI Level 1 Commands Test / test-cli-level1 (3.12) (push) Has been cancelled
AITBC CLI Level 1 Commands Test / test-cli-level1 (3.13) (push) Has been cancelled
AITBC CLI Level 1 Commands Test / test-summary (push) Has been cancelled
- Remove debugging service documentation (DEBUgging_SERVICES.md) - Remove development logs policy and quick reference guides - Remove E2E test creation summary - Remove gift certificate example file - Remove GitHub pull summary documentation
3.8 KiB
3.8 KiB
Mock Coordinator Services Removal - RESOLVED
Date: February 16, 2026
Status: Resolved
Severity: Low
Issue Description
Mock coordinator services were running on both localhost and AITBC server environments, creating potential confusion between development and production deployments. This could lead to testing against mock data instead of real production APIs.
Affected Components
- Localhost:
aitbc-mock-coordinator.service - AITBC Server:
aitbc-coordinator.service(mock version) - Production:
aitbc-coordinator-api.service(desired service)
Root Cause Analysis
Historical development setup included mock coordinator services for testing purposes. These were never properly cleaned up when moving to production deployment, leading to:
- Multiple coordinator services running simultaneously
- Potential routing to mock endpoints instead of production
- Confusion about which service was handling requests
Solution Implemented
1. Localhost Cleanup
# Stop and disable mock service
systemctl stop aitbc-mock-coordinator.service
systemctl disable aitbc-mock-coordinator.service
# Remove service file
rm /etc/systemd/system/aitbc-mock-coordinator.service
systemctl daemon-reload
2. AITBC Server Cleanup
# Stop and disable mock service
ssh aitbc "systemctl stop aitbc-coordinator.service"
ssh aitbc "systemctl disable aitbc-coordinator.service"
# Remove service file
ssh aitbc "rm /etc/systemd/system/aitbc-coordinator.service"
ssh aitbc "systemctl daemon-reload"
3. Production Service Verification
Confirmed production services running correctly:
- Localhost:
aitbc-coordinator-api.serviceactive on port 8000 - AITBC Server:
aitbc-coordinator-api.serviceactive in container
4. Database Configuration Fix
Fixed database configuration issue that was preventing localhost production service from starting:
- Added missing
effective_urlproperty toDatabaseConfigclass - Fixed module path in systemd service file
- Installed missing dependency (
python-json-logger)
Verification
Tested both production services:
# Localhost health check
curl -s http://localhost:8000/v1/health
# Response: {"status": "ok", "env": "dev"} ✅
# AITBC Server health check
curl -s https://aitbc.bubuit.net/api/health
# Response: {"status": "ok", "env": "dev"} ✅
Service Configuration Differences
Before Cleanup
- Localhost: Mock service + broken production service
- AITBC Server: Mock service + working production service
After Cleanup
- Localhost: Working production service only
- AITBC Server: Working production service only
Impact
- Clarity: Clear separation between development and production environments
- Reliability: Production requests no longer risk hitting mock endpoints
- Maintenance: Reduced service footprint and complexity
- Performance: Eliminated redundant services
Lessons Learned
- Service Hygiene: Always clean up mock/test services before production deployment
- Documentation: Keep accurate inventory of running services
- Configuration: Ensure production services have correct paths and dependencies
- Verification: Test both environments after configuration changes
Current Service Status
Localhost Services
- ✅
aitbc-coordinator-api.service- Production API (active) - ❌
aitbc-mock-coordinator.service- Mock API (removed)
AITBC Server Services
- ✅
aitbc-coordinator-api.service- Production API (active) - ❌
aitbc-coordinator.service- Mock API (removed)