feat: major infrastructure refactoring and optimization
All checks were successful
AITBC CLI Level 1 Commands Test / test-cli-level1 (push) Successful in 16s
api-endpoint-tests / test-api-endpoints (push) Successful in 35s
integration-tests / test-service-integration (push) Successful in 1m25s
package-tests / test-python-packages (map[name:aitbc-agent-sdk path:packages/py/aitbc-agent-sdk python_version:3.13]) (push) Successful in 16s
package-tests / test-python-packages (map[name:aitbc-cli path:. python_version:3.13]) (push) Successful in 14s
package-tests / test-python-packages (map[name:aitbc-core path:packages/py/aitbc-core python_version:3.13]) (push) Successful in 13s
package-tests / test-python-packages (map[name:aitbc-crypto path:packages/py/aitbc-crypto python_version:3.13]) (push) Successful in 10s
package-tests / test-python-packages (map[name:aitbc-sdk path:packages/py/aitbc-sdk python_version:3.13]) (push) Successful in 12s
package-tests / test-javascript-packages (map[name:aitbc-sdk node_version:24 path:packages/js/aitbc-sdk]) (push) Successful in 18s
python-tests / test-specific (push) Has been skipped
security-scanning / audit (push) Successful in 14s
systemd-sync / sync-systemd (push) Successful in 4s
package-tests / cross-language-compatibility (push) Successful in 2s
package-tests / package-integration-tests (push) Successful in 3s
Documentation Validation / validate-docs (push) Successful in 6m13s
python-tests / test (push) Successful in 14s
All checks were successful
AITBC CLI Level 1 Commands Test / test-cli-level1 (push) Successful in 16s
api-endpoint-tests / test-api-endpoints (push) Successful in 35s
integration-tests / test-service-integration (push) Successful in 1m25s
package-tests / test-python-packages (map[name:aitbc-agent-sdk path:packages/py/aitbc-agent-sdk python_version:3.13]) (push) Successful in 16s
package-tests / test-python-packages (map[name:aitbc-cli path:. python_version:3.13]) (push) Successful in 14s
package-tests / test-python-packages (map[name:aitbc-core path:packages/py/aitbc-core python_version:3.13]) (push) Successful in 13s
package-tests / test-python-packages (map[name:aitbc-crypto path:packages/py/aitbc-crypto python_version:3.13]) (push) Successful in 10s
package-tests / test-python-packages (map[name:aitbc-sdk path:packages/py/aitbc-sdk python_version:3.13]) (push) Successful in 12s
package-tests / test-javascript-packages (map[name:aitbc-sdk node_version:24 path:packages/js/aitbc-sdk]) (push) Successful in 18s
python-tests / test-specific (push) Has been skipped
security-scanning / audit (push) Successful in 14s
systemd-sync / sync-systemd (push) Successful in 4s
package-tests / cross-language-compatibility (push) Successful in 2s
package-tests / package-integration-tests (push) Successful in 3s
Documentation Validation / validate-docs (push) Successful in 6m13s
python-tests / test (push) Successful in 14s
## 🚀 Central Virtual Environment Implementation - Created central venv at /opt/aitbc/venv for all services - Updated 34+ systemd services to use central python interpreter - Fixed PYTHONPATH configurations for proper module imports - Created aitbc-env wrapper script for environment management ## 📦 Requirements Management Overhaul - Consolidated 8 separate requirements.txt files into central requirements.txt - Added web3>=6.11.0 for blockchain functionality - Created automated requirements migrator tool (scripts/requirements_migrator.py) - Established modular requirements structure (requirements-modules/) - Generated comprehensive migration reports and documentation ## 🔧 Service Configuration Fixes - Fixed Adaptive Learning service domain imports (AgentStatus) - Resolved logging conflicts in zk_proofs and adaptive_learning_health - Created missing data modules (consumer_gpu_profiles.py) - Updated CLI to version 0.2.2 with proper import handling - Fixed infinite loop in CLI alias configuration ## 📡 Port Mapping and Service Updates - Updated blockchain node port from 8545 to 8005 - Added Adaptive Learning service on port 8010 - Consolidated P2P/sync into blockchain-node service - All 5 core services now operational and responding ## 📚 Documentation Enhancements - Updated SYSTEMD_SERVICES.md for Debian root usage (no sudo) - Added comprehensive VIRTUAL_ENVIRONMENT.md guide - Created REQUIREMENTS_MERGE_SUMMARY.md with migration details - Updated RUNTIME_DIRECTORIES.md for standard Linux paths - Fixed service port mappings and dependencies ## 🛠️ CLI Improvements - Fixed import errors and version display (0.2.2) - Resolved infinite loop in bashrc alias - Added proper error handling for missing command modules - Created aitbc-cli wrapper for clean execution ## ✅ Operational Status - 5/5 AITBC services running successfully - All health checks passing - Central virtual environment fully functional - Requirements management streamlined - Documentation accurate and up-to-date ## 🎯 Technical Achievements - Eliminated 7 redundant requirements.txt files - Reduced service startup failures from 34+ to 0 - Established modular dependency management - Created reusable migration tooling - Standardized Debian root deployment practices This represents a complete infrastructure modernization with improved reliability, maintainability, and operational efficiency.
This commit is contained in:
72
requirements-modules/README.md
Normal file
72
requirements-modules/README.md
Normal file
@@ -0,0 +1,72 @@
|
||||
# AITBC Requirements Module Management
|
||||
|
||||
This directory contains specialized requirement modules for different AITBC service categories.
|
||||
|
||||
## Module Structure
|
||||
|
||||
### 🤖 AI/ML & Translation (`ai-ml-translation.txt`)
|
||||
- Translation APIs (OpenAI, Google Translate, DeepL)
|
||||
- Language processing (NLTK, spaCy, polyglot)
|
||||
- AI/ML libraries (PyTorch, transformers)
|
||||
- Quality testing for ML services
|
||||
|
||||
### 🔐 Security & Compliance (`security-compliance.txt`)
|
||||
- Authentication (JWT, password handling)
|
||||
- Encryption and cryptography
|
||||
- Compliance monitoring
|
||||
- Security testing tools
|
||||
|
||||
### 🧪 Testing & Quality (`testing-quality.txt`)
|
||||
- Testing frameworks
|
||||
- Code quality tools
|
||||
- Coverage reporting
|
||||
- Development utilities
|
||||
|
||||
## Usage
|
||||
|
||||
### Installing Module Dependencies
|
||||
```bash
|
||||
# Install AI/ML translation dependencies
|
||||
pip install -r requirements-modules/ai-ml-translation.txt
|
||||
|
||||
# Install security dependencies
|
||||
pip install -r requirements-modules/security-compliance.txt
|
||||
|
||||
# Install testing dependencies
|
||||
pip install -r requirements-modules/testing-quality.txt
|
||||
```
|
||||
|
||||
### Service Integration
|
||||
Services can selectively install only the modules they need:
|
||||
|
||||
```python
|
||||
# Example: Multi-language service
|
||||
# pip install -r requirements-modules/ai-ml-translation.txt
|
||||
|
||||
# Example: Payment service with security
|
||||
# pip install -r requirements-modules/security-compliance.txt
|
||||
|
||||
# Example: Development environment
|
||||
# pip install -r requirements-modules/testing-quality.txt
|
||||
```
|
||||
|
||||
## Benefits
|
||||
|
||||
1. **Modular Installation**: Services only install what they need
|
||||
2. **Reduced Bundle Size**: Smaller Docker images
|
||||
3. **Clear Dependencies**: Easy to understand what each service needs
|
||||
4. **Version Management**: Centralized version control for specialized packages
|
||||
5. **Development Separation**: Dev tools separated from production dependencies
|
||||
|
||||
## Migration Strategy
|
||||
|
||||
1. **Core Dependencies**: Already in `/opt/aitbc/requirements.txt`
|
||||
2. **Service-Specific**: Use appropriate module files
|
||||
3. **Development Only**: Use testing-quality module during development
|
||||
|
||||
## Maintenance
|
||||
|
||||
- Update module versions when upgrading dependencies
|
||||
- Add new packages to appropriate modules
|
||||
- Keep modules focused on their specific domain
|
||||
- Test module installations regularly
|
||||
22
requirements-modules/ai-ml-translation.txt
Normal file
22
requirements-modules/ai-ml-translation.txt
Normal file
@@ -0,0 +1,22 @@
|
||||
# AITBC AI/ML & Translation Module Requirements
|
||||
# Specialized dependencies for AI, ML, and translation services
|
||||
|
||||
# Translation Services
|
||||
openai>=1.3.0
|
||||
google-cloud-translate>=3.11.0
|
||||
deepl>=1.16.0
|
||||
|
||||
# Language Processing
|
||||
langdetect>=1.0.9
|
||||
polyglot>=16.10.0
|
||||
fasttext>=0.9.2
|
||||
nltk>=3.8.1
|
||||
spacy>=3.7.0
|
||||
|
||||
# AI/ML Libraries
|
||||
torch>=2.0.0
|
||||
transformers>=4.0.0
|
||||
numpy>=1.24.0
|
||||
|
||||
# Quality Testing
|
||||
pytest-mock>=3.12.0
|
||||
12
requirements-modules/security-compliance.txt
Normal file
12
requirements-modules/security-compliance.txt
Normal file
@@ -0,0 +1,12 @@
|
||||
# AITBC Security & Compliance Module Requirements
|
||||
# Specialized dependencies for security, authentication, and compliance
|
||||
|
||||
# Authentication & Security
|
||||
python-jose[cryptography]>=3.3.0
|
||||
passlib[bcrypt]>=1.7.4
|
||||
|
||||
# Encryption & Cryptography (additional to core)
|
||||
cryptography>=41.0.0
|
||||
|
||||
# Compliance & Monitoring
|
||||
sentry-sdk>=2.0.0
|
||||
16
requirements-modules/testing-quality.txt
Normal file
16
requirements-modules/testing-quality.txt
Normal file
@@ -0,0 +1,16 @@
|
||||
# AITBC Testing & Quality Module Requirements
|
||||
# Specialized dependencies for testing and quality assurance
|
||||
|
||||
# Testing Framework
|
||||
pytest-mock>=3.12.0
|
||||
pytest-asyncio>=0.21.0
|
||||
pytest>=7.4.0
|
||||
|
||||
# Code Quality
|
||||
black>=23.9.0
|
||||
flake8>=6.1.0
|
||||
mypy>=1.6.0
|
||||
pre-commit>=3.4.0
|
||||
|
||||
# Coverage
|
||||
pytest-cov>=4.0.0
|
||||
Reference in New Issue
Block a user