feat: organize documentation into logical subdirectories
✅ Created organized project documentation structure - project/ai-economics/: AI Economics Masters documentation - project/cli/: Command-line interface documentation - project/infrastructure/: System infrastructure and deployment docs - project/requirements/: Project requirements and migration docs - project/completion/: 100% project completion summary - project/workspace/: Workspace strategy and organization ✅ Updated MASTER_INDEX.md to reflect new organization - Added project documentation section with detailed breakdown - Updated navigation to include new subdirectory structure - Maintained existing documentation hierarchy ✅ Updated project/README.md for new organization - Complete project documentation overview - Directory structure explanation - Quick access guide for each subdirectory - Links to related documentation 📊 Documentation Organization Results: - 10 files moved into 6 logical subdirectories - Improved navigation and discoverability - Maintained all existing content and links - Enhanced project documentation structure 🎯 Documentation Status: 100% Organized and Complete
This commit is contained in:
aitbc
37
docs/project/infrastructure/LOGS_ORGANIZATION.md
Normal file
37
docs/project/infrastructure/LOGS_ORGANIZATION.md
Normal file
@@ -0,0 +1,37 @@
|
||||
# Logs Directory Organization
|
||||
|
||||
## Log Files Location
|
||||
|
||||
System logs are now properly organized in /var/log/aitbc/:
|
||||
|
||||
### Current Log Files:
|
||||
-
|
||||
- contract_endpoints_final_status.txt
|
||||
- final_production_ai_results.txt
|
||||
- final_testing_fixes.txt
|
||||
- issues_resolved.txt
|
||||
- marketplace_results_20260329_190503.txt
|
||||
- monitoring_report_20260329_192921.txt
|
||||
- monitoring_report_20260329_193125.txt
|
||||
- network_monitor.log
|
||||
- qa_cycle.log
|
||||
- qa-cycle.log
|
||||
- security_summary.txt
|
||||
- sync_detector.log
|
||||
- testing_completion_report.txt
|
||||
|
||||
### Log Categories:
|
||||
- **audit/**: Audit logs
|
||||
- **network_monitor.log**: Network monitoring logs
|
||||
- **qa_cycle.log**: QA cycle logs
|
||||
- **qa-cycle.log**: QA cycle testing logs (from temp directory)
|
||||
- **host_gpu_miner.log**: GPU miner client logs (2.4MB)
|
||||
- **contract_endpoints_final_status.txt**: Contract endpoint status
|
||||
- **final_production_ai_results.txt**: Production AI results
|
||||
- **monitoring_report_*.txt**: System monitoring reports
|
||||
- **testing_completion_report.txt**: Testing completion logs
|
||||
|
||||
## Change History
|
||||
- **2026-03-30**: Moved from /opt/aitbc/results/ to /var/log/aitbc/ for proper organization
|
||||
- **2026-03-30**: Consolidated /opt/aitbc/logs/host_gpu_miner.log to /var/log/aitbc/ for unified logging
|
||||
- **2026-03-30**: Cleaned up /opt/aitbc/temp/ directory and moved qa-cycle.log to proper logs location
|
||||
76
docs/project/infrastructure/PRODUCTION_ARCHITECTURE.md
Normal file
76
docs/project/infrastructure/PRODUCTION_ARCHITECTURE.md
Normal file
@@ -0,0 +1,76 @@
|
||||
# AITBC Production Environment
|
||||
|
||||
## 🏗️ Proper System Architecture
|
||||
|
||||
The AITBC production environment follows Linux Filesystem Hierarchy Standard (FHS) compliance:
|
||||
|
||||
### 📁 System Directory Structure
|
||||
|
||||
```
|
||||
/etc/aitbc/production/ # Production configurations
|
||||
├── .env # Production environment variables
|
||||
├── blockchain.py # Blockchain service config
|
||||
├── database.py # Database configuration
|
||||
├── services.py # Services configuration
|
||||
└── certs/ # SSL certificates
|
||||
|
||||
/var/lib/aitbc/production/ # Production services and data
|
||||
├── blockchain.py # Production blockchain service
|
||||
├── marketplace.py # Production marketplace service
|
||||
├── unified_marketplace.py # Unified marketplace service
|
||||
├── openclaw_ai.py # OpenClaw AI service
|
||||
└── backups/ # Production backups
|
||||
|
||||
/var/log/aitbc/production/ # Production logs
|
||||
├── blockchain/ # Blockchain service logs
|
||||
├── marketplace/ # Marketplace service logs
|
||||
└── unified_marketplace/ # Unified marketplace logs
|
||||
```
|
||||
|
||||
### 🚀 Launching Production Services
|
||||
|
||||
Use the production launcher:
|
||||
|
||||
```bash
|
||||
# Launch all production services
|
||||
/opt/aitbc/scripts/production_launcher.py
|
||||
|
||||
# Launch individual service
|
||||
python3 /var/lib/aitbc/production/blockchain.py
|
||||
```
|
||||
|
||||
### ⚙️ Configuration Management
|
||||
|
||||
Production configurations are stored in `/etc/aitbc/production/`:
|
||||
- Environment variables in `.env`
|
||||
- Service-specific configs in Python modules
|
||||
- SSL certificates in `certs/`
|
||||
|
||||
### 📊 Monitoring and Logs
|
||||
|
||||
Production logs are centralized in `/var/log/aitbc/production/`:
|
||||
- Each service has its own log directory
|
||||
- Logs rotate automatically
|
||||
- Real-time monitoring available
|
||||
|
||||
### 🔧 Maintenance
|
||||
|
||||
- **Backups**: Stored in `/var/lib/aitbc/production/backups/`
|
||||
- **Updates**: Update code in `/opt/aitbc/`, restart services
|
||||
- **Configuration**: Edit files in `/etc/aitbc/production/`
|
||||
|
||||
### 🛡️ Security
|
||||
|
||||
- All production files have proper permissions
|
||||
- Keystore remains at `/var/lib/aitbc/keystore/`
|
||||
- Environment variables are protected
|
||||
- SSL certificates secured in `/etc/aitbc/production/certs/`
|
||||
|
||||
## 📋 Migration Complete
|
||||
|
||||
The "box in box" structure has been eliminated:
|
||||
- ✅ Configurations moved to `/etc/aitbc/production/`
|
||||
- ✅ Services moved to `/var/lib/aitbc/production/`
|
||||
- ✅ Logs centralized in `/var/log/aitbc/production/`
|
||||
- ✅ Repository cleaned of production runtime files
|
||||
- ✅ Proper FHS compliance achieved
|
||||
39
docs/project/infrastructure/RUNTIME_DIRECTORIES.md
Normal file
39
docs/project/infrastructure/RUNTIME_DIRECTORIES.md
Normal file
@@ -0,0 +1,39 @@
|
||||
# AITBC Runtime Directory Structure
|
||||
|
||||
This document outlines the standard Linux system directories used by AITBC for runtime data.
|
||||
|
||||
## Standard System Directories
|
||||
|
||||
### `/var/lib/aitbc/`
|
||||
**Purpose**: Application data and databases
|
||||
- `keystore/` - Blockchain private keys and certificates
|
||||
- `data/` - Application databases (.db, .sqlite files)
|
||||
- `logs/` - Application log files
|
||||
|
||||
### `/etc/aitbc/`
|
||||
**Purpose**: Configuration files
|
||||
- Environment files (.env)
|
||||
- Service configuration
|
||||
- Network settings
|
||||
|
||||
### `/var/log/aitbc/`
|
||||
**Purpose**: System logging (symlinked from `/var/lib/aitbc/logs/`)
|
||||
|
||||
## Security & Permissions
|
||||
|
||||
- **Keystore**: Restricted permissions (600/700)
|
||||
- **Config**: Read-only for services, writable for admin
|
||||
- **Logs**: Writable by services, readable by admin
|
||||
|
||||
## Migration from Repo
|
||||
|
||||
Runtime data has been moved from `/opt/aitbc/data/` to system standard directories:
|
||||
- Old: `/opt/aitbc/data/keystore/` → New: `/var/lib/aitbc/keystore/`
|
||||
- Old: `/opt/aitbc/data/` → New: `/var/lib/aitbc/data/`
|
||||
|
||||
## SystemD Integration
|
||||
|
||||
Services should be updated to use these standard paths:
|
||||
- `Environment=KEYSTORE_PATH=/var/lib/aitbc/keystore`
|
||||
- `Environment=DB_PATH=/var/lib/aitbc/data`
|
||||
- `Environment=LOG_PATH=/var/log/aitbc`
|
||||
211
docs/project/infrastructure/VIRTUAL_ENVIRONMENT.md
Normal file
211
docs/project/infrastructure/VIRTUAL_ENVIRONMENT.md
Normal file
@@ -0,0 +1,211 @@
|
||||
# AITBC Central Virtual Environment Guide
|
||||
|
||||
**Last Updated**: 2026-03-29
|
||||
**Version**: 3.2 (Virtual Environment Standardization)
|
||||
|
||||
## Overview
|
||||
|
||||
AITBC now uses a central Python virtual environment to manage all dependencies consistently across services. This eliminates conflicts with system Python packages and provides a clean, isolated environment for all AITBC components.
|
||||
|
||||
## 🏗️ Virtual Environment Structure
|
||||
|
||||
```
|
||||
/opt/aitbc/
|
||||
├── venv/ # Central virtual environment
|
||||
│ ├── bin/ # Python executables and scripts
|
||||
│ ├── lib/ # Installed packages
|
||||
│ └── pyvenv.cfg # Virtual environment configuration
|
||||
├── requirements.txt # Central dependency list
|
||||
├── aitbc-env # Environment wrapper script
|
||||
└── apps/ # AITBC applications
|
||||
```
|
||||
|
||||
## 🚀 Quick Start
|
||||
|
||||
### Activate Virtual Environment
|
||||
```bash
|
||||
# Use the environment wrapper (recommended)
|
||||
/opt/aitbc/aitbc-env
|
||||
|
||||
# Or activate directly
|
||||
source /opt/aitbc/venv/bin/activate
|
||||
```
|
||||
|
||||
### CLI Usage
|
||||
```bash
|
||||
# Start interactive shell with CLI access
|
||||
/opt/aitbc/aitbc-env
|
||||
|
||||
# Use CLI commands directly
|
||||
/opt/aitbc/aitbc-env aitbc --help
|
||||
|
||||
# Run Python scripts with venv
|
||||
/opt/aitbc/aitbc-env python script.py
|
||||
```
|
||||
|
||||
## 📦 Package Management
|
||||
|
||||
### Dependencies Included
|
||||
- **Web Framework**: FastAPI, Uvicorn
|
||||
- **Database**: SQLAlchemy, SQLModel, Alembic
|
||||
- **Security**: Cryptography, PyNaCl
|
||||
- **CLI Tools**: Click, Rich, Typer
|
||||
- **AI/ML**: NumPy, Pandas, OpenCV
|
||||
- **Monitoring**: Prometheus Client, Structlog
|
||||
|
||||
### Installing New Packages
|
||||
```bash
|
||||
# Activate environment first
|
||||
/opt/aitbc/aitbc-env
|
||||
|
||||
# Install packages
|
||||
pip install package-name
|
||||
|
||||
# Update requirements.txt
|
||||
pip freeze > /opt/aitbc/requirements.txt
|
||||
```
|
||||
|
||||
## 🔧 Service Integration
|
||||
|
||||
### Updated Services
|
||||
All major AITBC services now use the central virtual environment:
|
||||
|
||||
- ✅ **Wallet Service**: `/opt/aitbc/venv/bin/python`
|
||||
- ✅ **Exchange API**: `/opt/aitbc/venv/bin/python`
|
||||
- ✅ **Coordinator API**: `/opt/aitbc/venv/bin/python`
|
||||
- ✅ **Blockchain Node**: `/opt/aitbc/venv/bin/python`
|
||||
|
||||
### SystemD Configuration
|
||||
Services automatically use the central venv via updated ExecStart paths:
|
||||
```ini
|
||||
[Service]
|
||||
ExecStart=/opt/aitbc/venv/bin/python service_script.py
|
||||
```
|
||||
|
||||
## 🛠️ Development Workflow
|
||||
|
||||
### Development Environment
|
||||
```bash
|
||||
# Activate for development
|
||||
/opt/aitbc/aitbc-env
|
||||
|
||||
# Run development servers
|
||||
cd /opt/aitbc/apps/coordinator-api
|
||||
uvicorn app.main:app --reload
|
||||
|
||||
# Run tests
|
||||
pytest tests/
|
||||
```
|
||||
|
||||
### Environment Variables
|
||||
The environment wrapper sets up:
|
||||
```bash
|
||||
PYTHONPATH=/opt/aitbc/packages/py/aitbc-sdk/src:/opt/aitbc/packages/py/aitbc-crypto/src
|
||||
AITBC_VENV=/opt/aitbc/venv
|
||||
PATH=/opt/aitbc/venv/bin:$PATH
|
||||
```
|
||||
|
||||
## 🔍 Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
**Service Not Starting**
|
||||
```bash
|
||||
# Check if venv exists
|
||||
ls -la /opt/aitbc/venv/
|
||||
|
||||
# Check service status
|
||||
sudo systemctl status aitbc-service-name
|
||||
|
||||
# Check logs
|
||||
sudo journalctl -u aitbc-service-name -n 20
|
||||
```
|
||||
|
||||
**Missing Packages**
|
||||
```bash
|
||||
# Install missing package
|
||||
/opt/aitbc/aitbc-env pip install package-name
|
||||
|
||||
# Update all services
|
||||
sudo systemctl restart aitbc-*
|
||||
```
|
||||
|
||||
**Import Errors**
|
||||
```bash
|
||||
# Check PYTHONPATH
|
||||
echo $PYTHONPATH
|
||||
|
||||
# Verify package installation
|
||||
/opt/aitbc/aitbc-env python -c "import package_name"
|
||||
```
|
||||
|
||||
### Recreate Virtual Environment
|
||||
```bash
|
||||
# Backup current requirements
|
||||
cp /opt/aitbc/requirements.txt /tmp/
|
||||
|
||||
# Remove and recreate
|
||||
sudo rm -rf /opt/aitbc/venv
|
||||
sudo python3 -m venv /opt/aitbc/venv
|
||||
sudo chown -R root:root /opt/aitbc/venv
|
||||
|
||||
# Install packages
|
||||
source /opt/aitbc/venv/bin/activate
|
||||
pip install -r /opt/aitbc/requirements.txt
|
||||
```
|
||||
|
||||
## 📋 Management Commands
|
||||
|
||||
### Virtual Environment
|
||||
```bash
|
||||
# Check Python version
|
||||
/opt/aitbc/aitbc-env python --version
|
||||
|
||||
# List installed packages
|
||||
/opt/aitbc/aitbc-env pip list
|
||||
|
||||
# Check package details
|
||||
/opt/aitbc/aitbc-env pip show package-name
|
||||
```
|
||||
|
||||
### Services
|
||||
```bash
|
||||
# Restart all services with venv
|
||||
sudo systemctl restart aitbc-wallet aitbc-exchange-api
|
||||
|
||||
# Check service status
|
||||
sudo systemctl status aitbc-*
|
||||
|
||||
# View service logs
|
||||
sudo journalctl -u aitbc-service-name -f
|
||||
```
|
||||
|
||||
## 🎯 Best Practices
|
||||
|
||||
1. **Always use the environment wrapper** (`/opt/aitbc/aitbc-env`) for consistency
|
||||
2. **Update requirements.txt** when adding new packages
|
||||
3. **Test services** after dependency updates
|
||||
4. **Monitor disk space** - venv can grow with many packages
|
||||
5. **Keep dependencies minimal** - only install what's needed
|
||||
|
||||
## 🔄 Migration Notes
|
||||
|
||||
### From System Python
|
||||
- No more `--break-system-packages` needed
|
||||
- Clean separation from OS packages
|
||||
- Consistent package versions across services
|
||||
|
||||
### From Multiple Venvs
|
||||
- Single source of truth for dependencies
|
||||
- Easier maintenance and updates
|
||||
- Reduced disk usage
|
||||
|
||||
## 📚 Additional Resources
|
||||
|
||||
- [Python Virtual Environments](https://docs.python.org/3/library/venv.html)
|
||||
- [Pip Documentation](https://pip.pypa.io/)
|
||||
- [AITBC Service Management](../infrastructure/SYSTEMD_SERVICES.md)
|
||||
|
||||
---
|
||||
|
||||
**Next Steps**: Use `/opt/aitbc/aitbc-env` for all AITBC development and operations.
|
||||
Reference in New Issue
Block a user