diff --git a/.windsurf/skills/aitbc-systemd-git-workflow.md b/.windsurf/skills/aitbc-systemd-git-workflow.md new file mode 100644 index 00000000..55f5033f --- /dev/null +++ b/.windsurf/skills/aitbc-systemd-git-workflow.md @@ -0,0 +1,106 @@ +# AITBC Systemd Git Workflow Skill + +## Description +Expert skill for managing systemd service files using proper git workflow instead of scp operations. Ensures systemd configurations are always synchronized via git repository rather than direct file copying. + +## Core Principles + +### Git-Tracked Files Only +- All systemd service files must be edited in `/opt/aitbc/systemd/` (git-tracked directory) +- NEVER edit files directly in `/etc/systemd/system/` +- NEVER use scp to copy systemd files between nodes + +### Symbolic Link Architecture +- `/etc/systemd/system/aitbc-*.service` -> `/opt/aitbc/systemd/aitbc-*.service` +- Symlinks ensure active systemd files always match repository +- Changes in repository automatically reflected in active configuration + +## Standard Workflow + +### Local Changes +1. Edit files in `/opt/aitbc/systemd/` +2. Commit changes: `git add systemd/ && git commit -m "description"` +3. Push to gitea: `git push` + +### Remote Sync (aitbc1) +1. Pull changes: `git pull` +2. Create/update symlinks: `/opt/aitbc/scripts/utils/link-systemd.sh` +3. Reload systemd: `systemctl daemon-reload` +4. Restart affected services: `systemctl restart aitbc-*` + +## Available Scripts + +### link-systemd.sh +- Location: `/opt/aitbc/scripts/utils/link-systemd.sh` +- Purpose: Creates symbolic links from `/etc/systemd/system/` to `/opt/aitbc/systemd/` +- Usage: `/opt/aitbc/scripts/utils/link-systemd.sh` +- Benefits: Automatic sync, no manual file copying needed + +### sync-systemd.sh +- Location: `/opt/aitbc/scripts/sync/sync-systemd.sh` +- Purpose: Copies repository files to active systemd (alternative to symlinks) +- Usage: `/opt/aitbc/scripts/sync/sync-systemd.sh` +- Note: Prefer link-systemd.sh for automatic sync + +## Common Issues + +### Git Conflicts on Remote Nodes +**Symptom**: `git pull` fails with "local changes would be overwritten" + +**Resolution**: +1. Discard local changes: `git reset --hard HEAD` +2. Pull changes: `git pull` +3. Re-run link-systemd.sh: `/opt/aitbc/scripts/utils/link-systemd.sh` + +### Broken Symlinks +**Symptom**: Systemd service fails to load or uses old configuration + +**Resolution**: +1. Verify symlinks: `ls -la /etc/systemd/system/aitbc-*` +2. Re-create symlinks: `/opt/aitbc/scripts/utils/link-systemd.sh` +3. Reload systemd: `systemctl daemon-reload` + +### SCP Usage Warning +**Symptom**: Direct scp to `/etc/systemd/system/` breaks symlinks + +**Resolution**: +1. Never use scp to `/etc/systemd/system/` +2. Always use git workflow +3. If scp was used, restore proper symlinks with link-systemd.sh + +## Verification Commands + +### Check Symlink Status +```bash +ls -la /etc/systemd/system/aitbc-* +readlink /etc/systemd/system/aitbc-blockchain-node.service +``` + +### Verify Git Status +```bash +git status +git diff systemd/ +``` + +### Check Service Configuration +```bash +systemctl cat aitbc-blockchain-node.service +``` + +## Best Practices + +1. **Always edit in git-tracked directory**: `/opt/aitbc/systemd/` +2. **Commit before pushing**: Ensure changes are properly committed +3. **Pull before link-systemd.sh**: Ensure repository is up-to-date +4. **Test locally first**: Verify changes work before syncing to remote +5. **Document changes**: Use descriptive commit messages +6. **Monitor logs**: Check service logs after changes +7. **Run as root**: No sudo needed - we are root on both nodes + +## Memory Reference +See memory entry `systemd-git-workflow` for detailed workflow documentation (no sudo needed - we are root on both nodes). + +## Related Skills +- aitbc-basic-operations-skill: Basic git operations +- aitbc-system-architect: System architecture understanding +- blockchain-troubleshoot-recovery: Service troubleshooting diff --git a/systemd/aitbc-gpu.service b/systemd/aitbc-gpu.service deleted file mode 100644 index 3f01e09a..00000000 --- a/systemd/aitbc-gpu.service +++ /dev/null @@ -1,43 +0,0 @@ -[Unit] -Description=AITBC Marketplace Service -After=network.target postgresql.service redis.service -Wants=postgresql.service redis.service - -[Service] -Type=simple -User=root -Group=root -WorkingDirectory=/opt/aitbc -Environment=PATH=/usr/bin:/usr/local/bin:/usr/bin:/bin -Environment=PYTHONPATH=/opt/aitbc/services -EnvironmentFile=/etc/aitbc/production.env - -# Marketplace execution -ExecStart=/opt/aitbc/venv/bin/python /opt/aitbc/services/gpu_marketplace_launcher.py -ExecReload=/bin/kill -HUP $MAINPID -KillMode=mixed -TimeoutStopSec=10 - -# Production reliability -Restart=always -RestartSec=5 -StartLimitBurst=5 -StartLimitIntervalSec=60 - -# Production logging -StandardOutput=journal -StandardError=journal -SyslogIdentifier=aitbc-marketplace - -# Production security -NoNewPrivileges=true -ProtectHome=true - -# Production performance -LimitNOFILE=65536 -LimitNPROC=4096 -MemoryMax=2G -CPUQuota=50% - -[Install] -WantedBy=multi-user.target