ci: enforce strict exit codes in workflow tests
Some checks failed
API Endpoint Tests / test-api-endpoints (push) Failing after 36s
CLI Tests / test-cli (push) Failing after 3m9s
Documentation Validation / validate-docs (push) Successful in 8s
Integration Tests / test-service-integration (push) Failing after 3s
JavaScript SDK Tests / test-js-sdk (push) Successful in 7s
Package Tests / test-python-packages (map[name:aitbc-agent-sdk path:packages/py/aitbc-agent-sdk]) (push) Failing after 8s
Package Tests / test-python-packages (map[name:aitbc-core path:packages/py/aitbc-core]) (push) Failing after 29s
Package Tests / test-python-packages (map[name:aitbc-crypto path:packages/py/aitbc-crypto]) (push) Failing after 13s
Package Tests / test-python-packages (map[name:aitbc-sdk path:packages/py/aitbc-sdk]) (push) Failing after 16s
Package Tests / test-javascript-packages (map[name:aitbc-sdk-js path:packages/js/aitbc-sdk]) (push) Successful in 7s
Package Tests / test-javascript-packages (map[name:aitbc-token path:packages/solidity/aitbc-token]) (push) Failing after 18s
Python Tests / test-python (push) Failing after 3m37s
Rust ZK Components Tests / test-rust-zk (push) Successful in 28s
Security Scanning / security-scan (push) Failing after 46s
Smart Contract Tests / test-solidity (map[name:aitbc-token path:packages/solidity/aitbc-token]) (push) Failing after 18s
Smart Contract Tests / test-solidity (map[name:zk-circuits path:apps/zk-circuits]) (push) Failing after 43s
Smart Contract Tests / lint-solidity (push) Failing after 12s
Staking Tests / test-staking-service (push) Failing after 2m33s
Staking Tests / test-staking-integration (push) Has been skipped
Staking Tests / test-staking-contract (push) Has been skipped
Staking Tests / run-staking-test-runner (push) Has been skipped
Systemd Sync / sync-systemd (push) Failing after 4s

- Remove `|| echo "⚠️ ..."` fallbacks that masked failures
- Add explicit `exit 1` on port readiness failures and missing test directories
- Track port_ready flag in health check loops to fail if services don't start
- Replace warning emoji (⚠️) with error emoji () for actual failures
- Fix docs-validation to use curated Markdown target list excluding high-noise directories
- Update rust-zk-tests paths from gpu_acceleration/research to dev
This commit is contained in:
aitbc
2026-04-18 11:57:35 +02:00
parent 40698f91fd
commit 23348892b9
34 changed files with 2680 additions and 1445 deletions

View File

@@ -2,7 +2,8 @@
## Overview
This document outlines the recommended branch protection settings for the AITBC repository to ensure code quality, security, and collaboration standards.
This document outlines the recommended branch protection settings for the AITBC
repository to ensure code quality, security, and collaboration standards.
## GitHub Branch Protection Settings
@@ -14,11 +15,13 @@ Navigate to: `Settings > Branches > Branch protection rules`
**Branch name pattern**: `main`
**Require status checks to pass before merging**
##### Require status checks to pass before merging
- ✅ Require branches to be up to date before merging
- ✅ Require status checks to pass before merging
**Required status checks**
##### Required status checks
- ✅ Lint (ruff)
- ✅ Check .env.example drift
- ✅ Test (pytest)
@@ -34,22 +37,28 @@ Navigate to: `Settings > Branches > Branch protection rules`
- ✅ security-scanning / trivy
- ✅ security-scanning / ossf-scorecard
**Require pull request reviews before merging**
##### Require pull request reviews before merging
- ✅ Require approvals
- **Required approving reviews**: 2
- ✅ Dismiss stale PR approvals when new commits are pushed
- ✅ Require review from CODEOWNERS
- ✅ Require review from users with write access in the target repository
- ✅ Limit the number of approvals required (2) - **Do not allow users with write access to approve their own pull requests**
- ✅ Limit the number of approvals required (2)
- **Do not allow users with write access to approve their own pull
requests**
##### Restrict pushes
**Restrict pushes**
- ✅ Limit pushes to users who have write access in the repository
- ✅ Do not allow force pushes
**Restrict deletions**
##### Restrict deletions
- ✅ Do not allow users with write access to delete matching branches
**Require signed commits**
##### Require signed commits
- ✅ Require signed commits (optional, for enhanced security)
### Develop Branch Protection
@@ -57,6 +66,7 @@ Navigate to: `Settings > Branches > Branch protection rules`
**Branch name pattern**: `develop`
**Settings** (same as main, but with fewer required checks):
- Require status checks to pass before merging
- Required status checks: Lint, Test, Check .env.example drift
- Require pull request reviews before merging (1 approval)
@@ -67,26 +77,39 @@ Navigate to: `Settings > Branches > Branch protection rules`
### Continuous Integration Checks
| Status Check | Description | Workflow |
|-------------|-------------|----------|
| `Lint (ruff)` | Python code linting | `.github/workflows/ci.yml` |
| `Check .env.example drift` | Configuration drift detection | `.github/workflows/ci.yml` |
| `Test (pytest)` | Python unit tests | `.github/workflows/ci.yml` |
| `contracts-ci / Lint` | Solidity linting | `.github/workflows/contracts-ci.yml` |
| `contracts-ci / Slither Analysis` | Solidity security analysis | `.github/workflows/contracts-ci.yml` |
| `contracts-ci / Compile` | Smart contract compilation | `.github/workflows/contracts-ci.yml` |
| `contracts-ci / Test` | Smart contract tests | `.github/workflows/contracts-ci.yml` |
| `dotenv-check / dotenv-validation` | .env.example format validation | `.github/workflows/dotenv-check.yml` |
| `dotenv-check / dotenv-security` | .env.example security check | `.github/workflows/dotenv-check.yml` |
| `security-scanning / bandit` | Python security scanning | `.github/workflows/security-scanning.yml` |
| `security-scanning / codeql` | CodeQL analysis | `.github/workflows/security-scanning.yml` |
| `security-scanning / safety` | Dependency vulnerability scan | `.github/workflows/security-scanning.yml` |
| `security-scanning / trivy` | Container security scan | `.github/workflows/security-scanning.yml` |
| `security-scanning / ossf-scorecard` | OSSF Scorecard analysis | `.github/workflows/security-scanning.yml` |
- **`Lint (ruff)`**: Python code linting. Workflow:
`.github/workflows/ci.yml`
- **`Check .env.example drift`**: Configuration drift detection. Workflow:
`.github/workflows/ci.yml`
- **`Test (pytest)`**: Python unit tests. Workflow:
`.github/workflows/ci.yml`
- **`contracts-ci / Lint`**: Solidity linting. Workflow:
`.github/workflows/contracts-ci.yml`
- **`contracts-ci / Slither Analysis`**: Solidity security analysis.
Workflow: `.github/workflows/contracts-ci.yml`
- **`contracts-ci / Compile`**: Smart contract compilation. Workflow:
`.github/workflows/contracts-ci.yml`
- **`contracts-ci / Test`**: Smart contract tests. Workflow:
`.github/workflows/contracts-ci.yml`
- **`dotenv-check / dotenv-validation`**: `.env.example` format validation.
Workflow: `.github/workflows/dotenv-check.yml`
- **`dotenv-check / dotenv-security`**: `.env.example` security check.
Workflow: `.github/workflows/dotenv-check.yml`
- **`security-scanning / bandit`**: Python security scanning. Workflow:
`.github/workflows/security-scanning.yml`
- **`security-scanning / codeql`**: CodeQL analysis. Workflow:
`.github/workflows/security-scanning.yml`
- **`security-scanning / safety`**: Dependency vulnerability scan. Workflow:
`.github/workflows/security-scanning.yml`
- **`security-scanning / trivy`**: Container security scan. Workflow:
`.github/workflows/security-scanning.yml`
- **`security-scanning / ossf-scorecard`**: OSSF Scorecard analysis.
Workflow: `.github/workflows/security-scanning.yml`
### Additional Checks for Feature Branches
For feature branches, consider requiring:
- `comprehensive-tests / unit-tests`
- `comprehensive-tests / integration-tests`
- `comprehensive-tests / api-tests`
@@ -94,7 +117,8 @@ For feature branches, consider requiring:
## CODEOWNERS Integration
The branch protection should be configured to require review from CODEOWNERS. This ensures that:
The branch protection should be configured to require review from CODEOWNERS.
This ensures that:
1. **Domain experts review relevant changes**
2. **Security team reviews security-sensitive files**
@@ -208,7 +232,9 @@ jobs:
run: python scripts/focused_dotenv_linter.py --check
- name: Test (pytest)
run: poetry run pytest --cov=aitbc_cli --cov-report=term-missing --cov-report=xml
run: >-
poetry run pytest --cov=aitbc_cli --cov-report=term-missing
--cov-report=xml
```
## Security Best Practices
@@ -386,6 +412,9 @@ New team members should be trained on:
## Conclusion
Proper branch protection configuration ensures code quality, security, and collaboration standards. By implementing these settings, the AITBC repository maintains high standards while enabling efficient development workflows.
Proper branch protection configuration ensures code quality, security, and
collaboration standards. By implementing these settings, the AITBC repository
maintains high standards while enabling efficient development workflows.
Regular review and updates to branch protection settings ensure they remain effective as the project evolves.
Regular review and updates to branch protection settings ensure they remain
effective as the project evolves.

View File

@@ -2,12 +2,16 @@
## 🔐 Security Overview
This document outlines the comprehensive security policy for CLI translation functionality in the AITBC platform, ensuring that translation services never compromise security-sensitive operations.
This document outlines the comprehensive security policy for CLI translation
functionality in the AITBC platform, ensuring that translation services never
compromise security-sensitive operations.
## ⚠️ Security Problem Statement
### Identified Risks
1. **API Dependency**: Translation services rely on external APIs (OpenAI, Google, DeepL)
1. **API Dependency**: Translation services rely on external APIs (OpenAI,
Google, DeepL)
2. **Network Failures**: Translation unavailable during network outages
3. **Data Privacy**: Sensitive command data sent to third-party services
4. **Command Injection**: Risk of translated commands altering security context
@@ -15,6 +19,7 @@ This document outlines the comprehensive security policy for CLI translation fun
6. **Audit Trail**: Loss of original command intent in translation
### Security-Sensitive Operations
- **Agent Strategy Commands**: `aitbc agent strategy --aggressive`
- **Wallet Operations**: `aitbc wallet send --to 0x... --amount 100`
- **Deployment Commands**: `aitbc deploy --production`
@@ -26,48 +31,63 @@ This document outlines the comprehensive security policy for CLI translation fun
### Security Levels
#### 🔴 CRITICAL (Translation Disabled)
**Commands**: `agent`, `strategy`, `wallet`, `sign`, `deploy`, `genesis`, `transfer`, `send`, `approve`, `mint`, `burn`, `stake`
**Commands**: `agent`, `strategy`, `wallet`, `sign`, `deploy`, `genesis`,
`transfer`, `send`, `approve`, `mint`, `burn`, `stake`
**Policy**:
- ✅ Translation: **DISABLED**
- ✅ External APIs: **BLOCKED**
- ✅ User Consent: **REQUIRED**
- ✅ Fallback: **Original text only**
**Rationale**: These commands handle sensitive operations where translation could compromise security or financial transactions.
**Rationale**: These commands handle sensitive operations where translation
could compromise security or financial transactions.
#### 🟠 HIGH (Local Translation Only)
**Commands**: `config`, `node`, `chain`, `marketplace`, `swap`, `liquidity`, `governance`, `vote`, `proposal`
**Commands**: `config`, `node`, `chain`, `marketplace`, `swap`, `liquidity`,
`governance`, `vote`, `proposal`
**Policy**:
- ✅ Translation: **LOCAL ONLY**
- ✅ External APIs: **BLOCKED**
- ✅ User Consent: **REQUIRED**
- ✅ Fallback: **Local dictionary**
**Rationale**: Important operations that benefit from localization but don't require external services.
**Rationale**: Important operations that benefit from localization but don't
require external services.
#### 🟡 MEDIUM (Fallback Mode)
**Commands**: `balance`, `status`, `monitor`, `analytics`, `logs`, `history`, `simulate`, `test`
**Commands**: `balance`, `status`, `monitor`, `analytics`, `logs`, `history`,
`simulate`, `test`
**Policy**:
- ✅ Translation: **EXTERNAL WITH LOCAL FALLBACK**
- ✅ External APIs: **ALLOWED**
- ✅ User Consent: **NOT REQUIRED**
- ✅ Fallback: **Local translation on failure**
**Rationale**: Standard operations where translation enhances user experience but isn't critical.
**Rationale**: Standard operations where translation enhances user experience
but isn't critical.
#### 🟢 LOW (Full Translation)
**Commands**: `help`, `version`, `info`, `list`, `show`, `explain`
**Policy**:
- ✅ Translation: **FULL CAPABILITIES**
- ✅ External APIs: **ALLOWED**
- ✅ User Consent: **NOT REQUIRED**
- ✅ Fallback: **External retry then local**
**Rationale**: Informational commands where translation improves accessibility without security impact.
**Rationale**: Informational commands where translation improves
accessibility without security impact.
## 🔧 Implementation Details
@@ -107,15 +127,26 @@ HIGH_POLICY = {
### Local Translation System
For security-sensitive operations, a local translation system provides basic localization:
For security-sensitive operations, a local translation system provides basic
localization:
```python
LOCAL_TRANSLATIONS = {
"help": {"es": "ayuda", "fr": "aide", "de": "hilfe", "zh": "帮助"},
"error": {"es": "error", "fr": "erreur", "de": "fehler", "zh": "错误"},
"success": {"es": "éxito", "fr": "succès", "de": "erfolg", "zh": "成功"},
"wallet": {"es": "cartera", "fr": "portefeuille", "de": "börse", "zh": "钱包"},
"transaction": {"es": "transacción", "fr": "transaction", "de": "transaktion", "zh": "交易"}
"wallet": {
"es": "cartera",
"fr": "portefeuille",
"de": "börse",
"zh": "钱包"
},
"transaction": {
"es": "transacción",
"fr": "transaction",
"de": "transaktion",
"zh": "交易"
}
}
```
@@ -237,7 +268,10 @@ from aitbc_cli.security import get_translation_security_report
report = get_translation_security_report()
print(f"Total security checks: {report['security_summary']['total_checks']}")
print(f"Critical operations: {report['security_summary']['by_security_level']['critical']}")
print(
f"Critical operations: "
f"{report['security_summary']['by_security_level']['critical']}"
)
print(f"Recommendations: {report['recommendations']}")
```
@@ -333,7 +367,8 @@ def handle_security_incident(incident_type: str):
### Key Performance Indicators
- **Translation Success Rate**: Percentage of successful translations by security level
- **Translation Success Rate**: Percentage of successful translations by
security level
- **Fallback Usage Rate**: How often local fallback is used
- **API Response Time**: External API performance metrics
- **Security Violations**: Attempts to bypass security policies
@@ -356,24 +391,32 @@ def get_security_metrics():
### Planned Security Features
1. **Machine Learning Detection**: AI-powered detection of sensitive command patterns
2. **Dynamic Policy Adjustment**: Automatic security level adjustment based on context
1. **Machine Learning Detection**: AI-powered detection of sensitive command
patterns
2. **Dynamic Policy Adjustment**: Automatic security level adjustment based on
context
3. **Zero-Knowledge Translation**: Privacy-preserving translation protocols
4. **Blockchain Auditing**: Immutable audit trail on blockchain
5. **Multi-Factor Authentication**: Additional security for sensitive translations
5. **Multi-Factor Authentication**: Additional security for sensitive
translations
### Research Areas
1. **Federated Learning**: Local translation models without external dependencies
2. **Quantum-Resistant Security**: Future-proofing against quantum computing threats
1. **Federated Learning**: Local translation models without external
dependencies
2. **Quantum-Resistant Security**: Future-proofing against quantum computing
threats
3. **Behavioral Analysis**: User behavior patterns for anomaly detection
4. **Cross-Platform Security**: Consistent security across all CLI platforms
---
**Security Policy Status**: ✅ **IMPLEMENTED**
**Last Updated**: March 3, 2026
**Next Review**: March 17, 2026
**Security Level**: 🔒 **HIGH** - Comprehensive protection for sensitive operations
- **Security Policy Status**: ✅ **IMPLEMENTED**
- **Last Updated**: March 3, 2026
- **Next Review**: March 17, 2026
- **Security Level**: 🔒 **HIGH** - Comprehensive protection for sensitive
operations
This security policy ensures that CLI translation functionality never compromises security-sensitive operations while providing appropriate localization capabilities for non-critical commands.
This security policy ensures that CLI translation functionality never
compromises security-sensitive operations while providing appropriate
localization capabilities for non-critical commands.

View File

@@ -2,7 +2,9 @@
## 🎯 Problem Solved
Having a `.env.example` file is good practice, but without automated checking, it can drift from what the application actually uses. This creates silent configuration issues where:
Having a `.env.example` file is good practice, but without automated
checking, it can drift from what the application actually uses. This creates
silent configuration issues where:
- New environment variables are added to code but not documented
- Old variables remain in `.env.example` but are no longer used
@@ -14,28 +16,35 @@ Having a `.env.example` file is good practice, but without automated checking, i
### **Focused Dotenv Linter**
Created a sophisticated linter that:
- **Scans all code** for actual environment variable usage
- **Filters out script variables** and non-config variables
- **Compares with `.env.example`** to find drift
- **Auto-fixes missing variables** in `.env.example
- **Auto-fixes missing variables** in `.env.example`
- **Validates format** and security of `.env.example`
- **Integrates with CI/CD** to prevent drift
### **Key Features**
#### **Smart Variable Detection**
- Scans Python files for `os.environ.get()`, `os.getenv()`, etc.
- Scans config files for `${VAR}` and `$VAR` patterns
- Scans shell scripts for `export VAR=` and `VAR=` patterns
- Filters out script variables, system variables, and internal variables
#### **Comprehensive Coverage**
- **Python files**: `*.py` across the entire project
- **Config files**: `pyproject.toml`, `*.yml`, `*.yaml`, `Dockerfile`, etc.
- **Shell scripts**: `*.sh`, `*.bash`, `*.zsh`
- **CI/CD files**: `.github/workflows/*.yml`
#### **Intelligent Filtering**
- Excludes common script variables (`PID`, `VERSION`, `DEBUG`, etc.)
- Excludes system variables (`PATH`, `HOME`, `USER`, etc.)
- Excludes external tool variables (`NODE_ENV`, `DOCKER_HOST`, etc.)
@@ -61,7 +70,7 @@ python scripts/focused_dotenv_linter.py --check
### **Output Example**
```
```text
🔍 Focused Dotenv Linter for AITBC
==================================================
📄 Found 111 variables in .env.example
@@ -140,28 +149,37 @@ Created `.github/workflows/dotenv-check.yml` with:
### **Workflow Triggers**
The dotenv check runs on:
- **Push** to any branch (when relevant files change)
- **Pull Request** (when relevant files change)
- **File patterns**: `.env.example`, `*.py`, `*.yml`, `*.toml`, `*.sh`
## 📊 Benefits Achieved
### ✅ **Prevents Silent Drift**
- **Automated Detection**: Catches drift as soon as it's introduced
- **CI/CD Integration**: Prevents merging with configuration issues
- **Developer Feedback**: Clear reports on what's missing/unused
### ✅ **Maintains Documentation**
- **Always Up-to-Date**: `.env.example` reflects actual usage
- **Comprehensive Coverage**: All environment variables documented
- **Clear Organization**: Logical grouping and naming
### ✅ **Improves Developer Experience**
- **Easy Discovery**: Developers can see all required variables
- **Auto-Fix**: One-command fix for missing variables
- **Validation**: Format and security checks
### ✅ **Enhanced Security**
- **No Secrets**: Ensures `.env.example` contains only placeholders
- **Security Scanning**: Detects potential actual secrets
- **Best Practices**: Enforces good naming conventions
@@ -210,7 +228,8 @@ r'([A-Z_][A-Z0-9_]*)='
```bash
# Checks for actual secrets vs placeholders
if grep -i "password=" .env.example | grep -v -E "(your-|placeholder|change-)"; then
if grep -i "password=" .env.example \
| grep -v -E "(your-|placeholder|change-)"; then
echo "❌ Potential actual secrets found!"
exit 1
fi
@@ -219,13 +238,16 @@ fi
## 📈 Statistics
### **Current State**
- **Variables in .env.example**: 111
- **Actual variables used**: 124
- **Missing variables**: 13 (auto-fixed)
- **Unused variables**: 0
- **Coverage**: 89.5%
### **Historical Tracking**
- **Before linter**: 14 variables, 357 missing
- **After linter**: 111 variables, 13 missing
- **Improvement**: 693% increase in coverage
@@ -233,12 +255,15 @@ fi
## 🔮 Future Enhancements
### **Planned Features**
- **Environment-specific configs**: `.env.development`, `.env.production`
- **Type validation**: Validate variable value formats
- **Dependency tracking**: Track which variables are required together
- **Documentation generation**: Auto-generate config documentation
### **Advanced Validation**
- **URL validation**: Ensure RPC URLs are properly formatted
- **File path validation**: Check if referenced paths exist
- **Value ranges**: Validate numeric variables have reasonable ranges
@@ -277,7 +302,9 @@ The dotenv configuration discipline ensures:
**Security**: Ensures no actual secrets in documentation
**Maintainability**: Clean, organized, and up-to-date configuration
This discipline prevents the common problem of configuration drift and ensures that `.env.example` always accurately reflects what the application actually needs.
This discipline prevents the common problem of configuration drift and ensures
that `.env.example` always accurately reflects what the application actually
needs.
---