Add comprehensive documentation for Concrete ML compatibility issue
- Document Python 3.13 incompatibility with Concrete ML library - Detail current TenSEAL-only implementation and its functionality - Provide multiple resolution options with effort assessments - Include testing procedures and monitoring guidelines - Create clear impact assessment and communication plan - Establish timeline for short, medium, and long-term solutions This documentation provides a complete reference for the Concrete ML compatibility limitation and guides future decision-making for FHE provider architecture in the AITBC platform.
This commit is contained in:
oib
257
docs/12_issues/concrete-ml-compatibility.md
Normal file
257
docs/12_issues/concrete-ml-compatibility.md
Normal file
@@ -0,0 +1,257 @@
|
||||
# Concrete ML Compatibility Issue
|
||||
|
||||
## Issue Summary
|
||||
|
||||
**Status**: ⚠️ **Known Limitation**
|
||||
**Severity**: 🟡 **Medium** (Functional limitation, no security impact)
|
||||
**Date Identified**: March 5, 2026
|
||||
**Last Updated**: March 5, 2026
|
||||
|
||||
## Problem Description
|
||||
|
||||
The AITBC Coordinator API service logs a warning message about Concrete ML not being installed:
|
||||
|
||||
```
|
||||
WARNING:root:Concrete ML not installed; skipping Concrete provider. Concrete ML requires Python <3.13. Current version: 3.13.5
|
||||
```
|
||||
|
||||
### Technical Details
|
||||
|
||||
- **Affected Component**: Coordinator API FHE (Fully Homomorphic Encryption) Service
|
||||
- **Root Cause**: Concrete ML library requires Python <3.13, but AITBC runs on Python 3.13.5
|
||||
- **Impact**: Limited to Concrete ML FHE provider; TenSEAL provider continues to work normally
|
||||
- **Error Type**: Library compatibility issue, not a functional bug
|
||||
|
||||
## Compatibility Matrix
|
||||
|
||||
| Python Version | Concrete ML Support | AITBC Status |
|
||||
|---------------|-------------------|--------------|
|
||||
| 3.8.x - 3.12.x | ✅ Supported | ❌ Not used |
|
||||
| 3.13.x | ❌ Not Supported | ✅ Current version |
|
||||
| 3.14+ | ❌ Unknown | ❌ Future consideration |
|
||||
|
||||
## Current Implementation
|
||||
|
||||
### FHE Provider Architecture
|
||||
|
||||
The AITBC FHE service supports multiple providers:
|
||||
|
||||
1. **TenSEAL Provider** (Primary)
|
||||
- ✅ **Fully Functional**
|
||||
- Supports BFV and CKKS schemes
|
||||
- Active and maintained
|
||||
- Compatible with Python 3.13
|
||||
|
||||
2. **Concrete ML Provider** (Optional)
|
||||
- ❌ **Unavailable** due to Python version incompatibility
|
||||
- Supports neural network compilation
|
||||
- Requires Python <3.13
|
||||
- Currently disabled gracefully
|
||||
|
||||
### Code Implementation
|
||||
|
||||
```python
|
||||
class FHEService:
|
||||
def __init__(self):
|
||||
providers = {"tenseal": TenSEALProvider()}
|
||||
|
||||
# Optional Concrete ML provider
|
||||
try:
|
||||
providers["concrete"] = ConcreteMLProvider()
|
||||
except ImportError as e:
|
||||
logging.warning("Concrete ML not installed; skipping Concrete provider. "
|
||||
"Concrete ML requires Python <3.13. Current version: %s",
|
||||
__import__('sys').version.split()[0])
|
||||
|
||||
self.providers = providers
|
||||
self.default_provider = "tenseal"
|
||||
```
|
||||
|
||||
## Impact Assessment
|
||||
|
||||
### Functional Impact
|
||||
|
||||
- **FHE Operations**: ✅ **No Impact** - TenSEAL provides full FHE functionality
|
||||
- **API Endpoints**: ✅ **No Impact** - All FHE endpoints work normally
|
||||
- **Performance**: ✅ **No Impact** - TenSEAL performance is excellent
|
||||
- **Security**: ✅ **No Impact** - Encryption schemes remain secure
|
||||
|
||||
### Feature Limitations
|
||||
|
||||
- **Neural Network Compilation**: ❌ **Unavailable** - Concrete ML specific feature
|
||||
- **Advanced ML Models**: ⚠️ **Limited** - Some complex models may require Concrete ML
|
||||
- **Research Features**: ❌ **Unavailable** - Experimental Concrete ML features
|
||||
|
||||
## Resolution Options
|
||||
|
||||
### Option 1: Current Status (Recommended)
|
||||
|
||||
**Approach**: Continue with TenSEAL-only implementation
|
||||
|
||||
**Pros**:
|
||||
- ✅ No breaking changes
|
||||
- ✅ Stable and tested
|
||||
- ✅ Python 3.13 compatible
|
||||
- ✅ Full FHE functionality
|
||||
|
||||
**Cons**:
|
||||
- ❌ Limited to TenSEAL features
|
||||
- ❌ No Concrete ML advanced features
|
||||
|
||||
**Implementation**: Already in place
|
||||
|
||||
### Option 2: Python Version Downgrade
|
||||
|
||||
**Approach**: Downgrade to Python 3.12 for Concrete ML support
|
||||
|
||||
**Pros**:
|
||||
- ✅ Full Concrete ML support
|
||||
- ✅ All FHE providers available
|
||||
|
||||
**Cons**:
|
||||
- ❌ Major infrastructure change
|
||||
- ❌ Python 3.13 features lost
|
||||
- ❌ Potential compatibility issues
|
||||
- ❌ Requires extensive testing
|
||||
|
||||
**Effort**: High (2-3 weeks)
|
||||
|
||||
### Option 3: Dual Python Environment
|
||||
|
||||
**Approach**: Maintain separate Python 3.12 environment for Concrete ML
|
||||
|
||||
**Pros**:
|
||||
- ✅ Best of both worlds
|
||||
- ✅ No main environment changes
|
||||
|
||||
**Cons**:
|
||||
- ❌ Complex deployment
|
||||
- ❌ Resource overhead
|
||||
- ❌ Maintenance complexity
|
||||
|
||||
**Effort**: Medium (1-2 weeks)
|
||||
|
||||
### Option 4: Wait for Concrete ML Python 3.13 Support
|
||||
|
||||
**Approach**: Monitor Concrete ML for Python 3.13 compatibility
|
||||
|
||||
**Pros**:
|
||||
- ✅ No immediate work required
|
||||
- ✅ Future-proof solution
|
||||
|
||||
**Cons**:
|
||||
- ❌ Timeline uncertain
|
||||
- ❌ No concrete ML features now
|
||||
|
||||
**Effort**: Minimal (monitoring)
|
||||
|
||||
## Recommended Solution
|
||||
|
||||
### Short Term (Current)
|
||||
|
||||
Continue with **Option 1** - TenSEAL-only implementation:
|
||||
|
||||
1. ✅ **Maintain current architecture**
|
||||
2. ✅ **Document limitation clearly**
|
||||
3. ✅ **Monitor Concrete ML updates**
|
||||
4. ✅ **Focus on TenSEAL optimization**
|
||||
|
||||
### Medium Term (6-12 months)
|
||||
|
||||
Evaluate **Option 4** - Wait for Concrete ML support:
|
||||
|
||||
1. 🔄 **Monitor Concrete ML releases**
|
||||
2. 🔄 **Test Python 3.13 compatibility when available**
|
||||
3. 🔄 **Plan integration if support added**
|
||||
|
||||
### Long Term (12+ months)
|
||||
|
||||
Consider **Option 3** if Concrete ML support remains unavailable:
|
||||
|
||||
1. 📋 **Evaluate business need for Concrete ML**
|
||||
2. 📋 **Implement dual environment if required**
|
||||
3. 📋 **Optimize for specific use cases**
|
||||
|
||||
## Testing and Validation
|
||||
|
||||
### Current Tests
|
||||
|
||||
```bash
|
||||
# Verify FHE service functionality
|
||||
curl -s http://localhost:8000/health
|
||||
# Expected: {"status":"ok","env":"dev","python_version":"3.13.5"}
|
||||
|
||||
# Test FHE provider availability
|
||||
python3 -c "
|
||||
from app.services.fhe_service import FHEService
|
||||
fhe_service = FHEService()
|
||||
print('Available providers:', list(fhe_service.providers.keys()))
|
||||
"
|
||||
# Expected: WARNING:root:Concrete ML not installed; skipping Concrete provider. Concrete ML requires Python <3.13. Current version: 3.13.5
|
||||
# Available providers: ['tenseal']
|
||||
```
|
||||
|
||||
### Validation Checklist
|
||||
|
||||
- [x] Coordinator API starts successfully
|
||||
- [x] FHE service initializes with TenSEAL
|
||||
- [x] API endpoints respond normally
|
||||
- [x] Warning message is informative
|
||||
- [x] No functional degradation
|
||||
- [x] Documentation updated
|
||||
|
||||
## Monitoring
|
||||
|
||||
### Key Metrics
|
||||
|
||||
- **Service Uptime**: Should remain 99.9%+
|
||||
- **API Response Time**: Should remain <200ms
|
||||
- **FHE Operations**: Should continue working normally
|
||||
- **Error Rate**: Should remain <0.1%
|
||||
|
||||
### Alerting
|
||||
|
||||
- **Service Down**: Immediate alert
|
||||
- **FHE Failures**: Warning alert
|
||||
- **Performance Degradation**: Warning alert
|
||||
|
||||
## Communication
|
||||
|
||||
### Internal Teams
|
||||
|
||||
- **Development**: Aware of limitation
|
||||
- **Operations**: Monitoring for issues
|
||||
- **Security**: No impact assessment
|
||||
|
||||
### External Communication
|
||||
|
||||
- **Users**: No impact on functionality
|
||||
- **Documentation**: Clear limitation notes
|
||||
- **Support**: Prepared for inquiries
|
||||
|
||||
## Related Issues
|
||||
|
||||
- [AITBC-001] Python 3.13 migration planning
|
||||
- [AITBC-002] FHE provider architecture review
|
||||
- [AITBC-003] Library compatibility matrix
|
||||
|
||||
## References
|
||||
|
||||
- [Concrete ML GitHub](https://github.com/zama-ai/concrete-ml)
|
||||
- [Concrete ML Documentation](https://docs.zama.ai/concrete-ml/)
|
||||
- [TenSEAL Documentation](https://github.com/OpenMined/TenSEAL)
|
||||
- [Python 3.13 Release Notes](https://docs.python.org/3.13/whatsnew.html)
|
||||
|
||||
## Change Log
|
||||
|
||||
| Date | Change | Author |
|
||||
|------|--------|--------|
|
||||
| 2026-03-05 | Initial issue documentation | Cascade |
|
||||
| 2026-03-05 | Added resolution options and testing | Cascade |
|
||||
|
||||
---
|
||||
|
||||
**Document Status**: 🟡 **Active Monitoring**
|
||||
**Next Review**: 2026-06-05
|
||||
**Owner**: AITBC Development Team
|
||||
**Contact**: dev@aitbc.dev
|
||||
Reference in New Issue
Block a user