Add comprehensive documentation for Concrete ML compatibility issue

- Document Python 3.13 incompatibility with Concrete ML library
- Detail current TenSEAL-only implementation and its functionality
- Provide multiple resolution options with effort assessments
- Include testing procedures and monitoring guidelines
- Create clear impact assessment and communication plan
- Establish timeline for short, medium, and long-term solutions

This documentation provides a complete reference for the Concrete ML
compatibility limitation and guides future decision-making for FHE
provider architecture in the AITBC platform.

docs/12_issues/concrete-ml-compatibility.md

@@ -0,0 +1,257 @@
+# Concrete ML Compatibility Issue
+
+## Issue Summary
+
+**Status**: ⚠️ **Known Limitation**  
+**Severity**: 🟡 **Medium** (Functional limitation, no security impact)  
+**Date Identified**: March 5, 2026  
+**Last Updated**: March 5, 2026  
+
+## Problem Description
+
+The AITBC Coordinator API service logs a warning message about Concrete ML not being installed:
+
+```
+WARNING:root:Concrete ML not installed; skipping Concrete provider. Concrete ML requires Python <3.13. Current version: 3.13.5
+```
+
+### Technical Details
+
+- **Affected Component**: Coordinator API FHE (Fully Homomorphic Encryption) Service
+- **Root Cause**: Concrete ML library requires Python <3.13, but AITBC runs on Python 3.13.5
+- **Impact**: Limited to Concrete ML FHE provider; TenSEAL provider continues to work normally
+- **Error Type**: Library compatibility issue, not a functional bug
+
+## Compatibility Matrix
+
+| Python Version | Concrete ML Support | AITBC Status |
+|---------------|-------------------|--------------|
+| 3.8.x - 3.12.x | ✅ Supported | ❌ Not used |
+| 3.13.x | ❌ Not Supported | ✅ Current version |
+| 3.14+ | ❌ Unknown | ❌ Future consideration |
+
+## Current Implementation
+
+### FHE Provider Architecture
+
+The AITBC FHE service supports multiple providers:
+
+1. **TenSEAL Provider** (Primary)
+   - ✅ **Fully Functional**
+   - Supports BFV and CKKS schemes
+   - Active and maintained
+   - Compatible with Python 3.13
+
+2. **Concrete ML Provider** (Optional)
+   - ❌ **Unavailable** due to Python version incompatibility
+   - Supports neural network compilation
+   - Requires Python <3.13
+   - Currently disabled gracefully
+
+### Code Implementation
+
+```python
+class FHEService:
+    def __init__(self):
+        providers = {"tenseal": TenSEALProvider()}
+        
+        # Optional Concrete ML provider
+        try:
+            providers["concrete"] = ConcreteMLProvider()
+        except ImportError as e:
+            logging.warning("Concrete ML not installed; skipping Concrete provider. "
+                          "Concrete ML requires Python <3.13. Current version: %s", 
+                          __import__('sys').version.split()[0])
+        
+        self.providers = providers
+        self.default_provider = "tenseal"
+```
+
+## Impact Assessment
+
+### Functional Impact
+
+- **FHE Operations**: ✅ **No Impact** - TenSEAL provides full FHE functionality
+- **API Endpoints**: ✅ **No Impact** - All FHE endpoints work normally
+- **Performance**: ✅ **No Impact** - TenSEAL performance is excellent
+- **Security**: ✅ **No Impact** - Encryption schemes remain secure
+
+### Feature Limitations
+
+- **Neural Network Compilation**: ❌ **Unavailable** - Concrete ML specific feature
+- **Advanced ML Models**: ⚠️ **Limited** - Some complex models may require Concrete ML
+- **Research Features**: ❌ **Unavailable** - Experimental Concrete ML features
+
+## Resolution Options
+
+### Option 1: Current Status (Recommended)
+
+**Approach**: Continue with TenSEAL-only implementation
+
+**Pros**:
+- ✅ No breaking changes
+- ✅ Stable and tested
+- ✅ Python 3.13 compatible
+- ✅ Full FHE functionality
+
+**Cons**:
+- ❌ Limited to TenSEAL features
+- ❌ No Concrete ML advanced features
+
+**Implementation**: Already in place
+
+### Option 2: Python Version Downgrade
+
+**Approach**: Downgrade to Python 3.12 for Concrete ML support
+
+**Pros**:
+- ✅ Full Concrete ML support
+- ✅ All FHE providers available
+
+**Cons**:
+- ❌ Major infrastructure change
+- ❌ Python 3.13 features lost
+- ❌ Potential compatibility issues
+- ❌ Requires extensive testing
+
+**Effort**: High (2-3 weeks)
+
+### Option 3: Dual Python Environment
+
+**Approach**: Maintain separate Python 3.12 environment for Concrete ML
+
+**Pros**:
+- ✅ Best of both worlds
+- ✅ No main environment changes
+
+**Cons**:
+- ❌ Complex deployment
+- ❌ Resource overhead
+- ❌ Maintenance complexity
+
+**Effort**: Medium (1-2 weeks)
+
+### Option 4: Wait for Concrete ML Python 3.13 Support
+
+**Approach**: Monitor Concrete ML for Python 3.13 compatibility
+
+**Pros**:
+- ✅ No immediate work required
+- ✅ Future-proof solution
+
+**Cons**:
+- ❌ Timeline uncertain
+- ❌ No concrete ML features now
+
+**Effort**: Minimal (monitoring)
+
+## Recommended Solution
+
+### Short Term (Current)
+
+Continue with **Option 1** - TenSEAL-only implementation:
+
+1. ✅ **Maintain current architecture**
+2. ✅ **Document limitation clearly**
+3. ✅ **Monitor Concrete ML updates**
+4. ✅ **Focus on TenSEAL optimization**
+
+### Medium Term (6-12 months)
+
+Evaluate **Option 4** - Wait for Concrete ML support:
+
+1. 🔄 **Monitor Concrete ML releases**
+2. 🔄 **Test Python 3.13 compatibility when available**
+3. 🔄 **Plan integration if support added**
+
+### Long Term (12+ months)
+
+Consider **Option 3** if Concrete ML support remains unavailable:
+
+1. 📋 **Evaluate business need for Concrete ML**
+2. 📋 **Implement dual environment if required**
+3. 📋 **Optimize for specific use cases**
+
+## Testing and Validation
+
+### Current Tests
+
+```bash
+# Verify FHE service functionality
+curl -s http://localhost:8000/health
+# Expected: {"status":"ok","env":"dev","python_version":"3.13.5"}
+
+# Test FHE provider availability
+python3 -c "
+from app.services.fhe_service import FHEService
+fhe_service = FHEService()
+print('Available providers:', list(fhe_service.providers.keys()))
+"
+# Expected: WARNING:root:Concrete ML not installed; skipping Concrete provider. Concrete ML requires Python <3.13. Current version: 3.13.5
+#          Available providers: ['tenseal']
+```
+
+### Validation Checklist
+
+- [x] Coordinator API starts successfully
+- [x] FHE service initializes with TenSEAL
+- [x] API endpoints respond normally
+- [x] Warning message is informative
+- [x] No functional degradation
+- [x] Documentation updated
+
+## Monitoring
+
+### Key Metrics
+
+- **Service Uptime**: Should remain 99.9%+
+- **API Response Time**: Should remain <200ms
+- **FHE Operations**: Should continue working normally
+- **Error Rate**: Should remain <0.1%
+
+### Alerting
+
+- **Service Down**: Immediate alert
+- **FHE Failures**: Warning alert
+- **Performance Degradation**: Warning alert
+
+## Communication
+
+### Internal Teams
+
+- **Development**: Aware of limitation
+- **Operations**: Monitoring for issues
+- **Security**: No impact assessment
+
+### External Communication
+
+- **Users**: No impact on functionality
+- **Documentation**: Clear limitation notes
+- **Support**: Prepared for inquiries
+
+## Related Issues
+
+- [AITBC-001] Python 3.13 migration planning
+- [AITBC-002] FHE provider architecture review
+- [AITBC-003] Library compatibility matrix
+
+## References
+
+- [Concrete ML GitHub](https://github.com/zama-ai/concrete-ml)
+- [Concrete ML Documentation](https://docs.zama.ai/concrete-ml/)
+- [TenSEAL Documentation](https://github.com/OpenMined/TenSEAL)
+- [Python 3.13 Release Notes](https://docs.python.org/3.13/whatsnew.html)
+
+## Change Log
+
+| Date | Change | Author |
+|------|--------|--------|
+| 2026-03-05 | Initial issue documentation | Cascade |
+| 2026-03-05 | Added resolution options and testing | Cascade |
+
+---
+
+**Document Status**: 🟡 **Active Monitoring**  
+**Next Review**: 2026-06-05  
+**Owner**: AITBC Development Team  
+**Contact**: dev@aitbc.dev