ml-ai-development
Interactive browser-based AppSec training game where players identify vulnerabilities in code and learn secure coding practices
- Category: discovered
- Tags: github, community
- Version: 1.0.0
- Author: walid
- Downloads: 0
- GitHub stars: 2
- Updated: 2025-11-21
Download the latest version (v1.0.0) from this page.
SKILL.md
# ML/AI Development Skill ## Activation Context Activates when working on machine learning models, AI features, data pipelines, or ML infrastructure. ## When to Invoke ml-ai-engineer Agent - Designing ML/AI features - Training machine learning models - Building data pipelines - Implementing AI/ML APIs - Model deployment and serving - Model monitoring and optimization - Feature engineering - Data preprocessing ## Responsibilities ### Model Development - Model selection and experimentation - Feature engineering - Training and validation - Hyperparameter tuning - Model evaluation - Model versioning ### Data Pipeline - Data collection and ingestion - Data cleaning and preprocessing - Feature extraction - Data validation - ETL processes - Data versioning ### ML Infrastructure - Training infrastructure - Model serving/deployment - Model monitoring - A/B testing infrastructure - Model registry - Experiment tracking ## Coordination Protocol ### Before Starting ML Work 1. **From product-manager**: Business requirements and success metrics 2. **From architect**: System architecture and integration points 3. **From backend-engineer**: API requirements and data access 4. **From database-engineer**: Data availability and schema 5. **From devsecopsengineer**: Infrastructure and deployment requirements ### ML Development Workflow 1. **Problem Definition**: Understand business problem 2. **Data Exploration**: Analyze available data 3. **Feature Engineering**: Create relevant features 4. **Model Selection**: Choose appropriate algorithms 5. **Training**: Train and validate models 6. **Evaluation**: Measure performance against metrics 7. **Deployment**: Deploy to production 8. **Monitoring**: Monitor performance and drift ### Handoff Points - **From database-engineer**: Data access and schemas - **To backend-engineer**: Model API specifications - **To qa-engineer**: Model performance testing requirements - **To devsecopsengineer**: Model deployment and serving infrastructure - **To documentation-engineer**: Model documentation and usage guides ## ML Project Structure ``` ml/ ├── data/ │ ├── raw/ # Original data │ ├── processed/ # Cleaned data │ └── features/ # Engineered features ├── notebooks/ # Jupyter notebooks for exploration ├── models/ # Trained models ├── src/ │ ├── data/ # Data processing │ ├── features/ # Feature engineering │ ├── models/ # Model definitions │ ├── training/ # Training scripts │ └── serving/ # Model serving code ├── tests/ # Tests for ML code └── experiments/ # Experiment tracking ``` ## ML Development Best Practices ### Data Management - Version control your datasets - Document data sources and transformations - Validate data quality - Handle missing data appropriately - Split data properly (train/val/test) - Prevent data leakage ### Experimentation - Track all experiments (MLflow, Weights & Biases) - Document hyperparameters - Use reproducible random seeds - Version control code and configs - Save model artifacts ### Model Evaluation - Define clear success metrics - Use appropriate evaluation metrics - Test on held-out test set - Check for bias and fairness - Validate on edge cases - Compare to baseline models ### Model Training - Use cross-validation - Monitor for overfitting - Implement early stopping - Save checkpoints - Log training metrics - Document training process ## Production ML Checklist ### Pre-Deployment - [ ] Model performance meets requirements - [ ] Model evaluated on diverse data - [ ] Bias and fairness checked - [ ] Model size and latency acceptable - [ ] Monitoring strategy defined - [ ] Rollback plan prepared - [ ] A/B testing plan ready ### Deployment - [ ] Model versioned and registered - [ ] Serving infrastructure ready (with devsecopsengineer) - [ ] API endpoints documented (with documentation-engineer) - [ ] Monitoring and alerting configured - [ ] Performance benchmarks met - [ ] Error handling implemented - [ ] Logging configured ### Post-Deployment - [ ] Monitor model performance - [ ] Track prediction latency - [ ] Monitor data drift - [ ] Monitor model drift - [ ] Collect feedback - [ ] Plan retraining schedule ## Model Serving Patterns ### Batch Prediction - Process data in batches - Store predictions in database - Lower latency requirements - Good for analytics ### Real-time Prediction - Low latency requirements (< 100ms) - Scalable serving infrastructure - Caching for common inputs - Good for user-facing features ### Streaming Prediction - Process streaming data - Near real-time predictions - Event-driven architecture - Good for continuous data flows ## Data Pipeline Best Practices ### Data Validation - Validate schema - Check data types - Validate ranges and distributions - Check for missing values - Detect anomalies ### Data Processing - Make pipelines reproducible - Document transformations - Test pipeline components - Handle errors gracefully - Log processing statistics ### Data Quality - Monitor data quality metrics - Alert on quality issues - Implement data checks - Document data lineage ## Model Monitoring ### Metrics to Monitor - **Performance Metrics**: Accuracy, precision, recall, F1, etc. - **Business Metrics**: Conversion rate, revenue impact, etc. - **System Metrics**: Latency, throughput, error rate - **Data Metrics**: Input distribution, missing values ### Drift Detection - **Data Drift**: Input distribution changes - **Concept Drift**: Relationship between features and target changes - **Prediction Drift**: Output distribution changes ### Alerts - Performance degradation - High error rates - Latency issues - Data quality issues - Drift detection ## ML Security Considerations (with security-engineer) ### Model Security - Protect model artifacts - Validate all inputs - Prevent model extraction - Monitor for adversarial attacks - Secure model serving endpoints ### Data Privacy - Comply with data regulations (GDPR, etc.) - Anonymize sensitive data - Implement access controls - Audit data access - Handle PII appropriately ### Responsible AI - Test for bias and fairness - Ensure transparency - Implement explainability - Document limitations - Monitor for unintended consequences ## Testing ML Systems ### Unit Tests - Test data processing functions - Test feature engineering - Test preprocessing pipelines - Test utility functions ### Integration Tests - Test end-to-end pipelines - Test model serving APIs - Test data ingestion ### Model Tests - Test on known inputs/outputs - Test edge cases - Test performance thresholds - Test for bias - Test prediction consistency ## Documentation Requirements (with documentation-engineer) ### Model Documentation - Model architecture - Training data description - Feature descriptions - Performance metrics - Limitations and assumptions - Use cases and intended use ### API Documentation - Endpoint specifications - Input/output formats - Example requests/responses - Error handling - Rate limits - Authentication ## Tools and Frameworks ### ML Frameworks - **PyTorch/TensorFlow**: Deep learning - **Scikit-learn**: Traditional ML - **XGBoost/LightGBM**: Gradient boosting - **Hugging Face**: NLP models ### MLOps Tools - **MLflow**: Experiment tracking and model registry - **Weights & Biases**: Experiment tracking - **DVC**: Data version control - **Kubeflow/MLflow**: Model deployment ### Data Tools - **Pandas/Polars**: Data manipulation - **Spark**: Big data processing - **Airflow/Dagster**: Workflow orchestration ## Best Practices - Start simple, iterate to complex - Baseline models first - Version everything (data, code, models) - Automate retraining pipelines - Monitor continuously - Document assumptions and limitations - Test thoroughly before production - Plan for model degradation