quality-monitor

Skill from fstr21/sportsbot
Category: discovered
Tags: github, community
Version: 1.0.0
Author: walid
Downloads: 0
Updated: 2025-11-22
Source: fstr21/sportsbot
Download the latest version (v1.0.0) from this page.
SKILL.md

# Quality Monitor Skill

**Purpose**: Monitor data quality by analyzing validation logs and generating reports.

## When Claude Should Use This Skill

Use this skill when:
- User asks "how's the data quality?" or "are there any errors?"
- User asks to "check for validation issues" or "check quality log"
- After running a pipeline, to verify data quality
- Working on TODO item 3 (investigate 45 quality log entries)
- User mentions debugging data issues or investigating failures
- User wants to know if today's pipeline run was clean
- Generating daily/weekly quality reports

## What This Skill Does

1. **Queries** the MySQL `data_quality_log` table
2. **Groups** errors by data_type, severity, issue_type
3. **Identifies** patterns in validation failures (recurring issues)
4. **Shows** top failing fields and common problems
5. **Calculates** validation pass rates
6. **Generates** reports in `reports/` directory
7. **Displays** results in easy-to-read table format (ASCII only)

## Critical Constraints

**MUST follow these rules**:
- NO emoji (Windows environment - use ASCII like [OK], [FAIL])
- Use MySQL connection from `.env.local` credentials
- Never clear the `data_quality_log` table (it's append-only by design)
- Display dates in ET timezone (user's timezone)
- Use Rich library for table display (already installed)

## How to Use This Skill

**Example 1**: User says "check data quality"
- Claude runs: `python .claude/skills/quality-monitor/scripts/check_quality.py`
- Claude shows: Summary table of recent errors
- Claude explains: What the errors mean in plain English

**Example 2**: User says "generate quality report"
- Claude runs: `python .claude/skills/quality-monitor/scripts/generate_report.py`
- Claude creates: `reports/quality_2025-10-21.txt`
- Claude summarizes: Key findings from the report

**Example 3**: User says "investigate quality log entries"
- Claude runs: `python .claude/skills/quality-monitor/scripts/analyze_patterns.py`
- Claude shows: Patterns in historical failures
- Claude recommends: What to fix based on patterns

## Expected Outcome

After using this skill:
- User knows if data quality is good or bad
- User knows which data types have the most errors
- User knows if errors are getting better or worse over time
- User has a written report they can reference
- User understands what needs to be fixed (in plain English)

## Technical Details

The quality-monitor scripts:
- `check_quality.py` - Quick health check (last 24 hours)
- `generate_report.py` - Full daily/weekly report
- `analyze_patterns.py` - Deep dive into recurring issues

All scripts:
- Connect to MySQL using `pipelines/shared/db/base_ingestor.py`
- Query `data_quality_log` table
- Use Rich for formatted output
- Save reports to `reports/quality_YYYY-MM-DD.txt`

## Success Criteria

Script is successful if:
- Connects to MySQL database successfully
- Queries data_quality_log without errors
- Displays results in readable table format
- Saves report to reports/ directory (if applicable)
- Explains findings in plain English
- User understands what (if anything) needs attention