learn-from-mistake
Investigate agent mistakes, perform root cause analysis, and update configurations to prevent recurrence
- Category: discovered
- Tags: github, community
- Version: 1.0.0
- Author: walid
- Downloads: 0
- GitHub stars: 2
- Updated: 2025-11-21
Download the latest version (v1.0.0) from this page.
SKILL.md
---
name: learn-from-mistake
description: Investigate agent mistakes, perform root cause analysis, and update configurations to prevent recurrence
allowed-tools: Read, Write, Edit, Bash, Grep, Glob
---
# Learn From Mistake Skill
**Purpose**: Investigate agent mistakes, perform root cause analysis, update agent configurations, and TEST fixes by reproducing the original mistake to verify prevention.
**When to Use**:
- After an agent makes a mistake that causes delays, rework, or violations
- When a pattern of similar mistakes is observed across tasks
- To proactively improve agent behavior based on lessons learned
- When protocol violations occur that should have been caught
## Skill Workflow
8-phase process:
1. **Mistake Identification** - Gather context
2. **Conversation Analysis** - Review logs
3. **Root Cause Analysis** - Categorize and investigate cause
4. **Configuration Updates** - Design prevention measures
5. **Implement Updates** - Apply changes
6. **Validation** - Review completeness
7. **Test by Reproduction** - ⚠️ Attempt to reproduce mistake to verify prevention
8. **Documentation** - Add inline comments and commit messages
**Critical**: Phase 7 requires actually reproducing the mistake (not mental simulation) to verify fixes work.
### Phase 1: Mistake Identification
**Input Requirements**:
- Agent name that made the mistake (e.g., `architect`, `style`, `main`)
- Task name if applicable (e.g., `implement-formatter-api`)
- Description of what went wrong
- Conversation ID or timestamp range (optional - will search recent if not provided)
**Questions to Ask User**:
1. Which agent made the mistake?
2. What task were they working on?
3. What specifically went wrong?
4. Do you have the conversation ID? (Can skip - will search recent)
### Phase 2: Conversation Analysis
**Access Conversation Logs**:
First, get the session ID from the SessionStart system reminder (provided by `get-session-id` skill):
```
✅ Session ID: 88194cb6-734b-498c-ab5d-ac7c773d8b34
```
Then access conversation logs:
```bash
# REQUIRED: Use session ID from system reminder
SESSION_ID="88194cb6-734b-498c-ab5d-ac7c773d8b34"
# Verify session ID is provided
if [ -z "$SESSION_ID" ]; then
echo "ERROR: Session ID not available in context." >&2
echo "Expected system reminder: '✅ Session ID: {uuid}'" >&2
echo "Provided by get-session-id skill at SessionStart." >&2
exit 1
fi
# Main session conversation
MAIN_CONVERSATION="/home/node/.config/projects/-workspace/${SESSION_ID}.jsonl"
# Verify conversation file exists
if [ ! -f "$MAIN_CONVERSATION" ]; then
echo "ERROR: Conversation file not found: $MAIN_CONVERSATION" >&2
exit 1
fi
# Agent sidechain conversations (if investigating agent-specific mistakes)
AGENT_CONVERSATIONS=$(ls -t /home/node/.config/projects/-workspace/agent-*.jsonl 2>/dev/null | head -10)
# Read and parse conversation for agent mentions
jq -r "select(.message.content | tostring | contains(\"$AGENT_NAME\"))" "$MAIN_CONVERSATION"
# For agent sidechain logs
for agent_log in $AGENT_CONVERSATIONS; do
grep -l "$AGENT_NAME" "$agent_log" 2>/dev/null && echo "Found in: $agent_log"
done
```
**What to Extract**:
1. **Agent Invocation Context**:
- What prompt was given to the agent
- What files/context were available
- What state the task was in
- Working directory specified
2. **Agent Actions**:
- Sequence of tool calls made
- Files read, written, edited
- Commands executed
- Directories changed to
3. **Mistake Manifestation**:
- Error messages
- Incorrect output
- Protocol violations
- Build failures
- Rework needed
4. **Recovery Actions**:
- What was done to fix the mistake
- How much rework was required
- What additional context was needed
### Phase 3: Root Cause Analysis
**⚠️ MANDATORY STEP 1: Check if Fix Already Exists (Temporal Analysis)**
**CRITICAL**: Before creating new fixes, determine if this is a NEW mistake or an INEFFECTIVE EXISTING FIX.
**Temporal Analysis Procedure**:
1. **Identify the mistake pattern** (e.g., "timing boundary misclassification", "worktree violation")
2. **Search git history for related fixes**:
```bash
# Search commit messages for related keywords
git log --all --format="%H %ai %s" --grep="timing.*boundary\|Error 8\|misclassification" | head -20
```
3. **Get violation timestamp from session**:
```bash
SESSION_ID="<from-system-reminder>"
VIOLATION_TIME=$(stat -c %y ~/.config/projects/-workspace/${SESSION_ID}.jsonl | cut -d' ' -f1-2)
echo "Violation occurred: $VIOLATION_TIME"
```
4. **Compare timestamps**:
```bash
# For each potentially related commit
COMMIT_HASH="<from-git-log>"
COMMIT_TIME=$(git show -s --format=%ai $COMMIT_HASH)
echo "Fix implemented: $COMMIT_TIME"
# Temporal relationship
if [[ "$COMMIT_TIME" < "$VIOLATION_TIME" ]]; then
echo "⚠️ FIX PREDATES VIOLATION - Existing fix is INEFFECTIVE"
else
echo "✅ Fix implemented AFTER violation - Reactive fix"
fi
```
**Decision Matrix**:
| Temporal Relationship | Classification | Action Required |
|----------------------|----------------|-----------------|
| Fix exists BEFORE violation | **INEFFECTIVE FIX** | Improve/replace existing fix, NOT create duplicate |
| No related fix exists | **NEW MISTAKE** | Create new prevention measures |
| Fix exists AFTER violation | **REACTIVE FIX** | Verify fix is comprehensive, test it |
**Example Analysis**:
```bash
# Scenario: Timing boundary misclassification in session bc6b74a7
# Step 1: Find related commits
git log --all --format="%H %ai %s" --grep="timing\|Error 8" | head -5
# Output:
# 63f27ed 2025-11-13 07:39:04 -0500 Improve shrink-doc to prevent timing boundary misclassification
# 39f8f39 2025-11-12 21:20:44 -0500 Strengthen shrink-doc to prevent repeated misclassification
# 088e259 2025-11-12 16:39:56 -0500 Learn from shrink-doc mistake: Add Error 8
# Step 2: Get violation time
stat -c %y ~/.config/projects/-workspace/bc6b74a7*.jsonl
# Output: 2025-11-13 03:38:41 (Nov 12 22:38 EST)
# Step 3: Temporal analysis
# 088e259: Nov 12 16:39 (6 hours BEFORE violation) ← INEFFECTIVE
# 39f8f39: Nov 12 21:20 (1 hour BEFORE violation) ← INEFFECTIVE
# 63f27ed: Nov 13 07:39 (9 hours AFTER violation) ← REACTIVE
# Conclusion: Error 8 fix was implemented 6 hours before violation occurred
# The fix is INEFFECTIVE and needs to be improved/replaced
```
**Output Required**:
Document the temporal analysis result:
```
TEMPORAL ANALYSIS RESULT:
- Mistake pattern: <pattern-name>
- Related commits: <commit-hash-list>
- Fix implementation date: <date-time>
- Violation date: <date-time>
- Classification: INEFFECTIVE FIX | NEW MISTAKE | REACTIVE FIX
- Evidence: <git-log-output-showing-timing>
```
If **INEFFECTIVE FIX** detected:
- Read the existing fix to understand what was tried
- Analyze why it didn't work (review conversation logs for agent behavior)
- Design improvement or replacement (not duplicate)
- Document in commit message: "Improve/Replace ineffective fix from commit <hash>"
**⚠️ MANDATORY STEP 2: Identify Triggering Thought Before Creating Fixes**
After temporal analysis, MUST:
1. Access `/home/node/.config/projects/-workspace/{session-id}.jsonl`
2. Find assistant message immediately before mistake
3. Extract exact text/thought before wrong decision
4. Document triggering quote
**Example**:
```bash
# Search for when file was created
grep -i "safety-analysis" conversation.jsonl | jq '.timestamp'
# Returns: 2025-11-02T04:31:37.673Z
# Get messages before that timestamp
jq 'select(.timestamp < "2025-11-02T04:31:37.673Z" and .timestamp > "2025-11-02T04:30:00.000Z")' conversation.jsonl
# Find triggering thought
jq 'select(.timestamp == "2025-11-02T04:31:10.998Z") | .message.content[].text' conversation.jsonl
# Result: "Given the complexity and length of fixing both skills completely,
# let me create a summary document showing the critical fixes needed"
```
Root cause: Agent used "complexity and length" to justify creating summary instead of doing work (Token Usage Policy violation).
**Without triggering thought, prevention will be ineffective.**
**Automated Detection**:
```bash
#!/bin/bash
# Helper: Find triggering thought automatically
SESSION_ID="${1:-$(grep 'Session ID:' /tmp/session_context.txt | cut -d: -f2 | tr -d ' ')}"
MISTAKE_INDICATOR="${2:-}" # e.g., "SAFETY-ANALYSIS.md", "BUILD FAILURE", "PROTOCOL VIOLATION"
if [[ -z "$SESSION_ID" || -z "$MISTAKE_INDICATOR" ]]; then
echo "Usage: $0 <session-id> <mistake-indicator>" >&2
echo "Example: $0 abc123 'SAFETY-ANALYSIS.md'" >&2
exit 1
fi
CONVERSATION="/home/node/.config/projects/-workspace/${SESSION_ID}.jsonl"
if [[ ! -f "$CONVERSATION" ]]; then
echo "ERROR: Conversation file not found: $CONVERSATION" >&2
exit 1
fi
# Find timestamp when mistake occurred
MISTAKE_TIMESTAMP=$(jq -r --arg indicator "$MISTAKE_INDICATOR" \
'select(.message.content | tostring | contains($indicator)) | .timestamp' \
"$CONVERSATION" | head -1)
if [[ -z "$MISTAKE_TIMESTAMP" ]]; then
echo "ERROR: Could not find mistake indicator '$MISTAKE_INDICATOR' in conversation" >&2
exit 1
fi
echo "Mistake occurred at: $MISTAKE_TIMESTAMP"
# Find last assistant message before mistake (within 5 minutes)
FIVE_MINUTES_BEFORE=$(date -d "$MISTAKE_TIMESTAMP - 5 minutes" -Iseconds 2>/dev/null || \
date -v-5M -jf "%Y-%m-%dT%H:%M:%S" "${MISTAKE_TIMESTAMP%+*}" "+%Y-%m-%dT%H:%M:%S" 2>/dev/null)
TRIGGERING_THOUGHT=$(jq -r --arg before "$MISTAKE_TIMESTAMP" --arg after "$FIVE_MINUTES_BEFORE" \
'select(.timestamp < $before and .timestamp > $after and .message.role == "assistant") |
{timestamp, content: .message.content}' \
"$CONVERSATION" | jq -s 'last')
if [[ -z "$TRIGGERING_THOUGHT" || "$TRIGGERING_THOUGHT" == "null" ]]; then
echo "WARNING: Could not find assistant message before mistake" >&2
exit 1
fi
echo ""
echo "TRIGGERING THOUGHT:"
echo "$TRIGGERING_THOUGHT" | jq -r '.content[] | select(.type == "text") | .text' | tail -20
# Also check for thinking blocks (often contain the decision reasoning)
echo ""
echo "ASSOCIATED THINKING (if any):"
echo "$TRIGGERING_THOUGHT" | jq -r '.content[] | select(.type == "thinking") | .thinking' | tail -20
```
**Usage**:
```bash
# Automatic detection
./find-triggering-thought.sh "$SESSION_ID" "PROTOCOL VIOLATION"
# Or inline
SESSION_ID="abc-123"
MISTAKE="wrong worktree"
jq -r --arg indicator "$MISTAKE" \
'select(.message.content | tostring | contains($indicator))' \
/home/node/.config/projects/-workspace/${SESSION_ID}.jsonl | \
jq -r '.timestamp' | head -1
```
**Categorize the Mistake**:
#### A. Missing Information
- Agent prompt lacked necessary context
- Required files not mentioned
- Assumptions not stated explicitly
- Working directory not clear
#### B. Misunderstood Requirements
- Ambiguous protocol documentation
- Conflicting instructions
- Unclear success criteria
- No examples for this scenario
#### C. Tool Usage Errors
- Wrong tool for the job
- Incorrect parameters
- Missing validation steps
- Tool limitations not understood
#### D. Logic Errors
- Incorrect algorithm
- Missing edge cases
- Faulty assumptions
- Incomplete understanding of domain
#### E. Protocol Violations
- Skipped required steps
- Wrong state for action
- Missing validations
- Worktree confusion
#### F. Configuration Gaps
- Agent config missing examples
- No guidance for this scenario
- Anti-patterns not documented
- Missing checklists
**Investigate**:
1. **Available Information**: Agent prompt, files accessed, missing context
2. **Correct Action**: Expected sequence, needed information, missing checks
3. **Wrong Choice Reason**: Documentation gaps, ambiguous instructions, misleading examples, insufficient validation
### Phase 4: Configuration Updates
**⚠️ CRITICAL: Active Enforcement vs Passive Policies**
**When policies exist but are violated, implement ACTIVE ENFORCEMENT (hooks), not just passive documentation.**
**Decision Tree**:
1. **Policy exists AND was violated?**
- ✅ YES → Create enforcement hook (PreToolUse/PostToolUse)
- ❌ NO (new issue) → Document policy first, then add hook if violations likely
2. **Policy enforceability:**
- ✅ Can be checked mechanically → MUST create hook
- ❌ Requires judgment → Document with examples, consider detection hook
**Why Active Enforcement Matters**:
- Passive policies (text in config) rely on agents reading and remembering
- Active hooks (validation scripts) prevent violations automatically
- **If it violated once, it WILL violate again without enforcement**
**Examples**:
✅ **CORRECT - Active Enforcement**:
```
Issue: Agent created files in wrong worktree (policy exists in docs)
Fix: Create pre-write.sh hook that validates worktree before Write tool
Result: Future violations automatically blocked
```
❌ **WRONG - Passive Policy Only**:
```
Issue: Agent created files in wrong worktree (policy exists in docs)
Fix: Add more warnings to documentation, bold the text
Result: Violation will recur because agent may not read/remember
```
**Hook Creation Priority**:
1. **HIGH**: Protocol violations (state machine, worktree, approval gates)
2. **MEDIUM**: Tool misuse (wrong parameters, missing validation)
3. **LOW**: Style preferences (can be post-hoc validated)
**⚠️ SPECIAL CASE: Ineffective Existing Fix**
**When temporal analysis identifies INEFFECTIVE FIX** (fix predates violation):
**DO NOT create duplicate fixes.** Instead:
1. **Analyze why existing fix failed**:
- Read the existing fix implementation (hook, documentation, validation)
- Review session logs to see agent behavior despite fix
- Identify the gap: What did the fix check? What did it miss?
- Example: Error 8 checked for section titles but not numeric boundary content
2. **Root cause categories for fix ineffectiveness**:
- **Incomplete detection**: Fix checks for pattern A but violation uses pattern B
- **Weak enforcement**: Warning message instead of blocking
- **Coverage gap**: Fix only applies to subset of cases
- **Agent workaround**: Agent found way to bypass check
- **Documentation clarity**: Guidance exists but is buried/vague
- **Validation timing**: Check happens too late (after damage done)
3. **Improvement strategies**:
**Strategy A: Strengthen detection logic**
```bash
# Before (INEFFECTIVE - only checked titles):
if [[ "$SECTION_TITLE" =~ "Clarification"|"Why" ]]; then
echo "High risk keyword detected"
fi
# After (EFFECTIVE - checks content for numeric boundaries):
if [[ "$CONTENT" =~ [0-9]+-[0-9]+[[:space:]]*(seconds|retries|MB) ]]; then
echo "⚠️ Numeric boundary detected - VERY LIKELY CONTEXTUAL"
exit 2 # Block if classified as EXPLANATORY
fi
```
**Strategy B: Add validation checkpoint**
```bash
# Add validation BEFORE removal (not just during classification)
# Check if removed content contains execution-critical patterns
```
**Strategy C: Multi-layered defense**
```bash
# Layer 1: Red flag keywords in classification
# Layer 2: Content pattern analysis (numeric boundaries)
# Layer 3: Validation checkpoint before removal
# Layer 4: Post-removal semantic integrity check
```
4. **Commit message pattern**:
```
Improve ineffective fix from commit <hash>: <pattern>
TEMPORAL ANALYSIS:
- Original fix: <commit-hash> on <date>
- Violation: session <session-id> on <date>
- Time delta: <fix-predates-violation-by-X-hours>
ROOT CAUSE OF FIX INEFFECTIVENESS:
- Original fix checked <what-it-checked>
- But violation occurred via <what-it-missed>
- Gap: <specific-gap-in-detection-logic>
IMPROVEMENT:
- <what-new-fix-does-differently>
- <why-this-should-work>
EVIDENCE FROM SESSION:
- <excerpt-showing-agent-behavior-despite-fix>
```
5. **Test the improvement**:
- Review session logs showing original violation
- Verify new fix would have caught it
- If possible, reproduce scenario with new fix active
**Update Strategy by Category**:
#### A. Missing Information → Update Agent Prompt Template
**File**: `.claude/agents/{agent-name}.md`
**Add to "Required Context" or "Before You Begin"**:
- Checklist to verify information
- Required file reads
- Questions for unclear context
- Working directory verification
```markdown
## Before You Begin - MANDATORY CHECKS
Before ANY implementation work:
- [ ] Verify working directory: `pwd`
- [ ] Confirm you're in: `/workspace/tasks/{task}/agents/{agent-name}/code/`
- [ ] Read task requirements: `/workspace/tasks/{task}/task.md`
- [ ] Check task state: `cat /workspace/tasks/{task}/task.json`
- [ ] Verify dependencies available
If ANY check fails: STOP and report the issue.
```
#### B. Misunderstood Requirements → Clarify Documentation
**File**: `docs/project/{relevant-protocol}.md`
**Add**:
- Explicit requirements
- ✅/❌ examples
- Edge cases
- Anti-pattern warnings
- Decision trees
```markdown
## Working Directory Requirements
⚠️ **CRITICAL**: Sub-agents MUST work in their agent worktree
**Your worktree**: `/workspace/tasks/{task}/agents/{agent-name}/code/`
**NOT the task worktree**: `/workspace/tasks/{task}/code/` ← WRONG
✅ **CORRECT**:
\```bash
cd /workspace/tasks/implement-api/agents/architect/code
Write: src/main/java/MyClass.java
\```
❌ **WRONG** (Protocol Violation):
\```bash
cd /workspace/tasks/implement-api/code
Write: src/main/java/MyClass.java # Creates in task worktree!
\```
```
#### C. Tool Usage Errors → Add Tool Guidance
**File**: `.claude/agents/{agent-name}.md`
**Add "Tool Usage Patterns"**:
- When to use each tool
- Common pitfalls
- Validation steps
- Example sequences
- Error recovery
```markdown
## Tool Usage Patterns
### Write Tool
**When**: Creating new files
**Before**: Verify directory with `pwd`
**After**: Verify file created with `ls -la {file}`
**Validation**: File must be in agent worktree
### Edit Tool
**When**: Modifying existing files
**Before**: Read file first to see exact content
**After**: Read file again to verify changes
**Pitfall**: Don't include line numbers in old_string
```
#### D. Logic Errors → Add Examples and Tests
**File**: `.claude/agents/{agent-name}.md`
**Add**:
- Worked examples
- Edge cases
- Validation checklist
- Self-test questions
- Common pitfalls
```markdown
## Common Scenarios
### Scenario: Creating New Maven Module
**Context**: Task requires new Maven module
**Steps**:
1. Create directory: `mkdir -p {module}/src/main/java`
2. Create POM: Use template from existing module
3. Add to parent POM: `<module>{module}</module>`
4. Create module-info.java: Define exports
5. Verify: `mvn compile -pl {module}`
**Edge Cases**:
- Module dependencies on other modules
- JPMS transitive requirements
- Test module descriptor
**Self-Test**:
- [ ] Did I update parent POM?
- [ ] Does module compile?
- [ ] Are exports correct?
```
#### E. Protocol Violations → Add Validation Hooks
**File**: `.claude/hooks/pre-{action}.sh`
**Create**:
- Pre-action validation
- Prerequisite checks
- State verification
- Helpful error messages
```bash
#!/bin/bash
# .claude/hooks/pre-write.sh
# Validates worktree before Write tool use
set -euo pipefail
# Extract agent name from context
AGENT_NAME=${CLAUDE_AGENT_NAME:-main}
# For sub-agents, verify working in agent worktree
if [[ "$AGENT_NAME" != "main" ]]; then
EXPECTED_PATH="/workspace/tasks/.*/agents/$AGENT_NAME/code"
if [[ ! "$PWD" =~ $EXPECTED_PATH ]]; then
echo "❌ ERROR: Agent $AGENT_NAME in wrong directory" >&2
echo "Current: $PWD" >&2
echo "Expected: /workspace/tasks/{task}/agents/$AGENT_NAME/code/" >&2
echo "" >&2
echo "SOLUTION: cd to your agent worktree first" >&2
exit 1
fi
fi
```
#### F. Configuration Gaps → Enhance Agent Config
**File**: `.claude/agents/{agent-name}.md`
**Add Sections**:
- "Common Scenarios" with examples
- "Anti-Patterns to Avoid"
- "Decision Trees" for complex