Back to catalog

sbt-benchmark

Use PROACTIVELY for JMH benchmarks in this Spark protobuf project. Handles sbt commands, always cleans before benchmarking, saves output to /tmp logs, and uses log-reader agent to parse results.

  • Category: discovered
  • Tags: github, community
  • Version: 1.0.0
  • Author: walid
  • Downloads: 0
  • GitHub stars: 1
  • Updated: 2025-11-21

Download the latest version (v1.0.0) from this page.

SKILL.md

---
name: sbt-benchmark
description: Use PROACTIVELY for JMH benchmarks in this Spark protobuf project. Handles sbt commands, always cleans before benchmarking, saves output to /tmp logs, and uses log-reader agent to parse results.
allowed-tools: Bash, Glob, Grep, Read, Edit, AskUserQuestion, Task, BashOutput, KillShell
---

Expert Scala/sbt build engineer specializing in JMH benchmarking for the Spark protobuf backport project.

## Workflow

### 1. Clarify Scope
If ambiguous, examine `bench/` and ask which benchmarks to run. Never run all benchmarks without asking.

### 2. Execute

**CRITICAL**: Always `sbt clean` before benchmarks to avoid incremental compilation errors.

```bash
# Specific benchmark
sbt clean bench/Test/compile 'bench/Jmh/run .*<BenchmarkName>.*' | tee /tmp/jmh_<name>.log

# Parser comparison
sbt clean bench/Test/compile 'bench/Jmh/run .*Scalar.*(anInlineParser|generatedWireFormatParser).*' | tee /tmp/jmh_scalar_comparison.log

# All benchmarks (ask first)
sbt clean jmh | tee /tmp/jmh_all.log
```

Run in background (`run_in_background: true`) and monitor with `.claude/skills/sbt-benchmark/scripts/monitor-jmh.sh -t <timeout> /tmp/jmh_<name>.log` (prefer using 30s for first monitor, then ETA). Do NOT use BashOutput.

### 3. Parse and Present

Use log-reader agent to parse `/tmp/jmh_<name>.log` and extract result table. Present raw JMH results without interpretation unless anomalies detected (noise, errors, warnings).

## sbt Commands

Key commands (see `build.sbt` for complete list):
- `sbt clean` - Required before JMH
- `sbt jmh` / `sbt jmhQuick` - Full/quick benchmarks
- `sbt unitTests` / `propertyTests` / `integrationTests` / `allTestTiers` - Test tiers
- `sbt assembly` - Build shaded JAR

## Benchmark Parameters

To discover parameters: Read `bench/` source files, look for `@Param` annotations, `@Benchmark` methods, and `@State` classes. Use `-p paramName=value` for custom values.

## Performance Analysis

Only when user requests. Use unambiguous language: "1.5x speedup" (old/new), "33% reduction", "2000ns → 1500ns". Not: "1.5x faster" (ambiguous).

## Monitoring

- **Live**: `.claude/skills/sbt-benchmark/scripts/monitor-jmh.sh -t <timeout> /tmp/jmh.log` - Real-time progress, auto-exits on completion/timeout (e.g., `-t 30` for 30s)
- **Status**: `.claude/skills/sbt-benchmark/scripts/jmh-status.sh /tmp/jmh.log` - One-shot status check