Benchmarks

All numbers below are real — run vibecop scan on any of these repos yourself to reproduce. Finding density = findings per 1,000 lines of code (findings/kLOC).

Methodology

How Findings Are Counted

vibecop scan <path> --format json --no-config --max-findings 2000

summary.total from JSON output. --no-config ensures no per-project suppression.

How LOC Is Counted

wc -l across all .ts, .tsx, .js, .jsx, .mjs, .cjs, .py files (excluding node_modules, dist, build).

Results

Established Projects (Professionally Maintained)

Project	Stars	Files	LOC	Findings	Density
fastify	65K	275	74,428	124	1.7/kLOC
date-fns	35K	1,543	99,859	308	3.1/kLOC
TanStack/query	43K	997	148,492	652	4.4/kLOC
express	66K	141	21,346	123	5.8/kLOC
zod	35K	356	70,886	964	13.6/kLOC

Median: 4.4/kLOC | Average: 5.7/kLOC

Vibe-Coded Projects (AI-Generated/Assisted)

Project	Stars	Files	LOC	Findings	Density
dyad	20K	956	147,284	1,179	8.0/kLOC
bolt.diy	19.2K	392	71,639	977	13.6/kLOC
code-review-graph	3.9K	95	27,119	361	13.3/kLOC
context7	51.3K	71	9,201	129	14.0/kLOC
vibe-check-mcp	480	55	5,964	119	20.0/kLOC
magic-mcp	4.6K	14	1,096	28	25.5/kLOC
browser-tools-mcp	7.2K	12	8,346	414	49.6/kLOC

Median: 14.0/kLOC | Average: 20.6/kLOC

Comparison

Metric	Established	Vibe-coded	Ratio
Median density	4.4/kLOC	14.0/kLOC	3.2x
Average density	5.7/kLOC	20.6/kLOC	3.6x

Vibe-coded projects consistently trigger more findings per line of code.

Notes on Established Repo Findings

Some established repos show higher density for valid reasons:

• zod (13.6/kLOC): 634 of 964 findings are excessive-any. Zod deliberately uses any for TypeScript type gymnastics — this is intentional, not a code smell.
• date-fns (3.1/kLOC): 218 of 308 findings are excessive-comment-ratio. date-fns has extensive JSDoc documentation — by design, not an AI pattern.
• express (5.8/kLOC): 73 of 123 findings are placeholder-in-production. Express uses example domains in comments.

vibecop detects patterns, not intent. Use .vibecop.yml to tune or disable detectors for your codebase.

v0.2 New Detector Impact

The 6 LLM/agent safety detectors added in v0.2 found 157 additional issues across the vibe-coded repos that v0.1 detectors missed entirely:

Detector	Findings	Example Repos
unsafe-shell-exec	63	dyad (47), context7 (6), code-review-graph (5)
llm-unpinned-model	53	bolt.diy (32), dyad (12), vibe-check-mcp (6)
llm-no-system-message	39	dyad (31), bolt.diy (7), vibe-check-mcp (1)
llm-call-no-timeout	2	vibe-check-mcp (2)

Zero v0.2 detector findings on the 5 established repos — they do not use LLM APIs or dynamic shell execution.

Reproducing

Fixture Benchmark

Run the benchmark script included in the vibecop repository:

bash scripts/benchmark.sh

Full Open-Source Benchmark

Clone any repo and scan it:

# Clone a repo
git clone --depth 1 https://github.com/fastify/fastify /tmp/fastify

# Scan with no config and high finding cap
vibecop scan /tmp/fastify --format json --no-config --max-findings 2000

The JSON output includes a summary.total field with the finding count. Count LOC with:

find /tmp/fastify -name '*.ts' -o -name '*.tsx' -o -name '*.js' -o -name '*.jsx' \
  | grep -v node_modules | xargs wc -l