MicroMind Experiment 001
PlannedCompare 0.8B variants (A-I) against raw 2B and 4B models
Models
Qwen2.5-0.5BQwen2.5-1.5BPhi-3-miniLlama-3.2-1BLlama-3.2-3B
Metrics
Task SuccessAccuracyLatencyCostMemory
Apple M2 (8GB RAM) Darwin 0.2
MindBench
Benchmark Infrastructure
Reproducible experiments with full provenance. 20 metrics. Hardware profiles. Config snapshots. No cherry-picking.
First Evolution Demo
PlannedIdentify weakness → generate candidates → sandbox → benchmark → promote
Models
Qwen2.5-0.5B + evolution
Metrics
Pre/Post evolution deltaRegression checkCost delta
Apple M2 (8GB RAM) Darwin 0.2
MindBench Metrics (20 Dimensions)
Every benchmark run captures all 20 metrics. No cherry-picking. Full provenance recorded.
Task Success↑ Better
%
—
Accuracy↑ Better
%
—
Factuality↑ Better
%
—
Hallucination↓ Better
%
—
Latency (p50)↓ Better
ms
—
Latency (p95)↓ Better
ms
—
Tokens/sec↑ Better
tok/s
—
RAM Peak↓ Better
GB
—
CPU Avg↓ Better
%
—
GPU Util↑ Better
%
—
Model Load↓ Better
s
—
Compute↓ Better
FLOPs
—
Energy↓ Better
Wh
—
Cost↓ Better
$
—
Tool Calls↓ Better
#
—
Escalation↓ Better
%
—
Mem Retrieval↑ Better
#
—
Skill Reuse↑ Better
#
—
Adaptation↑ Better
%
—
Forgetting↓ Better
%
—
Regression↓ Better
%
—
Reliability↑ Better
%
—
Reproducibility by Design
Fixed seeds, config snapshots, dependency hashes, hardware fingerprints. Anyone can re-run and verify.
Fixed SeedsConfig SnapshotsDependency HashesHardware FingerprintsFull Provenance
Run Your Own Benchmarks
Clone the repo, run seai doctor, and start benchmarking your own Minds.
One Command
seai init → seai run
Local First
Runs on your hardware
Evolves
Improves through experience