grid-bench

GitHub repository

Open-source power system tools timed on the same problems, where every result is graded by an oracle that no tool under test takes part in: no speed number without a correctness number. Choose a problem; how it is set up and graded is shown with its results.

Measure
Grid
Chart shows
Tools

Each cell counts its column's cases: ✓ / ✗ / FAILED. ✓: the oracle accepted the result. ✗: the tool returned a result the oracle rejected. FAILED: the tool raised or did not converge (see Failures). ·: the tool reads none of the column's cases, or was not asked to run them (the large cases are opt-in).

Every run that raised, with the tool's own exception.

As each tool's package reports them. A build from an untagged commit reports the last version it was given, so where a commit is listed, the commit is what ran. ·: a pinned release, or a build whose commit its results did not record.

How every problem is measured