Introducing PerfCompare’s Simple View

We heard you. You don’t want to open your PerfCompare performance comparison and feel like you need a background in statistics to interpret the Mann-Whitney-U results. Cliff’s Delta, CLES, and normality tests, oh my! It should not feel like you’ve blindly walked into a college stats exam, unless of course, you love statistics. For the rest, all you want is an easy answer to the question: Did my changes make things faster or slower?

So we fixed that. PerfCompare now shows the bare essentials by default and tucks the heavy statistics one click away.

An approachable default for beginners and occasional users

The results table now leads with the platform, the Base and New values, how far the median moved and its direction, the status (which includes the option to filter out noise) and the size of the change. To create a more advanced view, a single “Advanced options” dropdown allows you to add the expert columns (Cliff’s Delta, CLES, Significance) and the info-rich expanded row details below the full-width graph. A small help section at the top of the results provides a quick explanation of the columns. The purpose is to serve a diverse audience while preserving your selections in the URL so a shared link reproduces your view.

The “Advanced options” dropdown lets you add the precise statistics columns:

• Cliff’s Delta: An effect-size measure of how far the Base and New distributions have pulled apart
• CLES (Common Language Effect Size): The chance a random New run beats a random Base run, e.g. “New wins 78% of the time”
• Significance:  Whether the difference is a real signal or just noise

Each one toggles on independently, so you can surface only Significance or the whole trio.

Expand a result and you get a clean panel: a full-width graph of how your Base and New runs are spread out, a quick summary (platform, app, which direction is “better”), and a little “how to read this graph” tool-tip for anyone who hasn’t stared at a distribution curve before. You can still poke at the graph’s mode controls if you want to dig in.

You can access the extra expanded row details such as effect size and confidence intervals or the full stats table in the “Advanced options” dropdown. Turn on what you need, and the expanded extras lay out in neat two-column rows. If you turn on mode analysis and there aren’t multiple modes to show, we now say “No mode analysis available” instead of a blank space.

Lastly, the advanced columns and expanded-row details you’ve switched on, along with your filters and sorting, are all encoded in the URL. Paste it to a coworker and they’ll land on the same result you were looking at; for filters and sorting, the shared link takes precedence over whatever they had saved locally. (One exception: the “How to read the results” panel is a personal per-browser preference, so it stays with you rather than tagging along on the link.)

What’s next

Several follow-ups on the way:

• Remembering your advanced choices. Right now the advanced columns and expanded-row toggles start from the clean defaults each visit (they only persist in a link). Soon PerfCompare will remember them in your browser the same way we do for filters. However, if you open a shared link from a colleague, their specific view takes priority, overriding your own local settings so you both see the exact same data.
• Carrying your selections into subtests. Today, the Advanced options you pick on the main results page don’t follow you when you open a test’s subtests. We’re wiring it up so whatever you’ve enabled up top is automatically enabled in the subtests dropdown too, for one consistent view.
• Expanding the “How to read the results” cheat sheet to include Cliff’s Delta, CLES, and Significance definitions.

Share your feedback

We want PerfCompare to work for all users, whether you’re chasing your first regression, or you’re an expert who doesn’t want to lose their favorite metrics to a cleaner view. Our answer is separating the simple view from the power user view, and making the toggle obvious and shareable.

The great news is it’s live now! Go expand a Mann-Whitney-U result, click around the Advanced options, and tell us what you think. Please share your feedback, suggestions, or comments on the #perfcompare channel on Matrix. If you’ve encountered a bug or have a feature request, please file them in Bugzilla. We’re listening.