Overview

This methodology evaluates an interface against a structured set of usability and design rules. Each rule is scored as pass or fail, producing a conformance metric that shows how closely the interface matches the selected guideline set. Because each result is binary and traceable to a specific guideline, the score is repeatable, comparable over time, and useful for identifying where the design meets requirements or falls short.

Scoring Model

Each guideline is marked with one of the following:

• Pass = 1
• Fail = 0
• N/A and Unanswered = excluded from scoring

Only Pass and Fail are counted in the score calculation. No weighting or subjective scale is applied.

Category Scoring

For each category, let:

• Pass = number of guidelines marked Pass
• Fail = number of guidelines marked Fail

Then:

$$\text{CategoryScore} = \frac{\text{Pass}}{\text{Pass} + \text{Fail}}$$

Example:

$$\begin{aligned} \text{Pass} &= 6 \\ \text{Fail} &= 2 \\ \text{CategoryScore} &= \frac{6}{6 + 2} = 0.75 = 75\% \end{aligned}$$

If $\text{Pass} + \text{Fail} = 0$, the category is excluded from aggregation.

Overall Score

If there are $k$ valid category scores, the overall score is:

$$\text{OverallScore} = \frac{\sum_{i=1}^{k} \text{CategoryScore}_i}{k}$$

The result is expressed as a percentage.

Flat Score Variant

Without categories, let:

• TotalPass = total number of Pass across all guidelines
• TotalFail = total number of Fail across all guidelines

If categories are ignored:

$$\text{Score} = \frac{\text{TotalPass}}{\text{TotalPass} + \text{TotalFail}}$$

Worked Example

Using the table above:

$$\begin{aligned} A &= \frac{4}{4 + 1} = 0.80 \\ B &= \frac{2}{2 + 2} = 0.50 \\ C &= \frac{3}{3 + 0} = 1.00 \\ \text{OverallScore} &= \frac{0.80 + 0.50 + 1.00}{3} = 0.7667 = 76.67\% \end{aligned}$$

Grade Thresholds

Evaluation Method

A reviewer goes through each guideline and marks whether the current interface satisfies it. Each guideline is treated as a yes-or-no check:

$$g_i(\text{flow}) \in \{0, 1\}$$

The final score is the average of these binary outcomes across the set. This keeps the model consistent, easy to audit, and traceable to individual failed guidelines. It is intended to measure rule satisfaction and track change over time, not infer quality from ambiguous user-dependent signals that may vary between users or sessions.

What the Score Represents

• Percentage of defined rules that the interface satisfies
• Clear signal of how closely the interface matches standards
• Stable metric for comparing the same system over time
• Directional trend to track improvement or regression

What the Score Does Not Represent

• User satisfaction or emotional response
• Perceived ease of use or aesthetic appeal
• Efficiency, speed, or task success
• Cognitive demand or user behavior patterns

Using Results

• Focus on changes over time rather than single scores
• Review failed items to understand specific gaps
• Keep older evaluations to track trends and regressions
• Use the score to guide decisions, not to define success or failure