Adding Criteria

What are Questions?

Questions, also know as Guideline Criteria, are the specific evaluation criteria within each category. When you run an evaluation, you'll answer each question as Pass or Fail, and the results roll up into category scores and an overall flow score.

To avoid diminishing returns and keep evaluations actionable, the system enforces practical limits on guideline criteria. Heuristic evaluation is meant to quickly surface high-impact issues and support consistent design across products and flows. As criteria counts grow, usefulness drops. Large sets generate too many failures to realistically address and slow meaningful progress. In practice, sets above roughly 100 to 150 criteria tend to exceed the point of effectiveness, where redesign becomes more appropriate than continued incremental fixes.

Writing Effective Questions

A good question is:

Specific and Testable

❌ Bad: "Is the design good?"
✓ Good: "Can users complete checkout in 3 clicks or less?"

Single-focused

❌ Bad: "Are buttons clear and accessible?"
✓ Good: "Are call-to-action buttons clearly distinguishable from other page elements?"
✓ Good: "Can all buttons be activated by keyboard?"

Observable

❌ Bad: "Is the spacing adequate?"
✓ Good: "Is there at least 16px spacing between form fields?"

Question Structure

When adding a question:

Assign to Category - Which category does this question belong in?
Write the Question - Clear, specific, testable criteria
Define Pass Criteria - What does a "Pass" look like? (optional but recommended for consistency)
Define Fail Criteria - What triggers a "Fail"? (optional but recommended for consistency)

Example Guideline Set

Navigation & Information Architecture
├── Can users find the search functionality within 3 seconds?
│   Pass: Search bar visible above fold, prominently styled
│   Fail: Search is hidden, nested in menu, or requires scrolling
├── Is the main category menu showing 7 or fewer options?
│   Pass: Menu has ≤7 items
│   Fail: Menu has >7 items or requires hovering to see submenus
└── Can users understand their current location in the site?
    Pass: Breadcrumb visible OR current page highlighted in nav
    Fail: No indicator of current location

Accessibility
├── Is the color contrast ratio at least 4.5:1 for body text?
│   Pass: Contrast check passes WCAG AA standards
│   Fail: Contrast falls below 4.5:1

Tips for Building Strong Guidelines

Use consistent language - If one question says "3 clicks or less", don't say "quick navigation" in another. Consistency makes evaluation repeatable.
Test your questions - Run a pilot evaluation with your guidelines. Questions that cause confusion should be refined.
Document context - If "Pass" vs "Fail" is subjective, add notes explaining the distinction. This helps team members evaluate consistently.
Think longitudinally - These questions will be asked again in v2, v3, and beyond. Make them stable enough that trend tracking makes sense.