Measuring the User Experience on a Large Scale: User-Centered Metrics for Web Applications
Kerry Rodden, Hilary Hutchinson and Xin Fu (Google, CHI 2010)
Open resourceSenior product design: owning outcomes · Proficient · Lesson 66 of 99 · about 3 weeks
Project: own a shipped feature's outcome end to end
After this you have led one live feature from framing to measured result and written an honest outcome review of it.
Why it matters
Brief: choose one feature or meaningful change you can lead in a real product with real users (at work, or on a non-profit, open-source or side product with an audience). Required: (1) before design starts, write and date a one-page metric plan using goals-signals-metrics, with a baseline and at least one guardrail; (2) write a shaped pitch with an appetite and an explicit out-of-scope list; (3) interview at least three stakeholders and record their constraints; (4) design, test with users at least once, and run design QA on the build; (5) after launch, collect at least two weeks of data plus five follow-up user conversations; (6) write the outcome review described in the phase evidence. Constraints: no invented numbers; if instrumentation was not possible, say so and use the strongest qualitative evidence you can get. It is good when a reader can see the decision chain from goal to cut to result, and when the review would survive the PM and the lead engineer reading it.
How to use this resource
Use section 3's goals–signals–metrics table as the template for the metric plan you write before designing.