RefreshDC: Using Data to Inform Design

At Thursday night's RefreshDC panel, "Data-Driven Design", I had the opportunity to talk about the various kinds of research and tools we use at Clearspring to design the AddThis sharing platform. Using data to inform design is fundamental to how we approach user experience at Clearspring, and we do our best to incorporate different kinds of information to inform decision-making.

The tools we use could be placed into two groups:

• Quantitative tools, such as internal reporting, Google Anaytics, and Google Website Optimizer
• Qualitative tools, such as surveys, user testing, and field research

The real power of this suite of data-gathering methods, however, is when multiple tools can be used together to accomplish a larger objective.

Here are a several themes that we discussed on the panel or were brought up by the bunch of great questions from the audience: 

• Start with a question or hypothesis: any statistics can be interpreted multiple ways, so know what question or problem you're trying to solve and make sure your empirical design is sound
• Use multiple techniques to more thoroughly understand a problem:  I described a project where we used a combination of field research, A/B testing, multivariate testing, user testing and event tracking to optimize the process we have that allows publishers to get AddThis code for their sites.
• Not all tools are created equally: Paul Koch of Vigit Labs demonstrated how heatmaps can be used to understand how users are utilizing pages in ways that wouldn't be obvious with other tools
• Beware of local maximums: optimizing one particular stat can be a rabbit hole preventing you from stepping back and recognizing a much larger opportunity – be sure you're solving the right problems.
• Be data-informed, but use common-sense: data and research are powerful tools in your arsenal, but there are also reasons to choose not to follow the data; these include brand identify, business strategy, user interests, external forces like regulatory bodies, and other reasons. I included this topic in my November Refresh DC talk, Designing for One Billion People, and you read/listen more about this from Jeff Gothlef, Julie Zhou, and Adam Mosseri.
• Many tools are free or inexpensive: so there's no excuse not to be using data to inform your next design! I've included links below to some of the tools we discussed.

More about the event:

• Pictures from the event
• Panel participants, from the Eventbrite page for the event

Here are links to some of the tools we discussed:

• Google Analytics
• Google Website Optimizer
• Usertesting.com
• Optimizely
• BT Buckets
• Surveymonkey.com