This article comes from my Usability Chakra, a place that’s very near and dear to my heart. One of my favorite quotes that I refer to a lot in the field of usability is said by Sherlock Holmes, in A Scandal in Bohemia.
“It’s a capital mistake to theorize before one has data. Insensibly one begins to twist facts to suit theories, instead of theories to suit facts.”
The quote from Sherlock Holmes describes a problem I run into a lot in the field of UX, so I wanted to write something about this problem, and to champion the usage of analytical data. However, as I went down the rabbit hole the article turned into something more. I guess ultimately this article is built around the sentence: Common analytical data applications inform user research, and determines what you have and/or still need.
That sentence sounds simple enough, but I’ve come to understand that everyone’s experience with analytical data varies greatly. And so, this article is a list of software, methods, and resources – some of which you may currently use, some not – that provide valuable data and parameters when conducting audits, redesigns, research, or tests.
Common analytical data applications…
Broadly speaking, most analytical data applications are 3rd-party, from Adobe, exist in a web browser, or are from Google in nature.
3rd-Party Applications
The following are various popular data mining tools used to accomplish many different tasks:
- CrazyEgg *
Through Crazy Egg’s heat map and scroll map reports you can get an understanding of how your visitors engage with your website so you can boost your conversion rates. In CrazyEgg you create experiments that run for a certain amount of users, days, or both. - ForSee
ForSee turns customer insights into an action plan – with embedded polls, questionnaires, and surveys – with multichannel customer experience analytics for web, mobile, and contact centers. - Hodes
Redefining how brands and talent connect, Hodes is a full-service employer brand agency that uniquely connects companies to talent. Hodes combines analytics with spending to calculate CPH (cost per hire) and conversions. - PageFair
Adblocking has gone mainstream, and PageFair’s goal is to protect the future of the free web by re-establishing a fair deal between web users and the content creators they want to support. Adblocking can disable external fonts, social media iconography, pop-up windows, and more. PageFair detects what percentage of your visitors are using adblocking. - QR Codes
QR Codes (abbreviated from Quick Response Code) let you track the scan statistics – how many times, when, where and with what devices the codes have been scanned – allowing you to notice any changes in performance immediately, and gauge real world and app integration. - ShareThis
The ShareThis button is an all-in-one widget that lets people share any content on the internet with friends via e-mail, social media, instant messenger, or text message. The ShareThis Social Optimization Platform affords A/B testing and viral prediction. The ShareThis box and integration is 100% free, but any analytic requires a paid prescription. - Webalizer (or other server-statistics)
Webalizer is a website traffic analysis server-side application, produced by grouping and aggregating various data items. These data items are captured by the web server in the form of log files, while the website visitor is browsing the website. Comparing server-side statistics against both Adobe and Google Analytics identifies the number of real humans versus bots.
Adobe Marketing Cloud
Adobe Marketing Cloud consists of the following eight data resources:
- Analytics *
Adobe Analytics is a set of tools for predictive and real-time analytics that can be integrated into third-party sources. It includes the Marketing Reports and Analytics (formerly SiteCatalyst), Ad hoc analysis (formerly Adobe Discover) and Data Workbench (formerly Insight) applications to help create a holistic view of business activities by transforming customer interactions into insights. - Audience Manager
Adobe Audience Manager is a data management platform that can be used to create profiles of audience segments. These profiles can then be used for targeted ad campaigns. - Campaign
Adobe Campaign is an analytics tool that helps users build a personalized experience based on customer habits and preferences. It plans, manages and executes campaigns from a unique environment. You can now have an intuitive, automated way to deliver messages over marketing channels. The new Adobe Campaign, formerly Neolane, is now being integrated with Adobe Experience Manager to help predict customers needs. - Experience Manager
Adobe Experience Manager is a web content management system for organizing, managing, and delivering creative assets. The user can use templates to create targeted content and publish them securely in the cloud. It is derived from a product called CQ by Day Software, which Adobe acquired in 2010. - Media Optimizer
Adobe Media Optimizer is a tool that manages, forecasts and optimizes media. It provides a consolidated view of how media is performing together with tools to accurately forecast user media. Media Optimizer helps you manage search engine marketing, display, and social campaigns. - Primetime
Adobe Primetime is a video platform that can be used to create and monetize video content, and make it available across multiple types of devices. A strategic partnership with comScore, announced in March 2016, will promote the collection and interpretation of viewing metrics across a range of non-traditional TV devices. - Social
Adobe Social is a tool for managing social content and social campaigns. It’s a comprehensive solution for building stronger connections through data-driven content. It deals with relevant posts, insightful conversations, measurable results, and social activities connected to business. Adobe Social is about the discovery of precise content, social networks and business results. - Target
Adobe Target is a tool for testing and targeting digital experiences. It includes a user interface, built-in best practices, and robust optimization tools for following site visitors. With its self-learning algorithmic approach it is able to increase conversion and filter results precisely. Adobe Target also uses factorial testing to understand elements for real-time targeted content. Adobe Target uses automated behavioral targeting with acquired data such as IP addresses, time of day, referral URLs and brand affinity.
Browser Tools
Apple Safari, Google Chrome, Microsoft Internet Explorer/Edge, and Mozilla Firefox all have tools that help with JavaScript debugging/errors, performance load times, performance audits under different speeds, DOM inspection, CSS changes, and storage issues.
- Chrome DevTools
To access the DevTools, open a web page or web app in Google Chrome. Either: Select the Chrome menu at the top-right of your browser window, then select Tools > Developer Tools. Right-click on any page element and select Inspect Element. - Internet Explorer Developer Tools
On any site you want to debug, open the Developer Tools and switch to the Script tab, then click Start Debugging. When starting the debugging process, the Developer Tools will Refresh the page and Unpin the tools if it is pinned. - Firefox Developer Tools
There are a few different ways to open the Toolbox: select “Toggle Tools” from the Web Developer menu (under “Tools” on OS X and Linux, or “Firefox” on Windows) click the wrench icon, which is in the main toolbar or under the Hamburger menu, then select “Toggle Tools” - Safari Web Development Tools
To do so, enable the “Show Develop menu in menu bar” setting found in Safari’s preferences under the Advanced pane. You can then access Web Inspector through the Develop menu that appears in the menubar, or by pressing Command-Option-I.
Google AdWords
Google AdWords is an advertising service for those wanting to display ads on Google and its advertising network. AdWords enables businesses to set a budget for advertising and only pay when people click the ads. The ad service is largely focused on keywords, and consists of the following eight tools:
- Change History
AdWords account contains a history of changes that shows what you’ve done in the past. This change history can help you better understand what events may have triggered changes in your campaign’s’ performance. You can then filter the changes to see only the ones you’re interested in. You could filter by date range, campaign, ad group, or user, for example. - Conversions
Conversion tracking is a free tool that shows you what happens after a customer clicks on your ads – whether they purchased a product, signed up for your newsletter, called your business, or downloaded your app. - Attribution
You can use the Model Comparison Tool to compare how different attribution models impact the valuation of your marketing channels. In the tool, the calculated Conversion Value (and the number of conversions) for each of your marketing channels will vary according to the attribution model used. - Analytics *
Google Analytics Solutions offer free and enterprise analytics tools to measure website, app, digital, and offline data to gain customer insights. By enabling your Advertising Features, Google Analytics will collect additional data about your traffic (you may need to update your privacy policy before enabling Advertising Features). - Google Merchant Center
Google Merchant Center is a tool which helps you to upload your product listings for use with Google Shopping, Google Product Ads, and Google Commerce Search. - Keyword Planner
Keyword Planner is a free AdWords tool that helps you build Search Network campaigns by finding keyword ideas and estimating how they may perform. - Display Planner
An AdWords tool that provides ideas and estimates to help you plan a Display Network campaign that you can add to your account or download. Display Planner generates ideas for all the ways you can target the Display Network. Targeting ideas are based on your customers’ interests or your landing page. - Ad Preview and Diagnosis
A tool in your account that helps identify why your ad or ad extension might not be appearing. The tool also shows a preview of a Google search result page for a specific term.
* Adobe and Google Analytics don’t track external links by default. External links can be tracked by Google Analytics Solutions, but a line of JavaScript must be added to each external link. Adblocking software disables ShareThis and and JavaScript external link detection. Heatmaps are the only way to determine external link clicking without JavaScript detection.
Inform User Research, and…
For a UX professional, being able to identify competitive benchmarks, make informed design and Information Architecture (IA) decisions, construct realistic personas, and uncover existing usability concerns is integral to their position.
Competitive Benchmarking
- Established Lists – Use established metrics and lists based on research already conducted by other individuals. Also, aggregating or cross-referencing multiple lists can provide additional considerations and insights. The following are some ideas on how to determine competitive benchmarks:
- Benchmark Databases – Baymard Institute, NN/g, etc.
- Similar Grouping – HEPC, Accreditation, NRLA, Recruitment, Fortune 500, or Location Peers
- Paid Services – advertising agencies, consulting companies, etc.
- Reverse Meta Keyword Search – For a specific webpage, web application, section, or even product you can take the current meta keywords, and reverse search in a browse search engine that has no cache, cookies, or search history.
- Search Engine Optimization (SEO) Peers – Using Keyword Planner you can test your current SEO meta keywords, improve, and base your benchmarks on what people are actually finding through search.
Design & IA
- Data In Context – With in-page analytics you can see the data in the context of the design, which makes the data compelling and easy to understand. People may not be using links in your main navigation, sub navigation items, or clicking on images… in-page analytics will show this in context.
- Behavioral Flow – User flows illustrate the paths people take on a site. It shows you what people do and, more importantly, what they are not doing. It shows how do people come to your key pages, what do they explore next, and what are they missing. With a user flow you can see what happens before and after each step, where people get off track, and you can identify unexpected steps.
- Heat/Scroll Maps – See how users are digesting your page’s content, what they aren’t visually seeing, if they are scrolling to content that’s not been prioritized, and if they are even using external links.
- Low Usage – With in-page analytics, user flows, and heatmaps you can create a picture of why users may not be using links, sections, or features of your website/web application, which can inform deep dives and future tests.
- User Expectations – Google Analytic’s referrals, acquisitions, and bounce rates inform how people are finding your web application/website, what they are looking for, and when/where they leave.
- Wireframe Sizes – Are you still designing for a 960 pixel width? Should you be? Browser, device, and display statistics can show what sizes and devices the largest portion of your users are experiencing your website/web application.
Real Personas
By connecting Google’s SEO, AdWords, and Analytics to one another you can enable demographic and insight information. You can then combine the following metrics in Google Analytics –
- Device
- Location
- Age
- Gender
- User Flow
– to create real primary, secondary, and tertiary personas based on actual user data (e.g. for my company’s blog, in the month of October 2016, a primary persona is a 35 to 45 year-old female from New York, who is searching for ‘when it’s best to replace running shoes’ on their iPhone).
Existing UX Problems
- Discovering Pain Points – By looking at both feedback from ForSee cross referenced to user behavior for certain sections of the website – individual pages, web applications, products, etc. – it is possible identify and possibly diagnose pain points.
- What’s Broken/Taking Too Long – By using a web browser’s performance testing with variant connection speeds, or JavaScript debugging console, it’s possible to view and identify problems that people are experiencing on limited data networks.
Determines What You Have and/or Still Need.
It’s a good position to be in if you have a surplus of data, and your only concern is how to visualize that data. However, more often I find that data is lacking, and that decisions are being made based on conversion-only, crunched timelines, emphasis on deliverables, guessing, trial-and-error, opinion, and preference.
Any company, entity, or organization is able to easily capture all of the data that a UX professional needs to conduct further tests, inform design, iterate wireframes, and make decisions on behalf of users. If you find yourself needing more data, then don’t be afraid to ask for that data. If the data you need isn’t being captured, request that it be captured going forward.
When in doubt I always default to CYA (cover your ass) mode, and ask myself the following six questions:
- Do I have enough data to make an informed and valid decision?
- Does somebody already have the data I need to make an informed and valid decision?
- Do I need more data to make an informed and valid decision?
- How do I get the more data (surveying, setup, requests, etc.) that I need to make an informed and valid decision?
- Do I need to conduct user testing to make an informed and valid decision?
- Is remote versus in-person usability testing best to get the data I need to make an informed and valid decision?
In closure, I understand that everyone doesn’t have access to their server at work; however, it may be in your best interest to request this kind of data, and if this type of data isn’t available, inquire as to if the resource can be setup going forward.
This article is by no means complete. Feel free to comment if I missed any analytical data tools, need to make edits, or update as needed.