Show all answersShow just the questions

1. How Do In-Depth Usability Studies Differ from QuickTests™?

An In-Depth test is a combination of a market research study and a thorough usability test.

 

 

 

You can have several groups of shoppers, each with different tasks.

Sample sizes start at 100, and can be much more, if needed.

Testers can be pre-screened to make sure they exactly match your target market.

Can customize the questions to meet your unique needs.

Expert analysis and recommendations provided.

2. Why Would I Need An In-Depth Usability Study?

You can test more of your site, since different tasks will lead to people using different parts of the site.

You can probe for differences based on the task they were trying to accomplish, their educational background, types of computer systems, browsers, connection speeds, internet experience, age, etc.

The large sample size gives you more evidence for setting priorities for site improvements.

If you want to test something specific or unusual, customizing the questions provides maximum consumer insight.

Designed for people who don’t have the time and expertise to wade through hundreds of pages of results. We’ll highlight key tester comments and scores, and make recommendations about priorities for change, and what changes will be the most effective at improving your site’s effectiveness.

3. What Does the Report Include?

A blend of consulting, surveying and mystery shopping, our service collects qualitative and quantitative feedback from the mystery shoppers as they work their way through the site. Testers answer questions about the following as they work through the site:

• Home page - First impressions; is it clear what the site is/does; can they easily find the essential information they need/want to go further?

• Navigation - How easily can people find their way around the site and uncover the information they want?

• Customer service - Web Mystery Shoppers® can test all available customer service mechanisms, including e-mail and toll-free numbers.

• Search function - Is it easy to find; how useful are search results?

• Ordering or transacting process - What is involved? How smoothly does the process work? What errors are encountered, and can users fix them easily?

• Overall impressions, positive and negative.

Once all the feedback is in, we analyze the results and recommend specific changes you can make to your web site and site-related customer service to increase your conversion and usage rates.

4. What are the Benefits of This Type of In-Depth Market Research?

Benefits

• Set priorities with confidence. You'll have the statistics to back you up.
• Increase visitor satisfaction and conversion rates. Uncover the real problems people have using their quirky home or office computers, avoid false results from the stress of being filmed or watched in a lab, and the large sample size means you'll be able to find and fix more real-world problems. Get an in-depth understanding of your customers and prospects.
• Save time. We'll spend the days it takes to do the in-depth textual and statistical analysis for you.
• Save money by getting more things right the first time.

Unique features that create those benefits

• 100+ online testers per site. You can choose more or fewer testers to suit your needs and budget. Statistics experts have proven that you need at least 30 people per group for a statistically reliable sample. And a lot of our clients tell us their bosses feel more confident with a large sample size.
• Testers are demographically matched to your target market. If you are concerned about several different types of customers (different ages, income levels, geographic regions, levels of computer expertise, types of computer systems,...) you need at least 30 in each of those categories. There will be some overlap, so a group of 100 - 200 is often all you need, even to get 30 in each of several different categories.
• Can include screening survey, to help deepen market/customer knowledge.
• Testers work from their own computers, 24/7. This is a more realistic setting than lab-based usability tests or focus groups.
• Testers use varying computer systems, different browsers, and have differing levels of computer expertise and online shopping experience. You'll uncover problems that would be missed in lab-based setting, where only a small number of system combinations are available.
• Testers are trained and paid for their time, so they provide a thorough analysis.
• Testers give their feedback as they work through the site. This generates much more detailed and useful information than after-the-fact surveys.

5. How Much Does In-Depth Testing & Analysis Cost?

Three things affect the price:

• Consulting and administrative time, which varies with the complexity of the assignment.

• Extent of customization. Questions tailored to your unique needs can be added to the shoppers' evaluation forms. This entails special programming, for which there is a small additional cost.

• Mystery shoppers expense, which varies with the number of testers used and the complexity of the tasks required of them.

We can tell you that a full-scale study typically costs between $20,000 - $40,000, and produces hundreds of pages of evaluation and recommendations. If you can’t afford that, you may be able to benefit from a much less expensive QuickTest™. Why do In-Depth Studies cost so much more? Because they take lots more time and expertise. Why do those who can afford them buy them? Read what our clients have said.

6. What Have Others Said About Web Mystery Shoppers’ In-Depth Studies?

“I like the recommendations!”

Kelly Kruse
Bank of America

“At each phase of the project AADAC received reports of progress and insight into the current findings. The final report was delivered on time, on budget and, more importantly, it exceeded expectations. Web Mystery Shoppers’ delivered an extremely detailed report that included thoughtful recommendations, along with clearly detailed summaries of … feedback. Our web team has been able to use the report many times to clearly justify the changes we made to the Unity site.

AADAC would feel fully confident using Web Mystery Shoppers’ services again.”

Tannis Gibson
Alberta Alcohol and Drug Abuse Commission (AADAC)

“Over the past several years we have surveyed our online customers, conducted focus groups and usability tests. All these activities are useful, but have inherent flaws. Web Mystery Shoppers gave us the opportunity to obtain real user experiences, obtained from the comfort (no pressure) of customers’ own homes. It was tremendously valuable. They provided us with detailed information on what worked, didn't work, and why, including user experience statistics. Our previous efforts had, in many cases, given us a false sense of comfort that we were doing the right things. Now we know better and also have excellent ideas how we can make our site better.”

Martin Stevens
Royal Bank of Canada

To get started, call us toll-free 1-866-211-7027 or send an e-mail: indepth@webmysteryshoppers.com.

Not quite ready to talk to us? Why not download our free Testing Methodologies background paper to find out how our approach compares with other site-testing methods? And sign up for our monthly newsletter, which is full of tips and current research on website effectiveness.

If an in-depth study is more than you need (or can afford), check out our QuickTests™, for fast, inexpensive website usability testing.