Maze User Testing Review – The Good and Bad

Maze is a great UX research tool for teams that want an all-in-one quantitative and qualitative research tool that delivers actionable data fast. It’s also great if prototype testing is your main priority. But Maze isn’t perfect for everyone.

If you run a lot of research studies and have a hefty budget, then Maze should definitely be on your short list. Smaller organizations or anyone only doing occasional research testing might be better served with a different solution.

Likewise, if you want a research tool that gives you access to all qualitative and quantitative testing functionality without having to jump straight to the most expensive subscription plan, you may want to consider other options.

How Does Maze Compare to its Closest Competitors

Researchers on the hunt for a single tool to conduct both quantitative and qualitative testing usually see the same few names pop up on every short list. These include Optimal Workshop, UserTesting, and Lyssana. The tool that’s best for any given researcher largely depends on their needs and priorities.

Here’s how Maze stacks up against its closest competition.

Optimal Workshop vs. Maze

Optimal Workshop has you covered when it comes to both qualitative and quantitative testing. It offers a robust collection of testing tools, including interviews, first-click testing, tree testing, card sorting, and surveys. Maze offers most of these, but not first-click testing. In terms of available quantitative tests, Optimal Workshop brings more to the table.

Optimal Workshop also comes out ahead when it comes to the size of participant pools. Optimal Workshop offers more than 80 million study participants compared to Maze’s three million.

However, Maze dominates when it comes to prototype testing. In fact, it’s the standout feature that puts Maze on most short lists. Maze prototype testing is tried and true. Optimal Workshop didn’t offer prototype testing until just recently, when it deployed its beta version. It’s too soon to assess the robustness of this new offering. Stick with Maze for now.

Read our deeper dive into Optimal Workshop and Maze.

UserTesting vs. Maze

UserTesting and Maze both offer drag-and-drop functionality for creating new tests. However, UserTesting makes it easier to get a diverse array of new tests up and running with more than 100 testing templates compared to Maze’s 55.

UserTesting also wins when it comes to the depth and breadth of research insights. The EnjoyHG tool allows teams to quickly share highlight reels, feedback, video clips, and key insights with anyone in just a few clicks from a single dashboard. Maze does offer some similar functionality, but UserTesting is a better experience.

Maze does come out ahead of UserTesting when it comes to its flexible approach to pricing. While UserTesting doesn’t put pricing on its website, anecdotal information sourced from public review sites suggests that it costs upwards of $15,000 per year at the low end. Maze offers a free plan and a Starter plan for just $99 per month, although these lower priced plans do have functionality limitations.

Learn more about UserTesting in our review of the top usability testing tools.

Lyssna vs. Maze

Lyssna is another all-in-one UX research tool, offering both qualitative and quantitative testing options. Lyssna offers more quantitative testing options than Maze. It also includes unlimited access to its full research suite on every plan, including its free plan. Maze requires you to sign up for its Starter Plan ($99) to gain access to some testing tools. You need to jump to Maze’s Organizational Plan (priced upon request) to get full access to all Maze tools.

Lyssna is also very price friendly for startups and smaller organizations. Both its free and Basic plan (starting at $75/mo) offer more seats and testing options/volume than Maze’s comparable plans.

Maze blows Lyssna out of the water though when it comes to the size of its participant panel, offering more than three million participants compared to just 530,000+ with Lyssna. Maze also allows you to refine participant selection with more than 400 targeting attributes compared to Lyssna’s 35.

Take a deeper dive into Lyssna here.

Maze: The Good and The Bad

Maze excels at prototype testing, making it a standout among all UX research tools. It also has a number of other impressive features, like advanced AI analysis, numerous templates, and rapid delivery of testing results. Where Maze falls a bit short is in its reporting functionality, limited scaling capabilities, and high price point.

What Maze Is Good At

Robust prototype testing: Maze built its reputation on prototype testing. With it you can validate designs and reveal usability issues with actual users in real time, all before a single line of code is written. 

Prototypes can be imported from popular design tools, including Figma, Adobe XD, Sketch and Invision, and you can test up to five variants of a design simultaneously. Visual reports that you can filter are prepared in real-time and embedded in your productivity tool to ensure seamless collaboration across the entire organization.

Fast data delivery: Using Maze gives researchers and design teams the data they need to make decisions in hours as opposed to days or weeks as with some other research tools. Studies can be created in minutes, audiences can be recruited fast, and testing can begin within a day. Data is returned in real-time, providing actionable results right away.

Many integrations: Maze offers the most integrations among its competition, streamlining everything from design and productivity to calendaring and video conferencing. 

Design integrations include Adobe XD, Figma, Axure, and Sketch. Productivity tools like Atlassian, FigJam, Miro, Notion, and Slack keep projects moving along. Make calendaring a breeze with Exchange, Google Calendar, Office 365, Outlook, and iCloud integrations. Then streamline video conferencing when you connect Maze to Zoom, Google Meet, or Teams.

You can even add in-product prompts to boost testing participation with the Amplitude integration.

Diverse testing panel: Maze takes the guesswork out of participant selection. There’s a pool of more than three million pre-vetted participants from over 130 countries available, and you can drill down deep to find the exact ones right for your test. There are 400+ filters to choose from, letting you refine your participant qualifications as precisely as needed.

Recruiting for interviews is equally as precise. Choose from millions of B2B and B2C participants from around the world, then customize the screening process to ensure your future interview subjects meet your exact criteria before you ever invite them to participate. Maze also lets you test with your own user pool, giving you direct links and in-product prompts to start the process.

Using AI for good: Maze is on the leading edge of integrating AI within its suite of products. It uses AI to help reduce bias and other issues with testing questions, speed up data analysis, and reveal deeper insights.

• Better test questions: Let Maze help you craft the perfect test question. As you type, Maze AI identifies potential issues with wording and grammar, then offers alternatives designed to help you write the question to get the best response. Just one click revises the question automatically.
• Dynamic follow-up: AI can help you go beyond superficial answers by providing automated, follow-up questions based on a participant’s actual answer. This helps you dive deeper into the “why” behind a participant’s initial response.
• Theme identification: Maze eliminates the need for you to manually read through hundreds of responses to identify recurring themes. Maze AI reads through open-ended questions and identifies common themes in just minutes, then lets you filter by theme and sentiment, giving you quick and actionable insights.
• Interview analysis: Maze AI makes quick work of reading and summarizing interview transcripts. It also highlights key themes so reviewers can quickly scan any document and read adjacent responses for context.

Good selection of templates: There are currently 55 different templates available in Maze. These are organized both by use case and team needs. Designers, marketers, product managers, and researchers will all find an assortment of templates to help them jump-start their studies. 

From a use-case standpoint, templates cover a lot of territory, including concept validation (8), content testing (10), copy testing (9), feedback survey (12), idea validation (8), satisfaction survey (5), usability testing (12), and wireframe testing (6). If you can’t find a template to suit your situation, you can also suggest a new template to the Maze team and they’ll create it.

Potential Maze Drawbacks

Reporting is clunky: More than one Maze reviewer on public sites like G2 and TrustPilot noted that Maze reporting leaves a bit to be desired. Users cited lack of editability of pre-defined reports and no ability to combine multiple reports into a single document as drawbacks. Users also felt the robustness of study overviews was lacking and could be improved to add value when sharing data with stakeholders.

Inability to scale projects or track project changes: Users note that the lack of a change log or other tracking information on a project is problematic. There is no way to tell who made the most recent changes or when changes were made. Likewise there are limited project organization tools and no option for a folder structure within projects. Both of these omissions make it difficult to scale research projects in Maze.

Expensive for smaller organizations: We’ll dive deeper into pricing next, but generally users cite the lack of lower-cost pricing plans as an obstacle for startups, smaller organizations, or any size organization with a tight budget. While Maze doesn’t share pricing for its Organization Plan (the plan you need to have access to all Maze functionality), some internet sleuthing revealed that the average price for a Maze subscription is around $12,000 per month. For enterprise-level customers with bigger budgets, this may not be an issue. But for occasional testers or organizations with more limited budgets, there are more affordable options out there.

Maze Pricing, Plans, and Add-Ons

Maze offers three different plans. Pricing is available for two of the three plans on their website.

Free Plan

This plan is billed as the option for “individuals who want to get into usability testing.” It’s free which is great, but you definitely get what you pay for. With it, you’re limited to one unmoderated study or interview study per month, can have no more than one project active at a time, and may include up to three studies in any project. You’re also limited to no more than seven “blocks” (questions or missions) in a test. 

Given the limitations that come with a free plan, consider it more of a free demo. In fact, when you sign up to try out Maze for free, you’re actually signing up for the Free Plan. It will give you a feel for how the system works, but it isn’t going to get you far with usability testing.

Starter Plan

The next option is the Starter Plan for $99 per month on an annual plan. Quite a steep jump from the free option, with no middle ground in between. This plan is actually a slightly elevated variation of the Free Plan. Starter gives you a total of 12 studies (one study per month, basically), but lets you use them as you wish throughout the length of the plan. 

With the Starter Plan, you’ll also upgrade to unlimited blocks, access to all templates, the ability to make and share video clips, and get the AI question rephrasing feature. However, you’re still limited to five collaborators (called “seats”), just like with the Free Plan. But you will be able to archive and unarchive projects at will. If you really want to unleash all of Maze’s testing power, you’ll need to go with the Organization Plan.

Organization Plan

This is the third option available for Maze users. While no pricing is available on the Maze website, some internet sleuthing turned up information that suggests the average contract is around $12,000 per year. All plan pricing is fully customized, and varies based on the number of studies and number of seats needed.

It’s a hefty cost obviously intended for larger organizations with big research budgets, but it is the plan you must sign up for to get unfettered access to all Maze functionality.

Is Maze Right for You?

Maze is a solid research solution that gives you an array of qualitative and quantitative research tools in one package. It handles prototype testing with ease and leverages the power of AI to help build better studies and streamline data analysis. Where it falls a bit short is its lack of functionality at the lower price points, and the struggle some users have with generating adequate reports or scaling their research.

There are two scenarios where Maze is the easy choice, if your primary focus is prototype testing or you have a big research budget. With the former, you can pick the mid-level plan and trade full functionality for rock-solid prototype testing. With the latter, you’ll have a big monthly expense, but have the full power of Maze available.You can check out Maze by signing up for a free account. You do need to provide your contact information and a little bit of information about your organization to get started, though.