01. DISCOVERY
We jumped into the start-up mentality with fuguUX, founded in September 2023 with just two founding members. They shortly became a team of four, all usability advocates with an engineering mindset.
In collaboration with Carnegie Mellon University’s MHCI capstone team, they sought to build an automated website usability analyzer through the use of multi-modal LLMs. They envisioned a digital service that automates usability testing for websites and web applications.
But to develop and launch their MVP, they first needed to identify the right customer segments and prioritize the key features to design.
We all know that the web is flooded with usability problems, whether it’s booking a hotel stay or filing your taxes. But fuguUX also believes that persisting usability problems could be costing companies anything from hundreds to thousands of dollars.
(Is this a UX case study about building a UX product so UX teams’ products can have better usability? Yes, this gets real meta.)
So why are these usability problems allowed to persist? Is there a lack of time to fix them? Do they go under-looked? Do companies just not care?
fuguUX initially wanted our team to work within the constraints of their current understanding of the problem and solution. However, I believed it was critical to first step back and question whether we were even solving the right problem.
As UX practitioners, it’s easy to assume that the usability issues we encounter are crucial for companies, teams, and users. But as an advocate for fuguUX’s future users, I needed to first validate whether these pains were real—and if they were significant enough for users to pay for a solution.
Initial Problem Statement
02. Understanding Users and Market Gaps
I quickly recognized that the stakeholders involved in and impacted by usability are deeply complex and interconnected. Early discussions with diverse stakeholders highlighted just how varied UX roles, industries, and companies can be. This led to the question: How do we navigate the ambiguity of our users’ contexts?
We chose to research through design, using concepts and artifacts to provoke meaningful reactions from our users. This approach allowed us to gain a deeper understanding of their needs, pain points, and ultimately, identify our initial customer segment.
Our prototyping approach was hypothesis-driven, rooted in insights from our research. We crafted “magic moments” designed to provoke reactions, allowing us to test and validate our assumptions about user behavior.
.png)
.png)
Overwhelmed PM
Humanistic Researcher
Inexperienced Designer
We identified the core UX team—product managers, researchers, and designers—as our initial customer segment. These users are deeply immersed in web usability and are the most likely to champion it within their organizations. The ways in which they collaborate also presented the greatest opportunity for impactful gains.
Why not engineers or developers? While they’re crucial stakeholders, most developers we engaged prioritized functionality over usability. Even when they valued usability, they weren’t always involved in the key decision-making moments where it mattered most.
Through a competitive analysis of the usability and accessibility landscape, we discovered that tools similar to fuguUX’s proposal were already on the market.
To achieve product-market fit, we knew we had to deliver substantially greater value within the identified opportunity space.
.png)
Sharing Information
Writing usability reports isn’t a problem, but consuming, condensing, and sharing information is.
Persuading Stakeholders
User outcomes and business outcomes are equally important (and the relationship between them must be shown) to persuade stakeholders.
Big Data
Big data inhibits UX teams from efficiently capturing user emotions and frustrations across multiple flows
New Problem Space
03. Exploring and Testing Solutions
We adopted a "fail fast" approach, rapidly generating and testing new concepts. This enabled us to quickly advance successful ideas while discarding those that didn’t resonate. Though we initially faced lukewarm feedback, each iteration brought us closer to solutions that truly met our users’ needs.
Where better to test a usability analyzer than in the heart of a UX conference?
At Config ‘24, we guerrilla-recruited and tested with over 20 participants using two concepts. Since we had limited time, we engaged attendees while they waited in long lines for lunch or took breaks between sessions. Testing in this design-focused environment allowed me to directly observe their reactions, compare the concepts, and gather invaluable feedback.
Synthesizing the test results from our two concepts revealed that the strengths of one could address the weaknesses of the other. Recognizing this, I decided to converge the concepts, as I shifted gears towards building the final prototype.
Here's what I learned:
Quickly access critical usability insights at a glance.
Annotations clearly highlight successes and failures.
Comparative analysis falls short—"apples" can't be compared to "oranges."
Lacks value when free audit tools are available.
Visualize how usability issues interconnect across flows, pages, and problems.
Data-driven insights help persuade stakeholders.
Needs stronger alignment with real users and their needs.
Complexity of information makes it difficult to grasp.
04. Design Hurdles
Problem
Solution
Trusting AI to identify and resolve usability issues.
"This would be great… if I could trust it." This sentiment echoed throughout early iterations of fuguUX. While users saw value in our concepts, they questioned how AI could be trusted to support them in a field that relies so heavily on empathy for critical usability decisions.
Grounding fuguUX in real user context.
By grounding fuguUX's insights in real user data, I transformed the conversations I had with users. With usability information tied to actual user behavior, trust was no longer a concern.
Incorporating key metrics like user count, task success rate, and revenue, along with data visualizations, significantly boosted trust in fuguUX. By building off web analytics data, we improved transparency, making AI-driven decision-making more trustworthy.
Our users want to understand the complexity and messiness of their end-user experience… in a way that isn’t so complex and messy. By generating customer segments from real user behavior and enabling the viewing of user sessions, I built trust in AI while supporting their understanding of the end-user.
Problem
Solution
Finding issues isn’t enough for effective resolution.
Our users needed more than just issue detection—they required deeper insights to fully understand and prioritize usability problems. Finding an issue wasn't enough; they needed a comprehensive understanding to effectively address and resolve it.
End-to-end usability support.
I focused on supporting users holistically across the entire cycle. This approach empowers our users to not only find issues but also to fully understand, prioritize, and effectively resolve them.
Through ongoing scanning, we continuously monitor our user's web product for usability issues and points of struggle.
To effectively resolve issues, UX practitioners needed a deep understanding of their users and the usability challenges they face. I designed features specifically to enhance their ability to grasp their users' contexts, ensuring more informed decision-making.
One of the biggest challenges UX practitioners face is prioritizing usability issues when there are so many to tackle. To address this, I implemented flows to identify the most problematic areas and highlight critical issues. After prioritizing, users receive actionable insights on how to resolve these issues, along with the importance of each issue, enabling more effective conversations with stakeholders.
New Understanding
05. The Solution
By integrating web analytics with usability analysis, Fugu Flowfish gives UX practitioners a clear view of usability at every level. With usability scores, flow insights, and page-specific issues, our platform empowers users to create a better web experience.
"I would rather use this than Heap personally..." - UX Designer
"It would make my job easier when I present my designs to my stakeholders." - UX Designer
Cognizant that fuguUX's status is a start-up with limited funding and resources, we developed a streamlined version of our concept outlining immediate actions, future steps, and long-term goals.
As fuguUX evolves its product, our proposal and feature prioritization is now being integrated into its MVP.
06. Wrap-Up
Take Big Risks
I don’t just seek to validate ideas; I aim to test them in the real market. Feedback like 'Maybe' or 'Yeah, that could work' doesn’t push things forward. So, I’ve learned to take bold risks and get honest reactions, even if it means negative feedback. Those tougher moments reveal what users actually need.
You can always tone down risky ideas, but playing it safe from the start limits growth. By pushing boundaries early, I make sure real user insights drive the design and lead to stronger results.
Designing with AI
AI isn’t just some extra feature—it’s a core design material from start to finish. Figuring out how to use it to actually add value at every step is where the real impact happens.
Plus, AI is still pretty new, and since UX practitioners are right in the middle of that shift, making sure it’s explainable and that users can trust it was a huge takeaway for me. Trust and transparency go hand-in-hand with designing AI experiences.
Making Decisions
The truth is, there are always multiple ways to solve a problem. Just like there are different approaches to tackle global warming, improve public health, or cut down screentime, there are other ways to help UX practitioners make decisions too.
What I’ve learned is that it’s key to plant a flag. Sure, there might be a few right answers, but I know my decisions are backed by real conversations with users and solid data.
