Optimizing global content showcase for WWB using analytic tools and eye-tracking software
-Usability Testing & Eye-tracking study-
Duration
4 months
Team
4 Researchers
Client
Words Without Borders
My Role
-
Conducted 2 Eye-tracking experiments
-
Performed data analysis using Google Analytics & Hotjar
-
Reach out for recruitment of participants
-
Note-taking for teammates
-
Created 3 high-fidelity mockups for recommendations
Overview
Words Without Borders is an online literary magazine that showcases international writing translated into English, fostering cross-cultural understanding and appreciation of global literature. As part of the desktop team, we analyzed the website and conducted an eye-tracking study to find patterns backed by analytical data and behavioral data. Findings in the Multimedia, Search feature and the layout of the page were presented along with recommendations to improve the desktop experience on the website.
Here's our process in a gist:
2
Identify Patterns
2
3
4
5
1
INTRODUCTION
In our first meeting with the client, we discussed the goals of the project
Client goals
🔢
Expanding the number of readers on the webpage
📽️
Audio & Video content offering expansion
🎯
Find friction points for potential users and the connection for return users.
Research goals
🔎
Understand users’ behavior & visual attention
💻
How they navigate for meaningful content
🚀
Improve content showcase & user retention
We began by exploring the analytical data using Google Analytics and Hotjar
Here are some of our initial findings from Google Analytics:
90% of users who land on the website, explore at least one article.
Organic Search accounts for
62% of the user acquisition
23% comes from Direct
Interaction on Home Page:
Rage Click* on the scroll button for the 'Browse by topic' section.
*A rage click can be explained as repeated clicks in the same area of element, showing impatience or anger.
We observed behavioral analytics on Hotjar:
Interaction with the Article Page:
Users are going through the whole article. At least 75% of them reach mid-way and 50% of them read through the whole article.
SETTING UP THE STUDY
With our understanding of the website, we began setting up the study
To conduct the eye-tracking experiment on the website, we first formulated research questions aligned with our research plan objectives. Our recruitment strategy targeted 4 new users and 4 returning users through Google Forms distributed via LinkedIn, the client's newsletter, and email outreach to the Pratt community.
Screening for Eye-tracking study
Interested in reading
Availability to visit in person
Usage of screen readers*
*While testing website accessibility is essential, the software limitations prevented us from including participants who relied on screen readers for navigation.
Any visual impairments
Time to have a mid-point check-in with the client!
Clients Eric & Jessika
Given the project's extensive scope, keeping clients informed was vital to ensure we met the project goals.
As both groups conducted similar studies on different devices (mobile and desktop), presenting our initial findings to the clients was imperative to ensure our alignment with project objectives.
Desktop team
Mobile team
With our test plan and recruitment list ready, we began our eye-tracking study
8
participants
LEGEND
New Users
Current Users
Participant demographics
We recruited 8 participants in the age range of 24 - 44 yrs, around the NYC area who shared interests in reading and literature. Participants had a background in Library Sciences, Graduate student-level design, and Professional Designer roles.
Task list for participants
SCENARIO
Imagine you are interested in reading digital literature, and your friend introduces you to a literature website called “Words without Borders.” You’d like to check out this website and see what it has to offer.
Goal:
Our goal with this task was to identify what the users understood as the product offering and what they take away from the website.
Post-Test Questionnaire - SUS Evaluation
A post-test questionnaire was also developed for additional insight from participants.
SUS metrics provide a reliable and efficient way to assess user satisfaction and identify areas for improvement in the user experience.
Example of SUS Metric
ANALYZING - FINDINGS & RECOMMENDATIONS
We found patterns between the analytics data and the eye-tracking study
Insights from the tests and analytics data were compiled and analyzed to create a comprehensive set of findings and recommendations. Before we look at the problems, let's look at what's going well?
Finding #1
We noticed that users felt that manually browsing through the horizontal grid list required considerable effort
Users found it difficult to recognize the item they were searching for in the horizontal grid layout.
Video: Participant locating item
While the average time for task completion is 0:48 seconds, it took participants 1:10 minutes to complete this task. This indicates that the horizontal grid layout is not efficient in showcasing the content.
Fig. Usability Metric Results
Recommendation #1
We suggested having an interactive map along with intuitive search options to help users find items easily
Using an interactive map allows people to easily select the countries they are looking for.
Adding a search feature eliminates the long scroll action and also ensures that completing tasks of this nature would not depend on the user's geographical knowledge.
Finding #2
We also found that users did not know how to locate multimedia & multilingual content
6/8
participants searched the Navigation bar first, expecting to find Multimedia content.
4/8
participants expected to find Multilingual content in the Navigation Bar
Fig. User searching in the navigation bar expecting to find multimedia and multilingual content
Recommendation #2
We recommended adding Multimedia and Multilingual pages to the sub-menu of the navigation bar to help users easily access them.
Since users were intuitively searching for this content in the navigation bar, it would meet their expectations while searching for related content.
Adding additional information (Audio/Video) next to Multimedia to explain the term better
Finding #3
The labels for multimedia and multilingual are not noticeable.
Looking at results from the eye-tracking study and Hotjar, we found that users are not noticing the tags for multimedia and multi-lingual content. This was a problem since the client was trying to promote the multimedia content.
Reasons for this could be due to its positioning at the bottom or lack of color contrast from the main card.
Fig: The tags for the cards are not the popular heat spots within the page.
A few of the users also mentioned that they were not clear about what multimedia consisted of.
During the Retrospective-Think-Aloud, the users explained how they were not clear about what to expect with multimedia content. They were confused if it was poems, fiction, pictures, or videos.
Recommendation #3
Our recommendation for this problem was to have better surfacing of the tags and to add a description.
By moving the tags up, the user is more likely to notice it thanks to the contrast with the image as well as its priority in positioning.
Adding additional information about the kind of content (audio or video) it encompasses gives users more clarity and expedites their decision-making.
Adding additional information next to Multimedia to explain the term better
To maintain consistency, we also recommend adding language information for multilingual content.
Finding #4
The inconsistent experiences with using filters & search results caused confusion among users
Participant 1: Navigated back to search after using themes because of the absence of filters
Participant 2: Goes back and forth using the filter and search due to confusion and lack of consistency and confirmation from the website
Currently, users can only use filters under the search feature, meaning, other category pages such as fiction, poetry, etc. does not allow for filtering content.
3/8
participants went to theme then used search feature to be able to use the filter options
4/8
participants relied on the filters to confirm they were looking at 'fiction' related to 'dream'
We also noticed that participants saw different results based on how the paths they took to reach the results.
1 FUNNEL
Using search & filters to find content
2 FUNNEL
Using 'Themes' to find content
634 results
2
results
Recommendation #4
We suggested adding a filter option to all the exploratory pages under the Read tab in the Navigation menu.
BEFORE
By adding filters to all the pages such as Fiction, Poetry, Nonfiction, etc., we would be allowing them to more effectively search for the content they are looking for.
Additionally, we suggested having consistency in the results page by adopting one layout for showcasing number of results and the filters applied.
AFTER
To fix the funneling issue, we suggested incorporating the most used keywords of articles to respective themes to broaden user choices and results.
Since one of our key goals was to improve content showcase and user retention, this finding is crucial to be fixed. We believe showcasing more related options will allow users to identify the vast amount of resources available on WWB website.
Note: This is a backend fix, that would require more testing to ensure the correct keywords are being identified and added.
If you are interested in viewing the extensive list of problems and recommendations for this project, follow the link.
REFLECTION
What were some challenges we faced?
-
During the recruitment process, we received over 300+ responses. Out of these 90% were fake responses, leading us to spend time sorting out the potential participants from the bots.
-
We faced a location constraint because the study had to be done in-person @Pratt due to which, many participants were unable to confirm.
What was the client's feedback/response?
-
The client was happy with the presentation and commented that they were excited to implement changes such as the interactive map.
-
The client handoff included the presentation, eye-tracking videos, a problem list, and a one-page visual summary.
Next steps:
-
Creating wire flows out of the wireframes to test the recommendations.
-
Analyzing the usability testing results to identify any usability issues or areas for improvement.