MDN user testing for "Command & Query" Search
I recently finished a round of user testing for the Mozilla Developer Network's "Command & Query" search UI, the goal of which is a better way for developers to search and filter content on MDN. This started off as a hackday project in Paris and is now functioning behind a waffle flag on MDN. We knew this prototype could be improved, so I ran a series of test using usertesting.com to watch and listen to users as they discovered the new UI and walked through a series of tasks. Below is a summary of what I learned and the design recommendations that followed. Follow the tracking bug to stay up to date with the changes. [ bug 1182261 ] * videos from this user test are private, detailed presentations slides with additional test outcomes available on request.
What & Who did we test?
We had a high-level goal of learning from successful and unsuccessful elements of our search filter UI and use key findings to determine next steps to update and improve. While focusing mainly on the Command & Query search UI, users often went through an entire search flow from search > search results > wiki/docs page. They also touched on elements in the navigation and within our page templates that was useful to improving the overall design of MDN.
MDN focuses mainly on an intermediate to advanced developer audience. For this reason, I screened for users who self-identified as technically savvy users. Some coding knowledge was a plus and present in 60% of users, though typically ended up being between beginner and intermediate level. 70% of users used internet on a desktop or laptop computer for over 10 hours each day. 80% were male and 70% were using Windows. 80% of users were in the 18-25 yr age range.
5 Key findings for Search UI
- Users figured out value of filtering when encountering UI, but not until menu was opened
- When asked a leading question about filtering results, 90% of participants explored the down arrow to find these options.
- Users commented that search field should more clearly identify its functionality
- Users expected content at top of search results to reflect filter chosen in search field from prior step
- Users commented that # symbol was recognizable for filtering or tagging content (and easier than other filtering syntax - i.e., Gmail search) that they had used.
Based on user feedback, the following recommendations balance the lowest amount of dev and design work for the greatest outcome to improve the UI. Bugs have been filed for the following:
- Add magnifying glass icon to more clearly identify search bar functionality.
- Add tool-tip onboarding outside of field to point out new features and describe its value to the user before menu it opened.
- Reflect chosen filters on search results page. (in field at top of page and in right column)
- Once filter is chosen (either manually from menu or w/ # quick-command) alter instructional field text to say "add another term or filter" (exact copy tbd)
- Restrict users' global search history from populating automated search field terms. (user found this not valuable and sometimes intrusive - bonus: auto-populate from a library of key MDN terms)
- Explore new filter icon for down arrow (low priority as combined other changes in UI may render this unnecessary)
- A few additional bugs were found as users navigated through tasks. Bugs have been filed to solve, some of which are not limited to the search UI. Additional design recommendations pertaining to overall MDN website have been communicated and will be prioritized as a separate project.
- Present and discuss results with MDN team
- File bugs agreed on as a team
- Identify further user testing needed before considered ready to launch
- Once prioritized updates are implemented, review quality with leads and stakeholders
- Consider any further work necessary before launch
- Once launched, display post-launch survey on MDN to both introduce search UI to users and also ask for their feedback