An early prototype of the new WordPress editor was posted a few days ago by the WordPress core editor team. We ran a usability test at the February Brisbane WordPress meetup.
User Test Setup
Participants were presented with the following setup and asked to complete the task as set out in the instructions at left. The changes were to be made in the Gutenberg prototype window on the right.
A total of 5 participants took part in the test. All participants were experienced WordPress developers. You can watch the video footage of the user test session here (24:44).
Observations
The blocks user interface was considered intuitive. The minimalist “Medium” style aesthetic got a positive response.
The placement of the “+” insert button caused some confusion due to its low discoverability. Note that the test setup placed this button out of view. Participants felt that the more natural place for the “+” insert button would be at the start of a new block.
The block controls (up and down arrows) were considered intuitive. A number of participants commented that these would be slow in a larger document (too many clicks needed to reposition blocks). Suggestions were made to add drag and drop capability to the blocks.
A number of participants expected the block type indicator to be a button. From my observations, there are two options that could be further explored with this element:
-
Remove the block type indicator from the left, and add it into the block context menu
-
Remove the block context menu and add it to the block type indicator (which becomes a button)
We should test these options further – I suspect that the block menu and inline contextual menu may be too confusing, making the second option above (or variation thereof) a better choice.
Finally, the inline contextual menu was found to be intuitive.
Learn more
Read our initial thoughts where we ask how the WordPress editor could be designed to give content creators less homework. Also, don’t be afraid to share your thoughts and join the conversation in the Make WordPress #design group.
4 comments. Leave new
[…] Related material Gutenberg project on github – Prototype 1, Prototype 2, Ella’s prototype WordPress #design conversation from last week Observations from user tests of first gutenberg prototype […]
[…] As the user test script contained 18 tasks, the detailed observations and video footage make this for a very long post. If you are after a quick summary, the main takeaways are that participants love the overall minimalist and modern direction of Gutenberg (kudos to all the WordPressers!) and they find the general concept of blocks and contextual menus really intuitive to use, confirming earlier user test results. […]
This is a fascinating area.
I love testing out new pages and the various elements on them – although I have to admit I struggle with the technical side and have to work with our designer. It’s the analytical side of things that really gets me flowing.
But my question is;
How do you know when to stop testing?
This seems to be a never-ending mission and it’s taking a huge amount of my time.
Hi Henry! Great question – the short answer is that you stop testing when you see a pattern, plus a few more. How this equates to number of participants will depend on the type of test that you are running: for example, if you are doing in-person or on-line observational research, I find that patterns emerge after about 5 participants (so I tend to test to around 10 just to be sure). For surveys, the number is much larger and depends very much on how the question was phrased (for this reason, I think that observational style tests are actually a lot more insightful, and overall, cost-effective). Hope that helps 🙂