We put our Tone Analyzer to the test using the Validately service. In this post, we describe what we found.
Setting up the experiment
In October 2016, we ran an unmoderated observational research study to test the intuitiveness and utility of the Tone Analyzer. In this test, 20 participants were given a writing task, and asked to enrich their writing using the Cognitive Assistant.
The task prompted participants to write a blog post about an imagined trip to New York City, and to provide information about:
- The city
- Any attractions, landmarks, shows, movies or restaurants
- Any famous sports teams based in New York City
The participants were asked to adjust their writing tone (if needed) using the Tone Analyzer. The test task intentionally did not prompt the participants on how to achieve the task as we were interested in exploring the natural way that participants would interact with the Tone Analyzer.
What we found
We discovered that 89% of participants were able to intuitively interact with the Tone Analyzer feature. The majority 78% of participants reported that they found the Tone Analyzer useful.
“…So right now, it’s here, it’s Boring. OK. So, how do I make it exciting? [writes more stuff] Ooh! It went to passionate. OK, so it’s like it’s analyzing my whole thing [writing] and…that’s actually a really great idea. I really like the idea of it. I mean, you did not say what it was for, or explain what it was for which is fine, it’s. . . for the user to find out what this is for, which is great.”
Another participant said:
“[opens tone analyzer] Blunt. I really like this . . and that’s the tone I want to be able to take. I like how this tells me about what the tone of my writing is, and what the readers of my blog will think. I think that’s very useful.”
The influence that the Tone Analyzer feature had on the test participants’ editing could not be understated. It helped them become aware of their writing tone and allow them to consider edits, that they did not consider before:
“So mine is blunt. . . that’s so cool. Blunt, trustworthy and joyous. So that’s interesting…Should I fix it? Let me see…”
During our tests, we also discovered an unanticipated use case for the Tone Analyzer:
“Oh cool! It shows a bunch of adjectives. . . and defines what each means. Cool. So, if you wanted to impress someone, like a woman, you could use like all these adjectives at the bottom to make yourself seem smarter. That’s very clever!”
Below is a video summarizing the user test, revealing the sudden awareness that participants had of their own writing tone:
[youtube https://www.youtube.com/watch?v=I8NeIxM5ZA0]
