Are you in (voice) control?

Software that is used to transform Speech To Text is often abbreviated as STT or ASR (Automatic Speech Recognition). You can use speech recognition offered by your operating system (OS) or you can use dedicated commercial or open source software that you install on your computer. For your mobile device, there is GBoard (from Google) and SwiftKey (from Microsoft), two digital keyboards that supports speech recognition in many different languages.

The OS usually does not need to send data to an external website to help with the speech recognition. This is better from a privacy point of view, but as a result, the OS usually limits the number of supported languages. E.g. on Windows 11, there is no STT support for Dutch, the language that I write in. The digital keyboards on my mobile phone have a wider support for different languages, but data is sent to the servers of big tech to help with the speech recognition, and who knows what else they do with that data?

Lastly, there is a browser web API that adds speech recognition capabilities to your web browser. At the time of writing, this is supported by Google Chrome, Microsoft Edge and Apple Safari. Unfortunately, Firefox lacks support in this area. This web API allows applications that run in the web browser to offer support for speech recognition directly. But I have not seen any online writing tool implement it yet, also not digital-ink.app. I also have not seen any good plugins that offer STT for writing tools; plugins available seem to be targeted more on transcribing online meetings. On the other side, you can still dictate in these browser apps via your operating system’s STT support (for desktop and laptop computers) or via the digital keyboard apps (for handheld devices).

With the OS supplied software, you have support for a limited number of languages. The software is designed to support both dictation and controlling your computer with voice commands. This means: more complexity, higher change of miscommunication. I tried dictating a simple English text on Windows 10 and it was a disaster. The error rate is sky-high and the software clearly did not apply grammar checks to improve the speech recognition. Fortunately, the test on Windows 11 performed a lot better. But in the end, this is not the right choice for me, because, as I said, Dutch is not supported.

One of the benefits of writing on your mobile device is that you can carry it around everywhere, so you could turn waiting time into a little creative writing party. I tried the digital keyboards from both tech giants Microsoft and Google on my mobile phone. The speech recognition in those applications heavily depends on AI and that, in turn, depends heavily on the training of the underlying models. This means that the accuracy may differ per language and per vendor. Moreover, the results may change over time as more training is applied and the underlying AI models may evolve. But there are also differences in usability of the keyboard apps, so it is really worthwhile to give them both a try.

I dictated on the web browser of my mobile phone to write scenes with digital-ink.app under different circumstances. I dictated in Dutch and English, in a silent environment, in a parked car with background traffic noise and one time also with background music. I even tried whispering and applying a heavy accent.

Overall, the results were not bad. I think the recognition is good enough to write your first draft, provided that you carefully articulate. It helps when you first think of the entire sentence, before you say it out loud. The STT will produce errors, and when reading back, it may even be hard at times to recognize what you intended, so you will probably be spending a bit more time editing your scenes in the second draft, compared to a typed first draft.

In the usability department, I found significant differences between GBoard and SwiftKey. Especially in the area of inserting punctuation and microphone usage. But this is also an area of personal preferences, so please discover your own favorite tool. I liked it when Swiftkey beeped when the microphone stopped listening, so I did not have to watch my display as closely as with GBoard. I also had the impression that GBoard automatically switched off the microphone more frequently, and I found that annoying.

With Swiftkey, I could insert the basic punctuation that I need when writing: inserting a period, comma, double quote, question mark or exclamation mark worked well enough. With GBoard, I could not make it insert double quotes, which is not really helpful when writing dialog. And with both of them, I never succeeded starting a new paragraph or new line, so be prepared that you regularly have to fall back to typing.

If speed of words spoken > ( speed of words typed + extra time editing), then dictation offers productivity gains. And if it allows you to write more often, because you can do it in more situations and locations, that is also a productivity gain. But there is more. When I was experimenting with speech recognition, I noticed that I needed to focus less on the words in my editor in front of me. While dictating, my mind could be wondering off and that allowed for more creativity during the writing process. I did not expect that to happen. And therefore, I would advise every writer to experiment with dictation themselves.

See you at the next post, and until then: happy writing.