Designing audio-first reading experience

With text-to-speech models now widely available, voice interfaces are becoming increasingly common. I’ve started using tools like Super Whisper to dictate my messages and Voice Notes to document random ideas while away from computer. It’s not like I stopped using the keyboard entirely, but the shift towards audio-first interfaces is clear.

However, when it comes to my reading list, I’m still a long-time Pocket user. They do have basic text-to-speech, but its quality isn’t good enough for any serious “reading”. Now, imagine my shock when I learned that Mozilla shuts down Pocket later this year.

This got me thinking - what would be an audio-first reading experience? I’m quite certain it’s not just converting raw text into speech with AI. So I want to document the journey of building Katalog, the audio-first read-it-later app, and what I learned designing interfaces for an audio-first. Focusing on how UX and UI should change when there are no visuals to support the experience.

Why basic text-to-speech isn’t enough

Many existing read-it-later apps, like Pocket or Instapaper, have basic text-to-speech. While they technically convert text to audio, I found they don’t provide a good listening experience.

It’s hard to understand the article this way because the text, which is meant for visual consumption, is being directly translated into sound. Things like section breaks, list indicators, images are missing. All the structure and implicit affordances of written words is gone.

Listen to the text below and try to understand what’s going on without looking at it.

Imagine this list above containing 10 items instead of 4. Simply converting text to audio wouldn’t work. An audio-first approach means breaking down each element of an article and thinking how to convey its meaning and structure through sound.

Audio design examples

Translating the visual language of articles into an understandable audio format presented several design challenges. My approach was mostly “trial and error”, saving a bunch of articles, listening to the narration, noticing confusing parts, and then iterating on different solutions.

Lists (Especially Nested Lists):

Visually, lists use indentation and bullet points or numbers to show hierarchy and structure. It’s easy to scan and understand quickly.

In audio, a simple narration of list items falls flat. It’s just a sequence of spoken sentences. For a simple list, this might be okay, but for nested lists with multiple levels of indentation, it becomes impossible to understand the relationship between items just by listening.

I first tried adding a voice instruction before each item, like “List item one,” “List item two.” Or enunciating the beginning of the list and its end:

For simple lists, this was just annoying because it quickly became overly repetitive. For nested lists, it still didn’t clearly communicate the structure or the depth of the indentation.

My current solution involves using a subtle sound cue – a chime – played before each list item. The sound indicates that a new list item is starting.

For nested lists, I’m also exploring varying the sound slightly or repeating it to signify the indentation level. For example, one chime for the first level, two for a sub-item, and so on.

This way we use sound design to provide an “audio affordance,” a cue that signals the structure of the content, similar to how visual indentation works.

Images:

In an audio-only experience, images are completely inaccessible unless described in some way. Relying solely on the alt attribute is a start, but these are often too brief or missing entirely.

The first immediate solution was to check for the alt description in the HTML and narrate it after a short voice instruction like, “Here’s an image.” However, for more complex images like charts, diagrams, or screenshots, a short alt text isn’t enough. For these cases, I use AI to generate a more detailed description of the image’s content, focusing on the key information the image is meant to convey. The prompt needs a lot of tweaking, though. AI often over-emphasizes the style of the image over its content.

If the article includes a figcaption element with a detailed description written by the author, I prioritize narrating that after an instruction like, “Here is the image’s description.” So that the listener doesn’t miss out on important information presented visually.

Code Blocks:

Code blocks are a significant challenge. Visually, code has syntax highlighting, indentation, and structure that help with reading it.

Plainly narrating code line by line would be incredibly unpleasant to listen to. It wouldn’t be accessible or useful for understanding the code’s purpose within the article’s context.

I haven’t fully solved this yet, but I’m exploring using AI, similar to how I handle complex images. The idea is to have AI analyze the code block and generate a summary or explanation of what the code does or what key concept it illustrates, rather than narrating the code itself. This would allow the listener to get the main point the author was making with the code example.

Links:

Naturally, links are visually interactive elements. In audio, narrating the URL is useless. The listener can’t do anything with it immediately.

Eventually I decided not to narrate links directly within the article flow. Instead, I’d collect all the links from the article and provide them in a separate reference section that the user can access after listening. While this means the listener doesn’t know when a link is mentioned during narration, it avoids interrupting the listening experience with irrelevant information.

Quotes:

Blockquotes are visually set apart to highlight specific text. Translating them to audio was relatively simple. I use a brief voice instruction like, “Begin quote,” before narrating the quoted text, and then “End quote,” afterwards.

This clearly delineates the quoted section for the listener without being overly disruptive.

Beyond the narration

Katalog is still in its early stages, and there’s a lot to figure out here. But after the initial exploration, I can clearly see where this audio-first reading experience can go next.

Notes and highlights:

One area I’d love to explore is bi-directional audio communication. While I’m listening to an article, I often want to record my own thoughts, notes, or highlights using my voice, without pausing or interrupting the narration flow too much. This would make the listening experience more active and integrated into a workflow.

Expressive voices:

Another exciting possibility is using AI to implement variable tone and voice expressions. OpenAI’s text-to-speech API, for example, allows setting an emotional tone. I envision analyzing the article’s content to determine the most appropriate voice and emotional style for the narration. A scientific paper could have a calm, informative tone, while a dramatic piece might be narrated with more expression. This moves beyond a single monotone voice towards a more engaging and contextually rich listening experience.

Content discoverability:

Finally, I’d love to make the audio versions of articles created in Katalog more accessible. Imagine subscribing to articles from Katalog in your favourite podcast app. You also wouldn’t need to choose which articles you’ll read that day. They’d just arrive to you automatically based on all the content you saved before.

Summing up

Designing for audio consumption first is a different way of thinking about content and user experience. It’s not about adding text-to-speech as an extra button. It’s about fundamentally questioning how information, structure, and meaning can be conveyed solely through sound.

If you found examples above useful, you can follow @usekatalog on X or try the current version of Katalog web app.