Real-Time Speech-to-Speech Translation: Why Not All “Real-Time” Voice Translation Is the Same

Real-time voice translation is becoming a common feature in meeting platforms, communication tools, and AI applications.

But there is an important detail that is often hidden behind the words “real-time translation.”

Two products may both say they translate speech in real time while using very different technologies underneath.

One may process a conversation like this:

Speech → transcription → text translation → text-to-speech → translated audio

Another can process the interaction as a continuous speech-to-speech experience, where audio is understood and translated directly within a real-time multimodal model.

Both approaches can produce translated speech.

But they do not necessarily create the same conversation experience.

At Voicli, our live voice translation is designed around real-time speech-to-speech translation, allowing participants to speak naturally in their own languages while translated speech is generated for the people listening.

So what exactly is speech-to-speech translation, and why does the architecture matter?

What Is Real-Time Speech-to-Speech Translation?

Speech-to-speech translation (S2ST) is technology that receives spoken language and produces spoken output in another language.

In a live system, this process happens continuously while people are having a conversation.

For example:

A person speaks Ukrainian.

Another participant wants to hear English.

The system listens to the Ukrainian speech, understands its meaning, translates it, and generates English audio for the listener.

The goal is not simply to translate words.

The goal is to make communication between people speaking different languages feel as close as possible to a normal conversation.

This distinction becomes important when we compare modern speech-to-speech systems with traditional voice translation pipelines.

The Traditional Approach: STT → Translation → TTS

For many years, voice translation systems have been built from several separate technologies.

The architecture usually looks like this:

1. Speech-to-Text (STT)
The speaker's audio is converted into written text.

↓

2. Machine Translation (MT)
The written text is translated into another language.

↓

3. Text-to-Speech (TTS)
The translated text is converted back into artificial speech.

The complete pipeline becomes:

Audio → STT → Text → Translation → Text → TTS → Audio

This architecture works.

In fact, it has powered many speech translation systems for years.

It is also useful when the primary goal is transcription, subtitles, searchable meeting notes, or situations where developers need complete control over every processing stage.

But live human conversation introduces another requirement:

speed.

Why a Cascaded Translation Pipeline Can Feel Slow

Every step in a traditional voice translation pipeline takes time.

The speech recognition system needs enough audio to understand what was said.

Then the translation system processes the resulting text.

Then the text-to-speech system generates a new audio stream.

Each individual component may be fast.

But their delays can accumulate.

There is another problem.

Each stage depends on the output of the previous stage.

If speech recognition misunderstands a word, the translation model receives the wrong text.

If the translation changes the meaning, the TTS system simply speaks that translated text.

In other words, errors can propagate through the pipeline:

Speech recognition error → translation error → spoken translation error

During document translation this may be acceptable.

During a live conversation, even a few extra seconds or an awkward pause can completely change how communication feels.

People stop speaking naturally.

They wait.

They check whether the translation has finished.

Then the other person responds.

Instead of a conversation, communication begins to feel like taking turns with a machine.

Native Real-Time Speech-to-Speech Changes the Architecture

Modern multimodal AI models can work with audio input and audio output within the same real-time interaction.

Instead of requiring developers to connect three independent systems for every spoken phrase, a real-time model can continuously process incoming audio and generate spoken output.

From the user's perspective, the flow becomes much closer to:

Speech → AI understanding and translation → Speech

This is generally referred to as real-time speech-to-speech translation.

The important distinction is not that text can never exist anywhere inside an AI model.

Modern models may use many internal representations.

The difference is architectural:

The live conversation does not have to wait for a separate transcription service, then a separate text translation service, and then a separate speech synthesis service before translated audio can begin.

That makes a very different type of interaction possible.

“Real-Time” Does Not Always Mean the Same Thing

The term real-time translation is used broadly across the industry.

A product can technically call itself real-time if it generates a translation a few seconds after somebody speaks.

For subtitles, that may be perfectly acceptable.

But human conversation is much more sensitive to delay.

Consider two experiences.

Experience A: Cascaded Translation

You speak.

The system waits for enough audio.

Your speech becomes text.

The text is translated.

The translated text is sent to a voice generator.

The other participant hears the translation.

Then they respond.

Experience B: Real-Time Speech-to-Speech

You begin speaking.

The model processes the audio stream as part of a continuous session.

It understands the ongoing conversation and begins generating translated speech with significantly less dependency on separate processing stages.

The conversation becomes closer to natural turn-taking.

Both systems can technically be described as “real-time.”

But the experience can be very different.

Why Latency Matters More Than Most Translation Benchmarks

Translation products are often evaluated by accuracy.

Accuracy is obviously important.

But during live communication there is another equally important dimension:

conversation latency.

Imagine a business meeting with six people speaking several languages.

If every translated sentence introduces several seconds of silence, those delays multiply across the meeting.

Participants start interrupting translations.

Some people begin talking before previous translations have finished.

Others stop participating because following the conversation becomes difficult.

The translation itself may be correct while the meeting experience is still poor.

For live multilingual communication, translation quality therefore cannot be measured only by whether the words are correct.

It should also consider:

• how quickly translated speech begins;

• how naturally participants can take turns;

• whether interruptions can be handled;

• whether the system understands conversational context;

• whether users need to wait for complete sentences;

• how well the conversation flows across multiple languages.

This is why real-time speech-to-speech technology is especially important for meetings.

Speech Contains More Than Words

Another limitation of converting speech into plain text is that human speech contains information that is difficult to represent completely in a transcript.

People communicate through:

• tone;

• rhythm;

• pauses;

• emphasis;

• hesitation;

• speaking speed;

• conversational context.

Traditional STT systems first compress the spoken message into text.

Some information from the original audio can therefore be lost before translation even begins.

Modern speech-to-speech models can work directly with audio and have access to more of the original speech signal.

This does not mean every nuance will always survive translation.

But it creates the possibility of much more natural voice interactions than systems built only around text.

Speech-to-Speech vs Speech-to-Text Translation

These technologies are sometimes confused, but they solve different problems.

Speech-to-text translation converts spoken language into translated text.

For example:

Japanese speech → English subtitles

This is extremely useful for captions and accessibility.

Speech-to-speech translation produces translated audio:

Japanese speech → English speech

This allows the listener to follow the conversation without constantly reading subtitles.

Voicli supports live subtitles as part of the meeting experience, but voice translation is designed for something broader: allowing people to actually talk across languages, rather than simply read translations of what others said.

Why Voicli Uses Real-Time Speech-to-Speech Translation

Voicli was built around a simple idea:

People should be able to join the same meeting and speak the language in which they are most comfortable.

One participant might speak Ukrainian.

Another might speak Japanese.

Another might prefer English.

Another might speak Korean.

Instead of forcing everyone to switch to a common second language, Voicli handles translation during the conversation.

Each participant can select the language they want to hear.

The platform can provide translated voice, subtitles, or the original audio depending on the listener's preference.

For live voice translation, Voicli uses a real-time speech-to-speech architecture rather than relying exclusively on the traditional:

STT → text translation → TTS

pipeline for every translated utterance.

That architectural difference is important because Voicli is not trying to build a transcription tool that happens to speak translated text.

The goal is to build a multilingual communication layer for real conversations.

When STT + Translation + TTS Is Still the Right Solution

Speech-to-speech technology does not make traditional pipelines obsolete.

There are many situations where STT → translation → TTS is useful or even preferable.

For example:

Meeting transcripts need accurate text.

Searchable conversation history requires text representation.

Subtitles naturally depend on transcription.

Compliance systems may need to store exactly what was recognized.

Asynchronous translation does not require extremely low latency.

Specialized workflows may need developers to inspect and modify every intermediate stage.

That is why Voicli also uses different AI technologies depending on the task.

The key is choosing the correct architecture for the experience being built.

For a transcript, text is the product.

For a live multilingual conversation, the conversation itself is the product.

From Translation Tools to Multilingual Conversations

For decades, machine translation has primarily been about converting information from one language into another.

First documents.

Then websites.

Then messages.

Then subtitles.

Real-time speech-to-speech translation represents a different direction.

Instead of asking:

“How quickly can we translate this sentence?”

the more important question becomes:

“Can two people who speak different languages communicate naturally?”

That changes how translation systems need to be designed.

Latency becomes critical.

Turn-taking becomes critical.

Audio quality becomes critical.

Context becomes critical.

And translation becomes part of the communication infrastructure rather than a separate tool people consciously use.

The Future of Multilingual Meetings

The long-term goal of speech translation is not to make people think about translation technology more often.

It is the opposite.

Translation should gradually disappear into the conversation.

You speak your language.

The other person speaks theirs.

And the technology handles what happens between you.

There will still be challenges.

Accents, background noise, domain-specific terminology, overlapping speakers, names, and cultural context can all affect AI translation.

But real-time multimodal models are changing what is technically possible.

We are moving from:

record → transcribe → translate → synthesize → listen

toward:

speak → understand → translate → hear

And eventually, the most successful translation technology may be the technology participants barely notice.

Real-Time Translation Should Feel Like Communication

Calling a product “real-time” is easy.

Making multilingual communication actually feel real-time is much harder.

The difference is not just milliseconds on a benchmark.

It is whether people can speak naturally.

Whether they can respond without waiting.

Whether a multilingual meeting feels like a conversation rather than a sequence of translated messages.

That is the problem Voicli is working on.

Real-time speech-to-speech translation for conversations where everyone can speak their own language.

Frequently Asked Questions

What is speech-to-speech translation?

Speech-to-speech translation converts spoken language directly into spoken output in another language. Modern AI systems can process audio input and generate translated audio within a real-time session.

What is the difference between speech-to-speech and speech-to-text translation?

Speech-to-text translation produces translated written text, such as subtitles. Speech-to-speech translation produces translated audio that the listener can hear.

How does traditional voice translation work?

Traditional systems commonly use three stages: speech-to-text recognition, machine translation, and text-to-speech synthesis. This is often called a cascaded speech translation architecture.

Why can STT → translation → TTS have higher latency?

Each stage must process information and pass its output to the next component. The accumulated processing time can introduce additional delay, while errors from one stage can also affect subsequent stages.

Is speech-to-speech translation the same as simultaneous interpretation?

They are related concepts but not identical. Human simultaneous interpretation is performed by professional interpreters. AI speech-to-speech translation attempts to provide a similar low-delay multilingual communication experience automatically.

Can speech-to-speech translation be used for business meetings?

Yes. Real-time speech-to-speech translation is particularly useful for international meetings, customer calls, sales conversations, remote teams, interviews, and other situations where participants do not share the same language.

What does Voicli do?

Voicli is a multilingual video communication platform with real-time voice translation and live subtitles. Participants can speak their preferred languages while Voicli handles translation during the conversation.

Try Voicli and speak your language. Let the meeting handle the translation.