Realtime Speech-to-Text API

Real-time speech-to-text streaming over a single WebSocket connection.

This endpoint is a dedicated transcription stream: you push raw audio frames in and receive incremental transcriptions and, optionally, translations back as JSON.

Palabra API client

Consider using the Palabra API Python client and checking the code example with it.

Step 1. Get an API Key

Create an API Key on the Palabra API Keys page. See Authentication for details.

Step 2. Connect

Open a WebSocket to the endpoint below, passing your API Key as the token query parameter (or in the Authorization header). The server validates the key and creates a streaming session for the lifetime of the connection automatically. All other stream settings are passed as query parameters in the same URL.

wss://stream.palabra.ai/asr/v1/speech-to-text/stream?token=<API_KEY>&language=en&format=pcm_s16le&sample_rate=16000

import websockets

url = (
    "wss://stream.palabra.ai/asr/v1/speech-to-text/stream"
    f"?token={api_key}&language=en&format=pcm_s16le&sample_rate=16000"
)
ws = await websockets.connect(url)

Query parameters

Parameter	Required	Description
token	yes	Your API Key
format	yes	Audio format (see Audio formats)
sample_rate	conditional	Sample rate in Hz. Required for all raw PCM formats; for pcm_s16le required only when the rate is not 16000
language	no	Source language code. Defaults to auto
translate_languages	no	Comma-separated target languages, e.g. es,de,fr
enable_filler_filter	no	Whether to enable the filler filter. true by default for all languages except ja
finalization_mode	no	auto (default) or manual — see Finalization modes
finalization_timeout	no	Manual mode safety-net timeout in seconds (default 60)

turn_detection and turn_timeout are deprecated aliases of finalization_mode and finalization_timeout.

Supported languages

Code	Language
ar	Arabic
de	German
en	English
es	Spanish
fr	French
hi	Hindi
it	Italian
ja	Japanese
ko	Korean
nl	Dutch
pt	Portuguese
ru	Russian
zh	Chinese

Automatic source language detection (language=auto) is supported in experimental mode.

Step 3. Send audio

Send audio as raw binary WebSocket frames. Chunks of 320 ms are recommended.

await ws.send(data)

To finish the session, send an empty frame — see Ending the stream.

The finalize command

Besides binary audio, the client may send a JSON text frame with the finalize command — a manual end-of-segment:

{ "message_type": "finalize" }

It works in both finalization modes: the recognizer finalizes everything received so far and emits the pending segment as final (is_eos: true); recognition then continues as usual. Send it whenever your application knows the speaker's turn is over (push-to-talk release, your own turn detection), or after the last byte of a pre-recorded file (which often ends abruptly, with no trailing silence — so automatic, silence-driven segmentation would never finalize the tail). If the session ends there, end the stream instead: that finalizes the tail too.

The <fin> marker

Every finalize is answered. The final transcript that answers it carries the <fin> marker appended (without a space) to its text — that is how the client knows up to which point the transcription has been finalized:

{ "message_type": "transcription", "is_eos": true, "segment": { "text": "Hello world.<fin>", ... }, ... }

If the finalize produced no text (nothing new was recognized yet), the server still answers with a final transcription message (is_eos: true) whose text is just the marker:

{ "message_type": "transcription", "is_eos": true, "segment": { "text": "<fin>", ... }, ... }

The marker never appears in translated_transcription messages.

Ending the stream

To finish a session gracefully, send an empty WebSocket frame (binary or text) after the last audio chunk:

await ws.send(b"")  # or: await ws.send("")

The server then:

1. transcribes all audio received before the empty frame, including an unfinished phrase at the very end — no finalize is needed;
2. sends the remaining transcription messages, the last one final (is_eos: true), followed by its translated_transcription messages when translate_languages is set;
3. sends a stream_finished message and closes the connection with close code 1000 (normal closure).

This usually takes under a second (a bit longer with translations). Audio and finalize commands sent after the empty frame are ignored; a finalize sent before it that is still unanswered gets its <fin> answer before stream_finished.

await ws.send(b"")  # no more audio
async for message in ws:  # the last results, then stream_finished; the server closes the connection
    msg = json.loads(message)
    if msg["message_type"] == "stream_finished":
        print("done:", msg["final_audio_proc_ms"], "ms of audio transcribed")

Closing the connection (or sending end_task) without the empty frame drops the audio that has not been finalized yet.

Finalization modes

Segmentation ("when is the phrase finished?") is controlled by the finalization_mode query parameter:

• auto (default) — segments are finalized automatically. This is the current behavior and requires nothing from the client.
• manual — automatic finalization is suppressed: the client decides where turns end and sends the finalize command. As a safety net, if no finalize arrives within finalization_timeout seconds (default 60) since the last segment end, the nearest automatic end-of-segment signal is let through once; every finalize (and every delivered final segment) restarts the timer, so a client that keeps finalizing never hits it.

Use manual when your application has better knowledge of turn boundaries than the audio itself — push-to-talk UIs, external diarization/turn-taking logic, or benchmarks that need reproducible segmentation.

Audio formats

format	sample_rate	Notes
pcm_s16le	only if ≠ 16000	16-bit signed little-endian PCM. Recommended
pcm_f32le / pcm_f32be	required	32-bit float PCM
pcm_s32le / pcm_s32be	required	32-bit signed PCM
mulaw / alaw	required	G.711
webm / mp3 / aac / ogg / flac / wav	not used	Container formats; rate is read from the stream

Step 4. Receive messages

All server-to-client messages are JSON text frames. Switch on message_type.

transcription

Emitted continuously as speech is recognized.

{
  "message_type": "transcription",
  "transcription_id": "a1b2c3d4",
  "language": "en",
  "is_eos": false,
  "segment": {
    "text": "Hello world how are",
    "start_time": 0.32,
    "end_time": 1.84
  },
  "delta": {
    "text": "how are",
    "start_time": 1.20,
    "end_time": 1.84
  }
}

Field	Description
transcription_id	Stable id for the segment. All messages of one segment share the same id. A new id means a new segment has started
language	Detected (or configured) source language of this segment
is_eos	false — partial; the segment is still being updated. true — the segment is committed and final
segment.text	The full text of the segment so far
segment.start_time / end_time	Segment timing, in seconds relative to session start
delta	Incremental hint: the text added since the previous partial of the same segment (see below)

Working with delta

delta.text is append-only within a segment: each transcription message carries exactly the text appended since the previous partial of the same transcription_id, so you can concatenate deltas directly.

This holds with the filler filter enabled as well. A partial whose last word may still turn out to be a hesitation sound (for example um, эм) is held back until the next word or punctuation confirms it; the filler is then removed before the text is emitted, so previously delivered text is never rewritten. The final (is_eos: true) message carries the recognizer's text as is (no capitalisation or punctuation is added), so concatenated deltas match segment.text byte for byte.

translated_transcription

Sent only when translate_languages is set, once per target language, after each final (is_eos: true) transcription.

{
  "message_type": "translated_transcription",
  "transcription_id": "a1b2c3d4",
  "language": "es",
  "is_eos": true,
  "segment": {
    "text": "Hola mundo, ¿cómo estás?",
    "start_time": 0.32,
    "end_time": 1.84
  }
}

transcription_id matches the id of the source transcription (the is_eos: true one) this translation was produced from — use it to correlate a translation back to its original segment. language here is the target language, and is_eos is always true (translations are produced only for finalized segments).

stream_finished

The last message of a session ended with an empty frame (see Ending the stream); the server closes the connection right after it.

{
  "message_type": "stream_finished",
  "total_audio_proc_ms": 12480,
  "final_audio_proc_ms": 12480
}

Field	Description
total_audio_proc_ms	Audio, in milliseconds, processed into final and partial results
final_audio_proc_ms	Audio, in milliseconds, processed into final (is_eos: true) results

Both normally equal the duration of the audio sent: everything has been finalized. A final_audio_proc_ms below total_audio_proc_ms means the recognizer did not finish the last segment in time and its tail may be missing.

---

Errors

Authentication and routing failures are reported as HTTP status codes during the WebSocket upgrade, before the connection is established:

HTTP status	Meaning
401	Missing or invalid API Key / token
409	A session is already active for this identity

After a successful upgrade, the server does not send application-level error messages over the wire — it closes the connection with a standard WebSocket close frame.

---

Complete examples

Microphone, automatic finalization

Streams microphone audio and prints transcriptions (and translations, if PALABRA_LANGUAGE targets are configured).

pip install pyaudio websockets
export PALABRA_API_KEY=...        # from Step 1
export PALABRA_LANGUAGE=en        # source language

import json
import os
import asyncio
import threading
import queue

import pyaudio
import websockets

WS_URL = "wss://stream.palabra.ai/asr/v1/speech-to-text/stream"
LANGUAGE = os.environ.get("PALABRA_LANGUAGE", "en")

SAMPLE_RATE = 16000
CHANNELS = 1
CHUNK = 5120  # samples ≈ 320 ms at 16 kHz (recommended chunk size)


def mic_reader(audio_queue: queue.Queue, stop_event: threading.Event):
    pa = pyaudio.PyAudio()
    stream = pa.open(
        format=pyaudio.paInt16,
        channels=CHANNELS,
        rate=SAMPLE_RATE,
        input=True,
        frames_per_buffer=CHUNK,
    )
    print("Microphone open, speak now...")
    try:
        while not stop_event.is_set():
            audio_queue.put(stream.read(CHUNK, exception_on_overflow=False))
    finally:
        stream.stop_stream()
        stream.close()
        pa.terminate()


async def stream(token: str):
    url = (
        f"{WS_URL}?token={token}&language={LANGUAGE}"
        f"&format=pcm_s16le&sample_rate={SAMPLE_RATE}"
    )

    audio_queue: queue.Queue = queue.Queue()
    stop_event = threading.Event()
    threading.Thread(
        target=mic_reader, args=(audio_queue, stop_event), daemon=True
    ).start()

    async with websockets.connect(url) as ws:
        print("Connected")

        async def send_audio():
            loop = asyncio.get_event_loop()
            while True:
                data = await loop.run_in_executor(None, audio_queue.get)
                await ws.send(data)  # raw binary frame

        async def receive():
            async for message in ws:
                msg = json.loads(message)
                msg_type = msg.get("message_type")

                if msg_type == "transcription":
                    text = msg["segment"]["text"]
                    tid = msg.get("transcription_id", "")
                    if msg.get("is_eos"):
                        print(f"\n[EOS] {text} [{tid}]")
                    else:
                        # segment.text is the source of truth — render it whole
                        print(f"\r      {text}", end="", flush=True)

                elif msg_type == "translated_transcription":
                    lang = msg.get("language", "?")
                    tid = msg.get("transcription_id", "")
                    print(f"\n[{lang}] {msg['segment']['text']} [{tid}]")

        try:
            await asyncio.gather(send_audio(), receive())
        finally:
            stop_event.set()


if __name__ == "__main__":
    try:
        asyncio.run(stream(os.environ["PALABRA_API_KEY"]))
    except KeyboardInterrupt:
        print("\nStopped.")

Pre-recorded file, manual finalize

Streams a 16-bit mono WAV file in finalization_mode=manual, sends the finalize command after the last chunk, and exits once the answering final transcription (marked with <fin>) arrives. A pre-recorded file usually ends abruptly, with no trailing silence, so without the finalize the tail of the recording would never be finalized.

pip install websockets
export PALABRA_API_KEY=...        # from Step 1
export PALABRA_LANGUAGE=en        # source language
python transcribe_file.py speech.wav

import asyncio
import json
import os
import sys
import wave

import websockets

WS_URL = "wss://stream.palabra.ai/asr/v1/speech-to-text/stream"
LANGUAGE = os.environ.get("PALABRA_LANGUAGE", "en")

CHUNK_MS = 320  # recommended chunk duration


async def transcribe_file(token: str, path: str):
    wav = wave.open(path, "rb")
    assert wav.getnchannels() == 1 and wav.getsampwidth() == 2, "need 16-bit mono WAV"
    sample_rate = wav.getframerate()
    chunk_frames = sample_rate * CHUNK_MS // 1000

    url = (
        f"{WS_URL}?token={token}&language={LANGUAGE}"
        f"&format=pcm_s16le&sample_rate={sample_rate}"
        "&finalization_mode=manual"
    )

    async with websockets.connect(url) as ws:
        print("Connected")

        async def send_audio():
            while True:
                data = wav.readframes(chunk_frames)
                if not data:
                    break
                await ws.send(data)  # raw binary frame
                await asyncio.sleep(CHUNK_MS / 1000)  # real-time pace
            # The file is over — no trailing silence will ever arrive,
            # so tell the recognizer the turn is done:
            await ws.send(json.dumps({"message_type": "finalize"}))
            print("\n(finalize sent)")

        async def receive():
            async for message in ws:
                msg = json.loads(message)
                if msg.get("message_type") != "transcription":
                    continue
                text = msg["segment"]["text"]
                if msg.get("is_eos"):
                    print(f"[final] {text}")
                else:
                    print(f"\r        {text}", end="", flush=True)
                if "<fin>" in text:
                    # the answer to our finalize — everything sent
                    # before it has now been transcribed
                    return

        await asyncio.gather(send_audio(), receive())


if __name__ == "__main__":
    asyncio.run(
        transcribe_file(os.environ["PALABRA_API_KEY"], sys.argv[1])
    )