Glass TTS & ASR

Purpose

Describe offline TTS playback, cloud ASR capture, and offline wake-word registration paths on the glasses.

Entry points

Offline wake-word registration:

• HomeActivity

TTS / ASR tap targets:

• SendMessageActivity

Platform

• Glass

Main files

• HomeActivity.kt
• SendMessageActivity.kt

Walk the sources (recommended order)

1. HomeActivity.initSDK() — VoiceAction registration + GlassSdk.getGlassOfflineCmdService()?.add(...)
2. SendMessageActivity.toClick() — R.id.btTts, R.id.btAsr branches
3. SpeechCallback.Stub() overrides for streaming events

Related docs

• Glass3 Glass SDK API reference
• Quick start

Feature coverage

• Offline wake-word lexicon install
• Offline TTS playback
• Cloud ASR sessions
• Speech callback matrix (partial + final + intent JSON)

Snippets

Offline wake-word

jdVoiceAction = VoiceAction(
    "Turn on beacon #12 strobes",
    "da kai bian hao shi er de jing deng",
    object : IVoiceCallback.Stub() {
        override fun onVoiceTriggered() {
            Log.e(TAG, "Wake phrase: beacon #12")
        }
    }
)
GlassSdk.getGlassOfflineCmdService()?.add(jdVoiceAction)

Offline TTS

val str = "Autumn stayed away—I'm off to climb the trail"
GlassSdk.getGlassOfflineTtsService()?.playTtsMsg(str)

Cloud ASR

GlassSdk.getGlassAsrService()?.startSpeech(object : SpeechCallback.Stub() {
    override fun onStart() {}

    override fun onIntermediateVad(content: String) {}

    override fun onAsrComplete(content: String?) {}

    override fun onAsrCompleteWithIntent(
        content: String?,
        intent: Int,
        intentJson: String
    ) {}

    override fun onError(code: Int) {}
})

Flow outline

1) Register wake-words

After SDK init, HomeActivity builds VoiceAction objects and calls GlassSdk.getGlassOfflineCmdService()?.add(...). Upstream demos ship a handful of scripted wake phrases (siren/light control, snowfall, branded trigger, etc.—see upstream source strings).

2) Offline TTS tap path

SendMessageActivity routes the TTS button to GlassSdk.getGlassOfflineTtsService()?.playTtsMsg(...). Online TTS hooks exist but remain commented in source.

3) ASR tap path

The ASR button launches GlassSdk.getGlassAsrService()?.startSpeech(...) with the following callbacks:

• onStart()
• onIntermediateVad(content)
• onAsrComplete(content)
• onAsrCompleteWithIntent(...)
• onError(code)
• onServiceConnectState(connect)

Sequence sketch

sequenceDiagram
    participant U as User
    participant G as Glass UI
    participant ASR as Speech service

    U->>G: Trigger ASR
    G->>ASR: startSpeech(callback)
    ASR-->>G: onStart()
    U->>ASR: Speak
    ASR-->>G: onIntermediateVad(content)
    ASR-->>G: onAsrComplete(content)
    ASR-->>G: onAsrCompleteWithIntent(...)

Offline vs online matrix

Topic	Offline wake	Cloud ASR	Offline TTS
Intent	Hot command dispatch	Free dictation	Audible prompts
Network	None	Required	None
Output	Callback fire	Partial + finals + intent JSON	Audio stream
Sample host	HomeActivity	SendMessageActivity	SendMessageActivity
Best for	Low-latency macros	Subtitles, Q&A	Local feedback

FAQs

Offline vs online TTS

Sample highlights offline TTS so benches work air-gapped—flip commented online hooks once credentials validate.

onIntermediateVad vs onAsrComplete

onIntermediateVad streams partial hypotheses for live captions; onAsrComplete freezes the final hypothesis for deterministic actions.

onAsrCompleteWithIntent

Surfaces companion JSON + intent IDs for command routing without extra NLP.

Mixing modes

Common split: critical macros on offline wake-words, open conversation on cloud ASR.

Practical guidance

• Hardware control → offline wake-words first.
• Search / Q&A → cloud ASR.
• Status prompts → offline TTS.

Caveats

• Cloud ASR requires phone-side credential bootstrap to succeed.
• Real-world note from sample: avoid simultaneous screencast + TTS audio or routing may jump to the external display sink.
• Offline wake-words shine for low-latency, low-connectivity control surfaces.