# Glass TTS & ASR

## Purpose

Describe offline TTS playback, cloud ASR capture, and offline wake-word registration paths on the glasses.

## Entry points

Offline wake-word registration:

- `HomeActivity`

TTS / ASR tap targets:

- `SendMessageActivity`

## Platform

- Glass

## Main files

- [HomeActivity.kt](https://gitee.com/as_pixar/glass3sdkdemo/blob/main/glassdemo/app/src/main/java/com/rokid/glass/HomeActivity.kt)
- [SendMessageActivity.kt](https://gitee.com/as_pixar/glass3sdkdemo/blob/main/glassdemo/app/src/main/java/com/rokid/glass/SendMessageActivity.kt)

## Walk the sources (recommended order)

1. `HomeActivity.initSDK()` — `VoiceAction` registration + `GlassSdk.getGlassOfflineCmdService()?.add(...)`  
2. `SendMessageActivity.toClick()` — `R.id.btTts`, `R.id.btAsr` branches  
3. `SpeechCallback.Stub()` overrides for streaming events

## Related docs

- [Glass3 Glass SDK API reference](/en/terminal-sdk/api-reference/Glass3%20%20SDK(%E7%9C%BC%E9%95%9C%E7%AB%AF)%20API%E6%96%87%E6%A1%A3.md)
- [Quick start](/en/terminal-sdk/getting-started/快速开始.md)

## Feature coverage

- Offline wake-word lexicon install  
- Offline TTS playback  
- Cloud ASR sessions  
- Speech callback matrix (partial + final + intent JSON)

## Snippets

### Offline wake-word

```kotlin
jdVoiceAction = VoiceAction(
    "Turn on beacon #12 strobes",
    "da kai bian hao shi er de jing deng",
    object : IVoiceCallback.Stub() {
        override fun onVoiceTriggered() {
            Log.e(TAG, "Wake phrase: beacon #12")
        }
    }
)
GlassSdk.getGlassOfflineCmdService()?.add(jdVoiceAction)
```

### Offline TTS

```kotlin
val str = "Autumn stayed away—I'm off to climb the trail"
GlassSdk.getGlassOfflineTtsService()?.playTtsMsg(str)
```

### Cloud ASR

```kotlin
GlassSdk.getGlassAsrService()?.startSpeech(object : SpeechCallback.Stub() {
    override fun onStart() {}

    override fun onIntermediateVad(content: String) {}

    override fun onAsrComplete(content: String?) {}

    override fun onAsrCompleteWithIntent(
        content: String?,
        intent: Int,
        intentJson: String
    ) {}

    override fun onError(code: Int) {}
})
```

## Flow outline

### 1) Register wake-words

After SDK init, `HomeActivity` builds `VoiceAction` objects and calls `GlassSdk.getGlassOfflineCmdService()?.add(...)`. Upstream demos ship a handful of scripted wake phrases (siren/light control, snowfall, branded trigger, etc.—see upstream source strings).

### 2) Offline TTS tap path

`SendMessageActivity` routes the TTS button to `GlassSdk.getGlassOfflineTtsService()?.playTtsMsg(...)`. Online TTS hooks exist but remain commented in source.

### 3) ASR tap path

The ASR button launches `GlassSdk.getGlassAsrService()?.startSpeech(...)` with the following callbacks:

- `onStart()`  
- `onIntermediateVad(content)`  
- `onAsrComplete(content)`  
- `onAsrCompleteWithIntent(...)`  
- `onError(code)`  
- `onServiceConnectState(connect)`

## Sequence sketch

```mermaid
sequenceDiagram
    participant U as User
    participant G as Glass UI
    participant ASR as Speech service

    U->>G: Trigger ASR
    G->>ASR: startSpeech(callback)
    ASR-->>G: onStart()
    U->>ASR: Speak
    ASR-->>G: onIntermediateVad(content)
    ASR-->>G: onAsrComplete(content)
    ASR-->>G: onAsrCompleteWithIntent(...)
```

## Offline vs online matrix

| Topic | Offline wake | Cloud ASR | Offline TTS |
| --- | --- | --- | --- |
| Intent | Hot command dispatch | Free dictation | Audible prompts |
| Network | None | Required | None |
| Output | Callback fire | Partial + finals + intent JSON | Audio stream |
| Sample host | `HomeActivity` | `SendMessageActivity` | `SendMessageActivity` |
| Best for | Low-latency macros | Subtitles, Q&A | Local feedback |

## FAQs

### Offline vs online TTS

Sample highlights offline TTS so benches work air-gapped—flip commented online hooks once credentials validate.

### `onIntermediateVad` vs `onAsrComplete`

`onIntermediateVad` streams partial hypotheses for live captions; `onAsrComplete` freezes the final hypothesis for deterministic actions.

### `onAsrCompleteWithIntent`

Surfaces companion JSON + intent IDs for command routing without extra NLP.

### Mixing modes

Common split: critical macros on offline wake-words, open conversation on cloud ASR.

## Practical guidance

- Hardware control → offline wake-words first.  
- Search / Q&A → cloud ASR.  
- Status prompts → offline TTS.

## Caveats

- Cloud ASR requires phone-side credential bootstrap to succeed.  
- Real-world note from sample: avoid simultaneous screencast + TTS audio or routing may jump to the external display sink.  
- Offline wake-words shine for low-latency, low-connectivity control surfaces.
