vmodal_sdk_smartglass README

The project's README from github.com/v-modal/vmodal_sdk_smartglass. As one markdown file: README.md.

VModal owl wearing smart glasses

VModal for Smart Glasses

Turn a first-person view into a live, reliable video experience.

Capture on the glasses. Encode on Android. Stream securely to your ingest.
Keep your product focused on the experience—not the media plumbing.

Android API 31+, Kotlin 2.2, Rust core, and SRT live transport

Smart glasses sending a live first-person video stream to a cloud ingest

Build for the point of view no phone camera can reach.

Start here

Resource Link
💬 Developer support Join the VModal AI Discord
🚀 SDK quickstart Build and start your first stream
🧭 Developer portal Explore VModal developer resources
🌐 VModal platform Visit VModal

[!NOTE] This SDK streams live camera video from Android smart glasses. For video upload, indexing, and multimodal search, use the VModal Android SDK instead.

Make presence feel immediate

VModal combines an Android-native Kotlin client with a high-throughput Rust media core. Your app owns the product experience; the SDK owns the demanding path between a wearable camera and your live ingest service.

Your smart-glass experience VModal gives you
Remote expert support Reliable first-person video with reconnect behavior
Hands-free field work Meta DAT capture lifecycle and Android camera coordination
Live training and inspection Hardware H.264 encoding and MPEG-TS muxing
AR / VR companion experiences A compact Kotlin API using Flow and StateFlow
Production streaming Bounded queues, backpressure signals, metrics, and typed failures
Your own branded UX App-owned authentication, permissions, registration, and controls

A clean path from glasses to ingest

flowchart LR
    G[Smart glasses] --> D[Meta DAT]
    D --> K[Android H.264]
    K --> R[Rust media core]
    R --> S[Encrypted SRT]
    S --> I[Live ingest]

    A[Your Android app] --> C[SmartGlassClient]
    B[Authenticated backend] --> T[Short-lived target]
    T --> C
    C --> K
    C --> R

The separation is deliberate:

The Activity never needs to understand a JNI handle, SRT socket, or encoded GOP. The backend never needs to manage an Android camera session.

Build with the SDK

The implementation guide keeps the Kotlin and command-line setup in one place:

Open the Smart Glass SDK quickstart →

It covers prerequisites, local builds, SmartGlassClient creation, short-lived stream targets, state and metrics collection, and deterministic cleanup.

Engineered for wearable constraints

Bounded latency, not unbounded memory

GOP-aware backpressure and bounded queues keep a weak network from turning into ever-growing latency. The SDK can signal the Kotlin layer to drop safely or request a new keyframe when transport falls behind.

Recovery without a broken experience

SRT reconnects use configurable delays, jitter, and timeouts. When a target is expired or rejected, LiveStreamTargetProvider can fetch a fresh destination while the app continues to present one coherent stream lifecycle.

Native internals behind a Kotlin boundary

The public API does not expose Meta DAT types or Rust handles. Your host app owns the camera permission and registration experience; the SDK owns its one capture session only while streaming.

Metrics your support team can use

client.metrics exposes connection health, queue pressure, and transport behavior for a developer panel, field diagnostic screen, or support workflow.

Production integration checklist

The project provides two build flavors:

Flavor Purpose
offline Local lint and JVM tests without Meta artifacts or device credentials
meta Real-device integration with the Meta Wearables DAT dependencies

Keep credentials on the right side of the boundary

The app may briefly receive the SRT fields required to contribute live video. It should never contain account-level Cloudflare or live-platform credentials. After validating the app user, have your backend mint or lease a short-lived target; do not log or persist its stream ID or passphrase.

Explore the SDK

Build the view that only your user can see.

Resilient, native live video for Android smart glasses.

Talk with the VModal developer community →

vmodal_sdk_smartglass (v-modal/vmodal_sdk_smartglass)

Searching Database…

365 ·GitHub
Vmodal_sdk_smartglass
What is this?VModal Smart Glass SDK lets an Android app capture live first-person video from Meta smart glasses and stream it reliably to a live ingest service, pairing a Kotlin client with a high-throughput Rust media core.
Is this a standalone app or an SDK I integrate into my own app?It is an SDK, not an app: a Kotlin client plus a Rust media core that you embed in your own Android app, which keeps ownership of the UI, permissions, and authentication.
How do I get this running in my app?You follow the SDK quickstart to set up local builds, create a SmartGlassClient, wire up short-lived stream targets, and hook into state and metrics.
What do I need before integrating this?Android API 31+, JDK 17, Android compile SDK 36, plus a compatible Meta smart-glasses registration and the Meta Wearables DAT setup.
Any gotchas I should know about before shipping this?Only one SmartGlassClient can run per active capture owner and no other app can hold the camera while it streams; you also need your own authenticated backend to mint short-lived stream targets since the SDK never holds account credentials.
Does this work with any Android phone, or do I need real smart glasses?You need compatible Meta smart glasses with the Meta Wearables DAT integration; a separate 'offline' build flavor lets you run lint and JVM tests without the real hardware.
How are my streaming credentials kept safe?The device only ever gets short-lived SRT connection fields after your backend validates the user; it should never carry long-lived, account-level provider credentials.
Why split the SDK into a Kotlin client and a Rust core?The Kotlin layer owns the public API, Meta DAT integration, camera capture, and lifecycle, while the Rust core handles bounded media queues, H.264 normalization, MPEG-TS muxing, SRT transport, and reconnects — keeping native plumbing out of your app code.
What happens to the stream when the network gets weak?GOP-aware backpressure and bounded queues stop latency from growing unbounded; the SDK can signal the app to drop safely or request a new keyframe when transport falls behind.
Can I test my integration without real smart glasses?Yes, the 'offline' build flavor runs local lint and JVM tests without Meta artifacts or device credentials, while the 'meta' flavor does real-device integration.
What can I monitor once streaming is live?client.metrics exposes connection health, queue pressure, and transport behavior, which you can surface in a developer panel or field diagnostic screen.
What are similar projects?GSYVideoPlayer, PermissionsDispatcher, timber, Shizuku, RxTool, SmarterStreaming, Vector, AR.js
Which topics is it in?augmented-reality, h264, virtual-reality, audio-streaming, camera, vision, jvm, video-streaming, android-library, kotlin, authentication, sdk
Creator
v-modal
Size
4.3 MB
Stars
365
Updated
CommitOct 03
Age
2mo
Commits · 90 Days