vmodal_sdk_smartglass README

The project's README from github.com/v-modal/vmodal_sdk_smartglass. As one markdown file: README.md.

VModal owl wearing smart glasses

VModal for Smart Glasses

Turn a first-person view into a live, reliable video experience.

Capture on the glasses. Encode on Android. Stream securely to your ingest.
Keep your product focused on the experience—not the media plumbing.

Android API 31+, Kotlin 2.2, Rust core, and SRT live transport

Smart glasses sending a live first-person video stream to a cloud ingest

Build for the point of view no phone camera can reach.

Start here

Resource Link
💬 Developer support Join the VModal AI Discord
🚀 SDK quickstart Build and start your first stream
🧭 Developer portal Explore VModal developer resources
🌐 VModal platform Visit VModal

[!NOTE] This SDK streams live camera video from Android smart glasses. For video upload, indexing, and multimodal search, use the VModal Android SDK instead.

Make presence feel immediate

VModal combines an Android-native Kotlin client with a high-throughput Rust media core. Your app owns the product experience; the SDK owns the demanding path between a wearable camera and your live ingest service.

Your smart-glass experience VModal gives you
Remote expert support Reliable first-person video with reconnect behavior
Hands-free field work Meta DAT capture lifecycle and Android camera coordination
Live training and inspection Hardware H.264 encoding and MPEG-TS muxing
AR / VR companion experiences A compact Kotlin API using Flow and StateFlow
Production streaming Bounded queues, backpressure signals, metrics, and typed failures
Your own branded UX App-owned authentication, permissions, registration, and controls

A clean path from glasses to ingest

flowchart LR
    G[Smart glasses] --> D[Meta DAT]
    D --> K[Android H.264]
    K --> R[Rust media core]
    R --> S[Encrypted SRT]
    S --> I[Live ingest]

    A[Your Android app] --> C[SmartGlassClient]
    B[Authenticated backend] --> T[Short-lived target]
    T --> C
    C --> K
    C --> R

The separation is deliberate:

The Activity never needs to understand a JNI handle, SRT socket, or encoded GOP. The backend never needs to manage an Android camera session.

Build with the SDK

The implementation guide keeps the Kotlin and command-line setup in one place:

Open the Smart Glass SDK quickstart →

It covers prerequisites, local builds, SmartGlassClient creation, short-lived stream targets, state and metrics collection, and deterministic cleanup.

Engineered for wearable constraints

Bounded latency, not unbounded memory

GOP-aware backpressure and bounded queues keep a weak network from turning into ever-growing latency. The SDK can signal the Kotlin layer to drop safely or request a new keyframe when transport falls behind.

Recovery without a broken experience

SRT reconnects use configurable delays, jitter, and timeouts. When a target is expired or rejected, LiveStreamTargetProvider can fetch a fresh destination while the app continues to present one coherent stream lifecycle.

Native internals behind a Kotlin boundary

The public API does not expose Meta DAT types or Rust handles. Your host app owns the camera permission and registration experience; the SDK owns its one capture session only while streaming.

Metrics your support team can use

client.metrics exposes connection health, queue pressure, and transport behavior for a developer panel, field diagnostic screen, or support workflow.

Production integration checklist

The project provides two build flavors:

Flavor Purpose
offline Local lint and JVM tests without Meta artifacts or device credentials
meta Real-device integration with the Meta Wearables DAT dependencies

Keep credentials on the right side of the boundary

The app may briefly receive the SRT fields required to contribute live video. It should never contain account-level Cloudflare or live-platform credentials. After validating the app user, have your backend mint or lease a short-lived target; do not log or persist its stream ID or passphrase.

Explore the SDK

Build the view that only your user can see.

Resilient, native live video for Android smart glasses.

Talk with the VModal developer community →

vmodal_sdk_smartglass (v-modal/vmodal_sdk_smartglass)

Searching Database…

365 ·GitHub
Vmodal_sdk_smartglass
这是什么？VModal Smart Glass SDK 让 Android 应用能够从 Meta 智能眼镜采集第一人称实时视频，并可靠地串流到直播接入服务，由 Kotlin 客户端搭配高吞吐量的 Rust 媒体核心组成。
它是独立应用，还是我要集成到自己应用里的 SDK？它是 SDK，不是应用：一个 Kotlin 客户端加一个 Rust 媒体核心，嵌入到你自己的 Android 应用里，界面、权限和身份验证都由你的应用自己掌控。
怎么在我的应用里让它跑起来？按照 SDK 快速入门设置本地构建，创建 SmartGlassClient，接入短时效的串流目标，并挂接状态和指标回调。
集成之前需要准备什么？需要 Android API 31+、JDK 17、Android compile SDK 36，还要有兼容的 Meta 智能眼镜注册以及 Meta Wearables DAT 的设置。
上线之前有什么需要留意的坑吗？每个活跃的采集持有者只能运行一个 SmartGlassClient，而且串流期间其他应用无法占用摄像头；另外，由于 SDK 从不保存账号凭据，你还需要自己的已验证后端来签发短时效串流目标。
任何 Android 手机都能用吗，还是必须要有真的智能眼镜？需要支持 Meta Wearables DAT 集成的兼容 Meta 智能眼镜；单独的 'offline' 构建变体让你在没有真机的情况下运行 lint 和 JVM 测试。
我的串流凭据是如何保管安全的？只有在你的后端验证用户之后，设备才会拿到短时效的 SRT 连接信息；设备上绝不应该保存长效的账号级服务商凭据。
为什么把 SDK 拆成 Kotlin 客户端和 Rust 核心？Kotlin 层负责公开 API、Meta DAT 集成、摄像头采集和生命周期，Rust 核心负责有界媒体队列、H.264 归一化、MPEG-TS 封装、SRT 传输和重连——让原生底层细节远离你的应用代码。
网络变差时串流会怎样？感知 GOP 的背压机制和有界队列能防止延迟无限增长；当传输跟不上时，SDK 可以通知应用安全丢弃，或请求新的关键帧。
没有真的智能眼镜也能测试集成吗？可以，'offline' 构建变体不需要 Meta 构件或设备凭据就能运行本地 lint 和 JVM 测试，而 'meta' 变体用于真机集成。
串流开始后我能监控什么？client.metrics 会暴露连接健康状况、队列压力和传输行为，你可以在开发者面板或现场诊断界面中展示这些信息。
What are similar projects?GSYVideoPlayer, PermissionsDispatcher, timber, Shizuku, RxTool, SmarterStreaming, Vector, AR.js
Which topics is it in?augmented-reality, h264, virtual-reality, audio-streaming, vision, camera, jvm, video-streaming, android-library, kotlin, authentication, sdk
Creator
v-modal
Size
4.3 MB
Stars
365
Updated
CommitOct 03
Age
2mo
Commits · 90 Days