Decart Android SDK

Android SDK for Decart realtime streaming and batch video generation.

Features

Real-time video restyling and editing via LiveKit media transport
Batch video generation via /v1/jobs/* queue APIs
Built-in realtime and video model registries
Kotlin coroutines and Flow-based reactive state management
Observable connection state, remote media streams, errors, diagnostics, and publish stats
LiveKit camera publishing support

Requirements

Android API 24+ (Android 7.0)
Kotlin 2.1+
Java 17

Installation

Add the JitPack repository to your settings.gradle.kts:

dependencyResolutionManagement {
    repositories {
        google()
        mavenCentral()
        maven { url = uri("https://jitpack.io") }
    }
}

Add the dependency to your app's build.gradle.kts:

dependencies {
    implementation("com.github.DecartAI:decart-android:0.7.3")
}

Quick Start

import ai.decart.sdk.DecartClient
import ai.decart.sdk.DecartClientConfig
import ai.decart.sdk.realtime.ConnectOptions
import ai.decart.sdk.realtime.FacingMode
import ai.decart.sdk.realtime.InitialPrompt
import ai.decart.sdk.realtime.MirrorMode
import ai.decart.sdk.RealtimeModels

val client = DecartClient(context, DecartClientConfig(apiKey = "your-api-key"))
val realtime = client.realtime

// 1. Connect and publish the device camera through LiveKit
realtime.connect(
    ConnectOptions(
        model = RealtimeModels.LUCY_RESTYLE_2,
        initialPrompt = InitialPrompt("a cyberpunk cityscape"),
        facing = FacingMode.FRONT,
        mirror = MirrorMode.AUTO,
        publishCamera = true,
        onRemoteStream = { stream ->
            // Display stream.videoTrack with a LiveKit renderer.
        },
    ),
)

// 2. Change prompt during session and wait for the server ack.
try {
    realtime.setPrompt("a sunny beach scene", enhance = true)
} catch (e: Exception) {
    // ack failure, timeout, or websocket disconnect
}

// Or start immediately and keep a Deferred if you want JS Promise-style usage.
val promptAck = realtime.setPromptAsync("a sunny beach scene", enhance = true)
promptAck.await()

// 3. Disconnect when done
realtime.disconnect()
client.release()

LiveKit rendering

Realtime media tracks are LiveKit tracks. The Android publisher API currently surfaces remote video streams:

import io.livekit.android.renderer.SurfaceViewRenderer

realtime.remoteStreamUpdates.collect { stream ->
    stream.videoTrack?.addRenderer(remoteRenderer)
}

Realtime audio support

The Android LiveKit realtime publisher currently supports video only. publishMicrophone, includeMicrophone, and RealtimeMediaStream.audioTrack are retained for 0.7 source compatibility, but they are deprecated and ignored; SDK-created streams always expose audioTrack = null.

Realtime camera sizing and codec

Use the model registry to size camera input instead of hardcoding dimensions:

val model = RealtimeModels.LUCY_2_1
val localStream = realtime.createLocalVideoStream(model) // uses model.width/model.height

The Lucy 2.1 realtime model configs use 1088x624. The default LiveKit publisher codec is VP8; override it only when needed via RealtimeConfiguration.VideoConfig(preferredCodec = ...).

Realtime camera mirroring

Use mirror to pre-flip captured frames before they are sent to Decart. This is safer than flipping the rendered remote stream because server-baked pixels such as watermarks and overlays remain readable.

val localStream = realtime.createLocalVideoStream(
    model = RealtimeModels.LUCY_2_1,
    facing = FacingMode.FRONT,
    mirror = MirrorMode.AUTO, // default: front camera mirrored, back camera unmodified
)

When capture mirroring is enabled, render both local previews and remote streams as-is; do not also set renderer-level mirroring for those tracks.

You can also set the same behavior for SDK-owned camera capture:

realtime.connect(
    ConnectOptions(
        model = RealtimeModels.LUCY_2_1,
        facing = FacingMode.FRONT,
        mirror = MirrorMode.AUTO,
    )
)

Output resolution

Opt into 1080p output for a realtime session (defaults to 720p server-side):

import ai.decart.sdk.realtime.Resolution

realtime.connect(
    ConnectOptions(
        model = RealtimeModels.LUCY_2_1,
        resolution = Resolution.P1080, // default: server-side 720p
        onRemoteStream = { /* ... */ },
    )
)

Fast mode

Fast mode (speed = Speed.FAST) serves the session from a higher-compute tier for lower latency and higher throughput; output quality is unchanged. It is currently available for lucy-2.5 / lucy-latest and lucy-vton-3.5 / lucy-vton-latest, in the US region only, and is billed at 2x the standard realtime rate for those models. Other models ignore the option. Omit it (the default) for standard mode.

import ai.decart.sdk.realtime.Speed

realtime.connect(
    ConnectOptions(
        model = RealtimeModels.LUCY_2_5,
        speed = Speed.FAST, // default: null (standard mode)
        onRemoteStream = { /* ... */ },
    )
)

Each model advertises the tiers it supports via RealtimeModel.supportedSpeeds (e.g. Speed.FAST in RealtimeModels.LUCY_2_5.supportedSpeeds). Setting speed for a model that does not list it is not an error: the SDK logs a warning through the configured logger and the server serves the session from the standard tier at the standard rate.

Connection quality

Two layers, both on a shared GOOD | FAIR | POOR | CRITICAL scale — the SDK reports, you decide what to do (gate the UI, warn the user, etc.).

Preflight (before connecting). A fast, network-only reachability check — a throwaway peer connection against public STUN, so there's no session and no cost:

import ai.decart.sdk.realtime.ConnectionQuality

val report = realtime.checkConnectivity() // suspend
// report.metrics: transport (UDP | RELAY | FAILED), rttMs
if (report.quality == ConnectionQuality.CRITICAL) showFallbackUi(report.reasons)

In-session quality. While connected, the SDK derives a smoothed verdict from WebRTC stats (latency, packet loss, upstream bandwidth, frame rate) and tells you the limiting factor. Updates every stats sample (~few seconds) with fresh metrics; the level is debounced:

realtime.connect(
    ConnectOptions(
        model = RealtimeModels.LUCY_2_1,
        onConnectionQuality = { report ->
            // report.limitingFactor: BANDWIDTH | LATENCY | LOSS | STALL | CPU | NONE
            // report.metrics: rttMs, fps, packetLoss, availableUpstreamKbps, ...
        },
        onRemoteStream = { /* ... */ },
    )
)

// also a Flow + a getter:
realtime.connectionQuality.collect { /* ConnectionQualityReport? */ }
realtime.getConnectionQuality() // latest, or null before the first sample

Glass-to-glass latency (opt-in, diagnostic). Network RTT hides the dominant cost in real-time video — model inference — so a session can read GOOD while feeling laggy. Set debugQuality = true to measure the real camera→display latency: the SDK stamps a pixel marker into each outgoing frame and reads it back off the rendered output, surfacing startup (ttffMs) and steady-state (g2gMs) latency plus end-to-end drops (g2gDropRatio). When present, glass-to-glass drives the latency verdict instead of RTT.

⚠️ Diagnostic only. The marker is visible (bottom-left of the published + rendered video) and adds per-frame pixel work — don't enable it for production/end-user sessions. With a caller-provided stream, build it via createLocalVideoStream(..., debugQuality = true) so the same debugQuality is set on both the stream and connect().

realtime.connect(
    ConnectOptions(
        model = RealtimeModels.LUCY_2_1,
        debugQuality = true,
        onConnectionQuality = { report ->
            // report.metrics.ttffMs / g2gMs / g2gDropRatio
        },
    )
)

For a measured verdict before connecting (instead of the network-only check), use the deep probe — it briefly opens a real session with a synthetic source, measures glass-to-glass, then tears it down. It requires a model and costs a short GPU session:

import ai.decart.sdk.realtime.CheckConnectivityOptions

val probe = realtime.checkConnectivity(
    CheckConnectivityOptions(deep = true, model = RealtimeModels.LUCY_2_1),
)
// probe.metrics.g2gMs / ttffMs / g2gDropRatio

Batch Queue Example (Lucy 2 V2V)

import ai.decart.sdk.DecartClient
import ai.decart.sdk.DecartClientConfig
import ai.decart.sdk.VideoModels
import ai.decart.sdk.queue.FileInput
import ai.decart.sdk.queue.QueueJobResult
import ai.decart.sdk.queue.VideoEditInput

val client = DecartClient(context, DecartClientConfig(apiKey = "your-api-key"))

val input = VideoEditInput(
    prompt = "Cinematic color grade, soft contrast",
    data = FileInput.fromUri(videoUri),                     // required
    referenceImage = FileInput.fromUri(referenceImageUri),  // optional
    seed = 42,
    resolution = "720p",
    enhancePrompt = true,
)

when (val result = client.queue.submitAndPoll(VideoModels.LUCY_2_1, input)) {
    is QueueJobResult.Completed -> {
        // MP4 bytes
        val output = java.io.File(context.cacheDir, "output.mp4")
        output.writeBytes(result.data)
    }
    is QueueJobResult.Failed -> {
        // Job reached terminal failed state
        android.util.Log.e("Decart", "Job failed: ${result.error}")
    }
    else -> Unit
}

client.release()

Batch Progress Stream Example

client.queue.submitAndObserve(VideoModels.LUCY_2_1, input).collect { update ->
    when (update) {
        is QueueJobResult.InProgress -> {
            // pending / processing
            android.util.Log.d("Decart", "Status: ${update.status}")
        }
        is QueueJobResult.Completed -> {
            android.util.Log.d("Decart", "Completed: ${update.data.size} bytes")
        }
        is QueueJobResult.Failed -> {
            android.util.Log.e("Decart", "Failed: ${update.error}")
        }
    }
}

Other Batch Input Examples

import ai.decart.sdk.VideoModels
import ai.decart.sdk.queue.FileInput
import ai.decart.sdk.queue.VideoRestyleInput

// Restyle (reference-image mode)
val restyle = VideoRestyleInput(
    data = FileInput.fromUri(videoUri),
    referenceImage = FileInput.fromUri(styleImageUri),
    seed = 7,
)
client.queue.submit(VideoModels.LUCY_RESTYLE_2, restyle)

Available Models

Realtime Models

Model	Constant	Resolution	FPS
Lucy 2.1	`RealtimeModels.LUCY_2_1`	1088x624	30
Lucy 2.5	`RealtimeModels.LUCY_2_5`	1280x720	30
Lucy VTON 3.5	`RealtimeModels.LUCY_VTON_3_5`	1280x720	30
Lucy Restyle 2	`RealtimeModels.LUCY_RESTYLE_2`	1280x704	30

Batch Video Models

Model	Constant	Queue Path	Resolution	FPS
Lucy Clip	`VideoModels.LUCY_CLIP`	`/v1/jobs/lucy-clip`	1280x704	25
Lucy 2.1	`VideoModels.LUCY_2_1`	`/v1/jobs/lucy-2.1`	1088x624	20
Lucy 2.5	`VideoModels.LUCY_2_5`	`/v1/jobs/lucy-2.5`	1280x720	20
Lucy VTON 3.5	`VideoModels.LUCY_VTON_3_5`	`/v1/jobs/lucy-vton-3.5`	1280x720	20
Lucy Restyle 2	`VideoModels.LUCY_RESTYLE_2`	`/v1/jobs/lucy-restyle-2`	1280x704	22

Typed input helpers:

VideoEditInput (lucy-2.1, lucy-2.5, lucy-vton-3.5, lucy-clip)
VideoRestyleInput (lucy-restyle-2)

API Reference

Core Classes

Class	Description
`DecartClient`	Unified entry point exposing `realtime` and `queue` clients
`RealTimeClient`	Main entry point for real-time video streaming
`RealTimeClientConfig`	Client configuration (API key, base URL, logger)
`ConnectOptions`	Connection parameters (model, LiveKit stream callbacks, initial prompt, `resolution`, `speed`)
`InitialPrompt`	Initial prompt with optional enhancement
`Resolution`	Output resolution enum (`P720`, `P1080`) for `ConnectOptions.resolution`
`Speed`	Realtime compute tier enum (`FAST`) for `ConnectOptions.speed`; advertised per model via `RealtimeModel.supportedSpeeds`
`MirrorMode`	Camera-input mirroring enum (`OFF`, `ON`, `AUTO`)
`ConnectionState`	Connection lifecycle enum (`DISCONNECTED`, `CONNECTING`, `CONNECTED`, `GENERATING`, `RECONNECTING`)
`RealtimeModels`	Available AI model definitions
`VideoModels`	Available batch video model definitions
`ModelInputType`	Input category expected by each batch model
`QueueClient`	Batch queue client (`submit`, `status`, `result`, `submitAndPoll`, `submitAndObserve`)
`VideoEditInput`	Typed queue input for Lucy 2.1 V2V payload
`VideoRestyleInput`	Typed queue input for Lucy Restyle
`FileInput`	File wrappers for `Uri`, `File`, `ByteArray`, `InputStream`
`DecartError`	Error with code, message, and optional cause
`ErrorCodes`	Predefined error code constants

RealTimeClient

Methods:

Method	Description
`connect(options)`	Connect to a model, join the returned LiveKit room, and publish the camera by default
`disconnect()`	End the current session
`setPrompt(prompt, enhance, timeoutMs)`	suspend — update the prompt and wait for the server ack; throws on ack failure, timeout (default 15s), or disconnect
`setPromptAsync(prompt, enhance, timeoutMs)`	Starts the prompt update immediately and returns `Deferred<Unit>`; call `await()` to observe ack failure, timeout, or disconnect
`setImage(imageBase64, prompt, enhance, timeout)`	suspend — set a reference image and optional prompt, then wait for the server ack; throws on ack failure, timeout (default 30s), or disconnect
`setImageAsync(imageBase64, prompt, enhance, timeout)`	Starts the image/prompt update immediately and returns `Deferred<Unit>`; call `await()` to observe ack failure, timeout, or disconnect
`release()`	Release all resources

Observable State:

Property	Type	Description
`connectionState`	`StateFlow<ConnectionState>`	Current connection state
`connectionChange`	`StateFlow<ConnectionState>`	JS-aligned alias for `connectionState`
`errors`	`SharedFlow<DecartError>`	Error events
`generationTick`	`SharedFlow<GenerationTickMessage>`	Generation tick events
`localStreamUpdates`	`SharedFlow<RealtimeMediaStream>`	Local LiveKit stream updates
`localStream`	`SharedFlow<RealtimeMediaStream>`	JS-aligned alias for `localStreamUpdates`
`remoteStreamUpdates`	`SharedFlow<RealtimeMediaStream>`	Remote LiveKit stream updates
`remoteStream`	`SharedFlow<RealtimeMediaStream>`	JS-aligned alias for `remoteStreamUpdates`
`queuePositionUpdates`	`SharedFlow<QueuePositionMessage>`	Queue position updates while waiting for a server slot
`queuePosition`	`SharedFlow<QueuePositionMessage>`	JS-aligned alias for `queuePositionUpdates`
`generationEnded`	`SharedFlow<GenerationEndedMessage>`	Generation lifecycle end events
`sessionStarted`	`StateFlow<SessionStarted?>`	`(sessionId, subscribeToken)` once the LiveKit room info arrives
`subscribeToken`	`String?`	Base64 token to hand to viewer / subscribe clients
`diagnostics`	`SharedFlow<DiagnosticEvent>`	Connection diagnostic events (incl. `PublishStats`)
`diagnostic`	`SharedFlow<DiagnosticEvent>`	JS-aligned alias for `diagnostics`
`stats`	`SharedFlow<PublishStatsEvent>`	Publisher outbound video stats, aligned with the JS `stats` event

QueueClient

Method	Description
`submit(model, input)`	Create a job and return `{ jobId, status }`
`status(jobId)`	Read current job status
`result(jobId)`	Download completed job content as `ByteArray`
`submitAndPoll(model, input, onStatusChange?)`	Convenience method: submit and wait for terminal result
`submitAndObserve(model, input)`	`Flow` of in-progress updates followed by terminal result
`release()`	Close queue HTTP resources

Error Handling

Errors are emitted via the errors SharedFlow:

client.errors.collect { error ->
    when (error.code) {
        ErrorCodes.INVALID_API_KEY -> { /* handle auth error */ }
        ErrorCodes.WEBRTC_TIMEOUT_ERROR -> { /* handle timeout */ }
        ErrorCodes.WEBRTC_ICE_ERROR -> { /* handle ICE failure */ }
        ErrorCodes.WEBRTC_WEBSOCKET_ERROR -> { /* handle WS error */ }
        ErrorCodes.WEBRTC_SERVER_ERROR -> { /* handle server error */ }
        ErrorCodes.WEBRTC_SIGNALING_ERROR -> { /* handle signaling error */ }
    }
}

Queue APIs throw operation-specific exceptions:

QueueSubmitException
QueueStatusException
QueueResultException
InvalidInputException

Sample App

See the sample/ directory for a Jetpack Compose app with:

Realtime tab: camera + LiveKit streaming
Video tab: batch job submission, status updates, and result playback

Example App

For a more complete app showcasing real-world use cases -- video restyling, video editing, 90+ style presets, multiple view modes (fullscreen, PIP, split), and swipe-based navigation -- check out the Decart Android Example App.

Resources

License

MIT

Name		Name	Last commit message	Last commit date
gradle		gradle
sample		sample
sdk		sdk
.gitignore		.gitignore
README.md		README.md
build.gradle.kts		build.gradle.kts
gradle.properties		gradle.properties
gradlew		gradlew
gradlew.bat		gradlew.bat
jitpack.yml		jitpack.yml
settings.gradle.kts		settings.gradle.kts

Navigation Menu

Repository files navigation

Decart Android SDK

Features

Requirements

Installation

Quick Start

LiveKit rendering

Realtime audio support

Realtime camera sizing and codec

Realtime camera mirroring

Output resolution

Fast mode

Connection quality

Batch Queue Example (Lucy 2 V2V)

Batch Progress Stream Example

Other Batch Input Examples

Available Models

Realtime Models

Batch Video Models

API Reference

Core Classes

RealTimeClient

QueueClient

Error Handling

Sample App

Example App

Resources

License

About

Releases

Packages

Used by

Contributors

Languages

Latest commit

History

Folders and files

Repository files navigation

Decart Android SDK

Features

Requirements

Installation

Quick Start

LiveKit rendering

Realtime audio support

Realtime camera sizing and codec

Realtime camera mirroring

Output resolution

Fast mode

Connection quality

Batch Queue Example (Lucy 2 V2V)

Batch Progress Stream Example

Other Batch Input Examples

Available Models

Realtime Models

Batch Video Models

API Reference

Core Classes

RealTimeClient

QueueClient

Error Handling

Sample App

Example App

Resources

License

About

Resources

Stars

Watchers

Forks

Releases

Packages

Used by

Contributors

Languages