Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions packages/sdk/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,16 @@ realtimeClient.setPrompt("Cyberpunk city");
realtimeClient.disconnect();
```

#### Portrait mode

Pass `{ orientation: "portrait" }` to swap width and height for vertical streams:

```typescript
const model = models.realtime("lucy_2_rt", { orientation: "portrait" });
// model.width → 720
// model.height → 1280
```

### Async Processing (Queue API)

For video generation jobs, use the queue API to submit jobs and poll for results:
Expand Down
1 change: 1 addition & 0 deletions packages/sdk/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@ export {
type Model,
type ModelDefinition,
models,
type RealtimeModelOptions,
type RealTimeModels,
type VideoModelDefinition,
type VideoModels,
Expand Down
31 changes: 30 additions & 1 deletion packages/sdk/src/shared/model.ts
Original file line number Diff line number Diff line change
Expand Up @@ -378,12 +378,41 @@ const _models = {
},
} as const;

/**
* Options for configuring realtime model output.
*/
export type RealtimeModelOptions = {
/** The orientation of the output. When `"portrait"`, width and height are swapped. Defaults to `"landscape"`. */
orientation?: "landscape" | "portrait";
};

export const models = {
realtime: <T extends RealTimeModels>(model: T): ModelDefinition<T> => {
/**
* Get a realtime model definition.
*
* Available models:
* - `"mirage"` - Mirage v1
* - `"mirage_v2"` - Mirage v2
* - `"lucy_v2v_720p_rt"` - Lucy v2v 720p realtime
* - `"lucy_2_rt"` - Lucy 2 realtime
* - `"live_avatar"` - Live avatar
*
* @param model - The realtime model to use.
* @param options - Optional configuration. Set `orientation` to `"portrait"` to swap width and height.
* @returns The model definition, with dimensions adjusted for the requested orientation.
*/
realtime: <T extends RealTimeModels>(model: T, options?: RealtimeModelOptions): ModelDefinition<T> => {
const modelDefinition = _models.realtime[model];
if (!modelDefinition) {
throw createModelNotFoundError(model);
}
if (options?.orientation === "portrait") {
return {
...modelDefinition,
width: modelDefinition.height,
height: modelDefinition.width,
} as ModelDefinition<T>;
}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Portrait swap ignores schema dimension defaults

Medium Severity

When options.orientation === "portrait", models.realtime() swaps only width/height but leaves other dimension-coupled fields (notably inputSchema and any embedded/default resolution metadata) unchanged. If callers use inputSchema defaults/validation alongside returned dimensions, portrait mode can yield inconsistent sizing/validation behavior.

Fix in Cursor Fix in Web

return modelDefinition as ModelDefinition<T>;
},
/**
Expand Down
39 changes: 39 additions & 0 deletions packages/sdk/tests/unit.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -990,6 +990,45 @@ describe("Lucy 2 realtime", () => {
});
});

describe("models.realtime", () => {
it("returns correct landscape dimensions by default", () => {
const model = models.realtime("lucy_2_rt");
expect(model.width).toBe(1280);
expect(model.height).toBe(720);
expect(model.fps).toBe(20);
});

it("returns correct landscape dimensions when explicitly set", () => {
const model = models.realtime("lucy_2_rt", { orientation: "landscape" });
expect(model.width).toBe(1280);
expect(model.height).toBe(720);
expect(model.fps).toBe(20);
});

it("returns swapped dimensions in portrait mode", () => {
const model = models.realtime("lucy_2_rt", { orientation: "portrait" });
expect(model.width).toBe(720);
expect(model.height).toBe(1280);
expect(model.fps).toBe(20);
});

it("portrait mode preserves model name and urlPath", () => {
const model = models.realtime("lucy_2_rt", { orientation: "portrait" });
expect(model.name).toBe("lucy_2_rt");
expect(model.urlPath).toBe("/v1/stream");
});

it("throws on unknown model", () => {
expect(() => models.realtime("nonexistent_model" as any)).toThrow();
});

it("swaps dimensions for mirage in portrait mode", () => {
const model = models.realtime("mirage", { orientation: "portrait" });
expect(model.width).toBe(704);
expect(model.height).toBe(1280);
});
});

describe("WebRTCConnection", () => {
describe("setImageBase64", () => {
it("rejects immediately when WebSocket is not open", async () => {
Expand Down
Loading