# Create a new streaming session
Source: https://docs.vmonster.io/api-reference/create-a-new-streaming-session
apis/streams.yaml post /v1/streams
Create a new streaming session on VMONSTER Studio for a specified AI Avatar.
# Intro to VMONSTER
Source: https://docs.vmonster.io/introduction
Meet the next generation of Interactive AI.
Seamlessly integrate real-time AI Avatar video streaming into your service.
Built for developers, our lightweight SDK enables seamless integration into your service.
This guide introduces essential SDK functions—from quick authentication to flexible feature integration and practical use cases.
## Setting up
Implement your first AI Avatar application with our step-by-step guide
Learn how to install the VMONSTER Streaming SDK.
Learn about AI Avatar speech feature, using text and streaming input.
Learn how to transcribe user audio, using user audio track or blob.
Dive Deep into how to interact with with AI Avatars.
Javascript examples for the VMONSTER Streaming SDK.
# AI Avatar
Source: https://docs.vmonster.io/pages/aiAvatar
The AI Avatar is created within the [VMONSTER Studio](https://app.vmonster.io) designed to interact seamlessly with users.
You can develop a variety of AI Avatars to cater to diverse user needs.
## Key Features
***
### Text-to-Speech (TTS)
Employs TTS technology to convert text input into natural-sounding speech, enhancing user engagement.
### Lipsync Technology
Integrates lipsync technology to synchronize the Agent's lip movements with speech in real-time, creating a more realistic and immersive experience.
# Authentication
Source: https://docs.vmonster.io/pages/authentication
To use the VMONSTER API, an authentication process must be completed.
During this process, an API Key is required. After obtaining the key, follow the usage instructions below.
## Usages
1. Create an Enterprise account on the [VMONSTER Studio](https://app.vmonster.io) and obtain an API Key.
2. Ensure that the issued API Key is not recorded in client-side web browser code and is kept confidential.
3. When sending API requests, include the key in the HTTP headers using the `x-api-key` field:
```js theme={null}
headers: {
'x-api-key': 'YOUR_API_KEY',
}
```
## Errors
* Requests made with an invalid key will result in a `403 Forbidden` error.
* If the `x-api-key` header is missing, a `422 Unprocessable Entity` error will occur.
# Add Event Callback Function
Source: https://docs.vmonster.io/pages/callback_Functions
How to use callback functions with events in VmonsterRoom.
In the context of the VmonsterRoom instance, the callback function plays a vital role in responding to various Event types. Here's what you need to know about using callback functions:
* **Event Handling**: The callback function is invoked when its associated
Event takes place. This enables you to execute specific logic in response to
changes or actions within the Room.
* **Single Listener Binding**: When attaching a callback function via the
`on()` method, remember that only one listener can be attached per event type.
If a new callback is registered for the same event, it will replace the
previous listener.
* **Function Parameters**: Depending on the event type, the callback function
may receive parameters that provide context or data related to the event.
Tailor your callback to handle this data appropriately.
* **Integration with Room**: Utilizing the callback function allows you to
seamlessly integrate interactive features within the VmonsterRoom instance,
enhancing the responsiveness of your application to User and AI Avatar actions.
This approach ensures that your application remains dynamic and responsive, adapting to real-time events as they occur within the VmonsterRoom.
***
## on()
The `on()` method is used to attach an event listener to a Event.
```ts theme={null}
on(event: RoomEvent, callback: Function): void
```
You can only register one listener per event type. This means that if you register a new listener for an event, it will overwrite any previously registered listener for that event.
The callback function you provide will be executed when the specified event occurs. It should be tailored to handle the corresponding event type effectively.
| event | callback function type |
| ------------------------- | ------------------------- |
| `joined` | () ⇒ void |
| `joining` | () ⇒ void |
| `left` | () ⇒ void |
| `aiavatar-start-speaking` | () ⇒ void |
| `aiavatar-message` | (message: string) ⇒ void |
| `aiavatar-stop-speaking` | () ⇒ void |
| `stt-data` | (sttData: STTData) ⇒ void |
| `stt-error` | (error: any) ⇒ void |
# Create New Stream
Source: https://docs.vmonster.io/pages/create_New_Stream
Learn how to create a new AI Avatar stream.
To join [`VmonsterRoom`](/pages/vmonsterRoom), a new stream must be created each time. A new stream can be created by making an API call.
* [Authentication](/pages/authentication)
* Include the API key in the request headers for every API call.
* Increase the value of real-time interaction by adjusting the desired background / position / scale.
* After successful API request, you will receive:
* `session_id`: Unique identifier for the current session
* `stream_id`: Specific stream identifier
* `token`: Access token for entering the created room
```javascript theme={null}
const fetchNewStream = async () => {
const formData = new FormData();
// Enter the ID of the AI Avatar you want to use.
formData.append("aiavatar_id", "b9065cba-07ba-4720-8686-69dfca10a94c");
// Enter the language. The accent and speech pattern will vary based on the language.
formData.append("language", "en");
// [Optional] Set the session duration in seconds. Default is 3600 seconds (1 hour) if not specified.
formData.append("max_duration_s", 300);
const response = await fetch("https://api.vmonster.io/v1/streams", {
method: "POST",
headers: {
"x-api-key": process.env.VMONSTER_API_KEY,
},
body: formData,
});
const data = await response.json();
return data;
};
```
For detailed explanations and parameters, please refer to the API Reference.
API Reference for creating new stream.
# Demo
Source: https://docs.vmonster.io/pages/demo
Quickly test using the Javascript Demo. You can find this repository on our
[github](https://github.com/VMONSTER-AI/vmonster-streaming-js-vanilla-demo).
Or, To clone the repository,
```bash theme={null}
git clone https://github.com/VMONSTER-AI/vmonster-streaming-js-vanilla-demo.git
```
# Events
Source: https://docs.vmonster.io/pages/events
In the `VmonsterRoom`, several events of type `Event` may occur, which are essential for handling interactions and monitoring the state of the session.
The following are the key events:
***
Triggering when an attempt is made to connect the AI Avatar Stream. It occurs immediately upon calling the `join()` function.
Occuring when the AI Avatar Stream connection is successfully completed.
Triggering when the AI Avatar Stream connection is terminated.
Triggering when the AI Avatar begins speaking. It allows you to determine whether the AI Avatar is currently speaking.
Occuring when the AI Avatar sends a message. It enables you to review messages sent by the AI Avatar.
Triggering when the AI Avatar stops speaking. It allows you to determine whether the AI Avatar is currently speaking.
Triggering when the AI Avatar video track connected. Video MediaStream is passed to the callback.
Triggering when the AI Avatar audio track connected. Audio MediaStream is passed to the callback.
Triggered repeatedly while the user’s audio is unmuted and they are speaking. It sends speech-to-text (STT) results one by one to the callback as STTData.
```typescript theme={null}
type STTDataEventType = "transcript" | "start_of_speech" | "end_of_speech";
type STTData = {
text: string; // contains transcription text.
eventType: STTDataEventType;
type: "stt-data";
};
```
`STTDataEventType`
* `“transcript”` – The finalized transcription of the user’s speech audio.
* `“start_of_speech”` – Indicates the start of the user speaking. This means the Voice Activity Detection (VAD) system has detected the beginning of human speech.
* `“end_of_speech”` - Indicates the end of the user speaking. This means the Voice Activity Detection (VAD) system has detected the cessation of human speech.
Occurs when real-time STT cannot be performed successfully.
## Handling Events
To attach callbacks to these events, use `on()` method.
Learn about how to add callback function to Events
# Installation
Source: https://docs.vmonster.io/pages/installation
Install the VMONSTER Client Streaming SDK
**Prerequisite**: `npm` or `yarn` package manager
```bash npm theme={null}
npm install vmonster-streaming-js
```
```bash yarn theme={null}
yarn add vmonster-streaming-js
```
# Instance Methods
Source: https://docs.vmonster.io/pages/instance_Methods
The `VmonsterRoom` instance provides a variety of methods to manage and control the interactions and functionalities within the virtual space.
These methods enable developers to automate tasks and enhance user engagement efficiently.
The following are the instance methods.
***
## join()
```ts theme={null}
async join(options: JoinOptions): Promise
```
This method is used to connect to a `VmonsterRoom`.
If the method operates successfully, the `joined` event will be triggered.
#### Parameters
The config parameter accepts an object of type `JoinOptions.`
```typescript theme={null}
interface JoinOptions {
sessionId: string;
streamId: string;
token: string;
config?: SessionConfig;
}
```
**Error Occurrence**
* An error will be triggered if the method is called after the `joining` or `joined` events have already occurred.
***
## speak()
```ts theme={null}
async speak(
config?: MessageConfig
): Promise
```
This method requests the AI Avatar to speak based on the provided text or text stream. You can receive the AI Avatar's video stream data based on the input text. You can optionally configure the background, position, and scale of the AI Avatar for each request.
It supports standard text input as well as streaming input(AsyncIterable), allowing for real-time speech generation.
#### Parameters
The config parameter accepts an object of type `MessageConfig`, which is a union of the following: `TextMessageConfig` and `StreamMessageConfig`.
```ts theme={null}
// Standard text utterance
interface TextMessageConfig {
text: string;
isStream?: false;
background?: File | Blob | null;
positionX?: number | null;
positionY?: number | null;
scale?: number | null;
}
```
```ts theme={null}
// Streaming text utterance
interface StreamMessageConfig {
isStream: true;
stream: AsyncIterable;
background?: File | Blob | null;
positionX?: number | null;
positionY?: number | null;
scale?: number | null;
}
```
When speak() is called, the following steps occur sequentially.
1. The `AIAvatarState` transitions to `loading`.
2. When the AIAvatar begins speaking:
a. The `aiavatar-start-speaking` event is triggered.
b. The `AIAvatarState` transitions to `speaking`.
3. When the AI Avatar finishes speaking:
a. The `aiavatar-stop-speaking` event is triggered.
b. The `AIAvatarState` transitions to `idle`.
4. The AIAvatar's speech text can be verified through the callback argument of the `aiavatar-message` event.
**Error Occurrence**
* The `RoomState` is not `joined` when the method is called.
* The `AIAvatarState` is not `idle` when the method is called.
***
## stopSpeaking()
```ts theme={null}
async stopSpeaking(): Promise
```
This method is requests the AI Avatar to stop speaking
You can request to when the AI Avatar is speaking.
After the request, the speaking stops after 1 \~ 2 seconds.
* the `aiavatar-stop-speaking` event is triggered.
* The `AIAvatarState` transitions to idle.
**Error Occurrence:**
* The `AIAvatarState` is not speaking when the method is called.
***
## leave()
```ts theme={null}
leave(): void
```
This method is used to exit the `VmonsterRoom`.
Upon execution, the `left` event is triggered immediately.
**Error Occurrence**
* An error will be triggered if the method is called before the `joined` event has occurred.
***
## addVideo()
```ts theme={null}
addVideo(style?: Partial): void;
```
This method adds an AI Avatar Video Element to the DOM.
It checks for the presence of a `