Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -218,6 +218,21 @@ as well as to custom React-based frontends that handle message rendering themsel

In order to support the streaming of responses, when the MetadataEvent `TOCK_STREAM_RESPONSE` is set to true, a message with the same responseId will be replaced with the new version of the message.

The streaming contract (metadata of a response envelope):

| Metadata | Meaning |
|---|---|
| `TOCK_STREAM_PARTIAL=true` | Transient frame of an answer being generated. Not counted as a consumed message and not stored in the local history. |
| `TOCK_STREAM_REPLACE=<actionId>` | Final response: only the message whose `actionId` matches replaces the streamed bubble, the other messages are added in order. |
| `TOCK_STREAM_DISCARD=true` | The final response has no text: the streamed bubble is removed. |
| `TOCK_REQUEST_ID` | Identifier sent by the kit as `requestId` with each question and echoed on every frame. Frames of another in-flight request are ignored. |

By default, each streamed frame carries the whole text received so far. With the `streamMode: 'delta'` option, the kit adds `"streamMode": "delta"` to its requests; the server then sends frames flagged with the MetadataEvent `TOCK_STREAM_MODE` set to `delta`, whose text is only the end of the (already rendered) text. `TOCK_STREAM_KEEP` gives the number of characters of the current message to keep before appending it, so that the displayed text is the same as in the default mode. The final response always carries the complete text and replaces the message. Servers that do not support this option ignore it.

Frames flagged `TOCK_STREAM_PARTIAL` do not count as messages consumed from the POST response; every other SSE frame does, so that the POST response only adds the messages not already received over SSE. A `TOCK_STREAM_DISCARD` frame carries no message and counts for none.

Note: a delta frame received while no streamed bubble exists yet displays only its fragment until the final response replaces it. An invalid `TOCK_STREAM_KEEP` is ignored and the message waits for the final response.


## API Reference

Expand Down Expand Up @@ -382,6 +397,7 @@ Objects implementing this interface can be passed to `renderChat` or to `TockCon
|------------------------|------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `disableSse` | `boolean?` | If `true`, disables any SSE connection attempt |
| `extraHeadersProvider` | `() => Promise<Record<string, string>>?` | Provides extra HTTP headers for outgoing requests |
| `streamMode` | `'delta'?` | If `'delta'`, asks the server to stream only the new fragments of a response instead of the whole text so far (see [Streamed responses](#streamed-responses)) |
| `retryOnPingTimeoutMs` | `number?` | If SSE is enabled, when this duration in milliseconds elapses without receiving ping events from the backend, the SSE connection is considered to be in an error state and gets restarted |

#### `RendererSettings`
Expand Down Expand Up @@ -461,6 +477,7 @@ Contains all the properties from [`TockSettings`](#tocksettings) as well as the
| `timeoutBetweenMessage` | `number?` | Timeout between message |
| `widgets` | `any?` | Custom display component |
| `disableSse` | `boolean?` | Disable SSE (not even trying) |
| `streamMode` | `'delta'?` | Ask the server to stream only new fragments (see [Streamed responses](#streamed-responses)) |
| `accessibility` | `TockAccessibility` | Object for overriding role and label accessibility attributes |
| `localStorageHistory` | `TockLocalStorage?` | Object for history local storage |

Expand Down
1 change: 1 addition & 0 deletions src/TockOptions.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ export interface TockOptions extends TockOptionalSettings {
timeoutBetweenMessage?: number;
widgets?: { [id: string]: (props: unknown) => JSX.Element };
disableSse?: boolean;
streamMode?: 'delta';

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This new field is redundant, TockOptions already contains the network settings through inheritance. Most properties in this specific interface are legacy support.

accessibility?: TockAccessibility;
localStorageHistory?: TockLocalStorage;
}
Expand Down
63 changes: 51 additions & 12 deletions src/TockState.tsx
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import { Context, createContext, Dispatch, Reducer, useContext } from 'react';
import { QuickReply } from './model/buttons';
import { Message } from './model/messages';
import { Message, MessageType } from './model/messages';
import { parseKeep } from './streaming';

export const TockStateContext: Context<TockState | undefined> = createContext<
TockState | undefined
Expand Down Expand Up @@ -40,13 +41,15 @@ export interface TockAction {
| 'SET_QUICKREPLIES'
| 'ADD_MESSAGE'
| 'UPDATE_MESSAGE'
| 'DISCARD_STREAMED_MESSAGE'
| 'SET_METADATA'
| 'SET_LOADING'
| 'SET_SSE_INITIALIZING'
| 'CLEAR_MESSAGES'
| 'SET_ERROR';
quickReplies?: QuickReply[];
messages?: Message[];
responseId?: string;
loading?: boolean;
sseInitializing?: boolean;
metadata?: Record<string, string>;
Expand Down Expand Up @@ -81,28 +84,50 @@ export const tockReducer: Reducer<TockState, TockAction> = (
if (action.messages) {
let messageAlreadyExists = false;

const newMessages = state.messages.map((message) => {
const newMessages = state.messages.map((current) => {
// Search if a streamed message with the same responseId already exists
const messageExists = action.messages?.find((m) => {
const isMessageExists =
m.metadata?.RESPONSE_ID === message.metadata?.RESPONSE_ID &&
const incoming = action.messages?.find((m) => {
const exists =
m.metadata?.RESPONSE_ID === current.metadata?.RESPONSE_ID &&
m.metadata?.TOCK_STREAM_RESPONSE === 'true' &&
message.metadata?.TOCK_STREAM_RESPONSE === 'true';
current.metadata?.TOCK_STREAM_RESPONSE === 'true';

if (isMessageExists) {
if (exists) {
messageAlreadyExists = true;
}

return isMessageExists;
return exists;
});

// If the message already exists, replace with the new message
if (messageExists) {
return messageExists;
// If the message already exists, replace it with the new message,
// or append the fragment to it when the server streams deltas
if (incoming) {
if (
incoming.metadata?.TOCK_STREAM_MODE === 'delta' &&
current.type === MessageType.message &&
incoming.type === MessageType.message
) {
// Desynchronized stream: wait for the final response
if (current.metadata?.TOCK_STREAM_DESYNC === 'true') {
return current;
}
const keep = parseKeep(incoming.metadata?.TOCK_STREAM_KEEP);
if (keep === undefined || keep > current.message.length) {
return {
...current,
metadata: { ...current.metadata, TOCK_STREAM_DESYNC: 'true' },
};
}
return {
...incoming,
message: current.message.slice(0, keep) + incoming.message,
};
}
return incoming;
}

// Else return the original message
return message;
return current;
});

// If the message is new, add it to the messages
Expand Down Expand Up @@ -138,6 +163,20 @@ export const tockReducer: Reducer<TockState, TockAction> = (
};
}
break;
case 'DISCARD_STREAMED_MESSAGE':
if (action.responseId) {
return {
...state,
messages: state.messages.filter(
(m) =>
!(
m.metadata?.RESPONSE_ID === action.responseId &&
m.metadata?.TOCK_STREAM_RESPONSE === 'true'
),
),
};
}
break;
case 'SET_LOADING':
if (action.loading != undefined) {
return {
Expand Down
1 change: 1 addition & 0 deletions src/model/responses.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ export interface BotConnectorMessage {
widget?: BotConnectorWidget;
image?: BotConnectorImage;
deepLink?: string;
actionId?: string;
}

export interface BotConnectorCard {
Expand Down
3 changes: 3 additions & 0 deletions src/renderChat.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,9 @@ export const renderChat: (
if (options.disableSse) {
network.disableSse = true;
}
if (options.streamMode) {
network.streamMode = options.streamMode;
}
if (options.extraHeadersProvider) {
network.extraHeadersProvider = options.extraHeadersProvider;
}
Expand Down
1 change: 1 addition & 0 deletions src/settings/TockSettings.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ export interface LocalStorageSettings {

export interface NetworkSettings {
disableSse: boolean;
streamMode?: 'delta';
retryOnPingTimeoutMs: number;
extraHeadersProvider?: () => Promise<Record<string, string>>;
}
Expand Down
72 changes: 72 additions & 0 deletions src/streaming.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
import { BotConnectorResponse } from './model/responses';

type Metadata = Record<string, string> | undefined;

/**
* Streaming contract of the Tock web connector (metadata of a response envelope):
* - `TOCK_STREAM_RESPONSE=true`: the message replaces the one with the same `RESPONSE_ID`;
* - `TOCK_STREAM_PARTIAL=true`: transient frame of an answer being generated (never counted, never recorded);
* - `TOCK_STREAM_REPLACE=<actionId>`: the final message that replaces the streamed bubble, the others are added;
* - `TOCK_STREAM_DISCARD=true`: the streamed bubble must be removed;
* - `TOCK_REQUEST_ID`: identifier sent by the client with the question, echoed on every frame.
*/
export const isPartialFrame = (metadata: Metadata): boolean =>
metadata?.TOCK_STREAM_PARTIAL === 'true';

export const isDiscardFrame = (metadata: Metadata): boolean =>
metadata?.TOCK_STREAM_DISCARD === 'true';

/** True for the frames of the contract that designates (or removes) the streamed bubble. */
export const usesStreamContract = (metadata: Metadata): boolean =>
isPartialFrame(metadata) ||
isDiscardFrame(metadata) ||
metadata?.TOCK_STREAM_REPLACE !== undefined;

/** Metadata of a message that must not behave as the streamed bubble (stream flags removed). */
export const withoutStreamFlags = (
metadata: Record<string, string> | undefined,
): Record<string, string> | undefined => {
if (!metadata) return metadata;
const {
TOCK_STREAM_RESPONSE: _response,
TOCK_STREAM_PARTIAL: _partial,
TOCK_STREAM_REPLACE: _replace,
TOCK_STREAM_DISCARD: _discard,
TOCK_STREAM_MODE: _mode,
TOCK_STREAM_KEEP: _keep,
...rest
} = metadata;
return rest;
};

/** Parses TOCK_STREAM_KEEP: a non-negative integer, otherwise undefined. */
export const parseKeep = (value: string | undefined): number | undefined =>
value !== undefined && /^\d+$/.test(value) ? Number(value) : undefined;

/**
* A frame is correlated to a request (`TOCK_REQUEST_ID`) that this client is not
* handling, so it must be ignored. The rule is stable for the whole lifetime of a
* stream: a correlated frame is accepted only while its request is in flight here.
*
* It is deliberately NOT gated on `inFlightRequestIds.size > 0`: otherwise an idle
* tab (no request in flight) would accept a foreign tab's partial frames, then start
* rejecting the rest the moment it sends its own question — leaving an orphan
* provisional bubble that is never finalized nor recorded. For the same reason, a late
* frame arriving after the local request completed (success or error — the id is then
* removed from the in-flight set) is treated as foreign and dropped, so it cannot
* revive a closed/errored request.
*
* Frames without a request identifier (e.g. notifications) always pass.
*/
export const isFrameOfOtherRequest = (
botResponse: BotConnectorResponse,
inFlightRequestIds: ReadonlySet<string>,
): boolean => {
const requestId = botResponse.metadata?.TOCK_REQUEST_ID;
return !!requestId && !inFlightRequestIds.has(requestId);
};

export const newRequestId = (): string =>
typeof crypto !== 'undefined' && typeof crypto.randomUUID === 'function'
? crypto.randomUUID()
: `${Date.now().toString(36)}-${Math.random().toString(36).slice(2)}`;
Loading
Loading