note

This article was last updated on April 27, 2023, 1 year ago. The content may be out of date.

Previously I have written a post about ttyd that describes how to authenticate requests for ttyd sessions. The problem is that vanilla ttyd only supports running a predefined command, and it’s nearly impossible to customize what command to run based on the authenticated user. To solve this problem we need to port ttyd to another language. But first we need to understand how ttyd works.

Request Waterfall

When a browser connects to a ttyd instance, below is the order in which resources are requested.

Ttyd Requests Waterfall

Downloading Frontend

The first request, whose url depends on ttyd configuration, is where ttyd frontend is downloaded.

Frontend Response

Getting Auth Token

The second request is getting the auth token. It’s used later when establishing the websocket connection.

Auth Token

Connecting to Websocket

The third request is a websocket request. This is where most of the ttyd protocol is.

Websocket Message Exchange

ttyd Websocket Protocol

We could reverse engineer ttyd protocol using devtools. But since ttyd is an open source tool, we can simply inspect its source code to find out how it’s working.

The definitions of message types are listed in server.h:

 7
 8
 9
10
11
12
13
14
15
16
17

// client message
#define INPUT '0'
#define RESIZE_TERMINAL '1'
#define PAUSE '2'
#define RESUME '3'
#define JSON_DATA '{'

// server message
#define OUTPUT '0'
#define SET_WINDOW_TITLE '1'
#define SET_PREFERENCES '2'

client message is what clients send to the server and vice versa.

Client Message

protocol.c shows how ttyd servers are handling different types of client messages:

307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354

      switch (command) {
        case INPUT:
          if (server->readonly) break;
          int err = pty_write(pss->process, pty_buf_init(pss->buffer + 1, pss->len - 1));
          if (err) {
            lwsl_err("uv_write: %s (%s)\n", uv_err_name(err), uv_strerror(err));
            return -1;
          }
          break;
        case RESIZE_TERMINAL:
          if (pss->process == NULL) break;
          json_object_put(
              parse_window_size(pss->buffer + 1, pss->len - 1, &pss->process->columns, &pss->process->rows));
          pty_resize(pss->process);
          break;
        case PAUSE:
          pty_pause(pss->process);
          break;
        case RESUME:
          pty_resume(pss->process);
          break;
        case JSON_DATA:
          if (pss->process != NULL) break;
          uint16_t columns = 0;
          uint16_t rows = 0;
          json_object *obj = parse_window_size(pss->buffer, pss->len, &columns, &rows);
          if (server->credential != NULL) {
            struct json_object *o = NULL;
            if (json_object_object_get_ex(obj, "AuthToken", &o)) {
              const char *token = json_object_get_string(o);
              if (token != NULL && !strcmp(token, server->credential))
                pss->authenticated = true;
              else
                lwsl_warn("WS authentication failed with token: %s\n", token);
            }
            if (!pss->authenticated) {
              json_object_put(obj);
              lws_close_reason(wsi, LWS_CLOSE_STATUS_POLICY_VIOLATION, NULL, 0);
              return -1;
            }
          }
          json_object_put(obj);
          if (!spawn_process(pss, columns, rows)) return 1;
          break;
        default:
          lwsl_warn("ignored unknown message type: %c\n", command);
          break;
      }
CommandDescription
INPUTIf ttyd is configured as readonly, do nothing, else pass the data to the pty process
RESIZE_TERMINALttyp resizes the window of the pty process according to the size contained within the json data.
PAUSEttyp pauses reading from the pty process.
RESUMEttyp resumes reading from the pty process.
JSON_DATAThe opening brace is part of the data. The data contains the auth token and the window size of the client. ttyd checks if the auth token is valid and if so, spawns the predefined process with the window size.

Server Message

From the frontend codes:

289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314

    @bind
    private onSocketData(event: MessageEvent) {
        const { textDecoder } = this;
        const rawData = event.data as ArrayBuffer;
        const cmd = String.fromCharCode(new Uint8Array(rawData)[0]);
        const data = rawData.slice(1);

        switch (cmd) {
            case Command.OUTPUT:
                this.writeFunc(data);
                break;
            case Command.SET_WINDOW_TITLE:
                this.title = textDecoder.decode(data);
                document.title = this.title;
                break;
            case Command.SET_PREFERENCES:
                this.applyPreferences({
                    ...this.options.clientOptions,
                    ...JSON.parse(textDecoder.decode(data)),
                } as Preferences);
                break;
            default:
                console.warn(`[ttyd] unknown command: ${cmd}`);
                break;
        }
    }
CommandDescription
SET_WINDOW_TITLEchange the title of the browser tab in which the ttyd session is running to the specifiy value.
SET_PREFERENCESapply the preferences contained in the json data.
OUTPUTrender the data using xterm.js

Session Establishment

As can be seen from the first three messages of the websocket connection. A client first sends the JSON_DATA message to the server. The server responds by sending two messages: SET_WINDOW_TITLE and SET_PREFERENCES.

Flow Control

When rendering the data using xterm.js, xterm.js can be overwhelmed with too much data. ttyd solves this problem by using flow control:

192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213

    @bind
    public writeData(data: string | Uint8Array) {
        const { terminal, textEncoder } = this;
        const { limit, highWater, lowWater } = this.options.flowControl;

        this.written += data.length;
        if (this.written > limit) {
            terminal.write(data, () => {
                this.pending = Math.max(this.pending - 1, 0);
                if (this.pending < lowWater) {
                    this.socket?.send(textEncoder.encode(Command.PAUSE));
                }
            });
            this.pending++;
            this.written = 0;
            if (this.pending > highWater) {
                this.socket?.send(textEncoder.encode(Command.RESUME));
            }
        } else {
            terminal.write(data);
        }
    }

What it does is after having received data totalling a specific size, xterm.js renders the data and calls a callback function, increases the pending counter and reset the data counter. If the number of pending calls is above a certain threshold, frontend sends a RESUME message. When xterm.js finishes rendering the data, it calls the callback function and if the number of pending calls is below another threshold, frontend sends a PAUSE message.

info

RESUME and PAUSE should be swapped. It’s already fixed in 01f1ed5.

To Be Continued

The next part will deal with how to implement ttyd protocol in golang.