Interactive Typing

For the first code snippet, I want to talk about about something I like to call “interactive typing”.

Common Lisp

Inspired by a function from a project I was working on a while ago:

(defun wrap-sender (send &optional default-headers default-status)
  "Wrap a sender that accepts strings in a slightly HTTP-aware function.
  The returned function has an internal state and accepts the following
  arguments, in order:
  1. An integer representing the HTTP status code (optional)
  2. A cons representing a HTTP header (0 or more)
  3. A string to be sent verbatim to the client
  Once the third stage is reached the wrapper never changes state anymore.
  The headers are conses:
  - car: A symbol from the keyword package denoting the header
  - cdr: A string or other printable object for the value
  The default-headers argument is a list of headers that will be sent before the
  body if not overruled by an earlier header.
  "
  (labels ((send (&rest argv) (apply send argv)) ;; Bring send to function ns
           (header (x) (send (format NIL "~A: ~A~%" (car x) (cdr x))))
           (status (x) (header (cons 'status x))) ;; FCGI way to set status
           (end-head () (send (format NIL "~%"))))
    ;; State queue
    (let ((l (list (lambda (x) (if (numberp x)
                                 (progn (status x) #'rest)
                                 (when default-status
                                   (status default-status) NIL)))
                   (lambda (x)
                     (if (consp x)
                       (progn (hard-delete (car x) default-headers :key #'car)
                              (header x)
                              #'identity)
                       ;; Process remaining headers and advance (return NIL).
                       (dolist (h default-headers) (header h))))
                   (lambda (x) (declare (ignore x)) (end-head) NIL)
                   (lambda (x) (when x (send x)) #'identity))))
       ;; State machine. It keeps a reference to the current state in the state
       ;; queue.  Every incoming argument is dispatched to that. If the return
       ;; value is a function, that function is called on the remainder of the
       ;; queue and its result is set as the new state. If it returns NIL the
       ;; state advances one step and the action is repeated.
       (labels ((wrapped (x)
                  (let ((next (funcall (first l) x)))
                    (if next
                      (setf l (funcall next l))
                      (progn (pop l) (wrapped x)))
                  x)))
         #'wrapped))))

Python

I will recreate and explain it here using Python.

Interactive typing emphasises the temporal dynamic of the type of an object or function. The way you interact with it changes how you can use it.

Imagine you’re working on a web app. I know, highly unlikely, but bear with me. You want to craft a response. Typically it would go something like this:

response.setStatus(200)
response.setHeader('content-type', 'text/plain; charset=utf-8')
response.setText('Hello, line 1\n... and line 2.')
response.send()

Implementing this is easy enough. It uses buffering: just keep all the data in memory as members of the response object. When .send() is called, write all the data to the client.

This is called buffering.

Sometimes you don’t want to buffer. You want to send data to the client as soon as you have it. For example if the response value is huge (a multi-GB file), or if response lines come in very slowly and you want to feed them separately as soon as you can.

This is called streaming.

Let’s adapt the above snippet to support streaming:

response.sendStatus(200)
response.sendHeader('content-type', 'text/plain; charset=utf-8')
response.sendText('Hello, line 1\n')
time.sleep(5)
response.sendText('... and line 2.')

So far, so good.¹

But now, an error occurs. You want to change the response status to 500:

response.sendStatus(200)
response.sendHeader('content-type', 'text/plain; charset=utf-8')
try:
    response.sendText(getGreetingFromDB())
except DBException:
    response.sendStatus(500)
    response.sendText('There was a DB error.')

Because the API is streaming, reflecting the underlying HTTP connection directly, this is an error: you can’t modify the status once you’ve sent it.

This is not clear from the type of the object. It must be inferred from the documentation. Yuck.

A step up would be to encode this in the type of the response object. Here’s how this could be achieved:

response(200)
response('content-type', 'text/plain; charset=utf-8')
response('Hello, streaming object. ')
response("I've come to talk with you again")

The response object is now a function, but its signature depends on how it has been used before. I don’t know how to express this in any static type system I know of, but in a dynamically typed language, this works.

E.g. Python:

def wrapResponse(response):
    sentStatus = False
    sentHeaders = False
    def temporalResponse(arg1, arg2=None):
        if isinstance(arg1, int):
            if sentStatus:
                raise TypeError('Status code already sent')
            response.sendStatus(arg1)
            sentStatus = True
        elif arg2 is not None:
            if not sentStatus:
                raise TypeError('Must send status before headers')
            if sentHeaders:
                raise TypeError('Headers already sent')
            response.sendHeader(arg1, arg2)
        else:
            if not sentHeaders:
                raise TypeError('Must send headers before body')
            response.sendText(arg1)
    return temporalResponse

Plus some additional type checking, like ensuring a message body is a string.

It’s looking good, but it’s not quite there, yet. In this implementation, we always need to specify that the status is 200, and that the content-type is text/html.

What if that were default, instead?

try:
    response(helloTemplate('<p>Hello from a simple handler.'))
except SomeError as e:
    print(e)
    response(500)
    response('content-type', 'text/plain; charset=utf-8')
    response('Oh no, something went wrong!')

That’s more like it!

To create it, in Python:

class InteractiveResponse(object):
    def __init__(self, response, defaultStatus=200, defaultHeaders={'content-type': 'text/html; charset=latin1'}):
        self._response = response
        self._headers = dict(defaultHeaders)
        self._defaultStatus = defaultStatus
        self._handler = self._handleStatus

    def __call__(self, arg1, arg2=None):
        self._handler = self._handler(arg1, arg2) or self._handler

    def _flushHeaders(self):
        for k, v in self._headers.items():
            self._response.sendHeader(k, v)

    def _handleStatus(self, arg1, arg2=None):
        if not isinstance(arg1, int):
            self._response.sendStatus(self._defaultStatus)
            return self._handleHeader(arg1, arg2)
        else:
            self._response.sendStatus(arg1)
            return self._handleHeader

    def _handleHeader(self, arg1, arg2=None):
        if arg2 is None:
            self._flushHeaders()
            return self._handleText(arg1)
        else:
            self._headers[arg1] = arg2

    def _handleText(self, arg1):
        self._response.sendText(arg1)
        return self._handleText

This is the Python equivalent of the convoluted Lisp function wrap-sender.

It is semantically equivalent to the old streaming API, with custom method names for each task (sendStatus, etc), raising TypeErrors when they are invoked out of order. However, the function notation used here emphasises the order and interactivity of the response type. It reflects the underlying protocol better.

JavaScript

It can be particularly elegantly implemented in JavaScript:

function funcStateMachine(handler) {
    var that = this;
    return function () {
        handler = handler.apply(that, arguments) || handler;
    }
}

var handlers = {
    headers: { "content-type": "text/html; charset=latin1" },
    status: function (status) {
        if (+status !== status) {
            response.sendStatus(200);
            return this.header.apply(this, arguments);
        } else {
            response.sendStatus(status);
            return this.header;
        }
    },
    header: function (key, val) {
        if (val !== undefined) {
            this.headers[key] = val;
        } else {
            for (var k in this.headers) {
                if (this.headers.hasOwnProperty(k)) {
                    response.sendHeader(k, this.headers[k]);
                }
            }
            this.text("\n");
            return this.text.apply(this, arguments);
        }
    },
    text: function (str) {
        response.sendText(str);
        return this.text;
    }
};

var r1 = funcStateMachine.call(handlers, handlers.status);

I like to call it Interactive Typing.

Unfortunately, nobody would expect this type of API. That alone is reason enough not to use it.

Still, it’s a fun experiment.