EventSource: Native Browser Streaming API

Ever found yourself wrestling with a slow backend task, leaving users staring at a blank screen and wondering if your app has frozen? The HTML <progress> element – a perfect visual solution. But how to feed it real-time data without complex protocols or constant polling? Say hello to EventSource. This native browser API is built-in some browsers since 2010. It's super simple to use. Three lines of JavaScript for the (simplest) frontend, paired with a straightforward backend setup (we'll use Python with Django), and done.

EventSource API, the Frontend

EventSource is a Web API that enables real-time communication from a server to a client using server-sent events (SSE). It establishes a persistent connection, allowing the server to send updates in a simple text format. This unidirectional data flow is ideal for applications like live notifications and news feeds, as it automatically reconnects if the connection is lost, ensuring a seamless user experience.

EventSource benefits from HTTP/2, which allows multiple simultaneous streams and eliminates connection limit per domain found in older HTTP versions. The code is as simple as this:

const eventSource = new EventSource('/api/progress'); 
eventSource.onmessage = (event) => { 
    console.log(`Progress ${event.data}%`); 
};

I had not known about this until some weeks ago and the simplicity is amazing. I had once written a stream using JSONL (newline-delimited JSON), I thought it was elegant, but this is even simpler.

The backend

The code below creates a view which should be reachable at /api/progress as the frontend example uses above. The important thing here is that the view must return a a StreamingHttpResponse which is a django class. We use the according content-type text/event-stream and send a line of data for example for 42% we send data: 42\n\n. The two newlines (\n\n) at the end indicate the end of this message.

def progress_view(_):
    def stream():
        for percent in range(100):
            yield f'data: {percent}\n\n'
            # Do the slow work here.

    return StreamingHttpResponse(stream(), content_type='text/event-stream')

In Django we use a generator, which is a function that yields multiple results, in our case until we reached 100 and then it ends. We are just sending each percentage from 0 to 100 to the frontend.

In the browser's development tool you can see in the network traffic, that a stream of data like this is being sent:

data: 0

data: 1

data: 2

data: 3

data: 4

...

That is how little code is needed to send the data in a django app. Feel free to throw the code into your next AI and ask it how the backend is done in your preferred language/framework. Just make sure you are at least on HTTP2.

What is "text/event-stream"

Wikipedia says that the "media type for SSE is text/event-stream" (SSE means Server-Sent Events). It had first been specified in 2004, wow. Trying to understand where the format of the message is defined I am stumbling over MDN again, but this is not the specification, it's just a documentation. Anyway, in there it states that the "Event stream format":

• is a simple stream of text data

• encoded using UTF-8

• messages are separated by a pair of newline characters

• a colon as the first character of a line is a comment, and is ignored

• a message consists of one or more lines of text listing the fields for that message

• each field is represented by the field name, followed by a colon, followed by the text data for that field's value

This means our message we are sending data: value\n\n the data is just the field name, which allows us to access the data in the frontend using event.data.

So if we sent a message from the backend as foo: value\n\n would we use event.foo in the frontend? No, this field will be ignored. There is another restriction to the fields, only four fields are valid:

• data: We saw this one. We used Number() on it. It can be anything, for example a JSON string, just use JSON.parse() with it in the frontend.

• id: An ID, since all the data in our hands, I guess we can use it to make messages have a unique ID, or alike.

• retry: An integer, the time in milliseconds before attempting to reconnect if the server connection is lost.

• event: Fire a special event, if you set this eventSource.onmessage does not fire, you must use eventSource.addEventListener("eventName", ...) to receive the data.

• all other fields are ignored.

I found the place where the data format for the EventSource message is specified, it's in the HTML specification in the chapter 9.2.6 Interpreting an event stream (very well readable).

A Complete Frontend Example

The above frontend code was very much simplified and left out what a good code should also do: error handling and closing connections.

Below is the code, that handles the closing of the connection on the various occasions, when reaching 100%, on error and when the user leaves the page. The intention is to reduce the risk of running into memory leak issues because connections stay open of JS objects just linger around.

const updateProgress = (percentage) => { 
    // Instead of console.log this will probably update
    // the <progress> HTML element.
    console.log(`Progress ${percentage}%`); 

    if (percentage === 100) {
        // Close the connection, we are done.
        eventSource.close();
    }
};

const eventSource = new EventSource('/api/progress'); 
// Note: The following assumes the data sent is always numerical,
// otherwise an ugly `Progress NaN%` might show to the user. 
eventSource.onmessage = event => updateProgress(Number(event.data));
// Let's cleanup properly on error and when leaving the page.
eventSource.onerror = () => eventSource.close();
window.addEventListener('beforeunload', () => eventSource.close());

Have fun streaming data from the backend to the frontend the simplest way I know.

Do you have any comments or feedback? Please let me know.