Streaming text over HTTP with NextJS

AI chatbots often show you text coming in as it’s being generated.

Your browser makes a request to an API endpoint, and the endpoint streams the text back to you as it becomes available from the AI.

There are lots of tools that handle this kind of thing for you, but I think it’s fun and worthwhile to understand how you can do things yourself.

So today’s small learning objective was to stream Lorem Ipsum text over HTTP.

I used the Vercel AI SDK and NextJS. (and curl to test)

Woah there!

Hey, big shot — I thought you said you were building this yourself.

What’s with using the Vercel AI SDK?

Okay — yes, you’re right. I’m not fully building this myself. (are we ever?)

But my goal was to be able to take any arbitrary text and stream it back to the caller.

For example, I’m interested in running models locally with Ollama and streaming the responses to clients.

Let’s get right to it — here’s the code.

It generates and streams a bunch of Lorem Ipsum text. I added a sleep function so that the text would come in slower — making it more obvious that it was, in fact, streaming and not just coming in all at once.

To test it, you can use curl and add the -N flag (short for --no-buffer) to see the text immediately as it arrives. (otherwise curl will keep it all in a buffer until the connection closes before showing you anything)

My test command was: curl -N http://localhost:3000/lorem

Note: I’ve shortened the lorem ipsum text in the code example below, but if you’re copy/pasting and messing around, you’d want to make it longer.

 1// This is all in a file called: app/lorem/route.ts
 2 
 3import { StreamingTextResponse } from 'ai';
 4 
 5function sleep(ms: number): Promise<void> {
 6    return new Promise(resolve => setTimeout(resolve, ms));
 7}
 8 
 9function createLoremIpsumStream(wordCount: number): ReadableStream<string> {
10    // Swap this out for longer lorem ipsum text
11    const loremIpsumText = "Lorem ipsum dolor sit amet...";
12 
13    let words = loremIpsumText.split(' ');
14    let currentIndex = 0;
15 
16    return new ReadableStream<string>({
17        start(controller) {
18            async function push() {
19                if (currentIndex < wordCount) {
20                    controller.enqueue(words[currentIndex % words.length] + ' ');
21                    currentIndex++;
22                    await sleep(15);
23                    push();
24                } else {
25                    controller.close();
26                }
27            }
28 
29            push();
30        }
31    });
32}
33 
34export async function GET() {
35    const count = 100;
36    const stream = createLoremIpsumStream(count)
37 
38    return new StreamingTextResponse(stream)
39}