Coroutines - Why They Matter

Let say we want to write an Echo Server. Simplistically we will write:

void runSession(Socket session) {
    while (true) {
        auto data = session.read(255);
        if (data.length() = 0) {
            break;
        }
        session.write(data);
    }
    session.close();
}

void startServer() {
    while (notStopped) {
        Socket server;
        server.bind("", 7);
        server.listen(3);
        Socket session = server.accept();
        runSession(std::move(session));
        server.close();
    }
}

To keep things simple I have excluded error handling from this example. Also assume a socket class that wraps the Posix C APIs.

The above program follows a simple step-by-step implementation making it very straight forward to follow. It is very limited though - only one client can connect at a time.

This is because your program will be suspended while for data reads and writes to complete. While waiting, your program will be suspended by the operating system. Most servers however need to allow multiple clients to interact concurrently.

A simple strategy to allow multiple clients to connect is to give each an every session its own thread. the line runSession(std::move(session)); might become std::thread(runSession, std::move(session)); Now when one session is suspended the operating system has the opportunity to use is CPU time to process another session.

reads can be made non-blocking by calling the

1. Threads are an expensive operating system resource which will limit your to perhaps hundreds of sessions. Were you to have tens or hundred of thousand of sessions, you might run out of threads the OS is willing to provide.
2. How and when threads run is totally up to the operating system. Any synchronisation must therefore be protected somehow. This is easy in principle but can be very complex in practice.

So how can we then run multiple sessions concurrently without resorting to multi-threading?

On POSIX, we have the select and poll. Linux adds the epoll family of APIs on top of this. This will allow us to wait of a number of file descriptors. When of the file descriptors becomes readable, writable or registers an error, the API returns the now active file descriptor. Once a file descriptor become active, we can ensure that a read or write will complete immediately.

We might then have something like:

while (serverActive) {
    auto activeFds = doPoll(fds);
    for (fd : fds) {
        processFd(fd);
    }
}

void processFd(int fd)
{
    auto fdObject = lookupObject(fd);
    switch(fdObject->state) {
    case awaitingRead:
        fdObject->doRead();
        break;
    case awaitingRead:
        fdObject->doWrite();
        break;
    }
}

Where before we had a simple runSession function, we now have a fancy state-machine. Because an echo server is simple, we only have two states, but for anything more complicated, we can end up with 10, 20 or even more states.This all makes following the logic for a simple session difficult.

The issue is that a simple function or subroutine call enters at the beginning and once it exits, cannot be resumes. But what if we could suspend our session function if we were waiting a read and use the time to process another read. What if we could then resume the function once our read was ready.

Well that is exactly what coroutines do and it is because of their potential to simplify networking code that makes me really excited that the C++ standards committee has included them in the soon the be released C++ 20 standard. jump to its caller