JavaScript Promises ... in Wicked Detail

I've been using promises in my JavaScript code for a while now. They can be a little brain bending at first. I now use them pretty effectively, but when it came down to it, I didn't fully understand how they work. This article is my resolution to that. If you stick around until the end, you should understand promises well too.

We will be incrementally creating a promise implementation that by the end will mostly meet the Promises/A+ spec, and understand how promises meet the needs of asynchronous programming along the way. This article assumes you already have some familiarity with promises. If you don't, promisejs.org is a good site to checkout.

Translations

Japanese Translation, translated by Junpei Tajima

Change log

2021-01-07 Trimmed down further reading and translations due to the pages no longer existing :(
2017-10-07 Added a fiddle to the This Code is Brittle and Bad section to demonstrate how it can fail.
2014-12-23: Added Recovering from Rejection section. The article was a bit ambiguous on handling rejection, this new section should clear things up.

Why?

Why bother to understand promises to this level of detail? Really understanding how something works can increase your ability to take advantage of it, and debug it more successfully when things go wrong. I was inspired to write this article when a coworker and I got stumped on a tricky promise scenario. Had I known then what I know now, we wouldn't have gotten stumped.

The Simplest Use Case

Let's begin our promise implementation as simple as can be. We want to go from this

doSomething(function (value) {
    console.log('Got a value:' + value);
});

to this

doSomething().then(function (value) {
    console.log('Got a value:' + value);
});

To do this, we just need to change doSomething() from this

function doSomething(callback) {
    var value = 42;
    callback(value);
}

to this "promise" based solution

function doSomething() {
    return {
        then: function (callback) {
            var value = 42;
            callback(value);
        },
    };
}

JS Fiddle

This is just a little sugar for the callback pattern. It's pretty pointless sugar so far. But it's a start and yet we've already hit upon a core idea behind promises

Promises capture the notion of an eventual value into an object

This is the main reason promises are so interesting. Once the concept of eventuality is captured like this, we can begin to do some very powerful things. We'll explore this more later on.

Defining the Promise type

This simple object literal isn't going to hold up. Let's define an actual Promise type that we'll be able to expand upon

function Promise(fn) {
    var callback = null;
    this.then = function (cb) {
        callback = cb;
    };

    function resolve(value) {
        callback(value);
    }

    fn(resolve);
}

and reimplement doSomething() to use it

function doSomething() {
    return new Promise(function (resolve) {
        var value = 42;
        resolve(value);
    });
}

There is a problem here. If you trace through the execution, you'll see that resolve() gets called before then(), which means callback will be null. Let's hide this problem in a little hack involving setTimeout

function Promise(fn) {
    var callback = null;
    this.then = function (cb) {
        callback = cb;
    };

    function resolve(value) {
        // force callback to be called in the next
        // iteration of the event loop, giving
        // callback a chance to be set by then()
        setTimeout(function () {
            callback(value);
        }, 1);
    }

    fn(resolve);
}

JS Fiddle

With the hack in place, this code now works ... sort of.

This Code is Brittle and Bad

Our naive, poor promise implementation must use asynchronicity to work. It's easy to make it fail again, just call then() asynchronously and we are right back to the callback being null again. Why am I setting you up for failure so soon? Because the above implementation has the advantage of being pretty easy to wrap your head around. then() and resolve() won't go away. They are key concepts in promises.

Here is an example of what I mean:

JS Fiddle

If you open up the console, you'll see an error about the callback not being a function, because then() was called in a setTimeout.

Promises have State

Our brittle code above revealed something unexpectedly. Promises have state. We need to know what state they are in before proceeding, and make sure we move through the states correctly. Doing so gets rid of the brittleness.

A promise can be pending waiting for a value, or resolved with a value. Once a promise resolves to a value, it will always remain at that value and never resolve again.

(A promise can also be rejected, but we'll get to error handling later)

Let's explicitly track the state inside of our implementation, which will allow us to do away with our hack

function Promise(fn) {
    var state = 'pending';
    var value;
    var deferred;

    function resolve(newValue) {
        value = newValue;
        state = 'resolved';

        if (deferred) {
            handle(deferred);
        }
    }

    function handle(onResolved) {
        if (state === 'pending') {
            deferred = onResolved;
            return;
        }

        onResolved(value);
    }

    this.then = function (onResolved) {
        handle(onResolved);
    };

    fn(resolve);
}

JS Fiddle

It's getting more complicated, but the caller can invoke then() whenever they want, and the callee can invoke resolve() whenever they want. It fully works with synchronous or asynchronous code.

This is because of the state flag. Both then() and resolve() hand off to the new method handle(), which will do one of two things depending on the situation:

The caller has called then() before the callee calls resolve(), that means there is no value ready to hand back. In this case the state will be pending, and so we hold onto the caller's callback to use later. Later when resolve() gets called, we can then invoke the callback and send the value on its way.
The callee calls resolve() before the caller calls then(): In this case we hold onto the resulting value. Once then() gets called, we are ready to hand back the value.

Notice setTimeout went away? That's temporary, it will be coming back. But one thing at a time.

With promises, the order in which we work with them doesn't matter. We are free to call then() and resolve() whenever they suit our purposes. This is one of the powerful advantages of capturing the notion of eventual results into an object

We still have quite a few more things in the spec to implement, but our promises are already pretty powerful. This system allows us to call then() as many times as we want, we will always get the same value back

var promise = doSomething();

promise.then(function (value) {
    console.log('Got a value:', value);
});

promise.then(function (value) {
    console.log('Got the same value again:', value);
});

This is not completely true for the promise implementation in this article. If the opposite happens, ie the caller calls then() multiple times before resolve() is called, only the last call to then() will be honored. The fix for this is to keep a running list of deferreds inside of the promise instead of just one. I decided to not do that in the interest of keeping the article more simple, it's long enough as it is :)

Chaining Promises

Since promises capture the notion of asynchronicity in an object, we can chain them, map them, have them run in parallel or sequential, all kinds of useful things. Code like the following is very common with promises

getSomeData().then(filterTheData).then(processTheData).then(displayTheData);

getSomeData is returning a promise, as evidenced by the call to then(), but the result of that first then must also be a promise, as we call then() again (and yet again!) That's exactly what happens, if we can convince then() to return a promise, things get more interesting.

then() always returns a promise

Here is our promise type with chaining added in

function Promise(fn) {
    var state = 'pending';
    var value;
    var deferred = null;

    function resolve(newValue) {
        value = newValue;
        state = 'resolved';

        if (deferred) {
            handle(deferred);
        }
    }

    function handle(handler) {
        if (state === 'pending') {
            deferred = handler;
            return;
        }

        if (!handler.onResolved) {
            handler.resolve(value);
            return;
        }

        var ret = handler.onResolved(value);
        handler.resolve(ret);
    }

    this.then = function (onResolved) {
        return new Promise(function (resolve) {
            handle({
                onResolved: onResolved,
                resolve: resolve,
            });
        });
    };

    fn(resolve);
}

JS Fiddle

Hoo, it's getting a little squirrelly. Aren't you glad we're building this up slowly? The real key here is that then() is returning a new promise.

Since then() always returns a new promise object, there will always be at least one promise object that gets created, resolved and then ignored. Which can be seen as wasteful. The callback approach does not have this problem. Another ding against promises. You can start to appreciate why some in the JavaScript community have shunned them.

What value does the second promise resolve to? It receives the return value of the first promise. This is happening at the bottom of handle(), The handler object carries around both an onResolved callback as well as a reference to resolve(). There is more than one copy of resolve() floating around, each promise gets their own copy of this function, and a closure for it to run within. This is the bridge from the first promise to the second. We are concluding the first promise at this line:

var ret = handler.onResolved(value);

In the examples I've been using here, handler.onResolved is

function(value) {
    console.log("Got a value:", value);
}

in other words, it's what was passed into the first call to then(). The return value of that first handler is used to resolve the second promise. Thus chaining is accomplished

doSomething()
    .then(function (result) {
        console.log('first result', result);
        return 88;
    })
    .then(function (secondResult) {
        console.log('second result', secondResult);
    });

// the output is
//
// first result 42
// second result 88

doSomething()
    .then(function (result) {
        console.log('first result', result);
        // not explicitly returning anything
    })
    .then(function (secondResult) {
        console.log('second result', secondResult);
    });

// now the output is
//
// first result 42
// second result undefined

Since then() always returns a new promise, this chaining can go as deep as we like

doSomething()
    .then(function (result) {
        console.log('first result', result);
        return 88;
    })
    .then(function (secondResult) {
        console.log('second result', secondResult);
        return 99;
    })
    .then(function (thirdResult) {
        console.log('third result', thirdResult);
        return 200;
    })
    .then(function (fourthResult) {
        // on and on...
    });

What if in the above example, we wanted all the results in the end? With chaining, we would need to manually build up the result ourself

doSomething().then(function(result) {
    var results = [result];
    results.push(88);
    return results;
}).then(function(results) {
    results.push(99);
    return results;
}).then(function(results) {
    console.log(results.join(', ');
});

// the output is
//
// 42, 88, 99

Promises always resolve to one value. If you need to pass more than one value along, you need to create a multi-value in some fashion (an array, an object, concatting strings, etc)

A potentially better way is to use a promise library's all() method or any number of other utility methods that increase the usefulness of promises, which I'll leave to you to go and discover.

The Callback is Optional

The callback to then() is not strictly required. If you leave it off, the promise resolves to the same value as the previous promise

doSomething()
    .then()
    .then(function (result) {
        console.log('got a result', result);
    });

// the output is
//
// got a result 42

You can see this inside of handle(), where if there is no callback, it simply resolves the promise and exits. value is still the value of the previous promise.

if (!handler.onResolved) {
    handler.resolve(value);
    return;
}

Returning Promises Inside the Chain

Our chaining implementation is a bit naive. It's blindly passing the resolved values down the line. What if one of the resolved values is a promise? For example

doSomething()
    .then(function (result) {
        // doSomethingElse returns a promise
        return doSomethingElse(result);
    })
    .then(function (finalResult) {
        console.log('the final result is', finalResult);
    });

As it stands now, the above won't do what we want. finalResult won't actually be a fully resolved value, it will instead be a promise. To get the intended result, we'd need to do

doSomething()
    .then(function (result) {
        // doSomethingElse returns a promise
        return doSomethingElse(result);
    })
    .then(function (anotherPromise) {
        anotherPromise.then(function (finalResult) {
            console.log('the final result is', finalResult);
        });
    });

Who wants that crud in their code? Let's have the promise implementation seamlessly handle this for us. This is simple to do, inside of resolve() just add a special case if the resolved value is a promise

function resolve(newValue) {
    if (newValue && typeof newValue.then === 'function') {
        newValue.then(resolve);
        return;
    }

    state = 'resolved';
    value = newValue;

    if (deferred) {
        handle(deferred);
    }
}

JS Fiddle

We'll keep calling resolve() recursively as long as we get a promise back. Once it's no longer a promise, then proceed as before.

It is possible for this to be an infinite loop. The Promises/A+ spec recommends implementations detect infinite loops, but it's not required.

Also worth pointing out, this implementation does not meet the spec. Nor will we fully meet the spec in this regard in the article. For the more curious, I recommend reading the promise resolution procedure.

Notice how loose the check is to see if newValue is a promise? We are only looking for a then() method. This duck typing is intentional, it allows different promise implementations to interopt with each other. It's actually quite common for promise libraries to intermingle, as different third party libraries you use can each use different promise implementations.

Different promise implementations can interopt with each other, as long as they all are following the spec properly.

With chaining in place, our implementation is pretty complete. But we've completely ignored error handling.

Rejecting Promises

When something goes wrong during the course of a promise, it needs to be rejected with a reason. How does the caller know when this happens? They can find out by passing in a second callback to then()

doSomething().then(
    function (value) {
        console.log('Success!', value);
    },
    function (error) {
        console.log('Uh oh', error);
    }
);

As mentioned earlier, the promise will transition from pending to either resolved or rejected never both. In other words, only one of the above callbacks ever gets called.

Promises enable rejection by means of reject(), the evil twin of resolve(). Here is doSomething() with error handling support added

function doSomething() {
    return new Promise(function (resolve, reject) {
        var result = somehowGetTheValue();
        if (result.error) {
            reject(result.error);
        } else {
            resolve(result.value);
        }
    });
}

Inside the promise implementation, we need to account for rejection.

Let's see the full promise implementation again, this time with rejection support added

function Promise(fn) {
    var state = 'pending';
    var value;
    var deferred = null;

    function resolve(newValue) {
        if (newValue && typeof newValue.then === 'function') {
            newValue.then(resolve, reject);
            return;
        }

        state = 'resolved';
        value = newValue;

        if (deferred) {
            handle(deferred);
        }
    }

    function reject(reason) {
        state = 'rejected';
        value = reason;

        if (deferred) {
            handle(deferred);
        }
    }

    function handle(handler) {
        if (state === 'pending') {
            deferred = handler;
            return;
        }

        var handlerCallback;

        if (state === 'resolved') {
            handlerCallback = handler.onResolved;
        } else {
            handlerCallback = handler.onRejected;
        }

        if (!handlerCallback) {
            if (state === 'resolved') {
                handler.resolve(value);
            } else {
                handler.reject(value);
            }

            return;
        }

        var ret = handlerCallback(value);
        handler.resolve(ret);
    }

    this.then = function (onResolved, onRejected) {
        return new Promise(function (resolve, reject) {
            handle({
                onResolved: onResolved,
                onRejected: onRejected,
                resolve: resolve,
                reject: reject,
            });
        });
    };

    fn(resolve, reject);
}

JS Fiddle

Other than the addition of reject() itself, handle() also has to be aware of rejection. Within handle(), either the rejection path or resolve path will be taken depending on the value of state. This value of state gets pushed into the next promise, because calling the next promises' resolve() or reject() sets its state value accordingly.

When using promises, it's very easy to omit the error callback. But if you do, you'll never get any indication something went wrong. At the very least, the final promise in your chain should have an error callback. See the section further down about swallowed errors for more info.

Unexpected Errors Should Also Lead to Rejection

So far our error handling only accounts for known errors. It's possible an unhandled exception will happen, completely ruining everything. It's essential that the promise implementation catch these exceptions and reject accordingly.

This means that resolve() should get wrapped in a try/catch block

function resolve(newValue) {
    try {
        // ... as before
    } catch (e) {
        reject(e);
    }
}

It's also important to make sure the callbacks given to us by the caller don't throw unhandled exceptions. These callbacks are called in handle(), so we end up with

function handle(deferred) {
    // ... as before

    var ret;
    try {
        ret = handlerCallback(value);
    } catch (e) {
        handler.reject(e);
        return;
    }

    handler.resolve(ret);
}

Promises can Swallow Errors!

It's possible for a misunderstanding of promises to lead to completely swallowed errors! This trips people up a lot

Consider this example

function getSomeJson() {
    return new Promise(function (resolve, reject) {
        var badJson = 'uh oh, this is not JSON at all!
';
        resolve(badJson);
    });
}

getSomeJson().then(
    function (json) {
        var obj = JSON.parse(json);
        console.log(obj);
    },
    function (error) {
        console.log('uh oh', error);
    }
);

JS Fiddle

What is going to happen here? Our callback inside then() is expecting some valid JSON. So it naively tries to parse it, which leads to an exception. But we have an error callback, so we're good, right?

Nope. That error callback will not be invoked! If you run this example via the above fiddle, you will get no output at all. No errors, no nothing. Pure chilling silence.

Why is this? Since the unhandled exception took place in our callback to then(), it is being caught inside of handle(). This causes handle() to reject the promise that then() returned, not the promise we are already responding to, as that promise has already properly resolved.

Always remember, inside of then()'s callback, the promise you are responding to has already resolved. The result of your callback will have no influence on this promise

If you want to capture the above error, you need an error callback further downstream

getSomeJson()
    .then(function (json) {
        var obj = JSON.parse(json);
        console.log(obj);
    })
    .then(null, function (error) {
        console.log('an error occurred: ', error);
    });

Now we will properly log the error.

In my experience, this is the biggest pitfall of promises. Read onto the next section for a potentially better solution

done() to the Rescue

Most (but not all) promise libraries have a done() method. It's very similar to then(), except it avoids the above pitfalls of then().

done() can be called whenever then() can. The key differences are it does not return a promise, and any unhandled exception inside of done() is not captured by the promise implementation. In other words, done() represents when the entire promise chain has fully resolved. Our getSomeJson() example can be more robust using done()

getSomeJson().done(function (json) {
    // when this throws, it won't be swallowed
    var obj = JSON.parse(json);
    console.log(obj);
});

done() also takes an error callback, done(callback, errback), just like then() does, and since the entire promise resolution is, well, done, you are assured of being informed of any errors that erupted.

done() is not part of the Promises/A+ spec (at least not yet), so your promise library of choice might not have it. I have worked with one implementation as recently as 2018, you just never know with all the legacy code out there!

Recovering from Rejection

It is possible to recover from a rejected promise. If you pass in an errback to then(), from then on any further promises in this chain will be resolved instead of rejected:

aMethodThatRejects()
    .then(
        function (result) {
            // won't get here
        },
        function (err) {
            // since aMethodThatRejects calls reject()
            // we end up here in the errback
            return 'recovered!';
        }
    )
    .then(
        function (result) {
            console.log('after recovery: ', result);
        },
        function (err) {
            // we won't actually get here
            // since the rejected promise had an errback
        }
    );

// the output is
// after recovery: recovered!

If you don't pass in an errback, then the rejection propagates to the next promise in the chain:

// notice the two calls to then()
aMethodThatRejects()
    .then()
    .then(
        function (result) {
            // we won't get here
        },
        function (err) {
            console.log('error propagated');
        }
    );

// the output is
// error propagated

Promise Resolution Needs to be Async

Early in the article we cheated a bit by using setTimeout. Once we fixed that hack, we've not used setTimeout since. But the truth is the Promises/A+ spec requires that promise resolution happen asynchronously. Meeting this requirement is simple, we simply need to wrap most of handle()'s implementation inside of a setTimeout call

function handle(handler) {
    if (state === 'pending') {
        deferred = handler;
        return;
    }

    setTimeout(function () {
        // ... as before
    }, 1);
}

This is all that is needed. In truth, real promise libraries don't tend to use setTimeout. If the library is NodeJS oriented it will possibly use process.nextTick, for browsers it might use the new setImmediate or a setImmediate shim (so far only IE supports setImmediate), or perhaps an asynchronous library such as Kris Kowal's asap (Kris Kowal also wrote Q, a popular promise library)

Why Is This Async Requirement in the Spec?

It allows for consistency and reliable execution flow. Consider this contrived example

var promise = doAnOperation();
invokeSomething();
promise.then(wrapItAllUp);
invokeSomethingElse();

What is the call flow here? Based on the naming you'd probably guess it is invokeSomething() -> invokeSomethingElse() -> wrapItAllUp(). But this all depends on if the promise resolves synchronously or asynchronously in our current implementation. If doAnOperation() works asynchronously, then that is the call flow. But if it works synchronously, then the call flow is actually invokeSomething() -> wrapItAllUp() -> invokeSomethingElse(), which is probably bad.

To get around this, promises always resolve asynchronously, even if they don't have to. It reduces surprise and allows people to use promises without having to take into consideration asynchronicity when reasoning about their code.

Promises always require at least one more iteration of the event loop to resolve. This is not necessarily true of the standard callback approach.

Before We Wrap Up ... then/promise

There are many, full featured, promise libraries out there. The then organization's promise library takes a simpler approach. It is meant to be a simple implementation that meets the spec and nothing more. If you take a look at their implementation, you should see it looks quite familiar.

At the time this article was written, the final result looked very much like the then/promise implementation. That's no longer true, they've completely rewritten their promise implementation

There are some differences in the real implementation and what is here in this article. That is because there are more details in the Promises/A+ spec that I have not addressed. I recommend reading the spec, it is short and pretty straightforward.

Conclusion

If you made it this far, then thanks for reading! We've covered the core of promises, which is the only thing the spec addresses. Most implementations offer much more functionality, such as all(), spread(), race(), denodeify() and much more. I recommend browsing the API docs for Bluebird to see what all is possible with promises.

Once I came to understand how promises worked and their caveats, I came to really like them. They have led to very clean and elegant code in my projects. There's so much more to talk about too, this article is just the beginning!

JavaScript Promises ... in Wicked Detail

Translations

Change log

Table of Contents

Why?

The Simplest Use Case

Defining the Promise type

This Code is Brittle and Bad

Promises have State

Chaining Promises

The Callback is Optional

Returning Promises Inside the Chain

Rejecting Promises

Unexpected Errors Should Also Lead to Rejection

Promises can Swallow Errors!

done() to the Rescue

Recovering from Rejection

Promise Resolution Needs to be Async

Why Is This Async Requirement in the Spec?

Before We Wrap Up ... then/promise

Conclusion

Further Reading