Macaroons 101: Contextual Confinement

Elegent authorization, for a more civilized age

September 27, 2015

Macaroons, like Fezzes, are cool. If you find yourself disagreeing, it’s possible you’re thinking of the wrong sort of macaroons

, or you’ve yet to be convinced.11 or you have legitimate concerns This is the first in a series of posts in which I intend to explain why macaroons are so interesting. We’ll start by motivating them and defining them, and later expand on the theory behind them and study examples of how they’re applicable in the real world.

The name “macaroon” comes from the Google research paper that introduced them, a play on the ubiquitous browser “cookie.” (Naturally, by naming them macaroons, Google has made them nigh on ungooglable.)

What is a macaroon? Much like a signed cookie, a macaroon is a form of bearer credential that can be handed to a client and verified by a server at a later time. Unlike simple bearer tokens, macaroons embed “caveats” that confine the context in which they can be used. This allows decentralized access control that can be difficult with other methods, and in particular is simple, efficient, and flexible.

Before delving into the technical details of macaroons, lets look at some interesting22 contrived authorization problems (and some non-macaroon solutions).

Protecting Frank

Alice shares a home with a pretty adorable dachshund named Frank of whom she takes pretty adorable pictures. She likes to upload these to her Pics of Frank blog, but lately people have been hotlinking her images directly, which costs her a lot of bandwidth. She’d prefer that any view of her images count as a page view of her entire blog - she has a couple of unobtrusive ads there and she’d like to use the extra cash to buy toys for Frank.

Low Hanging Fruit: Checking the Referer

Any request to the server should include a Referer header indicating from where the request originated. Alice can add some checks to all image requests to see if the referer is her own blog, and reject (or redirect) requests that don’t match.

This works pretty well to prevent people who are innocently hotlinking her photos - but from monitoring her traffic she can see that there are some people who are intentionally stealing bandwidth by spoofing the Referer header!

Unique Image Keys

Alice decides to take her attackers head on. She makes all of her image files private - only accessible from within her server. Before every page load, her blog now does the following:

When a page on her blog renders, her server dereferences the unique urls into the corresponding image paths, and sends the image data. Her key-value store deletes the keys after 5 seconds, so that subsequent requests using them fail.

This strategy was a lot of work to implement, but it works well! Anyone who wishes to link to her photos directly must first render her blog in order to get access to them.

Unfortunately, she finds that the money she’s spending running her key-value store is greater than the additional money she’s making from increased page views. There will be no new toys for Frank.

Sharing Frank

Alice puts that problem aside for now. Her friend Bob is a dachshund fan and a photoshop wizard, and she’d like to share her pictures with him so that he can touch them up for her.

She needs to give him write access to her dachshund photos, but doesn’t want to give him full access to her server (she trusts him, but he’d probably just screw something up).

Given that she already has a system in place for assigning unique ids to her images, she figures she can do the same to allow write access. She could generate non-expiring random ids that allow write access, and only give them to Bob (over a secure channel). But Bob, though a talented photo manipulator, is not very careful with security - she wouldn’t be surprised if he shared the write-able link with someone, not realizing there was a difference between the read and write links.

After some thought, she comes up with the following solution:

This solution works pretty well for Alice. It effectively limits write-access to her photos to someone who can authenticate as Bob with Google, and places some limits to prevent him from sharing write access with anyone else. But she’s not happy about a couple of things:

Alice would like to redo this whole system without all of this rigmarole.

Going Stateless

There’s a common pattern of keeping some server state in an authenticated browser cookie33 In its simplest form, an authenticated cookie is just a blob of data with a signature attached. The unforgeable signature is created on the server, and can be verified later by the server. This is commonly done with an HMAC. - Alice realizes that the same pattern would remove her need for a key value store. She generates a nice, random secret for her server, which she’ll use to sign her new, stateless tokens.

For a reader of her blog, this is how it works:

token = (
    "file.jpg,2015-09-16 21:13:23;" +
    HMAC(secret_key, "file.jpg,2015-09-16 21:13:23")
)

Bob, who would additionally need write access, still needs to authenticate with Google. But let’s assume that Google provides Bob with a token that, when given to Alice’s website, can be verified as his and from Google. Any write request that includes both the “read” token from above, and includes Bob’s proof of authentication with Google, will be allowed. We’ll assume the token from Google has additional restrictions to be safe (only valid for a certain period of time, etc).

At this point, Alice has essentially created an ad-hoc, inextensible, standard-less form of macaroons. Macaroons improve on this authenticated-cookie strategy by:

Introducing Macaroons

One of the problems with plain bearer tokens is that they authorize unconditionally. In the scenario above, Alice can’t simply generate tokens to allow writing to her photos without worrying about who can get their hands on them (and then needs to go to great lengths to ensure they’re not used by anyone but Bob).

Macaroons solve this by introducing caveats. Caveats confine the context in which a macaroon is valid. A macaroon with appropriate caveats could confine a token to a user authorized as Bob with Google, restrict it to write access, and limit the time in which it can be used.

A macaroon consists of:

The identifier and location fields of a macaroon serve to identify the key to use for verification of the signature. The signature itself is a chained-MAC of the previous fields. That is, given a secret key key and identifier id, the signature s1 would be

s1 = HMAC(key, id)

If a validity-period caveat were added such as time < 2015-08-03T15:42:48, the new signature s2 would be

s2 = HMAC(s1, 'time < 2012-08-03T15:42:48-04:00')

The chaining allows macaroons to be extended (attenuated) with further caveats. There are several practical implications of this property:

First- and Third-party Caveats

Caveats come in two flavors: first- and third-party.

First party caveats make assertions that the target service can verify. Some example restrictions include:

These are all things that can be verified easily at the target service. If a macaroon restricts access to photo_of_frank.jpg and is read only, the server can reject any requests that are not for those specific actions.

Third-party caveats make assertions about external systems. In order to verify those assertions, the target service must have some pre-arranged relationship with them.55 This relationship could be almost anything: a shared symmetric key, an explicit API for creating ad-hoc keys, or a public/private key pair. The third-party caveat is actually the combination of the key for another macaroon that comes from the third party (a discharge macaroon66 The macaroon provided by a third-party service is called a discharge macaroon because it “discharges” the caveat’s assertion. Presence of a valid discharge macaroon means that the assertions made in the third-party caveat hold.), along with an identifier that matches the identifier of the discharge macaroon.

The target-service (Alice’s blog) will embed a caveat requiring the third party (Google) to issue a specific macaroon that will prove a user is authenticated. Because discharging the third-party caveat is simply verification of another macaroon, additional caveats can be included by the third-party in the discharge macaroon. These caveats will also need to be verifiable (and verified) by the target service in order for the macaroon to be considered valid.

An example helps to clarify. Suppose Alice’s blog wants to issue a macaroon for writing to photo_of_frank.jpg and needs to add a third-party caveat asserting that Bob can authenticate with Google.

Verification

The verification process for a macaroon is fairly simple:

  1. Collect all first party caveats from the root macaroon and all discharge macaroons.
  2. Verify that the first party caveats hold. If any fail, the request is rejected.
  3. Decrypt the keys for each discharge macaroon and verify their signatures.
  4. Find the key for the root macaroon and verify its signature.
  5. If none of the above verifications have failed, the request is allowed through.

It’s important to note that the macaroon should not be “read” to see what request should occur. If a macaroon authorizes write access to photo_of_frank.jpg, any request that includes that macaroon should not be assumed to be a write request to that photo. A holder of a macaroon should have to explicitly specify what action is being performed. It would be even better if a client had to choose a specific authorization macaroon to present for a particular action, though that isn’t possible when using macaroons as cookies.

Applications

Macaroons are attractive because of their flexibility. Alice’s dachshund blog is just one contrived example; macaroons can also be used for:

I intend to write more in-depth about these applications, but wanted to have some sort of “foundational” post to refer to. Rest assured that anything that is hand-waved above will be clarified in a later post. A careful reader will have noticed a lack of detail on the actual format of tokens, no mention of a standard format for caveats, no discussion of the security properties of macaroons, and nowhere near enough technical detail around third-party caveats and discharge macaroons. It’s also worth mentioning that although there are de-facto standards for macaroons, the definition in the research paper leaves a lot up to implementation.

Further Reading

If you’re interested in learning more, here are some links to current writing and projects related to macaroons:

Macaroons 101: Contextual Confinement - September 27, 2015 - Evan Cordell