Macaroons 101: Contextual Confinement

2015-09-27

12 min read

Macaroons, like Fezzes, are cool. If you find yourself disagreeing, it’s possible you’re thinking of the wrong sort of macaroons

macaroons

, or you’ve yet to be convinced. This is the first in a series of posts in which I intend to explain why macaroons are so interesting. We’ll start by motivating them and defining them, and later expand on the theory behind them and study examples of how they’re applicable in the real world.

The name “macaroon” comes from the Google research paper that introduced them, a play on the ubiquitous browser “cookie.” (Naturally, by naming them macaroons, Google has made them nigh on ungooglable.)

What is a macaroon? Much like a signed cookie, a macaroon is a form of bearer credential that can be handed to a client and verified by a server at a later time. Unlike simple bearer tokens, macaroons embed “caveats” that confine the context in which they can be used. This allows decentralized access control that can be difficult with other methods, and in particular is simple, efficient, and flexible.

A Contrived Problem

Before delving into the technical details of macaroons, lets look at some interesting (contrived) authorization problems (and some non-macaroon solutions).

Protecting Frank

frank the dachshund

Alice shares a home with a pretty adorable dachshund named Frank of whom she takes pretty adorable pictures. She likes to upload these to her Pics of Frank blog, but lately people have been hotlinking her images directly, which costs her a lot of bandwidth. She’d prefer that any view of her images count as a page view of her entire blog - she has a couple of unobtrusive ads there and she’d like to use the extra cash to buy toys for Frank.

Low Hanging Fruit: Checking the Referer

Any request to the server should include a Referer header indicating from where the request originated. Alice can add some checks to all image requests to see if the referer is her own blog, and reject (or redirect) requests that don’t match.

This works pretty well to prevent people who are innocently hotlinking her photos - but from monitoring her traffic she can see that there are some people who are intentionally stealing bandwidth by spoofing the Referer header!

Unique Image Keys

Alice decides to take her attackers head on. She makes all of her image files private - only accessible from within her server. Before every page load, her blog now does the following:

finds all photos on the page about to be rendered
generates a unique, random id for each of them
stores those ids in a key-value store along with the image they reference
replaces references to the image files with urls that include the new random keys

When a page on her blog renders, her server dereferences the unique urls into the corresponding image paths, and sends the image data. Her key-value store deletes the keys after 5 seconds, so that subsequent requests using them fail.

This strategy was a lot of work to implement, but it works well! Anyone who wishes to link to her photos directly must first render her blog in order to get access to them.

Unfortunately, she finds that the money she’s spending running her key-value store is greater than the additional money she’s making from increased page views. There will be no new toys for Frank.

frank the dachshund in paint

Alice puts that problem aside for now. Her friend Bob is a dachshund fan and a photoshop wizard, and she’d like to share her pictures with him so that he can touch them up for her.

She needs to give him write access to her dachshund photos, but doesn’t want to give him full access to her server (she trusts him, but he’d probably just screw something up).

Given that she already has a system in place for assigning unique ids to her images, she figures she can do the same to allow write access. She could generate non-expiring random ids that allow write access, and only give them to Bob (over a secure channel). But Bob, though a talented photo manipulator, is not very careful with security - she wouldn’t be surprised if he shared the write-able link with someone, not realizing there was a difference between the read and write links.

After some thought, she comes up with the following solution:

When Bob wants to work on one of the photos, he must first visit a special login page she made for him. /bobs-login/
The login page requires him to authenticate with Google OAuth as bob@example.com (she knows he has an account there already).
If he can do so successfully, the blog site creates a new record in the key value store: like before, the key is a unique, random id, but this time the value is Bob’s ip address when he authenticated. This entry is set to expire in 30 minutes, and for good measure the id is also stored in a cookie on his browser.
When Bob is ready to upload an image, he visits a special upload page /bob-upload/. It can only be viewed if the request includes a cookie with the random id and originates from the corresponding stored ip address.
When loading, the page generates a unique id for each image, and appends Bob’s unique key to each. These ids are only valid for 30 seconds.
Bob picks one of the images to overwrite, and issues a PUT request to the generated url.
If done within 30 seconds, the image is written.

This solution works pretty well for Alice. It effectively limits write-access to her photos to someone who can authenticate as Bob with Google, and places some limits to prevent him from sharing write access with anyone else. But she’s not happy about a couple of things:

she has to host a login page just for Bob
she’s storing a lot of state in the server

Alice would like to redo this whole system without all of this rigmarole.

Going Stateless

There’s a common pattern of keeping some server state in an authenticated browser cookie - Alice realizes that the same pattern would remove her need for a key value store. She generates a nice, random secret for her server, which she’ll use to sign her new, stateless tokens.

For a reader of her blog, this is how it works:

When the page is requested, like before, the server replaces references to her photos with tokens. This time, though, the tokens aren’t random, they are signed tokens that contain a reference to the file and a time that the tokens are no longer valid:

token = (
    "file.jpg,2015-09-16 21:13:23;" +
    HMAC(secret_key, "file.jpg,2015-09-16 21:13:23")
)

When the browser requests the photos, the server decodes the reference, verifies that it hasn’t been tampered with by recalculating the signature, and only then streams the photo data to the client.

Bob, who would additionally need write access, still needs to authenticate with Google. But let’s assume that Google provides Bob with a token that, when given to Alice’s website, can be verified as his and from Google. Any write request that includes both the “read” token from above, and includes Bob’s proof of authentication with Google, will be allowed. We’ll assume the token from Google has additional restrictions to be safe (only valid for a certain period of time, etc).

At this point, Alice has essentially created an ad-hoc, inextensible, standard-less form of macaroons. Macaroons improve on this authenticated-cookie strategy by:

Defining a standard interface for specifying and verifying caveats (examples: proof of authentication as Bob at Google, time restrictions)
Unifying the interface between local assertions and remote assertions (i.e. freshness can be verified in the same way as external authentication)
Permitting extensibility and delegation - Macaroons are like “layered” authentication cookies, where additional restrictions can be trivially added by anyone, or even passed to a different client.

Introducing Macaroons

macaroons in a row

One of the problems with plain bearer tokens is that they authorize unconditionally. In the scenario above, Alice can’t simply generate tokens to allow writing to her photos without worrying about who can get their hands on them (and then needs to go to great lengths to ensure they’re not used by anyone but Bob).

Macaroons solve this by introducing caveats. Caveats confine the context in which a macaroon is valid. A macaroon with appropriate caveats could confine a token to a user authorized as Bob with Google, restrict it to write access, and limit the time in which it can be used.

A macaroon consists of:

an identifier used to differentiate between macaroons
a location that identifies the service from which the macaroon originated
a series of caveats which confine the context for which the macaroon is valid
- first party caveats make assertions about the context of the target service
- third party caveats make assertions about the context of some third-party service (e.g. Google)
a signature that can be used to verify the macaroon has not been modified

The identifier and location fields of a macaroon serve to identify the key to use for verification of the signature. The signature itself is a chained MAC of the previous fields. That is, given a secret key key and identifier id, the signature s1 would be

s1 = HMAC(key, id)

If a validity-period caveat were added such as time < 2015-08-03T15:42:48, the new signature s2 would be

s2 = HMAC(s1, 'time < 2012-08-03T15:42:48-04:00')

The chaining allows macaroons to be extended (attenuated) with further caveats. There are several practical implications of this property:

A client can pass their authority to someone else, while attenuating it further (delegation)
A macaroon with few or no caveats could be cached, and then attenuated for specific situations
Multiple services could coordinate access by passing macaroons to each other, attenuating as appropriate
After attenuation, the target service can still easily (and quickly) verify the macaroon

First- and Third-party Caveats

Macaroon Diagram

Caveats come in two flavors: first- and third-party.

First party caveats make assertions that the target service can verify. Some example restrictions include:

which resources can be accessed
what operations can be performed (read, write, share, etc)
how long a macaroon is valid
a specific user or group of users
a specific ip address or other connection-specific information
a check against a revocation service

These are all things that can be verified easily at the target service. If a macaroon restricts access to photo_of_frank.jpg and is read only, the server can reject any requests that are not for those specific actions.

Third-party caveats make assertions about external systems. In order to verify those assertions, the target service must have some pre-arranged relationship with them. The third-party caveat is actually the combination of the key for another macaroon that comes from the third party (a discharge macaroon), along with an identifier that matches the identifier of the discharge macaroon.

The target-service (Alice’s blog) will embed a caveat requiring the third party (Google) to issue a specific macaroon that will prove a user is authenticated. Because discharging the third-party caveat is simply verification of another macaroon, additional caveats can be included by the third-party in the discharge macaroon. These caveats will also need to be verifiable (and verified) by the target service in order for the macaroon to be considered valid.

An example helps to clarify. Suppose Alice’s blog wants to issue a macaroon for writing to photo_of_frank.jpg and needs to add a third-party caveat asserting that Bob can authenticate with Google.

First, Alice needs to get the key that will be used to sign the discharge macaroon. For the sake of example, lets say she makes a request to Google for a key. This key is included in the third-party caveat of the macaroon that will be handed to Bob.
Bob gets this macaroon, sees that it has a third-party caveat for authenticating with Google. He talks to Google’s macaroon auth service and gets a discharge macaroon signed with the same key that Alice was given.
Bob sends both macaroons to Alice when he wants to write to photo_of_frank.jpg. Because the key to the discharge macaroon is embedded in the root macaroon’s third-party caveat (and is encrypted, though I’ve glossed over that so far), Alice can verify both macaroons and only allow the request through if they’re valid.
Additionally, Google could add a time limit to the discharge macaroon, which would further limit the context for which the macaroon pair is valid.

Verification

The verification process for a macaroon is fairly simple:

Collect all first party caveats from the root macaroon and all discharge macaroons.
Verify that the first party caveats hold. If any fail, the request is rejected.
Decrypt the keys for each discharge macaroon and verify their signatures.
Find the key for the root macaroon and verify its signature.
If none of the above verifications have failed, the request is allowed through.

It’s important to note that the macaroon should not be “read” to see what request should occur. If a macaroon authorizes write access to photo_of_frank.jpg, any request that includes that macaroon should not be assumed to be a write request to that photo. A holder of a macaroon should have to explicitly specify what action is being performed. It would be even better if a client had to choose a specific authorization macaroon to present for a particular action, though that isn’t possible when using macaroons as cookies.

Applications

Macaroons are attractive because of their flexibility. Alice’s dachshund blog is just one contrived example; macaroons can also be used for:

Distributing authorization data in a microservices architecture
Sharing structured auth information between external services
An alternative to JWTs for session management
Better management of session revocation and/or network-less session refreshing
Unifying disparate auth systems
Meaningful OAuth2 tokens

I intend to write more in-depth about these applications, but wanted to have some sort of “foundational” post to refer to. Rest assured that anything that is hand-waved above will be clarified in a later post. A careful reader will have noticed a lack of detail on the actual format of tokens, no mention of a standard format for caveats, no discussion of the security properties of macaroons, and nowhere near enough technical detail around third-party caveats and discharge macaroons. It’s also worth mentioning that although there are de-facto standards for macaroons, the definition in the research paper leaves a lot up to implementation.