SSO (aka S.S.Octopus) is one year old!

S.S. Octopus is celebrating a year! SSO was initially created to solve a problem here at BuzzFeed — we needed a way to reliably secure our 600+ microservices (check out this blog post for a look back in time and a more detailed overview of SSO). Not only has it continued to be a great solution for microservice […]

SSO (S.S. Octopus) logo with a birthday cake, confetti in the background
S.S. Octopus is celebrating a year!

SSO was initially created to solve a problem here at BuzzFeed — we needed a way to reliably secure our 600+ microservices (check out this blog post for a look back in time and a more detailed overview of SSO). Not only has it continued to be a great solution for microservice authentication for us, but it’s also been utilized by a range of companies and individuals outside of BuzzFeed; had ongoing support from the open source community, and had a whole host of improvements and bug fixes added.

This Fall, we celebrated SSO’s one year anniversary. We wanted to take this opportunity to reflect back on what we’ve done with SSO since we open sourced it, as well as thank everyone who has contributed to the project — especially our open-source contributors!

So, what’s changed?

We’ve gone Multi-Provider

Perhaps one of the largest changes is that SSO now supports Okta as an identity provider in addition to Google. Okta is used extensively here at BuzzFeed, and adding support for it in SSO is something that had been frequently requested by the open source community, so we saw a great opportunity to have a wide-reaching impact.

Internally for us, this was a big deal. Google had experienced several outages that affected our ability to authenticate through SSO. The introduction of another provider gave us something very important: it gave us options (that didn’t involve a potentially questionable change like drastically increasing the SSO session lifetime).

We are no longer tied to and reliant on any single provider, and in the event of an outage with either provider, we have the option of switching to authenticate via whichever one isn’t in the middle of a meltdown. It also meant that we could push more internal systems behind Okta — something that our IT team had been working towards for a while.

This Is Fine Dog Meme

It was, however, a big undertaking to migrate all of our 140+ SSO-protected upstreams (multiplied by multiple environments!) that originally used the Google provider to use the new Okta provider. We needed to work out a way to go through this process that didn’t require a hard change of provider for all upstreams at once — ultimately minimizing the risks that are typically associated with such large scale migrations.

We decided SSO should be able to:

  1. Use multiple identity providers at once
  2. Allow each individual upstream to easily set which provider to use (with as little as a one-line configuration change)

This meant we had much better control over the rollout: we could move upstreams to a new provider in batches while having full confidence that no other upstreams would be affected.

Implementing this functionality had its own set of hurdles to tackle. The idea of using multiple providers at once was entirely new to SSO, so we needed to modify and split up large parts of SSO’s code:

  • Authenticator logic needed to be changed to be on a one-to-one basis with upstreams, rather than one-to-many.
  • Routing needed to be restructured to support multiple provider routes.
  • Google has a unique way of providing the group data that we use for group-based authorization. Okta handles this very differently, so we needed to implement a new mechanism for caching these groups.
  • A new configuration variable mechanism needed to be introduced to support configuration for multiple, and different identity providers.
  • The UX while authenticating needed to be molded to allow this new flow with different providers and to understand a new set of possible error scenarios and states a request may come through with.

It often pays to put in the extra work and time to make migrations like this more manageable. By doing so we achieved an extremely ‘quiet’ rollout with no downtime to the services being migrated, minimal disruption to engineers and no one being woken up in the night by alerts.

We’re also looking to add support for more third-party identity providers in the near future!

Everyone loves environment variables, but…

The use of configuration variables and how they were being managed across the codebase was rigid and limiting. It involved a large ‘options’ object being initialized at runtime and passed around as needed. The limits of this method were evident when we needed to support the configuration of multiple and different identity providers at once.

With a ground-up rewrite of the configuration mechanism, we introduced a whole new backend for managing configuration variables within the SSO Authenticator component (SSO Proxy still to come!) using the open sourced library go-config.

This gave us the flexibility to maintain multiple configurations at once, and also opened up the doors for providing new ways of defining these variables; such as by CLI flags, from files or even something remote like Consul.

So many tests…

Tests are always a vital part of developing and maintaining a healthy codebase, but it can quickly become difficult to properly understand the usefulness of a set of tests and what is actually being tested. A good starting point to improve in this area is by having a mechanism to monitor the test coverage across the repo.

We integrated CodeCov into the repo and configured it to provide useful test coverage stats and details on pull requests. It helps us and other contributors gauge the impact of new tests and gives us detailed information around what is lacking tests.

Using the data CodeCov gives us, we can now confidently say that test coverage has increased overall!

Example CodeCov automated pull request comment showing detailed stats on test coverage
Example CodeCov automated pull request comment showing detailed stats on test coverage

All aboard the Kubernetes ship.

Originally SSO hadn’t really been tested on Kubernetes — there existed no I also don’t guides, configurations or quick-start steps for integrating the two together. Once again, thanks to open-source contributors (specifically @while1eq1) SSO can officially run on Kubernetes!

Check out this blog post @while1eq1 wrote for more information!

Here We Go (Mod)!

SSO started out by using gpm— an outdated package manager for Golang. The leap (a +243,015 / −165 line change leap, to be precise) was taken to migrate over to use go modand take that much-needed step into the future, and in this case…it is greener on the other side! Thanks, @slyr!

Spreading the word

We’ve been fortunate enough to give a whole multitude of talks and presentations on SSO over the last year in various formats and locations.

Seeing the interest that SSO has gotten at these events, feedback from its adopters, and community engagement on the repository has really helped us keep the momentum going.

“SSO drastically simplified an important part of our stack while also making it more robust. Our team depends on it daily and delivers in spades.” — Daniel McGrath, Director of Engineering @ Button

  • The SSO Docker image has been downloaded over 500k times
  • 1.5k 👏 on its original release blog post.
  • ⭐ ️by ~2.2k people on GitHub
  • Used in production by numerous other organizations such as Button, CommonBond and The Markup
  • 12 community contributors have committed a total of 29 commits
  • The repository receives thousands of views each month, with hundreds of unique visitors

It goes without saying but this progress wouldn’t have been possible without the help and support of everyone at BuzzFeed, and all of our open-source contributors.

Huge thanks to all that have been involved!

SSO Octopus image/mascot holding Google and Okta flag

BuzzFeed Tech is hiring. 👋 If you are interested in browsing openings, check out We have roles in Los Angeles, Minneapolis, London, and New York!

You can also find out about some of the new things we’re launching at BuzzFeed Tech by following us on Twitter, @buzzfeedexp!

SSO (aka S.S.Octopus) is one year old! was originally published in BuzzFeed Tech on Medium, where people are continuing the conversation by highlighting and responding to this story.

Source: Buzzfeed