How microservices fail and what to do about it
By Dean McDonnell

A presentation by Richard Rodger at the Microservices Dublin meetup, September 30, 2015

Want to know what are “the three most important lines of code you’ll ever write”? Read on! (Or skip to the end… I can take it. :))

The most recent Microservices Dublin meetup took place on September 30, 2015. The theme of the evening was “How microservices fail and what to do about it”, presented by nearForm CTO Richard Rodger.

Richard tackled head-on the issue of how to get the best use from microservices in message-oriented systems. He set out to answer three key questions:

  • How do you scale to hundreds of microservices?
  • How do you survive the dangers of poison messages and cascading system failure?
  • How can you keep your system healthy under continuous deployment?

The presentation took attendees through many of the failure modes that arise in production, and provided practical advice for dealing with them. According to Richard: “You won’t be able to eradicate failure, but you can control it”.


Richard’s talk was all about accepting and, more importantly, mitigating failure within a typical microservices system. He explained that failure is inevitable, regardless of how microservices interact. He then went a step further and detailed a number of tailored patterns to handle common scenarios.

For instance, if you have a slow downstream service that is causing issues, the key is to tailor your transport mechanism to simply drop the downstream so that it is no longer the point of failure. In cases where an upstream service becomes overloaded, callers should be able to handle backpack pressure and scale back accordingly.

The presentation covered all of the common scenarios that are used in microservice communication: synchronous, request / response, asynchronous, and fire-and-forget interactions.

Richard also demonstrated nodezoo - a package search engine powered by npm and GitHub – as a practical example of how to compose a product from a number of microservices. It uses Seneca, a message-based microservices framework built by Richard and others that has some goodies baked in to make building similar systems a breeze. It was great to see that nodezoo is available on GitHub for people to fork and tinker with.

And those three all-important lines of code? Take a look at Richard’s presentation and slides to find out.

PS. The next Microservices Dublin meetup is on Wednesday October 28, 2015, at 6:30 pm. Attendance is free, as always. Event details and registration are here.

New Call-to-action
join the discussion