OpenSourced: JustSaying (AWS MessageBus)

JUST EAT, AWS and a Message Bus

We’ve recently embraced Event Driven Architecture as a cornerstone for a decent chunk of our platform. This post introduces our latest open source library which is a super simple, easy to use Message Bus built on top of AWS. In later posts I’ll cover the motivations and benefits of our decision and some more in-depth first hand experience of our journey. So, for now…

Introducing “JustSaying“…

 

What is it?

JustSaying is a c# library giving very easy to use Message Bus type functionality on top of Amazon’s SNS and SQS services.
We’ve taken inspiration from the likes of Greg Young, MassTransit, NServiceBus et al. Ultimately we’ve come up with a simplistic, robust AWS centric way of getting messages published and consumed across component boundaries without the need to host any additional infrastructure.
We’ve put a lot of focus on the following:

  • Fluent, readable, extensible configuration syntax
  • Developer friendliness
  • Minimal configuration
  • Un-intrusive extension of existing/legacy codebases

 

Yeah Yeah…whatever – just show me the snippets…

Publishing a message:

public class OrderAccepted : Message
{
    public int OrderId { get; set; }
}
var publisher = JustSaying
    .CreateMeABus.InRegion(RegionEndpoint.EUWest1.SystemName)
    .WithSnsMessagePublisher();
publisher.Publish(new OrderAccepted { OrderId = 123456 });

 
Consuming a message

public class OrderNotifier : IHandler
{
    public bool Handle(OrderAccepted message)
    {
        // Some logic here ...
        return true;
    }
}
JustSaying.CreateMeABus
    .InRegion(RegionEndpoint.EUWest1.SystemName)
    .WithSqsTopicSubscriber()
    .IntoQueue("CustomerOrders")
    .WithMessageHandler(new OrderNotifier())
    .StartListening();

 
More advanced? Hit the read-me on GitHub: https://github.com/justeat/JustSaying
 

How does it work?

SNS as an exchange.
SQS as a delivery mechanism.
Enough said?
 

Why we built it

  1. One of the main things we were missing in JUST EAT’s platform until recently was a structured way to represent State Change across our services, causing us some difficulties in API design and working with our legacy components.
  2. We wanted a more robust system in place for dealing with outages, which resulted in less service interruption and data loss… Hey, we’re in The Cloud – instances go missing! 
  3. It isn’t an easy job to wire together the Amazon services; discovery can be an issue. A tool to make this easy was the only option in a multi (independent) team environment.
  4. Due to the above, we have been running it internally in an OpenSource fashion for some time.
  5. Using SNS and SQS together is extremely cost-effective and reliable.

 

 A voyage of discovery

We’ve taken a highly agile approach of only build what you need, and so the current state of the library is a direct reflection of our problem space. Here’s a (by no means exclusive) list of the features we needed and have built in:

  • Throttling consumers
  • Delayed message delivery
  • Configurable publish and consume retry attempts
  • Monitoring of key metrics
  • Extensible subscription and publishing configuration (for example delivery one per instance for load balanced components)
  • Guaranteed only once delivery
  • Strongly typed message delivery
  • Configurable serialisation strategy
  • Dead letter queues (for failed message handling)

 
Blog posts on some of these detailed topics to come.
 

Downloads

Source code is available on GitHub here:
https://github.com/justeat/JustSaying
NuGet packages available here:
http://www.nuget.org/packages/JustSaying

Leave a Reply to Jim Pelletier Cancel reply

Your email address will not be published. Required fields are marked *

  1. Love the library – Excellent stuff. It seems you have solved publishing events on the bus in a Pub/Sub model but does it work with Request/Response communication model?

    1. Hi there Abhaya,
      Thanks for the JustSaying love!
      The short answer to your question about Request/Response model is: no.
      When we created JustSaying, our goal was to give an easy Pub/Sub interface on top of AWS managed services. Our platform is heavily Http API based, which really covers our Request/Response needs. So, for that reason, we kept it to a simple Async Pub/Sub model.
      We only really build the things into it that we need at the time. However, if you think you have some cool ideas for extending JustSaying then we’d be really interested to have a chat / accept contributions to extend the library.

  2. Hi there, I really like this. Is there anyway to set up more complex routing? for example can I publish a message to multiple queues? or even all queues?
    I would also be interested in how you have handled service discovery?

  3. This is great, thank you for sharing this with the community. However, I’m struggling to get it working from the nuget package.
    I followed the example here:
    https://github.com/justeat/JustSaying/wiki/JustSaying-Installation
    But regardless of what I attempt (and I’ve attempted all variations) I cannot get the keys picked up and I constantly get an InvalidArgumentException trying to ‘CreateMeABus’ – what am I doing wrong?
    The same key to identify the profile works fine for the AWS SNS example.
    Suggestions welcome!
    Thanks
    IH

  4. Hi,
    Great looking library! Me and my team are looking to pick it up for a system we’re about build. This first hurdle we had was the version of the AWS library you use, which it looks like you have a PR in for. We were hoping however to target .net core. I haven’t been through the source yet but wanted to gauge what your interest would be if we submitted a PR porting to .net core?
    Cheers,
    Jim