How we use activity-oriented metrics to drive Engineering Excellence

At Just Eat we have had for a while a tool called “RAG report” to measure the level of compliance for every service (~450) we have in production.
We recently introduced something called Scorecard, to provide teams with better informations and to introduce some fun and gamification to this process. Scorecard use the RAG report data and provide an extra level of insights on a team by team basis.
As you can see below, Scorecard allows you to see data by feature (service), team, department and to see a full blown matrix of everything

Scorecard Matrix
Scorecard Matrix

Below you can see a sample of a matrix view. It lists all the services ( web sites, apis, databases, workers, lamdas), the team owning the service, the tiering of the service (Tier-1: order taking, Tier-2 order dispatching, etc…) and all the activities we want teams to complete and comply with.
There basic activities we track are:
Pipeline & Infrastructure: Service is in the pipeline and not in a stale status (unreleased code of infrastructure changes), Service is on the latest version of infrastructure as code (a migration we have in progress)
Security: Service is on the latest AMI (continuous activity), is using the latest standard to define Security Groups (a migration we have in progress), etc..
Monitoring and Alerting: Service has all the basic Seyren checks configured
As we become more sophisticated, we introduce new checks to make sure we keep the bar high across all of our services
Scorecard by Service

The last view (below) I would like to show is the team view.
This view is particularly useful in the Engineering / Product conversation of how much “Engineering Excellence” work we should have in each team backlog.
Every team should use this view to make sure there’s a clear balance between work that provide direct business value vs work that has to be done to keep the lights on.
Points = calculated based on #of services * # of things a service should comply with
In the Scorecard by Service, 10 points for green, 5 points for amber, 0 points for red ( or something similar, depending on the activity)
We have the ability to weight activities differently to calculate the final score
Features = Services ( we’ve historically called our services features )
The total amount of points per team is also a very significant indicator for how many complex services a team is supporting and helps to make decision on team size and team formation
Scorecard by Teams

Scorecard by Team
Big shout to Zac Charles ( for building the tool in his spare time 🙂