Monday 31 March 2014

Microservices and Agility

'Microservices' is a new buzz word our industry has found.  The idea is that a system is built with a set of independently managed services.  The idea is catching up as people are seeing more and more problems with the monolithic applications.


  • A bit of history..
Most enterprise applications today get developed in Java and JVM.  Around early 2000, when J2EE / Java was still gaining popularity,  distributed objects was the recommended way of developing enterprise applications. Books like J2EE design patterns pushed for that. People claimed that distributed objects were necessary for scaling.
In practice though, people were experiencing difficulties with the way distributed objects were creating problems. It was considered 'developer's nightmare'.
Rod Johnson, in his book 'Expert one - on -one J2EE development without EJB', claimed that POJO components with dependency injection was much better and lightweight approach to develop better applications. Monoliths are better for managing and can be scaled well as well.
Monolithic web applications using spring / or similar DI frameworks was the way most of the web applications were developed.

  • SOA

Came along, SOA, Service Oriented Architecture. The idea was to break the applications based on the different set of business capabilities and expose those as services, so that they can be consumed by outside clients. Unfortunately, SOA was quickly tied to SOAP and Enterprise Service Buses.
In last four to five years though, REST and HTTP have been clearly established as a de facto standard for exposing services.  This is the preferred 'light weight' approach.


  • Partitioning .
Exposing services along the lines of business capabilities is just one aspect of partitioning the system. By System I mean a software system which enables an organization to run their business.  There are complex relationships between software systems and an organization. The way organization is partitioned impacts the way software systems are partitioned and vice versa. Almost always, these decisions can not be taken by just business people or just software people. The whole point of Agile was to allow business people and software people to work together for that reason, as the quality of this relationship determines quality of software architecture. By Architecture I mean how the system is partitioned into modules or components. They should closely resemble the domain partitioning.



  • Forces for partitioning the system (organization and architecture)
Following are the most common reasons for partitioning the system.
  1. Different rate of change of parts of the system. :- There are parts of the system which change at different rate. For example on a flight booking website, insurance rules might change more frequently than say search options. In such scenario, it might be worth splitting system, so that insurance module can be changed and deployed indendently.
  2. Autonomy of teams working on different features :- Some times its worth splitting the system , so that different teams can work on different parts as independently as possible.  This is one of the common reasons for splitting monoliths. (This is actually driven by Conway's Law which states that parts of the systems reflects the way teams are structured)
  3. Independent domains :- Complex systems almost always have several independent business units. These are reflected in various different domains which are reflected in software system. Eric Evans calls these as 'bounded contexts' in his DDD book. It always beneficial to split the system on domain boundaries and have different modules (and teams) take care of these domains.
  4. Non functional aspects. :-  Different parts of the system can have different non functional requirements. Common requirements like scaling, availability, performance, security are some of the most important reasons for which  system needs to be split into parts. Monoliths often suffer with the problems that the whole monolith needs to be scaled even when a part of the system needs scaling. For example, in flight booking website, flight search is generally very expensive in terms of resources. Its well worth to split the application so that search components can be managed and scaled independently. It also better if these components can naturally map to teams, which can manage this component life cycle independently.  Key management parts of the API might need more secure storage, monitoring and network infrastructure than rest of the application. In that case, pulling key management as a separate service makes a lot of sense.
These four are the most common reasons to partition the system and organization. As we can easily see, the forces aren't just about the technical aspects, but in fact more about organizational aspects. This is the reason, we see Conway's Law mentioned almost always in any write up on micro services.
Of course just Conway's Law is not enough to guide us on how an organization should be structured and how the code should map to teams.
Jim Coplien's great book 'Organizational Patterns of Agile Software Development' has a pattern language called 'People and Code Pattern Language'. This language lists a set of patterns which are necessary for having successful partitioning of the system. Architect Controls the Product, Architect Also Implements, Architecture team, Stand Up Meeting are just to name the few.


  • Microservices or MicroApplications? 

As we can clearly see, there is broader problem of splitting or partitioning the software system and organization at stake here. One of the important parts of these systems is UI, either Web UI or otherwise. If we literally focus just on 'services', we can very easily fall in a trap of having a 'monolithic' web UI using granular back end services.  Considering 'verticals' including UI is essential when thinking of partitioning the systems. The term 'MicroServices' is probably misleading here. Its more about having 'Micro Applications'.


  • Its about being Agile
So, its all about individuals (Business people, architects and developers) and how they decide to implement software and build teams which develop this software. That's what being Agile is all about.
Mapping software artifacts to teams or people, and structuring them to make sure teams can work as effectively as possible is a very important aspect of software development. This is very commonly known as the 'Conway's Law'.
Conway's Law was at the heart of what Agile was. It was always the case. 
So when we say we need 'microservices architecture' we are really saying 'how can we be more Agile by having people and software in sync with business goals'. 
Unfortunately, Agile is thought of in a very very narrow sense now a days. Its generally limited to CI, CD, TDD, iterations etc etc..
Most of what we are talking about 'MicroServices', is really about 'Agile'. The meaning that was lost all these years, is getting rediscovered under a different name.