Modular Monoliths
A modular monolith is an architectural pattern which is seeing more and more traction at the moment. It’s not actually a particularly new concept. And like most things in software engineering, it has simply cycled back into the forefront of discussions.
This article provides some theory around the topic and an approach you might want to take when building with this architectural pattern in mind.
Table of Contents
Why are we talking about modular monoliths?
Modular monoliths are not a novel idea.
But they’ve recently come up in many discussions around architecture.
And it primarily seems to be as a backlash result of
the microservices movement which came over the last decade or so.
Microservices solves a set of problems under a distinct set of scenarios. They are not without their problems.
So of course, when the likes of Netflix built their system with the microservices approach
to solve their specific problems,
then the rest of our industry exhibited shiny toy syndrome and decided microservices was their silver bullet too.
You can probably guess where this is heading!
Why not microservices?
Microservices mean that your system is split across multiple repositories/components. Each service therefore designates its own boundaries.
The idea is that teams can develop their services in isolation, and each service can be scaled independently of each other.
But it also means that there are multiple CI/CD pipelines, codebases and overall system complexity. Modules now have to talk to each other over network instead of via processes. And deploying microservices necessitates additional orchestration efforts (hi Kubernetes).
Generally, architectures can and probably should reflect your team structure. So if you have a small team, then microservices can result in development friction without the right tooling, processes and design.
Without good design, microservices can often result in distributed monoliths, whereby services become too tightly coupled to each other and effectively exhibit the worst of both worlds.
And the vast majority of applications also do not need to support the kind of load and scale that Netflix does.
(Modular)
Right so what’s the difference between a monolith and a modular monolith? What does that extra word mean in this context?
Think of monoliths as the single-repository opposite of microservices. They are encompassed within the 1 codebase/component, which can mitigate system complexity. This also means just the 1 main CI/CD pipeline to maintain and no real requirement for all the bells and whistles of orchestrating deployments that come with microservices.
Of course, the tradeoffs mean that the artifact will be larger which will affect deployment times. And as you might have guessed, when you have a large team(s) working on the same codebase, development frictions can arise.
Monoliths are often be layered horizontally and there are no internal boundaries to be concerned with.
One might say that modular monoliths bridge the proverbial gap between monolithic architectures and microservices. With a modular monolith, we would draw distinct internal boundaries between modules. Often, we would want these modules to communicate with each other only via explicit interfaces.
Why would I want a modular monolith?
Managing a modular monolith can be much easier than say microservices, particularly for smaller teams.
Building a modular monolith means that you keep system complexity down in the short term. But it also gives you the freedom to split modules out into their own components later down the line when the time feels right.
And this is the crucial driving force here. No one can predict the future. So it seems sensible to build just enough for your system to deliver its functionality today. But leave the door open for large changes to your system to accommodate say larger scale.
Let’s say we had a greenfield project that eventually grew over time. With more usage and a larger development team, we might want to split a module out into its own service. In my opinion, this is a sensible and natural approach. Since our team has the freedom to make the necessary tradeoffs when the time is right.
In this scenario, the components are split into modules internally. The modules should ideally be independent of each other. This allows us to quite easily spin each module out into dedicated services later down the line should the need arise. And that is incredibly valuable for a development team.
Boundaries have to be drawn and respected. This requires discipline and thoughtful design.
Admittedly, development teams will have the same challenges with other architectural patterns. But this is all the more pronounced and important when it comes to building a modular monolith.
We can’t predict the future. But building just enough for what we need today, whilst positioning our systems to be nimble and easy to change is the goal.
The limitations
There are of course a number of limitations which you need to factor in when considering your architecture.
- With any flavour of a monolithic architecture, there is only the single runtime, and therefore you are restricted to selecting the 1 main language. With distributed architectures, each service can be written in any language, so long as they expose some form of API.
- With all the services contained within the 1 monolith, there are more tests to run and dependencies on other teams. This can result in slower CI/CD pipelines.
- The larger artifact also means that the entire system must be redeployed for every change. This of course, does not hold true for non-monolithic architectures.
Enforcing boundaries
For a modular monolithic architecture, the module boundaries have to be drawn into place and respected. A set of conventions or documentation which lives outside the confines of the codebase might feel somewhat superficial.
It’s one thing to decide on an architecture. But how do we apply it and more importantly enforce it? How do we ensure that existing and new team members don’t unknowingly break those boundaries?
Asking all engineers to build their system with those boundaries in mind with reference to some external document feels like additional unnecessary cognitive load. This can also be an intimidating thing, particularly for new engineers.
Let’s say we have the following structure:
In our simple example, we have 2 modules which should remain independent of each other.
module_a/
module_b/
If we broke this down into rules which could be understood at the application level. We would probably come up with the rule that code within module_a should not import code from module_b and vice versa.
This is where static analysis tooling can really help us.
We can enforce this rule along with any explicit exceptions to account for. For example, if we wanted to allow cross-modules communication via designated interfaces or if we wanted to wire in certain pieces of functionality to a central location.
With tooling in place, the engineers can get near-instant feedback in the form of continuous integration (CI) builds. Our CI builds will tell them if their code obeyed the architectural constraints imposed by the design of the system.
This in itself can help remove the friction between architect and development team. We typically use CI pipelines for automated testing and code quality/linting tasks. But verifying our code against the bigger picture design can also easily be enabled within the same pipeline.
Summary
Modular monoliths can offer the best of worlds in certain situations. Particularly for greenfield projects or small development teams.
They allow us to build our system with a monolithic framework whilst giving us the flexibility to break out from there when the need arises.
This requires a thoughtful design approach, particularly in regard to creating the right modules. Design the modules to be too small and there will be too much in the way of cross-module communication. Build the modules to be too big, and the eventual services to be broken out will be too big.
Related articles coming soon: