Break the monolith into microservices
Nowadays, the task of breaking a monolith into microservices has become really popular in the business environment. As if all the corporations of Russia suddenly realized all the prospects of the “new” architecture, they received a kick from their bosses and rushed to take a peak.
The zeal of corporate officials, as always, is crazy and merciless. And again, huge amounts of money allocated for pilots are spent mediocrely on what will never take off at best, and in the worst case, it will be implemented, despite all the mistakes. And, ladies and gentlemen, the most depressing thing is that this phenomenon is by no means an isolated one. It is typical for the entire IT industry in Russia.
In this article, I propose to discuss what errors occur when breaking a monolith into microservice architecture (MCA), why they arise, what they lead to. Well, in the end I will describe how an effective and correct process of transferring to ISA should be organized and what the architecture of a microservice system should be.
So, your organization decided to break up the good old monolith into microservices. Why? As a rule, no one can give a clear answer, but on the whole, everyone begins to describe how they live poorly with the monolith. Yes, release cycles are really very hooked on different teams. And why do you think that is different in ISA? Someone says that microservices scale much better. Well, you can do so, but why do you need it? No, maybe it’s necessary, but do you understand why or just use the spoken phrase? They say that each microservice is much less demanding on resources: memory, database, etc. So what? Have you run out of money on the server? Business value in what?
Colleagues, I do not want to say that MSA has no advantages. These advantages are. But I want to say that no matter what goals you set at the start of the breakdown project, you most likely set them incorrectly and never achieve them. Do not achieve it simply because the project will be implemented by those people who brought your monolithic project to the state of stupidly written code. Moreover, all approaches to the implementation of the ISA are so different from the monolithic, as far as the telephone noodles of the 20th century differ from the network technologies of the current 21st century. And the analogy here is really very successful. PSTN delivers one telephone line to the apartment. A break in any part of the line leads to its failure. Network technologies are dynamic and allow you to build alternative routes. Even the failure of 60% of the country's entire network infrastructure as a result of a nuclear war will not lead to the failure of the remaining networks.
That is, the main advantage of the ISA over the monolith is its dynamism, which can lead to extreme reliability of the system. Here it is necessary to clarify: it does not follow from the ISA itself that it will be reliable. ISA only gives you the opportunity to build reliable systems. But it is your responsibility to lay reliability and duplication of functionality in the architecture of your system.
And now we will analyze the ubiquitous errors in the development of the microservice system of the enterprise.
- Opt Out of Business Process Changes
They say that any business implements its business processes in code. And, if 20 years before that you built your business processes on the basis of your monolithic architecture, then any of your attempts to introduce ISAs will certainly be doomed to failure. Just because this requires changing business processes. An example? How many stages of the coordination of the issue do you have in your organization? So, in a microservice team, the number of such stages approaches zero. The vast majority of issues are resolved within the microservice team. The team is not required to bring questions about the language in which the microservice will be written to the corporate level. Some may choose Java, while others may choose Python. You may not even believe it now, but this issue is entirely in the competence of the team. And this is because in case of any problems, a microservice is easier to RENEW than to SUPPORT.
In my experience, the development of a simple microservice with a normally set production takes a week of work for two jones, including a bureaucracy to bring it to the industry, i.e. costs the company, including various overhead costs, about 50-100 thousand rubles.If your analytics only takes a month of work of 3 teams, then you do not have any ISA in the organization, you are again slipping into a fierce monolithic.
- Assign project management to experienced monolithic workers
After you have abandoned the modernization of your business processes, it is completely logical that you entrust a new project to a very experienced person who has shown himself to be the best and knows all your business processes in general. Perhaps someone nudged you that a new experienced person should look at old problems, but you waved him off like a silly annoying fly. And here he is. Your new/old development manager on the project. Do not be surprised that the result will be the same as before and even worse. When monolithic solutions begin to be applied in ISA, it will not lead to anything good. It was at this very moment that you signed the death sentence to your pilot. You may even recruit experienced people who have already devoured a pack of dogs on microservice projects, but this will not save you, simply because their boss will not listen. Just because his brains are not imprisoned under ISA, and he will take the advice of others as monstrous stupidity.
- Architecture designed by architects with no development experience
Here sits such an architect who in his life has not written a single line of code and issues instructions to developers. Very often, developers are more skilled than himself, but he gives orders as a napoleon. The orders are of course stupid. See examples of these orders.
- Repeat monolithic system functionality.
Well, this is logical: you must first fully reproduce what is already there, and then move on. Play all the bugs and crutches, all the crap that developers spit on for years.
One thing is not clear to me: why did these architects get the idea that then someone will give money for refactoring? Those. you copy your morally obsolete system, don’t bring anything positive from the point of view of functionality and with this 20 year old you will live at least 10 more years until the next refactoring.
- Code Reuse
Any monolithic knows that the code needs to be reused. If you made the User :: toString function, then within the framework of the ISA, you now need to separate it into a separate microservice and all 1000 other microservices will break into it to serialize each user. The fact that the first failure in the microservice puts the whole system in monoliths wasn’t like that.
For the first time, I came across military microservice architecture in 2017. By that time, the system had already been working for 7 years and during this time there was a complete change of team.
Hundreds of microservices. There is no documentation. There is not even a simple description of the purpose of each microservice. In general, what is the status of a microservice? It is functioning, deactivated, experimental, etc. In the event of a problem, the trace of microservices turned into a continuous nightmare. As a result, nobody just stayed on the team for a long time. The techie became the scourge of the team. For 3 months, the team could simply upgrade completely.
This is the true story of a poorly designed microservice system. Such a finale awaits the vast majority of pilots who have now started. And all because of the errors that I listed above.
And now I want to tell you how to develop a microservice system.
- Analyze your previous experience, make a list of what you think is successful and what you should review. Conduct a customer survey, let them request the necessary changes in functionality. Your goal is to achieve several business goals by means of one fine and fine-grained refactoring. Your goal is not to rewrite the same thing 10 times, but to create a business product that will make money. You do not just have to make a copy of the legacy system, but you must build the latest system that will take you to the forefront of the IT industry. Why do you need these old crutches?
- Review your business processes. If you have any agile always slipping into a hard waterfall, if approvals and analytics take months, then your software production requires a radical upgrade. And for this you need.
- ... find a specialist with extensive experience in developing microservice systems. Let him not be dedicated to your business processes, but this new person will look at your old problems with a fresh look and give really valuable recommendations.In addition, only a person with real experience in microservice development is able to foresee those pitfalls that you will meet. Your experienced monolithic will only plug everything.
- Legacy developers are your reserve, which in the new development will be able to act as a valuable expert link. But people who sawed the monolith in the form of crutches should by no means become the backbone of the development of a microservice system and determine its architecture. Monolithic workers should wait for an internship led by people with experience in building microservice systems. And the main thing that awaits monolithic people is a hard brain breakdown and a complete restructuring of thinking.
- Each microservice is not just a small backend function. This is a complete business function development cycle. One team should be fully responsible for the entire development of this business function both on the front-end and in the database. As soon as you begin to delegate part of the development to other teams, you immediately begin to wallow in integrations. Integrations in ISA are even more expensive, unreliable and inefficient than in a monolith. It is necessary to reduce their number. As a microservice, it’s quite normal to choose CRUD for an entity, for example, an entity of a client agreement. This is normal if the team provides both a backend and a frontend microservice (or library) for working with contracts: list, create, update, delete. The main thing is that the team bears full responsibility for how this function will work on the site and in mobiles.
- No need to engage in fanaticism in the implementation of microservice architecture. With the correct microarchitecture of services, you decide in what form to release the application: in a monolithic, microservice or serverless architecture. Expect your business staging to change at any time, and you must implement the changes with ease. Flexible development is something that simply without too much talk should really be applied at all levels.
- Distrust the reuse of functionality. On the contrary, any functionality should be duplicated. Your gain of 10 gigabytes of disk memory will not impress anyone if the directory using 30 microservices is stopped due to a failure, leading to a cascading shutdown of the entire microservice system. Microservices should BUY the failures of third-party services, and not strengthen them. And this principle is the main problem for monolithic. After 20 years of working with relational databases, after decades of experience in normalizing tables and data structures in applications, it is very difficult to accustom yourself to the idea that duplication, replication, sharding, denormalization is normal and useful for a project, and reuse and normalization are potentially unsuccessful places ready to put the whole system.
- And in general, what happens after the failure of one of the modules? You just have to simulate all such failures and properly handle them. Cascading the entire system is just one of the scenarios. There can be many troubles. It is in a monolith that all components are in one Jar-nick. In ISA, each microservice is separated by unreliable and compromised network communications. This is neither bad nor scary. You just need to keep this in mind when developing and provide scenarios for breaking these communications.
- If you really love Spring, Hibernate and OracleDB, then most likely you can never build a reliable and fast microservice system. Spring was invented during the formation of monolithic architecture. This is a huge monster, which even in the function of a simple Hello World will turn into a jar file the size of hundreds of megabytes. Just the start of such an application is delayed for minutes. And these gigabytes of pointless digital junk will gobble up your 24x7x365 server resources. Oracle, Postgres, MySQL - this is all terrible junk, which was built on the assumption that the database should spin on one server. Yes, time goes on and all these databases have acquired scaling tools, but so far they are completely clumsy and unreliable in this matter. There are entire classes of databases (NoSQL, NewSQL) that were created in the Big Data, High Availability, distributed storage, etc. paradigm. And these oracle with postgres with them simply are not able to compete in reliability and speed. All these “cool” springs, oracles and hibernates are the legacy of the outgoing monolith era.The era of microservices and cloud functions is a quick start of applications when a load occurs, the absence of states for high scalability and ruthless unloading of all applications and resources that are currently not needed and only uselessly consume RAM.
In conclusion, I want to note once again that in no case am I against splitting monolithic systems into microservices. On the contrary, I believe that this approach can lead to significantly greater resilience of your IT infrastructure. The thoughtless introduction of microservice architecture elements without understanding the general theory of building such systems, completely ignoring industry practices and common sense in general - this is what sends your pilots to the dustbin of history. Only the mistakes of your organization’s people bring tears and losses to your investors.
On the contrary, a smart approach can really speed up your services and make them extremely resilient. Therefore, I really want to wish wisdom to everyone involved in the development of IT in companies..