Extracting your first microservice
I argued before that a microservices architecture is seldom the way to go when starting out, but as a project matures it becomes more beneficial to migrate to this type of system architecture. There isn’t a hard rule when to do this, but it’s often triggered by one of the following needs:
Being able to individually scale the capacity of a single component.
The desire to isolate components so they can be reused or developed independently of the main application.
Extracting the first service is probably the hardest as it requires you to make decisions about a lot of aspects of your micro(service) architecture. How will your services communicate? Do multiple services share a single database? Do you need some kind of service discovery and configuration management? All these choices influence the amount of decoupling and the degree of service orientation.
In this case I will briefly discuss the considerations that went into extracting a credit check service. While the new microservice is far from perfect, it only took us 2 days of work to get it up and running. This gives us plenty of time to perfect it down the line.
The original credit check service was a simple Python module that retrieves company credit reports from upstream data providers using different APIs. Reports are retrieved using a combination of country and company registration number. The returned report is either a PDF, or a limited subset of properties (such as credit limit and rating) from the report.
The refactor
Fortunately the credit check service was already quite isolated from the rest of the application. I only spotted a single leak in our abstractions: instead of identifying a report by country and company registration number we also included an artifical identifier for the related entity. This could easily be fixed by changing the arguments of our retrieval methods:
get_credit_report(client_id): get_credit_report(country, company_number):
By doing this, the new credit check service can now function without knowing a thing about a client. We made the concious decision to keep the credit report data in the same application — at least for now. This eased our transition as we could read from our database as we used to, and only rebuild the way we write reports to the database. Database reads and writes are isolated, but the applications still share a single database.
To overcome availability issues at our upstream data providers we decided to introduce a message broker. We use RabbitMQ to queue the external retrieval of the reports, but the message broker also serves as a communication channel between our main application and the credit check service. When a new credit report is needed a “credit-report.created” message is published and retrieved in the background. The message is in turn consumed by the credit check service.
A message is structured as a JSON-API document, a specification we also use for our public RESTful API. By using an existing standard we can reuse some of our existing code and save ourselves from inventing something new. The contents of the message includes all the information required to retrieve a credit report.
Wrap-up
In this first iteration of our microservice we have a limited amount of decoupling. You could even argue that we’ve created a simple worker application, rather than a full-fledged service. In future iterations we plan on removing the read layer from the main application and move to an RPC style architecture (probably using gRPC or Thrift). The main application will then call the service, rather than access the database directly.
We used our existing Python code as the basis of the new service. Ultimately we might end up rewriting this code in Node.js or asynchronous Python, but throughput isn’t a concern right now. Getting up and running is.
The takeaway is that adopting a microservice architecture doesn’t have to be a big-bang event. If the team is disciplined enough about using the specified interface, future improvements are easy. Until then you are able to minimize operations overhead, while still enjoying the advantages of microservices.