More often than not, I start my projects thinking that they would be small and don't need a comprehensive framework like Django. Then I figure I need an authentication mechanism, SQL abstraction, an admin site, a pipeline to handle, verify the requests and so on. Then I switch back to Django mostly because I don't want to rediscover the world again. Surely given enough time I could rediscover, but why would one bother?
As per staying lean as stated in the article. I don't think that's staying lean at all. I must mention that I have no knowledge of the framework or codebase of the company in question, although rewriting a codebase is almost never a good idea. Well, it will require a lot of rewriting which is redundant, you would be ditching a lot of testing and bug fixing you have done ever since, and you will probably create a space a lot of new bugs.
I believe a better approach would be componentize your structure in a way that's still governed by django, every component split into apps. And if there are outlier apps that do not need to be within Django at all could be used by redirecting the requests to those services on the load balancer / reverse proxy level.
We're lucky enough that we decided early enough for it not to be too big an issue.
Also we split up the re-write over stages and haven't done it all in one go. I'm gonna write more about it in another blog post but essentially we still have our monolithic Django system.
It currently powers the majority of the services except we have changed the architecture so we can split things out overtime. This allowed us to achieve what we wanted to in 4 weeks yet lay the groundwork for the future.
This makes no sense from a business perspective. Letting developers work using any languages and technologies sounds like a good idea to individual developers maybe but how do you intend to maintain it? If you stick to a few core languages and technologies and hire / train for those tech then you reduce your risk if one or more developers leave and reduce the total complexity of your stack. You should have a good reason for introducing a new language or technology to your stack and think carefully before diving in. Just create separate Django apps and only switch to a different tech if Django is particularly ill suited for the task.
My thoughts as well. It seems like the problem here wasn't "monolithic Django codebase". The author mentions that he had a lot of trouble getting new developers up to speed because of package requirements.
That should be the simplest problem ever to solve. Use a requirements.txt, use vagrant, and maybe even Salt or Chef if you require more specific tools.
Microservices are great in theory, but every app ends up requiring a lot more interconnectedness as you grow. I'm all for splitting services off, but it's usually something to consider once you're a few iterations in.
Technically, one way of solving this is to make sure that the acceptance criteria for any single micro service are clear. For example, you have to expose these REST endpoints; given data set A, respond in this way to these requests, etc.
To me, a micro services architecture implies that the development team is responsible for the entire life span of the service, from conception to replacement. This includes maintenance. Insert the devops buzz here as well.
When the given solution can no longer be maintained because of failing knowledge (or even interest?) by other devs, it should be replaced. This should be no harder than refactoring a large method, except on the scale of an entire service.
Non-technical benefits include expanded responsibility and shiny architecture can possibly attract better developers/engineers. I know this has worked for us.
I think it would be reasonable to somewhat limit the available choices, especially at the start of a project.
I perhaps should have explained that a bit more. Technology choices have to make sense before implemented but giving developers the flexibility opens a lot of doors for creativity and exploration.
I strongly believe that learning new languages help people become better coders overall as it increases their knowledge of different problems and solutions which can be applied across most stacks.
Also no one developer should ever be solely responsible for a particular service. But that should also be the case for apps within a Django project.
My gut feeling is that microservices will lead to language unification within a team in the long run. Yes, initially, as you drop artificially imposed language restrictions, you will get a zoo of systems, but after a while (as the team experience grows) it will converge to a few or even a single language with DSL capabilities, it's just natural. In my experience, the driver for language heterogeneity is not languages themselves, it's craving for DSLs.
There is absolutely no need to use different languages and technologies within a microservices architecture if it is not necessary for your business. But to simply dismiss the business perspective from your perspective seems a little naive.
Some reasons why decoupling your architecture can make sense from a business perspective:
- You have a complicated product, and wish to break it up into more digestible chunks, with allocated teams working on each abstract business concept (Amazon and Spotify do this).
- You want to avoid having a single point of failure- if one service fails, the rest of the system might well keep on working. (Ever noticed how sometimes a panel on Amazon doesn't load?). If you're running a continuous development process, stuff breaks, a lot. It's good if that's locked down.
- You have a business requirement to build something with a different technology (from experience- rendering javascript in a node service was much, much easier and more maintainable than with an existing Django app- we tried both.)
- You want to experiment with new technologies without changing your whole stack.
I'm pretty sure that the author wasn't talking about moving to microservices so they can blindly start building every aspect of the application in a different stack- but more that if they need to use a new technology for their business requirements, or want to give something a trial without committing, they can. And with the uncertainty and rate things change in modern software, and in a startup, that flexibility is invaluable.
I can see a case for microservices as a way to break down complexity. But introducing languages willy-nilly could have maintenance nightmares along the road.
That just means you haven't properly aligned your developers' interests with your business interest.
Limiting developers in which languages and technologies they can use is not going to solve that problem, it merely addresses the symptom. Developers make hundreds of little less obvious and visible decisions that have a long term impact on your business. Picking the tools should always remain a developer decision.
If you a) hire the right people and b) make sure your goals are aligned, your developers limit themselves to a few core technologies. If not, you're screwed anyway.
(And yes, that does mean mutually accepting and respecting the fact that your developers' interests in exploring new things and your business interests in may not be aligned indefinitely.)
I don't get it: why does moving to microservices mean ditching Django? Couldn't you take your Django monolith and break it into separate (still-Django) microservices? I get that Django is seen as "too heavyweight" to do such a thing, though there are literally books (http://shop.oreilly.com/product/0636920032502.do) written about how to slim down Django.
I get that monolith to microservice is all the rage these days, I just don't understand (and the article doesn't really say) why you have to "ditch Django" to do that.
I thought Django and "microservices" are orthogonal. Can't Django be used to create any service, even a micro service? I certainly used Django in the past just as a REST server, without any "web site" to speak of. It just reduces some boiler plate.
I think the biggest reason people are switching to microservices is that it enforces protocols and documentation between components that would otherwise never be created.
Microservices create a boundary and that forces you to create a request/response protocol to enforce that boundary. You also have to create documentation for it to be useful.
There is nothing stopping you from creating your app as a bunch of small services without the added complexity of microservices/SOA. Each slice of your system could be an isolated library with a stronger enforced protocol and documentation and you'd get largely the same benefits without having to deal with the deployment headache.
The reason people don't do that is if they build isolated components in the same monolitic system, nobody takes the time to create a proper protocol and documentation.
The only way people seem to be willing to give separate services the appropriate design respect is to physically separate them and solve them as an isolated problem. The physical separation forces better behavior for microservices to even be useful.
It doesn't help that most languages have really weak capabilities to enforce strong protocols/boundaries beyond a simple type checker via complier. A language like JS, Python, PHP, Ruby, etc. don't even have a compiler so the ability/desire to create strong protocols in most code is almost nonexistent to most developers.
> A microservices architecture is (fairly) new and it doesn't tie you to a particular language. If you want to build a service in Lisp, Go, Elixir, etc then you can (assuming it makes sense). Good devs love to learn, experiment and hate being bored. A microservices approach encourages all of that
The developer in me kind of likes that - fun new toys! But from a business point of view, I'm not sure it sounds like such a good idea: lots of weird, unmaintainable stuff.
This is not as terrifying as you might think. You can use this well for prototyping. You can make a simple endpoint fairly quickly in Go, Erlang, Clojure, Ruby. It could do some small task and pop out an answer. You can deploy this behind the wall of an named service.
If the service proves useful, you can migrate it to a more appropriate language. By appropriate, I mean a language that fits your companies culture. Java Shoppe? Java. C# Shoppe? F#.
You can also get a better match for the domain. For example Python has some great math libraries. They are easy to pickup, pleasant to use and reasonably performant. Say you need such features. Boom! Python service doing what it does best while the rest of your shop can use the tools that work well for the domain.
Micro-services force separation of concerns, theoretically making a system more maintainable. Micro-services do little for complexity though - they simply move complexity from a monolithic app to somewhere else and spread it out.
Yeah, from my experiences with my current company, standard language/framework == good, one-off apps that now need to be maintained in production == bad.
Judging by the following statements: "we don't worry much about code quality", and "lots of moving parts isn't without its complexity", I'm just going to very politely wish him luck in his endeavors, and maybe make some popcorn.
One of my previous employers had let each new incoming VP of engineering use whatever he wanted to build out new parts of the app. With the high turnover rate they had, the app turned into the worst Frankenapp piece of crap imaginable. Significant portions of the app were built in php, java, python, javascript, perl, ruby, ruby on rails, mysql, sybase, and a few more I'm forgetting.
The result was that changing the title on the homepage took two weeks of manual regression testing and it took a new developer a minimum of two weeks to get a development instance up and running. I felt so sorry for the Ops guys, I can't even imagine how terrible it was for them. There was no one, ops or dev, that knew even 25% of how the whole system worked. It always felt like the leaning tower of pisa made toothpicks and popsicle sticks held together with bubble gum and boogers.
The cost to run that Rube Goldberg machine monstrosity ruined what could have been an amazingly profitable business and great place to work. I don't recommend it.
> it was becoming incrementally larger and more complicated for new team members. Because of this, the monolithic Django system was already proving a headache to get quickly set up, running and understood by our new hires.
So, now you have thirty smaller services for each new hire to install and link together? I can all but guarantee that this particular problem is better solved with Vagrant and some orchestration scripts. Takes your install from a list of packages to a single "vagrant up" command.
I've been a big fan of microservices for years. I believe that a good microservice has a lot in common with traditional Unix commands. And I find it funny that no one here has gone off on a tangent to talk about how horrible those are.
Unix's tools and good microservices have the following things in common:
1. They are small
2. They are loosely coupled
3. Shared-nothing state
4. They enforce standards for consuming and emitting data
5. They enforce a single standard for piping data from one process to another.
That said, I strongly believe that this won't work for every class of problem. And while I may be putting words into Tom Watson's mouth here, I believe he would say the same. But hey, guess what? This WOULD work for quite a few problems!
Some of the sentiment in the comments here feel defensive and knee-jerk. And I don't understand why. Tom Watson isn't telling you that you should use microservices. He's only explaining why he's using it at Hubble. I'm pretty sure how it turns out is going to have no impact on anyone else here, except (obviously) his customers.
But I hope it succeeds. I think there's a lot we can learn by applying the lessons Unix taught us to web-scale utilities.
1. They are small 2. They are loosely coupled 3. Shared-nothing state 4. They enforce standards for consuming and emitting data 5. They enforce a single standard for piping data from one process to another.
This can all be done via libraries with clear APIs, with none of the deployment headache of a microservice.
Doesn't Django already encourage this type of design? Each Django project is composed of several installed 'apps'. Each app should be portable enough that it could be the only app in any project.
I have a large app in Django at my work. The thing is it doesn't make sense to split this into smaller apps (except for the shopping cart equivalent). It tracks the workflow of our organization, so all queries need to be related from the very start to the very end of the process. (Ok, not all queries, but a lot of them).
True, but all these installed apps still increase the size of your one project which becomes larger and larger. You can't install it in parts, you can't make any architectural or language changes as it requires a rewrite of the whole project and each new release/ deploy effects the entire codebase.
Microservices doesn't fix everything and you're right about Django encouraging this type of design. Each approach has it's pros and cons.
> We don't worry too much about code quality as I believe that that is fixed at the recruitment level. Good people write good code and, more importantly, ship good products.
Good people make mistakes too. Having good tools to ensure code quality doesn't make it impossible to ship bad products, but it's definitely important.
Not worrying too much doesn't mean not worrying at all! We still have tools in place to ensure silly mistakes aren't made or are at least rectified quickly. Just we don't expect 100% test coverage on every single feature or fix we ship.
This approach will probably change as we get larger but being a small development team means that it is fine for now.
In my experience with microservices, the vast majority of bugs were in services not talking to each other properly, or in some variation of that interface. There are some security complications that means services aren't very free to talk to each other (which introduces complexity), but, overall, I think I prefer the monolithic model. Our Django app isn't too hard to set up, requirements.txt contains everything and the rest is in an Ansible playbook.
It depends on what you're building. For us, we haven't noticed any issues with latency just yet and there are plenty of steps you can take to optimise this down the line.
As for monitoring there are plenty of tools to do that. You have bigger problems if you expect your services to go down often.
It's amazing how easily people swap "I can't see how to use this in the contexts I'm familiar with" to "it doesn't make any sense at all".
Being myself - just a few years ago - a long time strong proponent of tightly coupled monoliths, I can't believe how blind and foolish I was in my fundamentalist rejection of loosely coupled architectures.
Monoliths do just fine up to a certain level of structural complexity. Above that, asynchronous service-like architectures are the only viable way to go.
Microservices is an attempt to see if the services patterns work below that waterline, down to the function level. That's why I find microservices at least interesting.
At any rate, it's a cost/benefit balance game, not an ideology.
We had similar issue's.
Our Designers couldn't design with that big Django app, so we split it up.
Everything talks ReST to a cached AngularJS Client.
We still have Authentication in Django and two of our Microservices are running with Django aswell. Some things were reimplemented in Golang, as we tried some new things out.
What people here are talking is mostly dumb, since Microservices are better in a lot of ways especially when it comes to multiple Developers.
Currently my company struggled a lot with Code that got changed by two guys at the same time, which was hard to fix.
We had less unit tests than now, since they were too complicated at a certain time.
Standardization is, generally, good for maintenance, learning curve and security, not always for retention (fun) though. Too much standardization is like the flip-side of too many one-offs: an extreme that can lead to unintended consequences. It's much easier to accrue technical debt and organizational bifurcation by having too many languages, with a big, painful shake out (technical debt repayment) into just a 2-3 (say Ruby|Python|Node & Go|Scala|Haskell|Erlang|C)... (I've seen this happen at a number of shops go through growing pains at scale).
Overall though, services should expose either a well/self-documented RESTful, zmq or similar API such that they can be rewritten in (your favorite whatever here). Being able to discover the API meta information automagically can be helpful for autogenerating clients.
The other aspect is that services have a risk budget: if the service is conventional and not as crucial, increase the experimentation. If the business/process model is risky/uncertain, use common, stable technologies.
(Gist: Pick from the toolbox, carefully... apply common-sense... Don't get stuck on perfect now or perfect waay later... Anticipate a little now to avoid pain of changing later.)
PS: Enterprise Rails is a great read. It's safe to ignore the Rails bits, because the architecture bits are universal and excellent.
Every time I see microservices, I think of Martin Fowler's First Law of Distributed Object Design: "Don't distribute your objects." http://martinfowler.com/bliki/FirstLaw.html
Splitting your app into multiple apps that send each other HTTP requests or MQ messages seems likely to just make your app slower.
Can I be the first to say that this is a terrible idea?
>We have a multitude of parts that each could easily be self contained as their own service. For example: Messaging, Search, Authentication, etc. These all have different requirements and focuses and therefore lend themselves better to different languages.
Yeah, not really. Python is not inherently less suitable for authentication than, say, Java. Both languages are designed for general purpose programming. The only real drawback to python is that it is relatively slow and this isn't even likely to be an issue at all if you are using django (you'll most likely bottleneck on I/O).
>Also, looking at our roadmap, we have lots of sprints and projects coming up that could be implemented in their own silos. By splitting these into services this allows us to quickly test and iterate without worrying about the rest of our product.
And splitting them into separate libraries is not sufficient because?
>Whereas in a monolithic system these services are bundled together and can create a spaghetti mess of workarounds and packages.
Newsflash: you're MORE likely to make a spaghetti mess of workarounds if you use a microservices architecture because of the impedance mismatch between API layers. You'll have to create a representation of, say, a user object on both sides of the microservice divide and write code to serialize it and deserialize it. Hello massive code repetition and obscure bugs.
>Having been a dev team of 1 for a long time I'd never experienced the headache of installing all the dependencies required by our Django system. It never really occurred to me that as I installed more packages, it was becoming incrementally larger and more complicated for new team members.
If you think your headache is bad with django (which has consistent opinions - e.g. on how templates should work - that makes it possible for modules to work together), get ready for a nightmare when you start using a hodgepodge of different systems written in different languages.
>We don't worry too much about code quality
This has officially moved this blog post from bad idea to outright parody.
>However, most of the cons are part and parcel with programming.
Here's a small laundry list of the exciting issues you'll now get to deal with:
* Writing serializers and deserializers for your objects so they can be passed from one service to another. And debugging them!
* Babysitting all the new services you will need. You'll need to keep them up and running, monitor them, have a backup plan for all of them going down and lots of lovely extra error conditions.
* The headache of trying to figure out how to scale your network of microservices - probably meaning lots of time configuring load balancers.
make sure to document well. When you're scaling up a team and moving from a coupled code base to independent codebases a good amount of assumption happens, especially around how mature each codebase is.
> For example: Messaging, Search, Authentication, etc. These all have different requirements and focuses and therefore lend themselves better to different languages.
What?
Every single one of those is a solved problem that has bindings to most major languages. Why in the world would you switch languages for an Auth mechanism (for example)?
> Having been a dev team of 1 for a long time I'd never experienced the headache of installing all the dependencies required by our Django system. It never really occurred to me that as I installed more packages, it was becoming incrementally larger and more complicated for new team members. Because of this, the monolithic Django system was already proving a headache to get quickly set up, running and understood by our new hires. Splitting it up into more digestible chunks (services) therefore should allow us to scale the team a lot quicker this year and beyond.
Yet, the dynamic language community constantly laughs at Java developers and their high overhead dependency management solutions, which just happen to have these kinds of problems solved since more than a decade.
I don't believe he's talking about code dependencies but rather things like LDAP, mySQL, MongoDB, etc which Java dependency management tools don't solve either. Managing code dependencies is solved in Python (as it is in most dynamic languages) using pip and virtual environments.
umurkontaci|11 years ago
As per staying lean as stated in the article. I don't think that's staying lean at all. I must mention that I have no knowledge of the framework or codebase of the company in question, although rewriting a codebase is almost never a good idea. Well, it will require a lot of rewriting which is redundant, you would be ditching a lot of testing and bug fixing you have done ever since, and you will probably create a space a lot of new bugs.
I believe a better approach would be componentize your structure in a way that's still governed by django, every component split into apps. And if there are outlier apps that do not need to be within Django at all could be used by redirecting the requests to those services on the load balancer / reverse proxy level.
tmwatson100|11 years ago
Also we split up the re-write over stages and haven't done it all in one go. I'm gonna write more about it in another blog post but essentially we still have our monolithic Django system.
It currently powers the majority of the services except we have changed the architecture so we can split things out overtime. This allowed us to achieve what we wanted to in 4 weeks yet lay the groundwork for the future.
bsdpython|11 years ago
adambratt|11 years ago
That should be the simplest problem ever to solve. Use a requirements.txt, use vagrant, and maybe even Salt or Chef if you require more specific tools.
Microservices are great in theory, but every app ends up requiring a lot more interconnectedness as you grow. I'm all for splitting services off, but it's usually something to consider once you're a few iterations in.
remmelt|11 years ago
Technically, one way of solving this is to make sure that the acceptance criteria for any single micro service are clear. For example, you have to expose these REST endpoints; given data set A, respond in this way to these requests, etc. To me, a micro services architecture implies that the development team is responsible for the entire life span of the service, from conception to replacement. This includes maintenance. Insert the devops buzz here as well. When the given solution can no longer be maintained because of failing knowledge (or even interest?) by other devs, it should be replaced. This should be no harder than refactoring a large method, except on the scale of an entire service.
Non-technical benefits include expanded responsibility and shiny architecture can possibly attract better developers/engineers. I know this has worked for us.
I think it would be reasonable to somewhat limit the available choices, especially at the start of a project.
tmwatson100|11 years ago
I strongly believe that learning new languages help people become better coders overall as it increases their knowledge of different problems and solutions which can be applied across most stacks.
Also no one developer should ever be solely responsible for a particular service. But that should also be the case for apps within a Django project.
mike_ivanov|11 years ago
chrisvxd|11 years ago
Some reasons why decoupling your architecture can make sense from a business perspective:
- You have a complicated product, and wish to break it up into more digestible chunks, with allocated teams working on each abstract business concept (Amazon and Spotify do this).
- You want to avoid having a single point of failure- if one service fails, the rest of the system might well keep on working. (Ever noticed how sometimes a panel on Amazon doesn't load?). If you're running a continuous development process, stuff breaks, a lot. It's good if that's locked down.
- You have a business requirement to build something with a different technology (from experience- rendering javascript in a node service was much, much easier and more maintainable than with an existing Django app- we tried both.)
- You want to experiment with new technologies without changing your whole stack.
I'm pretty sure that the author wasn't talking about moving to microservices so they can blindly start building every aspect of the application in a different stack- but more that if they need to use a new technology for their business requirements, or want to give something a trial without committing, they can. And with the uncertainty and rate things change in modern software, and in a startup, that flexibility is invaluable.
karuth|11 years ago
dominotw|11 years ago
santacluster|11 years ago
Limiting developers in which languages and technologies they can use is not going to solve that problem, it merely addresses the symptom. Developers make hundreds of little less obvious and visible decisions that have a long term impact on your business. Picking the tools should always remain a developer decision.
If you a) hire the right people and b) make sure your goals are aligned, your developers limit themselves to a few core technologies. If not, you're screwed anyway.
(And yes, that does mean mutually accepting and respecting the fact that your developers' interests in exploring new things and your business interests in may not be aligned indefinitely.)
drbryanadams|11 years ago
I get that monolith to microservice is all the rage these days, I just don't understand (and the article doesn't really say) why you have to "ditch Django" to do that.
eliben|11 years ago
I thought Django and "microservices" are orthogonal. Can't Django be used to create any service, even a micro service? I certainly used Django in the past just as a REST server, without any "web site" to speak of. It just reduces some boiler plate.
artursapek|11 years ago
programminggeek|11 years ago
Microservices create a boundary and that forces you to create a request/response protocol to enforce that boundary. You also have to create documentation for it to be useful.
There is nothing stopping you from creating your app as a bunch of small services without the added complexity of microservices/SOA. Each slice of your system could be an isolated library with a stronger enforced protocol and documentation and you'd get largely the same benefits without having to deal with the deployment headache.
The reason people don't do that is if they build isolated components in the same monolitic system, nobody takes the time to create a proper protocol and documentation.
The only way people seem to be willing to give separate services the appropriate design respect is to physically separate them and solve them as an isolated problem. The physical separation forces better behavior for microservices to even be useful.
It doesn't help that most languages have really weak capabilities to enforce strong protocols/boundaries beyond a simple type checker via complier. A language like JS, Python, PHP, Ruby, etc. don't even have a compiler so the ability/desire to create strong protocols in most code is almost nonexistent to most developers.
davidw|11 years ago
The developer in me kind of likes that - fun new toys! But from a business point of view, I'm not sure it sounds like such a good idea: lots of weird, unmaintainable stuff.
unknown|11 years ago
[deleted]
raverbashing|11 years ago
"Oh we'll just RabbitMQ everything together", sure, great, until you discovered certain services needed a smaller latency, more bandwidth, etc
And RabbitMQ libraries are a pain and are not smart
virmundi|11 years ago
If the service proves useful, you can migrate it to a more appropriate language. By appropriate, I mean a language that fits your companies culture. Java Shoppe? Java. C# Shoppe? F#.
You can also get a better match for the domain. For example Python has some great math libraries. They are easy to pickup, pleasant to use and reasonably performant. Say you need such features. Boom! Python service doing what it does best while the rest of your shop can use the tools that work well for the domain.
FourthProtocol|11 years ago
endemic|11 years ago
tmaly|11 years ago
mml|11 years ago
Mc_Big_G|11 years ago
The result was that changing the title on the homepage took two weeks of manual regression testing and it took a new developer a minimum of two weeks to get a development instance up and running. I felt so sorry for the Ops guys, I can't even imagine how terrible it was for them. There was no one, ops or dev, that knew even 25% of how the whole system worked. It always felt like the leaning tower of pisa made toothpicks and popsicle sticks held together with bubble gum and boogers.
The cost to run that Rube Goldberg machine monstrosity ruined what could have been an amazingly profitable business and great place to work. I don't recommend it.
falcolas|11 years ago
So, now you have thirty smaller services for each new hire to install and link together? I can all but guarantee that this particular problem is better solved with Vagrant and some orchestration scripts. Takes your install from a list of packages to a single "vagrant up" command.
roberthahn|11 years ago
Unix's tools and good microservices have the following things in common:
1. They are small 2. They are loosely coupled 3. Shared-nothing state 4. They enforce standards for consuming and emitting data 5. They enforce a single standard for piping data from one process to another.
That said, I strongly believe that this won't work for every class of problem. And while I may be putting words into Tom Watson's mouth here, I believe he would say the same. But hey, guess what? This WOULD work for quite a few problems!
Some of the sentiment in the comments here feel defensive and knee-jerk. And I don't understand why. Tom Watson isn't telling you that you should use microservices. He's only explaining why he's using it at Hubble. I'm pretty sure how it turns out is going to have no impact on anyone else here, except (obviously) his customers.
But I hope it succeeds. I think there's a lot we can learn by applying the lessons Unix taught us to web-scale utilities.
lumpypua|11 years ago
This can all be done via libraries with clear APIs, with none of the deployment headache of a microservice.
speg|11 years ago
collyw|11 years ago
tmwatson100|11 years ago
Microservices doesn't fix everything and you're right about Django encouraging this type of design. Each approach has it's pros and cons.
jexus2|11 years ago
Good people make mistakes too. Having good tools to ensure code quality doesn't make it impossible to ship bad products, but it's definitely important.
tmwatson100|11 years ago
This approach will probably change as we get larger but being a small development team means that it is fine for now.
statictype|11 years ago
I assume that REST or Thrift (or something similar) is the way most microservices talk to each other. What if the latency isn't acceptable?
I've thought about switching our architecture to microservices but these two potential issues are making me hesitate.
How important are these issues in practice?
StavrosK|11 years ago
tmwatson100|11 years ago
As for monitoring there are plenty of tools to do that. You have bigger problems if you expect your services to go down often.
mike_ivanov|11 years ago
Being myself - just a few years ago - a long time strong proponent of tightly coupled monoliths, I can't believe how blind and foolish I was in my fundamentalist rejection of loosely coupled architectures.
Monoliths do just fine up to a certain level of structural complexity. Above that, asynchronous service-like architectures are the only viable way to go.
Microservices is an attempt to see if the services patterns work below that waterline, down to the function level. That's why I find microservices at least interesting.
At any rate, it's a cost/benefit balance game, not an ideology.
Edit: clarity.
merb|11 years ago
What people here are talking is mostly dumb, since Microservices are better in a lot of ways especially when it comes to multiple Developers. Currently my company struggled a lot with Code that got changed by two guys at the same time, which was hard to fix.
We had less unit tests than now, since they were too complicated at a certain time.
fubarred|11 years ago
Overall though, services should expose either a well/self-documented RESTful, zmq or similar API such that they can be rewritten in (your favorite whatever here). Being able to discover the API meta information automagically can be helpful for autogenerating clients.
The other aspect is that services have a risk budget: if the service is conventional and not as crucial, increase the experimentation. If the business/process model is risky/uncertain, use common, stable technologies.
(Gist: Pick from the toolbox, carefully... apply common-sense... Don't get stuck on perfect now or perfect waay later... Anticipate a little now to avoid pain of changing later.)
PS: Enterprise Rails is a great read. It's safe to ignore the Rails bits, because the architecture bits are universal and excellent.
Update: RESTful API metadata formats http://apiux.com/2013/04/09/rest-metadata-formats/ (and also https://en.wikipedia.org/wiki/HATEOAS )
kyllo|11 years ago
Splitting your app into multiple apps that send each other HTTP requests or MQ messages seems likely to just make your app slower.
angersock|11 years ago
crdoconnor|11 years ago
>We have a multitude of parts that each could easily be self contained as their own service. For example: Messaging, Search, Authentication, etc. These all have different requirements and focuses and therefore lend themselves better to different languages.
Yeah, not really. Python is not inherently less suitable for authentication than, say, Java. Both languages are designed for general purpose programming. The only real drawback to python is that it is relatively slow and this isn't even likely to be an issue at all if you are using django (you'll most likely bottleneck on I/O).
>Also, looking at our roadmap, we have lots of sprints and projects coming up that could be implemented in their own silos. By splitting these into services this allows us to quickly test and iterate without worrying about the rest of our product.
And splitting them into separate libraries is not sufficient because?
>Whereas in a monolithic system these services are bundled together and can create a spaghetti mess of workarounds and packages.
Newsflash: you're MORE likely to make a spaghetti mess of workarounds if you use a microservices architecture because of the impedance mismatch between API layers. You'll have to create a representation of, say, a user object on both sides of the microservice divide and write code to serialize it and deserialize it. Hello massive code repetition and obscure bugs.
>Having been a dev team of 1 for a long time I'd never experienced the headache of installing all the dependencies required by our Django system. It never really occurred to me that as I installed more packages, it was becoming incrementally larger and more complicated for new team members.
If you think your headache is bad with django (which has consistent opinions - e.g. on how templates should work - that makes it possible for modules to work together), get ready for a nightmare when you start using a hodgepodge of different systems written in different languages.
>We don't worry too much about code quality
This has officially moved this blog post from bad idea to outright parody.
>However, most of the cons are part and parcel with programming.
Here's a small laundry list of the exciting issues you'll now get to deal with:
* Writing serializers and deserializers for your objects so they can be passed from one service to another. And debugging them!
* Babysitting all the new services you will need. You'll need to keep them up and running, monitor them, have a backup plan for all of them going down and lots of lovely extra error conditions.
* The headache of trying to figure out how to scale your network of microservices - probably meaning lots of time configuring load balancers.
And much, much more.
chhantyal|11 years ago
If OP is more worried about maintenance of project, Django gives foundation to build upon with sane conventions.
Anyone who reads few pages of Django Docs should be able to pick up quickly.
PS. I happen to currently maintain project written by other people with no docs.
knicholes|11 years ago
jdoconnor|11 years ago
omegote|11 years ago
mreiland|11 years ago
What?
Every single one of those is a solved problem that has bindings to most major languages. Why in the world would you switch languages for an Auth mechanism (for example)?
_pmf_|11 years ago
Yet, the dynamic language community constantly laughs at Java developers and their high overhead dependency management solutions, which just happen to have these kinds of problems solved since more than a decade.
Pathetic.
iends|11 years ago