- Split traffic between multiple servers. Probably with a load balancer in front, each server having the same code and same data and sharing the work round-robin or according to load.
- Split between functionality, with different servers for different classes of request. So, for example, logins go to one pool, search to another and so on. The idea being that with specialization servers can be more efficient, if only by what code or data they need to cache and the number of relationships with other servers they must maintain. Also, when managing upgrades on portions of the code base the effect can more easily be split away from impacting other services.
- Split by customer. This comes out as a class of specialisation of data; but in this case to do with user data rather than function or functional data.
So, can n+1 be applied to this?
My offering: "split by lumpiness".
Some traffic comes in bursts, can you identify and separate such traffic and route it to a dedicated server group? With cloud hosting this would map neatly to changing the size and/or specification of the servers. A similar effect might be obtained with a load balanced split. However, bursts of traffic will often have some commonality - a group of users, or a topic, or access to a suddenly popular page due to external references. See Barabasí's book Bursts for more on this. If the burst can be identified, e.g. by referrer page, then caching may help service those requests. This may be an emergent split, not a planned one - simply having planned the ability to make such a split.
Another lumpy split is down to the work involved in processing a request. Is there an advantage in provisioning a mix of traffic to one server or in having tasks of similar complexity competing with each other for resources? If the ratio between different classes of traffic varies then provisioning for a consistent response time gets harder. If more complex and non-time critical tasks can be handed off then the workload of a customer facing server isn't going to be affected by other tasks with a variable impact. Note that this is slightly more than simply offloading asynchronous tasks: if every request requires the same effort and causes an asynchronous task of consistent effort then the customer experience won't vary - so adding a server to handle the load is the first sort of split above and splitting the tasks is the second sort of split above (by function). If the task complexity varies, so that some are either naturally slow or might affect the usually quick, then a split by variation allows better resource planning and consistent response times.
No comments:
Post a Comment