There are two papers going round that everyone has an opinion about and rather fewer people have read. Google published one describing the storage system underneath its own products, and Amazon published one last autumn describing the store that keeps its shopping baskets alive.
Both describe systems that deliberately give up things a relational database guarantees. No joins. No transactions across multiple records. And in Amazon’s case, no promise that a read immediately after a write returns the new value rather than the old one.
Read out of context, that last one sounds like a defect. It is a considered trade and the reasoning is worth following.
If your data lives on hundreds of machines spread across several buildings, some of those machines are broken right now. Not might break. Are broken, at this moment, as a permanent condition of operating at that size. A system that insists every copy of a value agrees before answering has to wait for machines that may never reply, which means a failure anywhere becomes a stall everywhere. Amazon decided a shopping basket that occasionally shows a slightly stale item is better than a shopping basket that stops working when a switch fails in another city.
For a business where the checkout being down costs a fortune per minute, that is plainly the right call.
The excitement I am watching build around this is where I get uneasy, because the trade only pays if you have the problem.
Most of the people I have heard enthusing about giving up joins have a dataset that fits comfortably on one machine with room to spare. For them the relational database is not a bottleneck, it is thirty years of accumulated work on query planning, indexing, and crash recovery, available for nothing, understood by every person they might hire. Trading all of that away to solve a scaling problem you will not have for five years, if ever, is not architecture. It is anticipating a success you have not had yet.
The other thing being underplayed is where the complexity goes. It does not vanish when you drop transactions. It moves into your application, where you now have to handle the case of two versions of the same record disagreeing, and you have to handle it correctly in every place you touch that data, and there is no query planner to help you.
Amazon has engineers who can do that well. So does Google. That is part of what the papers are describing, and it is the part that does not transfer.
My guess is that a handful of genuinely large operations adopt these ideas and benefit enormously, a great many small ones adopt them because the papers are exciting, and roughly two years from now there is a quiet wave of migrations back to Postgres that nobody writes conference talks about.