Interesting read on SCM upending software and hardware architecture

https://queue.acm.org/detail.cfm?id=2874238 discusses how modern Storage 
Class Memory (SCM), such as PCIe SSD and NVDIMMs are completely upending 
every assumption made about storage. To put this in perspective, you can 
now see storage latency that is practically on-par with things like lock 
acquisition[1].

Presumably the bulk of this difference should be handled by the OS, but 
there's probably things we should be considering too:

Tiered storage will become common. That's going to make avoiding things 
like bulk scans even more important. There's a tie-in to partitioning 
and indexes too.

The days of a SAN may be over. With memory, network and storage latency 
approaching parity it's not practical to concentrate any of these 
resources; that creates a bottleneck. This means people will be even 
more resistant to the idea of a single database server.

The cost of temporary data becomes much lower. At some point it probably 
makes sense to just mmap what's needed and move on.

Fortunately, I think our traditional reliance on OS caching has helped 
us... to some degree we've always treated storage as fast because a lot 
of requests would be coming from RAM anyway.

[1] Ok, 25x isn't exactly on-par, but considering this used to be 
25,000x... "At 100K IOPS for a uniform random workload, a CPU has 
approximately 10 microseconds to process an I/O request. Because today's 
SCMs are often considerably faster at processing sequential or read-only 
workloads, this can drop to closer to 2.5 microseconds on commodity 
hardware. Even worse, since these requests usually originate from a 
remote source, network devices have to be serviced at the same rate, 
further reducing the available per-request processing time. To put these 
numbers in context, acquiring a single uncontested lock on today's 
systems takes approximately 20ns, while a non-blocking cache 
invalidation can cost up to 100ns, only 25x less than an I/O operation."
-- 
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com