Need to Feed Hungry CDNs? Put Them on an All-Object Diet
With the ever-increasing demand for higher-quality video content, you may be struggling to maintain a repository of content that can keep up with hungry viewers. Storage systems that served a menu of compressed standard definition fare can no longer keep up with consumers for which high definition is common and 4K is the special of the day.
Obviously, you’re not responsible for this explosion of storage capacity demand, but if you’re in the business of delivering video content, you are surely feeling the pressure of constant requests for more storage. When your conversations are moving from “how many terabytes” to “we need a petabyte,” you’re going to have to change the way you think about storage and start considering object storage. File systems and RAID arrays are perfectly appropriate for primary storage, where performance is king, collaborative creative application support is considered table stakes, and cost doesn’t matter. But when you allow your primary storage to be the place where content sits waiting to be delivered, that’s a poor use of resources and will ultimately hinder your work in progress by your best people. Enter object storage.
If you’re not already familiar with object storage, here’s an overview of the attributes that make it fundamentally different from RAID storage arrays and perfectly suited to act as a content repository for delivery workflows.
Object storage recreates the lost data from encoded copies of the rest of the object until the protection level is restored
There’s no file system
Every object in an object storage system has all the information it needs to be protected, found, accessed, and delivered. There’s no over-arching set of tables that link a name to a sector on a disk. Now you may be thinking, “So what? Why does that matter?” It matters a great deal when things get big. File systems are not infinite; ultimately there are only so many files they can address. That’s why most modern file systems have a logical or practical limit to how big they can become. Object storage has no limit. It will happily squirrel away content in the form of uniquely identifiable objects long after you run out of content to store.
RAID arrays get less reliable as they grow; object storage systems get more reliable as they grow. Don’t get me wrong, RAID is the best tool there is for the right job. But providing data protection for hundreds, if not thousands, of disk drives full of data is not one of them. Drives fail. It’s what they do. And when they fail as part of a RAID array, they become the data equivalent of moldy cheese in the back of the cold cuts drawer. You must get it out of there and clean up (or rebuild) the array before something else goes bad. Eventually, when an array gets big enough, it can even collapse under its own weight because you can’t rebuild the array faster than the drives fail.
Object storage doesn’t care about a failed component because it leverages concepts of slicing, distributing, encoding, and replicating to protect data without the need for a coherent set of devices. It's a bit like the telephone network. If a drive fails, the system patches around it. It recreates the lost data from encoded copies of the rest of the object until the protection level is restored. And rather than operate in a compromised state while recovering, like a RAID rebuild, object storage maintains full performance and access to content while getting incrementally healthier over time.
Leverages the Latest Storage Technology
For a storage system to be efficient over the long term, it needs to be able to adapt and incorporate new storage device technology as it becomes available. When a new higher-capacity, lower-cost-per-byte device comes on the market, you’d like to be able to utilize it without swapping out your entire system. But that’s exactly what RAID forces you to do. Every disk in a RAID array must be matched. With the march of disk drive innovation, you could be struggling to find matches for your array after only a few years of use. With object storage and its fully independent components, that 6 TB drive that was state of the art three years ago can be neatly replaced with the latest 10 TB unit, and all that capacity will be utilized.
Doesn’t Need Specific Operating System
For content delivery applications, this may be the biggest advantage of object storage. Most object storage systems support access to objects via simple and universal content delivery methods, such as REST API calls via HTML, the same hypertext markup language you use to access web pages. As part of its metadata, every object in the system has a unique identifier and a URL. This makes access to content in an object storage system completely platform—and operating system-independent. It also makes it very easy to write applications for object storage. We’re beginning to see that the software applications facilitating content browsing, transcoding, and delivery now can operate directly on objects without the need to bring them back into a file system. This not only creates an efficient delivery solution but also keeps valuable primary storage free to do what it does best—support your most valuable workers as they create even more content for delivery.
While object storage systems are built for the long haul and to scale as needed, they don’t have to be large. Object storage systems can start with as little as 24 terabytes of capacity. At 200 TB over three years, they begin to outpace deep cold storage in the cloud for fewer dollars per TB. So the next time someone offers you another helping of RAID storage for your content delivery pangs, push that plate aside and demand the object storage from the specials menu.