Containers Expo Blog Authors: Liz McMillan, Yeshim Deniz, Pat Romanski, Elizabeth White, Ravi Rajamiyer

Related Topics: Containers Expo Blog, Java IoT, Open Source Cloud, @CloudExpo

Containers Expo Blog: Blog Feed Post

Do Object Storage Plays Displace File Systems or Are They Absorbed?

Expect object storage systems to become more and more popular.

NAB kept me totally away from all the interesting online discussions last week. It’s too late to respond to @JoinToigo’s tweet (we’d call this Figs after Easter in Dutch), but I thought I’d share my thoughts in a bit more than 140 characters.

The short answer is no … but a better answer is very much *yes*.

The first file systems were not designed with the thought of petabytes of data. I don’t know what the exact projections were back then, but gigabytes must have sounded pretty sci-fi. Bytes and kilobytes were a lot more common. We didn’t think that we’d soon all be creating tens if not hundreds of multi-megabyte files per day.

File systems have of course evolved a lot and some have become so popular you could actually say they have a fan base (I’d need to do research on ZFS fan clubs). It is clear that the file system has played a very important role in the evolution of the computer industry. In my list of features that helped to make computers a commodity, the file system would probably be in the top three (with the windows-style GUI and the mouse). The file system enables the use of directories, which have been the most important tool to keep our data organized.

But like Robin Harris says “entropy refers to the inherent tendency for any organized system to disorder”. Especially with the amounts of data we are dealing with today, we have to put a lot of energy into keeping our data organized. We have come to a point where our directories are not that organized anymore because we have too much data. But that doesn’t matter all that much since there are so many applications out there (and a lot more coming) that can do this for us.

Take Google docs for example. Docs lets you star and share your documents, and organize them in collections. And no matter how you organize your stuff, Docs will find it back for you. Docs has a great search function (it’s Google after all) that is lot more powerful than the search in Windows Explorer or OSX’ Finder (although spotlight is actually pretty good). Picasa and Itunes are just two more examples of applications that help us keep our data organized with hardly any role for the file system. Eventually the applications will make the file system obsolete. Many of the applications we are using today are cloud based and run on object storage, with no file system involved, the application just masks the lack of a file system.

For businesses the situation is the same but different. Applications in the cloud are increasingly popular, so a lot of business data is already stored in a public or private object store. But a lot of business applications simply need a file system interface. For now, that is. If the current data growth continues, a lot of file systems will hit their scalability limits. And here object storage will play a very important role as object storage platforms have been (at least the good ones) designed to scale out big.

One interesting example is the media and entertainment industry. If there is one industry where data is big, it’s there: think of the 4k and soon 8k movies. Movies have become multi-petabyte projects (tens of petabytes). Companies in this industry understand they need more efficient storage and tape is no longer an option. All major studios are running object storage projects right now (mostly with file systems on top). This frees them from worrying about “how many files fit into a directory before it slows down”, and “how many directories can I have” and “how deep can my file system tree become” – especially as it relates to access performance.

So, expect object storage systems to become more and more popular. As long as needed, object storage will be implemented with some file system gateway on top but eventually, when the applications are ready, we will see less and less file systems. It just makes more sense to have the application talk directly to the storage. REST makes it all very simple. And fast. And economically feasible.

And now, anticipating the next question: Shouldn’t there be some standard REST API? I used to strongly believe so. But while doing research for this piece, I stumbled across Wikipedia’s list of file systems. With hardly a dozen object storage REST API’s on the market, it’s not all that bad in my opinion. Still, I believe object storage vendors all agree standardization is good. It’s just a matter of waiting to see which API will eventually become the most popular with the applications that use them.

Read the original blog entry...

More Stories By Tom Leyden

Tom Leyden is VP Product Marketing at Scality. Scality was founded in 2009 by a team of entrepreneurs and technologists. The idea wasn’t storage, per se. When the Scality team talked to the initial base of potential customers, the customers wanted a system that could “route” data to and from individual users in the most scalable, efficient way possible. And so began a non-traditional approach to building a storage system that no one had imagined before. No one thought an object store could have enough performance for all the files and attachments of millions of users. No one thought a system could remain up and running through software upgrades, hardware failures, capacity expansions, and even multiple hardware generations coexisting. And no one believed you could do all this and scale to petabytes of content and billions of objects in pure software.

IoT & Smart Cities Stories
The Internet of Things will challenge the status quo of how IT and development organizations operate. Or will it? Certainly the fog layer of IoT requires special insights about data ontology, security and transactional integrity. But the developmental challenges are the same: People, Process and Platform and how we integrate our thinking to solve complicated problems. In his session at 19th Cloud Expo, Craig Sproule, CEO of Metavine, demonstrated how to move beyond today's coding paradigm and sh...
The best way to leverage your Cloud Expo presence as a sponsor and exhibitor is to plan your news announcements around our events. The press covering Cloud Expo and @ThingsExpo will have access to these releases and will amplify your news announcements. More than two dozen Cloud companies either set deals at our shows or have announced their mergers and acquisitions at Cloud Expo. Product announcements during our show provide your company with the most reach through our targeted audiences.
@DevOpsSummit at Cloud Expo, taking place November 12-13 in New York City, NY, is co-located with 22nd international CloudEXPO | first international DXWorldEXPO and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time t...
What are the new priorities for the connected business? First: businesses need to think differently about the types of connections they will need to make – these span well beyond the traditional app to app into more modern forms of integration including SaaS integrations, mobile integrations, APIs, device integration and Big Data integration. It’s important these are unified together vs. doing them all piecemeal. Second, these types of connections need to be simple to design, adapt and configure...
Cell networks have the advantage of long-range communications, reaching an estimated 90% of the world. But cell networks such as 2G, 3G and LTE consume lots of power and were designed for connecting people. They are not optimized for low- or battery-powered devices or for IoT applications with infrequently transmitted data. Cell IoT modules that support narrow-band IoT and 4G cell networks will enable cell connectivity, device management, and app enablement for low-power wide-area network IoT. B...
Contextual Analytics of various threat data provides a deeper understanding of a given threat and enables identification of unknown threat vectors. In his session at @ThingsExpo, David Dufour, Head of Security Architecture, IoT, Webroot, Inc., discussed how through the use of Big Data analytics and deep data correlation across different threat types, it is possible to gain a better understanding of where, how and to what level of danger a malicious actor poses to an organization, and to determ...
Nicolas Fierro is CEO of MIMIR Blockchain Solutions. He is a programmer, technologist, and operations dev who has worked with Ethereum and blockchain since 2014. His knowledge in blockchain dates to when he performed dev ops services to the Ethereum Foundation as one the privileged few developers to work with the original core team in Switzerland.
Cloud-enabled transformation has evolved from cost saving measure to business innovation strategy -- one that combines the cloud with cognitive capabilities to drive market disruption. Learn how you can achieve the insight and agility you need to gain a competitive advantage. Industry-acclaimed CTO and cloud expert, Shankar Kalyana presents. Only the most exceptional IBMers are appointed with the rare distinction of IBM Fellow, the highest technical honor in the company. Shankar has also receive...
Digital Transformation and Disruption, Amazon Style - What You Can Learn. Chris Kocher is a co-founder of Grey Heron, a management and strategic marketing consulting firm. He has 25+ years in both strategic and hands-on operating experience helping executives and investors build revenues and shareholder value. He has consulted with over 130 companies on innovating with new business models, product strategies and monetization. Chris has held management positions at HP and Symantec in addition to ...
The hierarchical architecture that distributes "compute" within the network specially at the edge can enable new services by harnessing emerging technologies. But Edge-Compute comes at increased cost that needs to be managed and potentially augmented by creative architecture solutions as there will always a catching-up with the capacity demands. Processing power in smartphones has enhanced YoY and there is increasingly spare compute capacity that can be potentially pooled. Uber has successfully ...