Welcome!

Containers Expo Blog Authors: Mehdi Daoudi, Mark Leake, Todd Matters, Liz McMillan, Elizabeth White

Related Topics: Containers Expo Blog, @CloudExpo, SDN Journal

Containers Expo Blog: Blog Feed Post

Software Defined, Cloud, Bulk and Object Storage Fundamentals | @CloudExpo #SDN #Cloud #Storage

Not everything is the same in cloud, virtual and traditional data centers or information factories

Software Defined, Cloud, Bulk and Object Storage Fundamentals

Welcome to the Cloud, Big Data, Software Defined, Bulk and Object Storage fundamentals page part of the objectstoragecenter.com micro site collection of resources. Software defined, Bulk, Cloud and Object Storage exist to support expanding and diverse application data demands. There are various types of cloud, bulk and object storage including public services such as Amazon Web Services (AWS) Simple Storage Service (S3), Google, Microsoft Azure, IBM Softlayer, Rackspace among many others. There are also solutions for hybrid and private deployment from Cisco, DDN, Dell EMC, Fujitsu, HDS, HPE, IBM, NetApp, Noobaa, OpenStack, Quantum, Rackspace, Scality, Seagate, Spectra, Storpool, Suse, Swift and WD among others.

Cloud products and services among others, along with associated data infrastructures including object storage, file systems, repositories and access methods are at the center of bulk, big data, big bandwidth and little data initiatives on a public, private, hybrid and community basis. After all, not everything is the same in cloud, virtual and traditional data centers or information factories from active data to in-active deep digital archiving.

Cloud Object Storage Fundamentals Access and Architectures
There are many facets to object storage including technology implementation, products, services, access and architectures for various applications and use scenarios.

AWS S3 Cross region replication

Moving and Replicating Buckets/Containers, Subfolders and Objects

  • Project or Account - Top of the hierarchy that can represent the owner or billing information for a service that where buckets are also attached.
  • Region - Location where data is stored that can include one or more data centers also known as Availability Zones.
  • Availability Zone (AZ) or data center or server that implement durability and accessibility for availability within a region.

AWS Regions and Availability Zones AZs

Example of Regions and Availability Zones (AZs)

Object storage and cloud software defined

  • Bucket or Container - Where objects or sub-folders containing objects are attached and accessed.
  • Sub-folder - While object storage can be located in a flat namespace for commonality and organization some solutions and service support the notion of sub-folder that resemble traditional directory hierarchy.
  • Object - Byte (or bit) stream that can be as small as one byte to as large as several Tbytes (some solutions and services support up to 5TByte sized objects). The object contains whatever data in any organization along with metadata. Different solutions and services support from a couple hundred KBytes of meta-data to Mbytes worth of meta-data. Regarding what can be stored in an object, anything from files, videos, images, virtual disks (VMDKs, VHDX), ZIP or tar files, backup and archive save sets, executable images or ISO's, anything you want.
  • End-point - Where or what your software, application or tool and utilities along with gateways attach to for accessing buckets and objects.

object and cloud storage example

A common theme for object storage is flexibility, along with scaling (performance, availability, capacity, economics) along with extensibility without compromise or complexity. From those basics, there are many themes and variations from how data is protected (RAID or no RAID, hardware or software), deployed as a service or as tin wrapped software (an appliance), optimized for archiving or video serving or other applications.

Many facets of cloud and object storage access
One aspect of object and cloud storage is accessing or using object methods including application programming interfaces (APIs) vs. traditional block (LUN) or NAS (file) based approaches. Keep in mind that many object storage systems, software, and services support NAS file-based access including NFS, CIFS, HDFS  among others for compatibility and ease of use.

Likewise various APIs can be found across different object solutions, software or services including Amazon Web Services (AWS) Simple Storage Service (S3) HTTP REST based, among others. Other API's will vary by specific vendor or product however can include IOS (e.g. Apple iPhone and iPad), WebDav, FTP, JSON, XML, XAM, CDMI, SOAP, and DICOM among others. Another aspect of object and cloud storage are expanded  and dynamic metadata.

While traditional file systems and NAS have simple or fixed metadata, object and cloud storage systems, services and solutions along with some scale-out file systems have ability to support user defined metadata. Specific systems, solutions, software, and services will vary on the amount of metadata that could range on the low-end from 100s of KBytes  to tens or more Mbytes.

cloud object storage

Where to learn more
The following resources provide additional information about big data, bulk, software defined, cloud and object storage.

Click here to view software defined, bulk, cloud and object storage trend news.

Via StorageIOblog: AWS EFS Elastic File System (Cloud NAS) First Preview Look

Via InfoStor: Cloud Storage Concerns, Considerations and Trends

Via InfoStor: Object Storage Is In Your Future

Via Server StorageIO: April 2015 Newsletter Focus on Cloud and Object storage

Via StorageIOblog: AWS S3 Cross Region Replication storage enhancements

Cloud conversations: AWS EBS, Glacier and S3 overview

AWS (Amazon) storage gateway, first, second and third impressions

Cloud and Virtual Data Storage Networking (CRC Book)

Videos and podcasts at storageio.tv also available via Applie iTunes.

Human Face of Big Data

Human Face of Big Data (Book review)

Seven Databases in Seven weeks

Seven Databases in Seven Weeks (Book review)

Wrap up and summary
Object and cloud storage are in your future, the questions are when, where, with what and how among others.

Watch for more content and links to be added here soon to this object storage center page including posts, presentations, pod casts, polls, perspectives along with services and product solutions profiles.

Ok, nuff said (for now...).

Cheers
Gs

Greg Schulz - Microsoft MVP Cloud and Data Center Management, VMware vExpert (and vSAN). Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier) and twitter @storageio. Watch for the spring 2017 release of his new book "Software-Defined Data Infrastructure Essentials" (CRC Press).

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2017 Server StorageIO(R) and UnlimitedIO All Rights Reserved

Read the original blog entry...

More Stories By Greg Schulz

Greg Schulz is founder of the Server and StorageIO (StorageIO) Group, an IT industry analyst and consultancy firm. Greg has worked with various server operating systems along with storage and networking software tools, hardware and services. Greg has worked as a programmer, systems administrator, disaster recovery consultant, and storage and capacity planner for various IT organizations. He has worked for various vendors before joining an industry analyst firm and later forming StorageIO.

In addition to his analyst and consulting research duties, Schulz has published over a thousand articles, tips, reports and white papers and is a sought after popular speaker at events around the world. Greg is also author of the books Resilient Storage Network (Elsevier) and The Green and Virtual Data Center (CRC). His blog is at www.storageioblog.com and he can also be found on twitter @storageio.

@ThingsExpo Stories
SYS-CON Events announced today that Enzu will exhibit at SYS-CON's 21st Int\ernational Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Enzu’s mission is to be the leading provider of enterprise cloud solutions worldwide. Enzu enables online businesses to use its IT infrastructure to their competitive advantage. By offering a suite of proven hosting and management services, Enzu wants companies to focus on the core of their ...
With major technology companies and startups seriously embracing Cloud strategies, now is the perfect time to attend 21st Cloud Expo October 31 - November 2, 2017, at the Santa Clara Convention Center, CA, and June 12-14, 2018, at the Javits Center in New York City, NY, and learn what is going on, contribute to the discussions, and ensure that your enterprise is on the right path to Digital Transformation.
Internet of @ThingsExpo, taking place October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 21st Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The Internet of Things (IoT) is the most profound change in personal and enterprise IT since the creation of the Worldwide Web more than 20 years ago. All major researchers estimate there will be tens of billions devic...
Amazon started as an online bookseller 20 years ago. Since then, it has evolved into a technology juggernaut that has disrupted multiple markets and industries and touches many aspects of our lives. It is a relentless technology and business model innovator driving disruption throughout numerous ecosystems. Amazon’s AWS revenues alone are approaching $16B a year making it one of the largest IT companies in the world. With dominant offerings in Cloud, IoT, eCommerce, Big Data, AI, Digital Assista...
SYS-CON Events announced today that Cloud Academy named "Bronze Sponsor" of 21st International Cloud Expo which will take place October 31 - November 2, 2017 at the Santa Clara Convention Center in Santa Clara, CA. Cloud Academy is the industry’s most innovative, vendor-neutral cloud technology training platform. Cloud Academy provides continuous learning solutions for individuals and enterprise teams for Amazon Web Services, Microsoft Azure, Google Cloud Platform, and the most popular cloud com...
When growing capacity and power in the data center, the architectural trade-offs between server scale-up vs. scale-out continue to be debated. Both approaches are valid: scale-out adds multiple, smaller servers running in a distributed computing model, while scale-up adds fewer, more powerful servers that are capable of running larger workloads. It’s worth noting that there are additional, unique advantages that scale-up architectures offer. One big advantage is large memory and compute capacity...
No hype cycles or predictions of zillions of things here. IoT is big. You get it. You know your business and have great ideas for a business transformation strategy. What comes next? Time to make it happen. In his session at @ThingsExpo, Jay Mason, Associate Partner at M&S Consulting, presented a step-by-step plan to develop your technology implementation strategy. He discussed the evaluation of communication standards and IoT messaging protocols, data analytics considerations, edge-to-cloud tec...
"When we talk about cloud without compromise what we're talking about is that when people think about 'I need the flexibility of the cloud' - it's the ability to create applications and run them in a cloud environment that's far more flexible,” explained Matthew Finnie, CTO of Interoute, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
SYS-CON Events announced today that MobiDev, a client-oriented software development company, will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. MobiDev is a software company that develops and delivers turn-key mobile apps, websites, web services, and complex software systems for startups and enterprises. Since 2009 it has grown from a small group of passionate engineers and business...
SYS-CON Events announced today that GrapeUp, the leading provider of rapid product development at the speed of business, will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Grape Up is a software company, specialized in cloud native application development and professional services related to Cloud Foundry PaaS. With five expert teams that operate in various sectors of the market acr...
SYS-CON Events announced today that Ayehu will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on October 31 - November 2, 2017 at the Santa Clara Convention Center in Santa Clara California. Ayehu provides IT Process Automation & Orchestration solutions for IT and Security professionals to identify and resolve critical incidents and enable rapid containment, eradication, and recovery from cyber security breaches. Ayehu provides customers greater control over IT infras...
We build IoT infrastructure products - when you have to integrate different devices, different systems and cloud you have to build an application to do that but we eliminate the need to build an application. Our products can integrate any device, any system, any cloud regardless of protocol," explained Peter Jung, Chief Product Officer at Pulzze Systems, in this SYS-CON.tv interview at @ThingsExpo, held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA
With the introduction of IoT and Smart Living in every aspect of our lives, one question has become relevant: What are the security implications? To answer this, first we have to look and explore the security models of the technologies that IoT is founded upon. In his session at @ThingsExpo, Nevi Kaja, a Research Engineer at Ford Motor Company, discussed some of the security challenges of the IoT infrastructure and related how these aspects impact Smart Living. The material was delivered interac...
Artificial intelligence, machine learning, neural networks. We’re in the midst of a wave of excitement around AI such as hasn’t been seen for a few decades. But those previous periods of inflated expectations led to troughs of disappointment. Will this time be different? Most likely. Applications of AI such as predictive analytics are already decreasing costs and improving reliability of industrial machinery. Furthermore, the funding and research going into AI now comes from a wide range of com...
In his session at Cloud Expo, Alan Winters, an entertainment executive/TV producer turned serial entrepreneur, presented a success story of an entrepreneur who has both suffered through and benefited from offshore development across multiple businesses: The smart choice, or how to select the right offshore development partner Warning signs, or how to minimize chances of making the wrong choice Collaboration, or how to establish the most effective work processes Budget control, or how to ma...
IoT solutions exploit operational data generated by Internet-connected smart “things” for the purpose of gaining operational insight and producing “better outcomes” (for example, create new business models, eliminate unscheduled maintenance, etc.). The explosive proliferation of IoT solutions will result in an exponential growth in the volume of IoT data, precipitating significant Information Governance issues: who owns the IoT data, what are the rights/duties of IoT solutions adopters towards t...
In his session at @ThingsExpo, Eric Lachapelle, CEO of the Professional Evaluation and Certification Board (PECB), provided an overview of various initiatives to certify the security of connected devices and future trends in ensuring public trust of IoT. Eric Lachapelle is the Chief Executive Officer of the Professional Evaluation and Certification Board (PECB), an international certification body. His role is to help companies and individuals to achieve professional, accredited and worldwide re...
SYS-CON Events announced today that CA Technologies has been named "Platinum Sponsor" of SYS-CON's 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. CA Technologies helps customers succeed in a future where every business - from apparel to energy - is being rewritten by software. From planning to development to management to security, CA creates software that fuels transformation for companies in the applic...
SYS-CON Events announced today that IBM has been named “Diamond Sponsor” of SYS-CON's 21st Cloud Expo, which will take place on October 31 through November 2nd 2017 at the Santa Clara Convention Center in Santa Clara, California.
The current age of digital transformation means that IT organizations must adapt their toolset to cover all digital experiences, beyond just the end users’. Today’s businesses can no longer focus solely on the digital interactions they manage with employees or customers; they must now contend with non-traditional factors. Whether it's the power of brand to make or break a company, the need to monitor across all locations 24/7, or the ability to proactively resolve issues, companies must adapt to...