Welcome!

Containers Expo Blog Authors: Liz McMillan, Pat Romanski, Yeshim Deniz, Elizabeth White, Zakia Bouachraoui

Related Topics: @CloudExpo, Containers Expo Blog, @DXWorldExpo

@CloudExpo: Blog Feed Post

Data Infrastructure Primer | @CloudExpo #SDN #AI #Storage #DataCenter

Data Infrastructures exists to support business, cloud and information technology (IT)

Data Infrastructures exists to support business, cloud and information technology (IT) among other applications that transform data into information or services. The fundamental role of data infrastructures is to provide a platform environment for applications and data that is resilient, flexible, scalable, agile, efficient as well as cost-effective. Put another way, data infrastructures exist to protect, preserve, process, move, secure and serve data as well as their applications for information services delivery. Technologies that make up data infrastructures include hardware, software, cloud or managed services, servers, storage, I/O and networking along with people, processes, policies along with various tools spanning legacy, software-defined virtual, containers and cloud.

Data
Various Types and Layers of Infrastructures
Depending on your role or focus, you may have a different view than somebody else of what is infrastructure, or what an infrastructure is. Generally speaking, people tend to refer to infrastructure as those things that support what they are doing at work, at home, or in other aspects of their lives. For example, the roads and bridges that carry you over rivers or valleys when traveling in a vehicle are referred to as infrastructure.

Similarly, the system of pipes, valves, meters, lifts, and pumps that bring fresh water to you, and the sewer system that takes away waste water, are called infrastructure. The telecommunications network. This includes both wired and wireless, such as cell phone networks, along with electrical generating and transmission networks are considered infrastructure. Even the airplanes, trains, boats, and buses that transport us locally or globally are considered part of the transportation infrastructure. Anything that is below what you do, or that supports what you do is considered infrastructure.

Software Defined Data Infrastructure overview

Figure 1: Business, IT Information, Data and other Infrastructures

This is also the situation with IT systems and services where, depending on where you sit or use various services, anything below what you do may be considered infrastructure. However, that also causes a context issue in that infrastructure can mean different things. For example in Figure 1, the user, customer, client, or consumer who is accessing some service or application may view IT in general as infrastructure, or perhaps as business infrastructure.

Those who develop, service, and support the business infrastructure and its users or clients may view anything below them as infrastructure, from desktop to database, servers to storage, network to security, data protection to physical facilities. Moving down a layer (lower altitude) in figure 1 is the information infrastructure which, depending on your view, may also include servers, storage, and I/O hardware and software.

To help make a point, let's think of the information infrastructure as the collection of databases, key-value stores, repositories, and applications along with development tools that support the business infrastructure. This is where you may find developers who maintain and create real business applications for the business infrastructure. Those in the information infrastructure usually refer to what's below them as infrastructure. Meanwhile, those lower in the stack shown in figure 1 may refer to what's above them as the customer, user, or application, even if the real user is up another layer or two.

Whats inside a data infrastructure
Context matters in the discussion of infrastructure. So for our of server storage I/O fundamentals, the data infrastructures support the databases and applications developers as well as things above, while existing above the physical facilities infrastructure, leveraging power, cooling, and communication network infrastructures below.

SDDI and Data Infrastructure building blocks
Figure 2 Data Infrastructure fundamental building blocks (hardware, software, services)

Figure 2 shows the fundamental pillars or building blocks for a data infrastructure, including servers for computer processing, I/O networks for connectivity, and storage for storing data. These resources including both hardware and software as well as services and tools. The size of the environment, organization, or application needs will determine how large or small the data infrastructure is or can be.

For example, at one extreme you can have a single high-performance laptop with a hypervisor running OpenStack; along with various operating systems along with their applications leveraging flash SSD and high-performance wired or wireless networks powering a home lab or test environment. On the other hand, you can have a scenario with tens of thousands (or more) servers, networking devices, and hundreds of petabytes (PBs) of storage (or more).

In Figure 2 the primary data infrastructure components or pillar (server, storage, and I/O) hardware and software resources are packaged and defined to meet various needs. Software-defined storage management includes configuring the server, storage, and I/O hardware and software as well as services for use, implementing data protection and security, provisioning, diagnostics, troubleshooting, performance analysis, and other activities. Server storage and I/O hardware and software can be individual components, prepackaged as bundles or application suites and converged, among other options.

Figure 3 shows a deeper look into the data infrastructure shown at a high level in figure 2. The lower left of figure 2 shows the common-to-all-environments hardware, software, people, processes, and practices that include tradecraft (experiences, skills, techniques) and "valueware". Valueware is how you define the hardware and software along with any customization to create a resulting service that adds value to what you are doing or supporting. Also shown in figure 3 are common application and services attributes including performance, availability, capacity, and economics (PACE), which vary with different applications or usage scenarios.

Data Infrastructure components

Figure 3: Data Infrastructure server storage I/O hardware and software components.

Applications are what transform data into information. Figure 4 shows how applications, which are software defined by people and software, consist of algorithms, policies, procedures, and rules that are put into some code to tell the server processor (CPU) what to do.

SDDI and SDDC server storage I/O

Figure 4: How data infrastructure resources transform data into information

Application programs include data structures (not to be confused with infrastructures) that define what data looks like and how to organize and access it using the "rules of the road" (the algorithms). The program algorithms along with data structures are stored in memory, together with some of the data being worked on (i.e., the active working set). Additional data is stored in some form of extended memory storage devices such as Non-Volatile Memory (NVM) solid-state devices (SSD), hard disk drives (HDD), or tape, among others, either locally or remotely. Also shown in Figure 4 are various devices that do input/output (I/O) with the applications and server, including mobile devices as well as other application servers.

Bringing IT All Together (for now)

Software Defined Data Infrastructure overview

Figure 5: Data Infrastructure  fundamentals "big picture"

A fundamental theme is that servers process data using various applications programs to create information; I/O networks provide connectivity to access servers and storage; storage is where data gets stored, protected, preserved, and served from; and all of this needs to be managed. There are also many technologies involved, including hardware, software, and services as well as various techniques that make up a server, storage, and I/O enabled data infrastructure.

Server storage I/O and data infrastructure fundamental focus areas include:

  • Organizations: Markets and industry focus, organizational size
  • Applications: What's using, creating, and resulting in server storage I/O demands
  • Technologies: Tools and hard products (hardware, software, services, packaging)
  • Trade craft: Techniques, skills, best practices, how managed, decision making
  • Management: Configuration, monitoring, reporting, troubleshooting, performance, availability, data protection and security, access, and capacity planning

Where to Learn More

StorageIO.com (events, news, tips, resources) and StorageIOblog.com
Cloud and Virtual Data Storage Networking
(CRC)
And watch for my new book Software-Defined Data Infrastructure Essentials (CRC)

What This All Means
Whether you realize it or not, you may already be using, rely upon, affiliated with, support or otherwise involved with data infrastructures. Granted what you or others generically refer to as infrastructure or the data center may, in fact, be the data infrastructure. Watch for more discussions and content about as well as related technologies, tools, trends, techniques and tradecraft in future posts as well as other venues, some of which involve legacy, others software-defined, cloud, virtual, container and hybrid.

Ok, nuff said, for now...

Cheers
Gs

Greg Schulz - Microsoft MVP Cloud and Data Center Management, vSAN and VMware vExpert. Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier) and twitter @storageio. Watch for the spring 2017 release of his new book "Software-Defined Data Infrastructure Essentials" (CRC Press).

All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2017 Server StorageIO(R) and UnlimitedIO All Rights Reserved

More Stories By Greg Schulz

Greg Schulz is founder of the Server and StorageIO (StorageIO) Group, an IT industry analyst and consultancy firm. Greg has worked with various server operating systems along with storage and networking software tools, hardware and services. Greg has worked as a programmer, systems administrator, disaster recovery consultant, and storage and capacity planner for various IT organizations. He has worked for various vendors before joining an industry analyst firm and later forming StorageIO.

In addition to his analyst and consulting research duties, Schulz has published over a thousand articles, tips, reports and white papers and is a sought after popular speaker at events around the world. Greg is also author of the books Resilient Storage Network (Elsevier) and The Green and Virtual Data Center (CRC). His blog is at www.storageioblog.com and he can also be found on twitter @storageio.

IoT & Smart Cities Stories
The deluge of IoT sensor data collected from connected devices and the powerful AI required to make that data actionable are giving rise to a hybrid ecosystem in which cloud, on-prem and edge processes become interweaved. Attendees will learn how emerging composable infrastructure solutions deliver the adaptive architecture needed to manage this new data reality. Machine learning algorithms can better anticipate data storms and automate resources to support surges, including fully scalable GPU-c...
Machine learning has taken residence at our cities' cores and now we can finally have "smart cities." Cities are a collection of buildings made to provide the structure and safety necessary for people to function, create and survive. Buildings are a pool of ever-changing performance data from large automated systems such as heating and cooling to the people that live and work within them. Through machine learning, buildings can optimize performance, reduce costs, and improve occupant comfort by ...
The explosion of new web/cloud/IoT-based applications and the data they generate are transforming our world right before our eyes. In this rush to adopt these new technologies, organizations are often ignoring fundamental questions concerning who owns the data and failing to ask for permission to conduct invasive surveillance of their customers. Organizations that are not transparent about how their systems gather data telemetry without offering shared data ownership risk product rejection, regu...
René Bostic is the Technical VP of the IBM Cloud Unit in North America. Enjoying her career with IBM during the modern millennial technological era, she is an expert in cloud computing, DevOps and emerging cloud technologies such as Blockchain. Her strengths and core competencies include a proven record of accomplishments in consensus building at all levels to assess, plan, and implement enterprise and cloud computing solutions. René is a member of the Society of Women Engineers (SWE) and a m...
Poor data quality and analytics drive down business value. In fact, Gartner estimated that the average financial impact of poor data quality on organizations is $9.7 million per year. But bad data is much more than a cost center. By eroding trust in information, analytics and the business decisions based on these, it is a serious impediment to digital transformation.
Digital Transformation: Preparing Cloud & IoT Security for the Age of Artificial Intelligence. As automation and artificial intelligence (AI) power solution development and delivery, many businesses need to build backend cloud capabilities. Well-poised organizations, marketing smart devices with AI and BlockChain capabilities prepare to refine compliance and regulatory capabilities in 2018. Volumes of health, financial, technical and privacy data, along with tightening compliance requirements by...
Predicting the future has never been more challenging - not because of the lack of data but because of the flood of ungoverned and risk laden information. Microsoft states that 2.5 exabytes of data are created every day. Expectations and reliance on data are being pushed to the limits, as demands around hybrid options continue to grow.
Digital Transformation and Disruption, Amazon Style - What You Can Learn. Chris Kocher is a co-founder of Grey Heron, a management and strategic marketing consulting firm. He has 25+ years in both strategic and hands-on operating experience helping executives and investors build revenues and shareholder value. He has consulted with over 130 companies on innovating with new business models, product strategies and monetization. Chris has held management positions at HP and Symantec in addition to ...
Enterprises have taken advantage of IoT to achieve important revenue and cost advantages. What is less apparent is how incumbent enterprises operating at scale have, following success with IoT, built analytic, operations management and software development capabilities - ranging from autonomous vehicles to manageable robotics installations. They have embraced these capabilities as if they were Silicon Valley startups.
As IoT continues to increase momentum, so does the associated risk. Secure Device Lifecycle Management (DLM) is ranked as one of the most important technology areas of IoT. Driving this trend is the realization that secure support for IoT devices provides companies the ability to deliver high-quality, reliable, secure offerings faster, create new revenue streams, and reduce support costs, all while building a competitive advantage in their markets. In this session, we will use customer use cases...