Welcome!

Containers Expo Blog Authors: Liz McMillan, Pat Romanski, APM Blog, Lori MacVittie, Elizabeth White

Related Topics: IBM Cloud, @CloudExpo

IBM Cloud: Blog Post

Databases in the Cloud

Cloud computing is such a fascinating topic because ...

Cloud computing is such a fascinating topic because, to anybody who can get enough distance from the marketing hype, it really represents a significant departure from the way the software and the hardware industry have been operating in the last decades. In fact, cloud computing is to the traditional IT industry what the Internet has been to the music and film industry: an unwelcome and very threatening development dictating completely different business models.

The basic technical premises behind cloud computing are known since many years. Already in the 80's, as the number of computers around constantly increased, it did not take long until people realized many of them were idle for most of time. The result was the first cluster management tools capable of moving simple jobs between machines. From there, the advances in networking during the 90's brought Grid Computing and then the large computer farms behind the Web brought what we now call Cloud Computing. On the software side, a similar development took place over the years going from monolithic designs to multi-tier architectures and from tightly coupled systems to service oriented architectures. Add virtualization, and all the basic tools for building computing clouds are in place.

Whether Software as a Service (SaaS) or Platform as a Service (PaaS), there are many examples out there of services that work well; make economic sense to all parties involved; and are destined to grow both in the number of users as well as in the functionality they provide. This, however, does not seem to be the case for databases. At least not for databases containing important data.

This is an interesting observation since, technically, there is nothing that prevents databases from residing in the cloud. To understand the complex relation between databases and the cloud, one needs to understand the complex chain of problems that need to be solved before a database with important data resides in the cloud. These problems are: 

- legal aspects of where the data resides

- long term custody warranties

- trust in the cloud

If the data residing in a database is of any real value to anybody except a small group of individuals, it is likely that there are many regulations imposing a wide variety of constraints on where the data can be, who can look at it, and whether it can be moved anywhere. For instance, in federal countries, local governments often have legislation imposing that the data must be stored within the region. At larger scale, it seems unlikely that a country would agree to have government data stored in a different country. In the private sector, many software development outsourcing efforts have failed because of the difficulty to provide realistic data for testing without giving any confidential information away. And if the database stays where it is, it is unlikely that the software stack built on top of it will move to the cloud.

Assuming there is a cloud in the vicinity that fulfills all the locality requirements, the next hurdle is the legal custodian warranties imposed on data. Important and relevant data must be by law available and searchable for long periods of time, often many decades. Clouds cannot provide such guarantees today and, in this matter, the IT industry has never dealt with such time horizons before.

Finally, even there is a cloud in the proper place that guarantees that it will stay there for the nest 50 years, the question that remains is whether it can be trusted to do so. What happens to the data if the cloud simply disappears? Replication makes the location problem even more difficult and it certainly does not help to reduce the cost of the cloud. It also does not solve the problem of a company simply shutting down the service. Without very strong, enforceable guarantees -as it happens in other branches of industry that are critical to the economy - there will be not enough trust to move databases with important data into a commercial cloud.

Does this imply that we will never see databases in the cloud? Not at all. However, the clouds were important databases will reside might be different from the commercial ones that are attracting so much attention these days.

First, the clouds where databases may live very comfortably will be private clouds. Governments, for instance, are likely to own (or contract) such clouds to offer cloud services to the public sector. Second, community clouds linking the private clouds of partner companies are also likely to be common since they spread the costs among several participants while still giving access to more resources that anyone of them directly owns. Being a federation of private clouds, they are easier to protect, organize under well defined contractual agreements, and to tailor to the particular application by using, e.g., application aware networks. Third, public clouds will be used not necessarily for storing the data but for scalability and processing purposes in all those cases where parts of the data can be safely brought into the open. For instance, a company can keep the confidential data within the private cloud but place databases with copies of the publicly available data on a public cloud. By keeping the master copy of the data, the company takes advantage of the public cloud for scalability but can make sure all regulations are followed in house using conventional solutions.

The challenges to bring databases into the cloud are both technical and regulatory. Until the regulatory problems are solved -and that may take a long time- the key to putting databases into the cloud will be to have infrastructures that give users flexibility and complete control over the databases and the data inside, regardless of the type of cloud used. If users can easily create copies of their databases or part of their databases and place those into the cloud; can guarantee that they have within their premises a consistent copy of the data at all times; and can take advantage of the cloud to reduce the costs of provisioning, scaling out, and adding functionality to their data management systems, then databases will move to the cloud. If all these chores are not provided through automatic tools, the overhead, costs, and risks involved will be too high to justify moving enterprise class databases to the cloud.

More Stories By Maximilian Ahrens

Ahrens is an expert and frequent speaker on international conferences for service oriented architecture and virtualization. Before co-founding Zimory, he served as a project manager and research scientist at the innovation development entity of Deutsche Telekom Laboratories. Responsible for infrastructure and enterprise IT projects spanning multiple divisions of the Deutsche Telekom group -- Ahrens is an expert on enterprise IT and business processes. Before Deutsche Telekom, he led several business process reengineering projects for major German companies. Ahrens received his degree in computer science and business administration from Technische Universität Berlin.

@ThingsExpo Stories
Digital transformation is too big and important for our future success to not understand the rules that apply to it. The first three rules for winning in this age of hyper-digital transformation are: Advantages in speed, analytics and operational tempos must be captured by implementing an optimized information logistics system (OILS) Real-time operational tempos (IT, people and business processes) must be achieved Businesses that can "analyze data and act and with speed" will dominate those t...
Almost two-thirds of companies either have or soon will have IoT as the backbone of their business in 2016. However, IoT is far more complex than most firms expected. How can you not get trapped in the pitfalls? In his session at @ThingsExpo, Tony Shan, a renowned visionary and thought leader, will introduce a holistic method of IoTification, which is the process of IoTifying the existing technology and business models to adopt and leverage IoT. He will drill down to the components in this fra...
As ridesharing competitors and enhanced services increase, notable changes are occurring in the transportation model. Despite the cost-effective means and flexibility of ridesharing, both drivers and users will need to be aware of the connected environment and how it will impact the ridesharing experience. In his session at @ThingsExpo, Timothy Evavold, Executive Director Automotive at Covisint, will discuss key challenges and solutions to powering a ride sharing and/or multimodal model in the a...
Internet of @ThingsExpo, taking place November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with the 19th International Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world and ThingsExpo Silicon Valley Call for Papers is now open.
I'm a lonely sensor. I spend all day telling the world how I'm feeling, but none of the other sensors seem to care. I want to be connected. I want to build relationships with other sensors to be more useful for my human. I want my human to understand that when my friends next door are too hot for a while, I'll soon be flaming. And when all my friends go outside without me, I may be left behind. Don't just log my data; use the relationship graph. In his session at @ThingsExpo, Ryan Boyd, Engi...
SYS-CON Events announced today that Pulzze Systems will exhibit at the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Pulzze Systems, Inc. provides infrastructure products for the Internet of Things to enable any connected device and system to carry out matched operations without programming. For more information, visit http://www.pulzzesystems.com.
Why do your mobile transformations need to happen today? Mobile is the strategy that enterprise transformation centers on to drive customer engagement. In his general session at @ThingsExpo, Roger Woods, Director, Mobile Product & Strategy – Adobe Marketing Cloud, covered key IoT and mobile trends that are forcing mobile transformation, key components of a solid mobile strategy and explored how brands are effectively driving mobile change throughout the enterprise.
SYS-CON Events announced today that ReadyTalk, a leading provider of online conferencing and webinar services, has been named Vendor Presentation Sponsor at the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. ReadyTalk delivers audio and web conferencing services that inspire collaboration and enable the Future of Work for today’s increasingly digital and mobile workforce. By combining intuitive, innovative tec...
If you’re responsible for an application that depends on the data or functionality of various IoT endpoints – either sensors or devices – your brand reputation depends on the security, reliability, and compliance of its many integrated parts. If your application fails to deliver the expected business results, your customers and partners won't care if that failure stems from the code you developed or from a component that you integrated. What can you do to ensure that the endpoints work as expect...
WebRTC adoption has generated a wave of creative uses of communications and collaboration through websites, sales apps, customer care and business applications. As WebRTC has become more mainstream it has evolved to use cases beyond the original peer-to-peer case, which has led to a repeating requirement for interoperability with existing infrastructures. In his session at @ThingsExpo, Graham Holt, Executive Vice President of Daitan Group, will cover implementation examples that have enabled ea...
There is growing need for data-driven applications and the need for digital platforms to build these apps. In his session at 19th Cloud Expo, Muddu Sudhakar, VP and GM of Security & IoT at Splunk, will cover different PaaS solutions and Big Data platforms that are available to build applications. In addition, AI and machine learning are creating new requirements that developers need in the building of next-gen apps. The next-generation digital platforms have some of the past platform needs a...
So, you bought into the current machine learning craze and went on to collect millions/billions of records from this promising new data source. Now, what do you do with them? Too often, the abundance of data quickly turns into an abundance of problems. How do you extract that "magic essence" from your data without falling into the common pitfalls? In her session at @ThingsExpo, Natalia Ponomareva, Software Engineer at Google, provided tips on how to be successful in large scale machine learning...
In his session at @ThingsExpo, Kausik Sridharabalan, founder and CTO of Pulzze Systems, Inc., will focus on key challenges in building an Internet of Things solution infrastructure. He will shed light on efficient ways of defining interactions within IoT solutions, leading to cost and time reduction. He will also introduce ways to handle data and how one can develop IoT solutions that are lean, flexible and configurable, thus making IoT infrastructure agile and scalable.
SYS-CON Events announced today that Numerex Corp, a leading provider of managed enterprise solutions enabling the Internet of Things (IoT), will exhibit at the 19th International Cloud Expo | @ThingsExpo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Numerex Corp. (NASDAQ:NMRX) is a leading provider of managed enterprise solutions enabling the Internet of Things (IoT). The Company's solutions produce new revenue streams or create operating...
In his general session at 18th Cloud Expo, Lee Atchison, Principal Cloud Architect and Advocate at New Relic, discussed cloud as a ‘better data center’ and how it adds new capacity (faster) and improves application availability (redundancy). The cloud is a ‘Dynamic Tool for Dynamic Apps’ and resource allocation is an integral part of your application architecture, so use only the resources you need and allocate /de-allocate resources on the fly.
IoT is fundamentally transforming the auto industry, turning the vehicle into a hub for connected services, including safety, infotainment and usage-based insurance. Auto manufacturers – and businesses across all verticals – have built an entire ecosystem around the Connected Car, creating new customer touch points and revenue streams. In his session at @ThingsExpo, Macario Namie, Head of IoT Strategy at Cisco Jasper, will share real-world examples of how IoT transforms the car from a static p...
Fifty billion connected devices and still no winning protocols standards. HTTP, WebSockets, MQTT, and CoAP seem to be leading in the IoT protocol race at the moment but many more protocols are getting introduced on a regular basis. Each protocol has its pros and cons depending on the nature of the communications. Does there really need to be only one protocol to rule them all? Of course not. In his session at @ThingsExpo, Chris Matthieu, co-founder and CTO of Octoblu, walk you through how Oct...
Web Real-Time Communication APIs have quickly revolutionized what browsers are capable of. In addition to video and audio streams, we can now bi-directionally send arbitrary data over WebRTC's PeerConnection Data Channels. With the advent of Progressive Web Apps and new hardware APIs such as WebBluetooh and WebUSB, we can finally enable users to stitch together the Internet of Things directly from their browsers while communicating privately and securely in a decentralized way.
"My role is working with customers, helping them go through this digital transformation. I spend a lot of time talking to banks, big industries, manufacturers working through how they are integrating and transforming their IT platforms and moving them forward," explained William Morrish, General Manager Product Sales at Interoute, in this SYS-CON.tv interview at 18th Cloud Expo, held June 7-9, 2016, at the Javits Center in New York City, NY.
The Internet of Things can drive efficiency for airlines and airports. In their session at @ThingsExpo, Shyam Varan Nath, Principal Architect with GE, and Sudip Majumder, senior director of development at Oracle, will discuss the technical details of the connected airline baggage and related social media solutions. These IoT applications will enhance travelers' journey experience and drive efficiency for the airlines and the airports. The session will include a working demo and a technical d...