Welcome!

Containers Expo Blog Authors: Liz McMillan, Yeshim Deniz, Elizabeth White, Zakia Bouachraoui, Pat Romanski

Related Topics: @CloudExpo, Microservices Expo, @DXWorldExpo, SDN Journal, @ThingsExpo

@CloudExpo: Blog Post

How to Avoid the #BigData Black Hole | @BigDataExpo [#IoT]

It's hard to avoid the world of opportunities that has been opened in data collection and analysis

It takes a lot of will power, in our data obsessed world to say "too much!" However, there are many ways where too much information is destroying productivity, and actually causing bad decision making, not good. But it is hard to avoid the world of opportunities that has been opened in data collection and analysis. So how do you balance the two? The first step is to understand there is a big difference between data collection, and it's utilization. While it seems subtle, the difference is key, and utilization is where many make mistakes.

Many confuse data collection and data utilization as the same thing, or at least being very similar.
Collection merely indicates the storage and analysis of data by automated systems, maybe never viewed by human eye or mind. It is when you spend the time to access and understand the data collected where there is a human cost.

"When information is cheap, attention becomes expensive." - James Gleick

New tools mask human effort by taking raw data (in hard to consume formats) and making it much easier to understand, and more importantly; query upon. This makes users feel like they need to look at it all, and the UI is saving them all the time they need. But, in most cases, we did not previously collect this information, nor analyze it. So the opportunity is higher, but any effort is net new.

big-data-blackhole

When I get in front of a great log management tool, I sometimes get that same feeling as when I get addicted to a new iPad game; crushing candies, or tending to farms. Because there is a sort of gamification when you dig, and if I just dig a little deeper I will reach the goal. Next thing you know, you have spent three hours without any results.

"Information is not knowledge and knowledge is not wisdom." - James Gleick,

It is a people problem too
There is effort required to first understand, and then act on the information we receive. But the enthusiasm for learning more, and "information is power" creates a sort of blindness. A blindness that skirts the need to quantify our effort spent in analyzing the data against the results (decisions, actions items) that from it. To make things worse, the amount of data it takes for one decision is often way more than we realize.

So, what is the impact of spending too much time trying to utilize the new pile of information?

  • Time suck: Being taken away from daily tasks, by asking a question of the platform that takes way more time than expected.
  • Dwelling on details that do not impact the business: Leading to zero actionable results.
  • The anchoring effect: If you are familiar with the psychological anchoring effect, where new decisions are based on an existing foundation of positions, and opinions. With all the data in your log analysis platform, if you are not aware of anchoring, you will the ask questions that produce the answers you seek.
  • "That is what the numbers say ...." Platforms that only produce time-series dashboards and numbers lead to the habit of using this data as an default answer to all questions, and deflecting mechanism. It's easy to put the decisions on the system, instead of the people who were previously responsible for them.

These are not light topics I know, but I have seen time and time again where the abuse of data, and the time wasted on collecting it eats organizations from within.

Does the tool save your time or steal your eyeballs?
Part of this whole situation has a lot to do with the tools. Tools can be good or bad at surfacing information. All the latest APM, Log Analysis, Error Logging, and usage analytics tools have ways for you to dig-in deep. Putting the burden of when enough is enough on the user. Who often is too absorbed to keep track.

In fact, many of these tools encourage you to go deep, eventually spending hours sifting thought information that provides no real value real, but perhaps a lot of little "oh that is interesting". Those little hits are just like pulling the lever on a casino machine. If you have to become specialized to use the platform, and make it a large portion of your daily effort, you are committed. After all, churn is a major concern for tech companies. But should it be?

"Business is not just doing deals; business is having great products, doing great engineering, and providing tremendous service to customers." - Ross Perot

A great tool should help you get the data, make decisions (or prepare them for the expanded team), and move on.

It should prevent you from entering the data black hole, and guide you to what is most useful. The tool should also put a layer between you and the data, not to encourage the diving-in too deep which we know can happen quickly.

Here are some ways to avoid these problems:

  • Hire a dedicated analysis person: Definitely a nice-to-have. But really, this is just a workaround for a bigger problem.
  • Ask discrete questions: If you keep your questions simple and small, then you can move faster. But we all know that there are bigger questions we want answered.
  • Plan your logs in advance for utilization, not collection: If you think about how you want to use the information, instead of just gathering data, you will make the analysis go much faster. In a crunch, this seems like a wasted effort.
  • Focusing: There are a lot of "Fun" questions you can ask. But time is better spent on questions decided-on in advance and related to current initiatives and activity. At the same time, wouldn't it be nice for the data to tell you what is interesting without you having to predict it in advance?
  • FIND THE RIGHT TOOL! Many analysis tools only help "send information and put in a nice format ", aka data collection. While this is a must, the focus of the tool should be on utilization. And this should be their roadmap focus.

Find it, act on it, move on
Part of creating this value layer is discovering what is happening, but also what is not happening, and what is out of normal. Watching the flow of data can be memorizing, but decisions are made on things that are not consistent with the steady state of systems and applications. Or with sudden changes. Things that normally occur in the system, but for some recently did not.

Data collection should be synthesized into meaningful events. Getting users addicted to a platform by the quality and frequency of decisions versus encouraging them to spin the wheel to see what happens and becoming a 5th limb.

First, admit to yourself that there is too much information, more than you could every analyze on your own. And then find a tool that takes over analysis, and only shows you what is valuable so that you and your team can get on with your day.

Conference Schedule Announced

Are you ready to put your data in the cloud?

What is the future of security in the cloud?

Does Docker quickly advance the development of an IoT application?

What are the implications of Moore's Law on Hadoop deployments?

Get all these questions and hundreds more like them answered at the 15th Cloud Expo, November 4-6, 2014, at the Santa Clara Convention Center, in Santa Clara, CA. The Cloud Expo / Big Data Expo / @ThingsExpo / DevOps Summit programs are now available for you to inspect and investigate in advance.

Our upcoming November 4-6 event in Santa Clara, California will present a total of 10 simultaneous tracks by an all-star faculty, over three days, plus a two-day "Cloud Computing Bootcamp" presented by Janakiram MSV, an Analyst with the Gigaom Research analyst network, where he covers the Cloud Services landscape.

Cloud and Big Data topics and tracks include: Enterprise Cloud Adoption, APM & Cloud Computing | Hot Topics, Cloud APIs & Business, Cloud Security | Mobility, Big Data | Analytics.

@ThingsExpo content tripled from a single track in New York to three simultaneous tracks: Consumer IoT, Enterprise IoT, IoT Developer | WebRTC Convergence.

DevOps Summit also doubled from a single track in New York to two simultaneous tracks: "Dev" Developer Focus and "Ops" Operations Focus.

Schedule for Cloud Expo / Big Data Expo / @ThingsExpoHere

Schedule for DevOps SummitHere

Now that we have published the full conference schedule, please check back for daily updates as we finalize new session abstracts by working with our distinguished faculty members. For your questions please contact us at events (at) sys-con.com. Last but not least we will announce our keynotes on the hottest subjects to be delivered by world-class speakers!

The largest 'Internet of Things' event in the world has announced "sponsorship opportunities" and "call for papers."

The 1st International Internet of @ThingsExpo was launched this June at the Javits Center in New York City with over 6,000 delegates in attendance. The 2nd International Internet of @ThingsExpo will take place November 4-6, 2014, at the Santa Clara Convention Center in Santa Clara, California, with an estimated 7,000 plus delegates attending over three days.

@ThingsExpo is co-located with 15th International Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading IoT industry players in the world. In 2014, more than 200 companies will be present on the @ThingsExpo show floor, including global players and the hottest new technology pioneers.

Sponsorship and Exhibit Opportunities for @ThingsExpo Silicon Valley and New York Are Now Available
Sponsors and exhibitors of Internet of @ThingsExpo will benefit from unmatched branding, profile building and lead generation opportunities through:

  • Featured on-site presentation and ongoing on-demand webcast exposure to a captive audience of industry decision-makers.
  • Showcase exhibition during our new extended dedicated expo hours
  • Breakout Session Priority scheduling for sponsors that have been guaranteed a 35-minute technical session
  • Online advertising in SYS-CON's i-Technology publications
  • Capitalize on our comprehensive marketing efforts leading up to the show with print mailings, e-newsletters and extensive online media coverage.
  • Unprecedented PR Coverage: Editorial coverage on IoT.sys-con.com, Tweets to our 75,000 plus followers, press releases sent on major wire services to over 500 combined analysts and press members who attended Internet of @ThingsExpo - making it the best-covered "Internet of Things" conference in the world

For more information on sponsorship, exhibit, and keynote opportunities contact Carmen Gonzalez by email at events (at) sys-con.com, or by phone 201 802-3021. Book both events for additional savings!

@ThingsExpo Silicon Valley (November 4-6, 2014, Santa Clara, CA)
@ThingsExpo New York (June 9-11, 2015, New York, NY)

Secure Your VIP Pass to Attend @ThingsExpo Silicon Valley
Internet of @ThingsExpo announced today a limited time free "Expo Plus" registration option. The onsite registration price of $600 will be set to 'free' for delegates who register during September.

To take advantage of this opportunity, attendees can use the coupon code "IoTSeptember" and secure their "@ThingsExpo Plus" registration to attend all keynotes and general sessions, as well as a limited number of technical sessions each day of the show, in addition to full access to the expo floor and the @ThingsExpo hackathon.

The registration page is located at the @ThingsExpo site here.


@ThingsExpo New York 2015 'Call for Papers' Now Open
The 3rd International Internet of @ThingsExpo, to be held June 9-11, 2015, at the Javits Center in New York City, New York announces that its 'Call for Papers' is now open. The event will feature a world class, all-star faculty with the hottest IoT topics covered in three distinct tracks.

Track 1 - Consumer IoT and Wearables: Smart Appliances, Wearables, Smart Cars, Smartphones 2.0, Smart Travel, Personal Fitness, Health Care, Personalized Marketing, Customized Shopping, Personal Finance, The Digital Divide, Mobile Cash & Markets, Games & the IoT, The Future of Education, Virtual Reality

Track 2 - Enterprise IoT: The Business Case for IoT, Smart Grids, Smart Cities, Smart Transportation, The Smart Home, M2M, Authentication/Security, Wiring the IoT, The Internet of Everything, Digital Transformation of Enterprise IT, Agriculture, Transportation, Manufacturing, Local & State Government, Federal Government

Track 3 - Developer IoT: WebRTC, Eclipse Foundation, Cloud Foundry, Docker & Linux Containers, Node-Red, Open Source Hardware, Leveraging SOA, Multi-Cloud IoT, Evolving Standards, WebSockets, Security & Privacy Protocols, GPS & Proximity Services, Bluetooth/RFID/etc., XMPP, Nest Labs


@ThingsExpo billboard is viewed by more than 1.3 million motorists per week on Highway 101, in the heart of Silicon Valley

Help plant your flag in the fast-expanding business opportunity that is the Internet of Things: Submit your speaking proposal today here!

Download @ThingsExpo Newsletter Today Here

Chris Matthieu Named @ThingsExpo Tech Chair

Internet of @ThingsExpo named Chris Matthieu tech chair of Internet of @ThingsExpo 2014 Silicon Valley.

Chris Matthieu has two decades of telecom and web experience. He launched his Teleku cloud communications-as-a-service platform at eComm in 2010, which was acquired by Voxeo. Next he built an open source Node.JS PaaS called Nodester, which was acquired by AppFog. His latest startups include Twelephone. Leveraging HTML5 and WebRTC, Twelephone's BHAG (Big Hairy Audacious Goal) is to become the next generation telecom company running in the Web browser. Chris is currently co-founder and CTO of Octoblu.

Website: http://www.ThingsExpo.com

Twitter: http://www.Twitter.com/ThingsExpo

About SYS-CON Media & Events
SYS-CON Media (www.sys-con.com) has since 1994 been connecting technology companies and customers through a comprehensive content stream - featuring over forty focused subject areas, from Cloud Computing to Web Security - interwoven with market-leading full-scale conferences produced by SYS-CON Events. The company's internationally recognized brands include among others Cloud Expo® (CloudComputingExpo.com / @CloudExpo), Big Data Expo (BigDataExpo.net / @BigDataExpo), DevOps Summit (DevOpsSummit.sys-con.com / @DevOpsSummit), Internet of @ThingsExpo (ThingsExpo.com / @ThingsExpo) and Cloud Computing Bootcamp (CloudComputingBootcamp.com).

Cloud Expo® and Big Data Expo® are registered trademarks of Cloud Expo, Inc., a SYS-CON Events company.

More Stories By Trevor Parsons

Trevor Parsons is Chief Scientist and Co-founder of Logentries. Trevor has over 10 years experience in enterprise software and, in particular, has specialized in developing enterprise monitoring and performance tools for distributed systems. He is also a research fellow at the Performance Engineering Lab Research Group and was formerly a Scientist at the IBM Center for Advanced Studies. Trevor holds a PhD from University College Dublin, Ireland.

IoT & Smart Cities Stories
Druva is the global leader in Cloud Data Protection and Management, delivering the industry's first data management-as-a-service solution that aggregates data from endpoints, servers and cloud applications and leverages the public cloud to offer a single pane of glass to enable data protection, governance and intelligence-dramatically increasing the availability and visibility of business critical information, while reducing the risk, cost and complexity of managing and protecting it. Druva's...
At CloudEXPO Silicon Valley, June 24-26, 2019, Digital Transformation (DX) is a major focus with expanded DevOpsSUMMIT and FinTechEXPO programs within the DXWorldEXPO agenda. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term. A total of 88% of Fortune 500 companies from a generation ago are now out of business. Only 12% still survive. Similar percentages are found throug...
BMC has unmatched experience in IT management, supporting 92 of the Forbes Global 100, and earning recognition as an ITSM Gartner Magic Quadrant Leader for five years running. Our solutions offer speed, agility, and efficiency to tackle business challenges in the areas of service management, automation, operations, and the mainframe.
The Jevons Paradox suggests that when technological advances increase efficiency of a resource, it results in an overall increase in consumption. Writing on the increased use of coal as a result of technological improvements, 19th-century economist William Stanley Jevons found that these improvements led to the development of new ways to utilize coal. In his session at 19th Cloud Expo, Mark Thiele, Chief Strategy Officer for Apcera, compared the Jevons Paradox to modern-day enterprise IT, examin...
With 10 simultaneous tracks, keynotes, general sessions and targeted breakout classes, @CloudEXPO and DXWorldEXPO are two of the most important technology events of the year. Since its launch over eight years ago, @CloudEXPO and DXWorldEXPO have presented a rock star faculty as well as showcased hundreds of sponsors and exhibitors! In this blog post, we provide 7 tips on how, as part of our world-class faculty, you can deliver one of the most popular sessions at our events. But before reading...
DSR is a supplier of project management, consultancy services and IT solutions that increase effectiveness of a company's operations in the production sector. The company combines in-depth knowledge of international companies with expert knowledge utilising IT tools that support manufacturing and distribution processes. DSR ensures optimization and integration of internal processes which is necessary for companies to grow rapidly. The rapid growth is possible thanks, to specialized services an...
At CloudEXPO Silicon Valley, June 24-26, 2019, Digital Transformation (DX) is a major focus with expanded DevOpsSUMMIT and FinTechEXPO programs within the DXWorldEXPO agenda. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term. A total of 88% of Fortune 500 companies from a generation ago are now out of business. Only 12% still survive. Similar percentages are found throug...
There are many examples of disruption in consumer space – Uber disrupting the cab industry, Airbnb disrupting the hospitality industry and so on; but have you wondered who is disrupting support and operations? AISERA helps make businesses and customers successful by offering consumer-like user experience for support and operations. We have built the world’s first AI-driven IT / HR / Cloud / Customer Support and Operations solution.
Codete accelerates their clients growth through technological expertise and experience. Codite team works with organizations to meet the challenges that digitalization presents. Their clients include digital start-ups as well as established enterprises in the IT industry. To stay competitive in a highly innovative IT industry, strong R&D departments and bold spin-off initiatives is a must. Codete Data Science and Software Architects teams help corporate clients to stay up to date with the mod...
Scala Hosting is trusted by 50 000 customers from 120 countries and hosting 700 000+ websites. The company has local presence in the United States and Europe and runs an internal R&D department which focuses on changing the status quo in the web hosting industry. Imagine every website owner running their online business on a fully managed cloud VPS platform at an affordable price that's very close to the price of shared hosting. The efforts of the R&D department in the last 3 years made that pos...