|By Don MacVittie||
|December 4, 2012 07:45 AM EST||
Test results provided by vendors and “independent testing labs” often test for things that just don’t matter in the datacenter. Know what you’re getting.
When working in medicine, you don’t glance over a patient, then ask them “so how do you feel when you’re at your best?” You ask them what is wrong, then run a ton of tests – even if the patient thinks they know what is wrong – then let the evidence determine the best course of treatment.
Sadly, when determining the best tools for your IT staff to use, we rarely follow the same course. We invite a salesperson in, ask them “so, what do you do?”, and let them tell us with their snippets of reality or almost-reality why their product rocks the known world. Depending upon the salesperson and the company, their personal moral code or corporate standards could limit them to not bringing up the weak points of their products to outright lying about its capabilities.
“But Don!”, you say, “you’re being a bit extreme, aren’t you?” Not in my experience I am not. From being an enterprise architect to doing comparative reviews, I have seen it all. Vendor culture seems to infiltrate how employees interact with the world outside their HQ – or more likely (though I’ve never seen any research on it), vendors tend to hire to fit their culture, which ranges from straight-up truth about everything to wild claims that fall apart the first time the device is put into an actual production-level network.
The most common form of disinformation that is out there is to set up tests so they simply show the device operating at peak efficiency. This is viewed as almost normal by most vendors – why would you showcase your product in less than its best light? and as a necessary evil by most of the few who don’t have that view – every other vendor in the space is using this particular test metric, we’d better too or we’ll look bad. Historically, in network gear, nearly empty communications streams have been the standard for high connection rates, very large window sizes the standard for manipulating throughput rates. While there are many other games vendors in the space play to look better than they are, it is easy to claim you handle X million connections per second if those connections aren’t actually doing anything. It is also easier to claim you handle a throughput of Y Mbps if you set the window size larger than reality would ever see it.
Problem with this kind of testing is that it seeps into the blood, after a while, those test results start to be sold as actual… And then customers put the device into their network, and needless to say, they see nothing of the kind. You would be surprised the number of times when we were testing for Network Computing that a vendor downright failed to operate as expected when put into a live network, let alone met the numbers the vendor was telling customers about performance.
One of the reasons I came to F5 way back when was that they did not play these games. They were willing to make the marketing match the product and put a new item on the roadmap for things that weren’t as good as they wanted. We’re still out there today helping IT staff understand testing, and what testing will show relevant numbers to the real world. By way of example, there is the Testing Configuration Guide on F5 DevCentral.
As Application Delivery Controllers have matured and solidified, there has been much change in how they approach network traffic. This has created an area we are now starting to talk more about, which is the validity of throughput testing in regards to ADCs in general. The thing is, we’ve progressed to the point that simply “we can handle X Mbps!” is no longer a valid indication of the workloads an ADC will be required to handle in production scenarios. The real measure for application throughput that matters is requests per second. Vendors generally avoid this kind of testing, because response is also limited by the capacity of the server doing the actual responding, so it is easy to get artificially low numbers.
At this point in the evolution of the network, we have reached the reality of that piece of utility computing. Your network should be like electricity. You should be able to expect that it will be on, and that you will have enough throughput to handle incoming load. Mbps is like measuring amperage… When you don’t have enough, you’ll know it, but you should, generally speaking, have enough. It is time to focus more on what uses you put that bandwidth to, and how to conserve it. Switching to LED bulbs is like putting in an ADC that is provably able to improve app performance. LEDs use less electricity, the ADC reduces bandwidth usage… Except that throughput or packets per second isn’t measuring actual improvements of bandwidth usage. It’s more akin to turning off your lights after installing LED bulbs, and then saying “lookie how much electricity those new bulbs saved!”
Seriously, do you care if your ADC can delivery 20 million Megabits per second in throughput, or that it allows your servers to respond to requests in a timely manner? Seriously, the purpose of an Application Delivery Controller is to facilitate and accelerate the delivery of applications, which means responses to requests. If you’re implementing WAN Optimization functionality, throughput is still a very valid test. If you’re looking at the Application Delivery portion of the ADC though, it really has no basis in reality, because requests and responses are messy, not “as large a string of ones as I can cram through here”. From an application perspective – particularly from a web application perspective – there is a lot of “here’s a ton of HTML, hold on, sending images, wait, I have a video lookup…” Mbps or MBps just doesn’t measure the variety of most web applications. But users are going to feel requests per second just as much as testing will show positive or negative impacts. To cover the problem of application servers actually having a large impact on testing, do what you do with everything else in your environment, control for change. When evaluating ADCs, simply use the same application infrastructure and change only the ADC out. Then you are testing apples-to-apples, and the relative values of those test results will give you a gauge for how well a given ADC will perform in your environment.
In short, of course the ADC looks better if it isn’t doing anything. But ADCs do a ton in production networks, and that needs to be reflected in testing. If you’re fortunate enough to have time and equipment, get a test scheduled with your prospective vendor, and make sure that test is based upon the usage your actual network will expose the device to. If you are not, then identify your test scenarios to stress what’s most important to you, and insist that your vendor give you test results in those scenarios. In the end, you know your network far better than they ever will, and you know they’re at least not telling you the whole story, make sure you can get it.
Needless to say, this is a segue into the next segment of our #BareMetalBlog series, but first I’m going to finish educating myself about our use of FPGAs and finish that segment up.
In his keynote at @ThingsExpo, Chris Matthieu, Director of IoT Engineering at Citrix and co-founder and CTO of Octoblu, focused on building an IoT platform and company. He provided a behind-the-scenes look at Octoblu’s platform, business, and pivots along the way (including the Citrix acquisition of Octoblu).
Feb. 25, 2017 07:30 PM EST Reads: 1,797
You think you know what’s in your data. But do you? Most organizations are now aware of the business intelligence represented by their data. Data science stands to take this to a level you never thought of – literally. The techniques of data science, when used with the capabilities of Big Data technologies, can make connections you had not yet imagined, helping you discover new insights and ask new questions of your data. In his session at @ThingsExpo, Sarbjit Sarkaria, data science team lead ...
Feb. 25, 2017 07:00 PM EST Reads: 8,804
SYS-CON Events announced today that IoT Now has been named “Media Sponsor” of SYS-CON's 20th International Cloud Expo, which will take place on June 6–8, 2017, at the Javits Center in New York City, NY. IoT Now explores the evolving opportunities and challenges facing CSPs, and it passes on some lessons learned from those who have taken the first steps in next-gen IoT services.
Feb. 25, 2017 07:00 PM EST Reads: 1,669
SYS-CON Events announced today that WineSOFT will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Based in Seoul and Irvine, WineSOFT is an innovative software house focusing on internet infrastructure solutions. The venture started as a bootstrap start-up in 2010 by focusing on making the internet faster and more powerful. WineSOFT’s knowledge is based on the expertise of TCP/IP, VPN, SSL, peer-to-peer, mob...
Feb. 25, 2017 06:45 PM EST Reads: 1,932
The Internet of Things can drive efficiency for airlines and airports. In their session at @ThingsExpo, Shyam Varan Nath, Principal Architect with GE, and Sudip Majumder, senior director of development at Oracle, discussed the technical details of the connected airline baggage and related social media solutions. These IoT applications will enhance travelers' journey experience and drive efficiency for the airlines and the airports.
Feb. 25, 2017 06:30 PM EST Reads: 2,163
Big Data, cloud, analytics, contextual information, wearable tech, sensors, mobility, and WebRTC: together, these advances have created a perfect storm of technologies that are disrupting and transforming classic communications models and ecosystems. In his session at @ThingsExpo, Erik Perotti, Senior Manager of New Ventures on Plantronics’ Innovation team, provided an overview of this technological shift, including associated business and consumer communications impacts, and opportunities it m...
Feb. 25, 2017 06:00 PM EST Reads: 7,689
With billions of sensors deployed worldwide, the amount of machine-generated data will soon exceed what our networks can handle. But consumers and businesses will expect seamless experiences and real-time responsiveness. What does this mean for IoT devices and the infrastructure that supports them? More of the data will need to be handled at - or closer to - the devices themselves.
Feb. 25, 2017 05:45 PM EST Reads: 2,216
Things are changing so quickly in IoT that it would take a wizard to predict which ecosystem will gain the most traction. In order for IoT to reach its potential, smart devices must be able to work together. Today, there are a slew of interoperability standards being promoted by big names to make this happen: HomeKit, Brillo and Alljoyn. In his session at @ThingsExpo, Adam Justice, vice president and general manager of Grid Connect, will review what happens when smart devices don’t work togethe...
Feb. 25, 2017 05:00 PM EST Reads: 1,519
SYS-CON Events announced today that Dataloop.IO, an innovator in cloud IT-monitoring whose products help organizations save time and money, has been named “Bronze Sponsor” of SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Dataloop.IO is an emerging software company on the cutting edge of major IT-infrastructure trends including cloud computing and microservices. The company, founded in the UK but now based in San Fran...
Feb. 25, 2017 05:00 PM EST Reads: 2,917
A strange thing is happening along the way to the Internet of Things, namely far too many devices to work with and manage. It has become clear that we'll need much higher efficiency user experiences that can allow us to more easily and scalably work with the thousands of devices that will soon be in each of our lives. Enter the conversational interface revolution, combining bots we can literally talk with, gesture to, and even direct with our thoughts, with embedded artificial intelligence, whic...
Feb. 25, 2017 04:45 PM EST Reads: 1,736
In his session at @ThingsExpo, Sudarshan Krishnamurthi, a Senior Manager, Business Strategy, at Cisco Systems, will discuss how IT and operational technology (OT) work together, as opposed to being in separate siloes as once was traditional. Attendees will learn how to fully leverage the power of IoT in their organization by bringing the two sides together and bridging the communication gap. He will also look at what good leadership must entail in order to accomplish this, and how IT managers ca...
Feb. 25, 2017 04:15 PM EST Reads: 1,816
SYS-CON Events announced today that CA Technologies has been named “Platinum Sponsor” of SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY, and the 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. CA Technologies helps customers succeed in a future where every business – from apparel to energy – is being rewritten by software. From ...
Feb. 25, 2017 03:30 PM EST Reads: 2,492
In his keynote at 18th Cloud Expo, Andrew Keys, Co-Founder of ConsenSys Enterprise, provided an overview of the evolution of the Internet and the Database and the future of their combination – the Blockchain. Andrew Keys is Co-Founder of ConsenSys Enterprise. He comes to ConsenSys Enterprise with capital markets, technology and entrepreneurial experience. Previously, he worked for UBS investment bank in equities analysis. Later, he was responsible for the creation and distribution of life settle...
Feb. 25, 2017 03:30 PM EST Reads: 1,591
TechTarget storage websites are the best online information resource for news, tips and expert advice for the storage, backup and disaster recovery markets. By creating abundant, high-quality editorial content across more than 140 highly targeted technology-specific websites, TechTarget attracts and nurtures communities of technology buyers researching their companies' information technology needs. By understanding these buyers' content consumption behaviors, TechTarget creates the purchase inte...
Feb. 25, 2017 02:30 PM EST Reads: 1,749
The best way to leverage your Cloud Expo presence as a sponsor and exhibitor is to plan your news announcements around our events. The press covering Cloud Expo and @ThingsExpo will have access to these releases and will amplify your news announcements. More than two dozen Cloud companies either set deals at our shows or have announced their mergers and acquisitions at Cloud Expo. Product announcements during our show provide your company with the most reach through our targeted audiences.
Feb. 25, 2017 02:15 PM EST Reads: 2,305
SYS-CON Events announced today that Cloud Academy will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Cloud Academy is the industry’s most innovative, vendor-neutral cloud technology training platform. Cloud Academy provides continuous learning solutions for individuals and enterprise teams for Amazon Web Services, Microsoft Azure, Google Cloud Platform, and the most popular cloud computing technologies. Ge...
Feb. 25, 2017 02:15 PM EST Reads: 1,737
SYS-CON Events announced today that Fusion, a leading provider of cloud services, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Fusion, a leading provider of integrated cloud solutions to small, medium and large businesses, is the industry’s single source for the cloud. Fusion’s advanced, proprietary cloud service platform enables the integration of leading edge solutions in the cloud, including cloud...
Feb. 25, 2017 02:00 PM EST Reads: 4,058
Your homes and cars can be automated and self-serviced. Why can't your storage? From simply asking questions to analyze and troubleshoot your infrastructure, to provisioning storage with snapshots, recovery and replication, your wildest sci-fi dream has come true. In his session at @DevOpsSummit at 20th Cloud Expo, Dan Florea, Director of Product Management at Tintri, will provide a ChatOps demo where you can talk to your storage and manage it from anywhere, through Slack and similar services ...
Feb. 25, 2017 01:45 PM EST Reads: 2,037
20th Cloud Expo, taking place June 6-8, 2017, at the Javits Center in New York City, NY, will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud strategy.
Feb. 25, 2017 01:15 PM EST Reads: 2,225
Have you ever noticed how some IT people seem to lead successful, rewarding, and satisfying lives and careers, while others struggle? IT author and speaker Don Crawley uncovered the five principles that successful IT people use to build satisfying lives and careers and he shares them in this fast-paced, thought-provoking webinar. You'll learn the importance of striking a balance with technical skills and people skills, challenge your pre-existing ideas about IT customer service, and gain new in...
Feb. 25, 2017 12:45 PM EST Reads: 2,882