Click here to close now.

Welcome!

Virtualization Authors: Liz McMillan, Pat Romanski, Carmen Gonzalez, Elizabeth White, Ian Khan

Related Topics: Virtualization

Virtualization: Article

The Next Generation of Test Environment Management

Application-Behavior Virtualization

Traditional hardware and OS virtualization technology reduce software development/testing infrastructure costs and increase access to constrained systems. Yet, it's not always feasible to leverage hardware or OS virtualization for many large systems such as mainframes and ERPs. More pointedly, configuring and maintaining the environment and data needed to support development and test efforts still requires considerable time and resources. As a result, keeping a complex staged environment in synch with today's constantly-evolving Agile projects is a time-consuming, never-ending task.

Complementing traditional virtualization, Application-Behavior Virtualization (ABV) provides a new way for developers and testers to exercise their applications in incomplete, constantly evolving, and/or difficult-to-access environments. Rather than virtualizing entire applications and/or databases, Application-Behavior Virtualization focuses on virtualizing only the specific behavior that is exercised as developers and testers execute their core use cases. Beyond "service virtualization," it extends across all aspects of composite applications - services, mainframes, web and mobile device UIs, ERPs, ESB/JMS, legacy systems, and more.

This new breed of virtualization radically reduces the configuration time, hardware overhead, and data management efforts involved in standing up and managing a realistic and sustainable dev/test environment.

The Complexity of Quality
Today's complex, interdependent systems wreak havoc on parallel development and functional/performance testing efforts - significantly impacting productivity, quality, and project timelines. As systems become more complex and interdependent, development and quality efforts are further complicated by constraints that limit developer and tester access to realistic test environments. These constraints often include:

  • Missing/unstable components
  • Evolving development environments
  • Inaccessible third-party/partner systems and services
  • Systems that are too complex for test labs (mainframes or large ERPs)
  • Internal and external resources with multiple "owners"

The scope of what needs to be tested is increasing exponentially. With multiple new interfaces and ways for people to access core technology, systems and architectures have grown broader, larger, and more distributed - with multiple endpoints and access points. For example, you might have a thick client, a web browser, a device, and a mobile application all accessing the same critical component. Not surprisingly, testing in this environment has become very difficult and time consuming.

Furthermore, the number and range of people involved with software quality is rising. Advancements in development methodologies such as Agile are drawing more and more people into quality matters throughout the SDLC. For instance, business analysts are increasingly involved with user acceptance testing, QA has become responsible for a broader and more iterative quality cycle, and the development team is playing a more prominent role in the process of software quality and validation. Moreover, today's large distributed teams also exhibit a similar increase in team members involved with quality.

Also increasing are the permutations of moving parts - not only hardware and operating systems, but also client/server system upgrades, patches, and dependent third-party applications. As the service-oriented world broke apart many monolithic applications, service orientation also increased and distributed the number of connections and integration points involved in executing a business process.

Hardware and OS Virtualization Lowers Cost & Increases Access - Yet Significant Gaps Remain
In an attempt to provide all of the necessary team members ubiquitous access to realistic dev/test environments in light of these complexities, many organizations have turned to hardware and OS virtualization. Virtualizing the core test foundations - specific operating systems, configurations, platforms, etc. - has been a tremendous step forward for dev/test environment management. This virtualization provides considerable freedom from the live system, simultaneously reducing infrastructure costs and increasing access to certain types of systems. Moreover, leveraging the cloud in concert with virtualization provides a nearly unlimited bandwidth for scaling dependent systems.

Nevertheless, in terms of development or test environments, some significant gaps remain. First, some assets cannot be easily virtualized. For example, it's often unfeasible to leverage hardware or OS virtualization technology for large mainframe applications, third-party applications, or large ERPs.

Moreover, even when virtualization can be completed, you still need to configure and manage each one of those applications on top of the virtualized stack. Managing and maintaining the appropriate configuration and data integrity for all the dependent systems remains an ominous and time-consuming task. It is also a task that you will need some outside help with - you will inevitably be relying on other groups, such as operations or DevOps, to assist with at least certain aspects of the environment configuration and management.

Application-Behavior Virtualization reduces this configuration and data management overhead by enabling the developer or tester to rapidly isolate and virtualize just the behavior of the specific dependent components that they need to exercise in order to complete their end-to-end transactions. Rather than virtualizing entire systems, you virtualize only specific slices of dependent behavior critical to the execution of development and testing tasks.

It is completely feasible to use the cloud for scalability with Application-Behavior Virtualization. Nevertheless, since you're virtualizing only the specific behavior involved in dev/test transactions (not entire systems), the scope of what's being virtualized is diminished... and so is the need for significant incremental scalability.

What Is Application-Behavior Virtualization?
Application-Behavior Virtualization is a more focused and efficient strategy for eliminating the system and environment constraints that impede the team's ability to test their heterogeneous component-based applications. Instead of trying to virtualize the complete dependent component - the entire database, the entire third-party application, and so forth - you virtualize only the specific behavior that developers and testers actually need to exercise as they work on their particular applications, components, or scenarios.

For instance, instead of virtualizing an entire database (and performing all associated test data management as well as setting up the database for each test session), you monitor how the application interacts with the database, then you virtualize the related database behavior (the SQL queries that are passed to the database, the corresponding result sets that are returned, and so forth). This can then be accessed and adjusted as needed for different development and test scenarios.

To start, you designate which components you want to virtualize, then - as the application is exercised - the behavior of the associated transactions, messages, services, etc., is captured in what we call a "virtual asset." You can then configure this virtual asset by parameterizing its conditional behavior, performance criteria, and test data. This virtual asset can then emulate the actual behavior of the dependent system from that point forward, even if the live system is no longer accessible for development and testing.

Test data can be associated with these virtual assets, reducing the need for a dependent database and the need to configure and manage the dependent database that, if shared, usually gets corrupted.

By applying Application-Behavior Virtualization in this manner, you can remove the dependency on the actual live system/architecture while maintaining access to the dependent behavior. This ultra-focused approach significantly reduces the time and cost involved in managing multiple environments as well as complex test data management.

What Does Application-Behavior Virtualization Involve?
Application-Behavior Virtualization is achieved via the following phases:

  • Capture or model the real behavior of dependent systems
  • Configure the virtualized asset to meet demands of the test scenarios
  • Provision the virtualized asset for the appropriate team members or partners to access and test on their schedule

Phase 1: Capture
Real system behavior is captured using monitors to record live transaction details on the system under test by analyzing transaction logs, or by modeling behavior from a simple interface.

The intent here is to capture the behavior and performance of the dependent application for the system under test and leverage that behavior for development and testing efforts. This capturing can be done in three ways:

  1. If you have access to the live system, you can capture behavior by monitoring live system traffic. With a proxy monitoring traffic on the dependent system, the related messages are monitored, then the observed behavior is represented in a virtualized asset. This capturing can cover simple or composite behavior (e.g., a call to transfer funds in one endpoint can trigger an account balance update on another).
  2. If you want to emulate the behavior represented in transaction logs, virtual assets can be created by analyzing those logs. This is a more passive (and less politically volatile) approach to capturing the system behavior.
  3. If you're working in an environment that is evolving to include new functionality, you might want to model the behavior of the "not yet implemented" functionality within the Application-Behavior Virtualization interface. Leveraging the broad scope of protocol support available to facilitate modeling, you can rapidly build a virtual asset that emulates practically any anticipated behavior. For instance, you can visually model various message formats such as XML, JSON, and various legacy, financial, healthcare, and other domain-specific formats.

Phase 2: Configure
The virtualized asset's behavior can be fine-tuned, including performance, data source usage, and conditional response criteria.

After you use any of the three above methods to create a virtual asset, you can then instruct that asset to fine-tune or extend the behavior that it emulates. For instance, you can apply Quality of Service metrics so you can alter how you would like the asset to behave from the performance (timing, latency, and delay) perspective. You can also apply and modify test data for each particular asset to reproduce specific conditions critical for completing dev/test tasks. For example, you can configure various error and failure conditions that are difficult to reproduce or replicate with real systems. By adding data sources and providing conditional response criteria, you can tune the virtualized asset to perform as expected - or as unexpected (for negative testing).

Phase 3: Provision and Test
The environment is then provisioned for secure access across teams and business partners. The virtualized asset can then be leveraged for testing.

Once a virtualized asset is created, it can be provisioned for simplified uniform access across teams and business partners - either locally or globally (on a globally-accessible server, or in the cloud). They can then be used in unit, functional, and performance tests. Since virtual assets leverage a wide array of native protocols, they can be accessed for manual testing or automated testing by any test suite or any test framework, including Parasoft Test, HP Quality Center suite, IBM Rational Quality Management suite, and Oracle ATS. It is also easy to scale virtualized assets to support large-scale, high-throughput load and performance tests.

Even after the initial provisioning, these virtual assets are still easily modifiable and reusable to assist you in various dev/test scenarios. For instance, one of your test scenarios might access a particular virtual asset that applies a certain set of conditional responses. You can instantly construct an additional virtual asset that inherits those original conditions and then you can adjust them as needed to meet the needs of a similar test scenario.

How Application-Behavior Virtualization Speeds Testing and Cuts Costs: Three Common Use Cases
To conclude, let's look at how organizations have successfully applied Application-Behavior Virtualization to address dev/test environment management challenges in three common contexts:

  1. Performance/capacity-constrained environment
  2. Complex, difficult-to-access systems (mainframes, large ERPs, 3rd party systems)
  3. Parallel development (Agile or other iterative processes)

Performance/Capacity-Constrained Environments
Staged environments frequently lack the infrastructure bandwidth required to deliver realistic performance. Placing multiple virtualized applications on a single piece of hardware can increase access to a constrained resource, but the cost of this increased access is often degraded performance. Although the increased access could technically enable the execution of performance and load tests, the results typically would not reflect real-world behavior, significantly undermining the value of such testing efforts.

Application-Behavior Virtualization allows you to replicate realistic performance data independent of the live system. Once you create a virtual asset that captures the current performance, you can adjust the parameters to simulate more realistic performance. Performance tests can then run against the virtual asset (with realistic performance per the Quality of Service agreement) rather than the staged asset (with degraded performance).

Controlling the virtual asset's performance criteria is simply a matter of adjusting controls for timing, latency, and delay. In addition to simulating realistic behavior, this can also be used to instantly reproduce performance conditions that would otherwise be difficult to set up and control. For instance, you can simulate various levels of slow performance in a dependent component, then zero in on how your application component responds to such bottlenecks.

Even when it is possible to test against systems that are performing realistically, it is often not feasible to hit various components with the volume typical of effective load/stress tests. For example, you might need to validate how your application responds to extreme traffic volumes simulating peak conditions, but how do you proceed if your end-to-end transactions pass through a third-party service that charges per-transaction access fees?

If your performance tests pass through a component that you cannot (or do not want to) access under extreme load testing conditions, Application-Behavior Virtualization enables you to capture its behavior under a low-volume test (e.g., a single user transaction), adjust the captured performance criteria as desired, then perform all subsequent load testing against that virtualized component instead of the actual asset. In the event that the constrained component is not available for capture, you can create a virtual asset from scratch - using Application-Behavior Virtualization visual modeling interfaces to define its expected behavior and performance.

Complex, Difficult-to-Access Systems (Mainframes, Large ERPs, Third-Party Systems)
With large complex systems (mainframes, large ERPs, third-party systems), multiple development and test teams are commonly vying for limited system access for testing. Most of these systems are too complex for a test lab or a staged environment. To exercise end-to-end transactions involving these components, teams usually need to schedule (and pay for) access to a shared resource. This approach commonly causes test efforts to be delayed and/or prevents the team from performing the level and breadth of testing that they would like. For iterative development processes (e.g., Agile), the demand for frequent and immediate testing increases the severity of these delays and fees exponentially.

Even if organizations manage to use virtualization for these complex systems, proper configuration for the team's distinct testing needs would require a tremendous amount of work. And once that obstacle is overcome, another is right on its heels: developing and managing the necessary set of test data can also be overwhelming.

When teams use Application-Behavior Virtualization in such contexts, they only need to access the dependent resources long enough to capture the specific functionality related to the components and transactions they are working on. With this behavior captured in virtual assets, developers and testers can then access it continuously, allowing them to exercise end-to-end transactions at whatever time they want (without scheduling) and as frequently as they want (without incurring exorbitant transaction/access fees).

Parallel Development (Agile or Other Iterative Processes)
Even for simple applications, providing continued access to a realistic test environment can be challenging for teams engaged in parallel development (Agile or other iterative processes). A wide range of team members, including developers, testers, sometimes business analysts, all need easy access to a dev/test environment that is evolving in synch with their application. If the team decided to take the traditional virtualization route here, they would not only face all the initial setup overhead, but also be mired in constant work to ensure that the virtualized systems remain in step with the changes introduced in the latest iteration. When the team ends up waiting for access to dependent functionality, agility is stifled

Application-Behavior Virtualization reduces these constraints and associated delays by giving developers and testers the ability to rapidly emulate the needed behavior rather than having to wait for others to upgrade, configure, and manage the dependent systems. Even if anticipated functionality or components are not yet implemented, their behavior can be modeled rapidly, then deployed so team members can execute the necessary end-to-end transactions without delay, And if the dependent functionality recently changed, previously captured behavior can be easily modified either by re-capturing key transactions or by adjusting behavior settings in a graphical interface (without scripting or coding).

For example, many organizations are developing mobile applications, and this development is typically performed by a separate mobile development team. Since mobile applications commonly depend on core application components developed and maintained by other teams, the mobile team is often delayed as they wait for the other teams to complete work on the core components that their own mobile apps need to interact with. Application-Behavior Virtualization can eliminate these delays by allowing the mobile development team to emulate the behavior of the dependent components even if the actual components are incomplete, evolving, or otherwise difficult-to-access during the parallel development process.

Key Takeaways
Leveraging Application-Behavior Virtualization, teams reduce the complexity and the costs of managing multiple environments while providing ubiquitous access for development and test. Application-Behavior Virtualization helps you:

  • Reduce infrastructure costs
  • Improve provisioning/maintenance of test environments
  • Increase test coverage
  • Reduce defects
  • Improve predictability/control of software cycle times
  • Increase development productivity
  • Reduce third-party access fees

More Stories By Wayne Ariola

Wayne Ariola is Vice President of Strategy and Corporate Development at Parasoft, a leading provider of integrated software development management, quality lifecycle management, and dev/test environment management solutions. He leverages customer input and fosters partnerships with industry leaders to ensure that Parasoft solutions continuously evolve to support the ever-changing complexities of real-world business processes and systems. Ariola has more than 15 years of strategic consulting experience within the technology and software development industries. He holds a BA from the University of California at Santa Barbara and an MBA from Indiana University.

Comments (1) View Comments

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


Most Recent Comments
erikagd 07/29/11 02:11:00 PM EDT

For more information about Application Behavior Virtualization, please visit: www.parasoft.com/virtualize

@ThingsExpo Stories
The 4th International Internet of @ThingsExpo, co-located with the 17th International Cloud Expo - to be held November 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA - announces that its Call for Papers is open. The Internet of Things (IoT) is the biggest idea since the creation of the Worldwide Web more than 20 years ago.
There's no doubt that the Internet of Things is driving the next wave of innovation. Google has spent billions over the past few months vacuuming up companies that specialize in smart appliances and machine learning. Already, Philips light bulbs, Audi automobiles, and Samsung washers and dryers can communicate with and be controlled from mobile devices. To take advantage of the opportunities the Internet of Things brings to your business, you'll want to start preparing now.
With major technology companies and startups seriously embracing IoT strategies, now is the perfect time to attend @ThingsExpo in Silicon Valley. Learn what is going on, contribute to the discussions, and ensure that your enterprise is as "IoT-Ready" as it can be! Internet of @ThingsExpo, taking place Nov 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 17th Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The Internet of Things (IoT) is the most profound change in personal an...
P2P RTC will impact the landscape of communications, shifting from traditional telephony style communications models to OTT (Over-The-Top) cloud assisted & PaaS (Platform as a Service) communication services. The P2P shift will impact many areas of our lives, from mobile communication, human interactive web services, RTC and telephony infrastructure, user federation, security and privacy implications, business costs, and scalability. In his session at @ThingsExpo, Robin Raymond, Chief Architect at Hookflash, will walk through the shifting landscape of traditional telephone and voice services ...
The 17th International Cloud Expo has announced that its Call for Papers is open. 17th International Cloud Expo, to be held November 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA, brings together Cloud Computing, APM, APIs, Microservices, Security, Big Data, Internet of Things, DevOps and WebRTC to one location. With cloud computing driving a higher percentage of enterprise IT budgets every year, it becomes increasingly important to plant your flag in this fast-expanding business opportunity. Submit your speaking proposal today!
Explosive growth in connected devices. Enormous amounts of data for collection and analysis. Critical use of data for split-second decision making and actionable information. All three are factors in making the Internet of Things a reality. Yet, any one factor would have an IT organization pondering its infrastructure strategy. How should your organization enhance its IT framework to enable an Internet of Things implementation? In his session at Internet of @ThingsExpo, James Kirkland, Chief Architect for the Internet of Things and Intelligent Systems at Red Hat, described how to revolutioniz...
All major researchers estimate there will be tens of billions devices - computers, smartphones, tablets, and sensors - connected to the Internet by 2020. This number will continue to grow at a rapid pace for the next several decades. With major technology companies and startups seriously embracing IoT strategies, now is the perfect time to attend @ThingsExpo, June 9-11, 2015, at the Javits Center in New York City. Learn what is going on, contribute to the discussions, and ensure that your enterprise is as "IoT-Ready" as it can be
The security devil is always in the details of the attack: the ones you've endured, the ones you prepare yourself to fend off, and the ones that, you fear, will catch you completely unaware and defenseless. The Internet of Things (IoT) is nothing if not an endless proliferation of details. It's the vision of a world in which continuous Internet connectivity and addressability is embedded into a growing range of human artifacts, into the natural world, and even into our smartphones, appliances, and physical persons. In the IoT vision, every new "thing" - sensor, actuator, data source, data con...
Container frameworks, such as Docker, provide a variety of benefits, including density of deployment across infrastructure, convenience for application developers to push updates with low operational hand-holding, and a fairly well-defined deployment workflow that can be orchestrated. Container frameworks also enable a DevOps approach to application development by cleanly separating concerns between operations and development teams. But running multi-container, multi-server apps with containers is very hard. You have to learn five new and different technologies and best practices (libswarm, sy...
SYS-CON Events announced today that DragonGlass, an enterprise search platform, will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. After eleven years of designing and building custom applications, OpenCrowd has launched DragonGlass, a cloud-based platform that enables the development of search-based applications. These are a new breed of applications that utilize a search index as their backbone for data retrieval. They can easily adapt to new data sets and provide access to both structured and unstruc...
There's Big Data, then there's really Big Data from the Internet of Things. IoT is evolving to include many data possibilities like new types of event, log and network data. The volumes are enormous, generating tens of billions of logs per day, which raise data challenges. Early IoT deployments are relying heavily on both the cloud and managed service providers to navigate these challenges. In her session at Big Data Expo®, Hannah Smalltree, Director at Treasure Data, discussed how IoT, Big Data and deployments are processing massive data volumes from wearables, utilities and other machines...
Buzzword alert: Microservices and IoT at a DevOps conference? What could possibly go wrong? In this Power Panel at DevOps Summit, moderated by Jason Bloomberg, the leading expert on architecting agility for the enterprise and president of Intellyx, panelists will peel away the buzz and discuss the important architectural principles behind implementing IoT solutions for the enterprise. As remote IoT devices and sensors become increasingly intelligent, they become part of our distributed cloud environment, and we must architect and code accordingly. At the very least, you'll have no problem fil...
SYS-CON Events announced today that MetraTech, now part of Ericsson, has been named “Silver Sponsor” of SYS-CON's 16th International Cloud Expo®, which will take place on June 9–11, 2015, at the Javits Center in New York, NY. Ericsson is the driving force behind the Networked Society- a world leader in communications infrastructure, software and services. Some 40% of the world’s mobile traffic runs through networks Ericsson has supplied, serving more than 2.5 billion subscribers.
The worldwide cellular network will be the backbone of the future IoT, and the telecom industry is clamoring to get on board as more than just a data pipe. In his session at @ThingsExpo, Evan McGee, CTO of Ring Plus, Inc., discussed what service operators can offer that would benefit IoT entrepreneurs, inventors, and consumers. Evan McGee is the CTO of RingPlus, a leading innovative U.S. MVNO and wireless enabler. His focus is on combining web technologies with traditional telecom to create a new breed of unified communication that is easily accessible to the general consumer. With over a de...
Disruptive macro trends in technology are impacting and dramatically changing the "art of the possible" relative to supply chain management practices through the innovative use of IoT, cloud, machine learning and Big Data to enable connected ecosystems of engagement. Enterprise informatics can now move beyond point solutions that merely monitor the past and implement integrated enterprise fabrics that enable end-to-end supply chain visibility to improve customer service delivery and optimize supplier management. Learn about enterprise architecture strategies for designing connected systems tha...
Cloud is not a commodity. And no matter what you call it, computing doesn’t come out of the sky. It comes from physical hardware inside brick and mortar facilities connected by hundreds of miles of networking cable. And no two clouds are built the same way. SoftLayer gives you the highest performing cloud infrastructure available. One platform that takes data centers around the world that are full of the widest range of cloud computing options, and then integrates and automates everything. Join SoftLayer on June 9 at 16th Cloud Expo to learn about IBM Cloud's SoftLayer platform, explore se...
SYS-CON Media announced today that 9 out of 10 " most read" DevOps articles are published by @DevOpsSummit Blog. Launched in October 2014, @DevOpsSummit Blog offers top articles, news stories, and blog posts from the world's well-known experts and guarantees better exposure for its authors than any other publication. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time to wait for long development cycles that produce softw...
15th Cloud Expo, which took place Nov. 4-6, 2014, at the Santa Clara Convention Center in Santa Clara, CA, expanded the conference content of @ThingsExpo, Big Data Expo, and DevOps Summit to include two developer events. IBM held a Bluemix Developer Playground on November 5 and ElasticBox held a Hackathon on November 6. Both events took place on the expo floor. The Bluemix Developer Playground, for developers of all levels, highlighted the ease of use of Bluemix, its services and functionality and provide short-term introductory projects that developers can complete between sessions.
From telemedicine to smart cars, digital homes and industrial monitoring, the explosive growth of IoT has created exciting new business opportunities for real time calls and messaging. In his session at @ThingsExpo, Ivelin Ivanov, CEO and Co-Founder of Telestax, shared some of the new revenue sources that IoT created for Restcomm – the open source telephony platform from Telestax. Ivelin Ivanov is a technology entrepreneur who founded Mobicents, an Open Source VoIP Platform, to help create, deploy, and manage applications integrating voice, video and data. He is the co-founder of TeleStax, a...
The Internet of Things (IoT) promises to evolve the way the world does business; however, understanding how to apply it to your company can be a mystery. Most people struggle with understanding the potential business uses or tend to get caught up in the technology, resulting in solutions that fail to meet even minimum business goals. In his session at @ThingsExpo, Jesse Shiah, CEO / President / Co-Founder of AgilePoint Inc., showed what is needed to leverage the IoT to transform your business. He discussed opportunities and challenges ahead for the IoT from a market and technical point of vie...