Due to the sporadic nature of the QA environments, moving QA processes to cloud are an obvious choice for the organizations deciding to make a cloud journey. While cloud governance is not a new topic, still there are a very few articles that focus on moving QA processes to cloud.
Before we discuss the governance aspects, let me explain the two popular approaches to move QA processes to cloud – “Testing on cloud” and “Cloud testing”. Testing on a cloud is a form where the test environment is provisioned on the cloud. QA team still does the setup & tear down of the environment on the cloud. “Cloud Testing” is a form of SaaS for simulating the real world traffic by utilizing the cloud resources. In the later case, the SaaS provider takes care of interacting with the underlying IaaS/PaaS providers to provision the environment. Both the models require different governance considerations and pose different responsibilities on the business owners.
QA environments are different for the following reasons. At the same time, the delegation of responsibilities between the QA and the service provider depends on the the testing model chosen.
QA has to repeat the setup for different platforms, system configurations over different versions of the release. Moreover, the frequency of quickly setting up and tearing down the environment can also pose challenges. In case of “testing on cloud” model, this would require automation of deployments and configuration management to achieve efficiency.
Performance and scalability tests require a real world load to be simulated in the QA environment. A lack of control on the budget would result in a huge cost to the company. In case of a “testing on cloud” model, the business owners need to set controls on the spending on scale/performance tests.
Security Audits, Penetration Testing
Penetration tests happen as a part of security audits. If the QA environment is in house, then the security audits are much easier and well within the IT control. The moment the infrastructure is hosted on a cloud, the security audits and related penetration testing have limitations as well. Say in case of “testing on a cloud” model, the penetration testing can be done only to the level of application or operating system visibility. Any test that touches the infrastructure like penetrating the network needs contract discussions with the service provider. The hosting service providers might not allow a certain set of tests to be performed like SQL injections or DoS due to the multi-tenancy nature of the cloud. Most often the hosting service providers may have an in-house penetration testing team which can provide these test results. When it comes to “cloud testing” model, since it is a SaaS model, security auditing and controls are owned by the SaaS provider who in-turn can co-ordinate with the hosting service providers. Here is a great article from CISCO on the separation of responsibilities in case of on-prem or IaaS or PaaS or SaaS. SOASTA & Core Cloud Inspect are some of the popular SaaS penetration testing solutions available in the market.
Collaborative QA Teams
QA teams can be distributed across the globe or can be in-house and centralized. In either case, QA has to closely interact with the Dev team(s) to simulate a Dev environment or isolate the QA environment for debugging. This would also mean sharing the QA environments across multiple teams that perform different roles. While testing on cloud, the teams have to put in place appropriate access control and sharing mechanisms.
In all the above aspects, a “Cloud Testing” SaaS provider can offer solutions that can automate the configurations/deployments, budget the spending on the QA processes, own the security of the underlying infrastructure by co-coordinating with the hosting providers and provide access controls for the test resources. The businesses can delegate some of these governance considerations to the Cloud Testing SaaS providers and thus save cost and time.