We keep hearing about new solutions for test automation and continuous testing. Such solutions aim to increase the test automation authoring as well as the maintenance associated with these tests as the product evolves.
With this trend, many software quality engineers, SDET, and test automation architects are asking themselves whether their job is at risk, and what’s the future holds for them?
Prior to answering these questions, and re-profiling the modern testers role, let’s examine the terms ML/AI, the algorithms behind these tools, as well as the tools landscape as of today.
Artificial Intelligence
Artificial Intelligence: Sometimes called machine intelligence, is intelligence demonstrated by machines, in contrast to the natural intelligence displayed by humans and other animals. In computer science AI research is defined as the study of “intelligent agents”: any device that perceives its environment and takes actions that maximize its chance of successfully achieving its goals. The term “artificial intelligence” is applied when a machine mimics “cognitive” functions that humans associate with other human minds, such as “learning” and “problem solving”.
Machine Learning
Machine Learning: Is a subset of artificial intelligence in the field of computer science that often uses statistical techniques to give computers the ability to “learn” (i.e., progressively improve performance on a specific task) with data, without being explicitly programmed. In a recent blog post from Mabl, ML was also defined as “Machine learning is the process of continuously presenting a machine with a well defined data sample so that behavior can be developed“
Common Methods for Developing ML/AI
- Gradient Decent – Is a first-order iterative optimization algorithm for finding the minimum of a function. To find a local minimum of a function using gradient descent, one takes steps proportional to the negative of the gradient (or approximate gradient) of the function at the current point. If instead one takes steps proportional to the positive of the gradient, one approaches a local maximum of that function; the procedure is then known as gradient ascent.
- Convolutional Neural Networks – In machine learning, a convolutional neural network (CNN, or ConvNet) is a class of deep, feed-forward artificial neural networks, most commonly applied to analyzing visual imagery. The convolutional layer is the core building block of a CNN. The layer’s parameters consist of a set of learnable filters, that have a small receptive field, but extend through the full depth of the input volume. During the forward pass, each filter is convolved across the width and height of the input volume, computing the dot product between the entries of the filter and the input and producing a 2-dimensional activation map of that filter. As a result, the network learns filters that activate when it detects some specific type of feature at some spatial position in the input.
- Lookahead and Backchaining – are the leading models for decision making as part of machine learning.
- Backchaining is a technique used in teaching oral language skills, especially with polysyllabic or difficult words. The teacher pronounces the last syllable, the student repeats, and then the teacher continues, working backwards from the end of the word to the beginning. For example, to teach the name ‘Kinsbruner‘ a teacher will pronounce the last syllable: – ner, and have the student repeat it. Then the teacher will repeat it with ––bru– attached before: –bru-ner, after which all that remains is the first syllable: Kins-bru-ner
- The lookahead–based algorithms is used for induction of decision trees, allowing trade-off between tree quality and learning times.
- Forward-Backward Algorithm – In an inference algorithm that computes the posterior marginals of all hidden state variables given a sequence of observations. The algorithm makes use of the principle of dynamic programming to efficiently compute the values that are required to obtain the posterior marginal distributions in two passes. The first pass goes forward in time while the second goes backward in time; hence the name forward–backward algorithm. In the first pass, the forward–backward algorithm computes a set of forward probabilities which provide the probability of ending up in any particular state given the first observations in the sequence. In the second pass, the algorithm computes a set of backward probabilities which provide the probability of observing the remaining observations given any starting point. These two sets of probability distributions can then be combined to obtain the distribution over states at any specific point in time given the entire observation sequence
ML and AI Tools Landscape
In the mobile and desktop web testing landscape today, we see few vendors starting to operate.
Without drilling into each specific solution, here are the list of vendors that offer specific ML/AI set of solutions:
- Mabl – desktop web ML test automation solution that leverages a chrome add-on trainer to build code-less test automation
- Testim.IO – Desktop and Mobile Android code-less test automation solution that also uses a browser add-on to build robust test automation that aims to address the problem of dynamic objects
- TestCraft – web based code-less selenium test automation solution for testing web apps
- Applitools – visual test automation and monitoring for mobile and web apps
- Test.AI – ML based mobile native test automation
- Perfecto – Cloud based solution for testing mobile native apps and web based applications, offers AI based reporting with error classifications, and analytics to optimize the entire CI/CD workflows.
What Does all Of the Above Means to Modern Testers?
As mentioned above, there are plethora of tools evolving these days aiming to solve test authoring, analysis, and maintenance problems. While these are all awesome initiatives that will position testing higher and smarter in the overall DevOps processes, this does not translate into the extinction of the tester. Each of the above tools, as well as new tools that will rise are rising to help the existing testers to become more agile, smarter, and efficient.
AI and ML today mean the following to Test Engineers:
- A change in mind set – Aid Humans vs. Replacing Humans
- Training on modern ML/AI tools and techniques is required TODAY
- Use the tools to solve complex testing activities, don’t neglect existing testing methods, they are still relevant, not all are supported by ML/AI
- Keep humans in control of these tools, evolve productivity – become a DevOps champion by embracing new tools and lead innovation
- Modify working processes accordingly (Go/No GO criteria)
- AI/ML tools are solving specific rather than holistic problems – keep that in mind
- Match proper AI/ML tools to existing pains in test automation
