Impact of Visual AI on Test Automation Shows 5.8x Faster Authoring, 5.9x Better Test Code Efficiency, and a 4.6x Increase in Test Stability

SAN MATEO, Calif., April 7, 2020 /PRNewswire/ -- Applitools, the leading provider of Visual AI powered test automation, today announced an industry report that reveals the impact of Visual AI on the performance of the most popular open source test frameworks - Cypress, Selenium, and WebdriverIO. Data was sourced from 288 quality engineers across 101 countries, spending an average of 11 hours during November 2019. This amounts to a combined 80 work weeks, or well over a year and a half of quality engineering hours, making it the largest industry study of its kind to date.

To download the "Impact of Visual AI on Test Automation Report" visit: (https://www.applitools.com/impact-of-visual-ai/)

"The Impact of Visual AI on Test Automation Report" provides important learning including:

• 5.8x Faster Test Creation: When using existing open source code-based frameworks in isolation, it took testers 7 hours to create test code in contrast to just 1.2 hours using Visual AI. As a result, testers are able to greatly expand test coverage while simultaneously releasing faster than ever before.
• 3.8x Increase in Test Code Stability: Participants used 32 locators and labels on average to provide test coverage using code based frameworks alone. This number dropped to just 7 with Visual AI. This increased stability greatly reduces test flakiness, broken CI/CD builds, and time spent on maintenance. Better test stability leads to efficiency gains along whatever dimension the quality engineering team values most - either more coverage, more time to manage quality, or even faster test cycles in support of CI/CD.
• 45% More Efficient at Catching Bugs: Participants caught 19 out of 20 bugs (95%) using Visual AI compared to just 13 (65%) when using code based frameworks in isolation. When test builds are faster, provide more coverage, and are more stable and easily maintained, testers catch more bugs pre-production while still delivering more features, faster than previously thought possible.
• Visual AI Helped Testers Succeed in Their Work When Used in Conjunction with Open Source Frameworks: All 288 testers that participated in the study achieved success rates of 79% when using their preferred code-based framework vs. 88% with Visual AI. This is a nine point jump in their performance after just a single hour of Visual AI training. Eric Davidson, Software Engineer from Chick-Fil-A comments, "This was an incredible opportunity to experience the power of Visual AI. Applitools has created a tool that easily integrates with existing frameworks and quickly augments functional test suites..."

To dive deeper into this study, register for the upcoming webinar on April 23, 2020, "The Impact of Visual AI on Selenium, Cypress, and WebdriverIO: Side-by-Side Test Code Comparison From 288 Quality Engineers" hosted by Raja Rao and James Lamberti of Applitools (https://go.applitools.com/200423-Hackathon-insights-webinar.html).

"Visual AI is a tremendous efficiency multiplier for automated testing," said Gil Sever, CEO of Applitools. "It improves the productivity of developers and test automation engineers more than 5X, delivers huge time and money savings in the app delivery cycle, removes the testing bottleneck in the CI/CD process, and helps deliver software applications with unprecedented visual and functional quality on any computer and mobile device."

"Testers need to look at Visual AI as a superpower. A virtual assistant that can access every screen and page in minutes, inspect it for changes and interpret every one of them the same way the human eye and brain would, but do so without ever getting tired or making any mistakes. We believe this capability is a major part of the future of test automation."

According to Gartner's Critical Capabilities for Software Test Automation (December 18, 2019), "61% of [its 2019 Software Quality Tools and Practices Survey] respondents said that AI/ML features would be very valuable in software testing tools. Improved defect detection (48%), reduction in test maintenance cost (42%) and improved test coverage (41%) were seen as the top benefits expected (multiple answers were allowed)." "The Impact of Visual AI on Test Automation Report" provides empirical evidence that Applitools Visual AI delivers exactly what the market expects from the addition of AI/ML into automated testing.

Data was sourced via the "Global Visual AI Rockstar Hackathon" which challenged software testers to create test suites for five real-world functional testing use-cases using their preferred code-based approach, including Selenium, Cypress, or WebdriverIO, and then repeat the process after adding Applitools Visual AI. This approach provided an unbiased, highly controlled environment from 288 quality engineers from 101 different countries during the 30 days ending November 30, 2019.

"Completing the hackathon was a keystone achievement in my career!" said Tracy Mazelin, QA Engineer at Paylocity. "I learned more by participating in this hackathon than any other automation instruction I've taken in the past number of years. I'm now 100 percent convinced that Visual AI testing is an essential tool for efficiently validating web and mobile software applications."

"This was the most interesting and useful event of the year in the field of test automation," said Viktar Silakou, Lead QA Automation Engineer at Globant S.A. "This allows you to take a look at test automation from a different point of view and gives an opportunity to radically improve your existing approaches."

About Applitools
Applitools enables Visual AI powered test automation to help teams release high-quality web and mobile apps faster and more efficiently.

Applitools Visual AI modernizes important test automation use cases -- Functional Testing, Visual Testing, Web and Mobile UI/UX Testing, Cross Browser Testing, Responsive Web Design Testing, Cross Device Testing, PDF Testing, Accessibility Testing and Compliance Testing -- to transform the way organizations deliver innovation at the speed of CI/CD at a significantly lower Total Cost of Ownership (TCO).

Hundreds of companies from verticals such as Software, Banking, Insurance, Retail, Pharmaceuticals, and Publishing -- including 50 of the Fortune 100 -- use Applitools to deliver the best possible digital experiences to millions of customers on any device and browser, and across every screen size and operating system.

Applitools is headquartered in San Mateo, California, with an R&D center in Tel Aviv, Israel. For more information, please visit applitools.com.

Media Contact:
Jeremy Douglas
Catapult PR-IR
+1 303-581-7760, ext. 16
[email protected]

SOURCE Applitools

Related Links

http://applitools.com