Buildkite Eradicates "Nightmare" Flaky Coding Tests for Developers With Revolutionary New Tool

Buildkite launches Test Analytics, an industry-first tool for identifying and eliminating problematic tests — saving developers and companies time, money and effort

MELBOURNE, Australia, June 2, 2022 /PRNewswire/ -- A game-changing new tool by Buildkite, an industry-leading CI/CD platform used by the likes of Shopify, Venmo, Wayfair and Pinterest, launches today, promising to fix one of the biggest and most expensive problems in software development — flaky tests.

Buildkite's new product Test Analytics speeds up delivery pipelines for DevOps engineers by eliminating waste in the development process. The tool works seamlessly with Buildkite's CI/CD platform and integrates with any CI system, including CircleCI, GitHub Actions and Jenkins, providing continuous performance monitoring and real-time insights for test suites. These insights help software development teams identify, optimize, and resolve flaky and unreliable tests faster than ever before, freeing up teams to focus on fixing bugs and rolling out new features.

According to a recent study, 59% of developers deal with flaky tests on a daily, weekly, or monthly basis. On Buildkite's platform alone, over 800 million seconds (25 years) of flaky tests are manually re-run by developers each month. Buildkite's CI/CD platform helps developers run software quickly and reliably, with testing and deployment tools that work for all software projects, whether for building mobile apps, backend applications, or custom hardware.

Tim Lucas, co-founder and co-CEO of Buildkite, said: "Flaky and unreliable tests are a nightmare for the developer world. There are three traditional ways of dealing with the issue. The first is to spend an inordinate amount of time and energy identifying the root cause of a flaky test. The second is to run your test repeatedly until it passes, wasting compute capacity and increasing costs. And the third is to ignore the failure altogether, which makes for unhappy developers and erodes trust in your entire testing and delivery process.

"With Test Analytics, developers can performance-tune their test suites to identify problems more quickly off the back of clear insight and analysis. Development teams not only have access to creating highly performant and stable build infrastructure, they now have the tools for making their own test suites faster and less flaky."

A clear dashboard within Test Analytics shows the slowest and least reliable tests, prioritizing the most critical fixes. Historical views and trend graphs provide further insight into diagnosing recurring problems. User-defined thresholds provide granularity in defining slow or unreliable tests and creating custom alerts. Integrations with commonly-used testing frameworks allow for deep inspection of where tests are spending time, from long-running database queries to API calls timing out.

For more information on Test Analytics, visit the company blog post here.

For visual assets surrounding Test Analytics, visit here.

Buildkite is the fastest and most secure way to test and deploy software at any scale. The company's continuous integration and continuous development (CI/CD) platform allows teams of all sizes to run fast, secure and scalable pipelines on their own infrastructure. Founded in 2013 by former developers Tim Lucas and Keith Pitt, the company was the first to introduce a self-hosted agent model and now supports over 1,000 customers across the world, including Canva, Intercom, Shopify, Venmo, Webflow, and more.

SOURCE Buildkite