A global survey of 820 IT decision makers and DevOps practitioners finds that half of respondents (53%) report that developers in the age of artificial intelligence (AI) are now authoring more tests directly.
Conducted by Perforce, that shift also appears to be enabling a similar percentage of organizations (55%) to provide quality assurance (QA) teams with more time to focus on analytics.
Perforce CTO Anjali Arora said it appears that organizations are investing more time and effort in testing to prevent suboptimal code, otherwise known as AI slop, from being incorporated into software builds.
That effort, in fact, also appears to be spurring more adoption of best DevSecOps practices, with 52% of respondents reporting their software development teams are embedding secure coding practices into the continuous integration/continuous delivery (CI/CD) platform. Half (50%) are also embedding security practices in code review, while 49% also extend security practices into runtime or production environments, the survey finds.
Similarly, 39% report fully automated compliance workflows, compared to 47% that have manual or partially automated processes.
Overall, the survey suggests that many organizations are moving to prevent software engineering from becoming a wild, wild west environment where best practices are largely ignored, noted Arora. In many cases, organizations are simply overconfident in their ability to extend existing workflows to manage what soon may be hundreds of bots accessing a wide range of tools and platforms that are evolving into headless services that AI agents invoke to perform various software engineering tasks, she added. More organizations will need to rely more on experienced software engineers to successfully manage those interactions, noted Arora.
It’s not clear how rapidly that transition is occurring, especially among the nearly half of respondents that are not reporting any changes to application development workflows. A total of 44% cite limited skills or training as the biggest barrier to adopting secure coding practices, followed by 39% that cited time pressure.
Overall, 63% of organizations report using hybrid test infrastructures that combine cloud scalability with on-premises control. In comparison, 16% report relying on primarily cloud-based platforms, while 14% exclusively use on-premises infrastructure.
Specific metrics tracked include lead time to validated release (39%), customer-reported defects post-release (33%) and defect escape rates (28%). Half (50%) also track improved customer retention or acquisition, faster delivery of new features (48%) and increased revenue or market share (44%).
Among respondents that work for organizations that have embedded AI into testing workflows, 44% use AI-powered tools integrated into existing platforms compared to 31% that are using AI-native tooling. Another 13% are relying on standalone or home-grown scripts.
Respondents are also paying more attention to costs, with 43% reporting comprehensive tracking of AI-powered testing costs, compared to 40% that are partially tracking costs, the survey finds.
There has always been a significant amount of discrepancy when it comes to assessing the DevOps maturity of organizations, especially as organizations try to strike a balance between the speed at which software is created and QA. However, as the pace of application development continues to increase, more organizations will need to improve. The challenge is that not all issues, including the technical debt generated, of course, are of equal weight.