New Relic has made available an open source extension to its observability platform for coding tools at no additional cost.
Nic Benders, chief technical strategist for New Relic, said the New Relic AI Coding Observability capability will make it simpler for DevOps teams to centrally monitor usage of a diverse range of artificial intelligence (AI) coding tools, including the cost of the tokens consumed, using the same platform they already have to observe IT operations.
Regardless of the type of AI coding tool employed, New Relic AI Coding Observability normalizes the data collected, he added.
That capability makes it possible for organizations to employ multiple AI coding tools as they best see fit, or swap one out for another as additional advances are made, noted Benders.
The arrival of New Relic AI Coding Observability coincides with the sharing of a survey of 200 technology decision makers in the U.S. that finds a full 94% rate code generated by AI as being of a higher quality than human-authored code at the time of review, with 67% reporting AI now generates or significantly refactors between 51% and 75% of their organization’s weekly code output.
Nearly two-thirds (62%) also said their engineering teams often trust AI-generated code enough to ship it to production without line-by-line manual verification. A full 88% also lead organizations that made vibe coding a part of their workflows, with only 5% restricting it to non-production environments, and no respondents banning the practice outright.
However, more than three quarters (78%) also report there has been an increase in incidents. In fact, 86% report an increase in time senior staff spends fixing code and 74% report at least 25% of AI code needs significant rework when considering the past 12 months. Eighty-two percent experienced at least one production failure tied to AI-generated code in the past six months compared to 19% reporting no AI-generated code challenges in this time period.
A full 96% also rate observability as very or extremely important when working with AI-generated code. Well over three quarters (78%) now routinely prompt AI tools to include specific telemetry—such as logs, traces and metrics—directly into generated code to ensure it is observable.
Collectively, the results suggest that IT leaders are overestimating the quality of the code finding its way into their production environments, noted Benders. Many of those organizations are also about to discover that maintaining code over an extended period of time is a lot harder than writing code, he added. After all, when there is an issue the only source of information will be the AI tool that wrote it, which may not always surface a reliable explanation for why it opted for one approach or another, said Benders.
As such, it’s no longer a question of whether organizations will need to revisit how they manage the software development lifecycle (SDLC), but to what degree, he noted.
Regardless of approach, software engineering in the AI era is never going to be the same again. The issue now is how best to observe and manage massive volumes of code moving through DevOps pipelines that is of uncertain quality and, often, provenance.