Harness today extended the reach of its core continuous integration/continuous delivery (CI/CD) platform to enable DevOps teams to build and deploy artificial intelligence (AI) agents.
At the core of that capability are five additions to the Harness portfolio, including an AI Asset Catalog that automatically discovers every agent, skill, and plugin that exists within repositories and, just as importantly, who within the organization owns them.
There is also a Harness AI Evals tool that measures the quality of an AI agent by creating gates that automatically catch regressions whenever an agent or model changes.
Additionally, an Agent Deployments offering extends the canary releases, approvals, and Open Policy Agent (OPA) guardrails that Harness already applies to Kubernetes deployments to managed agent runtimes like Amazon Bedrock AgentCore and Google’s Agent Runtime. AI Configs, meanwhile, provides access to a feature management tool to manage prompts and model changes at runtime.
At the same time, Harness has added a Harness AgentTrace tool to record what happens during a single agent run to show which path an agent took during a multi-step session. Harness also announced that it is open-sourcing the foundational components behind AgentTrace, including harness-sdk and harness-evals to enable developers to embed tracing primitives into their own AI applications.
Finally, Harness has added security capabilities that constrain what agents can do before they ship and scan for misconfigurations in agent skills, prompts, and models. An AI Bill of Materials (AIBOM) captures every model, tool, and dependency an agent has, while an AI Testing tool runs agents against adversarial inputs based on the OWASP Top 10 large language model (LLM) and Agentic AI risks framework.
There is also now an Agent Discovery and Posture Management tool that continuously maps agents as they spin up and connect across the organization and an AI Firewall that enforces policy in real time against prompt injection, tool misuse, and data exfiltration.
Trevor Stuart, senior vice president and general manager for Harness, said AI agents are essentially a new type of software artifact that DevOps teams need to be able to build and deploy much like any other application. The challenge is that AI agents are probabilistic, so scoring for accuracy, for example, requires a different approach to testing, said Stuart.
Ultimately, the underlying language model used by the AI agent decides how to complete a task, and the same agent, given the same input, can choose a different tool or take a different action from one run to the next. As a result, a test that passes once offers no guarantee it will pass the next time, noted Stuart.
More challenging still, incidents are not reproducible on demand, which means the current methodologies for catching and fixing bugs in software can’t be applied, he added. Harness already provides other tools that, for example, enable DevOps teams to apply policies to control token costs, noted Stuart.
Collectively, those capabilities provide the governance controls that organizations are going to need to safely deploy AI agents at scale, he added.
Regardless of the approach taken to building and deploying AI agents, what is clear is that there may soon be thousands of them. The challenge, as always, is keeping track of all the dependencies that exist between not just an AI agent and the infrastructure it depends on but all the relationships that continuously evolve between the AI agents themselves.