{"id":171,"date":"2024-03-17T08:58:57","date_gmt":"2024-03-17T08:58:57","guid":{"rendered":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/2024\/03\/17\/build-secure-ai-applications-on-azure-with-hashicorp-terraform-and-vault\/"},"modified":"2024-03-17T18:34:32","modified_gmt":"2024-03-17T18:34:32","slug":"build-secure-ai-applications-on-azure-with-hashicorp-terraform-and-vault","status":"publish","type":"post","link":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/2024\/03\/17\/build-secure-ai-applications-on-azure-with-hashicorp-terraform-and-vault\/","title":{"rendered":"Build secure AI applications on Azure with HashiCorp Terraform and Vault"},"content":{"rendered":"

HashiCorp and Microsoft have partnered to create Terraform modules<\/a> that follow Microsoft’s Azure Well-Architected Framework<\/a> and best practices. In a previous blog post, we demonstrated how to accelerate AI adoption on Azure with Terraform<\/a>. This post covers how to use a simple three-step process to build, secure, and enable OpenAI applications on Azure with HashiCorp Terraform and Vault. <\/p>\n

The code for this demo can be found on GitHub<\/a>. You can leverage the Microsoft application outlined in this post and the Microsoft Azure Kubernetes Service (AKS) to integrate with OpenAI. You can also read more about how to deploy an application that uses OpenAI on AKS<\/a> on the Microsoft website.<\/p>\n

Key considerations of AI<\/h2>\n

The rise in AI workloads is driving an expansion of cloud operations. Gartner predicts<\/a> that cloud infrastructure will grow 26.6% in 2024, as organizations deploying generative AI (GenAI) services look to the public cloud. To create a successful AI environment, orchestrating the seamless integration of artificial intelligence and operations demands a focus on security, efficiency, and cost control. <\/p>\n

Security<\/h3>\n

Data integration, the bedrock of AI, not only requires the harmonious assimilation of diverse data sources but must also include a process to safeguard sensitive information. In this complex landscape, the deployment of public key infrastructure (PKI)<\/a> and robust secrets management<\/a> becomes indispensable, adding cryptographic resilience to data transactions and ensuring the secure handling of sensitive information. For more information on the HashiCorp Vault solution, see our use-case page on Automated PKI infrastructure<\/a><\/p>\n

Machine learning models, pivotal in anomaly detection, predictive analytics, and root-cause analysis, not only provide operational efficiency but also serve as sentinels against potential security threats. Automation and orchestration, facilitated by tools like HashiCorp Terraform, extend beyond efficiency to become critical components in fortifying against security vulnerabilities. Scalability and performance, guided by resilient architectures and vigilant monitoring, ensure adaptability to evolving workloads without compromising on security protocols.<\/p>\n

Efficiency and cost control<\/h3>\n

In response, platform teams are increasingly adopting infrastructure as code (IaC) to enhance efficiency and help control cloud costs. HashiCorp products underpin some of today\u2019s largest AI workloads, using infrastructure as code to help eliminate idle resources and overprovisioning, and reduce infrastructure risk.<\/p>\n

Automation with Terraform<\/h3>\n

This post delves into specific Terraform configurations tailored for application deployment within a containerized environment. The first step looks at using IaC principles to deploy infrastructure to efficiently scale AI workloads, reduce manual intervention, and foster a more agile and collaborative AI development lifecycle on the Azure platform. The second step focuses on how to build security and compliance into an AI workflow. The final step shows how to manage application deployment on the newly created resources. <\/p>\n

Prerequisites<\/h2>\n

For this demo, you can use either Azure OpenAI service or OpenAI service:<\/p>\n

To use Azure OpenAI service, enable it on your Azure subscription using the Request Access to Azure OpenAI Service form<\/a>.
\nTo use OpenAI, sign up on the OpenAI website<\/a>.<\/p>\n

Step one: Build<\/h3>\n

First let’s look at the Helm provider<\/a> block in main.tf<\/a>:<\/p>\n

provider “helm” {
\n kubernetes {
\n host = azurerm_kubernetes_cluster.tf-ai-demo.kube_config.0.host
\n username = azurerm_kubernetes_cluster.tf-ai-demo.kube_config.0.username
\n password = azurerm_kubernetes_cluster.tf-ai-demo.kube_config.0.password
\n client_certificate = base64decode(azurerm_kubernetes_cluster.tf-ai-demo.kube_config.0.client_certificate)
\n client_key = base64decode(azurerm_kubernetes_cluster.tf-ai-demo.kube_config.0.client_key)
\n cluster_ca_certificate = base64decode(azurerm_kubernetes_cluster.tf-ai-demo.kube_config.0.cluster_ca_certificate)
\n }
\n}<\/p>\n

This code uses information from the AKS resource to populate the details in the Helm<\/a> provider, letting you deploy resources into AKS pods using native Helm charts. <\/p>\n

With this Helm chart method, you deploy multiple resources using Terraform in the helm_release.tf file. This file sets up HashiCorp Vault, cert-manager, and Traefik Labs\u2019 ingress controller within the pods. The Vault configuration shows the Helm set functionality to customize the deployment:<\/p>\n

resource “helm_release” “vault” {
\n name = “vault”
\n chart = “hashicorp\/vault”<\/p>\n

set {
\n name = “server.dev.enabled”
\n value = “true”
\n }
\n set {
\n name = “server.dev.devRootToken”
\n value = “AzureA!dem0”
\n }
\n set {
\n name = “ui.enabled”
\n value = “true”
\n }
\n set {
\n name = “ui.serviceType”
\n value = “LoadBalancer”
\n }
\nset {
\n name = “ui.serviceNodePort”
\n value = “null”
\n}
\n set {
\n name = “ui.externalPort”
\n value = “8200”
\n }
\n}<\/p>\n

In this demo, the Vault server is customized to be in Dev Mode, have a defined root token, and enable external access to the pod via a load balancer using a specific port.<\/p>\n

At this stage you should have created a resource group with an AKS cluster and servicebus established. The containerized environment should look like this:<\/p>\n

If you want to log in to the Vault server at this stage, use the EXTERNAL-IP load balancer address with port 8200 (like this: http:\/\/[EXTERNAL_IP]:8200\/<\/a>) and log in using AzureA!dem0.<\/p>\n

Step two: Secure<\/h3>\n

Now that you have established a base infrastructure in the cloud and the microservices environment, you are ready to configure Vault resources to integrate PKI into your environment. This centers around the pki_build.tf.second file, which you need to rename to remove the .second extension and make it executable as a Terraform file. After performing a terraform apply, as you are adding to the current infrastructure, add the elements to set up Vault with a root certificate and issue this within the pod.<\/p>\n

To do this, use the Vault provider<\/a> and configure it to define a mount point for the PKI, a root certificate, role cert URL, issuer, and policy needed to build the PKI:<\/p>\n

resource “vault_mount” “pki” {
\n path = “pki”
\n type = “pki”
\n description = “This is a PKI mount for the Azure AI demo.”<\/p>\n

default_lease_ttl_seconds = 86400
\n max_lease_ttl_seconds = 315360000
\n}<\/p>\n

resource “vault_pki_secret_backend_root_cert” “root_2023” {
\n backend = vault_mount.pki.path
\n type = “internal”
\n common_name = “example.com”
\n ttl = 315360000
\n issuer_name = “root-2023”
\n}<\/p>\n

Using the same Vault provider you can also configure Kubernetes authentication to create a role named “issuer” that binds the PKI policy with a Kubernetes service account named issuer:<\/p>\n

resource “vault_auth_backend” “kubernetes” {
\n type = “kubernetes”
\n}
\nresource “vault_kubernetes_auth_backend_config” “k8_auth_config” {
\n backend = vault_auth_backend.kubernetes.path
\n kubernetes_host = azurerm_kubernetes_cluster.tf-ai-demo.kube_config.0.host
\n}<\/p>\n

resource “vault_kubernetes_auth_backend_role” “k8_role” {
\n backend = vault_auth_backend.kubernetes.path
\n role_name = “issuer”
\n bound_service_account_names = [“issuer”]
\n bound_service_account_namespaces = [“default”,”cert-manager”]
\n token_policies = [“default”, “pki”]
\n token_ttl = 60
\n token_max_ttl = 120
\n}<\/p>\n

The role connects the Kubernetes service account, issuer, which is created in the default namespace with the PKI Vault policy. The tokens returned after authentication are valid for 60 minutes. The Kubernetes service account name, issuer, is created using the Kubernetes provider, discussed in step three, below. These resources are used to configure the model to use HashiCorp Vault to manage the PKI certification process.<\/p>\n

The image below shows how HashiCorp Vault interacts with cert-manager to issue certificates to be used by the application:<\/p>\n

Step three: Enable<\/h3>\n

The final stage requires another tf apply as you are again adding to the environment. You now use app_build.tf.third to build an application. To do this you need to rename app_build.tf.third to remove the .third extension and make it executable as a Terraform file. <\/p>\n

Interestingly, the code in app_build.tf uses the Kubernetes provider resource <\/a>kubernetes_manifest. The manifest values are the HCL (HashiCorp Configuration Language) representation of a Kubernetes YAML manifest. (We converted an existing manifest from YAML to HCL to get the code needed for this deployment. You can do this using Terraform\u2019s built-in yamldecode()<\/a> function or the HashiCorp tfk8s tool<\/a>.)<\/p>\n

The code below represents an example of a service manifest used to create a service on port 80 to allow access to the store-admin app that was converted using the tfk8s tool:<\/p>\n

resource “kubernetes_manifest” “service_tls_admin” {
\n manifest = {
\n “apiVersion” = “v1”
\n “kind” = “Service”
\n “metadata” = {
\n “name” = “tls-admin”
\n “namespace” = “default”
\n }
\n “spec” = {
\n “clusterIP” = “10.0.160.208”
\n “clusterIPs” = [
\n “10.0.160.208”,
\n ]
\n “internalTrafficPolicy” = “Cluster”
\n “ipFamilies” = [
\n “IPv4”,
\n ]
\n “ipFamilyPolicy” = “SingleStack”
\n “ports” = [
\n {
\n “name” = “tls-admin”
\n “port” = 80
\n “protocol” = “TCP”
\n “targetPort” = 8081
\n },
\n ]
\n “selector” = {
\n “app” = “store-admin”
\n }
\n “sessionAffinity” = “None”
\n “type” = “ClusterIP”
\n }
\n }
\n}<\/p>\n

Putting it all together<\/h3>\n

Once you\u2019ve deployed all the elements and applications, you use the certificate stored in a Kubernetes secret to apply the TLS configuration to inbound HTTPS traffic. In the example below, you associate “example-com-tls” \u2014 which includes the certificate created by Vault earlier \u2014 with the inbound IngressRoute deployment using the Terraform manifest:<\/p>\n

resource “kubernetes_manifest” “ingressroute_admin_ing” {
\n manifest = {
\n “apiVersion” = “traefik.containo.us\/v1alpha1”
\n “kind” = “IngressRoute”
\n “metadata” = {
\n “name” = “admin-ing”
\n “namespace” = “default”
\n }
\n “spec” = {
\n “entryPoints” = [
\n “websecure”,
\n ]
\n “routes” = [
\n {
\n “kind” = “Rule”
\n “match” = “Host(`admin.example.com`)”
\n “services” = [
\n {
\n “name” = “tls-admin”
\n “port” = 80
\n },
\n ]
\n },
\n ]
\n “tls” = {
\n “secretName” = “example-com-tls”
\n }
\n }
\n }
\n}<\/p>\n

To test access to the OpenAI store-admin site, you need a domain name. You use a FQDN<\/a> to access the site that you are going to protect using the generated certificate and HTTPS. <\/p>\n

To set this up, access your AKS cluster. The Kubernetes command-line client, kubectl<\/a>, is already installed in your Azure Cloud Shell. You enter:<\/p>\n

kubectl get svc<\/p>\n

And should get the following output:<\/p>\n

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
\nhello LoadBalancer 10.0.23.77 20.53.189.251 443:31506\/TCP 94s
\nkubernetes ClusterIP 10.0.0.1 443\/TCP 29h
\nmakeline-service ClusterIP 10.0.40.79 3001\/TCP 4h45m
\nmongodb ClusterIP 10.0.52.32 27017\/TCP 4h45m
\norder-service ClusterIP 10.0.130.203 3000\/TCP 4h45m
\nproduct-service ClusterIP 10.0.59.127 3002\/TCP 4h45m
\nrabbitmq ClusterIP 10.0.122.75 5672\/TCP,15672\/TCP 4h45m
\nstore-admin LoadBalancer 10.0.131.76 20.28.162.45 80:30683\/TCP 4h45m
\nstore-front LoadBalancer 10.0.214.72 20.28.162.47 80:32462\/TCP 4h45m
\ntraefik LoadBalancer 10.0.176.139 20.92.218.96 80:32240\/TCP,443:32703\/TCP 29h
\nvault ClusterIP 10.0.69.111 8200\/TCP,8201\/TCP 29h
\nvault-agent-injector-svc ClusterIP 10.0.31.52 443\/TCP 29h
\nvault-internal ClusterIP None 8200\/TCP,8201\/TCP 29h
\nvault-ui LoadBalancer 10.0.110.159 20.92.217.182 8200:32186\/TCP 29h<\/p>\n

Look for the traefik entry and note the EXTERNALl-IP (yours will be different than the one shown above). Then, on your local machine, create a localhost entry for admin.example.com to resolve to the address. For example on MacOS, you can use sudo nano \/etc\/hosts. If you need more help, search \u201ccreate localhost\u201d for your machine type.<\/p>\n

Now you can enter https:\/\/admin.example.com<\/a> in your browser and examine the certificate.<\/p>\n

This certificate is built from a root certificate authority (CA) held in Vault (example.com) and is valid against this issuer (admin.example.com) to allow for secure access over HTTPS. To verify the right certificate is being issued, expand the detail on our browser and view the cert name and serial number:<\/p>\n

You can then check this in Vault and see if the common name and serial numbers match.<\/p>\n

Terraform has configured all of the elements using the three-step approach shown in this post. To test the OpenAI application, follow Microsoft\u2019s instructions<\/a>. Skip to Step 4 and use https:\/\/admin.example.com<\/a> to access the store-admin and the original store-front load balancer address to access the store-front.<\/p>\n

DevOps for AI app development<\/h3>\n

To learn more and keep up with the latest trends in DevOps for AI app development, check out this Microsoft Reactor session with HashiCorp Co-Founder and CTO Armon Dadgar: Using DevOps and copilot to simplify and accelerate development of AI apps<\/a>. It covers how developers can use GitHub Copilot<\/a> with Terraform to create code modules for faster app development. You can get started by signing up for a free Terraform Cloud account<\/a>.<\/p>","protected":false},"excerpt":{"rendered":"

HashiCorp and Microsoft have partnered to create Terraform modules that follow Microsoft’s Azure Well-Architected Framework and best practices. In a […]<\/p>\n","protected":false},"author":1,"featured_media":158,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"site-sidebar-layout":"default","site-content-layout":"","ast-site-content-layout":"default","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","ast-disable-related-posts":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"footnotes":""},"categories":[6],"tags":[],"class_list":["post-171","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-terraform"],"_links":{"self":[{"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/posts\/171","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/comments?post=171"}],"version-history":[{"count":1,"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/posts\/171\/revisions"}],"predecessor-version":[{"id":244,"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/posts\/171\/revisions\/244"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/media\/158"}],"wp:attachment":[{"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/media?parent=171"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/categories?post=171"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/rssfeedtelegrambot.bnaya.co.il\/index.php\/wp-json\/wp\/v2\/tags?post=171"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}