This project costs $2.00 or more per hour to run!
A port of Google Cloud's Online Boutique to Azure, using Terraform, Azure Verified Modules where they fit, direct azurerm_* resources where they do not, ArgoCD for GitOps delivery, and SigNoz for monitoring and tracing.
- Private AKS cluster Kubernetes with Azure CNI Overlay networking and Workload Identity (OIDC)
- OpenAI - Customer chat bot for shopping recommendations
- VNet with 5 purpose-built subnets and per-subnet NSGs for network segmentation
- Azure Container Registry (Premium SKU, private endpoint)
- Azure Managed Redis (TLS, private endpoint) replacing the in-cluster Redis
- Application Gateway with AGIC for ingress
- TLS/HTTPS via cert-manager and Let's Encrypt with Azure DNS integration (optional, disabled by default because it requires a real domain name)
- Private DNS zones for all private endpoints
- Log Analytics and Container Insights for monitoring
- ArgoCD for GitOps-based application delivery
- Azure Bastion for secure cluster access
- SigNoz with OpenTelemetry for distributed tracing (optional)
The default deployment is AKS, App Gateway, ACR, Redis, ArgoCD, Bastion, private DNS, and the Azure GitOps overlay.
Optional on/off toggles are available for OpenAI, PostgreSQL, SigNoz, and CosmosDB
Sizes are demo-sized; I would use larger VMs, higher k8s limits, and more autoscaling if this were a production systems but it's designed and built to be a small system.
| Tool | Version | Purpose |
|---|---|---|
| Terraform | >= 1.9 | Infrastructure provisioning |
| Azure CLI | latest | Authentication and az aks command invoke |
| kubectl | >= 1.28 | Kustomize builds, offline tests |
- Microsoft Entra tenant and user/service principal access to the target Azure subscription. A subscription ID by itself is not enough; the identity running Terraform must exist in, or be invited into, the tenant that owns the subscription.
- Azure Subscription with billing enabled and quota available in your target region for AKS, Application Gateway v2, Bastion, Premium Redis, Public IPs, private endpoints, and Log Analytics.
- Azure RBAC permissions to create resource groups, AKS clusters, Redis, ACR, App Gateway, Bastion, managed identities, federated identity credentials, private endpoints, DNS records, and role assignments. In practice this usually means
Owner, orContributorplusUser Access Administrator, on the deployment scope. - Azure subscription ID — required in
terraform.tfvars. - Registered Azure resource providers in the subscription. At minimum, verify the providers used by this stack are registered before
terraform apply, includingMicrosoft.ContainerService,Microsoft.Network,Microsoft.ContainerRegistry,Microsoft.Cache,Microsoft.OperationalInsights,Microsoft.Insights,Microsoft.ManagedIdentity,Microsoft.Authorization, andMicrosoft.CognitiveServiceswhen Azure OpenAI is enabled. - Region support for every feature you plan to enable. Confirm your chosen region supports private AKS, Application Gateway v2, Bastion, Premium Redis, and Azure OpenAI if
enable_openai = true.
- Git repo URL — a public (or private + PAT) repo containing this codebase, so ArgoCD can sync from it over HTTPS.
- Outbound connectivity from the operator environment and the deployed platform to required public services such as Azure control-plane endpoints, Microsoft Entra token endpoints, GitHub/raw GitHub content, Helm chart repositories, and Let's Encrypt when TLS is enabled.
You'll also need a read-only GitHub PAT (or equivalent token) with repo access. If your GitHub organization uses SSO, the token must be SSO-authorized for the repo. Set these in terraform.tfvars:
gitops_repo_is_private = true
gitops_repo_token = "github_pat_..."Prerequisites:
- Control of the public DNS domain used by
tls_hostname. The domain must already exist and be hosted in Azure DNS. - Azure DNS zone already created for
tls_dns_zone_name. - Access to the DNS zone scope for the Terraform identity, plus permission for Terraform to create the cert-manager managed identity role assignment on that zone.
- A valid email address for Let's Encrypt registration.
Terraform can manage the public storefront Ingress, Azure DNS record, cert-manager bootstrap, and a Let's Encrypt certificate for you.
Where to put the TLS HCL:
- Copy
terraform.tfvars.exampletoterraform/azure/terraform.tfvars. - Put the TLS settings in that
terraform/azure/terraform.tfvarsfile alongside yoursubscription_idandgitops_repo_url.
Example terraform/azure/terraform.tfvars snippet:
enable_tls = true
tls_hostname = "shop.example.com"
tls_dns_zone_name = "example.com"
tls_dns_zone_resource_group_name = "shared-dns-rg"
tls_letsencrypt_email = "ops@example.com"
tls_letsencrypt_environment = "staging"
tls_manage_dns_record = trueRequired values when enable_tls = true:
enable_tls = truetls_hostnameThis is the public hostname users will browse to, for exampleshop.example.com.tls_dns_zone_nameThis must be the Azure DNS zone that contains the hostname, for exampleexample.com.tls_dns_zone_resource_group_nameThis is the resource group where that Azure DNS zone already exists.tls_letsencrypt_emailThis is the email address registered with Let's Encrypt for certificate issuance.cert_manager_manifest_sha256Required for supply-chain integrity checks during cert-manager install.
Optional or defaulted values:
tls_letsencrypt_environmentOptional. Defaults tostaging.staginguses Let's Encrypt's test environment, which is meant for proving that DNS, cert-manager, workload identity, and ACME challenge solving all work end to end. Certificates issued bystagingare not browser-trusted.productionuses the real Let's Encrypt service and issues browser-trusted certificates. Recommended sequence: start withstaging, runterraform apply, confirm certificate issuance and ingress routing work, then switch toproductionand apply again. This avoids burning real Let's Encrypt rate limits while you are still debugging the setup.tls_manage_dns_recordOptional. Defaults totrue. Whentrue, Terraform creates the Azure DNSArecord fortls_hostname. Whenfalse, you must create and maintain the DNS record yourself, pointing the hostname at the Application Gateway public IP.
Preconditions before terraform apply:
- An Azure DNS public zone matching
tls_dns_zone_name - Permission for the Terraform identity to read that DNS zone and create records there when
tls_manage_dns_record = true - A hostname where
tls_hostnameis either the zone apex or a subdomain oftls_dns_zone_name
Hostname relationship examples:
- Valid:
tls_hostname = "shop.example.com"withtls_dns_zone_name = "example.com" - Valid:
tls_hostname = "example.com"withtls_dns_zone_name = "example.com" - Invalid:
tls_hostname = "shop.other.com"withtls_dns_zone_name = "example.com"
What Terraform creates when TLS is enabled:
- installs cert-manager into the cluster
- creates a workload identity for cert-manager to solve Azure DNS challenges
- creates the Let's Encrypt
ClusterIssuer - creates the storefront
Certificate - manages the public
Ingress - optionally creates the public Azure DNS
Arecord
Recommended rollout order:
- Start with
tls_letsencrypt_environment = "staging". - Run
terraform apply. - Confirm certificate issuance and ingress routing work.
- Switch to
productionand apply again when you are ready for a browser-trusted certificate.
Notes:
- Start with
tls_letsencrypt_environment = "staging"until the full flow works end to end. - The DNS zone must already exist in Azure DNS. Terraform does not create the public DNS zone for you.
- Even if
tls_manage_dns_record = false, the DNS zone settings are still required by this configuration because cert-manager uses Azure DNS for the ACME DNS-01 challenge. - Terraform now owns the public Ingress so hostname and certificate wiring stay environment-specific and deterministic.
Prerequisites:
- Azure OpenAI access approved for the subscription and tenant.
- Model/deployment quota in the target region for the model this repo deploys.
Set enable_openai = true in terraform/azure/terraform.tfvars to provision the Azure OpenAI account, deployment, Kubernetes secret, workload identity service account, and the azure-openai GitOps overlay.
When enabled, Terraform keeps the default gitops_app_path input but automatically points ArgoCD at gitops/apps/online-boutique/overlays/azure-openai/. That overlay:
- enables the frontend assistant route
- deploys
shoppingassistantservice - uses the Azure OpenAI endpoint and deployment name from the Terraform-managed
openai-credentialsSecret - authenticates to Azure OpenAI with AKS workload identity bearer tokens instead of API keys
Current guardrails in shoppingassistantservice:
- rate limiting is set to 100 requests per 1-second window
- outbound Azure OpenAI calls are capped at 10 concurrent requests
These limits are currently hardcoded in the assistant service implementation.
Note: the auto-switch to the azure-openai overlay only happens when gitops_app_path is left at its default value. If you override gitops_app_path, you are responsible for pointing ArgoCD at the overlay you want.
Useful outputs:
terraform output openai_endpoint
terraform output openai_deployment_name
terraform output openai_workload_identity_client_idIf enable_cosmosdb = true, Terraform provisions a private Cosmos DB account and private endpoint.
Important scope note:
- This repo currently provisions the Azure Cosmos DB infrastructure only.
- The Online Boutique workloads are not reconfigured to consume Cosmos DB by the current GitOps overlays.
- The deployed application is not rewired to store cart or order data in Cosmos DB.
If enable_postgresql = true, Terraform provisions a private PostgreSQL Flexible Server, creates an admin password, stores that password in Azure Key Vault, and exposes retrieval outputs.
Required input when PostgreSQL is enabled:
keyvault_allowed_ipsmust include at least one public IP/CIDR, and it must include the operator IP runningterraform applyso Terraform can write the admin password secret into Key Vault.
Example:
enable_postgresql = true
keyvault_allowed_ips = ["203.0.113.42/32"] # replace with your public IP/CIDRRetrieve the admin credentials:
terraform output postgres_admin_username
eval "$(terraform output -raw postgres_admin_password_command)"Important scope note:
- This repo currently provisions the Azure PostgreSQL infrastructure and credentials only.
- The Online Boutique workloads are not reconfigured to consume PostgreSQL by the current GitOps overlays.
- The deployed application still uses Redis-backed
cartservicefor cart state. - Checkout and order flows are not rewired to persist order data into PostgreSQL.
Password lifecycle notes:
- Re-running
terraform applywithout replacing therandom_password.pg_adminresource keeps the same password. - The password is regenerated if the PostgreSQL feature is destroyed and recreated, if the
random_passwordresource is replaced, or if its generation settings change. - Setting
enable_postgresql = falsecauses Terraform to destroy the PostgreSQL server, the generated password resource, the Key Vault secret, and the PostgreSQL Key Vault created for that feature.
Set enable_signoz = true in terraform/azure/terraform.tfvars to deploy SigNoz onto the AKS cluster via Helm. This installs the SigNoz frontend (UI on port 3301), an OpenTelemetry Collector (gRPC on port 4317), ClickHouse (trace storage), and a query service.
When enabled, Terraform auto-switches ArgoCD to the azure-signoz overlay (or azure-signoz-openai if both enable_signoz and enable_openai are true). The overlay injects two env vars into all 11 microservice deployments:
COLLECTOR_SERVICE_ADDR=signoz-otel-collector.signoz:4317ENABLE_TRACING=1
Each service also gets a unique OTEL_SERVICE_NAME for trace identification. The loadgenerator's USERS is bumped from 10 to 25 for denser trace data.
All 11 services receive the tracing env vars, but only 7 have OTel SDK instrumentation that actively emits traces (frontend, checkoutservice, productcatalogservice, recommendationservice, currencyservice, emailservice, paymentservice). The other 4 (shippingservice, cartservice, adservice, loadgenerator) receive the vars but do not act on them.
If both SigNoz and Azure OpenAI are enabled, the assistant deployment also receives tracing env vars through the combined overlay. That does not fully instrument the assistant service by itself; the current Python implementation is not yet emitting OpenTelemetry spans.
As with enable_openai, the automatic overlay switch only applies when gitops_app_path remains at its default.
Access the SigNoz UI:
eval "$(terraform output -raw signoz_port_forward_command)"
# Browse to http://localhost:3301Useful outputs:
terraform output signoz_port_forward_command
terraform output signoz_otel_collector_addrPrepare your Terraform inputs:
cd terraform/azure
cp terraform.tfvars.example terraform.tfvars
# Edit terraform.tfvars — set required variables for your deployment mode
# Optional: add the TLS variables to provision HTTPS with Let's Encrypt
# Optional: set enable_openai = true to sync the Azure OpenAI assistant overlayMinimum base deployment example:
subscription_id = "00000000-0000-0000-0000-000000000000"
gitops_repo_url = "https://github.com/your-org/azure-microservices-demo"
gitops_repo_revision = "main" # required — pin to a tag or commit SHA for production
argocd_manifest_sha256 = "9246f6725ada1399ec0550a0ceaf0bccaf872ed151616025786cc0f9edf6c89c" # required
# Optional base overrides
# location = "eastus2"Required variables for a base deployment (enable_tls = false):
subscription_idgitops_repo_urlgitops_repo_revisionargocd_manifest_sha256
Additional required variable when enable_tls = true:
cert_manager_manifest_sha256
Verify your Azure CLI context before you create anything:
az login
az account set --subscription <your-subscription-id>
az account show --query '{tenantId:tenantId, subscriptionId:id, name:name}' -o yamlRecommended preflight checks:
# Replace with the same location you set in terraform.tfvars.
LOCATION=eastus2
# Confirm the required resource providers are registered
az provider show --namespace Microsoft.ContainerService --query registrationState -o tsv
az provider show --namespace Microsoft.Network --query registrationState -o tsv
az provider show --namespace Microsoft.ContainerRegistry --query registrationState -o tsv
az provider show --namespace Microsoft.Cache --query registrationState -o tsv
az provider show --namespace Microsoft.ManagedIdentity --query registrationState -o tsv
# Confirm the target region supports the features you plan to enable
az account list-locations --query "[?name=='${LOCATION}'].[name,displayName]" -o table
# Check quota before apply, especially in new or constrained subscriptions
az vm list-usage --location "${LOCATION}" -o tableIf you plan to enable optional features, verify those prerequisites too before terraform apply:
enable_openai = true: confirm Azure OpenAI access approval, regional model availability, and quota.enable_tls = true: confirm the Azure DNS zone already exists and your identity can create role assignments and records on that zone.
Deploy:
terraform init
terraform applyDeployment takes ~15 minutes but may run for up to 45 due to IAM role propagation and Ingress spin-up time. Terraform waits for ArgoCD to sync and the frontend to become reachable.
- Most bootstrap and inspection steps in this repo use
az aks command invoke, so they work without direct kube-apiserver network access as long as your Azure CLI identity can operate on the cluster. - Commands that use
az aks get-credentialspluskubectl port-forwardneed network reachability to the private AKS API server from the machine where you run them. - The outputs in
terraform outputmix both models. Read the output description before assuming a command will work from an arbitrary laptop or CI runner.
By default, the Application Gateway accepts HTTP/HTTPS traffic from the entire internet (allowed_ingress_source = "Internet"). To lock it down so only your machine can reach the storefront, set allowed_ingress_source to your public IP in terraform.tfvars:
allowed_ingress_source = "203.0.113.42/32" # replace with your public IPFind your public IP:
curl -s ifconfig.meThis controls the NSG rules on the Application Gateway subnet — only the specified CIDR is allowed inbound on ports 80 and 443. All other external traffic is denied.
The in-cluster loadgenerator (traffic simulation) is not affected by this setting. It talks directly to the frontend Kubernetes Service over the cluster network (frontend:80) and never routes through the Application Gateway or its public IP.
To allow multiple sources, use a summary CIDR that covers them, or widen the range as needed (e.g. 203.0.113.0/24). The variable accepts a single address prefix — if you need multiple disjoint CIDRs, you would need to add additional NSG rules to the Terraform configuration.
Base deployment:
# Browse the storefront
terraform output frontend_url
# Check pod status
eval "$(terraform output -raw get_pods_command)"
# Check ArgoCD sync status
eval "$(terraform output -raw argocd_sync_status_command)"If TLS is enabled:
terraform output frontend_hostname
eval "$(terraform output -raw certificate_status_command)"If Azure OpenAI is enabled:
- Confirm the storefront assistant UI appears.
- Ask the assistant for a product recommendation and verify it returns product IDs plus linked product cards.
If SigNoz is enabled:
eval "$(terraform output -raw signoz_port_forward_command)"
# Browse to http://localhost:3301./tests/test-offline.sh # offline only (default)
./tests/test-offline.sh --help # show all optionsThe test suite has three tiers:
Offline (default) — no network or credentials needed. Validates Terraform formatting, Kustomize builds, manifest correctness (no :latest tags, no redis-cart resources, overlay/service/serviceAccount cross-references), and Terraform code quality (pinned providers, AGIC lifecycle rules, TLS resources, no committed secrets).
Terraform online — downloads providers from the Terraform registry to run terraform init and terraform validate. Requires internet access but no Azure credentials.
RUN_TERRAFORM_ONLINE_TESTS=1 ./tests/test-offline.shLive Azure and AKS — queries the Azure control plane and the running AKS cluster to verify the deployed infrastructure matches expectations: cluster state, AGIC add-on, node pool autoscaling, private endpoints, RBAC role assignments, ArgoCD sync status, pod health, and live ingress routing. Requires:
az loginwith read access to the deployed subscription- A completed
terraform apply(or manually provided resource names)
RUN_AZ_ONLINE_TESTS=1 ./tests/test-offline.shThe script infers resource names from Terraform outputs automatically. If outputs aren't available locally, provide them explicitly:
RUN_AZ_ONLINE_TESTS=1 \
AZ_TEST_RESOURCE_GROUP=<resource-group> \
AZ_TEST_AKS_CLUSTER=<aks-name> \
./tests/test-offline.shAdditional overrides: AZ_TEST_APP_GATEWAY, AZ_TEST_ACR_NAME.
terraform destroyThe ArgoCD Application has a resources-finalizer that cascade-deletes all managed K8s resources automatically.
| Item | Details |
|---|---|
| SigNoz Remaining Instrumentation | Add OTlel to the following services: shippingservice, cartservice, adservice, loadgenerator |
| SigNoz Host Metrics | Gather per-host metrics |
| k8s NetworkPolicy | Change to active / enforcing |
| Terraform Workflow | Build a real one, just laptop based right now |
| infracost | Add this tool to the pipeline |
| Images from ACR not Google | Currently pulling from Google's container registry |