Azure (AKS)

The layout

Hospital ──ExpressRoute / VPN──▶ VNet ──▶ internal Azure Load Balancer (private IP):7890
                                               │
                         AKS (zones 1, 2, 3)   ▼
                         hl7-edge-gateway × 2  (spread across zones)  ──HTTPS──▶ your API
                         NATS JetStream × 3    (one per zone, Premium SSD)
                         images from Azure Container Registry

The three cloud examples share one Kubernetes base. The gateway is a stateless Deployment of two pods; NATS is a StatefulSet from the official chart, on persistent disks. The base is tested on k3s by deleting a gateway pod and a NATS pod while messages flow. This example adds what's specific to Azure.

The sample is in 07-azure-aks in the examples repository.

Diagrams

The shared base

shared/kubernetes/hl7-edge-gateway.toml — a ConfigMap; every pod's node ID is its pod name

# HL7 Edge Gateway on Kubernetes: the same file for every pod (a ConfigMap). The node ID is the pod
# name; secrets come from the Secret hl7edge-secrets as environment variables.

node_id = "${HL7EDGE_NODE_ID}"

[admin]
listen = "0.0.0.0:8080"
token = "${HL7EDGE_ADMIN_TOKEN}"

[store]
backend = "jetstream"
urls = ["nats://nats.hl7edge.svc.cluster.local:4222"]   # the NATS chart's client Service
token = "${NATS_TOKEN}"
replicas = 3
ack_wait = "30s"

[[route]]
name = "lab-results"
mode = "store_and_forward"
request_timeout = "15s"
source = { type = "mllp", listen = "0.0.0.0:7890" }

[route.destination]
type = "http"
url = "${LAB_API_URL}"
headers = { Authorization = "Bearer ${LAB_API_TOKEN}" }

[route.store_and_forward]
on_reject = "dead_letter"
retry = { initial = "2s", max = "60s", multiplier = 2.0 }

shared/kubernetes/gateway.yaml

# HL7 Edge Gateway: two stateless pods (the queue is in NATS JetStream), spread across zones and
# nodes, behind one MLLP Service. The cloud folders patch the Service (internal load balancer) and
# the image (your registry).
apiVersion: apps/v1
kind: Deployment
metadata:
  name: hl7-edge-gateway
  labels: { app: hl7-edge-gateway }
spec:
  replicas: 2
  selector:
    matchLabels: { app: hl7-edge-gateway }
  template:
    metadata:
      labels: { app: hl7-edge-gateway }
      annotations:
        prometheus.io/scrape: "true"
        prometheus.io/port: "8080"
    spec:
      securityContext:
        runAsNonRoot: true
        runAsUser: 10001
        fsGroup: 10001
        seccompProfile: { type: RuntimeDefault }
      terminationGracePeriodSeconds: 30
      containers:
        - name: gateway
          image: lambogenius/hl7-edge-gateway:0.3.0
          args: ["run", "--config", "/etc/hl7-edge-gateway/hl7-edge-gateway.toml"]
          ports:
            - { name: mllp, containerPort: 7890 }
            - { name: admin, containerPort: 8080 }
          env:
            - name: HL7EDGE_NODE_ID
              valueFrom: { fieldRef: { fieldPath: metadata.name } }
          envFrom:
            - secretRef: { name: hl7edge-secrets }
          startupProbe:            # NATS may still be starting: allow up to 5 minutes before liveness applies
            httpGet: { path: /health, port: admin }
            periodSeconds: 5
            failureThreshold: 60
          readinessProbe:          # ready = config loaded and JetStream reachable
            httpGet: { path: /ready, port: admin }
            periodSeconds: 2
            failureThreshold: 2
          livenessProbe:
            httpGet: { path: /health, port: admin }
            periodSeconds: 10
            failureThreshold: 3
          resources:
            requests: { cpu: 100m, memory: 128Mi }
            limits: { memory: 512Mi }
          securityContext:
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            capabilities: { drop: [ALL] }
          volumeMounts:
            - { name: config, mountPath: /etc/hl7-edge-gateway, readOnly: true }
            - { name: tmp, mountPath: /tmp }
      volumes:
        - name: config
          configMap: { name: hl7edge-config }
        - name: tmp
          emptyDir: {}
      topologySpreadConstraints:
        - maxSkew: 1
          topologyKey: topology.kubernetes.io/zone
          whenUnsatisfiable: ScheduleAnyway
          labelSelector: { matchLabels: { app: hl7-edge-gateway } }
        - maxSkew: 1
          topologyKey: kubernetes.io/hostname
          whenUnsatisfiable: ScheduleAnyway
          labelSelector: { matchLabels: { app: hl7-edge-gateway } }
---
# MEDITECH connects here. Both pods serve every route (active/active); a pod that is not ready gets
# no new connections.
apiVersion: v1
kind: Service
metadata:
  name: hl7edge-mllp
  labels: { app: hl7-edge-gateway }
spec:
  type: LoadBalancer
  selector: { app: hl7-edge-gateway }
  ports:
    - { name: lab-results, port: 7890, targetPort: mllp }
---
# Admin console and API, inside the cluster only: kubectl -n hl7edge port-forward svc/hl7edge-admin 8080
apiVersion: v1
kind: Service
metadata:
  name: hl7edge-admin
  labels: { app: hl7-edge-gateway }
spec:
  type: ClusterIP
  selector: { app: hl7-edge-gateway }
  ports:
    - { name: admin, port: 8080, targetPort: admin }
---
# Keep one gateway running during node drains and cluster upgrades.
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: hl7-edge-gateway
spec:
  minAvailable: 1
  selector:
    matchLabels: { app: hl7-edge-gateway }

shared/kubernetes/nats-values.yaml

# Values for the official NATS Helm chart (https://github.com/nats-io/k8s), release name "nats":
#   helm upgrade --install nats nats/nats --version 2.15.1 -n hl7edge -f nats-values.yaml [-f <cloud>-nats.yaml]
# A 3-node JetStream cluster on persistent disks, one pod per zone, with token auth.
config:
  cluster:
    enabled: true
    replicas: 3
  jetstream:
    enabled: true
    fileStore:
      pvc:
        size: 20Gi
        # storageClassName: set per cloud
  merge:
    authorization:
      token: << $NATS_TOKEN >>
container:
  env:
    NATS_TOKEN:
      valueFrom:
        secretKeyRef: { name: hl7edge-secrets, key: NATS_TOKEN }
podTemplate:
  topologySpreadConstraints:
    kubernetes.io/hostname:
      maxSkew: 1
      whenUnsatisfiable: DoNotSchedule
    topology.kubernetes.io/zone:
      maxSkew: 1
      whenUnsatisfiable: ScheduleAnyway
natsBox:
  enabled: false

What Azure adds

07-azure-aks/kustomization.yaml

# Sample 07: the shared base on AKS. setup.sh fills in your registry.
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: hl7edge
resources:
  - ../shared/kubernetes
images:
  - name: lambogenius/hl7-edge-gateway
    newName: REGISTRY.azurecr.io/hl7-edge-gateway   # setup.sh sets your ACR name
    newTag: "0.3.0"
patches:
  # MLLP on an internal Azure Load Balancer (a private IP in the cluster's virtual network), never public.
  - target: { kind: Service, name: hl7edge-mllp }
    patch: |
      - op: add
        path: /metadata/annotations
        value:
          service.beta.kubernetes.io/azure-load-balancer-internal: "true"
          # Optional: put it in a specific subnet, or pin its address:
          # service.beta.kubernetes.io/azure-load-balancer-internal-subnet: "snet-hl7"
          # service.beta.kubernetes.io/azure-load-balancer-ipv4: "10.224.0.50"
      - op: add
        path: /spec/externalTrafficPolicy
        value: Local      # keep the sender's address (for allow_from); the LB probes each node

07-azure-aks/nats-azure.yaml

# AKS additions to ../shared/kubernetes/nats-values.yaml: zonal Premium SSD disks for JetStream.
config:
  jetstream:
    fileStore:
      pvc:
        storageClassName: managed-csi-premium

Deploy

You need the Azure CLI (az login), kubectl, helm 3, Docker, and the image archive in the folder. The script creates billable resources: three 2-vCPU nodes (AZ_NODE_SIZE in .env, Standard_D2s_v5 by default), three Premium SSD disks, a registry and a load balancer. Pick a node size your subscription has quota for in all three zones (az vm list-usage -l <region>).

07-azure-aks/setup.sh

#!/usr/bin/env sh
# Sample 07: HL7 Edge Gateway with NATS JetStream on AKS, across three availability zones.
# Needs the Azure CLI (az login), kubectl, helm and Docker. Creates billable resources; ./teardown.sh
# deletes them.
set -eu
cd "$(dirname "$0")"
[ -f .env ] || { echo "copy .env.example to .env and fill it in"; exit 1; }
set -a; . ./.env; set +a
step() { printf '\n== %s\n' "$*"; }

step "Resource group, container registry and AKS cluster (3 nodes, one per zone)"
az group create -n "$AZ_RESOURCE_GROUP" -l "$AZ_LOCATION" -o none
az acr create -g "$AZ_RESOURCE_GROUP" -n "$AZ_ACR_NAME" --sku Standard -o none
az aks create -g "$AZ_RESOURCE_GROUP" -n "$AZ_AKS_NAME" \
  --node-count 3 --zones 1 2 3 --node-vm-size "${AZ_NODE_SIZE:-Standard_D2s_v5}" \
  --network-plugin azure --enable-managed-identity --attach-acr "$AZ_ACR_NAME" \
  --generate-ssh-keys -o none
az aks get-credentials -g "$AZ_RESOURCE_GROUP" -n "$AZ_AKS_NAME" --overwrite-existing

step "The gateway image, into the registry"
docker load -i "$GATEWAY_IMAGE_ARCHIVE"
az acr login -n "$AZ_ACR_NAME"
docker tag "lambogenius/hl7-edge-gateway:$GATEWAY_VERSION" "$AZ_ACR_NAME.azurecr.io/hl7-edge-gateway:$GATEWAY_VERSION"
docker push "$AZ_ACR_NAME.azurecr.io/hl7-edge-gateway:$GATEWAY_VERSION"
sed -i.bak "s|newName: .*azurecr.io/hl7-edge-gateway.*|newName: $AZ_ACR_NAME.azurecr.io/hl7-edge-gateway|; s|newTag: .*|newTag: \"$GATEWAY_VERSION\"|" kustomization.yaml && rm -f kustomization.yaml.bak

step "Namespace and secrets"
kubectl create namespace hl7edge --dry-run=client -o yaml | kubectl apply -f -
kubectl -n hl7edge create secret generic hl7edge-secrets --dry-run=client -o yaml \
  --from-literal=LAMBOGENIUS_LICENSE="$LAMBOGENIUS_LICENSE" --from-literal=HL7EDGE_ADMIN_TOKEN="$HL7EDGE_ADMIN_TOKEN" \
  --from-literal=NATS_TOKEN="$NATS_TOKEN" --from-literal=LAB_API_URL="$LAB_API_URL" --from-literal=LAB_API_TOKEN="$LAB_API_TOKEN" |
  kubectl apply -f -

step "NATS JetStream (3 pods, one per zone, Premium SSD)"
helm repo add nats https://nats-io.github.io/k8s/helm/charts/ >/dev/null 2>&1 || true
helm repo update nats >/dev/null
helm upgrade --install nats nats/nats --version 2.15.1 -n hl7edge \
  -f ../shared/kubernetes/nats-values.yaml -f nats-azure.yaml --wait --timeout 10m

step "The gateway (2 pods) and its internal load balancer"
kubectl apply -k .
kubectl -n hl7edge rollout status deploy/hl7-edge-gateway --timeout=5m
echo "Waiting for the internal load balancer's address..."
for i in $(seq 1 60); do
  ip=$(kubectl -n hl7edge get svc hl7edge-mllp -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
  [ -n "$ip" ] && break; sleep 5
done
kubectl -n hl7edge get pods -o wide
echo
echo "MEDITECH connects to ${ip:-<pending>}:7890 (a private address in the AKS virtual network)."
echo "Admin console: kubectl -n hl7edge port-forward svc/hl7edge-admin 8080, then http://localhost:8080"

When you are done, teardown.sh deletes the whole resource group:

07-azure-aks/teardown.sh

#!/usr/bin/env sh
# Deletes everything setup.sh created (the whole resource group).
set -eu
cd "$(dirname "$0")"
set -a; . ./.env; set +a
az group delete -n "$AZ_RESOURCE_GROUP" --yes --no-wait
echo "Deleting resource group $AZ_RESOURCE_GROUP (takes a few minutes)."

Check it

kubectl -n hl7edge get pods -o wide        # 2 gateway and 3 NATS pods, on different nodes and zones
kubectl -n hl7edge get svc hl7edge-mllp    # EXTERNAL-IP: a private address in the VNet
kubectl -n hl7edge port-forward svc/hl7edge-admin 8080      # console at http://localhost:8080
hl7-edge-gateway send --to <private-ip>:7890 oru-r01.hl7  # from a VM in the VNet
../shared/kubernetes/check.sh              # the failover test, automated

check.sh runs the failover test on the real cluster. It checks that the NATS pods are in three zones, starts a stand-in lab API and MEDITECH in the cluster, sends results through the Service and the internal load balancer, and deletes a gateway pod and a NATS pod while sending more. Every result must arrive once, in order. For this check, set LAB_API_URL=http://lab-api:8000/hl7/results in .env before ./setup.sh; the test pods go away with ./teardown.sh.

shared/kubernetes/check.sh — the same for AKS, EKS and GKE

#!/usr/bin/env sh
# Checks a cloud deployment of samples 07-09 after its setup.sh, with the current kubectl context:
#   - the 3 NATS pods are in 3 zones;
#   - a stand-in lab API and a MEDITECH pod are started in the cluster (the lab API URL in .env must be
#     http://lab-api:8000/hl7/results for this check);
#   - K8S-1 goes through the hl7edge-mllp Service, K8S-2 to K8S-4 through the internal load balancer;
#   - a gateway pod and a NATS pod are deleted while messages flow, and K8S-1 to K8S-4 must arrive once each, in order.
# Run it from a sample folder:  ../shared/kubernetes/check.sh     (the test pods go away with ./teardown.sh)
set -eu
EX=$(cd "$(dirname "$0")/../.." && { pwd -W 2>/dev/null || pwd; })   # pwd -W: a C:/… path for kubectl.exe in Git Bash
export MSYS_NO_PATHCONV=1   # Git Bash on Windows: keep /messages/… as it is (set KUBECONFIG as C:/…)
k() { kubectl -n hl7edge "$@"; }
step() { printf '\n== %s\n' "$*"; }
fail() { echo "FAILED: $*" >&2; exit 1; }
retry() { n=$1; shift; for i in $(seq 1 "$n"); do "$@" >/dev/null 2>&1 && return 0; sleep 2; done; return 1; }
send() { k exec meditech -- hl7-edge-gateway send --to "$1" "/messages/k8s-$2.hl7" 2>&1 | grep -q 'MSA|AA|'; }
received() {
  k exec deploy/lab-api -- python -c "import urllib.request;print(urllib.request.urlopen('http://localhost:8000/received').read().decode())" |
    grep -o 'K8S-[0-9]*' | tr '\n' ' '
}
at_least() { [ "$(received | wc -w)" -ge "$1" ]; }

step "Nodes and pods, by zone"
kubectl get nodes -L topology.kubernetes.io/zone --no-headers | awk '{print "  node " $1 "  zone " $NF}'
zone_of() { kubectl get node "$1" -o jsonpath='{.metadata.labels.topology\.kubernetes\.io/zone}'; }
for p in $(k get pods -o jsonpath='{range .items[*]}{.metadata.name}={.spec.nodeName}{"\n"}{end}'); do
  echo "  pod ${p%%=*}  zone $(zone_of "${p#*=}")"
done
nats_zones=$(for n in $(k get pods -l app.kubernetes.io/name=nats -o jsonpath='{.items[*].spec.nodeName}'); do zone_of "$n"; echo; done | sort -u | grep -c .)
[ "$nats_zones" -eq 3 ] || fail "the 3 NATS pods should be in 3 zones, are in $nats_zones"
echo "NATS: 3 pods in 3 zones"

step "A stand-in lab API and MEDITECH, in the cluster"
IMAGE=$(k get deploy hl7-edge-gateway -o jsonpath='{.spec.template.spec.containers[0].image}')
k create configmap lab-api --from-file=app.py="$EX/01-meditech-to-rest-api/lab-api/app.py" --dry-run=client -o yaml | k apply -f - >/dev/null
k create configmap messages --from-file="$EX/shared/kubernetes/test/messages" --dry-run=client -o yaml | k apply -f - >/dev/null
k apply -f - >/dev/null <<EOF
apiVersion: apps/v1
kind: Deployment
metadata: { name: lab-api }
spec:
  selector: { matchLabels: { app: lab-api } }
  template:
    metadata: { labels: { app: lab-api } }
    spec:
      containers:
        - name: lab-api
          image: python:3.12-slim
          command: ["python", "-u", "/app/app.py"]
          env: [{ name: LAB_API_TOKEN, valueFrom: { secretKeyRef: { name: hl7edge-secrets, key: LAB_API_TOKEN } } }]
          volumeMounts: [{ name: app, mountPath: /app }]
      volumes: [{ name: app, configMap: { name: lab-api } }]
---
apiVersion: v1
kind: Service
metadata: { name: lab-api }
spec: { selector: { app: lab-api }, ports: [{ port: 8000 }] }
---
apiVersion: v1
kind: Pod
metadata: { name: meditech }
spec:
  containers:
    - name: meditech
      image: $IMAGE
      command: ["sleep", "infinity"]
      volumeMounts: [{ name: m, mountPath: /messages }]
  volumes: [{ name: m, configMap: { name: messages } }]
EOF
k rollout status deploy/lab-api --timeout=5m >/dev/null || fail "the lab API did not start"
k wait --for=condition=Ready pod/meditech --timeout=5m >/dev/null || fail "the meditech pod did not start"
LB=$(k get svc hl7edge-mllp -o jsonpath='{.status.loadBalancer.ingress[0].ip}{.status.loadBalancer.ingress[0].hostname}')
[ -n "$LB" ] || fail "the MLLP Service has no load balancer address"
echo "MLLP load balancer: $LB"

step "1. K8S-1 through the Service, K8S-2 through the internal load balancer"
retry 30 send hl7edge-mllp:7890 1 || fail "K8S-1 should be accepted through the Service"
retry 60 send "$LB:7890" 2 || fail "K8S-2 should be accepted through the load balancer at $LB"
retry 30 at_least 2 || fail "2 results should be delivered: $(received)"
echo "Received: $(received)"

step "2. A gateway pod is deleted, then a NATS pod"
pod=$(k get pods -l app=hl7-edge-gateway -o jsonpath='{.items[0].metadata.name}')
k delete pod "$pod" --wait=false >/dev/null; echo "deleted $pod"
retry 30 send "$LB:7890" 3 || fail "K8S-3 should be accepted by the remaining gateway pod"
k delete pod nats-1 --wait=false >/dev/null; echo "deleted nats-1"
sleep 3
retry 30 send "$LB:7890" 4 || fail "K8S-4 should be accepted with one NATS pod gone"
retry 60 at_least 4 || fail "4 results should be delivered: $(received)"
ids=$(received); echo "Received: $ids"
[ "$ids" = "K8S-1 K8S-2 K8S-3 K8S-4 " ] || fail "expected K8S-1 to K8S-4 once each, in order"
retry 60 k rollout status statefulset/nats --timeout=5s || fail "nats-1 did not come back"
retry 60 k rollout status deploy/hl7-edge-gateway --timeout=5s || fail "the gateway pod was not replaced"
k get pods -o wide | sed 's/^/  /'

step "Passed"

For production

• Pin the address MEDITECH uses with azure-load-balancer-ipv4 and -internal-subnet.
• Keep secrets in Key Vault (Secrets Store CSI driver) rather than .env.
• For Service Bus or Entra ID-protected APIs, use AKS workload identity: auth = { type = "workload_identity" }.
• Alert on hl7edge_queue_depth and hl7edge_dead_letters (the pods have Prometheus annotations).