Windows compute pool

A VirtualMachinePool that clones the golden ws2019-symphony image and, on every clone’s sysprep specialize pass, injects per-VM identity and joins it to the grid’s masters. This is the component that makes the "minutes, not procurement cycles" elasticity claim concrete — clone-to-AgentConnected was measured at ~5 minutes on real reclones — and it is also, honestly, the one place the architecture’s story is not yet fully closed: the join is fixed in source but not independently re-verified against a fresh clone.

Purpose

2 replicas exist on sym-aws today, cloning successfully; join status is the open item (see Maturity below). The VirtualMachinePool (sym-compute-win) uses the d1.2xlarge instancetype — dedicated CPU, hard-requiring the CPU Manager KubeletConfig from OpenShift Virtualization — clones DataSource ws2019-symphony into a 70Gi rootdisk, and attaches a sysprep CD-ROM volume from the win-sysprep-join ConfigMap. That ConfigMap and the win-admin Secret it depends on are rendered and applied directly by scripts/create-secrets.sh, not kustomize-owned — per-cluster rendering with live master IPs, not static git content.

No egoconfig join subcommand exists on Windows. The Windows egoconfig.exe (egocore 4.0 MSI) has no join subcommand at all — given one, it prints a paginated usage screen and blocks on stdin forever. The durable fix, and the one currently in source: skip egoconfig entirely for the join and directly rewrite EGO_MASTER_LIST="<master1> <master2>" in kernel\conf\ego.conf. Ports are already correct from the MSI’s BASEPORT=7869 install-time property.

Placement

Property Value

Cluster / Namespace

sym-aws / symphony

Compute

virt-metal node — the same single m5zn.metal shared with the RHEL masters

Network

Grid layer2 UDN, l2bridge binding, same 10.10.0.0/20 subnet as the masters — no NAT, no DNS; identity resolved via /etc/hosts entries written at join time

Storage

gp3-csi (EBS, RWO), 70Gi clone disks — compute hosts are cattle (rejoin on reboot/reclone), so RWO’s lack of live migration is an accepted trade here, unlike for the masters

Security posture

WinRM is baked into the golden image and enabled as the very first action at firstboot — before hosts entries, before the EGO join logic — so a guest that later hangs or crashes mid-script is still remotely inspectable (ADR 0012). Basic auth, unencrypted, port 5985; accepted only because the grid UDN is private and non-NAT’d, and this is a PoC image, not a production one

ROKS burst pool

Phase 3+, a separate overlay whose replicas KEDA or the Host Factory provider drives independently — not yet built

netsh advfirewall, not -NetFirewallRule/WMI cmdlets, is used for every firewall rule added during specialize — the WMI/CIM firewall provider is not functional that early in boot (every -NetFirewallRule call dies with HRESULT 0x8004100a, fleet-confirmed). The LIM service ships Manual startup type from the golden image, not Automatic — if it auto-started before the master-list rewrite, a host heading its own master list acts as a master and demands entitlement.

Interfaces

Peer Direction Port / Protocol Purpose

sym-master1/sym-master2

outbound (register)

7869 TCP+UDP

Compute host’s LIM registers with the master(s) in EGO_MASTER_LIST

sym-master1/sym-master2

inbound (dial-back)

7869-7875 TCP + 7869 UDP

The master’s LIM dials back to the compute host after registration — the build’s single most costly firewall gap: Windows blocks unsolicited inbound by default on every profile, so a host can be fully outbound-reachable and still sit silently unavail

installer-repo

outbound

HTTP 5985 (WinRM), N/A (image build only)

Diagnostic/debug channel — not used for the join logic itself

Operator/debug workstation

inbound

5985 TCP (WinRM, Basic/unencrypted)

Guest inspection without a VNC rig — grid-UDN-only, PoC posture

`firstboot-symphony.ps1’s own probe

outbound

$MasterIP:7869 TCP

Reachability gate only (waits up to 5 min) — diagnostic false trail: each probe logs a harmless master-side not using privileged port line, not evidence of a real join attempt (cost hours in build)

Maturity

  • Clone-to-AgentConnected measured at ~5 minutes on real reclones: VM Running phase reached ~2-3 min from delete, AgentConnected=True by ~T0+5min; CSI smart-clone of both root disks completed within ~3 min. Full clone-to-joined timing is the Phase-2 exit measurement and has not been closed out.

  • Stale-ConfigMap re-render discipline (recurring build trap): win-sysprep-join is rendered and applied directly by create-secrets.sh, not reconciled by Argo — every fix to firstboot-symphony.ps1 requires an explicit re-run and a reclone of every existing compute VM, since the sysprep CD-ROM is only mounted at VM create time. Treat "is the live ConfigMap current vs git" as a standing verification step, not an assumption.

  • Four compounding root causes were found and fixed in the join path (git commit 5158077): no egoconfig join subcommand on Windows → direct EGO_MASTER_LIST rewrite; -NetFirewallRule/WMI cmdlets fail with 0x8004100a during specialize → netsh advfirewall; the installed service is named LIM, not EGO; and the MTU 8901-vs-1500 mismatch between masters and Windows guests silently dropped the master’s UDP LIM join payload.

  • Honest status as of this writing: the commit that landed fixes 1-4 claims the join was live-proven working on running clones before the golden image was rebaked with the fix baked in — but that confirmation has not yet been independently re-verified against a fresh clone, and a separate, still-open blocker (WinRM auth rejecting the win-admin password on post-re-render clones, suspected create-secrets.sh password desync) was outstanding as of the last logged handoff. Treat the join as fixed-in-source, not yet independently confirmed until a fresh egosh resource list on the master closes it out. This is the single open item blocking Phase 2 exit criteria.

  • KEDA scale-in against this pool is explicitly not grid-aware (see KEDA metrics shim) — deletes VMs without draining, which is why this pool’s replicas is hand-set (2) rather than autoscaler-driven in the current build.

Verify

# 1. Pool and its VMs/VMIs are healthy at the KubeVirt level
oc get vmpool sym-compute-win -n symphony
oc get vmi -n symphony -l app=sym-compute-win -o wide

# 2. Confirm the LIVE win-sysprep-join ConfigMap matches git (the stale-ConfigMap check --
#    run this before trusting any join-path fix is actually in effect)
oc get configmap win-sysprep-join -n symphony -o jsonpath='{.data.firstboot-symphony\.ps1}' \
  | diff - images/windows/firstboot-symphony.ps1

# 3. WinRM probe pattern from an in-cluster pod on the grid UDN (no passwords in examples --
#    supply WIN_ADMIN_PASSWORD via env, never inline)
oc run winrm-check --rm -i --restart=Never --image=registry.access.redhat.com/ubi9/ubi-minimal -n symphony -- \
  bash -c 'microdnf install -y nmap-ncat && nc -zv <compute-vm-ip> 5985'

# 4. The definitive join proof: run on the master, over SSH, from a pod that can reach the
#    grid UDN -- a new compute host must appear "ok", not "unavail"
ssh egoadmin@10.10.0.31 'source /opt/ibm/spectrumcomputing/profile.platform && egosh resource list'

# 5. If a host is unavail, confirm the two most common causes first (bidirectional firewall,
#    then MTU) before chasing anything else
ssh egoadmin@10.10.0.31 'source /opt/ibm/spectrumcomputing/profile.platform && egosh resource view <host>'

Continue to GitOps, the reconciliation model that deploys everything above.