vmware-api-simulator/pulumi-tests

pulumi hybrid lab suite for vmware-api-simulator

Hybrid suite under pulumi-tests/:

1. Layer A (required for “100%”) — HTTP contract matrix: all IMPLEMENTED REST verb+path × catalog majors 6–9, plus synthetic HEAD on every GET
2. Layer B (smoke / lifecycle) — official pulumi-vsphere programs
3. Deep REST CRUD (session / folder / tag / content library / VM)
4. SOAP WSDL ops already advertised on /sdk/vimService.wsdl (not a new SOAP universe)

pulumi-vsphere alone cannot hit ~1092 REST routes — 100% means the HTTP matrix (coverage probed/declared, critical=0), not provider resource count.

Pass rules (Layer A)

Check	Rule
Declared	methods_for_major(M) minus session DELETE, plus HEAD per GET
Probed	Exactly one auth’d probe per declared route (probed == declared)
Critical	0 — no 5xx/501; no "stub": true / empty body on inventory-critical GETs (major 9)
Coverage line	{probed - critical}/{declared} (aggregate + per major)
Verb histogram	Includes HEAD
Stubs	Spec-shaped durable JSON OK; reported as stub, not durable CRUD
Auth	Fresh vmware-api-session-id session for each major pass

Pass rules (Layer B + extras)

Class	Pass rule
pulumi-vsphere	Existing cases + nonempty exports (lifecycle smoke, not full API)
REST deep CRUD	create → GET nonempty → update (where supported) → delete → GET missing
SOAP WSDL ops	POST /sdk without 5xx; Create/Power/Clone/Destroy checked via inventory

What runs

Case	Layer	Notes
PU-INV	B · pulumi-vsphere	Inventory data sources
PU-FOLDER	B · pulumi-vsphere	Folder create/destroy
PU-VM	B · pulumi-vsphere	VirtualMachine create/destroy
PU-TAG	B · pulumi-vsphere	TagCategory + Tag
PU-REST	A · HTTP matrix	IMPLEMENTED × majors 6–9 + HEAD (smoke: major 9)
PU-CRUD	Deep REST	Session, folder, tagging, content library, VM
PU-SOAP	SOAP	Existing WSDL ops only

Artifacts: HTML + JSON + JUnit under the lab-reports volume. Look for rest.coverage_line, rest.critical, rest.by_verb (incl. HEAD).

Quick start

From the repo root:

make pulumi-tests          # full hybrid suite
make pulumi-tests-smoke    # PU-INV + Layer A major-9 smoke

Or from this directory:

cd pulumi-tests
make up
make test-pulumi           # or: make test-pulumi-smoke

Gateway (in-compose): https://api-gateway (runner → service name). Optional host map: 127.0.0.1:28443 (does not replace main lab https://localhost / :18443 — stop this stack with make down when done). Seed profile: small → Datacenter / Cluster / datastore1 / VM Network / web-01 / esxi01.lab.local / /Datacenter/vm/production.

Make targets

Target	Meaning
make test-pulumi / pulumi-tests	Full hybrid: Layer A ×6–9 + Layer B + CRUD + SOAP
make test-pulumi-smoke	PU-INV + Layer A major-9 smoke (no VM/tags/CRUD/SOAP)
make up / down / seed	Lab stack lifecycle

Layout

pulumi-tests/
  run_suite.py                 # Automation API + REST/SOAP probes
  report_html.py               # coverage line + verb histogram
  lib/assert_nonempty.py
  lib/rest_matrix.py           # Layer A: IMPLEMENTED × majors + HEAD
  lib/rest_crud.py             # deep create/read/update/delete
  lib/soap_ops.py              # WSDL SOAP ops
  programs/inventory/
  programs/folders/
  programs/vm_lifecycle/
  programs/tags/
  docker/Dockerfile.pulumi-runner
  docker-compose.yml

PYTHONPATH mounts /workspace so probes import app.vsphere.* coverage/matrix.