DevOpsTools/vmware-api-simulator
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Expand lab seed tiers, OpenAPI PARAMS, and console DATA/Params UX.

Browse Source
This commit is contained in:
Sergey Antropoff
2026-07-18 08:46:39 +03:00
parent f8d3cbdd59
commit 63cc409424
71 changed files with 38380 additions and 796 deletions
Download Patch File Download Diff File Expand all files Collapse all files
pulumi-tests/README.md
+39 -24
View File
@@ -2,37 +2,50 @@
Hybrid suite under `pulumi-tests/`:
1. Official [`pulumi-vsphere`](https://www.pulumi.com/registry/packages/vsphere/) (SOAP/VIM via govmomi)
2. Full REST `IMPLEMENTED` × majors **6–9** matrix (deep + stub response checks)
1. **Layer A (required for “100%”)** — HTTP contract matrix: all `IMPLEMENTED`
REST `verb+path` × catalog majors **6–9**, plus synthetic **HEAD** on every GET
2. **Layer B (smoke / lifecycle)** — official
[`pulumi-vsphere`](https://www.pulumi.com/registry/packages/vsphere/) programs
3. Deep REST CRUD (session / folder / tag / content library / VM)
4. All SOAP WSDL ops advertised on `/sdk/vimService.wsdl`
4. SOAP WSDL ops already advertised on `/sdk/vimService.wsdl` (not a new SOAP universe)
`pulumi-vsphere` alone cannot hit ~1092 REST routes — the HTTP matrix is required for full-surface confidence.
`pulumi-vsphere` alone cannot hit ~1092 REST routes — **100% means the HTTP
matrix** (`coverage probed/declared`, `critical=0`), not provider resource count.
## Pass rules
## Pass rules (Layer A)
| Check | Rule |
|-------|------|
| Declared | `methods_for_major(M)` minus session DELETE, plus `HEAD` per GET |
| Probed | Exactly one auth’d probe per declared route (`probed == declared`) |
| Critical | `0` — no 5xx/501; no `"stub": true` / empty body on inventory-critical GETs (major 9) |
| Coverage line | `{probed - critical}/{declared}` (aggregate + per major) |
| Verb histogram | Includes `HEAD` |
| Stubs | Spec-shaped durable JSON OK; reported as `stub`, not durable CRUD |
| Auth | Fresh `vmware-api-session-id` session for each major pass |
## Pass rules (Layer B + extras)
| Class | Pass rule |
|-------|-----------|
| REST GET (inventory-critical / deep) | 2xx, body non-empty, not `"stub": true` on major 9 critical paths |
| REST stubs (universe) | no 5xx/501; response present; reported as `stub` (not claimed durable CRUD) |
| pulumi-vsphere | Existing cases + nonempty exports (lifecycle smoke, not full API) |
| REST deep CRUD | create → GET nonempty → update (where supported) → delete → GET missing |
| SOAP WSDL ops (~45) | POST `/sdk` without 5xx; Create/Power/Clone/Destroy checked via inventory |
| pulumi-vsphere | existing cases + nonempty exports |
| SOAP WSDL ops | POST `/sdk` without 5xx; Create/Power/Clone/Destroy checked via inventory |
## What runs
| Case | Layer | Notes |
|------|-------|-------|
| `PU-INV` | pulumi-vsphere | Inventory data sources |
| `PU-FOLDER` | pulumi-vsphere | Folder create/destroy |
| `PU-VM` | pulumi-vsphere | VirtualMachine create/destroy |
| `PU-TAG` | pulumi-vsphere | TagCategory + Tag |
| `PU-REST` | REST matrix | `IMPLEMENTED` × majors 6–9 (smoke: major 9 only) |
| `PU-CRUD` | REST CRUD | Session, folder, tagging, content library, VM |
| `PU-SOAP` | SOAP | All WSDL ops |
| `PU-INV` | B · pulumi-vsphere | Inventory data sources |
| `PU-FOLDER` | B · pulumi-vsphere | Folder create/destroy |
| `PU-VM` | B · pulumi-vsphere | VirtualMachine create/destroy |
| `PU-TAG` | B · pulumi-vsphere | TagCategory + Tag |
| `PU-REST` | **A · HTTP matrix** | `IMPLEMENTED` × majors 6–9 + HEAD (smoke: major 9) |
| `PU-CRUD` | Deep REST | Session, folder, tagging, content library, VM |
| `PU-SOAP` | SOAP | Existing WSDL ops only |
Artifacts: HTML + JSON + JUnit under the `lab-reports` volume. JSON includes
`rest.total` / `rest.failed`, `crud.failed`, `soap.failed`.
Artifacts: HTML + JSON + JUnit under the `lab-reports` volume. Look for
`rest.coverage_line`, `rest.critical`, `rest.by_verb` (incl. HEAD).
## Quick start
@@ -40,7 +53,7 @@ From the **repo root**:
```bash
make pulumi-tests # full hybrid suite
make pulumi-tests-smoke # PU-INV + one-major REST smoke
make pulumi-tests-smoke # PU-INV + Layer A major-9 smoke
```
Or from this directory:
@@ -51,7 +64,9 @@ make up
make test-pulumi # or: make test-pulumi-smoke
```
Gateway (in-compose): `https://api-gateway` (host map `127.0.0.1:18443`).
Gateway (in-compose): `https://api-gateway` (runner → service name).
Optional host map: `127.0.0.1:28443` (does **not** replace main lab
`https://localhost` / `:18443` — stop this stack with `make down` when done).
Seed profile: `small` → `Datacenter` / `Cluster` / `datastore1` / `VM Network` /
`web-01` / `esxi01.lab.local` / `/Datacenter/vm/production`.
@@ -59,8 +74,8 @@ Seed profile: `small` → `Datacenter` / `Cluster` / `datastore1` / `VM Network`
| Target | Meaning |
|--------|---------|
| `make test-pulumi` / `pulumi-tests` | Full hybrid: pulumi-vsphere + REST×6–9 + CRUD + SOAP |
| `make test-pulumi-smoke` | `PU-INV` + REST major-9 smoke (no VM/tags/CRUD/SOAP) |
| `make test-pulumi` / `pulumi-tests` | Full hybrid: Layer A ×6–9 + Layer B + CRUD + SOAP |
| `make test-pulumi-smoke` | `PU-INV` + Layer A major-9 smoke (no VM/tags/CRUD/SOAP) |
| `make up` / `down` / `seed` | Lab stack lifecycle |
## Layout
@@ -68,9 +83,9 @@ Seed profile: `small` → `Datacenter` / `Cluster` / `datastore1` / `VM Network`
```
pulumi-tests/
run_suite.py # Automation API + REST/SOAP probes
report_html.py
report_html.py # coverage line + verb histogram
lib/assert_nonempty.py
lib/rest_matrix.py # IMPLEMENTED × majors
lib/rest_matrix.py # Layer A: IMPLEMENTED × majors + HEAD
lib/rest_crud.py # deep create/read/update/delete
lib/soap_ops.py # WSDL SOAP ops
programs/inventory/