Document 2026-07-18 lab test results across EN/RU guides.
Record offline 161 passed, green test-vsphere probes, and full pulumi-tests 7/7 with REST matrix 1987/1987 (critical=0); note the Terraform cookbook binary mismatch separately.
This commit is contained in:
+35
-2
@@ -1,5 +1,7 @@
|
||||
# pulumi hybrid lab suite for vmware-api-simulator
|
||||
|
||||
**Language / Язык:** [English](README.md) | [Русский](README.ru.md)
|
||||
|
||||
Hybrid suite under `pulumi-tests/`:
|
||||
|
||||
1. **Layer A (required for “100%”)** — HTTP contract matrix: all `IMPLEMENTED`
|
||||
@@ -12,6 +14,28 @@ Hybrid suite under `pulumi-tests/`:
|
||||
`pulumi-vsphere` alone cannot hit ~1092 REST routes — **100% means the HTTP
|
||||
matrix** (`coverage probed/declared`, `critical=0`), not provider resource count.
|
||||
|
||||
## Latest lab run (2026-07-18)
|
||||
|
||||
Full hybrid (`make pulumi-tests`, seed profile `small`):
|
||||
|
||||
| Check | Result |
|
||||
|-------|--------|
|
||||
| Cases | **7 / 7 passed** (`PU-INV`, `PU-FOLDER`, `PU-VM`, `PU-TAG`, `PU-REST`, `PU-CRUD`, `PU-SOAP`) |
|
||||
| Layer A REST matrix | **1987 / 1987** (`critical=0`) across majors 6–9 |
|
||||
| Verb histogram | GET 682 · HEAD 682 · POST 301 · PUT 96 · PATCH 91 · DELETE 135 |
|
||||
|
||||
Artifacts (local copy from the `lab-reports` volume; directory is gitignored):
|
||||
`pulumi-tests/reports/pulumi-report.html`,
|
||||
`pulumi-tests/reports/pulumi-summary.json`.
|
||||
|
||||
Related offline / gateway probes from the same day:
|
||||
|
||||
| Suite | Result |
|
||||
|-------|--------|
|
||||
| `make test` (offline unit + contract) | **161 passed** |
|
||||
| `make test-vsphere` pytest + surface/matrix/dump/nonempty | **73** pytest passed; matrix/surface/nonempty **0 failures** |
|
||||
| Client cookbooks (part of `test-vsphere`) | Python / Ansible URI / Pulumi-style OK; **Terraform cookbook** failed (`Exec format error` on `.tools/terraform` binary — host/arch mismatch, not an API regression) |
|
||||
|
||||
## Pass rules (Layer A)
|
||||
|
||||
| Check | Rule |
|
||||
@@ -44,8 +68,17 @@ matrix** (`coverage probed/declared`, `critical=0`), not provider resource count
|
||||
| `PU-CRUD` | Deep REST | Session, folder, tagging, content library, VM |
|
||||
| `PU-SOAP` | SOAP | Existing WSDL ops only |
|
||||
|
||||
Artifacts: HTML + JSON + JUnit under the `lab-reports` volume. Look for
|
||||
`rest.coverage_line`, `rest.critical`, `rest.by_verb` (incl. HEAD).
|
||||
Artifacts: HTML + JSON + JUnit under the `lab-reports` volume (and optionally
|
||||
`pulumi-tests/reports/` on the host). Look for `rest.coverage_line`,
|
||||
`rest.critical`, `rest.by_verb` (incl. HEAD). Open the HTML report after a run:
|
||||
|
||||
```bash
|
||||
mkdir -p pulumi-tests/reports
|
||||
cd pulumi-tests && docker compose --profile test run --rm --no-deps \
|
||||
-v "$(pwd)/reports:/out" pulumi-runner \
|
||||
cp /reports/pulumi-report.html /reports/pulumi-summary.json /out/
|
||||
open reports/pulumi-report.html
|
||||
```
|
||||
|
||||
## Quick start
|
||||
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# Гибридный lab-набор pulumi для vmware-api-simulator
|
||||
|
||||
**Language / Язык:** [English](README.md) | [Русский](README.ru.md)
|
||||
|
||||
Гибрид под `pulumi-tests/`:
|
||||
|
||||
1. **Layer A (обязателен для «100%»)** — HTTP contract matrix: все `IMPLEMENTED`
|
||||
@@ -12,6 +14,28 @@
|
||||
Одним `pulumi-vsphere` закрыть ~1092 REST `verb×path` нельзя — **100% = HTTP-матрица**
|
||||
(`coverage probed/declared`, `critical=0`), а не число provider-ресурсов.
|
||||
|
||||
## Последний lab-прогон (2026-07-18)
|
||||
|
||||
Полный гибрид (`make pulumi-tests`, seed-профиль `small`):
|
||||
|
||||
| Проверка | Результат |
|
||||
|----------|-----------|
|
||||
| Кейсы | **7 / 7 passed** (`PU-INV`, `PU-FOLDER`, `PU-VM`, `PU-TAG`, `PU-REST`, `PU-CRUD`, `PU-SOAP`) |
|
||||
| Layer A REST matrix | **1987 / 1987** (`critical=0`) по majors 6–9 |
|
||||
| Verb histogram | GET 682 · HEAD 682 · POST 301 · PUT 96 · PATCH 91 · DELETE 135 |
|
||||
|
||||
Артефакты (локальная копия с тома `lab-reports`; каталог в `.gitignore`):
|
||||
`pulumi-tests/reports/pulumi-report.html`,
|
||||
`pulumi-tests/reports/pulumi-summary.json`.
|
||||
|
||||
Связанные прогоны того же дня:
|
||||
|
||||
| Набор | Результат |
|
||||
|-------|-----------|
|
||||
| `make test` (offline unit + contract) | **161 passed** |
|
||||
| `make test-vsphere` pytest + surface/matrix/dump/nonempty | **73** pytest passed; matrix/surface/nonempty **0 failures** |
|
||||
| Client cookbooks (часть `test-vsphere`) | Python / Ansible URI / Pulumi-style OK; **Terraform cookbook** упал (`Exec format error` на `.tools/terraform` — несовпадение host/arch, не регрессия API) |
|
||||
|
||||
## Критерии pass (Layer A)
|
||||
|
||||
| Проверка | Правило |
|
||||
@@ -37,6 +61,16 @@ make pulumi-tests # полный гибрид
|
||||
make pulumi-tests-smoke # PU-INV + Layer A major-9
|
||||
```
|
||||
|
||||
После прогона HTML-отчёт:
|
||||
|
||||
```bash
|
||||
mkdir -p pulumi-tests/reports
|
||||
cd pulumi-tests && docker compose --profile test run --rm --no-deps \
|
||||
-v "$(pwd)/reports:/out" pulumi-runner \
|
||||
cp /reports/pulumi-report.html /reports/pulumi-summary.json /out/
|
||||
open reports/pulumi-report.html
|
||||
```
|
||||
|
||||
Шлюз (в compose): `https://api-gateway`. Хост-алиас: `127.0.0.1:28443`
|
||||
(не занимает `https://localhost` / `:18443` основного lab — после тестов
|
||||
`make down`).
|
||||
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user