DevOpsTools/proxmox-api-simulator
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Align sized cluster seeds and GET dumps with PVE wire shapes; restyle DATA panel.

Browse Source 
- Scale small/large/big seeds (3×50 / 10×1000 / 20×2000) with proportional
  backups, snapshots, HA, replication, Ceph capacity, and OSD totals
  (10 / 100 / 500) plus matching node disks and crush/pg metadata
- Enrich handler responses for apt, certificates, qemu/lxc status, storage,
  SDN, metrics export, and related cluster/node dumps
- Flatten nested body_example fields into PARAMS and sync the request body
  via dotted paths (oVirt-style)
- Restyle DATA controls as size cards with full-width Reset to minimal /
  Refresh stats; unload reloads the minimal cluster
This commit is contained in:
Sergey Antropoff
2026-07-18 08:46:11 +03:00
parent 48df10b17e
commit 0773f721ea
77 changed files with 4870 additions and 855 deletions
Download Patch File Download Diff File Expand all files Collapse all files
pulumi-tests/README.md
+24 -3
View File
@@ -6,10 +6,12 @@ Docker-only suite against the Proxmox API simulator.
| Layer | What it covers |
|---|---|
| **Surface** | **100%** of declared contract methods for PVE majors **6–9** (every path+verb) via HTTP (`pvelib/surface.py`). Suite PASS requires `declared == probed` per major and zero critical failures (501, “not supported” messages, 5xx, exceptions, unknown verbs). |
| **Lifecycle** | **`pulumi-proxmoxve` only** (BPG Terraform bridge) — Provider, inventory data sources, `VmLegacy` + non-empty checks. Negative auth via httpx. This is **not** full API coverage; the provider exposes dozens of resources, not thousands of contract methods. Provider uses internal HTTPS gateway; surface stays HTTP. |
| **A — Surface (HTTP contract matrix)** | **100%** means every declared contract method for PVE majors **6–9** (path+verb: GET/PUT/POST/DELETE), plus **synthetic HEAD on each GET path**. Ticket + CSRF on mutations; form-urlencoded bodies; Proxmox `{ "data": … }` envelope. PASS requires `declared == probed` per major **and** aggregate, `critical=0` (501, “not supported”, 5xx, exceptions, empty/wrong 2xx envelopes). HEAD is in the verb histogram but **not** in the declared/probed denominator. |
| **B — Lifecycle** | **`pulumi-proxmoxve` smoke only** (BPG Terraform bridge) — Provider, inventory data sources, `VmLegacy` + non-empty checks. **Not** 100% API and **not** provider resource-count coverage. HTTPS via lab TLS gateway; surface stays HTTP. |
Reports (`pulumi/reports/report.html`, `results.json`, `junit.xml`) include a **Full contract coverage** summary with per-major and total `declared`/`probed` counts (e.g. `Coverage: 2324/2324 methods across majors 6–9`).
Reports (`pulumi/reports/report.html`, `results.json`, `junit.xml`) include **Full contract coverage**, a **verb histogram (incl. HEAD)**, and a line like:
`Coverage: N/N methods across majors 6–9 (critical=0)`
## Layout
@@ -43,7 +45,26 @@ open pulumi/reports/report.html
make down
```
### Quick curl (ticket → GET/POST + report)
Against a running suite stack (`http://localhost:8006`):
```bash
TICKET=$(curl -s -c /tmp/pve.ck -b /tmp/pve.ck \
-d 'username=root@pam&password=secret' \
http://localhost:8006/api2/json/access/ticket)
echo "$TICKET" | jq .
CSRF=$(echo "$TICKET" | jq -r .data.CSRFPreventionToken)
curl -s -b /tmp/pve.ck http://localhost:8006/api2/json/version | jq .
curl -s -b /tmp/pve.ck -H "CSRFPreventionToken: $CSRF" \
-d 'vmid=100&name=demo' \
http://localhost:8006/api2/json/nodes/pve01/qemu | jq .
```
Open the HTML report: `pulumi-tests/pulumi/reports/report.html`.
Provider env (set by Compose): `PROXMOX_VE_ENDPOINT=https://tls-gateway:8443/`
(internal suite TLS — `pulumi-proxmoxve` rejects `http://`), plus username/password
/`INSECURE`. Surface probe uses `API_URL=http://simulator:8006`. Host lab URL
remains plain `http://localhost:8006/` (Kubernetes HTTPS is Ingress-only).
Seed for CI/suite: `SEED_PROFILE=small` (Compose default).
pulumi-tests/README.ru.md
+24 -3
View File
@@ -6,10 +6,12 @@ Suite только в Docker против симулятора Proxmox API.
| Слой | Покрытие |
|---|---|
| **Surface** | **100%** объявленных методов контракта majors **6–9** (каждый path+verb) по HTTP (`pvelib/surface.py`). PASS только при `declared == probed` на каждый major и нуле критичных сбоев (501, «not supported», 5xx, exceptions, неизвестные verb). |
| **Lifecycle** | Только **`pulumi-proxmoxve`** (мост BPG) — Provider, data sources инвентаря, `VmLegacy` + проверки непустых outputs. Это **не** полное покрытие API: в провайдере десятки ресурсов, не тысячи contract methods. HTTPS через lab TLS-шлюз; surface — HTTP. |
| **A — Surface (HTTP contract matrix)** | **100%** = каждый объявленный метод контракта majors **6–9** (path+verb: GET/PUT/POST/DELETE) плюс **синтетический HEAD на каждый GET path**. Ticket + CSRF на мутациях; form-urlencoded; оболочка `{ "data": … }`. PASS при `declared == probed` на major и в сумме, `critical=0` (501, «not supported», 5xx, exceptions, пустые/неверные 2xx envelope). HEAD — в гистограмме глаголов, **не** в знаменателе declared/probed. |
| **B — Lifecycle** | Только smoke **`pulumi-proxmoxve`** (мост BPG) — Provider, data sources инвентаря, `VmLegacy` + непустые outputs. Это **не** 100% API и **не** покрытие по числу ресурсов провайдера. HTTPS через lab TLS-шлюз; surface — HTTP. |
В отчётах (`pulumi/reports/report.html`, `results.json`, `junit.xml`) — блок **Full contract coverage** с итогами `declared`/`probed` по major и суммой (например `Coverage: 2324/2324 methods across majors 6–9`).
В отчётах (`pulumi/reports/report.html`, `results.json`, `junit.xml`) — **Full contract coverage**, **verb histogram (вкл. HEAD)** и строка вида:
`Coverage: N/N methods across majors 6–9 (critical=0)`
## Структура
@@ -43,7 +45,26 @@ open pulumi/reports/report.html
make down
```
### Короткий curl (ticket → GET/POST + отчёт)
При поднятом стеке (`http://localhost:8006`):
```bash
TICKET=$(curl -s -c /tmp/pve.ck -b /tmp/pve.ck \
-d 'username=root@pam&password=secret' \
http://localhost:8006/api2/json/access/ticket)
echo "$TICKET" | jq .
CSRF=$(echo "$TICKET" | jq -r .data.CSRFPreventionToken)
curl -s -b /tmp/pve.ck http://localhost:8006/api2/json/version | jq .
curl -s -b /tmp/pve.ck -H "CSRFPreventionToken: $CSRF" \
-d 'vmid=100&name=demo' \
http://localhost:8006/api2/json/nodes/pve01/qemu | jq .
```
Отчёт: `pulumi-tests/pulumi/reports/report.html`.
Env провайдера (Compose): `PROXMOX_VE_ENDPOINT=https://tls-gateway:8443/`
(внутренний TLS suite — `pulumi-proxmoxve` не принимает `http://`), плюс
username/password/`INSECURE`. Surface probe: `API_URL=http://simulator:8006`.
Хостовый lab URL — plain `http://localhost:8006/` (HTTPS в K8s — только Ingress).
Seed для CI/suite: `SEED_PROFILE=small` (default Compose).
pulumi-tests/pulumi/programs/lifecycle/Pulumi.hxlife11784346334.yaml
+1
View File
@@ -0,0 +1 @@
encryptionsalt: v1:Im3SuXTFiBk=:v1:oUxFIClVSpmc6Jkb:AXfMbHESIZWB3tnQLGNdQNp2ICM+UA==
pulumi-tests/pulumi/programs/lifecycle/Pulumi.hxlife11784346404.yaml
+3
View File
@@ -0,0 +1,3 @@
encryptionsalt: v1:7PSO8VI3JSQ=:v1:LKcWFamTow4SZLl4:ItaMZLsAtkKhooVaycXnDR7F2dWwmA==
config:
hx-lifecycle:smoke: "0"
pulumi-tests/pulumi/pvelib/surface.py
+156 -4
View File
@@ -2,6 +2,12 @@
Mirrors ``app/surface_probe.py`` classification and path/body synthesis, but
talks HTTP to ``API_URL`` instead of an in-process ASGI app.
Layer A matrix:
- every declared path+verb (GET/PUT/POST/DELETE)
- synthetic HEAD on each GET path (histogram only; not counted in declared)
- ticket + CSRF on mutations; form-urlencoded bodies
- Proxmox ``{ "data": … }`` envelope checks on 2xx
"""
from __future__ import annotations
@@ -31,6 +37,11 @@ _FORBIDDEN = re.compile(
r"handler pending for this contract method|is not supported in the (emulator|simulator)",
re.I,
)
_UPID_RE = re.compile(
r"^UPID:[A-Za-z0-9][A-Za-z0-9_-]*:"
r"[0-9A-Fa-f]{8}:[0-9A-Fa-f]{8}:[0-9A-Fa-f]{8}:"
r"[A-Za-z0-9_-]+:[^:]*:[^:]+:$"
)
_PATH_PARAM_EXAMPLES: dict[str, object] = {
"node": "pve01",
@@ -50,6 +61,11 @@ _PATH_PARAM_EXAMPLES: dict[str, object] = {
"key": "cpu",
"digest": "00000000",
"name": "example",
"size": "1G",
"filename": "vm-100-disk-0.raw",
"certificates": "-----BEGIN CERTIFICATE-----\nMIIB\n-----END CERTIFICATE-----\n",
"contact": "mailto:admin@example.com",
"clustername": "lab",
}
_EXTRA_PATH: dict[str, object] = {
@@ -85,7 +101,36 @@ _EXTRA_PATH: dict[str, object] = {
}
CRITICAL_BUCKETS = frozenset(
{"unimplemented_501", "unsupported_message", "server_5xx", "exception"}
{
"unimplemented_501",
"unsupported_message",
"server_5xx",
"exception",
"empty_success_body",
"wrong_returns_envelope",
}
)
# Path suffixes that real PVE usually runs via fork_worker → UPID string.
_WORKER_SUFFIXES = (
"/status/start",
"/status/stop",
"/status/shutdown",
"/status/reboot",
"/status/reset",
"/status/suspend",
"/status/resume",
"/clone",
"/migrate",
"/remote_migrate",
"/snapshot",
"/rollback",
"/template",
"/resize",
"/move_disk",
"/move_volume",
"/vzdump",
"/apt/update",
)
@@ -195,6 +240,64 @@ def classify(status: int, text: str) -> str:
return f"other_{status}"
def returns_type(method: dict[str, Any]) -> str:
returns = method.get("returns") or {}
if isinstance(returns, dict):
return str(returns.get("type") or "")
return ""
def looks_like_worker(path_template: str, verb: str) -> bool:
if verb.upper() not in {"POST", "PUT"}:
return False
lowered = path_template.lower()
return any(lowered.endswith(suffix) for suffix in _WORKER_SUFFIXES)
def classify_envelope(
*,
status: int,
text: str,
method: dict[str, Any],
path_template: str,
verb: str,
) -> str | None:
"""Return an extra critical bucket for 2xx envelope/returns mismatches, else None."""
if not (200 <= status < 300) or verb.upper() == "HEAD":
return None
returns = method.get("returns") or {}
rtype = returns_type(method)
try:
payload = json.loads(text) if text else None
except json.JSONDecodeError:
return "empty_success_body"
if not isinstance(payload, dict) or "data" not in payload:
return "empty_success_body"
data = payload.get("data")
if rtype == "null":
return None
if rtype == "string":
if data is None:
return "wrong_returns_envelope"
if looks_like_worker(path_template, verb):
if not isinstance(data, str) or not _UPID_RE.fullmatch(data):
return "wrong_returns_envelope"
return None
if isinstance(data, str):
return None
# Older PVE schemas sometimes mark list endpoints as opaque ``string``
# (empty properties, no description). Accept non-null structured data.
if isinstance(returns, dict) and not (
returns.get("description") or returns.get("properties") or returns.get("enum")
):
return None
return "wrong_returns_envelope"
if rtype in {"array", "object"} and data is None:
return "wrong_returns_envelope"
return None
def probe_major(
client: httpx.Client,
csrf: str,
@@ -209,6 +312,8 @@ def probe_major(
by_verb: dict[str, Counter[str]] = defaultdict(Counter)
failures: list[dict[str, Any]] = []
method_results: list[dict[str, Any]] = []
head_results: list[dict[str, Any]] = []
get_paths: set[str] = set()
methods: list[tuple[str, dict[str, Any]]] = [
(path["path"], method)
@@ -225,6 +330,7 @@ def probe_major(
params = body_for(method, path_template)
try:
if verb == "GET":
get_paths.add(path_template)
response = client.get(url, headers=headers, params=params or None)
elif verb == "PUT":
response = client.put(url, data=params or {}, headers=headers)
@@ -232,9 +338,7 @@ def probe_major(
response = client.post(url, data=params or {}, headers=headers)
elif verb == "DELETE":
# Proxmox accepts delete identifiers as form or query params.
response = client.request(
"DELETE", url, data=params or {}, headers=headers
)
response = client.request("DELETE", url, data=params or {}, headers=headers)
else:
by_verb[verb or "UNKNOWN"]["exception"] += 1
item = {
@@ -262,6 +366,15 @@ def probe_major(
text = response.text
bucket = classify(response.status_code, text)
envelope = classify_envelope(
status=response.status_code,
text=text,
method=method,
path_template=path_template,
verb=verb,
)
if envelope is not None:
bucket = envelope
by_verb[verb][bucket] += 1
ok = bucket not in CRITICAL_BUCKETS
item = {
@@ -270,12 +383,40 @@ def probe_major(
"status": response.status_code,
"bucket": bucket,
"ok": ok,
"returns": returns_type(method),
}
if not ok:
item["body"] = text[:240]
failures.append(item)
method_results.append(item)
# Synthetic HEAD for every GET path (Starlette mirrors GET handlers).
for path_template in sorted(get_paths):
url = f"/api2/json{render_path(path_template)}"
try:
response = client.request("HEAD", url)
text = response.text
bucket = classify(response.status_code, text)
except Exception as exc: # noqa: BLE001
bucket = "exception"
text = str(exc)[:200]
response = None
by_verb["HEAD"][bucket] += 1
ok = bucket not in CRITICAL_BUCKETS
item = {
"verb": "HEAD",
"path": path_template,
"status": getattr(response, "status_code", None),
"bucket": bucket,
"ok": ok,
"synthetic": True,
}
if not ok:
item["body"] = text[:240]
item["error"] = text[:200]
failures.append(item)
head_results.append(item)
version, _ = MAJOR_REVISIONS[major]
declared = int(snapshot.get("method_count") or len(method_results))
critical = len(failures)
@@ -309,13 +450,23 @@ def probe_major(
)
critical = len(failures)
verb_histogram = {
verb: {
"total": sum(counter.values()),
"buckets": dict(counter),
}
for verb, counter in sorted(by_verb.items())
}
return {
"major": major,
"version": snapshot.get("source_version") or version,
"apply": applied,
"declared": declared,
"probed": probed,
"head_probed": len(head_results),
"by_verb": {verb: dict(counter) for verb, counter in by_verb.items()},
"verb_histogram": verb_histogram,
"success_2xx": success_2xx,
"client_4xx": sum(
c.get("client_4xx", 0) + c.get("auth_401_403", 0) for c in by_verb.values()
@@ -325,6 +476,7 @@ def probe_major(
"ok": critical == 0,
"time": time.monotonic() - started,
"methods": method_results,
"head_methods": head_results,
}
pulumi-tests/pulumi/report.py
+41
View File
@@ -73,6 +73,33 @@ def _surface_error(major: dict[str, Any]) -> str:
return f"{len(fails)} critical: " + "; ".join(parts)
def _verb_histogram_rows(surface: list[dict[str, Any]]) -> list[str]:
rows: list[str] = []
for major in surface:
histogram = major.get("verb_histogram") or {}
if not histogram:
# Fall back to by_verb totals when slim payload lacks histogram.
by_verb = major.get("by_verb") or {}
histogram = {
verb: {"total": sum(buckets.values()), "buckets": buckets}
for verb, buckets in by_verb.items()
}
for verb, info in sorted(histogram.items()):
buckets = info.get("buckets") or {}
bucket_txt = ", ".join(
f"{name}={count}" for name, count in sorted(buckets.items()) if count
)
rows.append(
"<tr>"
f"<td>{html.escape(str(major.get('version') or major.get('major')))}</td>"
f"<td>{html.escape(str(verb))}</td>"
f"<td>{html.escape(str(info.get('total') or 0))}</td>"
f"<td><code>{html.escape(bucket_txt)}</code></td>"
"</tr>"
)
return rows
def write_html(payload: dict[str, Any], path: Path) -> None:
surface = payload.get("surface") or []
scenarios = payload.get("scenarios") or []
@@ -240,6 +267,20 @@ code {{ font-family: var(--mono); font-size: .85em; }}
</tbody>
</table>
<h2>Verb histogram (incl. synthetic HEAD)</h2>
<p class="lead">
Contract verbs GET/PUT/POST/DELETE count toward coverage.
HEAD is probed on every GET path for the matrix but is not part of declared/probed.
</p>
<table>
<thead><tr>
<th>Major</th><th>Verb</th><th>Total</th><th>Buckets</th>
</tr></thead>
<tbody>
{''.join(_verb_histogram_rows(surface)) or '<tr><td colspan="4" class="empty">No histogram</td></tr>'}
</tbody>
</table>
<h2>Critical surface failures</h2>
<table>
<thead><tr>
pulumi-tests/pulumi/run_suite.py
+33 -17
View File
@@ -22,9 +22,10 @@ sys.path.insert(0, str(ROOT))
def run_lifecycle(*, smoke: bool) -> dict[str, Any]:
from pulumi import automation as auto
import uuid
program_dir = PROGRAMS / "lifecycle"
stack_name = f"hxlife{os.getpid()}{int(time.time())}"
stack_name = f"hxlife-{uuid.uuid4().hex[:12]}"
os.environ.setdefault("PULUMI_CONFIG_PASSPHRASE", "hx-test-passphrase")
state = HERE / ".pulumi-state"
state.mkdir(parents=True, exist_ok=True)
@@ -43,6 +44,21 @@ def run_lifecycle(*, smoke: bool) -> dict[str, Any]:
started = time.monotonic()
stack = None
def _cleanup() -> None:
if stack is None:
return
try:
stack.destroy(on_output=lambda _: None)
except Exception as exc: # noqa: BLE001
message = str(exc).lower()
if "no stack named" not in message and "not found" not in message:
raise
try:
stack.workspace.remove_stack(stack_name)
except Exception:
pass
try:
# Pass env (incl. PULUMI_BACKEND_URL) at workspace creation — assigning
# workspace.env_vars after create_or_select_stack can lose stack selection.
@@ -67,11 +83,7 @@ def run_lifecycle(*, smoke: bool) -> dict[str, Any]:
item = outputs.get(key)
if item is None or getattr(item, "value", None) in (None, "", {}, []):
raise RuntimeError(f"lifecycle export {key!r} is empty")
stack.destroy(on_output=lambda _: None)
try:
stack.workspace.remove_stack(stack_name)
except Exception:
pass
_cleanup()
elapsed = time.monotonic() - started
if ids:
return {
@@ -89,15 +101,10 @@ def run_lifecycle(*, smoke: bool) -> dict[str, Any]:
stderr = getattr(exc, "stderr", None)
if stderr:
detail = f"{detail}\n{stderr}"
if stack is not None:
try:
stack.destroy(on_output=lambda _: None)
except Exception:
pass
try:
stack.workspace.remove_stack(stack_name)
except Exception:
pass
try:
_cleanup()
except Exception as cleanup_exc: # noqa: BLE001
detail = f"{detail}\ncleanup: {cleanup_exc}"
return {
"ok": False,
"scenarios": [
@@ -166,9 +173,17 @@ def main() -> int:
status = "ok" if item["ok"] else "FAIL"
print(
f" {status} PVE {item['version']}: declared={item['declared']} "
f"probed={item['probed']} critical={item['failure_count']} "
f"probed={item['probed']} head={item.get('head_probed', 0)} "
f"critical={item['failure_count']} "
f"2xx={item['success_2xx']} 4xx={item['client_4xx']}"
)
hist = item.get("verb_histogram") or {}
if hist:
parts = [
f"{verb}:{info.get('total', 0)}"
for verb, info in sorted(hist.items())
]
print(f" verbs: {', '.join(parts)}")
if not item["ok"]:
for fail in (item.get("failures") or [])[:10]:
print(
@@ -177,11 +192,12 @@ def main() -> int:
file=sys.stderr,
)
# Slim JSON: drop full method lists (keep failures)
# Slim JSON: drop full method lists (keep failures + verb histogram)
surface_slim = []
for item in surface:
slim = dict(item)
slim.pop("methods", None)
slim.pop("head_methods", None)
surface_slim.append(slim)
coverage = build_coverage(surface_slim, majors=list(majors))