vmware-api-simulator/docs/troubleshooting.md

Language / Язык: English | Русский

Troubleshooting

Ready stays unavailable

1. Confirm Postgres: make logs / Compose health.
2. Run make db-migrate.
3. Hit /health/ready again.

Task workers may retry until migrations catch up after a late migrate.

Unexpected HTTP 501

Every registered route should have a real handler or DB-backed stub — 501 should not appear for a known path. If you see it:

• Confirm you are calling the exact registered path/verb (check app/vsphere/rest/coverage.py or /docs).
• 501 from the optional legacy stub (ENABLE_PVE_STUB=true) is expected for undeclared PVE-style methods when CONTRACT_FALLBACK=error; it is unrelated to the vSphere surface.
• Report a regression — full registry coverage is expected on the native vSphere plane.

401 / 403

• Session expired (2-hour sliding TTL) or vmware-api-session-id header/cookie not sent.
• Basic auth malformed on /api/session (missing header, wrong user:password base64).
• Privilege denial — try administrator@vsphere.local vs readonly@vsphere.local to compare (see Authorization).
• In the Web UI, a 401 clears the stored session and switches the header badge to Guest with a “Session expired — sign in again” toast.

Ingress returns branded HTML 404 / nginx 405 instead of JSON

The simulator answers API errors as JSON (detail / error_type / messages). If you see a site HTML “page not found” or plain nginx 405 page, the Ingress / reverse proxy replaced the upstream body (often via custom-http-errors on the ingress-nginx controller).

Fix on the Ingress for this host (see helm/vmware-api-simulator/values-ingress-example.yaml):

annotations:
  nginx.ingress.kubernetes.io/proxy-intercept-errors: "false"
  nginx.ingress.kubernetes.io/custom-http-errors: "502,503"

Then re-check with Accept: application/json. A missing host should look like:

{
  "detail": {
    "error_type": "not_found",
    "messages": [
      {
        "default_message": "No such host ('host-999')",
        "id": "com.vmware.vapi.std.errors.not_found",
        "args": ["host-999"]
      }
    ],
    "data": { "host": "No such host ('host-999')" }
  }
}

Seeded host ids for small / large / big start at host-11. Cookbook VMs include vm-101 (web-01).

Correct authenticated mutation (vSphere Automation)

Session cookie/header + JSON body (not form-urlencoded):

SID=$(curl -sk -u 'administrator@vsphere.local:VMware1!' -X POST \
  "https://HOST/api/session" | tr -d '"')
curl -sk -X POST "https://HOST/api/vcenter/vm" \
  -H "Accept: application/json" \
  -H "Content-Type: application/json" \
  -H "vmware-api-session-id: $SID" \
  -d '{"name":"lab-vm","guest_os":"OTHER_GUEST_64","placement":{"folder":"group-v23","host":"host-11","datastore":"datastore-31","resource_pool":"resgroup-22"}}'

Task never finishes

• Inspect /api/cis/tasks/{task}.
• Check worker/simulator logs (make logs).
• Verify TASK_WORKER_CONCURRENCY > 0 and database leases can be claimed.
• Extremely high SIMULATION_TIME_SCALE slowdowns are unusual (higher = faster simulation); mis-set worker leases are more common culprits.

TLS / gateway failures

• Use host port 443 (gateway) for TLS clients — pyvmomi, govmomi, Terraform's hashicorp/vsphere provider, Pulumi.
• Set verify_ssl=False / allow_unverified_ssl=true only for the local self-signed development certificate.
• Inside Compose, target simulator:8080 directly (plain HTTP, no gateway).
• Seeded VM names for small are web-01, web-02, db-01, app-01, jumpbox — not Proxmox-style pve01/VMIDs.

Terraform / Pulumi / Ansible drift after reseed

Reseed replaces PostgreSQL inventory (MOIDs and VM names can change); external tool state does not update automatically. Refresh, import, or rebuild stacks after make seed.

Hot-swap "did nothing"

• Catalog browse ≠ apply. Use Apply as runtime or POST /ui/api/contract/apply?major=N.
• Applying a major changes the Web UI catalog / evidence view, not the live route table — the runtime always serves the full registered surface. See API versions.
• Apply is process-local; a Compose restart returns to the default (major 9).

Demo unload surprised you

POST /ui/api/demo/unload clears API-created state and loads small. Re-run make seed (or load demo-cluster again) to restore a richer fixture.