Using ipsonic-probe for Web Project Acceptance

This guide explains how a project team can use ipsonic-probe to define, execute, and document technical acceptance criteria for a website or web API. It is intended for customers, project owners, developers, operations teams, and acceptance testers.

1. Agree on the acceptance contract before implementation

At project kickoff, the customer and delivery team should agree on:

• The target environments (staging, pre-production, production) and stable endpoint URLs.
• Which behavior is contractually required and which is advisory.
• Expected HTTP status codes, headers, response-body markers, TLS policy, DNS records, and performance thresholds.
• Who maintains the YAML files, who approves changes, and how failures are handled.
• Which requirements cannot be proven by automated probes (visual quality, usability, business workflow, accessibility audits, security penetration testing).

Treat a version-controlled YAML file as an executable technical acceptance specification, not as the entire contractual acceptance process.

2. Translate requirements into testable assertions

Be precise: a body substring test does not prove a complete user journey, and a response-time threshold from one monitoring location is not a load or capacity test.

3. Define the acceptance YAML

Create acceptance/web-project-staging.yaml based on examples/91-web-project-acceptance.yaml.

name: "Customer Portal - Staging Acceptance"
url: "https://portal.example.com/health"
request:
  method: GET
  user_agent: "ipsonic-probe-acceptance/1.0"
  headers:
    Accept: "application/json"
timeouts:
  connect: 3
  response: 10
tls:
  verify: true
  min_valid_days: 30
tests:
  status:
    allow: [200]
  headers:
    exists: [Content-Type, Strict-Transport-Security]
    contains:
      Content-Type: ["application/json"]
  body:
    contains: ['"status"', '"healthy"']
    not_contains: ["Traceback", "Internal Server Error"]
monitoring:
  service_name: "Customer Portal Staging Acceptance"
  output: text
  on_failure: critical
  thresholds:
    response_time:
      warning: 2
      critical: 5
  assertions:
    - id: http.header
      severity: warning
    - id: http.status
      severity: critical
    - id: http.body
      severity: critical

This example uses documentation-only domains and must be adapted before execution. Check the exact assertion IDs against the version being deployed; an unmatched failure uses monitoring.on_failure.

Acceptance policy: A test with INFO or WARNING failures is not necessarily accepted. If a customer requires all mandatory criteria to pass, use critical for mandatory failures and agree separately whether WARNING is permissible. info produces an OK monitoring exit code, so do not use it for contractual must-pass criteria.

4. Keep environments separate

Use a directory structure such as:

acceptance/
  staging/
    web-home.yaml
    api-health.yaml
    dns-public.yaml
  production/
    web-home.yaml
    api-health.yaml
  evidence/

Avoid storing real passwords or API tokens in YAML. Use environment variables (token_env, username_env, password_env) and your organization's secret management process. Do not commit secrets or generated evidence containing sensitive responses.

For load-balanced services, use execution.backends to probe each agreed backend while preserving the public URL, HTTP Host header, and TLS SNI.

5. Execute the acceptance checks

Install dependencies using standard Linux distribution packages, without pip or external repositories. On Debian/Ubuntu:

sudo apt-get update
sudo apt-get install -y python3 python3-requests python3-yaml

On AlmaLinux/Rocky Linux:

sudo dnf install -y python3 python3-requests python3-pyyaml

For air-gapped hosts, use approved internal APT/DNF mirrors containing all dependencies. See the Linux installation section of the README.

Run a single acceptance file:

/usr/bin/python3 ipsonic-probe.py \
  --output text acceptance/staging/web-home.yaml

Run all agreed checks (shell glob expansion):

/usr/bin/python3 ipsonic-probe.py \
  --output json acceptance/staging/*.yaml \
  > acceptance/evidence/staging-results.json
status=$?
printf 'ipsonic-probe exit code: %s\n' "$status"

Keep both the output and the exit code. When using JSON, inspect the structure from your installed release before connecting it to downstream automation. Nagios and Icinga 2 plugin output is suitable for continuous operational monitoring; Checkmk output is intended for Checkmk local checks.

6. Record verifiable evidence

For each acceptance run, record:

1. Project, environment, target URLs, and test operator.
2. The approved YAML files and their SHA-256 hashes.
3. The ipsonic-probe release/version and system hostname.
4. Date/time (UTC), test output, and process exit code.
5. Any failures, exceptions, approved deviations, and retest results.
6. Customer and delivery-team sign-off (outside ipsonic-probe).

Example evidence commands:

date -u +'%Y-%m-%dT%H:%M:%SZ' > acceptance/evidence/executed-at.txt
sha256sum acceptance/staging/*.yaml > acceptance/evidence/yaml-sha256.txt
/usr/bin/python3 ipsonic-probe.py --output text \
  acceptance/staging/*.yaml \
  > acceptance/evidence/probe-output.txt 2>&1
printf '%s\n' "$?" > acceptance/evidence/exit-code.txt

Do not publish internal IP addresses, authorization tokens, detailed error pages, or other confidential evidence on a public static documentation site.

7. Decide on acceptance

Suggested acceptance rule:

• PASS: All agreed mandatory checks pass; no unapproved CRITICAL failures.
• CONDITIONAL PASS: Only explicitly approved non-blocking deviations remain, with owner and remediation deadline.
• FAIL: Any mandatory criterion fails, evidence is incomplete, or the tested environment differs from the agreed environment.

A monitoring exit code alone is not a legal or business acceptance decision. Obtain explicit customer sign-off on the criteria and the evidence.

8. Reuse the YAML after go-live

Move approved technical checks into scheduled Nagios, Icinga 2, or Checkmk monitoring. The same YAML can then detect regressions against the original acceptance baseline. Review thresholds and credentials before moving from staging to production.

9. Maintain documentation and the static site

The project website is generated from README.md and examples/. Keep this guide in the source tree and link to it from the README. Generate the static site offline:

./tools/generate-site.sh --output ./site-output

The generated site is documentation, not a live test-results dashboard. Store acceptance evidence separately, with appropriate access controls.

10. Acceptance checklist

• [ ] Customer-approved scope and target environment
• [ ] YAML criteria reviewed and versioned
• [ ] Required status, content, security, DNS, and timing checks included
• [ ] Mandatory criteria mapped to CRITICAL
• [ ] Secrets handled outside version-controlled YAML
• [ ] Tests executed from the agreed network vantage point
• [ ] Output, exit code, timestamp, and YAML hashes archived
• [ ] Deviations recorded and retested
• [ ] Customer acceptance decision signed off
• [ ] Approved checks transferred to operational monitoring

Limitations

ipsonic-probe provides active technical probes, not a browser-based end-to-end test framework. It does not replace user acceptance testing, visual regression testing, full accessibility audits, penetration tests, performance/load testing, or contract sign-off. DNSSEC record and resolver AD-flag checks do not independently validate the DNSSEC cryptographic chain.

Maintainer and disclaimer

Maintained by mati peterson (ipsonic@mailbox.org). Official project URL: https://ipsonic.com/probe/.

Disclaimer: This software and its examples are provided AS IS, without warranty of any kind. Use at your own risk. Review and validate configurations before production deployment.