HTTP(S) proxies and private CA chains

HTTP and HTTPS checks can use a corporate web proxy without extra Python packages. Proxy use is opt-in; the default proxy.mode: none ignores ambient proxy variables to keep checks deterministic.

• proxy.mode: environment uses the standard HTTP_PROXY, HTTPS_PROXY, ALL_PROXY, and NO_PROXY variables (including lowercase variants), as supported by Requests. NO_PROXY bypasses the proxy for matching hosts.
• proxy.mode: explicit uses proxy.http / proxy.https or their safer environment-based forms proxy.http_env / proxy.https_env. Values must be HTTP(S) proxy URLs, e.g. http://proxy.internal.example:3128. Avoid credentials in YAML and command output; use environment variables if proxy credentials are necessary.
• proxy.mode: none (default) disables environment proxy inheritance.
• tls.ca_file accepts a path to a PEM CA bundle; tls.ca_file_env reads the PEM bundle path from an environment variable. Use these for internal/private PKI. Certificate verification stays enabled by default. The bundle must be readable and contain the trusted issuing chain (normally root CA and any needed intermediates).

Example environment:

export PROBE_HTTPS_PROXY='http://proxy.internal.example:3128'
export PROBE_CA_BUNDLE='/etc/ipsonic-probe/private-roots.pem'
python3 ipsonic-probe.py examples/118-https-explicit-proxy-private-ca.yaml

For conventional environment proxy discovery, set HTTPS_PROXY and optionally NO_PROXY, and use example 117. For direct private-CA verification and certificate expiry inspection, use example 120.

Important limits: Proxy routing applies to HTTP(S) requests, not TCP or DNS checks. connection.resolve cannot be combined with proxy mode because the proxy determines upstream routing. Detailed TLS certificate/version/cipher inspection (min_valid_days, min_version, max_version, certificate, cipher) currently uses a direct TLS socket, so it is rejected when proxy mode is enabled rather than silently bypassing the proxy. Through a proxy, normal HTTPS certificate chain and hostname verification still occurs via Requests and the configured CA bundle. For an HTTPS-intercepting proxy, the trusted CA must match the certificate chain presented by that proxy. A custom CA bundle is a trust store, not a client certificate for mutual TLS authentication.

The existing native Linux packages python3, python3-requests, and python3-yaml (Debian/Ubuntu) or python3-pyyaml (AlmaLinux/Rocky) remain sufficient. No pip, SOCKS extension, or external package source is required.

ipsonic-probe — YAML-driven TCP and HTTP(S) validation

Why ipsonic-probe exists — decades of operating experience, one simple idea

Operating systems and applications for decades, we have worked with many monitoring products and integration approaches. Powerful tools have their place, but too often a straightforward health check comes with a complex stack, numerous dependencies, costly integration and ongoing maintenance.

We built ipsonic-probe around the Unix philosophy: KISS — Keep It Simple, Stupid. A small CLI, plain YAML files, standard Linux distribution packages and meaningful exit codes should be enough to verify a service and pass the result to an existing monitoring system.

Bridge development and operations: developers describe expected application behavior in readable YAML; operators reuse and adjust those checks for deployment, acceptance, production monitoring, patching, upgrades and migrations. As an application changes through its lifecycle, its checks can evolve without introducing another monitoring platform to maintain.

Complement, don't replace: ipsonic-probe fits alongside Nagios, Icinga 2 and Checkmk. It is not a replacement for metrics, logging, tracing or a full monitoring platform. The goal is a focused, low-resource, low-dependency tool that does one job well.

A lightweight, YAML-driven network validation tool for Linux, including Debian, Ubuntu, AlmaLinux, and Rocky Linux. It supports TCP, HTTP(S), DNS, and monitoring integrations in connected or air-gapped environments. Installation uses distribution-managed packages only: no pip, PyPI, virtual environment, or third-party package repository is required.

First test — TCP 127.0.0.1:22

New to ipsonic-probe? Follow the Hello first TCP test guide from release download and Linux package installation to your first local tcp://127.0.0.1:22 check. The ready-to-run example is examples/138-hello-first-tcp-ssh.yaml.

Best practices

See the ipsonic-probe best-practice guide for reliable monitoring design, alert severities, secrets, private CAs, CI/CD integration, application lifecycle use and operational runbooks.

Installation and release downloads

• Raspberry Pi OS installation — ARM-based Debian-family systems.
• Ansible installation — optional host provisioning from an Ansible controller.
• Release downloads — versioned tar.gz release archives and SHA-256 checksums, generated by tools/build-release.sh.

The release includes a requirements.txt documentation file listing standard Linux distribution packages. It is not a pip requirements file: install dependencies using APT or DNF, never pip.

Requirements and installation (Linux)

Runtime dependencies are Python 3, Requests, and PyYAML, all provided as standard distribution packages. Requests and PyYAML are Python libraries, but no external package source is necessary. The DNS implementation and website generator use the Python standard library.

Debian / Ubuntu

Use the official distribution repositories or an approved internal APT mirror:

sudo apt-get update
sudo apt-get install -y python3 python3-requests python3-yaml

Raspberry Pi OS (Raspbian)

Raspberry Pi OS is Debian-based. Install from the official Raspberry Pi OS repositories or an approved internal APT mirror:

sudo apt-get update
sudo apt-get install -y python3 python3-requests python3-yaml ca-certificates

The same packages work on supported ARM 32-bit and ARM 64-bit releases where available in the configured repository. No pip or external repositories are required. For automated host preparation, see Raspberry Pi OS installation. For optional automated provisioning, see Ansible installation. Ansible runs on a separate controller, not on the probe host.

AlmaLinux / Rocky Linux

Use the official distribution repositories or an approved internal DNF mirror:

sudo dnf install -y python3 python3-requests python3-pyyaml

Air-gapped installation

Configure a local APT/DNF mirror containing the required packages and their transitive dependencies. On RPM-based systems, for example:

sudo dnf --disablerepo='*' --enablerepo='YOUR_LOCAL_REPO_ID' install -y \
  python3 python3-requests python3-pyyaml

On Debian/Ubuntu, configure an internal APT source and use apt-get install as above. If no mirror is available, prepare a complete set of matching DEB/RPM packages on a connected machine and transfer them through the approved process. The project ZIP contains no DEB/RPM files. Do not use pip or public repositories on an isolated host.

Verify installation

python3 -c 'import requests, yaml, urllib3; print("OS packages OK")'
python3 ipsonic-probe.py --help

Probes need network connectivity only to their configured targets. No internet service is required by the framework itself.

Why use ipsonic-probe for existing applications?

ipsonic-probe is not limited to new software projects. Application operators can monitor already installed applications from outside the target, without changing application code or installing an agent. This is particularly useful for legacy systems, commercial/vendor products, WordPress sites, internal portals, reverse-proxy frontends and private-PKI services.

• Monitor the user-visible contract: check an expected HTTP response, body marker, TLS trust and certificate expiry, DNS records and response time rather than only process state or port availability.
• Start with one YAML: capture a known-good baseline and reuse it after maintenance, patches, application upgrades, DNS changes and data-center migrations.
• Fit existing operations: execute from cron or a deployment script, or consume Nagios, Icinga 2 and Checkmk output with existing alerting. No replacement monitoring platform is required.
• Avoid target changes: probe from a monitoring host or approved network location. For internal services, configure HTTP(S) proxies and private CA trust where necessary.
• Make handover easier: keep small, reviewable YAML checks with application runbooks and version-control changes to expected behavior.
• Help SecOps spot drift: monitor certificate validity, HTTPS behavior, DNS answers and selected security headers as observable signals; these are not substitutes for vulnerability scans or a SIEM.
• Keep the dependency inventory small: use Linux distribution packages (python3, Requests and PyYAML), without pip, hosted services, databases, message brokers or a container orchestrator. The OS and transitive packages still belong in the SBOM.

Incremental adoption for operators

1. Choose a business-critical installed application and identify its stable health or public URL.
2. Copy a relevant YAML from examples/ and set the expected status, content, latency and TLS policy.
3. Run it from the actual monitoring network; verify proxy and private-CA settings if needed.
4. Review failure severity and expected maintenance behavior with the application owner.
5. Schedule the check through an existing monitoring platform or scheduler; retain YAML in version control.
6. Reuse the check after upgrades, infrastructure changes and disaster-recovery tests.

Across the lifecycle: development → CI/CD → deployment → technical acceptance → operations → security monitoring → upgrade/migration → retirement. Use ipsonic-probe as a focused assertion runner alongside, not instead of, full observability and security tools.

Microsoft Exchange infrastructure monitoring

Use the Exchange infrastructure monitoring guide and examples 121–137 to monitor OWA, ECP, Autodiscover, EWS, MAPI, ActiveSync, SMTP, IMAP, POP3, mail DNS and Exchange HTTPS backends. These are layered reachability and HTTPS/DNS checks, not authenticated mailbox or end-to-end mail-delivery tests. Enable only services present in your deployment.

Web project acceptance guide

For a step-by-step customer acceptance workflow, see WEB-PROJECT-ACCEPTANCE.md and example 91. The guide explains requirements, executable YAML acceptance criteria, evidence collection, sign-off, and operational reuse.

Quick start

/usr/bin/python3 ipsonic-probe.py examples/02-http-get-status-allow.yaml
/usr/bin/python3 ipsonic-probe.py /etc/httptest/*.yaml
printf 'exit code: %s\n' "$?"

Examples are templates, not an immediately runnable test suite: change example hostnames, URLs, paths, expected response contents, and credentials to match your service. .invalid hostnames are intentionally nonresolvable placeholders. Do not run the whole examples/*.yaml directory without adapting it.

Exit status: 0 when every configuration passes; 1 if any configuration fails or raises an error; argparse may return 2 for invalid CLI usage.

Configuration reference

name: Example name                   # optional, informational
url: https://service.example.invalid/health  # required; tcp://host:port also accepted
request:                              # HTTP(S) only; optional
  method: GET                         # GET or POST, default GET
  user_agent: httptest/1.0            # optional
  headers:                            # optional request headers
    Accept: application/json
  json:                               # optional JSON-serializable POST payload
    check: ping
  # body: 'raw request body'          # alternative to json
  follow_redirects: false             # default false

auth:                                 # optional, HTTP(S) only
  type: bearer                        # none | basic | bearer
  token_env: HTTPTEST_API_TOKEN       # or token: literal
  # For basic: username/password or username_env/password_env

timeouts:
  connect: 3                          # seconds, default 3
  response: 10                        # seconds, default 10

tls:                                  # HTTPS only
  verify: true                        # default true
  min_valid_days: 30                  # optional, only checked when verify=true

tests:                                # all assertions optional
  status:
    allow: [200, 204]                 # if provided, status must be in list
    deny: [401, 403, 500]            # if provided, status must not be in list
  headers:
    exists: [Content-Type]            # header names are case-insensitive
    absent: [X-Debug]
    contains:                          # each listed substring must appear
      Content-Type: [application/json]
    not_contains:                      # none of these substrings may appear
      Server: [nginx/1.14]
  body:
    contains: ['"status"']            # case-sensitive substring checks
    not_contains: [traceback]

Assertion semantics

• All specified assertions are combined using AND. Every failed assertion is printed.
• status.allow and status.deny can be used independently or together. An empty allow list rejects every status.
• Header names are case-insensitive; header values are checked by case-sensitive substring match. contains on a missing header fails; not_contains on a missing header passes unless exists is also set.
• Body assertions operate on decoded response text (requests.Response.text), not raw bytes, JSON paths, regular expressions, or exact equality.
• HTTP errors such as 404 are only failures if the configured assertions reject them. Without status assertions, receiving an HTTP response is sufficient.
• A tcp:// URL only checks whether a TCP connection can be established; HTTP request, auth, TLS, and response assertions do not apply.

Authentication

auth.type: basic requires exactly one of username / username_env and one of password / password_env. auth.type: bearer requires exactly one of token / token_env. Missing/empty environment variables fail the configuration. Do not define a manual Authorization request header together with auth.type: basic or bearer.

export HTTPTEST_USER='monitor'
export HTTPTEST_PASSWORD='example-secret'
export HTTPTEST_API_TOKEN='example-token'
/usr/bin/python3 ipsonic-probe.py examples/20-http-bearer-env.yaml

Use secret injection (for example Ansible Vault plus no_log: true on sensitive tasks). Avoid literal credentials in version control. Bearer tokens and Basic credentials should normally be sent only over HTTPS.

TLS behavior

• tls.verify: true uses the OS/Requests trust configuration for certificate chain and hostname validation.
• tls.verify: false disables both certificate chain and hostname validation. Only for isolated test systems, never as a production default.
• tls.min_valid_days performs an additional TLS handshake to inspect the leaf certificate expiry; this requires verify: true. The script uses the system trust store for this check. The extra handshake is separate from the measured HTTP response duration.
• For internal CAs, deploy the CA into the OS trust store using your organization's trust-management procedure (typically update-ca-trust), and ensure the Python Requests CA bundle is configured appropriately. On RPM installations, confirm the effective CA path with python3 -c 'import requests; print(requests.certs.where())'. No per-YAML custom CA file option is implemented.

Timeout behavior and limitations

• timeouts.connect sets the socket connection timeout used by Requests and the TCP check.
• timeouts.response is the maximum wait for the HTTP operation from the caller's perspective, implemented with a worker thread and a deadline. On timeout the CLI returns an error without waiting for the worker. The underlying request/thread is not forcibly terminated, so this is not a hard socket cancellation guarantee.
• Requests additionally uses response as its read-inactivity timeout. DNS resolution, the separate certificate-expiry handshake, and some OS socket operations may not obey the same overall deadline.
• A TCP connection is timed using connect; its measured elapsed time is also compared with response.
• For strict wall-clock enforcement across the entire process, use an external supervisor such as timeout 15s /usr/bin/python3 ipsonic-probe.py ... (GNU coreutils).

Output example

PASS /etc/httptest/health.yaml (0.081s)
FAIL /etc/httptest/api.yaml (0.142s)
  - HTTP 503 not in allowed statuses [200]
  - Body does not contain 'healthy'
ERROR /etc/httptest/tcp.yaml: ConnectionRefusedError: [Errno 111] Connection refused

Ansible example

- name: Run httptest checks
  ansible.builtin.command:
    argv:
      - /usr/bin/python3
      - /opt/httptest/httptest.py
      - /etc/httptest/api.yaml
  environment:
    HTTPTEST_API_TOKEN: "{{ vault_api_token }}"
  register: httptest_result
  changed_when: false
  no_log: true  # prevents sensitive environment values leaking to logs

To retain non-sensitive output for monitoring, separate the secret-bearing execution task from a safe summary/reporting step. no_log: true hides the registered output in normal Ansible logs.

Example catalog

All files are located in examples/. These are representative feature and interaction combinations, not a Cartesian product of every possible parameter/value. You can freely combine independent assertions in a single file; see examples 11, 25, and 26.

• 01-tcp-connect.yaml — TCP connect
• 02-http-get-status-allow.yaml — GET allowed status codes
• 03-http-get-status-deny.yaml — GET denied status codes
• 04-http-status-allow-and-deny.yaml — Combined status acceptance and rejection
• 05-http-header-exists.yaml — Required response headers
• 06-http-header-absent.yaml — Forbidden response headers
• 07-http-header-contains.yaml — Required response header substrings
• 08-http-header-not-contains.yaml — Forbidden response header substrings
• 09-http-body-contains.yaml — Required body substrings
• 10-http-body-not-contains.yaml — Forbidden body substrings
• 11-http-headers-body-combined.yaml — Combined HTTP acceptance checks
• 12-http-post-json.yaml — POST JSON payload
• 13-http-post-form.yaml — POST form-encoded payload
• 14-http-user-agent-custom-headers.yaml — Custom User-Agent and request headers
• 15-http-follow-redirects.yaml — Follow redirects
• 16-http-do-not-follow-redirects.yaml — Inspect redirect response
• 17-http-basic-auth-literal.yaml — HTTP Basic auth literal (demo only)
• 18-http-basic-auth-env.yaml — HTTP Basic auth from environment
• 19-http-bearer-literal.yaml — Bearer token literal (demo only)
• 20-http-bearer-env.yaml — Bearer token from environment
• 21-https-verify-secure.yaml — Verify TLS certificate and hostname
• 22-https-insecure.yaml — Disable TLS validation for isolated preprod ONLY
• 23-https-min-valid-days.yaml — Verify TLS and require 30 days certificate validity
• 24-http-timeouts.yaml — Short connection and total response deadline
• 25-https-post-bearer-full.yaml — Full HTTPS POST with bearer token and assertions
• 26-https-post-basic-full.yaml — HTTPS POST with Basic authentication
• 27-http-no-assertions.yaml — HTTP connectivity only
• 28-https-insecure-bearer.yaml — Insecure preprod HTTPS with bearer auth

Operational notes

• The tool intentionally makes one HTTP request per YAML file, plus an extra TLS handshake if certificate lifetime is checked.
• A URL can include a custom port. No curl --resolve-style DNS override or SNI override is implemented.
• Redirects are disabled by default. follow_redirects: true follows them. Review redirect destinations carefully when using credentials.
• The HTTP client disables environment proxy configuration (session.trust_env = False), which also affects some Requests environment settings.
• This tool does not currently provide JSON/JUnit output, regex/JSONPath assertions, client certificates, or retries.
• Keep test YAML files access-controlled if they contain credentials.

Extended HTTP/HTTPS checks (version 2)

The following checks use the existing Python requests, urllib3 (a dependency of requests), and PyYAML RPM packages. No pip installation or outbound internet access is required. Examples 29–36 illustrate the additions.

Per-request DNS override (preproduction)

url: https://api.example.com/health
connection:
  resolve:
    api.example.com: 10.20.30.40

Connects directly to the supplied IP while retaining the original URL hostname for the HTTP Host header, TLS SNI, and certificate hostname verification. Only a single mapping matching the URL hostname is supported. Overrides apply to the initial target; redirect destinations are not automatically overridden. Proxies inherited from the environment are disabled. IPv4 and IPv6 literals are accepted. TLS certificate validity is still checked against the URL hostname.

Redirect assertions

request:
  follow_redirects: true
tests:
  redirects:
    max_count: 2
    require_https: true
    allowed_hosts: [example.com, www.example.com]
    final_url: https://www.example.com/new

max_count checks followed redirects; require_https checks all URLs in the followed chain; allowed_hosts checks the hostname of every URL in the chain. final_url compares the resulting URL exactly. With follow_redirects: false, checks only observe the initial response, and max_count will be zero. Redirect checks do not prevent a redirect before it occurs: do not enable following redirects to untrusted hosts with credentials.

Response metadata

tests:
  response:
    min_bytes: 20
    max_bytes: 1048576
    content_type: application/json

Size is the length of the downloaded/decoded response bytes. Content type matches the media type case-insensitively, ignoring parameters such as charset. This is a post-download assertion, not a streaming download-size cap.

Structured JSON assertions

tests:
  json:
    paths:
      - {path: status, equals: healthy}
      - {path: database.connected, equals: true}
      - {path: services, min_items: 2}
      - {path: version, exists: true}
      - {path: error, exists: false}

Paths are dot-separated dictionary keys; numeric segments address list indexes. This is a simple path selector, not full JSONPath. equals compares both JSON type and value; min_items / max_items apply to lists or objects. Keys containing dots are not supported.

Cookies and HSTS

tests:
  cookies:
    require:
      sessionid:
        secure: true
        httponly: true
        samesite: Lax
  security:
    hsts_min_age: 31536000

Cookie rules check the named cookie's attributes on the final response. HSTS parses numeric max-age from the final response's Strict-Transport-Security header. Cookie attributes are parsed using Python's http.cookies.SimpleCookie; unusual or non-standard cookie syntax may not be recognized.

Linux air-gapped deployment

dnf --disablerepo='*' --enablerepo=YOUR_INTERNAL_REPO install -y python3 python3-requests python3-pyyaml
python3 httptest.py examples/36-combined-preprod.yaml
python3 httptest.py examples/*.yaml

Use the appropriate standard packages for your distribution (python3-yaml on Debian/Ubuntu; python3-pyyaml on AlmaLinux/Rocky Linux). The archive does not bundle DEB/RPM packages. Validate the installed Python and urllib3 versions. Example hostnames and addresses are placeholders.

Limitations

The total request deadline uses a background worker; it bounds the CLI's waiting time but does not guarantee immediate termination of the underlying socket operation. TLS min-valid-days uses a separate certificate handshake. Redirect chains, JSON paths, and cookies are evaluated after the request completes. No direct TLS socket inspection, cipher auditing, OCSP, or HTTP/2 support has been added.

Extended checks (v2)

All features below are implemented in ipsonic-probe.py. See examples/29- through examples/40- for individual and combined YAML files. One YAML file describes one request and multiple assertions (logical AND).

Direct target IP / curl --resolve equivalent

url: https://api.example.invalid/health
connection:
  resolve:
    api.example.invalid: 10.20.30.40

This maps the URL hostname to a literal IPv4/IPv6 destination without modifying DNS or /etc/hosts. The HTTP Host header, TLS SNI and certificate hostname remain the URL hostname. Proxy environment variables are ignored (Session.trust_env = False). The mapping must contain exactly the URL hostname. Automatic redirects cannot be combined with connection.resolve to prevent a redirected connection from silently bypassing the override. Use separate YAML files to test redirects in this situation. This override is for HTTP(S), not TCP.

TLS version, certificate and cipher assertions

tls:
  verify: true
  ca_file: /etc/pki/httptest/internal-ca.pem # optional custom CA bundle
  min_valid_days: 30
  min_version: TLSv1.2
  max_version: TLSv1.3
  certificate:
    issuer_contains: Example CA
    subject_contains: api.example.invalid
    san_contains: [api.example.invalid]
    # fingerprint_sha256: '64 lowercase hex characters'
  cipher:
    allow: [TLS_AES_256_GCM_SHA384, TLS_AES_128_GCM_SHA256]
    deny: [AES128-SHA]

TLS checks establish a separate handshake before the HTTP request; they validate that connection's negotiated version, cipher and peer certificate. min_version/max_version restrict the probe's TLS context; the actual HTTP connection may negotiate independently. cipher.allow and cipher.deny compare the OpenSSL negotiated cipher name (not IANA names). Fingerprint is SHA-256 of DER-encoded leaf certificate, compared case-insensitively with optional colon separators. SAN checks are exact matches for DNS SAN entries. Issuer/subject are substring checks on flattened distinguished names. CA trust and hostname validation use the OS/OpenSSL trust configuration or ca_file. min_valid_days, issuer/subject/SAN assertions require verify: true; fingerprint also works when insecure. Certificate revocation / OCSP, all supported TLS versions, and exhaustive server cipher enumeration are not tested. A minimum-version assertion demonstrates that a connection can negotiate at or above the configured minimum; it does not prove the server rejects older protocols.

Redirect policy

request:
  follow_redirects: true
tests:
  redirects:
    max_count: 2
    require_https: true
    allowed_hosts: [api.example.invalid]
    final_url: https://api.example.invalid/health

Redirect assertions inspect response.history and the final URL. require_https checks every URL in the chain, including the initial URL. allowed_hosts checks all visited URL hostnames. Do not enable redirects to untrusted hosts when sending credentials. Redirect behavior is not a substitute for an open-redirect security scan.

Response size and MIME type

tests:
  response:
    min_bytes: 10
    max_bytes: 1048576
    content_type: application/json

Byte length is measured after Requests content decoding (including gzip decompression). MIME comparison ignores parameters such as charset=UTF-8 and is case-insensitive. Response download is buffered; max_bytes is an assertion after download, not a streaming memory cap.

Structured JSON paths

tests:
  json:
    paths:
      - {path: status, equals: healthy}
      - {path: database.connected, equals: true}
      - {path: services.0.name, exists: true}
      - {path: services, min_items: 2, max_items: 10}
      - {path: error, exists: false}

Path syntax is simple dot-separated object keys and zero-based array indices, not full JSONPath: no filters, wildcards or escaping dots in keys. equals uses exact JSON value and type comparison (boolean is distinct from integer). Missing paths fail unless exists: false is requested.

Cookie flags and HSTS

tests:
  cookies:
    require:
      session:
        secure: true
        httponly: true
        samesite: Strict
  security:
    hsts_min_age: 31536000

Cookie assertions inspect Set-Cookie headers on the final response. Secure, HttpOnly and SameSite are checked per named cookie. HSTS verifies the numeric max-age directive, not preload eligibility or subdomain inclusion. Cookie parsing uses Python http.cookies.SimpleCookie; malformed or unusual cookie headers may not parse as expected.

Air-gapped Linux constraints

Runtime dependencies remain only OS-managed packages: python3, python3-requests, and python3-yaml (Debian/Ubuntu) or python3-pyyaml (AlmaLinux/Rocky Linux), plus their distribution-managed dependencies. Python standard library provides TLS, socket, JSON, hashing and cookie parsing. No pip, cryptography, jsonpath-ng, httpx, external openssl command or online service is required. Check /usr/bin/python3 and /usr/bin/python3 -c 'import requests, yaml, urllib3' on the target. The archive contains source code, examples and tests but no DEB/RPM packages. Use an internal repository or transfer dependency-complete packages matching the OS release and CPU architecture.

Run offline unit tests:

/usr/bin/python3 -m unittest discover -s tests -v

Operational limitations

timeouts.response is an outer deadline for the CLI result implemented using a worker thread. When the deadline expires, the thread may continue until the socket's own timeout; it is not a guaranteed immediate cancellation of the underlying socket. The TLS preflight uses a separate connection and is not included in the measured HTTP elapsed time. Avoid pointing these examples at production systems until URLs, secrets and policies are reviewed. TLS probe supports TLS 1.2 and 1.3 configuration with the OpenSSL library supplied by the Linux distribution.

Monitoring integration (Nagios, Icinga 2, Checkmk)

Each YAML file can specify a monitoring mapping. The command-line --output option overrides monitoring.output for every supplied file. Supported values: text (default), nagios, icinga2, checkmk, json. Output formatting does not change the assertions or connection behavior.

monitoring:
  service_name: "API Health"
  output: nagios
  thresholds:
    response_time:
      warning: 2      # seconds, strictly greater => WARNING
      critical: 5     # seconds, strictly greater => CRITICAL

Thresholds are optional, numeric, positive and finite. When both are set, warning must be less than critical. Failed assertions take precedence over latency warnings and produce CRITICAL. Invalid YAML, invalid configuration, connection errors, and timeouts produce UNKNOWN. A configured response-time threshold is evaluated against the HTTP operation measured by the engine; the separate TLS inspection handshake is not included. timeouts.response remains an independent operation deadline; a deadline error is UNKNOWN rather than a latency threshold violation.

Nagios and Icinga 2 use the conventional plugin output with performance data:

OK - API Health: all assertions passed | response_time=0.184s;2;5;0;

Checkmk output follows the local-check format (one line per service):

0 "API Health" response_time=0.184s;2;5;0; all assertions passed

For Checkmk, the leading state field is authoritative. Deploy the command as a local check under the appropriate agent local-check directory and run it with --output checkmk. Avoid using --output json for a Checkmk local check. Checkmk performance-data parsing and unit handling should be verified with your installed Checkmk version. The JSON mode emits one JSON object per file (JSON Lines), not one combined JSON array.

/usr/bin/python3 ipsonic-probe.py --output nagios examples/41-monitoring-nagios.yaml
/usr/bin/python3 ipsonic-probe.py --output icinga2 examples/41-monitoring-icinga2.yaml
/usr/bin/python3 ipsonic-probe.py --output checkmk examples/41-monitoring-checkmk.yaml
/usr/bin/python3 ipsonic-probe.py --output json examples/41-monitoring-json.yaml

Nagios / Icinga 2: invoke a single YAML file per monitored service so that each service has its own status and performance data. When passing multiple YAML files, the CLI prints one result line per file and returns the maximum status code; this is suitable for batch use but is not a single-service Nagios plugin response. Checkmk: multiple files may be used to generate multiple local-check lines, but service names must be unique. Do not mix output modes in one Checkmk local-check invocation.

service_name is optional and defaults to name or ipsonic-probe. Keep names unique and free of quotes for Checkmk integration. See examples/41-monitoring-*.yaml, 42-monitoring-tcp.yaml, 43-monitoring-thresholds.yaml, and 44-monitoring-no-thresholds.yaml.

Packaging

The distribution provides ipsonic-probe.py and a compatibility copy named httptest.py. Both run on system Python with distribution-managed Requests and PyYAML; no pip or external package repositories are required. The archive contains no DEB/RPM binaries.

Assertion failure severity (monitoring.on_failure)

Each YAML configuration represents one test service. Set monitoring.on_failure to control the monitoring status when one or more assertions fail. The default is critical.

monitoring:
  service_name: "API Health"
  output: icinga2       # text, nagios, icinga2, checkmk, json
  on_failure: warning   # info, warning, critical (default: critical)
  thresholds:
    response_time:
      warning: 2
      critical: 5

This option applies to all assertions in the YAML file, not individual assertion expressions. For different severities, use separate YAML files/services. Network, TLS handshake, timeout, and configuration exceptions remain UNKNOWN (3) rather than being downgraded. Response-time monitoring thresholds remain independent and may raise WARNING/CRITICAL if no assertion failed. info is a descriptive OK state, not a fifth Nagios/Icinga status.

Per-assertion monitoring severity

monitoring.assertions provides ordered, per-failure severity overrides. Each rule matches a literal, case-sensitive substring of the generated failure message (contains). The first matching rule wins. If no rule matches, monitoring.on_failure applies; its default is critical.

monitoring:
  output: icinga2
  service_name: API Health
  on_failure: critical
  assertions:
    - contains: "Missing header: X-Request-ID"
      severity: info
    - contains: "Body does not contain 'healthy'"
      severity: warning
    - contains: "HTTP 503"
      severity: critical

A single failed assertion can therefore be INFO (Nagios/Icinga OK exit 0), WARNING (exit 1), or CRITICAL (exit 2). The highest severity of all failures and independently evaluated response-time thresholds determines the final state. Errors executing the check remain UNKNOWN (exit 3). Individual failure severities are included in JSON as assertion_results and are prefixed in human-readable monitoring output. Matching is based on diagnostic text, so review rules after upgrading the engine; rules are not a general expression language. Avoid matching broad fragments that may affect unrelated checks.

Static documentation site (offline)

The static documentation website is generated from the current README.md and every YAML file under examples/. It is documentation, not a live monitoring dashboard. No pip packages, Markdown tools, JavaScript libraries, CDN, network access, or external assets are needed.

./tools/generate-site.sh
# or choose an explicit separate directory:
./tools/generate-site.sh --output /srv/www/ipsonic-probe-docs

The default output directory is site-output/ at the project root. It contains index.html (rendered README), README.md (original source), examples/index.html (example catalog), one HTML page per YAML example, downloads/examples/ (unmodified YAML files), and assets/style.css.

Serve the generated output directory as static files with an existing Apache/nginx installation, or open index.html directly. The generator refuses to overwrite an existing directory unless it contains its own .ipsonic-generated marker; subsequent runs regenerate it from source. Do not edit generated files, as they will be replaced. Markdown rendering supports common headings, lists, tables, code fences, links and emphasis; it is intentionally a lightweight renderer, not a full CommonMark implementation.

DNS record validation (A, AAAA, CNAME, TXT)

ipsonic-probe supports DNS-only YAML files (no url field). It sends DNS wire-protocol queries using Python's standard library; no dnspython or pip installation is required. DNS checks use the same monitoring outputs, response-time thresholds, monitoring.on_failure, and per-assertion severity matching as HTTP checks.

name: Internal DNS

dns:
  name: api.example.com
  nameserver: 10.10.10.53  # optional IP literal; IPv4 or IPv6
  port: 53                 # optional, default 53
  transport: auto          # auto (UDP with TCP truncation fallback), udp, tcp
  timeout: 3               # seconds
  records:
    - type: A
      exists: true
      equals: [10.20.30.40]
      not_equals: [127.0.0.1]
      min_count: 1
      max_count: 4
      min_ttl: 60
    - type: AAAA
      exists: false
    - type: TXT
      contains: ["v=spf1"]
      not_contains: ["+all"]
monitoring:
  service_name: Internal DNS
  output: icinga2
  on_failure: critical

DNS configuration reference

The DNS client requests one record type per dns.records entry. A CNAME check tests the CNAME response itself; it does not recursively chase aliases. TXT character-string fragments belonging to one TXT record are concatenated before assertion checks. Comparison of record content is case-sensitive except for DNS owner names. DNS response codes such as NXDOMAIN are assertions; socket errors, timeouts and malformed responses result in UNKNOWN.

Air-gapped operation: DNS queries only target the selected nameserver, with no HTTP requests or external dependencies. An explicit resolver must be an IP literal to avoid accidental DNS bootstrap queries to another resolver. Examples use documentation-only IPs and domains and must be adapted. DNSSEC validation, DoH, DoT, SRV, MX and iterative resolution are not implemented.

Examples: 50-dns-a.yaml through 57-dns-multiple-records.yaml. Run them using python3 ipsonic-probe.py examples/54-dns-custom-nameserver.yaml. Regenerate the static documentation website with ./tools/generate-site.sh.

Extended DNS records and execution controls

Supported DNS resource record types: A, AAAA, CNAME, TXT, MX, NS, SOA, PTR, SRV, CAA, DS, DNSKEY, RRSIG. The DNS wire implementation uses the Python standard library; no dnspython or pip is needed.

Records are represented as normalized text for equals, contains, and other existing assertions:

• MX: priority hostname.
• SOA: mname. rname. serial refresh retry expire minimum
• SRV: priority weight port target.
• CAA: flags tag value
• DS: keytag algorithm digesttype HEXDIGEST
• DNSKEY: flags protocol algorithm BASE64KEY
• RRSIG: typecovered algorithm labels originalttl expiration inception keytag signer. HEXSIGNATURE

DNSSEC limitation: dnssec: true sets EDNS DO and enables inspection of DNSKEY, DS and RRSIG data. authenticated_data: true checks the AD flag reported by the selected recursive resolver. This is not independent cryptographic DNSSEC chain validation. Use a trusted validating resolver when depending on AD; DNSSEC failures may surface as SERVFAIL. No external resolver is contacted unless explicitly configured.

Compare nameservers

dns:
  name: api.example.com
  compare_nameservers: [192.0.2.53, 192.0.2.54]
  require_identical: true
  records:
    - type: A
      exists: true
      max_response_time: 2.0

The comparison checks RCODE and sorted, unique answer values; TTLs are deliberately ignored. Different answers from load-balanced DNS may legitimately differ.

Retries and multiple backends

execution:
  retries: 2
  retry_delay: 1
  success_required: 2
  backends:
    - name: node-a
      resolve: {api.example.com: 192.0.2.10}
    - name: node-b
      resolve: {api.example.com: 192.0.2.11}

retries means additional attempts; default 0. success_required counts successful attempts per backend (default 1). All backends must satisfy their success quota. Backends may override url or resolve for HTTP, or nameserver for DNS. Attempts execute sequentially. With retries configured, connection errors are treated as unsuccessful attempts; without execution, legacy UNKNOWN handling is retained. Avoid configuring excessive retry counts in latency-sensitive monitoring checks.

Stable assertion category IDs

monitoring:
  on_failure: critical
  assertions:
    - id: dns.a.comparison
      severity: warning
    - id: http.body
      severity: info

Rules accept exactly one of id or legacy contains, plus severity. IDs currently identify assertion categories, not each individual YAML assertion. DNS identifiers include dns.<recordtype>.record, .rcode, .ttl, .comparison, .dnssec_ad, .response_time. HTTP categories include http.status, http.body, http.header, response.time and generic probe.assertion. The first matching severity rule wins; unmatched failures use monitoring.on_failure (default critical). JSON output exposes assertion IDs.

Air-gapped Linux

Install python3, python3-requests, and the distribution YAML package (python3-yaml on Debian/Ubuntu; python3-pyyaml on AlmaLinux/Rocky Linux) from approved local repositories, including dependencies. No pip, runtime internet access, or third-party DNS module is required. DNS nameserver addresses must be IP literals. Examples use documentation-only IP ranges and must be adapted.

Generate the static documentation site

./tools/generate-site.sh --output ./site-output

The site generator reads this README and the YAML examples, and writes its output to a separate directory.

Extended configuration examples (71–90)

These additional YAML templates illustrate the extended DNS record types, DNSSEC metadata, resolver comparison, retry success quotas, independent backends, stable assertion-category severity IDs and monitoring outputs. All IP addresses in 192.0.2.0/24 are documentation placeholders. Replace names, records and addresses before running checks. authenticated_data trusts the resolver AD flag and is not independent DNSSEC cryptographic validation.

monitoring.output supports text (default), json, nagios, icinga2, and checkmk. CLI --output overrides YAML. To test all formats against the same YAML, run python3 ipsonic-probe.py --output FORMAT examples/90-monitoring-all-outputs.yaml with each supported FORMAT.

WordPress website checks

Start with 92-wordpress-homepage.yaml for a public WordPress homepage smoke test. It checks HTTPS, certificate validity, successful HTML response, redirect policy, response time, and two common WordPress failure messages. Replace www.example.com / example.com and the allowed hosts with your actual domain.

Optional checks: 93-wordpress-rest-api.yaml tests /wp-json/ only when the REST API is intended to be publicly available; 94-wordpress-login-page.yaml checks /wp-login.php only when the standard login endpoint is intentionally exposed. Hardened sites may disable or rename either endpoint. These are separate YAML files so a customer can explicitly choose which endpoints are contractual acceptance criteria.

Run one check with python3 ipsonic-probe.py examples/92-wordpress-homepage.yaml, or multiple files by passing each path. No login credentials, administrative actions, or WordPress plugins are needed. These are availability and response checks, not proof that WordPress is patched, secure, accessible, or functionally correct.

Combined real-world scenarios (examples 95–109)

These YAML files demonstrate practical combinations of existing assertions rather than isolated features. All hostnames and addresses are documentation placeholders; adapt them to your environment, security policy, and acceptance contract before execution.

Important: The probe runs one configured target per YAML. For a complete customer journey or infrastructure acceptance, combine multiple YAML files and archive their outputs. A presence check is not equivalent to a security audit, a full mail-delivery test, or cryptographic DNSSEC validation. Mandatory acceptance checks should use CRITICAL severity; the project owner must still sign off on the evidence.

Example descriptions

The example catalog includes a plain-English explanation for every YAML file, including the checks performed and usage notes. Descriptions are maintained in examples-descriptions.json and rendered into each static HTML example page; no runtime dependency is added.

Maintainer and disclaimer

Maintained by mati peterson (ipsonic@mailbox.org). Official project URL: https://ipsonic.com/probe/.

Disclaimer: This software and its examples are provided AS IS, without warranty of any kind. Use at your own risk. Review and validate configurations before production deployment.

Project main website: https://ipsonic.com/

Examples

Browse all 138 YAML examples →