Docs / Reachability API

Reachability API

Whether a watched URL answered us, why not if it did not, and how long it has been that way. Six targets today, probed once a day. No page content is read, stored or served here — a status code is a fact about a request we made, which is also why this is the one dataset on this platform with nothing to license. A false “reachable” is not a claim that a site is down: two of these six sit behind bot protection that answers our client a challenge page rather than the site, and which answer we get has been measured changing with nothing but the HTTP version we negotiated. So every row carries the class of failure, the vantage the request left from, and the number of consecutive probes that have failed — because one refusal is noise and sixty-one in a row is a finding.

Live https://api.softon.dev/v1 v1 · stable

Endpoints

Endpoint	Route	Returns
List watched targets	`GET /v1/reachability`	collection
Retrieve a watched target	`GET /v1/reachability/{id}`	single object

The Reachability object

The fields below appear on the dataset's primary resource. Endpoint pages list any fields specific to them.

Field	Type	Description
`id`	string	`<source>:<target_id>` — the prober and the target it asked. Stable for the life of the target, and what `/v1/reachability/{id}` takes.
`source`	string	Which prober made the request. This is the one collection here where `source` is softon and not an upstream publisher: a row is a record of a request this platform made, so the source is the client that made it. A second prober at a second vantage is a second source, and a second row for the same target.
`target_id`	string	The stable key for one watched URL. Chosen when the target is registered and never derived from the URL, so a target that changes host keeps the history in the three fields at the bottom of this table.
`url`	string	The URL that was requested, exactly as registered.
`watchlist`	string	The named set of targets this one belongs to, and the value `?watchlist=` takes. `middle-corridor` is the first one. Open rather than a closed set — a new watchlist is a row, not a release — so an unknown value answers an empty page rather than a 400.
`vantage`	string	Where the request was made from, and the field that stops this row being read as a verdict on the site. `hetzner-nbg1` is a datacenter address in Nuremberg, which is this platform's ingestion box. The answer genuinely depends on where you ask from: the same URL has been measured answering `403` to a datacenter address and `200` to a residential one on the same day. A reachability figure quoted without the vantage attached is a measurement of a pair reported as a property of one half of it — which is also why the prober is carried as `source`, so that a second client probing the same target is a second row rather than a silent redefinition of this one.
`reachable`	boolean	Whether the last probe got an HTTP 200. Strictly 200: a `204` or a `206` reads as false here, deliberately, because that is what the pipeline this replaces means by the word. `false` is not a claim that the site is down. Two of the six targets served today sit behind a bot-protection edge whose answer is decided by the request rather than by the site: measured with one client, one user agent, one address and one minute, both answered `403` on every HTTP/2 sample and `200` on every HTTP/1.1 one, and one of them is not stable even within a single client. So read `failure_class` before turning this into a count, and `consecutive_failures` before turning one row into a finding. A single `blocked_by_bot_protection` is noise; sixty-one consecutive ones are a fact about a relationship with that site.
`http_status`	integer · nullable	The final status after redirects. Null when no HTTP response was obtained at all: a name that did not resolve, a handshake that was refused, a timeout, or a target whose robots.txt means we did not ask. Always present as a key, null where inapplicable.
`final_url`	string · nullable	Where the request ended up after redirects. Null when there was no response.
`redirected`	boolean	Whether the request moved before answering. False when there was no response.
`failure_class`	string · nullable	Why it was not a 200, from a closed set, and null exactly when `reachable` is true — the database refuses either half without the other, so there is no such row as one that failed without saying why. `http_error` — the host answered, with something that is not 200. `blocked_by_bot_protection` — the host answered 403 with a bot-protection challenge page. Our client failed a check; the site is up. It is separate from `http_error` because it is a fact about the request rather than about the target, and separate from an outage because collapsing the two publishes a finding about a site that is serving fine. `tls_error` — the certificate or the handshake was refused; a browser would refuse it too. `dns_error` — the name did not resolve. `timeout` — nothing came back inside the budget. `connection_error` — the connection failed or was reset below HTTP. `robots_disallowed` — we did not ask. The target's robots.txt refuses this crawler for this path, so `http_status` is null because no request was made, not because one failed.
`error`	string · nullable	A short plain sentence, at most 200 characters, or null. `"TLS certificate expired"`, not a stack trace. Treat it as user-facing copy, because on the page this dataset was built for it is rendered verbatim to the public. The pipeline it replaces writes `str(exception)` into the equivalent field, which is how a visitor to a transport-analysis page came to read a urllib3 connection-pool repr complete with a CPython source-file line number. Null rather than truncated when a sentence cannot be vouched for: a shortened exception repr is still an exception repr, and the meaning is in `failure_class`, which is never null on a failing row.
`response_ms`	integer · nullable	How long the request took, in milliseconds. Null when nothing came back. One measurement from one vantage on a shared box — useful for noticing a target getting slower, not a latency SLA.
`robots_allowed`	boolean	What the target's own `robots.txt` says about this crawler fetching this path. Nothing in the pipeline this replaces checks robots at all. Fails open when robots.txt cannot be fetched — a 404 or a timeout there is not a refusal. When it is false no request is made, and `failure_class` is `robots_disallowed`.
`checked_at`	timestamp	When the probe ran, on the prober's clock. Date a finding by this, not by `ingested_at`: the pipeline this replaces stamps its rows with the date of the build that read them, so a row could claim it was checked on a day nothing was.
`last_reachable_at`	timestamp · nullable	The last probe that got a 200. Null for a target that has never answered one.
`first_failed_at`	timestamp · nullable	When the current run of failures began. Null while the target is reachable, and reset the moment it answers 200 again.
`consecutive_failures`	integer	How many probes in a row have not got a 200; 0 while the target is reachable. This and the two fields above are the value this collection has that a one-shot check cannot: "this target has been refusing us for sixty-one straight probes" is an editorial signal, and "it refused us once, today" is not. They are accumulated where the row is stored, so they survive a redeploy and the prober never has to remember anything. Counts probes, not days — the same number at the daily cadence this runs at, and named for the fact so that changing the cadence cannot silently redefine it. All three reset if `vantage` changes, because a counter spanning two vantages answers neither question.
`ingested_at`	timestamp	When this platform stored the observation. Differs from `checked_at` by the time the batch spent in flight; compare the two to see whether probe runs are landing at all.