Short answer
To check a relay against NIP-01, publish a known, labelled dataset, compute the expected result of every filter locally and compare exactly. Report a failure only where a binding requirement is violated: filters on authors, ids, kinds, since, until, limit 0, replaceable event handling, rejecting forged events. Tag-filter indexing and limit behaviour are conventions or recommendations and deserve warnings.
Don't believe, recompute
A filter like {"authors": [A], "since": 1700000600} is easy to write and hard to test if you do not know what the relay holds. The reliable way is to turn the test around: publish a known dataset, compute the expected matches locally with your own reference implementation of the filter semantics, then compare exactly. Is anything missing? Anything extra? Anything twice?
A good dataset is reproducible. Derive keys and events from a run id, use deterministic signatures, tag every event so it can be told apart from real data and mark it as synthetic. Include notes with different tags (e, p, a, t), a few reactions, events of a second author and two events with an identical timestamp to provoke ties.
The basics: EVENT, OK, REQ, EOSE, CLOSE
- EVENT and OK. Every EVENT must be answered with a well-formed OK. If the relay refuses, the message must start with a machine-readable prefix such as
invalid:orblocked:. - REQ, EOSE, CLOSE. Stored events come first, then EOSE. After CLOSE nothing may arrive. New events must arrive live after EOSE.
- Several subscriptions must run independently. The same subscription id on two connections must be handled separately.
- Subscription ids from one up to 64 characters must work. How a relay reacts to empty or too-long ids is not specified, so you can only observe it.
- Invalid events. Wrong id, wrong signature, wrong field types. Accepting a forged event is a failure.
- Malformed messages. Not JSON, empty arrays, incomplete messages. The relay may answer with
NOTICEor close the connection, but it must stay functional afterwards.
Filters: what is required and what is convention
Every filter feature deserves its own check: authors, ids, kinds, since, until, both together, the inclusive boundaries, limit, #e, #p, #a, #t, several values, several filters in one REQ, overlapping filters. Weigh the results, because not every deviation is a bug:
- Required: the fields whose meaning NIP-01 defines:
authors,ids,kinds,since,until, AND within a filter, OR between filters. Alsolimit: 0, which means no stored events but an EOSE. - Convention: tag filters. NIP-01 calls the indexing of single-letter tags a convention. A relay that does not index
#tdeserves a warning, not a failure. - Recommended: the behaviour of
limit. Returning fewer events is acceptable; returning far more should be avoided.
Only report a failure where a binding requirement is violated.
Event semantics
- Regular events (kind 1 and most others) are stored.
- Replaceable events (kinds 10000 to 19999): the newest must be stored. Older versions still being delivered is a warning at most. If an older event replaces a newer one, that is a failure. On equal timestamps the lowest id should remain.
- Addressable events (kinds 30000 to 39999): the same, per
dtag. - Ephemeral events (kinds 20000 to 29999) are distributed live, not stored.
- Order: newest first, ties by lowest id first.
Deletion requests (NIP-09)
NIP-09 says relays should delete, so that is checked as a recommendation. One check is stricter: deletion requests from other keys must be ignored. A relay that deletes foreign events lets anyone remove other people's data. The wording is not a MUST, but it breaks data integrity, so it counts as essential.
What a finding looks like
A made-up example: a relay ignores since when it is combined with authors. The check runs, 11 events are expected, 15 arrive. A useful report shows the filter, the expected and received counts, a few missing and unexpected event ids, and the classification Required. That is a finding you can reproduce and fix.
Running it with labs
The Conformance check of labs publishes the known dataset, recomputes the expected results locally and compares them exactly. The NIP-01 basics, the filter checks, the event semantics and the NIP-09 checks run in one pass. Every item is listed as advertised, observed and verdict. From a script it is one call with "modules":["conformance"], 800 Payloads (0.08 EUR), the test dataset included; see How to test a Nostr relay with one API call.
Before this, the document that tells clients what to expect is covered in NIP-11. Next: NIP-42, authentication.
FAQ
What does a relay have to do with a rejected event?
It answers with OK false and a message that starts with a machine-readable prefix such as invalid: or blocked:.
Is a relay that does not index tag filters broken?
No. NIP-01 calls indexing of single-letter tags a convention, so it is a warning, not a failure.
What counts as a failure for replaceable events?
If an older replaceable event replaces a newer one. Still delivering older versions is a warning at most.
Do deletion requests from other keys have to be ignored?
They should be: a relay that deletes events of other keys lets anyone remove other people's data, so labs treats it as essential.
Which labs test covers this?
The Conformance test, 800 Payloads (0.08 EUR), with the known test dataset included.
