A robots.txt file is not read top to bottom. Each crawler first picks exactly one group, and inside that group the longest matching pattern wins, with Allow beating Disallow on a tie. This tool applies those three rules from RFC 9309 and names the winning line for every combination.
Matching ignores case, and a name in the file counts when the identifier starts with it.
Paths only, starting with a slash. A query string may be included.
The address is fetched once from this server. Only that request is stored, for the hourly limit.
Eight queries per hour. Only /robots.txt is fetched, at most 500 KiB, private address ranges are refused.