Robots.txt Validator & Tester

Validate robots.txt syntax, inspect parsed rule groups, and test whether a URL path would be allowed or blocked for any user-agent.

Robots.txt Content
Validation Results
✓ No syntax errors or warnings found
URL Tester
BLOCKED— Matched disallow: /admin/ (line 4)
Parsed Rule Groups
User-agent:Googlebotline 2
allow/public/:3
disallow/admin/:4
disallow/api/:5
crawl-delay10:6
User-agent:*line 8
disallow/private/:9
allow/:10
Sitemaps
sitemaphttps://example.com/sitemap.xml:12

How It Works

1

Paste your robots.txt

Paste the contents of your robots.txt file into the editor. A sample is pre-loaded to get started.

2

Review validation results

The tool checks syntax, groups rules by User-agent, and flags errors and warnings.

3

Test specific URLs

Enter a user-agent name and URL path to check whether that path would be allowed or blocked.

FAQ

User-agent, Allow, Disallow, Sitemap, Crawl-delay, and Host. Unknown directives trigger a warning.
Patterns support * (wildcard) and $ (end anchor) per the robots.txt specification. The most specific matching rule wins — longer patterns take priority.
Yes. 'Disallow: /' blocks all paths for the specified user-agent. To allow everything, use 'Allow: /' or leave Disallow empty.
No. All parsing and validation runs in your browser. To validate your site's Open Graph tags, try the OG Tag Validator.

Varstatt Toolkit

Each tool works standalone, runs entirely in your browser, and requires no signup.

JSON FormatterYAML ValidatorJSON ↔ YAMLRegex TesterUUID GeneratorHash GeneratorPassword GeneratorCrontab GeneratorTimestamp ConverterBase64 EncoderJWT DecoderImage to Base64Encrypt / DecryptColor ConverterCSV EditorMarkdown PreviewText DiffHTML ↔ MarkdownMarkdown to PDFMarkdown to DOCXQR Code GeneratorBarcode GeneratorMesh GradientCSS Cover ArtImage ConverterText to GradientFavicon GeneratorAspect Ratio CalculatorSVG OptimizerImage PlaceholderDNS LookupOG Tag ValidatorUser Agent ParserHTTP Status CodesRobots.txt ValidatorSitemap ValidatorCORS TesterSSL CheckerCase ConverterWord CounterSlug GeneratorCopy Paste Characters

Built & Maintained by Varstatt

Varstatt is a one-person product studio run by Jurij Tokarski, product engineer since 2011. These tools are free and open — no signup, no catch.