{"id":10255,"date":"2026-08-15T00:42:19","date_gmt":"2026-08-14T22:42:19","guid":{"rendered":"https:\/\/www.lukaswojcik.com\/blog\/narzedzia\/robots-llms-txt-ai-crawler-generator\/"},"modified":"2026-08-15T00:42:19","modified_gmt":"2026-08-14T22:42:19","slug":"robots-llms-txt-ai-crawler-generator","status":"publish","type":"page","link":"https:\/\/www.lukaswojcik.com\/blog\/pl\/narzedzia\/robots-llms-txt-ai-crawler-generator\/","title":{"rendered":"Generator robots.txt i llms.txt dla crawler\u00f3w AI"},"content":{"rendered":"<div class=\"gtm-analyser-container\" style=\"background: var(--bg-panel, #1e1e24); padding: 25px; border-radius: 8px; border: 1px solid var(--border, #2a2a35);\">\n<p style=\"color: var(--text-secondary, #a0a0b0); margin-bottom: 20px;\">Crawler wyszukiwarki, crawler trenuj\u0105cy model i crawler odpowiadaj\u0105cy na pytanie tu i teraz to trzy r\u00f3\u017cne rzeczy, a blokada jednego nie ma nic wsp\u00f3lnego z pozosta\u0142ymi. Ten generator je rozdziela i nazywa koszt ka\u017cdej decyzji.<\/p>\n<div style=\"display: flex; gap: 12px; flex-wrap: wrap;\">\n<div style=\"flex: 2; min-width: 240px;\">\n            <label for=\"rb-host\" style=\"color: var(--text-secondary, #a0a0b0); display: block; font-size: 0.85rem; margin-bottom: 5px;\">Adres witryny<\/label><br \/>\n            <input type=\"text\" id=\"rb-host\" class=\"form-control\" style=\"width: 100%; padding: 10px; background: var(--bg-body, #14141a); border: 1px solid var(--border, #2a2a35); color: var(--text-primary, #e8e8ee); border-radius: 6px; font-family: monospace; box-sizing: border-box;\" value=\"https:\/\/www.example.com\">\n        <\/div>\n<div style=\"flex: 2; min-width: 240px;\">\n            <label for=\"rb-name\" style=\"color: var(--text-secondary, #a0a0b0); display: block; font-size: 0.85rem; margin-bottom: 5px;\">Nazwa witryny<\/label><br \/>\n            <input type=\"text\" id=\"rb-name\" class=\"form-control\" style=\"width: 100%; padding: 10px; background: var(--bg-body, #14141a); border: 1px solid var(--border, #2a2a35); color: var(--text-primary, #e8e8ee); border-radius: 6px; font-family: monospace; box-sizing: border-box;\" value=\"Example\">\n        <\/div>\n<\/p><\/div>\n<div style=\"margin-top: 12px;\">\n        <label for=\"rb-zusammenfassung\" style=\"color: var(--text-secondary, #a0a0b0); display: block; font-size: 0.85rem; margin-bottom: 5px;\">Jedno zdanie o witrynie, do llms.txt<\/label><br \/>\n        <input type=\"text\" id=\"rb-zusammenfassung\" class=\"form-control\" style=\"width: 100%; padding: 10px; background: var(--bg-body, #14141a); border: 1px solid var(--border, #2a2a35); color: var(--text-primary, #e8e8ee); border-radius: 6px; font-family: monospace; box-sizing: border-box;\" placeholder=\"Materia\u0142y techniczne o analityce, sieciach i w\u0142asnej infrastrukturze\">\n    <\/div>\n<div style=\"display: flex; gap: 14px; flex-wrap: wrap; margin-top: 15px;\">\n<div style=\"flex: 1; min-width: 240px;\">\n<div style=\"color: var(--text-secondary, #a0a0b0); font-size: 0.85rem; margin-bottom: 8px;\">Kto ma wst\u0119p<\/div>\n<div id=\"rb-gruppen\"><\/div>\n<\/p><\/div>\n<div style=\"flex: 1; min-width: 240px;\">\n            <label for=\"rb-pfade\" style=\"color: var(--text-secondary, #a0a0b0); display: block; font-size: 0.85rem; margin-bottom: 5px;\">\u015acie\u017cki, kt\u00f3rych nie ma widzie\u0107 \u017caden crawler, po jednej w wierszu<\/label><br \/>\n            <textarea id=\"rb-pfade\" class=\"form-control\" style=\"width: 100%; height: 120px; padding: 10px; background: var(--bg-body, #14141a); border: 1px solid var(--border, #2a2a35); color: var(--text-primary, #e8e8ee); border-radius: 6px; font-family: monospace; font-size: 0.85rem; box-sizing: border-box;\">\/wp-admin\/<br \/>\n\/*?s=<br \/>\n\/danke\/<\/textarea>\n        <\/div>\n<\/p><\/div>\n<div style=\"display: flex; gap: 10px; flex-wrap: wrap; margin: 18px 0 0;\">\n        <button id=\"rb-btn\" class=\"button\" style=\"background: var(--accent, #7ee787); color: #0b1114; border: none; padding: 12px 24px; border-radius: 6px; font-weight: 700; cursor: pointer;\">Utw\u00f3rz pliki<\/button>\n    <\/div>\n<div id=\"rb-ergebnis\" style=\"margin-top: 22px;\"><\/div>\n<\/div>\n<p><script>\n(function () {\n    'use strict';<\/p>\n<p>    const T = {\"einleitung\":\"Crawler wyszukiwarki, crawler trenuj\u0105cy model i crawler odpowiadaj\u0105cy na pytanie tu i teraz to trzy r\u00f3\u017cne rzeczy, a blokada jednego nie ma nic wsp\u00f3lnego z pozosta\u0142ymi. Ten generator je rozdziela i nazywa koszt ka\u017cdej decyzji.\",\"label_host\":\"Adres witryny\",\"label_name\":\"Nazwa witryny\",\"label_zusammenfassung\":\"Jedno zdanie o witrynie, do llms.txt\",\"ph_zusammenfassung\":\"Materia\u0142y techniczne o analityce, sieciach i w\u0142asnej infrastrukturze\",\"label_gruppen\":\"Kto ma wst\u0119p\",\"label_pfade\":\"\u015acie\u017cki, kt\u00f3rych nie ma widzie\u0107 \u017caden crawler, po jednej w wierszu\",\"knopf\":\"Utw\u00f3rz pliki\",\"fehler_host\":\"Brakuje adresu witryny.\",\"fehler_host_form\":\"Adres musi by\u0107 samym \u017ar\u00f3d\u0142em, jak https:\\\/\\\/www.example.com, bez \u015bcie\u017cki.\",\"fehler_pfad\":\"\u015acie\u017cka musi zaczyna\u0107 si\u0119 od uko\u015bnika: %s\",\"kopf_hinweis\":\"Utworzone generatorem robots.txt na lukaswojcik.com\",\"gruppe_suche\":\"Wyszukiwarki\",\"gruppe_suche_text\":\"Googlebot, Bingbot i reszta klasycznej wyszukiwarki. Blokada usuwa witryn\u0119 z wynik\u00f3w wyszukiwania.\",\"gruppe_training\":\"Trenowanie AI\",\"gruppe_training_text\":\"GPTBot, Google-Extended, ClaudeBot i inne, kt\u00f3re zbieraj\u0105 materia\u0142 do trenowania. Blokada nie wp\u0142ywa na pozycj\u0119 w wynikach wyszukiwania.\",\"gruppe_assistent\":\"Asystenci AI\",\"gruppe_assistent_text\":\"OAI-SearchBot, PerplexityBot i crawlery, kt\u00f3re pobieraj\u0105 stron\u0119, aby odpowiedzie\u0107 na pytanie w\u0142a\u015bnie teraz. Blokada trzyma witryn\u0119 poza odpowiedziami AI.\",\"gruppe_archiv\":\"Archiwa i zbieracze danych\",\"gruppe_archiv_text\":\"CCBot, Diffbot i podobne, kt\u00f3rych zbiory zasilaj\u0105 od razu wiele modeli.\",\"h_robots\":\"robots.txt\",\"h_llms\":\"llms.txt\",\"h_kennungen\":\"Co dostaje ka\u017cdy crawler\",\"h_befunde\":\"Co to oznacza\",\"llms_platzhalter\":\"Jedno zdanie o tym, czego dotyczy ta witryna.\",\"llms_wichtig\":\"Najwa\u017cniejsze strony\",\"llms_optional\":\"Dodatkowo\",\"llms_start\":\"Strona g\u0142\u00f3wna\",\"llms_start_text\":\"punkt wej\u015bcia i przegl\u0105d\",\"llms_blog\":\"Blog\",\"llms_blog_text\":\"artyku\u0142y i poradniki\",\"llms_ueber\":\"O witrynie\",\"llms_ueber_text\":\"kto za ni\u0105 stoi\",\"llms_sitemap\":\"Mapa witryny\",\"b_suche_gesperrt\":\"Wyszukiwarki s\u0105 zablokowane. W ci\u0105gu kilku tygodni usunie to witryn\u0119 z Google i Bing.\",\"b_training_ohne_assistent\":\"Trenowanie jest zablokowane, asystenci pozostaj\u0105 dozwoleni: witryn\u0119 nadal mo\u017cna zacytowa\u0107 w odpowiedzi, po prostu nie trafi do kolejnego modelu.\",\"b_google_extended\":\"Google-Extended reguluje wy\u0142\u0105cznie trenowanie Gemini i Vertex AI. Blokada niczego nie zmienia w pozycji w wyszukiwarce Google.\",\"b_rueckwirkend\":\"Blokada dzia\u0142a dopiero od teraz. To, co ju\u017c pobrano, zostaje w zbudowanych z tego zbiorach.\",\"b_assistent_gesperrt\":\"Asystenci AI s\u0105 zablokowani, wi\u0119c witryna nie mo\u017ce pojawi\u0107 si\u0119 w ich odpowiedziach jako \u017ar\u00f3d\u0142o - a to w\u0142a\u015bnie ta widoczno\u015b\u0107, kt\u00f3r\u0105 rozdaj\u0105 systemy generatywne.\",\"b_alles_gesperrt\":\"Jedna ze \u015bcie\u017cek to sam uko\u015bnik, co zamyka ka\u017cdemu crawlerowi ca\u0142\u0105 witryn\u0119.\",\"b_platzhalter\":\"\u015acie\u017cki u\u017cywaj\u0105 * albo $. Google i Bing rozumiej\u0105 oba znaki, wiele mniejszych crawler\u00f3w nie i czyta wiersz dos\u0142ownie.\",\"b_kein_schutz\":\"robots.txt to pro\u015bba, a nie zamek. Tak wykluczona strona i tak mo\u017ce pojawi\u0107 si\u0119 w wyszukiwarce, je\u015bli inni do niej odsy\u0142aj\u0105 - potrzebny jest nag\u0142\u00f3wek noindex albo prawdziwa kontrola dost\u0119pu.\",\"b_freiwillig\":\"Ka\u017cdy crawler sam decyduje, czy tego przestrzega. Powy\u017csza lista obejmuje te, kt\u00f3re publikuj\u0105 swoj\u0105 nazw\u0119 i j\u0105 respektuj\u0105.\"};<\/p>\n<p>    \/\/ --- reine Rechenlogik ---<\/p>\n<p>    \/\/ Die Kennungen stammen aus den Angaben der jeweiligen Betreiber. Wer\n    \/\/ eine Kennung sperrt, sperrt genau diesen Zugriff und nicht die Firma.\n    const BOTS = [\n        { name: 'Googlebot', gruppe: 'suche' },\n        { name: 'Googlebot-Image', gruppe: 'suche' },\n        { name: 'Bingbot', gruppe: 'suche' },\n        { name: 'DuckDuckBot', gruppe: 'suche' },\n        { name: 'Applebot', gruppe: 'suche' },\n        { name: 'YandexBot', gruppe: 'suche' },\n        { name: 'Seznam', gruppe: 'suche' },<\/p>\n<p>        { name: 'GPTBot', gruppe: 'training' },\n        { name: 'Google-Extended', gruppe: 'training' },\n        { name: 'ClaudeBot', gruppe: 'training' },\n        { name: 'anthropic-ai', gruppe: 'training' },\n        { name: 'Applebot-Extended', gruppe: 'training' },\n        { name: 'meta-externalagent', gruppe: 'training' },\n        { name: 'Bytespider', gruppe: 'training' },\n        { name: 'Amazonbot', gruppe: 'training' },\n        { name: 'cohere-ai', gruppe: 'training' },<\/p>\n<p>        { name: 'OAI-SearchBot', gruppe: 'assistent' },\n        { name: 'ChatGPT-User', gruppe: 'assistent' },\n        { name: 'PerplexityBot', gruppe: 'assistent' },\n        { name: 'Perplexity-User', gruppe: 'assistent' },\n        { name: 'YouBot', gruppe: 'assistent' },<\/p>\n<p>        { name: 'CCBot', gruppe: 'archiv' },\n        { name: 'Diffbot', gruppe: 'archiv' },\n        { name: 'Omgilibot', gruppe: 'archiv' },\n        { name: 'ImagesiftBot', gruppe: 'archiv' }\n    ];<\/p>\n<p>    const GRUPPEN = ['suche', 'training', 'assistent', 'archiv'];<\/p>\n<p>    function pfadeLesen(text) {\n        return String(text === undefined || text === null ? '' : text)\n            .split(\/\\r?\\n\/)\n            .map(function (s) { return s.trim(); })\n            .filter(function (s) { return s !== ''; });\n    }<\/p>\n<p>    function bauen(eingabe) {\n        const ergebnis = { ok: false, fehler: '', robots: '', llms: '', befunde: [], bots: [] };\n        const e = eingabe || {};<\/p>\n<p>        const host = String(e.host === undefined ? '' : e.host).trim().replace(\/\\\/+$\/, '');\n        if (host === '') { ergebnis.fehler = T.fehler_host; return ergebnis; }\n        if (!\/^https?:\\\/\\\/[^\\\/\\s]+$\/i.test(host)) { ergebnis.fehler = T.fehler_host_form; return ergebnis; }<\/p>\n<p>        const name = String(e.name === undefined ? '' : e.name).trim() || host.replace(\/^https?:\\\/\\\/\/i, '');\n        const pfade = pfadeLesen(e.pfade);\n        const erlaubt = {};\n        GRUPPEN.forEach(function (g) {\n            erlaubt[g] = e[g] === undefined ? (g === 'suche') : !!e[g];\n        });<\/p>\n<p>        \/\/ Ungueltige Pfade abfangen, bevor sie in der Datei stehen.\n        for (let i = 0; i < pfade.length; i++) {\n            if (pfade[i].charAt(0) !== '\/') {\n                ergebnis.fehler = T.fehler_pfad.replace('%s', pfade[i]);\n                return ergebnis;\n            }\n        }\n\n        const zeilen = [];\n        zeilen.push('# robots.txt fuer ' + host);\n        zeilen.push('# ' + T.kopf_hinweis);\n        zeilen.push('');\n\n        \/\/ Erst die allgemeine Gruppe, danach die einzelnen Kennungen.\n        zeilen.push('User-agent: *');\n        if (pfade.length) {\n            pfade.forEach(function (p) { zeilen.push('Disallow: ' + p); });\n        } else {\n            zeilen.push('Disallow:');\n        }\n        zeilen.push('');\n\n        const gesperrt = [];\n        GRUPPEN.forEach(function (gruppe) {\n            const botsDerGruppe = BOTS.filter(function (b) { return b.gruppe === gruppe; });\n            if (erlaubt[gruppe]) {\n                botsDerGruppe.forEach(function (b) {\n                    ergebnis.bots.push({ name: b.name, gruppe: gruppe, erlaubt: true });\n                });\n                return;\n            }\n            botsDerGruppe.forEach(function (b) {\n                ergebnis.bots.push({ name: b.name, gruppe: gruppe, erlaubt: false });\n                gesperrt.push(b.name);\n            });\n            zeilen.push('# ' + T['gruppe_' + gruppe]);\n            botsDerGruppe.forEach(function (b) { zeilen.push('User-agent: ' + b.name); });\n            zeilen.push('Disallow: \/');\n            zeilen.push('');\n        });\n\n        zeilen.push('Sitemap: ' + host + '\/sitemap.xml');\n\n        ergebnis.robots = zeilen.join('\\n');\n\n        \/\/ llms.txt ist Markdown und beschreibt, was auf der Seite steht.\n        const llms = [];\n        llms.push('# ' + name);\n        llms.push('');\n        llms.push('> ' + (String(e.zusammenfassung || '').trim() || T.llms_platzhalter));\n        llms.push('');\n        llms.push('## ' + T.llms_wichtig);\n        llms.push('');\n        llms.push('- [' + T.llms_start + '](' + host + '\/): ' + T.llms_start_text);\n        llms.push('- [' + T.llms_blog + '](' + host + '\/blog\/): ' + T.llms_blog_text);\n        llms.push('- [' + T.llms_ueber + '](' + host + '\/about\/): ' + T.llms_ueber_text);\n        llms.push('');\n        llms.push('## ' + T.llms_optional);\n        llms.push('');\n        llms.push('- [' + T.llms_sitemap + '](' + host + '\/sitemap.xml)');\n        ergebnis.llms = llms.join('\\n');<\/p>\n<p>        \/\/ --- Befunde ---\n        if (!erlaubt.suche) {\n            ergebnis.befunde.push({ schwer: true, text: T.b_suche_gesperrt });\n        }\n        if (!erlaubt.training && erlaubt.assistent) {\n            ergebnis.befunde.push({ schwer: false, text: T.b_training_ohne_assistent });\n        }\n        if (!erlaubt.training) {\n            ergebnis.befunde.push({ schwer: false, text: T.b_google_extended });\n            ergebnis.befunde.push({ schwer: false, text: T.b_rueckwirkend });\n        }\n        if (!erlaubt.assistent) {\n            ergebnis.befunde.push({ schwer: true, text: T.b_assistent_gesperrt });\n        }\n        if (pfade.some(function (p) { return p === '\/'; })) {\n            ergebnis.befunde.push({ schwer: true, text: T.b_alles_gesperrt });\n        }\n        if (pfade.some(function (p) { return p.indexOf('*') !== -1 || p.indexOf('$') !== -1; })) {\n            ergebnis.befunde.push({ schwer: false, text: T.b_platzhalter });\n        }\n        ergebnis.befunde.push({ schwer: false, text: T.b_kein_schutz });\n        ergebnis.befunde.push({ schwer: false, text: T.b_freiwillig });<\/p>\n<p>        ergebnis.gesperrt = gesperrt;\n        ergebnis.ok = true;\n        return ergebnis;\n    }<\/p>\n<p>    if (typeof window !== 'undefined') {\n        window.LW_TEST = window.LW_TEST || {};\n        window.LW_TEST.RB = { bauen: bauen, pfadeLesen: pfadeLesen, BOTS: BOTS };\n    }<\/p>\n<p>    \/\/ --- Anbindung an das Dokument ---<\/p>\n<p>    const btn = document.getElementById('rb-btn');\n    const out = document.getElementById('rb-ergebnis');\n    const gruppenFeld = document.getElementById('rb-gruppen');\n    if (!btn || !out || !gruppenFeld) { return; }<\/p>\n<p>    const KASTEN = 'background: var(--bg-body, #14141a); border: 1px solid var(--border, #2a2a35); border-radius: 6px; padding: 14px; margin-bottom: 18px;';\n    const UEBERSCHRIFT = 'color: var(--text-primary, #e8e8ee); font-weight: 700; margin-bottom: 8px;';<\/p>\n<p>    function el(tag, css, text) {\n        const knoten = document.createElement(tag);\n        if (css) { knoten.setAttribute('style', css); }\n        if (text !== undefined && text !== null) { knoten.textContent = text; }\n        return knoten;\n    }<\/p>\n<p>    GRUPPEN.forEach(function (gruppe) {\n        const zeile = el('label', 'display: flex; gap: 10px; align-items: flex-start; padding: 7px 0; cursor: pointer;');\n        const kasten = document.createElement('input');\n        kasten.type = 'checkbox';\n        kasten.id = 'rb-' + gruppe;\n        kasten.checked = gruppe === 'suche';\n        kasten.setAttribute('style', 'margin-top: 3px;');\n        zeile.appendChild(kasten);\n        const text = el('span', '');\n        text.appendChild(el('div', 'color: var(--text-primary, #e8e8ee); font-size: 0.9rem;', T['gruppe_' + gruppe]));\n        text.appendChild(el('div', 'color: var(--text-secondary, #a0a0b0); font-size: 0.8rem; line-height: 1.4;', T['gruppe_' + gruppe + '_text']));\n        zeile.appendChild(text);\n        gruppenFeld.appendChild(zeile);\n    });<\/p>\n<p>    function codeKasten(titel, inhalt, dateiname) {\n        const teil = document.createDocumentFragment();\n        teil.appendChild(el('div', UEBERSCHRIFT, titel));\n        teil.appendChild(el('div', 'color: var(--text-secondary, #a0a0b0); font-size: 0.82rem; margin-bottom: 8px;', dateiname));\n        teil.appendChild(el('pre', 'background: var(--bg-body, #14141a); border: 1px solid var(--border, #2a2a35); color: var(--text-primary, #e8e8ee); padding: 16px; border-radius: 6px; overflow-x: auto; font-family: monospace; font-size: 0.82rem; line-height: 1.55; white-space: pre; margin-bottom: 18px;', inhalt));\n        return teil;\n    }<\/p>\n<p>    btn.addEventListener('click', function () {\n        const eingabe = {\n            host: (document.getElementById('rb-host') || {}).value,\n            name: (document.getElementById('rb-name') || {}).value,\n            zusammenfassung: (document.getElementById('rb-zusammenfassung') || {}).value,\n            pfade: (document.getElementById('rb-pfade') || {}).value\n        };\n        GRUPPEN.forEach(function (g) {\n            const knoten = document.getElementById('rb-' + g);\n            eingabe[g] = knoten ? knoten.checked : false;\n        });<\/p>\n<p>        const erg = bauen(eingabe);\n        out.textContent = '';\n        if (!erg.ok) {\n            out.appendChild(el('div', 'background: var(--bg-body, #14141a); border: 1px solid var(--border, #2a2a35); border-left: 3px solid #ffb86c; color: var(--text-secondary, #a0a0b0); padding: 14px; border-radius: 6px;', erg.fehler));\n            return;\n        }<\/p>\n<p>        out.appendChild(codeKasten(T.h_robots, erg.robots, '\/robots.txt'));\n        out.appendChild(codeKasten(T.h_llms, erg.llms, '\/llms.txt'));<\/p>\n<p>        out.appendChild(el('div', UEBERSCHRIFT, T.h_kennungen));\n        const tabelle = el('div', KASTEN);\n        GRUPPEN.forEach(function (gruppe) {\n            const bots = erg.bots.filter(function (b) { return b.gruppe === gruppe; });\n            if (!bots.length) { return; }\n            tabelle.appendChild(el('div', 'color: var(--text-primary, #e8e8ee); font-size: 0.85rem; font-weight: 700; padding: 8px 0 4px;', T['gruppe_' + gruppe]));\n            const reihe = el('div', 'display: flex; gap: 8px; flex-wrap: wrap; padding-bottom: 6px;');\n            bots.forEach(function (b) {\n                reihe.appendChild(el('span', 'font-family: monospace; font-size: 0.78rem; padding: 3px 8px; border-radius: 4px; border: 1px solid var(--border, #2a2a35); color: ' + (b.erlaubt ? 'var(--accent, #7ee787)' : '#ffb86c') + ';', b.name));\n            });\n            tabelle.appendChild(reihe);\n        });\n        out.appendChild(tabelle);<\/p>\n<p>        out.appendChild(el('div', UEBERSCHRIFT, T.h_befunde));\n        const befunde = el('div', KASTEN);\n        erg.befunde.forEach(function (b) {\n            const z = el('div', 'display: flex; gap: 10px; padding: 5px 0;');\n            z.appendChild(el('span', 'color: ' + (b.schwer ? '#ffb86c' : 'var(--text-secondary, #a0a0b0)') + '; font-family: monospace;', b.schwer ? '!' : '-'));\n            z.appendChild(el('span', 'color: var(--text-secondary, #a0a0b0); font-size: 0.9rem; line-height: 1.5;', b.text));\n            befunde.appendChild(z);\n        });\n        out.appendChild(befunde);\n    });\n})();\n<\/script><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Tworzy plik robots.txt, kt\u00f3ry rozdziela wyszukiwarki, crawlery trenuj\u0105ce modele i asystent\u00f3w AI, dodaje szkielet llms.txt i wyja\u015bnia, co naprawd\u0119 daje ka\u017cda blokada.<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":9284,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"template-tool-base.php","meta":{"footnotes":""},"tags":[92061,92038,92077],"class_list":["post-10255","page","type-page","status-publish","hentry","tag-automation-pl","tag-marketing-cyfrowy-pl-pl","tag-web-security-pl"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v28.1 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Generator robots.txt i llms.txt dla crawler\u00f3w AI - Lukas Wojcik - Blog<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.lukaswojcik.com\/blog\/pl\/narzedzia\/robots-llms-txt-ai-crawler-generator\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Generator robots.txt i llms.txt dla crawler\u00f3w AI - Lukas Wojcik - Blog\" \/>\n<meta property=\"og:description\" content=\"Tworzy plik robots.txt, kt\u00f3ry rozdziela wyszukiwarki, crawlery trenuj\u0105ce modele i asystent\u00f3w AI, dodaje szkielet llms.txt i wyja\u015bnia, co naprawd\u0119 daje ka\u017cda blokada.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.lukaswojcik.com\/blog\/pl\/narzedzia\/robots-llms-txt-ai-crawler-generator\/\" \/>\n<meta property=\"og:site_name\" content=\"Lukas Wojcik - Blog\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.lukaswojcik.com\/blog\/wp-content\/uploads\/2026\/08\/og-default.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"630\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.lukaswojcik.com\\\/blog\\\/pl\\\/narzedzia\\\/robots-llms-txt-ai-crawler-generator\\\/\",\"url\":\"https:\\\/\\\/www.lukaswojcik.com\\\/blog\\\/pl\\\/narzedzia\\\/robots-llms-txt-ai-crawler-generator\\\/\",\"name\":\"Generator robots.txt i llms.txt dla crawler\u00f3w AI - Lukas Wojcik - Blog\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.lukaswojcik.com\\\/blog\\\/#website\"},\"datePublished\":\"2026-08-14T22:42:19+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.lukaswojcik.com\\\/blog\\\/pl\\\/narzedzia\\\/robots-llms-txt-ai-crawler-generator\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.lukaswojcik.com\\\/blog\\\/pl\\\/narzedzia\\\/robots-llms-txt-ai-crawler-generator\\\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.lukaswojcik.com\\\/blog\\\/pl\\\/narzedzia\\\/robots-llms-txt-ai-crawler-generator\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/www.lukaswojcik.com\\\/blog\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Toolbox\",\"item\":\"https:\\\/\\\/www.lukaswojcik.com\\\/blog\\\/toolbox-3\\\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Generator robots.txt i llms.txt dla crawler\u00f3w AI\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.lukaswojcik.com\\\/blog\\\/#website\",\"url\":\"https:\\\/\\\/www.lukaswojcik.com\\\/blog\\\/\",\"name\":\"Lukas Wojcik - Blog\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\\\/\\\/www.lukaswojcik.com\\\/blog\\\/#\\\/schema\\\/person\\\/895f7604f9b6b71aad9bba33af28d0f9\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.lukaswojcik.com\\\/blog\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":[\"Person\",\"Organization\"],\"@id\":\"https:\\\/\\\/www.lukaswojcik.com\\\/blog\\\/#\\\/schema\\\/person\\\/895f7604f9b6b71aad9bba33af28d0f9\",\"name\":\"luky\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/www.lukaswojcik.com\\\/blog\\\/wp-content\\\/uploads\\\/2026\\\/07\\\/lw-x2.jpg\",\"url\":\"https:\\\/\\\/www.lukaswojcik.com\\\/blog\\\/wp-content\\\/uploads\\\/2026\\\/07\\\/lw-x2.jpg\",\"contentUrl\":\"https:\\\/\\\/www.lukaswojcik.com\\\/blog\\\/wp-content\\\/uploads\\\/2026\\\/07\\\/lw-x2.jpg\",\"width\":424,\"height\":636,\"caption\":\"luky\"},\"logo\":{\"@id\":\"https:\\\/\\\/www.lukaswojcik.com\\\/blog\\\/wp-content\\\/uploads\\\/2026\\\/07\\\/lw-x2.jpg\"},\"sameAs\":[\"https:\\\/\\\/www.lukaswojcik.com\\\/blog\"]}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Generator robots.txt i llms.txt dla crawler\u00f3w AI - Lukas Wojcik - Blog","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.lukaswojcik.com\/blog\/pl\/narzedzia\/robots-llms-txt-ai-crawler-generator\/","og_locale":"en_US","og_type":"article","og_title":"Generator robots.txt i llms.txt dla crawler\u00f3w AI - Lukas Wojcik - Blog","og_description":"Tworzy plik robots.txt, kt\u00f3ry rozdziela wyszukiwarki, crawlery trenuj\u0105ce modele i asystent\u00f3w AI, dodaje szkielet llms.txt i wyja\u015bnia, co naprawd\u0119 daje ka\u017cda blokada.","og_url":"https:\/\/www.lukaswojcik.com\/blog\/pl\/narzedzia\/robots-llms-txt-ai-crawler-generator\/","og_site_name":"Lukas Wojcik - Blog","og_image":[{"width":1200,"height":630,"url":"https:\/\/www.lukaswojcik.com\/blog\/wp-content\/uploads\/2026\/08\/og-default.jpg","type":"image\/jpeg"}],"twitter_card":"summary_large_image","schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.lukaswojcik.com\/blog\/pl\/narzedzia\/robots-llms-txt-ai-crawler-generator\/","url":"https:\/\/www.lukaswojcik.com\/blog\/pl\/narzedzia\/robots-llms-txt-ai-crawler-generator\/","name":"Generator robots.txt i llms.txt dla crawler\u00f3w AI - Lukas Wojcik - Blog","isPartOf":{"@id":"https:\/\/www.lukaswojcik.com\/blog\/#website"},"datePublished":"2026-08-14T22:42:19+00:00","breadcrumb":{"@id":"https:\/\/www.lukaswojcik.com\/blog\/pl\/narzedzia\/robots-llms-txt-ai-crawler-generator\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.lukaswojcik.com\/blog\/pl\/narzedzia\/robots-llms-txt-ai-crawler-generator\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.lukaswojcik.com\/blog\/pl\/narzedzia\/robots-llms-txt-ai-crawler-generator\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.lukaswojcik.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Toolbox","item":"https:\/\/www.lukaswojcik.com\/blog\/toolbox-3\/"},{"@type":"ListItem","position":3,"name":"Generator robots.txt i llms.txt dla crawler\u00f3w AI"}]},{"@type":"WebSite","@id":"https:\/\/www.lukaswojcik.com\/blog\/#website","url":"https:\/\/www.lukaswojcik.com\/blog\/","name":"Lukas Wojcik - Blog","description":"","publisher":{"@id":"https:\/\/www.lukaswojcik.com\/blog\/#\/schema\/person\/895f7604f9b6b71aad9bba33af28d0f9"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.lukaswojcik.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":["Person","Organization"],"@id":"https:\/\/www.lukaswojcik.com\/blog\/#\/schema\/person\/895f7604f9b6b71aad9bba33af28d0f9","name":"luky","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.lukaswojcik.com\/blog\/wp-content\/uploads\/2026\/07\/lw-x2.jpg","url":"https:\/\/www.lukaswojcik.com\/blog\/wp-content\/uploads\/2026\/07\/lw-x2.jpg","contentUrl":"https:\/\/www.lukaswojcik.com\/blog\/wp-content\/uploads\/2026\/07\/lw-x2.jpg","width":424,"height":636,"caption":"luky"},"logo":{"@id":"https:\/\/www.lukaswojcik.com\/blog\/wp-content\/uploads\/2026\/07\/lw-x2.jpg"},"sameAs":["https:\/\/www.lukaswojcik.com\/blog"]}]}},"_links":{"self":[{"href":"https:\/\/www.lukaswojcik.com\/blog\/wp-json\/wp\/v2\/pages\/10255","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.lukaswojcik.com\/blog\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/www.lukaswojcik.com\/blog\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/www.lukaswojcik.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.lukaswojcik.com\/blog\/wp-json\/wp\/v2\/comments?post=10255"}],"version-history":[{"count":0,"href":"https:\/\/www.lukaswojcik.com\/blog\/wp-json\/wp\/v2\/pages\/10255\/revisions"}],"up":[{"embeddable":true,"href":"https:\/\/www.lukaswojcik.com\/blog\/wp-json\/wp\/v2\/pages\/9284"}],"wp:attachment":[{"href":"https:\/\/www.lukaswojcik.com\/blog\/wp-json\/wp\/v2\/media?parent=10255"}],"wp:term":[{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.lukaswojcik.com\/blog\/wp-json\/wp\/v2\/tags?post=10255"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}