# PINtPOINT — pintpoint.co.uk # Allow search engines and AI assistants to index the site # Block only training-focused crawlers # Content Signals (contentsignals.org) — declared per User-agent block below. # - ai-train=no : do not use my content to train foundation models # - search=yes : do index my content for search-engine results # - ai-input=yes : do use my content as context for live AI assistant answers User-agent: * Content-Signal: search=yes, ai-input=yes, ai-train=no Allow: / # Netfax app feed packs aren't for indexing. The app fetches them directly. Disallow: /netfax/starter-pack # Build machinery, not reading matter: the generators and the data files # they read. _data/retired-strings.json is the corrections guard's record of # lines we have taken off pages, which is our process rather than our # writing. A Disallow only stops crawlers that obey it, and it names the path # to anyone reading this file, so it is a stopgap and not the fix. Disallow: /_data/ Disallow: /scripts/ # Allow AI search/assistant crawlers (these surface your site in AI answers) User-agent: GPTBot Allow: / User-agent: ClaudeBot Allow: / User-agent: PerplexityBot Allow: / User-agent: Applebot-Extended Allow: / User-agent: Google-Extended Allow: / # CCBot (Common Crawl) — re-allowed 2026-04-20 so the site surfaces in # the AI-search systems that seed from CC. Our Content-Signal still # declares ai-train=no; CCBot honours it. User-agent: CCBot Allow: / # Bytespider (ByteDance / TikTok Search) — kept blocked. Historically # aggressive crawler with spotty Content-Signal honouring; the GEO # upside is marginal until ByteDance's AI search footprint grows. User-agent: Bytespider Disallow: / # Meta AI's standalone agent — blocked. Their AI search uses different # entry points; this one is closer to pure training-data collection. User-agent: meta-externalagent Disallow: / Sitemap: https://pintpoint.co.uk/sitemap.xml # Context files for AI crawlers and assistants. Not a standard robots # field, so parsers that do not know it ignore the line; the point is # that a crawler reading this file as text finds the path without # having to guess at the root. llms.txt is the curated index, with the # full editorial text in llms-full.txt. Llms-txt: https://pintpoint.co.uk/llms.txt Llms-full-txt: https://pintpoint.co.uk/llms-full.txt