# JourneyForHealth — robots.txt (updated 2026-10-28) # Optimised for traditional search + AI search (GEO/AEO 2026). # All legitimate search/AI crawlers are allowed via this single group. User-agent: * Allow: / # PII / authenticated / commerce surfaces — keep out of all crawlers' index Disallow: /admin/ Disallow: /dashboard/ Disallow: /account/ Disallow: /checkout/ Disallow: /cart/ Disallow: /api/ Allow: /api/blog/sitemap Disallow: /auth/ Disallow: /login Disallow: /signup Disallow: /profile/ Disallow: /booking/ Disallow: /quote/ Disallow: /aftercare/ Disallow: /coaching/ Disallow: /clinician Disallow: /agency/ Disallow: /b2b/ Disallow: /clinic/ Disallow: /hospital-portal # Block known scraper/spam bots (their own group intentionally overrides *) User-agent: DotBot Disallow: / User-agent: MJ12bot Disallow: / User-agent: BLEXBot Disallow: / User-agent: DataForSeoBot Disallow: / User-agent: serpstatbot Disallow: / User-agent: Bytespider Disallow: / User-agent: PetalBot Disallow: / User-agent: SemrushBot Disallow: / User-agent: AhrefsBot Disallow: / Sitemap: https://jforh.com/sitemap.xml Sitemap: https://jforh.com/sitemap-doctors.xml Sitemap: https://jforh.in/sitemap-jforh-in.xml # llms.txt — LLM-friendly content index (https://llmstxt.org) # https://jforh.com/llms.txt # https://jforh.com/llms-full.txt