Can AI Read Your Healthcare Site? We Scanned 100

In September 2026 we scanned 100 healthcare websites and one in five would not let an AI crawler in at all. The sample covers telehealth, behavioral health, virtual care, pharmacy, diagnostics, the software hospitals run on, the big insurers, and a dozen of the best-known hospital systems. We picked the names by hand, so this is a convenience sample rather than a random one, and no company here is named as failing anything.

Each domain went through three passes. Our free scanner ran 22 checks on whether a machine can fetch and parse the site. A second pass read each sitemap. A third pulled one real article out of the sitemap and looked for a date on it.

Healthcare's problem is not how the pages are built. It is who gets to see them.

One site in five turns the AI crawlers away

Twenty-one of the 98 sites we could measure refused a request carrying an AI crawler's name. That is the highest rate of any industry we have scanned, well above the 11 we found in business software. It splits into three cases that need completely different responses, and telling them apart is most of the work.

The 13 in the first row are mostly hospital systems and insurers, and their security is doing what it was bought to do. A plain request with an ordinary Chrome user-agent got challenged exactly the same way. We confirmed this by hand from a second network before describing any of them. These sites may still allow verified crawlers through by IP range in ways nobody outside the organization can test, and their own logs would answer it in a minute.

The six in the second row are the ones worth acting on. A browser gets the page. Change the name on the request to GPTBot or ClaudeBot and the same URL returns a 403. Nothing in their robots.txt mentions the decision, which is usually how you can tell it was made inside a security console rather than by anyone weighing what it costs in AI search.

The last two are the honest case, and both are medical content publishers whose business is people reading their articles. They named the crawlers in robots.txt and disallowed them. Someone decided that, wrote it down, and can reverse it in one line. Whatever you think of the choice, it is a policy rather than an accident.

If you run one of these sites, the useful question is not whether your robots.txt allows GPTBot. It is whether your edge does.

The pages themselves are fine

This is the surprise. Healthcare builds pages more conservatively than any industry we have looked at except business software, and it shows. Only 23% returned a homepage with nothing on it before JavaScript ran, against 52% of online stores. Half publish sitemap dates saying what changed, roughly the industry average.

So the content is mostly reachable, mostly parseable, and mostly server-rendered. Then a fifth of it sits behind a door that does not open.

Nearly nobody can tell an engine who they are

Ninety-five of the 98 are missing pieces of the block of structured data that states the organization's contact details, address, and links to its real profiles. That is the highest rate in the series, and it matters more here than elsewhere.

An engine answering a health question is trying to work out whether the source is a real, identifiable organization. That block is how a site says, in a form a machine can check, that it is a specific hospital at a specific address with a phone number and a verified profile somewhere else. Leaving it out does not make you look untrustworthy. It makes you look like a string of text.

Related, and cheap to fix: 80% are missing at least one of an about, contact, or privacy page at a predictable URL, and 79% publish no llms.txt.

Dates are shown to people and hidden from machines

We found and fetched a real article on 34 of the 100 sites. Of those, 79% printed a date a reader could see and only 50% carried one a parser could read. Thirty-eight percent showed a date to the human and handed the machine nothing.

Six percent is the lowest use of that one HTML element anywhere in the series. In an industry where "last reviewed" dates are a clinical convention and often a regulatory expectation, the date is nearly always there on the page. It is just written as ordinary text, which a parser cannot tell from any other sentence.

Fixing it takes minutes. Wrap the date you already print in <time datetime="2026-09-07">, and make the datePublished and dateModified in your structured data say the same thing the page says. On a templated site that is one change to one template.

Where our checks are opinions rather than standards

Two of our results look catastrophic and are not. Every site in the sample failed to link an llms.txt file, and 97% do not serve markdown to clients that ask for it. Both are Growthr bets on where agent tooling is going, and we have written before that no major AI engine has published a ranking benefit for llms.txt. Reading those as an industry failure would be dishonest.

The checks that matter are the ones with evidence behind them: whether the crawler gets a page, whether that page has anything on it, whether it carries structure and a date, and whether other sites say the same things about you.

What we would fix first

1. Ask your security team what it is turning away. Filter your edge logs for the GPTBot, ClaudeBot, PerplexityBot, and Google-Extended user-agents and look at the status codes you returned. Six of the sites here are almost certainly refusing crawlers without anyone having decided to.
2. Fill in the organization block. Contact point, address, and the profile URLs that let an engine resolve you to a named institution. Ninety-five of 98 are missing part of it.
3. Mark up the review dates you already print. A time element and matching structured data, on the template rather than per article.
4. Publish about, contact, and privacy at predictable URLs. Four in five are missing at least one, and they are the pages an engine checks to confirm an organization is real.
5. Add alt text. Half the sample fails it. This one is worth doing for its own sake before anyone mentions AI.

None of this decides whether a model recommends you, which is governed mostly by what other sites say about you. It decides whether you are eligible to be read at all.

Method and limits

We scanned in September 2026. The 100 domains were chosen by hand, so the sample is not random and the numbers describe these organizations rather than the industry. Each domain got a 22-check scan, a sitemap read, and one article pulled from that sitemap. The scanner ran 22 checks at the time of this scan; a 23rd, covering sitemap dates, was added afterwards, so a score you run today will not match one quoted here. An article was found for 34 of the 100, so every date percentage is out of 34 rather than 100. Two domains rate-limited the scan and are excluded from the scores, leaving 98. Every block was re-tested by hand from a second network before we described it.

The sample is named below; the results are not attributed to it. Every domain we scanned is listed, so anyone can reproduce this, but no organization is identified as passing or failing any individual check. The point of the exercise is the pattern, and a per-company scoreboard would be a pile-on rather than research.

Frequently asked questions

© 2026 Growthr. All rights reserved. · llms.txt · About · Privacy · Terms