The three user-agents, and why the distinction decides everything
OpenAI operates three separate crawlers, and conflating them is the single most expensive mistake in this area. GPTBot gathers content for model training. OAI-SearchBot builds the search index that ChatGPT draws on when it answers with sources. ChatGPT-User fetches a specific page because a user asked about it in that moment.
Only the last two affect citation. This matters because the widely-circulated advice for publishers who did not want to contribute to model training was to block GPTBot — sound advice in itself, frequently implemented as a blanket Disallow under the wildcard group. The result is a site that opted out of training and, without intending to, out of every ChatGPT answer as well.
The check takes seconds and almost nobody runs it. Fetch your robots.txt and resolve each of the three agents against it separately, or use the crawler access checker, which does exactly that and reports which rule decided each verdict.
Getting the access layer right
Access is more than robots.txt. A page can be permitted by your robots file and still be unreachable in practice, and the failure modes are mundane.
CDN and WAF bot filtering is the most common. Rules written to stop scrapers routinely return 403 to OAI-SearchBot, because from a firewall's perspective a well-behaved AI crawler and a badly-behaved scraper look similar. This is invisible in analytics and does not appear in Search Console, since Googlebot is unaffected.
Client-side rendering is the second. A fetcher working under a timeout may receive a shell document with no content where a browser would eventually render a full article. If your primary content only exists after hydration, assume some fetchers will never see it.
Interstitials are the third — cookie walls, age gates, region redirects and login prompts that a crawler cannot dismiss. Whatever a fetcher receives at that URL is what it has to work with.
Writing a passage that can be lifted
Once a crawler can reach the page, the question becomes whether anything on it can be used. Retrieval operates on passages, so the unit that matters is not the article but the paragraph, and a paragraph that only makes sense in sequence is a paragraph that cannot be quoted.
The practical test is to copy any substantive paragraph into a blank document and read it cold. If it opens with "This means that", "As we saw above", or "It also improves performance", it has failed — the subject is missing. Rewritten so the subject is explicit, the same sentence becomes usable without changing its meaning or lengthening the page meaningfully.
State conclusions before you support them. Journalistic structure that builds to a finding works well for a reader who starts at the top and works badly for a system retrieving the middle of your page. Put the answer in the first two sentences of the section, then develop it.
Keep numbers adjacent to their sources. A statistic in one paragraph with its citation in a footnote is, at the passage level, an unsourced claim — the retrieval unit contains the number and not its provenance.
Being a low-risk source
An engine choosing between two adequate passages is making a risk decision, because a bad citation costs it more than a missed one. Everything that reduces that risk helps you.
Name a real author with genuine credentials rather than attributing everything to a house byline. Publish a visible date and keep dateModified accurate — a page that cannot demonstrate currency loses comparisons it would otherwise win. Link the primary sources behind your claims rather than describing them. State the scope of what you are covering, and what you are not: scope statements measurably reduce the risk of your content being applied to a question you did not answer.
None of this is novel. It is ordinary editorial standards, applied because a machine is now checking them.
Testing whether it worked
Build twenty questions a prospective customer would actually type, phrased as questions. Ask each in ChatGPT with browsing enabled. Record whether you were cited, which URL was returned, which competitors appeared, and the date. Archive the full answer text, not a yes or no.
Repeat the identical set a fortnight later. The comparison between dated runs is the entire value, because a single run cannot distinguish a real absence from sampling variance. When a citation disappears, the archived answers usually show which competitor took your place and what they said that you did not — which is a more actionable finding than the disappearance itself.
Deep research and agentic browsing
The citation surface inside ChatGPT is no longer a single behaviour. Alongside ordinary browsing, extended research modes visit many sources across a longer session and produce a report with substantial reference lists. The economics of that are different and worth understanding.
Those modes reward depth over immediacy. Where a quick answer needs one clean liftable sentence, a long research pass reads structure — methodology sections, stated limitations, primary data, comparisons against alternatives — because it is assembling something more than a single claim. Pages that are thin but well-optimised for the quick path often contribute nothing here.
They also read more of your site. A research pass that finds one useful page will frequently follow internal links to related material, which makes internal linking a retrieval mechanism rather than only a navigation aid. Orphaned pages, however good, are unlikely to be found this way.
None of this changes the fundamentals, and it does not warrant a separate programme. It does mean that the substantive, well-sourced pages that look uncompetitive against short-form content in quick answers may be earning citations in a place you are not measuring. If you only sample quick browsing responses, that contribution is invisible to you.
The blockers we see most often
In rough order of frequency: a wildcard Disallow that captured the search crawlers; CDN bot filtering returning 403; primary content only present after client-side hydration; every material claim unsourced; no author and no date anywhere on the template; and evidence trapped inside images, where the most valuable numbers on the page are invisible to any text extractor.
All six are mechanical, all six are fixable in days rather than quarters, and none require producing a single new word of content.
Primary references
Common questions
Do I need to be in ChatGPT's training data to be cited?+
No, and this is the most common misconception. Citations in ChatGPT come from retrieval at query time, not from training. That is why declining GPTBot while allowing OAI-SearchBot and ChatGPT-User is coherent rather than contradictory: you opt out of training and remain fully citable.
Which user-agent actually matters for citation?+
Two of them. OAI-SearchBot builds the search index that answers draw on, and ChatGPT-User fetches a page when someone asks about it directly. GPTBot is the training crawler and has no bearing on whether you are cited.
How long after publishing can a page be cited?+
There is no fixed interval. A page has to be crawled and indexed before it can be retrieved, so the practical lower bound is however long discovery takes on your site. Strong internal linking and an accurate sitemap shorten it; a page reachable only from a paginated archive can take considerably longer.
Does ChatGPT prefer certain domains?+
It behaves conservatively about attribution, which in practice favours sources with clear authorship, visible dates, established reputation and checkable claims. That is not a fixed allow-list, and small sites are cited regularly when their evidence is stronger and better structured than a larger competitor's.
Why did I get cited last week and not this week?+
These systems are non-deterministic, and the retrieved candidate set varies between runs. A disappearance across a single pair of runs is usually noise. A disappearance sustained across several dated runs, with a competitor consistently in your place, is a signal worth investigating.