A third of travel prompts in ChatGPT don’t leave the training data

We logged ChatGPT’s hidden metadata, and watched it decide when to search the web and when to answer from memory (training data).

Every time you send a prompt to ChatGPT, something happens before a single word of the answer is generated: the system decides what kind of turn this is.

• Is it a question it can answer from its training data?
• Does it need to go and search the web?
• Should it engage its slower reasoning mode?
• Or is this about you (the user), requiring a look at its stored memory?

ChatGPT’s web client shows turn-level metadata that reveals it (visible via Dev Tools), and we captured these response logs through a custom Chrome plugin we built.

We logged two fields attached to every message:

• turn_use_case, which records how ChatGPT classified the turn
• result_source, which records which retrieval backends supplied results

Together, they offer a window into the routing layer that sits between a user’s prompt and the answer they receive, and the findings have direct implications for anyone working in SEO or optimising for visibility in LLMs.

There are six buckets we’ve been able to identify for turn_use_case;

• text, relies entirely on the AI’s pre-trained data, skipping the live web for definitions, code, and translations.
• instant_search, performs fast, standard web searches to quickly pull current facts or real-time news.
• thinking, activates a reasoning model that fires 15 to 40 targeted sub-queries to deeply cross-reference data.
• shopping, triggers commercial web scrapers for transactional queries like product recommendations and buying guides.
• local, routes location-based queries through local business, review, or regional press indexes.
• image_generation, directs asset-creation requests straight to the image generation pipeline instead of text retrieval.

If ChatGPT responds it’s using text, then no matter how much optimisation you do for that prompt – unless you’re in the training data (historically), and relevant, you’re not going to be surfaced.

There are also four buckets for result_source:

Every prompt is triaged before it is answered

The most common was instant_search (62% of this responses), which always appeared as a pair of values (instant and instant search) marking a fast-path answer augmented by live web retrieval.

Next came text (31%), pure model generation, answered entirely from the model’s weights, meaning that a third of the travel prompts didn’t trigger live web search and were answered from ChatGPT’s training data, which doesn’t go past August 2025 (at the time of writing).

A single turn was classified thinking, engaging the slower reasoning mode, and one was classified personal/personal context, drawing on stored user memory rather than the web.

What makes this more than a curiosity is how it lines up against the retrieval data. Every single instant search turn retrieved results from external sources. Every single text, thinking and personal turn retrieved nothing, not one external fetch between them. There was no overlap.

In other words, this is not a probabilistic blend where the model “sometimes leans on search a bit”.

It is a hard routing decision, made per turn, as either the pipeline goes out to the live web, or the answer comes entirely from what the model already knows. Two fundamentally different answer-generation paths, chosen before generation begins.

A layered stack of search backends

The result_source field revealed three distinct backends and a clear hierarchy between them.

bright is the always-on, default search backend and no retrieval happens without it.

A second source, labrador, appeared on 25 of the 48 (52%), and never on its own.

Labrador only ever fired alongside bright, which suggests a supplementary index consulted to enrich or cross-check the primary results rather than a co-equal alternative. A third source, serp, appeared exactly once (2%), alongside both of the others, behaving like a rarely triggered fallback or a specialised search-results fetch.

We should be plain about what we do and do not know here. These codenames are undocumented, and we are reading behaviour, not architecture diagrams.

But the pattern is consistent with a plausible two-tier design, a primary backend derived from conventional search-engine results (bright’s universal presence and the existence of a separate serp value both point this way), layered with a secondary source that may reflect OpenAI’s own crawling and processing of the web.

If that reading is right, it matters enormously, because it would mean ChatGPT’s answers are fed simultaneously by classic search rankings and by direct crawler access to your site.

Retrieval is multi-pass, not one-and-done

Looking at the raw log data we collected, individual search turns frequently recorded the same source firing two or three times. A single “instant search” answer routinely involved multiple rounds of fetching — bright twice, labrador twice, occasionally three passes of each.

ChatGPT is not running one search and summarising the top results. It is iterating, fetching, evaluating, and fetching again within a single answer.

Each pass is another moment at which content is either selected or passed over, which raises the bar for how retrievable, parseable and directly useful a page needs to be.

What this means for travel SEO and LLM/AI visibility

A third of the classified turns in this session were text turns that never touched the web. For those queries (typically definitional, conversational, or evergreen) no amount of crawling, structured data, or ranking strength puts you in the answer, because the answer is generated from training data.

Influence on that path is a function of your brand’s presence in the broader public corpus.

These are the mentions, citations, coverage and authority signals that were absorbed when the model was trained. This is brand-building and digital PR territory, on a timescale of training cycles rather than crawl cycles.

The retrieval path is different, and it appears to be double-doored. If the primary backend is SERP-derived, then classic organic visibility still feeds travel AI answers, ranking well remains an entry blocker to AI visibility, not a legacy metric.

And if a secondary source reflects OpenAI’s own index, then direct crawler accessibility matters independently of your Google rankings.

The strategic question this reframes is one most travel SEO teams are not yet asking, which prompts about your brand, products or category trigger search, and which are answered from memory?