Why treating ChatGPT like Google will guarantee your failure in the age of RAG, reranking, and probabilistic systems
Look, I get it. Every time a new search technology appears, we try to map it to what we already know. When mobile search exploded, we called it “mobile SEO.” When voice assistants arrived, we coined “voice search optimization” and told everyone this would be the new hype.
Having reverse-engineered ChatGPT’s ranking system and discovered RRF in the code, I now have more questions than answers.
I’ve been doing SEO for years. I know how Google works – or at least I thought I did. Then I started digging into how ChatGPT picks citations, how Perplexity ranks sources, and how Google’s AI Overviews select content.
I’m not here to declare that SEO is dead or to state that everything has changed. I’m here to share the questions that keep me up at night – questions that suggest we might be dealing with fundamentally different systems that require fundamentally different thinking.
The Questions I Can’t Stop Asking
After months of analyzing AI search systems at AEOVision, documenting ChatGPT’s behavior, and reverse-engineering Perplexity’s ranking factors, these are the questions that challenge most of the things I thought I knew about search optimization.
When Math Stops Making Sense
I understand PageRank. I understand link equity. But when I discovered Reciprocal Rank Fusion in ChatGPT’s code, I realized I don’t understand this:
- Why does RRF mathematically reward mediocre consistency over single-query excellence? Is ranking #4 across 10 queries really better than ranking #1 for one?
- How do vector embeddings determine semantic distance differently from keyword matching? Are we optimizing for meaning or words?
- Why does temperature=0.7 create non-reproducible rankings? Should we test everything 10 times over now?
- How do cross-encoder rerankers evaluate query-document pairs differently than PageRank? Is real-time relevance replacing pre-computed authority?
These are also SEO concepts. However, they appear to be entirely different mathematical frameworks within LLMs. Or are they?
When Scale Becomes Impossible
Google indexes trillions of pages. ChatGPT retrieves 38-65. This isn’t a small difference – it’s a 99.999% reduction, resulting in questions that haunt me:
- Why do LLMs retrieve 38-65 results while Google indexes billions? Is this temporary or fundamental?
- How do token limits establish rigid boundaries that don’t exist in traditional searches? When did search results become limited in size?
- How does the k=60 constant in RRF create a mathematical ceiling for visibility? Is position 61 the new page 2?
Maybe they’re just current limitations. Or maybe, they represent a different information retrieval paradigm.
The 101 Questions That Haunt Me:
- Is OpenAI also using CTR for citation rankings?
- Does AI read our page layout the way Google does, or only the text?
- Should we write short paragraphs to help AI chunk content better?
- Can scroll depth or mouse movement affect AI ranking signals?
- How do low bounce rates impact our chances of being cited?
- Can AI models use session patterns (like reading order) to rerank pages?
- How can a new brand be included in offline training data and become visible?
- How do you optimize a web/product page for a probabilistic system?
- Why are citations continuously changing?
- Should we run multiple tests to see the variance?
- Can we use long-form questions with the “blue links” on Google to find the exact answer?
- Are LLMs using the same reranking process?
- Is web_search a switch or a chance to trigger?
- Are we chasing ranks or citations?
- Is reranking fixed or stochastic?
- Are Google & LLMs using the same embedding model? If so, what’s the corpus difference?
- Which pages are most requested by LLMs and most visited by humans?
- Do we track drift after model updates?
- Why is EEAT easily manipulated in LLMs but not in Google’s traditional search?
- How many of us drove at least 10x traffic increases after Google’s algorithm leak?
- Why does the answer structure always change even when asking the same question within a day’s difference? (If there is no cache)
- Does post-click dwell on our site improve future inclusion?
- Does session memory bias citations toward earlier sources?
- Why are LLMs more biased than Google?
- Does offering a downloadable dataset make a claim more citeable?
- Why do we still have very outdated information in Turkish, even though we ask very up-to-date questions? (For example, when asking what’s the best e-commerce website in Turkiye, we still see brands from the late 2010s)
- How do vector embeddings determine semantic distance differently from keyword matching?
- Do we now find ourselves in need to understand the “temperature” value in LLMs?
- How can a small website appear inside ChatGPT or Perplexity answers?
- What happens if we optimize our entire website solely for LLMs?
- Can AI systems read/evaluate images in webpages instantly, or only the text around them?
- How can we track whether AI tools use our content?
- Can a single sentence from a blog post be quoted by an AI model?
- How can we ensure that AI understands what our company does?
- Why do some pages show up in Perplexity or ChatGPT, but not in Google?
- Does AI favor fresh pages over stable, older sources?
- How does AI re-rank pages once it has already fetched them?
- Can we train LLMs to remember our brand voice in their answers?
- Is there any way to make AI summaries link directly to our pages?
- Can we track when our content is quoted but not linked?
- How can we know which prompts or topics bring us more citations? What’s the volume?
- What would happen if we were to change our monthly client SEO reports by just renaming them to “AI Visibility AEO/GEO Report”?
- Is there a way to track how many times our brand is named in AI answers? (Like brand search volumes)
- Can we use Cloudflare logs to see if AI bots are visiting our site?
- Do schema changes result in measurable differences in AI mentions?
- Will AI agents remember our brand after their first visit?
- How can we make a local business with a map result more visible in LLMs?
- Will Google AI Overviews and ChatGPT web answers use the same signals?
- Can AI build a trust score for our domain over time?
- Why do we need to be visible in query fanouts? For multiple queries at the same time? Why is there synthetic answer generation by AI models/LLMs even when users are only asking a question?
- How often do AI systems refresh their understanding of our site? Do they also have search algorithm updates?
- Is the freshness signal sitewide or page-level for LLMs?
- Can form submissions or downloads act as quality signals?
- Are internal links making it easier for bots to move through our sites?
- How does the semantic relevance between our content and a prompt affect ranking?
- Can two very similar pages compete inside the same embedding cluster?
- Do internal links help strengthen a page’s ranking signals for AI?
- What makes a passage “high-confidence” during reranking?
- Does freshness outrank trust when signals conflict?
- How many rerank layers occur before the model picks its citations?
- Can a heavily cited paragraph lift the rest of the site’s trust score?
- Do model updates reset past re-ranking preferences, or do they retain some memory?
- Why can we find better results by 10 blue links without any hallucination? (mostly)
- Which part of the system actually chooses the final citations?
- Do human feedback loops change how LLMs rank sources over time?
- When does an AI decide to search again mid-answer? Why do we see more/multiple automatic LLM searches during a single chat window?
- Does being cited once make it more likely for our brand to be cited again? If we rank in the top 10 on Google, we can remain visible while staying in the top 10. Is it the same with LLMs?
- Can frequent citations raise a domain’s retrieval priority automatically?
- Are user clicks on cited links stored as part of feedback signals?
- Are Google and LLMs using the same deduplication process?
- Can citation velocity (growth speed) be measured like link velocity in SEO?
- Will LLMs eventually build a permanent “citation graph” like Google’s link graph?
- Do LLMs connect brands that appear in similar topics or question clusters?
- How long does it take for repeated exposure to become persistent brand memory in LLMs?
- Why doesn’t Google show 404 links in results but LLMs in answers?
- Why do LLMs fabricate citations while Google only links to existing URLs?
- Do LLMs retraining cycles give us a reset chance after losing visibility?
- How do we build a recovery plan when AI models misinterpret information about us?
- Why do some LLMs cite us while others completely ignore us?
- Are ChatGPT and Perplexity using the same web data sources?
- Do OpenAI and Anthropic rank trust and freshness the same way?
- Are per-source limits (max citations per answer) different for LLMs?
- How can we determine if AI tools cite us following a change in our content?
- What’s the easiest way to track prompt-level visibility over time?
- How can we make sure LLMs assert our facts as facts?
- Does linking a video to the same topic page strengthen multi-format grounding?
- Can the same question suggest different brands to different users?
- Will LLMs remember previous interactions with our brand?
- Does past click behavior influence future LLM recommendations?
- How do retrieval and reasoning jointly decide which citation deserves attribution?
- Why do LLMs retrieve 38-65 results per search while Google indexes billions?
- How do cross-encoder rerankers evaluate query-document pairs differently than PageRank?
- Why can a site with zero backlinks outrank authority sites in LLM responses?
- How do token limits create hard boundaries that don’t exist in traditional search?
- Why does temperature setting in LLMs create non-deterministic rankings?
- Does OpenAI allocate a crawl budget for websites?
- How does Knowledge Graph entity recognition differ from LLM token embeddings?
- How does crawl-index-serve differ from retrieve-rerank-generate?
- How does temperature=0.7 create non-reproducible rankings?
- Why is a tokenizer important?
- How does knowledge cutoff create blind spots that real-time crawling doesn’t have?
When Trust Becomes Probabilistic
This one really gets me. Google links to URLs that exist, whereas AI systems can completely make things up:
- Why can LLMs fabricate citations while Google only links to existing URLs?
- How does a 3-27% hallucination rate compare to Google’s 404 error rate?
- Why do identical queries produce contradictory “facts” in AI but not in search indices?
- Why do we still have outdated information in Turkish even though we ask up-to-date questions?
Are we optimizing for systems that might lie to users? How do we handle that?
Where This Leaves Us
I’m not saying AI search optimization/AEO/GEO is completely different from SEO. I’m just saying that I have 100+ questions that my SEO knowledge can’t answer well, yet. Maybe you have the answers. Maybe nobody does (yet). But as of now, I don’t have the answers to these questions.
What I do know, however, is this: These questions aren’t going anywhere. And, there will be new ones. The systems that generate these questions aren’t going anywhere either. We need to engage with them, test against them, and maybe – just maybe – develop new frameworks to understand them.
I’ll also publish a new post soon! “XX Questions Show That SEO isn’t different than AEO/GEO”
The winners in this new field won’t be those who have all the answers. There’ll be those asking the right questions and testing relentlessly to find out what works.
These are my questions. What are yours?
SPECIAL NOTE:
Happy Republic Day!
Today we celebrate the vision, courage, and brilliance of Mustafa Kemal Atatürk, who turned hope into a nation.
A reminder that modern Turkiye was built on reason, science, and independence, values that continue to guide us forward.
SPECIAL NOTE 2:
Huge thanks to Marc-Aurele Legoux for making this post even better