Google and bing have both transitioned from robot txt. crawling technology to something much more powerful with AI. Many SEO companies went under, however the trick is training the powerful LLMs. Today, generative content should to be trained on a brand-first perspective in addition to topic-specific keywords, holding the LLMs on to your content for better chat-search results.