xAI delayed model release to improve Baldur’s Gate performance

According to a report by the publication Business Insider, xAI, the artificial intelligence startup recently acquired by SpaceX, delayed a model release for several days last year. People familiar with the matter reported that the delay occurred because Elon Musk, the CEO of xAI, was dissatisfied with how the chatbot answered detailed questions about the video game “Baldur’s Gate.” High-level engineers were reportedly pulled from other projects to improve the responses before launch, they said. The incident highlights how xAI has been placing particular emphasis on video-game walkthroughs, focusing on reaching parity with other AI models in niche areas.

To evaluate the current capabilities of xAI’s chatbot, Grok, RPG enthusiast Ram Iyer put together a set of five general questions about “Baldur’s Gate.” This test, a quasi-benchmark created by the author called “BaldurBench,” compared Grok against competing models including OpenAI’s ChatGPT, Anthropic’s Claude, and Gemini. The testing showed that Grok actually gives good information, though its responses were dense with gamer jargon such as “save-scumming”—the practice of saving and reloading—and “DPS,” or damage per second. Ultimately, Grok’s advice turned out about the same as the other models, with the primary differences being stylistic. While Grok favored tables and “theorycraft”—the analytical breakdown of game mechanics—its competitors took different approaches. ChatGPT relied on bulleted lists and sentence fragments, while Gemini frequently bolded key terms. The biggest surprise came from the AI model Claude, which focused on preserving the player’s experience by avoiding spoilers, concluding its advice with: “Don’t stress too much and just play what sounds fun to you.” The models generally drew from the same guides, making style the primary differentiator.

The anecdote regarding the delayed release highlights broader internal dynamics at the startup. Business Insider reported on how the 54-year-old Musk is making life difficult for employees, pointing to the frustration of engineers being pulled from core machine intelligence projects to address his personal preferences. The report suggests that high-level engineers had to pivot from fundamental problems of knowledge and machine intelligence to help resolve Musk’s specific complaints about the chatbot’s gaming performance. This direct executive intervention shows how personal interests can disrupt standard development timelines.

Why it matters

The incident illustrates how xAI’s development priorities are being shaped by direct executive intervention, even in niche domains like gaming, as the company races to reach parity with competitors.