X open sources its algorithm amid regulatory scrutiny

On Tuesday, social media platform X provided an accessible write-up about its feed-generating code on GitHub, open sourcing its algorithm again. The release fulfills a promise made by Tesla billionaire Elon Musk, who had stated, “We will make the new 𝕏 algorithm, including all code used to determine what organic and advertising posts are recommended to users, open source in 7 days”.

According to the GitHub write-up, the recommendation system “relies entirely” on a Grok-based transformer to learn relevance from user engagement sequences. The system considers a user’s engagement history and surveys recent in-network posts, alongside a machine-learning-based analysis of out-of-network posts. It then filters out content from blocked accounts, muted keywords, or posts deemed too violent or spam-like, before ranking the remaining content based on the likelihood of user engagement, such as replies, reposts, or favorites. X noted there is no manual feature engineering for content relevance, which reduces complexity in its data pipelines.

The code release arrives as X faces significant regulatory pressure. In the European Union, regulators fined the platform $140 million for violating “transparency obligations” under the Digital Services Act (the EU’s content regulation framework). Regulators argued that the platform’s verification check mark system made it more difficult for users to judge the authenticity of accounts.

Simultaneously, the platform is under scrutiny in California. The California Attorney General’s office and congressional lawmakers have scrutinized the platform following claims that its Grok chatbot has been used to create “sexualized content”, including “naked images of women and minors”.

While X frames this code release as a step toward openness, some may view this appeal to openness as just more “transparency theater”. When Musk bought Twitter in 2022, the platform transitioned from a public company to a private one. Although the site previously released multiple transparency reports a year, X did not release its first transparency report under the new ownership until September of 2024. Critics had previously labeled the platform’s 2023 partial open-sourcing of its algorithm as “transparency theater” as well, noting that the release was incomplete and failed to reveal the inner workings of the organization.

Why it matters

X is attempting to bolster its transparency credentials by open-sourcing its algorithm, even as it faces regulatory fines and scrutiny over its AI chatbot’s content policies.