x-algorithm README

The project's README from github.com/xai-org/x-algorithm. As one markdown file: README.md.

X For You Feed Algorithm

This repository contains the core code that determines which posts a viewer sees in the For You feed on X. It combines in-network content (from accounts the viewer follows) with out-of-network content (discovered through ML-based retrieval and other mechanisms), filters content based on a variety of inputs, and ranks posts using a transformer model.

Table of Contents


Notable Updates

August 14th, 2026

Notable updates:

August 13th, 2026

This release:

Among new systems included are:

  1. Visibility filtering: visibility-filtering/ determines whether to show a post, drop it, or show it behind an interstitial.
  2. The systems that produce labels that drive visibility filtering's responses: rules that apply labels (botmaker/, botmaker-rules/, scarecrow/), models that score accounts on various dimensions (agatha/, bdsm/, user-cred-v2/), models that examine images and video (media-model-proxy/, clip/), and enforcement (abuse-enforcement-service/).
  3. Phoenix model code: phoenix/ now contains code that trains and runs the model, plus synthetic data generation.
  4. SimClusters: simclusters/, an additional source of posts from accounts the viewer does not follow that is called in retrieval alongside Thunder and Phoenix retrieval.

This update is also paired with a new Under the Hood transparency tool that allows people to see aggregate statistics about the labels on their account and posts that can limit visibility.


Overview

The For You feed is assembled per request. Posts come from two places:

  1. In-Network — thunder/ keeps recent posts from the accounts a viewer follows in memory
  2. Out-of-Network — phoenix/ retrieval and simclusters/ find posts from accounts the viewer does not follow

Both are ranked together by the same model. Phoenix reads the viewer's recent engagement history and predicts, for each post, how likely the viewer is to take each action on it. Those predictions are combined into one score using weights held in the code — see Scoring and Ranking.

Two pipelines do the work. The Post Pipeline finds, ranks and filters posts. The Blending Pipeline wraps it and adds what the model does not rank: ads, Who to Follow recommendations, prompts.

Ranking sets the order. Whether a post can be shown at all is decided separately, by visibility-filtering/, from the viewer's own actions such as blocks and mutes and from labels that other systems here attach to posts and accounts.


System Architecture

Request Path

┌──────────────────────────────────────────────────────────────────────────────────────────┐
│                                   FOR YOU FEED REQUEST                                   │
└──────────────────────────────────────────────────────────────────────────────────────────┘
                                             ▼
┌──────────────────────────────── HOME MIXER   home-mixer/ ────────────────────────────────┐
│                                                                                          │
├───────────────────────  POST PIPELINE   PhoenixCandidatePipeline  ───────────────────────┤
│                                                                                          │
│  ┌────────────────────────────────────────────────────────────────────────────────────┐  │
│  │ 1. QUERY HYDRATION                                                                 │  │
│  │    user action sequence — the viewer's recent engagements, and the                 │  │
│  │    main input to the model · following list · blocks and mutes · muted             │  │
│  │    keywords · posts already seen and served · followed topics, etc.                │  │
│  └────────────────────────────────────────────────────────────────────────────────────┘  │
│                                            ▼                                             │
│  ┌────────────────────────────────────────────────────────────────────────────────────┐  │
│  │ 2. CANDIDATE SOURCES — queried in parallel                                         │  │
│  │    ┌───────────────────────────────┐ ┌────────────────────────────────────────┐    │  │
│  │    │ IN-NETWORK                    │ │ OUT-OF-NETWORK                         │    │  │
│  │    │ Thunder                       │ │ Phoenix retrieval   retrieval model    │    │  │
│  │    │   recent posts from the       │ │ SimClusters         cluster similarity │    │  │
│  │    │   accounts the viewer follows │ │                                        │    │  │
│  │    └───────────────────────────────┘ └────────────────────────────────────────┘    │  │
│  └────────────────────────────────────────────────────────────────────────────────────┘  │
│                                            ▼                                             │
│  ┌────────────────────────────────────────────────────────────────────────────────────┐  │
│  │ 3. CANDIDATE HYDRATION                                                             │  │
│  │    post text and media · author details and account labels · quoted post ·         │  │
│  │    language · engagement counts · subscription status, etc.                        │  │
│  └────────────────────────────────────────────────────────────────────────────────────┘  │
│                                            ▼                                             │
│  ┌────────────────────────────────────────────────────────────────────────────────────┐  │
│  │ 4. PRE-SCORING FILTERS                                                             │  │
│  │    duplicates across sources · older than 48 hours · the viewer's own              │  │
│  │    posts · blocked and muted accounts · muted keywords · already seen              │  │
│  │    or served · subscriber-only posts the viewer cannot access, etc.                │  │
│  └────────────────────────────────────────────────────────────────────────────────────┘  │
│                                            ▼                                             │
│  ┌────────────────────────────────────────────────────────────────────────────────────┐  │
│  │ 5. SCORING                                                                         │  │
│  │    PhoenixScorer   a probability for each action the viewer might take             │  │
│  │    RankingScorer   weighted sum, then repeated-author decay, an                    │  │
│  │                    out-of-network discount, a new-author boost                     │  │
│  │    VMRanker        calls the reranking service in vm-ranker/                       │  │
│  └────────────────────────────────────────────────────────────────────────────────────┘  │
│                                            ▼                                             │
│  ┌────────────────────────────────────────────────────────────────────────────────────┐  │
│  │ 6. SELECTION — TopKScoreSelector                                                   │  │
│  │    sort by final score, keep the top K                                             │  │
│  └────────────────────────────────────────────────────────────────────────────────────┘  │
│                                            ▼                                             │
│  ┌────────────────────────────────────────────────────────────────────────────────────┐  │
│  │ 7. POST-SELECTION FILTERS — after the order is fixed                               │  │
│  │    VFCandidateHydrator  asks visibility-filtering/ per post and viewer             │  │
│  │    VFFilter             removes the posts it said to drop                          │  │
│  │    DedupConversationFilter  collapses branches of one conversation                 │  │
│  │                         ◄── these labels come from the Labeling Path               │  │
│  └────────────────────────────────────────────────────────────────────────────────────┘  │
│                                                                                          │
├─────────────────────  BLENDING PIPELINE   ForYouCandidatePipeline  ──────────────────────┤
│                                                                                          │
│  ┌────────────────────────────────────────────────────────────────────────────────────┐  │
│  │ the ranked posts are one source here; the others add non-post items:               │  │
│  │    ads · Who to Follow · prompts · push-to-home, etc.                              │  │
│  │                                                                                    │  │
│  │ BlenderSelector interleaves them. The default ads blender reorders                 │  │
│  │ posts for ad adjacency. Who to Follow and prompts go at fixed positions.           │  │
│  └────────────────────────────────────────────────────────────────────────────────────┘  │
│                                            ▼                                             │
│  ┌────────────────────────────────────────────────────────────────────────────────────┐  │
│  │ SIDE EFFECTS — after the response is sent                                          │  │
│  │    record which posts were served · refresh the post cache · log ad                │  │
│  │    and client events, etc.                                                         │  │
│  └────────────────────────────────────────────────────────────────────────────────────┘  │
│                                                                                          │
└──────────────────────────────────────────────────────────────────────────────────────────┘
                                             ▼
┌──────────────────────────────────────────────────────────────────────────────────────────┐
│                                 RANKED FOR YOU TIMELINE                                  │
└──────────────────────────────────────────────────────────────────────────────────────────┘

Stages can be switched on and off individually, with defaults in home-mixer/params/param.rs — see Experiments and Configuration for how those defaults relate to what runs in production.

Labeling Path

┌───────  1. CONTENT UNDERSTANDING — happens continuously, not on the request path  ───────┐
│                                                                                          │
│    POSTS AND MEDIA                    ACCOUNTS                                           │
│    grox/          classifiers for     agatha/        blocks and reports                  │
│                   text and media                     relative to favorites               │
│    media-model-   image and video     bdsm/          inauthentic behavior                │
│      proxy/       models              user-c

x-algorithm (xai-org/x-algorithm)

Searching Database…

这是什么?这是 X(原 Twitter)“为你推荐”信息流算法的开源代码。它负责找出、打分、排序并过滤每位用户看到的帖子,把关注账号的帖子与机器学习挖掘出的非关注账号内容混合在一起。
这是一个完整可运行的应用,还是算法各组件的参考代码?这是各组件(Thunder、Phoenix、SimClusters、visibility-filtering、home-mixer 等)按功能拆分的实际源代码,并不是一个可直接安装的应用。各个流水线环节可以单独开启或关闭,而不是部署整个“为你推荐”信息流。
我能自己运行排序模型吗?可以,但只是概念验证。仓库中包含 Phoenix(排序模型)的训练代码以及合成数据生成代码,因此无需 X 的真实用户数据,就能运行一次概念验证训练。
排序是否只是在最大化原始互动数?不是。权重作用于每位用户自身对点赞、分享、拉黑、举报等行为的预测概率(以及停留时长等预测值),而不是直接乘以原始互动数。因此,举报的权重较高,并不意味着一次举报就能抵消几百个赞。
可见性过滤与排序有什么区别?排序(RankingScorer、VMRanker)决定帖子的先后顺序;visibility-filtering 则针对每条帖子和每位用户单独运行,决定是显示、隐藏,还是把帖子放在提示页之后展示。判断依据来自拉黑、静音,以及 Botmaker、Scarecrow、Agatha 等系统打上的标签。
什么是 SimClusters?SimClusters 是一个额外的站外内容召回来源,它基于兴趣簇的相似度,推送用户未关注账号的帖子。它与 Thunder(站内内容召回)和 Phoenix 召回并行运行。
信息流是否会应用特定地区的内容规则?是的。截至 2026 年 8 月,它会运行 Brazil2026ElectionFilter,移除被巴西选举法院举报为与 2026 年选举相关的账号所发布的帖子,除非用户明确关注了该账号。
我能看到哪些标签正在影响我账号的可见性吗?仓库中附带了一个“Under the Hood”(幕后)透明度工具,它会汇总展示与你的账号及可能限制其可见性的帖子相关联的标签统计数据。
Creator
xai-org
Size
5.8 MB
Stars
33.7K
Updated
CommitOct 17
Age
8mo
Commits · 90 Days