Implementing Weighted Scorers

In the X recommendation engine, the final ranking of a post is determined by the Scoring stage. While the Grok-based transformer (Phoenix) outputs raw probabilities for various engagement types (likes, replies, retweets), the Weighted Scorer is responsible for combining these signals into a single scalar value that determines the post's final position in the "For You" feed.

This guide walks you through implementing a custom weighted scorer within the home-mixer and candidate-pipeline services.

1. Define the Scoring Logic

Weighted scorers live within the home-mixer/src/scorers directory. A scorer's primary job is to take the engagement probabilities generated by the Phoenix model and apply weights to them.

For example, if you want to prioritize "Conversational Depth," you might create a scorer that weights Replies and Quote Tweets more heavily than Likes.

Engagement Signals

The Phoenix model provides the following primary signals for each candidate post:

• p_like: Probability the user will like the post.
• p_reply: Probability the user will reply to the post.
• p_retweet: Probability the user will retweet the post.
• p_quote: Probability the user will quote the post.
• p_stay_on_post: Probability the user will dwell on the post.

2. Implement the Scorer Trait

To create a new scorer, you must implement the Scorer trait. This involves defining how the scores are calculated for a batch of candidates.

// home-mixer/src/scorers/conversation_heavy_scorer.rs

use crate::scorers::Scorer;
use xai_home_mixer_proto::Candidate;

pub struct ConversationHeavyScorer {
    reply_weight: f64,
    quote_weight: f64,
    like_weight: f64,
}

impl Scorer for ConversationHeavyScorer {
    fn score(&self, candidates: Vec<Candidate>) -> Vec<Candidate> {
        candidates.into_iter().map(|mut candidate| {
            // Retrieve probabilities from the Phoenix model output
            let p_reply = candidate.get_score("p_reply");
            let p_quote = candidate.get_score("p_quote");
            let p_like = candidate.get_score("p_like");

            // Calculate weighted sum
            let final_score = (p_reply * self.reply_weight) + 
                              (p_quote * self.quote_weight) + 
                              (p_like * self.like_weight);

            candidate.set_final_score(final_score);
            candidate
        }).collect()
    }
}

3. Register the Scorer in the Pipeline

Once the scorer is implemented, it must be registered within the candidate_pipeline module so the Home Mixer can utilize it during a request.

1. Open home-mixer/src/candidate_pipeline.rs.
2. Add your new scorer to the scoring stage of the orchestration logic:

let scorer = ConversationHeavyScorer {
    reply_weight: 15.0,
    quote_weight: 10.0,
    like_weight: 1.0,
};

pipeline.add_scorer(scorer);

4. Configure Weights via Parameters

Hardcoding weights is discouraged. The system is designed to use a parameter-loading mechanism (often via params.rs) that allows for dynamic updates without recompiling the entire service.

Update your implementation to fetch values from the configuration:

impl ConversationHeavyScorer {
    pub fn new(params: &Params) -> Self {
        Self {
            reply_weight: params.get_f64("scorers.conversation.reply_weight", 12.0),
            quote_weight: params.get_f64("scorers.conversation.quote_weight", 8.0),
            like_weight: params.get_f64("scorers.conversation.like_weight", 0.5),
        }
    }
}

5. Validating with Metrics

Because the scoring stage is critical to the user experience, you should monitor the distribution of scores. Use the built-in metrics library found in thunder/metrics.rs or home-mixer/util to track how your scorer behaves.

Common metrics to track include:

• Score Distribution: Ensure your weights aren't causing "score explosion" where one signal dominates all others.
• Average Freshness: Check if your weighted scores are inadvertently penalizing new content.
• In-Network vs. Out-of-Network Balance: Ensure that your weights don't accidentally filter out all Phoenix (out-of-network) or Thunder (in-network) content.

// Example of reporting a metric within the scorer
metrics::SCORE_VALUE_OBSERVATION.observe(final_score);

Summary Checklist

• [ ] Created a new module in home-mixer/scorers.
• [ ] Implemented the score logic using Phoenix engagement probabilities.
• [ ] Added the scorer to the pipeline in candidate_pipeline.rs.
• [ ] Externalized weights into the system parameters.
• [ ] Added Prometheus metrics to monitor score distribution.