Writing Weighted Scorers

The final ranking of the "For You" feed is determined by a Weighted Scorer. While the Phoenix model (the Grok-based transformer) outputs raw probabilities for various engagement types—such as the likelihood of a user liking, replying to, or reposting a tweet—it is the Weighted Scorer that combines these disparate signals into a single numerical value used for sorting.

This guide walks you through creating and configuring a Weighted Scorer to tune the behavior of the recommendation engine.

Understanding the Scoring Formula

The core logic of a weighted scorer is a linear combination of engagement probabilities. For any given post, the final score is calculated as:

$$Score = \sum_{i=1}^{n} (Probability_{i} \times Weight_{i})$$

Where:

• Probability: The ML model's prediction that a user will perform action $i$.
• Weight: A configurable constant representing the "value" of that action to the platform.

Step 1: Define Engagement Signals

Before writing the scorer, identify which engagement logits from the Phoenix model you wish to include. Standard signals available in RecsysModelOutput include:

• like_probability
• reply_probability
• repost_probability
• quote_probability
• stay_on_post_probability (Long dwell time)

Step 2: Implement the Scorer Logic

Weighted Scorers are implemented within the home-mixer component. To create a new scorer, you define a structure that implements the scoring logic.

// home-mixer/src/scorers/weighted_engagement_scorer.rs

pub struct WeightedEngagementScorer {
    pub like_weight: f32,
    pub reply_weight: f32,
    pub repost_weight: f32,
    pub dwell_time_weight: f32,
}

impl WeightedEngagementScorer {
    /// Calculates the final rank score based on model logits.
    /// 
    /// # Arguments
    /// * `logits` - The raw output probabilities from the Phoenix transformer.
    pub fn calculate_score(&self, logits: &EngagementLogits) -> f32 {
        let mut total_score = 0.0;

        total_score += logits.like_prob * self.like_weight;
        total_score += logits.reply_prob * self.reply_weight;
        total_score += logits.repost_prob * self.repost_weight;
        total_score += logits.dwell_prob * self.dwell_time_weight;

        total_score
    }
}

Step 3: Configure Weights

Weights are typically managed via a configuration layer to allow for rapid tuning without recompiling the entire pipeline. When setting weights, consider the "relative value" of actions.

For example, if you want to prioritize meaningful conversations over passive likes, you might set the reply_weight significantly higher than the like_weight.

Example Weight Configuration:

| Signal | Weight | Logic | | :--- | :--- | :--- | | Like | 1.0 | The baseline engagement signal. | | Reply | 5.0 | High value; indicates deep engagement. | | Repost | 3.0 | Medium-high value; indicates amplification intent. | | Dwell Time | 0.5 | Low value; used to prevent "clickbait" that users quickly abandon. |

Step 4: Register the Scorer in the Pipeline

Once your scorer is implemented, it must be registered in the HomeMixerServer to be used during the orchestration phase. This happens in the candidate_pipeline module of the Home Mixer.

// home-mixer/src/candidate_pipeline.rs

let custom_scorer = WeightedEngagementScorer {
    like_weight: 1.0,
    reply_weight: 5.0,
    repost_weight: 3.0,
    dwell_time_weight: 0.5,
};

// Apply the scorer to the combined candidates from Thunder and Phoenix
let scored_candidates = pipeline.apply_scorer(candidates, custom_scorer).await?;

Best Practices for Tuning

1. Normalization: Ensure that the weights account for the base frequency of the action. Users "Like" content much more often than they "Reply" to it, so reply weights often need to be higher to remain competitive.
2. Avoid Zero Weights: Instead of setting a weight to zero to ignore a signal, remove it from the calculation entirely to save compute cycles.
3. Iterative Testing: Use A/B testing frameworks to evaluate weight changes. Small adjustments (e.g., increasing repost_weight by 10%) can have significant impacts on the overall health of the feed.

Troubleshooting

• Negative Scores: Ensure all weights are positive unless you explicitly want to penalize certain behaviors (e.g., penalizing the probability of a user "blocking" an author).
• Scale Mismatch: If the Phoenix model updates and its output probability distribution shifts, you may need to recalibrate your weights to maintain the same feed characteristics.