Scaling the Home Mixer

The Home Mixer serves as the central orchestration layer for the "For You" feed. As your user base or request volume grows, you must balance throughput, memory usage, and latency. Scaling the Home Mixer effectively involves configuring concurrency limits, optimizing memory retention, and tuning gRPC communication.

This guide walks you through the steps to tune the system for production-scale loads.

1. Managing Request Throughput

To prevent the service from being overwhelmed by spikes in traffic, the Home Mixer and its downstream components (like Thunder) use semaphore-based concurrency limiting.

Step: Configure Concurrency Limits When starting the thunder or home-mixer services, use the --max-concurrent-requests flag to set a ceiling on active operations.

# Example: Limit Thunder to 500 concurrent requests
./thunder \
  --grpc-port 50051 \
  --max-concurrent-requests 500

How it works: If the service reaches this limit, incoming requests are queued or rejected (returning a ResourceExhausted gRPC status) rather than consuming more memory and degrading the performance of existing requests.
Best Practice: Set this value based on the available CPU cores and the average latency of your retrieval sources.

2. Optimizing Memory Retention

The Thunder component maintains an in-memory PostStore to serve in-network content with sub-millisecond latency. Memory usage is directly proportional to the number of posts stored and how long they are kept.

Step: Set Retention and Trim Intervals Adjust the retention window to match your available RAM. If you have a high-volume feed, lowering retention will significantly reduce the memory footprint.

# Example: Retain posts for 2 days (172,800 seconds)
./thunder \
  --post-retention-seconds 172800 \
  --request-timeout-ms 200

Auto-Trim: The system automatically runs a background "auto-trim" task every 2 minutes to purge expired posts.
Calculation: Total Memory $\approx$ (Avg. Post Size) $\times$ (Posts per Second) $\times$ (Retention Seconds).

3. Tuning gRPC Communication

The Home Mixer handles large payloads of candidate posts. Efficient serialization and compression are critical for reducing network tail latency (P99).

Step: Enable Compression and Message Limits The Home Mixer supports both Gzip and Zstd compression. You should configure the maximum message size to accommodate high-volume candidate lists returned by Phoenix.

In your service configuration (or via environment variables/params):

Compression: Ensure both client and server are configured to accept Zstd for the best balance between CPU overhead and compression ratio.
Chunk Size: Use the --chunk-size CLI argument to control how many posts are processed in a single batch during the scoring and filtering stages.

./home-mixer \
  --grpc-port 8080 \
  --chunk-size 128 \
  --reload-interval-minutes 60

4. Scaling the Retrieval Pipeline

The "For You" feed relies on heavy-duty ML ranking. If scoring becomes a bottleneck, you can scale the Home Mixer horizontally.

Stateless Orchestration: The Home Mixer is stateless. You can deploy multiple instances behind a standard Load Balancer (LB).
Shard Downstreams: While the Home Mixer scales horizontally, ensure that Thunder (In-Network) and Phoenix (Out-of-Network) are scaled or sharded appropriately to handle the increased query volume from multiple Mixer instances.

5. Monitoring for Bottlenecks

Use the built-in metrics port to identify when it's time to increase resources.

Step: Monitor Key Scaling Metrics Access the metrics endpoint (default port metrics_port) to track:

in_flight_requests: If this is consistently hitting your max_concurrent_requests, increase instances.
rejected_requests: A sign that your downstream services are too slow or your concurrency cap is too low.
post_store_stats: Tracks the total number of posts in memory and freshness.

# Example: Check metrics via curl
curl http://localhost:metrics_port/metrics | grep get_in_network_posts

By following these steps, you can ensure the Home Mixer remains responsive as your post corpus and user traffic expand.