Tuning In-Memory Post Retention

Thunder maintains an in-memory cache of "In-Network" posts to ensure the Home Mixer can retrieve content from followed accounts with sub-millisecond latency. Because this data is stored entirely in RAM, tuning the retention period is critical to balancing content freshness against server memory limits.

This guide walks you through configuring retention settings and monitoring the health of the in-memory PostStore.

Configuring Post Retention

The retention policy is governed by command-line arguments passed to the Thunder service at startup.

1. Set the Retention Window

The primary setting is --post-retention-seconds. This determines how long a post remains eligible for retrieval after it is first indexed from the Kafka stream.

To adjust this, modify your startup flags:

./thunder \
  --post-retention-seconds 172800 \ # Keep posts for 48 hours
  --grpc-port 50051 \
  --http-port 8080

• Higher Retention: Provides more historical "In-Network" context for the ranking models but increases RAM usage.
• Lower Retention: Reduces memory footprint but may result in "empty" feeds if followed accounts post infrequently.

2. Configure Request Timeouts

To prevent long-running queries from blocking the PostStore, you should also tune the request timeout. This ensures that even if the store is under heavy load, the service remains responsive.

./thunder \
  --post-retention-seconds 86400 \
  --request-timeout-ms 200 \ # Time out retrieval after 200ms
  --max-concurrent-requests 1000

How Auto-Trim Works

Thunder runs an internal Auto-Trim task. Once the service is in "serving" mode, it launches a background thread that executes every 2 minutes.

This task scans the in-memory indices and removes any LightPost objects whose created_at timestamp is older than the configured post_retention_seconds. This prevents memory fragmentation and ensures the system doesn't grow indefinitely.

Monitoring Memory & Health

After adjusting retention settings, you should monitor the Thunder service's metrics to ensure the PostStore is performing optimally.

Key Metrics to Watch

The Thunder service exports several Prometheus metrics that indicate if your retention settings are too aggressive or too lenient:

| Metric | Description | | :--- | :--- | | GET_IN_NETWORK_POSTS_FOUND_FRESHNESS_SECONDS | The age of the most recent post found. If this is very high, your retention window might be too long. | | GET_IN_NETWORK_POSTS_FOUND_TIME_RANGE_SECONDS | The "depth" of posts retrieved (Oldest vs. Newest). | | REJECTED_REQUESTS | Increments if the service is overloaded. Often happens if the PostStore is too large to scan efficiently. |

Accessing the Stats Logger

When Thunder is started with is_serving=true, the PostStore automatically logs its internal state (total post count, memory overhead, and author density) to the standard output. Look for these logs to verify that the auto-trim task is successfully reducing the post count every 2 minutes:

INFO - Started PostStore auto-trim task (interval: 2 minutes, retention: 2.0 days)
INFO - Started PostStore stats logger

Practical Example: Optimizing for Low-Memory Environments

If you are running Thunder on a machine with limited RAM, use the following configuration to maintain a "fresh-only" cache:

1. Reduce Retention: Set --post-retention-seconds to 43200 (12 hours).
2. Strict Timeouts: Set --request-timeout-ms to 50 to fail fast.
3. Monitor Density: Watch the GET_IN_NETWORK_POSTS_FOUND_UNIQUE_AUTHORS metric. If the number of unique authors drops significantly, your retention window is likely too short to capture active content from the user's following list.