Setting up the Rust Environment

This guide walks you through preparing your local development environment to build, test, and run the Rust-based components of the X algorithm, specifically the Home Mixer and Thunder services.

Prerequisites

Before setting up the Rust environment, ensure your system has the following dependencies installed:

• Rust Toolchain: The latest stable version of Rust (1.75+ recommended).
• Protocol Buffers Compiler (protoc): Required to compile the gRPC service definitions (.proto files) used for communication between components.
• OpenSSL: Required for secure Kafka connections and internal service communication.
• Build Essentials: Standard C compiler and linkers (e.g., build-essential on Ubuntu or Xcode tools on macOS).

Installing System Dependencies

On Ubuntu/Debian:

sudo apt-get update
sudo apt-get install -y build-essential protobuf-compiler libssl-dev pkg-config

On macOS:

brew install protobuf openssl pkg-config

Step 1: Install Rust

We recommend using rustup to manage your Rust installation. If you don't have it installed, run:

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

After installation, ensure your shell is configured:

source $HOME/.cargo/env

Verify the installation:

rustc --version
cargo --version

Step 2: Clone the Repository

Clone the x-algorithm repository and navigate to the project root:

git clone https://github.com/xai-org/x-algorithm.git
cd x-algorithm

Step 3: Build the Components

The repository is organized as a Rust workspace. You can build all components at once or target specific services like home-mixer or thunder.

Build All Components

cargo build --release

Build a Specific Service

To build only the Home Mixer (the orchestration layer):

cargo build -p xai-home-mixer --release

To build only Thunder (the in-network content service):

cargo build -p xai-thunder --release

The compiled binaries will be located in target/release/.

Step 4: Run Tests

Before deploying or making changes, verify the environment by running the test suite.

Run Workspace Tests

cargo test

Run Service-Specific Tests

# Test the Candidate Pipeline logic
cargo test -p xai-candidate-pipeline

# Test Thunder's internal logic
cargo test -p xai-thunder

Step 5: Configuration and Environment Variables

The Rust services utilize environment variables for sensitive configurations, particularly for Kafka and gRPC settings found in thunder/kafka_utils.rs and home-mixer/main.rs.

Create a .env file or export the following variables for local testing (Note: internal SASL credentials are required for actual production data access):

# Example Kafka Configuration
export KAFKA_GROUP_ID="local-dev-group"
export SECURITY_PROTOCOL="SASL_SSL"
export PRODUCER_SASL_MECHANISM="PLAIN"

# Service Ports
export GRPC_PORT=50051
export METRICS_PORT=9090

Troubleshooting

Protobuf Compilation Errors

If you encounter errors related to PROTOC not being found, ensure protoc is in your $PATH. You may also need to set the PROTOC environment variable:

export PROTOC=/usr/bin/protoc

Linking Errors (OpenSSL)

If pkg-config cannot find OpenSSL, ensure libssl-dev (Linux) or openssl (macOS) is installed correctly. On macOS, you may need to point pkg-config to the Homebrew OpenSSL directory:

export PKG_CONFIG_PATH="/usr/local/opt/openssl@3/lib/pkgconfig"

Missing Proto Definitions

If the build fails due to missing xai-home-mixer-proto or similar crates, ensure you have initialized submodules if they are used to host shared protocol definitions:

git submodule update --init --recursive