Skip to main content
The AI Gateway (router), Request Logger, and Evaluator Runner are lightweight services that handle incoming LLM requests, log them, and run evaluations. By default they run inside the main Docker Compose setup via the gateway profile. For production, you can break them out to separate instances for horizontal scaling and maximum uptime.

​
Architecture Overview

When scaled out, the gateway services run independently and connect back to the main Datawizz instance for configuration:

​
Running the AI Gateway

The AI Gateway fetches its configuration (projects, endpoints, prompts) from the Datawizz API via Supabase. It caches configurations in-memory and uses Supabase Realtime for instant cache invalidation when configurations change in the dashboard.

​
Docker Run

docker run -d \
    --name datawizz-gateway \
    -p 3000:3000 \
    -e DW_API_URL="http://your-datawizz-host/sbapi" \
    -e DW_API_KEY="your-service-role-key" \
    -e DW_CACHE_TTL="3600" \
    -e DW_LOGGER_URL="http://your-logger-host:3000" \
    -e DW_LOGGER_API_KEY="your-service-role-key" \
    -e DW_LOGGER_BATCH_SIZE="10" \
    -e DW_LOGGER_FLUSH_INTERVAL_MS="5000" \
    -e DWGW_PORT="3000" \
    -e DWGW_HOST="0.0.0.0" \
    -e DWGW_CORS_ENABLED="true" \
    -e DWGW_CORS_ALLOW_ORIGIN="*" \
    -e DWGW_OPTIONS_ENABLED="true" \
    ghcr.io/datawizzai/ai-gateway-cf:latest

​
Environment Variables

VariableDescriptionDefault
DW_API_URLURL to the Datawizz Supabase API (e.g. http://datawizz-host/sbapi)Required
DW_API_KEYSupabase service role keyRequired
DW_CACHE_TTLConfig cache TTL in seconds3600
DW_LOGGER_URLURL of the Request Logger serviceRequired
DW_LOGGER_API_KEYAPI key for the loggerRequired
DW_LOGGER_BATCH_SIZENumber of logs per batch10
DW_LOGGER_FLUSH_INTERVAL_MSFlush interval in milliseconds5000
DWGW_PORTPort to listen on3000
DWGW_HOSTHost to bind to0.0.0.0
DWGW_CORS_ENABLEDEnable CORS headersfalse
DWGW_CORS_ALLOW_ORIGINAllowed CORS origins*
DWGW_OPTIONS_ENABLEDHandle OPTIONS preflight requestsfalse
DWGW_FEEDBACK_JWT_ENABLEDEnable JWT tokens for feedback authfalse
DWGW_FEEDBACK_JWT_SECRETSecret for signing feedback JWTs (min 32 chars)—
DWGW_FEEDBACK_JWT_EXPIRATIONFeedback JWT expiration in seconds86400

​
Cache Invalidation

The gateway subscribes to Supabase Realtime for automatic cache invalidation. When you update a project, endpoint, or prompt in the Datawizz dashboard, all gateway instances invalidate their cached data within seconds. If the Realtime connection fails, the gateway continues operating normally and cached entries expire based on the TTL setting.

​
Health Check

The gateway exposes a health endpoint at /health: 
curl http://your-gateway-host:3000/health
# {"status": "ok"}

​
Running the Request Logger

The Request Logger batches inference logs and writes them to ClickHouse. It also forwards results to the Evaluator Runner for automated scoring. 
docker run -d \
    --name datawizz-request-logger \
    -p 3000:3000 \
    -v logger-data:/app/data \
    -e PORT="3000" \
    -e DATA_DIR="/app/data" \
    -e SUPABASE_URL="http://your-datawizz-host/sbapi" \
    -e SUPABASE_SERVICE_ROLE_KEY="your-service-role-key" \
    -e CLICKHOUSE_URL="http://your-clickhouse-host:8123" \
    -e CLICKHOUSE_DATABASE="datawizz" \
    -e CLICKHOUSE_USERNAME="your_user" \
    -e CLICKHOUSE_PASSWORD="your_password" \
    -e DISABLE_AUTH="false" \
    -e EVALUATOR_RUNNER_API_URL="http://your-evaluator-host:3000" \
    -e EVALUATOR_RUNNER_API_KEY="your-evaluator-key" \
    ghcr.io/datawizzai/logging-service:latest
VariableDescriptionDefault
PORTPort to listen on3000
DATA_DIRDirectory for local data/buffer/app/data
SUPABASE_URLDatawizz Supabase API URLRequired
SUPABASE_SERVICE_ROLE_KEYSupabase service role keyRequired
CLICKHOUSE_URLClickHouse HTTP URLRequired
CLICKHOUSE_DATABASEClickHouse database namedefault
CLICKHOUSE_USERNAMEClickHouse usernamedefault
CLICKHOUSE_PASSWORDClickHouse passworddefault
DISABLE_AUTHDisable auth for internal networkstrue
EVALUATOR_RUNNER_API_URLEvaluator Runner URL—
EVALUATOR_RUNNER_API_KEYEvaluator Runner API key—

​
Running the Evaluator Runner

The Evaluator Runner executes custom evaluators on inference results. 
docker run -d \
    --name datawizz-evaluator-runner \
    -p 3000:3000 \
    -e PORT="3000" \
    -e ENVIRONMENT="production" \
    -e JWT_SECRET="your-jwt-secret" \
    -e API_KEYS="your-evaluator-key" \
    -e SUPABASE_URL="http://your-datawizz-host/sbapi" \
    -e SUPABASE_SERVICE_KEY="your-service-role-key" \
    ghcr.io/datawizzai/evaluator-runner-cf:latest
VariableDescriptionDefault
PORTPort to listen on3000
ENVIRONMENTEnvironment namelocal
JWT_SECRETJWT secret (must match main instance)Required
API_KEYSComma-separated API keys for service-to-service authRequired
SUPABASE_URLDatawizz Supabase API URLRequired
SUPABASE_SERVICE_KEYSupabase service role keyRequired

​
Updating the Main Instance

After deploying the gateway services externally, update the .env on the main Datawizz instance so the app knows where to find them: 
# Point the app to the external gateway and evaluator
NEXT_PUBLIC_GATEWAY_BASE_URL=http://your-gateway-host:3000
NEXT_PUBLIC_EVALUATOR_TEST_API_URL=http://your-evaluator-host:3000
Then disable the gateway profile and restart the app: 
# Start without the gateway profile (services are external now)
docker compose up -d

# Or restart just the app if already running
docker compose restart datawizz-app
If you had previously used the gateway profile, stop and remove the local gateway containers: docker compose --profile gateway down.

​
Horizontal Scaling

You can run multiple instances of the AI Gateway behind a load balancer for high availability. Each instance independently fetches and caches configuration from the Datawizz API, so no shared state is needed between gateway nodes. A typical setup:
  1. Run 2+ AI Gateway containers across separate machines
  2. Place them behind a load balancer (e.g. AWS ALB, Nginx, Caddy)
  3. All gateway instances point to the same Request Logger (or a logger cluster)
  4. The Request Logger writes to ClickHouse
The gateway is stateless and lightweight, so it scales horizontally with minimal resource overhead.