MĀRGA — LLM RouterQuick Start

Quick Start Guide

Get MĀRGA up and running in 5 minutes.

Prerequisites

  • Docker (recommended) OR Go 1.21+
  • API keys for at least one provider (OpenAI, Anthropic, etc.)

Option 1: Docker (Recommended)

Step 1: Pull the Image

docker pull ghcr.io/gaurav21/marga:latest

Step 2: Create Environment File

# Create a simple .env file
cat > .env << EOF
OPENAI_API_KEY=sk-your-openai-api-key-here
ANTHROPIC_API_KEY=sk-ant-your-anthropic-key-here
EOF

Step 3: Run MĀRGA

docker run -d \
  --name marga \
  -p 8080:8080 \
  --env-file .env \
  ghcr.io/gaurav21/marga:latest

Step 4: Verify It’s Running

# Check health
curl http://localhost:8080/health
 
# Expected response:
# {"status":"ok","service":"marga","version":"0.1.0","providers":2}

Option 2: Docker Compose

Step 1: Clone Repository

git clone https://github.com/gaurav21/avyay-marga
cd avyay-marga

Step 2: Configure Environment

# Copy example environment file
cp .env.example .env
 
# Edit with your API keys
nano .env  # or vi .env

Step 3: Start All Services

# Start MĀRGA with optional Ollama and Prometheus
docker-compose up -d
 
# Check status
docker-compose ps

Option 3: From Source

Step 1: Clone and Build

git clone https://github.com/gaurav21/avyay-marga
cd avyay-marga
 
# Install dependencies
go mod download
 
# Build
go build -o marga ./cmd/server

Step 2: Configure

# Copy default config
cp config.yaml my-config.yaml
 
# Set environment variables
export OPENAI_API_KEY="sk-your-key-here"
export ANTHROPIC_API_KEY="sk-ant-your-key-here"

Step 3: Run

# Start the server
CONFIG_FILE=my-config.yaml ./marga

Making Your First Request

Once MĀRGA is running, test it with a chat completion request:

curl -X POST http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer dummy-key" \
  -d '{
    "model": "gpt-4",
    "messages": [
      {
        "role": "user", 
        "content": "Hello! Can you tell me about MĀRGA?"
      }
    ],
    "max_tokens": 150
  }'

Expected response:

{
  "id": "chatcmpl-abc123",
  "object": "chat.completion",
  "created": 1699649600,
  "model": "gpt-4o",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Hello! MĀRGA is an enterprise LLM router..."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 15,
    "completion_tokens": 45,
    "total_tokens": 60
  }
}

Verification Steps

1. Check Health Endpoint

curl http://localhost:8080/health

Should return {"status":"ok","service":"marga","version":"0.1.0","providers":N}

2. Check Configuration

curl http://localhost:8080/v1/config

Shows enabled providers and routing configuration.

3. View Metrics

curl http://localhost:8080/v1/metrics

Prometheus-style metrics for monitoring.

4. Test Different Models

# Test Claude
curl -X POST http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer dummy-key" \
  -d '{"model": "claude-3-sonnet", "messages": [{"role": "user", "content": "Hi Claude!"}]}'
 
# Test with model mapping
curl -X POST http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer dummy-key" \
  -d '{"model": "gpt-4", "messages": [{"role": "user", "content": "Hi GPT-4!"}]}'

Common Issues & Solutions

Issue: “no healthy provider found”

Cause: API keys not set or invalid

Solution:

# Check environment variables
echo $OPENAI_API_KEY
echo $ANTHROPIC_API_KEY
 
# Verify keys work directly
curl https://api.openai.com/v1/models \
  -H "Authorization: Bearer $OPENAI_API_KEY"

Issue: Connection refused

Cause: MĀRGA not running or wrong port

Solution:

# Check if running
docker ps | grep marga
# or
ps aux | grep marga
 
# Check port
netstat -tulpn | grep 8080

Issue: Provider timeouts

Cause: Network connectivity or provider issues

Solution: Check provider status and network connectivity:

# Test direct connectivity
curl https://api.openai.com/v1/models
curl https://api.anthropic.com/v1/messages

Next Steps

  1. Configure routing - Customize provider selection and failover
  2. Set up monitoring - Add Datadog integration and alerting
  3. Deploy to production - Kubernetes, Cloud Run, or Docker Swarm
  4. Explore use cases - Learn about cost optimization and A/B testing

Production Checklist

Before deploying to production:

  • Set secure API keys in production environment
  • Configure proper CORS origins
  • Set up rate limiting
  • Enable authentication with real API keys
  • Configure monitoring and alerting
  • Set up log aggregation
  • Test failover scenarios
  • Document your routing rules

Getting Help

OverviewAPI Reference