Reverse Proxy / Load Balancer

MezHub can be deployed behind a TLS-terminating reverse proxy or load balancer. This is the recommended production topology — the load balancer handles TLS certificates, provides DDoS protection, and allows you to scale horizontally with multiple MezHub instances.

Architecture

When deployed behind a load balancer, the traffic flow is:

Client                    Load Balancer               MezHub
  |                           |                          |
  |---[TLS]--- HTTPS :443 --->|---[plaintext]--- :3080 ->|  Web UI + API
  |---[TLS]--- SSH :3023 ---->|---[TCP]--------- :3023 ->|  SSH proxy
  |---[TLS]--- Tunnel :3024 ->|---[TCP]--------- :3024 ->|  Agent reverse tunnels
  |---[TLS]--- gRPC :3025 --->|---[h2c]--------- :3025 ->|  Auth gRPC API

The load balancer terminates TLS on all ports and forwards plaintext to MezHub. MezHub runs with MEZITE_AUTH_H2C=true so it accepts plaintext connections on all ports. Clients still use TLS — they connect to the load balancer, which handles the encryption.

Why h2c for gRPC?

gRPC requires HTTP/2. When a load balancer terminates TLS, the connection between the load balancer and MezHub is plaintext. Standard HTTP/2 requires TLS (via ALPN negotiation), so plaintext HTTP/2 needs a special mode called h2c (HTTP/2 cleartext).

When MEZITE_AUTH_H2C=true is set, MezHub's gRPC auth server (port 3025) automatically runs in h2c mode. This wraps the gRPC server in an h2c handler that accepts both:

• h2c prior-knowledge — direct gRPC clients that send HTTP/2 without negotiation
• HTTP/1.1 Upgrade: h2c — reverse proxies (like nginx) that upgrade from HTTP/1.1 to HTTP/2

Enable h2c with MEZITE_GRPC_ALLOW_HTTP=true (recommended for production) or MEZITE_AUTH_H2C=true (which also disables mTLS on all ports). No other gRPC configuration is needed.

MezHub Configuration

cluster_name: my-cluster
data_dir: /var/lib/mezite

database:
  driver: sqlite          # or: postgresql
  url: /data/mezhub.db    # or: postgres://user:pass@host/mezite

auth:
  enabled: true
  listen_addr: 0.0.0.0:3025

proxy:
  enabled: true
  listen_addr: 0.0.0.0:3080
  ssh_listen_addr: 0.0.0.0:3023
  tunnel_listen_addr: 0.0.0.0:3024

  # The public address that clients use to reach this cluster.
  # This must match the hostname that resolves to your load balancer.
  public_addr: mezite.example.com:443

  # Disable TLS on the proxy — the load balancer handles TLS.
  tls_mode: "off"

# Enable h2c (plaintext HTTP/2) for the gRPC auth service.
# This is the targeted option — it only affects the gRPC port (3025).
MEZITE_GRPC_ALLOW_HTTP=true

# Alternative: MEZITE_AUTH_H2C=true enables h2c AND disables mTLS
# on all ports. Use MEZITE_GRPC_ALLOW_HTTP instead in production.

# Optional: trust the X-Forwarded-For header from the load balancer
# for rate limiting and audit logging of real client IPs.
# Set this to the header your load balancer uses.
MEZITE_TRUSTED_IP_HEADER=X-Forwarded-For

Preserving the real client IP — PROXY protocol v2

MEZITE_TRUSTED_IP_HEADER only works on HTTP listeners: it reads an HTTP header the load balancer injects. For the SSH, tunnel, and (in single-port mode) gRPC listeners there is no HTTP layer, so an HTTP header can't carry the source IP. Any deployment where the load balancer operates in TLS passthrough / L4 mode (e.g. AWS NLB with TCP targets, HAProxy in mode tcp) hits the same limit even on the HTTPS port: the LB never sees cleartext, so it can't add headers.

MezHub's listeners support PROXY protocol v1 and v2 — a binary header the LB prepends to each TCP connection that carries the original client address. It survives TLS passthrough because it sits in front of the TLS handshake rather than inside HTTP. Enable it on the listener and configure your LB to send it on every port you expose through MezHub.

proxy:
  proxy_protocol: "on"

  # Only peers whose direct TCP source IP falls in one of these CIDRs
  # are allowed to send PROXY headers. Keep this as tight as the LB's
  # actual egress range — any host inside the CIDR that can reach the
  # listener could otherwise forge client_ip in the audit log by
  # sending a crafted PROXY header.
  proxy_protocol_trusted_cidrs:
    - 10.0.0.0/24    # example: the LB's private subnet
    - 192.0.2.5/32   # example: a single-IP LB

Peers whose source IP is NOT in the trusted list are rejected if they send a PROXY header — so an attacker who can reach the port directly can't spoof client_ip. Trusted peers that send a connection without a PROXY header fall through to the raw TCP address, which means in-cluster health probes (kubelet, etc.) continue to work when you flip the flag on.

Once enabled, every downstream code path that previously saw the LB's IP — audit event client_ip, the per-IP connection limiter, role IP pinning — transparently sees the real caller. No code or query changes needed.

Port Mapping

Your load balancer needs to forward traffic to MezHub's internal ports. The mapping depends on whether you expose each service on its standard port or consolidate onto fewer external ports.

Standard Port Mapping

Agent Configuration

Agents connecting through a TLS-terminating load balancer need MEZITE_TLS_WRAP=true. This wraps the tunnel connection in TLS so the load balancer can terminate it, then the SSH tunnel protocol runs over the plaintext connection inside.

# Connect to the load balancer's tunnel port
MEZITE_PROXY_ADDR=mezite.example.com:3024

# Auth gRPC address (through the load balancer)
MEZITE_AUTH_ADDR=mezite.example.com:3025

# Wrap the tunnel in TLS for the load balancer
MEZITE_TLS_WRAP=true

# Standard agent config
MEZITE_JOIN_TOKEN=<your-join-token>
MEZITE_NODE_NAME=web-server-01
MEZITE_DATA_DIR=/var/lib/mezite

Without MEZITE_TLS_WRAP=true, the agent sends raw SSH protocol on the tunnel port. The load balancer wouldn't know how to handle this (it expects TLS on the external port). With TLS wrapping, the agent sends TLS on the wire, the load balancer terminates it, and the plaintext SSH tunnel protocol reaches MezHub.

Client Configuration

Clients (msh, mezctl) connect to the load balancer's external address. No special flags are needed — TLS is handled by the load balancer, and clients use standard TLS to connect.

# Login (connects to LB on :443, LB forwards to :3080)
msh login --proxy=mezite.example.com

# SSH (connects to LB on :3023, LB forwards to :3023)
msh ssh --login=ubuntu web-server-01

# Admin CLI (connects to LB on :3025, LB forwards h2c to :3025)
mezctl --auth-server=mezite.example.com:3025 nodes list

Nginx Example

# HTTPS → MezHub proxy (Web UI + API)
server {
    listen 443 ssl;
    server_name mezite.example.com;

    ssl_certificate /etc/ssl/mezite.crt;
    ssl_certificate_key /etc/ssl/mezite.key;

    location / {
        proxy_pass http://mezhub:3080;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        # WebSocket support (for web terminal)
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }
}

# gRPC → MezHub auth (port 3025)
server {
    listen 3025 ssl http2;
    server_name mezite.example.com;

    ssl_certificate /etc/ssl/mezite.crt;
    ssl_certificate_key /etc/ssl/mezite.key;

    location / {
        grpc_pass grpc://mezhub:3025;
        grpc_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

# SSH proxy (port 3023) — TCP passthrough with TLS termination
stream {
    upstream mezhub_ssh {
        server mezhub:3023;
    }
    server {
        listen 3023 ssl;
        ssl_certificate /etc/ssl/mezite.crt;
        ssl_certificate_key /etc/ssl/mezite.key;
        proxy_pass mezhub_ssh;
    }

    # Agent tunnel (port 3024) — TCP passthrough with TLS termination
    upstream mezhub_tunnel {
        server mezhub:3024;
    }
    server {
        listen 3024 ssl;
        ssl_certificate /etc/ssl/mezite.crt;
        ssl_certificate_key /etc/ssl/mezite.key;
        proxy_pass mezhub_tunnel;
    }
}

Troubleshooting

Agent tunnel fails with "SSH handshake: EOF"

The agent connected to the wrong port. Make sure MEZITE_PROXY_ADDR points to the tunnel port (3024), not the HTTPS port (443). The HTTPS port serves the Web UI and API — it does not speak the SSH tunnel protocol.

mezctl returns "connection refused" or TLS errors

mezctl connects directly to the gRPC port (3025). Make sure your load balancer exposes port 3025 and forwards it as h2c (HTTP/2 cleartext) to MezHub's internal port 3025.

Agent recording upload fails with 502

The agent uploads recordings via gRPC on port 3025. If your load balancer does not support HTTP/2 on this port, the upload will fail. Ensure the load balancer is configured for gRPC/h2c on port 3025. Set MEZITE_AUTH_H2C=true on MezHub so it runs the gRPC server in h2c mode.

Next Steps

• SSH Access — Set up agents and connect via SSH.
• Session Recording — Configure recording modes and storage.
• Configuration Reference — Full list of environment variables and config options.