Architecture

Mezite is built around three core components: the Auth Service, the Proxy Service, and Agents. Together they provide authenticated, authorized, and audited SSH access to infrastructure without exposing backend servers directly to the network.

This page covers the system topology, component responsibilities, port layout, certificate trust model, SSH connection data flow, database schema, and deployment topologies.

System Overview

The diagram below shows how clients, the proxy, the auth service, agents, and target SSH servers relate to each other. In a combined-mode deployment (the default), mezhub runs both Auth and Proxy in a single process. 

 ┌──────────────┐         HTTPS :3080          ┌──────────────────────────────────┐
 │              │ ──────────────────────────▶   │            mezhub               │
 │    msh CLI   │         SSH :3023             │  ┌────────────┐ ┌────────────┐  │
 │    Web UI    │ ──────────────────────────▶   │  │   Proxy    │ │    Auth    │  │
 │              │                               │  │  Service   │ │  Service   │  │
 └──────────────┘                               │  │            │ │            │  │
                                                │  │ :3080 HTTPS│ │ :3025 gRPC │  │
                                                │  │ :3023 SSH  │ │            │  │
                                                │  │ :3024 Tun  │ │  User CA   │  │
                                                │  │            │ │  Host CA   │  │
                                                │  │            │ │  Sessions  │  │
                                                │  └─────┬──────┘ └──────┬─────┘  │
                                                │        │    gRPC :3025 │        │
                                                │        └───────┬───────┘        │
                                                └────────────────┼────────────────┘
                                                                 │
                                                          ┌──────┴──────┐
                                                          │ PostgreSQL  │
                                                          │ or SQLite   │
                                                          └──────┬──────┘
                                                                 │
                      ┌──────────────────────────────────────────┘
                      │
       Reverse Tunnel :3024
                      │
        ┌────────────────────────────┐       ┌─────────────────────────┐
        │       mezd                 │       │      mezd               │
        │       (node-01)            │       │      (node-02)          │
        │                            │       │                         │
        │       SSH server           │       │       SSH server        │
        └────────────────────────────┘       └─────────────────────────┘

Components

Auth Service

The Auth service is the cluster's brain. It runs as a gRPC server on port 3025 and is responsible for:

  • User authentication — Verifying passwords (with optional TOTP), validating OIDC tokens, SAML assertions, LDAP credentials, and GitHub OAuth tokens, and issuing short-lived SSH certificates.
  • Certificate authorities — Managing the User CA (signs user certificates) and Host CA (signs host certificates). Both use ECDSA P-256 keys. CA private keys are stored encrypted in the database.
  • RBAC enforcement — Evaluating role bindings against requested actions. Uses label-based matching with deny-overrides-allow semantics and template variables.
  • Node registration — Accepting join tokens from agents, issuing host certificates, and maintaining the inventory of registered nodes.
  • Session management — Tracking active SSH sessions, enforcing session TTLs, and storing session recordings.
  • Audit event storage — Writing structured audit events to the configured database (SQLite or PostgreSQL) for every authentication, authorization decision, session start/end, and administrative action.

Proxy Service

The Proxy service is the public-facing entry point. It terminates all client connections and routes them to the appropriate agent. It exposes three listeners:

  • HTTPS (:3080) — Web UI, REST API, WebSocket session channels, and OIDC callback endpoints. The same listener also serves application access: when proxy.apps_domain is set, requests to an app's public hostname are SSO-gated, RBAC-checked, and forwarded down a serving agent's reverse tunnel to the app's internal address.
  • SSH (:3023) — Native SSH protocol. Clients connecting via msh ssh land here. The proxy authenticates the user's certificate, resolves the target node, and forwards the connection through the agent's reverse tunnel.
  • Tunnel (:3024) — Persistent reverse-tunnel connections from agents. Each agent maintains an SSH connection to this port (over plain TCP, or wrapped in TLS / ALPN when fronted by a TLS-terminating load balancer); the proxy multiplexes client connections over these tunnels.

Agent (mezd)

Agents run on every target machine. On first start they generate a local host key pair, then call GenerateHostCerts on the Auth service to obtain a host certificate. Once they hold a host certificate they call RegisterAgent — or, when joining via AWS IAM, RegisterUsingIAMMethod — which records the node in the inventory and exchanges the one-time join token (or IAM identity proof) for ongoing authentication material. With registration complete the agent establishes a persistent reverse tunnel to the Proxy. Key responsibilities:

  • Reverse tunnel — Maintains a persistent outbound connection to the proxy's tunnel port (:3024). This means agents do not need any inbound firewall rules.
  • SSH server — Runs an SSH server that accepts connections forwarded through the tunnel. Verifies user certificates against the User CA.
  • Session recording — Captures terminal I/O at the PTY level after SSH decryption. The agent sees the clean text that the user sees in their terminal, not encrypted SSH protocol bytes. Recordings are streamed to the Auth service in real-time (node-sync mode) or uploaded after the session ends (node mode).

Certificate Authorities

Mezite operates four certificate authorities, all managed by the Auth service and all created on first start:

  • User CA (ECDSA P-256) — Issues short-lived SSH certificates to authenticated users. These certificates encode the user's identity, roles, and permitted principals.
  • Host CA (ECDSA P-256) — Issues host certificates to agents during registration. These certificates identify the host to connecting users, preventing MITM attacks. The same CA also signs the X.509 server certificate that mezhub presents on its gRPC listener.
  • SPIFFE CA (ECDSA P-256) — Issues X.509-SVIDs and signs JWT-SVIDs for the workload-identity surface exposed by mezd identity. Workloads on a registered node connect to a local Unix socket and receive short-lived SPIFFE certificates whose URI SAN encodes the cluster's trust domain plus the workload's role, suitable for mTLS or token-bound auth between services on the node.
  • App CA (ECDSA P-256) — Signs the short-lived identity JWT the proxy injects into requests it forwards to a protected application. It is a separate trust domain from the User CA so application-access assertions can be rotated independently of SSH access. See Application Access.

CA private keys are stored in the certificate_authorities table in the configured database (SQLite or PostgreSQL). They are AES-256-GCM encrypted at rest when ca_key_passphrase is set; the key is derived from the passphrase via Argon2id. With kms.enabled=true, CA signing is routed through AWS KMS instead and the raw private key never leaves the KMS HSM boundary — the database row holds the KMS key alias rather than the key material.

CA Rotation

Each CA can be rotated without interrupting in-flight sessions. Rotation runs as a multi-phase state machine driven by mezctl ca rotate / mezctl ca advance / mezctl ca rollback:

  1. Init — mezctl ca rotate generates a new key pair and makes it the active signing key immediately. The old key is retained and stays in the trust bundle, so every certificate already in circulation keeps validating. Clients and agents fetch the two-key trust bundle and pin both.
  2. Update Clients — A checkpoint, not a behaviour change: advance here once users have re-logged in, so every user certificate in circulation is signed by the new key.
  3. Update Servers — The second checkpoint: advance once agents and the proxy have reissued their host certificates under the new key and no live server is still presenting an old-key cert.
  4. Complete — The previous key material is deleted from the database (or, under KMS, scheduled for deletion) and the rotation state returns to standby. Trust is single-key again.

mezctl ca rollback restores the previous key and is the escape hatch if a phase advance breaks a population of clients you do not control. It works in init, update_clients and update_servers; advancing to complete is the point of no return, because the old key material is gone by then. mezctl ca status renders the current phase and certificate expiry so an operator can verify intent against the audit log.

Port Reference

PortProtocolComponentDescription
3025gRPCAuth ServiceInternal auth API. Used by proxies, agents, and admin CLI. Should not be exposed publicly.
3080HTTPSProxy ServiceWeb UI, REST API, OIDC callbacks. The primary public endpoint.
3023SSHProxy ServiceSSH client connections. Clients connect here via msh.
3024SSHProxy ServiceAgent reverse-tunnel connections. Agents connect outbound to this port.
5432PostgreSQLDatabaseState store (only when using the PostgreSQL backend; the default SQLite backend has no network port).

ALPN Single-Port Mode

With proxy.single_port=true (env MEZITE_SINGLE_PORT), the proxy accepts SSH, agent tunnel, and HTTPS traffic on the single HTTPS listener. Clients negotiate one of Mezite's ALPN protocol IDs during the TLS handshake; the proxy demultiplexes the connection to the right internal handler based on the negotiated protocol. This lets a cluster be reached entirely on :443, which is the right shape when the only port that reliably traverses corporate firewalls and L4 load balancers is :443.

Single-port mode does not change the rest of the system. The internal :3023 SSH and :3024 tunnel listeners still exist for environments that prefer port-per-service routing, and the same auth/RBAC/recording paths run regardless of which listener accepted the connection.

SSH Connection Data Flow

When a user runs msh ssh --login=user node-01, the following sequence occurs:

SSH connection flow — step by step text 
1. msh opens an SSH connection to Proxy :3023
2. Proxy verifies the user's SSH certificate (signed by User CA)
3. Proxy evaluates RBAC: does this user's role allow login "user" on node-01?
4. Proxy looks up node-01 in the Auth service node registry
5. Proxy resolves node-01 to the agent's reverse tunnel
6. Proxy forwards the SSH channel through the tunnel to the agent
7. Agent verifies the user certificate against the User CA public key
8. Agent starts an SSH session as the requested login user
9. Agent allocates a PTY and records terminal I/O at the PTY level
10. Agent streams recording chunks to Auth via gRPC in real-time (node-sync mode)
11. Agent streams terminal I/O back through the tunnel to the client
12. Auth stores the recording and writes audit events: session.start, session.end

Certificate Trust Model

Mezite uses a mutual SSH certificate trust model. Both users and hosts present certificates signed by trusted CAs. This eliminates the need for password-based SSH, shared keys, or authorized_keys files. 

 ┌─────────────────────────────────────────────────────────┐
 │                    Auth Service                         │
 │                                                         │
 │   ┌──────────────┐              ┌──────────────┐        │
 │   │   User CA    │              │   Host CA    │        │
 │   │  (ECDSA P-256)│              │ (ECDSA P-256)│        │
 │   └──────┬───────┘              └──────┬───────┘        │
 │          │                             │                │
 └──────────┼─────────────────────────────┼────────────────┘
            │                             │
            ▼                             ▼
   ┌────────────────┐            ┌────────────────┐
   │ User Certs     │            │ Host Certs     │
   │                │            │                │
   │ identity: bob  │            │ host: node-01  │
   │ roles: [admin] │            │ cluster: prod  │
   │ principals:    │            │ valid: 24h     │
   │   [root, bob]  │            │                │
   │ valid: 12h     │            │                │
   └────────────────┘            └────────────────┘
          │                             │
          │ presented to                │ presented to
          ▼                             ▼
   ┌──────────────┐             ┌──────────────┐
   │ Agent (host) │             │  msh (user)  │
   │ trusts       │             │  trusts      │
   │ User CA      │             │  Host CA     │
   └──────────────┘             └──────────────┘

The trust chain works as follows:

  1. User login — When a user authenticates (password+TOTP, OIDC, SAML, LDAP, or GitHub OAuth), the Auth service issues a short-lived SSH certificate signed by the User CA. The certificate encodes the user's name, roles, and allowed principals.
  2. Host registration — When an agent joins the cluster with a valid token, the Auth service issues a host certificate signed by the Host CA. The certificate encodes the node's hostname and cluster membership.
  3. Mutual verification — During an SSH connection, the agent verifies the user's certificate against the User CA public key, and the user's client verifies the host's certificate against the Host CA public key. Both sides reject connections if the certificate is expired, revoked, or signed by an unknown CA.

Database Schema

All cluster state is stored in the database (SQLite or PostgreSQL). The schema is managed by mezhub using golang-migrate and auto-migrates on startup. Below are the primary tables and their purpose.

TableDescriptionKey Columns
usersUser accounts. Stores credentials (bcrypt hashes for local auth), roles, and metadata.id, username, password_hash, roles, created_at, updated_at
rolesRBAC role definitions. The role spec (allowed logins, node label selectors, deny rules, session TTLs) is serialized into the spec JSON column.id, name, spec, builtin, created_at, updated_at
certificate_authoritiesCA keypairs (User CA and Host CA), each holding both an SSH and an X.509 key. Private keys are AES-256-GCM encrypted when a passphrase is set. Rotation state lives on the same row.id, ca_type, cluster_name, ssh_public_key, ssh_private_key, x509_cert_pem, x509_key_pem, rotation_state, rotation_phase
registered_agentsRegistered SSH agents (nodes). Updated by agents on heartbeat.agent_id, hostname, labels, status, last_heartbeat
session_recordingsMetadata for active and completed SSH session recordings.session_id, user_name, login, server_hostname, storage_backend, storage_path, started_at, ended_at
audit_eventsImmutable audit log. Every authentication, authorization decision, session lifecycle event, and administrative action is recorded here.id, event_type, event_code, user_name, resource_type, resource_name, success, details, timestamp
access_requestsJust-in-time access requests.id, requester, requested_roles, reason, state, created_at, resolved_at, access_expires_at
auth_connectorsSSO connector configurations. connector_type is one of oidc, saml, github or ldap.id, name, connector_type, spec, enabled, created_at

Migrations are embedded in the mezhub binary and applied automatically at startup — there is no separate migration command to run, and no migration files to ship alongside the binary. On PostgreSQL the migrator takes an advisory lock, so replicas starting at the same time serialise instead of racing.

Deployment Topologies

Single Instance

One mezhub process runs Auth and Proxy together — they are not separable, and there is no proxy-only or auth-only mode. This is the simplest deployment and the right starting point for any cluster.

Single-instance topology text 
┌───────────────────────────┐
│         mezhub            │
│  Auth :3025 + Proxy :3080  │
│         + SSH :3023        │
│         + Tunnel :3024     │
└─────────────┬─────────────┘
              │
    SQLite file or PostgreSQL :5432

Why There Is No Multi-Instance Topology Yet

An agent's reverse tunnel is held in the memory of exactly one mezhub process, and there is no proxy-to-proxy forwarding: an SSH connection can only reach a node whose tunnel terminates on the instance that accepted the connection. Agents also connect to a single proxy address rather than fanning out across instances.

Put several mezhub instances behind a round-robin load balancer and a client will intermittently land on an instance that holds no tunnel for the node it asked for, and the connection fails. Run one instance per cluster. Scale it vertically, and use a highly-available PostgreSQL if you want the database tier redundant.

Next Steps