roxy

Roxy the Frontend Proxy

View on GitHub

Roxy logo

Roxy: The Full Configuration Reference

Roxy’s configuration files use JSON format. The primary config file lives at /etc/opt/roxy/config.json, while the MIME rules live at /etc/opt/roxy/mime.json.

The overall layout of config.json looks like this:

{
  "global": {...},     # Section "global" (an Object)
  "hosts": [...],      # Section "hosts" (an Array of String)
  "frontends": {...},  # Section "frontends" (an Object)
  "rules": [...]       # Section "rules" (an Array of Object)
}

All sections are technically optional, but nearly every setup will want to define "hosts", "frontends", and "rules". Advanced users will also care about "global".

Full working example

Here is an example Roxy configuration for config.json that demonstrates both static content serving and reverse proxying, plus a few simple header rewrites:

{
  "hosts": [
    "*.example.com",
    "subdomain.example.org",
    "other.example.org"
  ],
  "frontends": {
    "fs-example-com": {
      "type": "fs",
      "path": "/srv/www/example.com/htdocs"
    },
    "fs-subdomain-example-org": {
      "type": "fs",
      "path": "/srv/www/subdomain.example.org/htdocs"
    },
    "http-other-example-org": {
      "type": "http",
      "client": {
        "target": "dns:///127.0.0.1:8001"
      }
    }
  },
  "rules": [
    {
      "match": {
        "Host": "(.*\\.)?example\\.com"
      },
      "frontend": "fs-example-com"
    },
    {
      "match": {
        "Host": "subdomain\\.example\\.org"
      },
      "frontend": "fs-subdomain-example-org"
    },
    {
      "match": {
        "Host": "other\\.example\\.org"
      },
      "mutations": [
        {
          "type": "response-header-post",
          "header": "Server",
          "search": ".*",
          "replace": "Apache/2.4"
        },
        {
          "type": "response-header-post",
          "header": "Content-Security-Policy",
          "search": "(.*);",
          "replace": "\\1; image-src 'self' https://*.example.com;"
        }
      ],
      "frontend": "http-other-example-org"
    },
    {
      "frontend": "ERROR:404"
    }
  ]
}

Section "global"

Section "global" groups together miscellaneous configuration items that don’t fit in any other category.

It contains the following fields and subsections:

{
  "global": {
    "mimeFile": "...",                # Path to mime.json (a String)
    "acmeDirectoryURL": "...",        # ACME server to use (a String)
    "acmeRegistrationEmail": "...",   # E-mail address to report to ACME server (a String)
    "acmeUserAgent": "...",           # User agent to report to ACME server (a String)
    "maxCacheSize": 65536,            # Max file size for in-RAM cache (a Number)
    "maxComputeDigestSize": 4194304,  # Max file size for automatic Digest/ETag headers (a Number)
    "zk": {...},                      # Subsection "global.zk" (an Object)
    "etcd": {...},                    # Subsection "global.etcd" (an Object)
    "storage": {...},                 # Subsection "global.storage" (an Object)
    "pages": {...}                    # Subsection "global.pages" (an Object)
  },
  ...
}

Field "global.mimeFile"

Field "global.mimeFile" specifies the path to the mime.json ancillary configuration file. If empty or not specified, it defaults to "/etc/opt/roxy/mime.json".

If the file does not exist, it is not a fatal error. Instead, Roxy will fall back on its built-in defaults, which should be identical to the mime.json.example file that ships with Roxy.

The MIME file itself is in JSON format and contains an Array of Objects. It has the following structure:

[
  ...
  {
    "suffixes": ["..."],       # Literal suffixes matched against the full path (an Array of String)
    "contentType": "...",      # "Content-Type" header (a String; default `application/octet-stream`)
    "contentLanguage": "...",  # "Content-Language" header (a String; default absent)
    "contentEncoding": "..."   # "Content-Encoding" header (a String; default absent)
  },
  ...
]

Here’s a concrete example:

[
  {
    "suffixes": [".html", ".htm"],
    "contentType": "text/html; charset=utf-8",
    "contentLanguage": "en-US"
  }
]

You can override the “Content-Type”, “Content-Language”, and “Content-Encoding” headers on a file-by-file basis using extended attributes (“xattrs”), as described in the docs for field "global.maxComputeDigestSize".

Field "global.acmeDirectoryURL"

Field "global.acmeDirectoryURL" controls which ACME server endpoint Roxy uses to obtain TLS certificates. The default is https://acme-v02.api.letsencrypt.org/directory, which is the endpoint for Let’s Encrypt.

You can change it to the endpoint for another ACME provider if you wish, such as https://acme.zerossl.com/v2/DV90 to connect to ZeroSSL.

Field "global.acmeRegistrationEmail"

Field "global.acmeRegistrationEmail" controls the e-mail address associated with your ACME client account. Your ACME provider may use this e-mail from time to time to warn you about problems with your TLS certificates. You can omit this, if you prefer, but it is recommended.

Roxy itself does not care about your e-mail address, and will not reveal it to anyone other than your ACME provider.

Field "global.acmeUserAgent"

Field "global.acmeUserAgent" controls the user agent string which Roxy presents to your ACME provider. This helps your ACME provider track down Roxy’s creators if Roxy is misbehaving. The default is roxy/<version>, which is probably what you want.

Field "global.maxCacheSize"

Field "global.maxCacheSize" controls the maximum size of files stored in the in-memory file cache. The file cache is used to accelerate serving of static content, at the cost of RAM. The default is 65536 bytes, or 64 KiB. You can set this to -1 to disable in-memory caching entirely.

NB: the file’s size and last modified time are still checked on every request, even after a cache hit. This ensures that stale files are never served.

Field "global.maxComputeDigestSize"

Field "global.maxComputeDigestSize" controls the maximum size of files for which Roxy will automatically generate the “Digest” and “ETag” headers. These headers are used for integrity checking and for resuming interrupted downloads. However, computing these headers requires scanning through all bytes of the file before delivering the first byte to the waiting HTTP client, so this is a trade-off between functionality and timeliness. The default is 4194304 bytes, or 4 MiB, which is a trade-off chosen by the Roxy developers to be appropriate for low-end SSDs or fast spinning disks. You can set this to -1 to disable automatic computation of the “Digest” and “ETag” headers for non-cached files. (These headers are always computed for files served from the in-memory cache.)

NB: you can also set the “Digest” and “ETag” headers using ext4 extended attributes (“xattrs”). Roxy respects the following xattrs:

You can set the xattrs on a file using the setfattr command.

Subsection "global.zk"

Subsection "global.zk" enables the use of Apache ZooKeeper to store TLS certificates (subsection "global.storage") and to look up servers (section "frontends"). It has the following structure:

{
  "global": {
    ...
    "zk": {
      "servers": ["..."],       # Addresses of the ZK cluster members (an Array of String; required)
      "sessionTimeout": "30s",  # Max time that the ZK cluster should keep our session alive if we get disconnected (a String)
      "auth": {
        "scheme": "digest",     # An auth scheme; "digest" is common, other schemes are possible, see ZooKeeper docs (a String; required if "auth" present)
        "raw": "...",           # Auth data; most users will use "username" and "password" instead (a String; base-64 format)
        "username": "...",      # Username to use (a String)
        "password": "..."       # Password to use (a String)
      }
    },
    ...
  },
  ...
}

For a single-homed ZK cluster running on localhost with no authentication, this simplifies to:

{
  "global": {
    "zk": {
      "servers": ["127.0.0.1:2181"]
    }
  }
}

Managing a ZooKeeper cluster is beyond the scope of this documentation.

Subsection "global.etcd"

Subsection "global.etcd" enables the use of Etcd to store TLS certificates (see subsection "global.storage") and to look up servers (see section "frontends"). It has the following structure:

{
  "global": {
    ...
    "etcd": {
      "endpoints": ["..."],     # Addresses of the etcd cluster members (an Array of String; required)
      "tls": {...},             # TLS client configuration (an Object; see below)
      "username": "...",        # Etcd username (a String)
      "password": "...",        # Etcd password (a String)
      "dialTimeout": "5s",      # Max time to wait when connecting (a String)
      "keepAliveTime": "30s",   # Time between sending of keep-alive requests (a String)
      "keepAliveTimeout": "2m"  # Max time without receiving a keep-alive reply (a String)
    },
    ...
  },
  ...
}

For a single-homed etcd cluster running on localhost with no TLS and no authentication, this simplifies to:

{
  "global": {
    "etcd": {
      "endpoints": ["http://localhost:2379"]
    }
  }
}

Managing an etcd cluster is beyond the scope of this documentation.

Subsection "global.storage"

Subsection "global.storage" determines where Roxy will store TLS certificates obtained via the ACME protocol, as well as its long-lived private key for speaking with the ACME server. It has the following structure:

{
  "global": {
    ...
    "storage": {
      "engine": "...",  # Which storage engine to use; one of "fs", "etcd", or "zk" (a String; required)
      "path": "..."     # Configuration for the storage engine (a String; required)
    },
    ...
  },
  ...
}

The "global.storage.engine" field is the name of a storage engine:

(NB: Etcd 3.x does not have “paths” and “directories”, per se; instead, "path" is suffixed with "/" to form a search prefix, and each “file” is a key-value pair formed by concatenating the search prefix with the filename. This feels enough like a directory that the filesystem-like nomenclature still fits, with only a few caveats.)

The default, which takes effect only if there is no "global.storage" subsection at all, is:

{
  "global": {
    "storage": {
      "engine": "fs",
      "path": "/var/opt/roxy/lib/acme"
    }
  }
}

Subsection "global.pages"

Subsection "global.pages" tells Roxy where to find your custom HTML templates for error pages, redirects, and filesystem index pages. It has the following structure:

{
  "global": {
    ...
    "pages": {
      "rootDir": "...",                 # Path to the template directory (a String; required)
      "defaultContentType": "...",      # Default value for the "Content-Type" header (a String)
      "defaultContentLanguage": "...",  # Default value for the "Content-Language" header (a String)
      "defaultContentEncoding": "...",  # Default value for the "Content-Encoding" header (a String)
      "map": {...}                      # Used to customize individual error codes (an Object)
    },
    ...
  },
  ...
}

The "global.pages.rootDir" field is a filesystem path that points to a directory containing HTML templates (in Go’s html/template format).

The "global.pages.map" field is structured as:

"map": {
  ...
  "<key>": {
    "fileName": "...",          # Relative path to HTML template file (a String)
    "contentType": "...",       # "Content-Type" header (a String)
    "contentLanguage": "...",   # "Content-Language" header (a String)
    "contentEncoding": "..."    # "Content-Encoding" header (a String)
  },
  ...
}

For HTTP 4xx and 5xx errors, the following keys are checked in "global.pages.map":

For HTTP 3xx redirects, the following keys are checked:

For automatically-generated filesystem directory indexes, the following key is checked:

All fields within "global.pages.map" are optional. In fact, all entries in the map are optional. If there is no entry in the map for a given key, then:

If the computed filename does not exist, then the next key is tried. If the "error", "redir", or "index" keys do not exist, then the Roxy built-in default templates are used.

Section "hosts"

Section "hosts" is a list of host patterns. A host pattern is a string, which matches one of the following patterns: a domain name, a domain name prefixed with "*.", or the exact string "*".

This controls which domains Roxy is willing to serve, and more importantly, which domains Roxy is willing to obtain ACME certificates for.

Section "frontends"

Section "frontends" is a map from frontend names (strings) to frontend configurations (objects). A frontend name is a unique identifier that will be used by the "rules" section to refer back to the frontend configuration.

The frontend configuration has the following structure:

{
  ...
  "frontends": {
    ...
    "<name>": {
      "type": "...",    # The frontend type (a String; one of "fs", "http", or "grpc"; required)
      "path": "...",    # Path to the directory to serve (a String)
      "client": {...}   # HTTP or gRPC client configuration (an Object)
    },
    ...
  },
  ...
}

The "<name>" key is a unique identifier for this frontend configuration. The name must consist of letters, numbers, or the punctuation characters _, ., +, and -. The name cannot begin with a number or punctuation, nor can it end with punctuation, nor can two punctuation characters appear next to each other.

The "type" field selects which frontend type to use for this frontend configuration:

The "path" field is required for "type": "fs", and is forbidden for other types. It specifies the local filesystem directory out of which static content is served. See "global.maxCacheSize" and "global.maxComputeDigestSize" for configuration options.

NB: The "fs" type does not support disabling of automatically-generated directory indexes, and only supports index files with the exact name index.html.

The "client" field is required for "type": "http" and "type": "grpc", and is forbidden for other types. It specifies the “client config”, i.e. how to connect to the hosts being reverse proxied. See the “Client configs” heading.

A simple frontend for static file serving might look like this:

{
  "frontends": {
    "my-frontend-name": {
      "type": "fs",
      "path": "/srv/www"
    }
  }
}

A frontend that uses mutual TLS (“mTLS”) to connect to an HTTPS backend, on the other hand, might look like this:

{
  "frontends": {
    "my-frontend-name": {
      "type": "http",
      "client": {
        "target": "dns:///backend.internal:443",
        "tls": {
          "clientCert": "/path/to/client/cert.pem",
          "clientKey": "/path/to/client/key.pem"
        }
      }
    }
  }
}

Section "rules"

Section "rules" is structured as an Array of Objects, where each Object represents a single “rule”. A rule is an optional set of matching criteria, an optional list of mutations to apply, and an optional frontend name. It does not make sense to specify a rule with no mutations and no frontend, but nothing prevents you from doing this.

"rules": [
  ...
  {
    "match": {...},      # Headers to match (Object of String)
    "mutations": [...],  # Mutations to apply to matching requests (Array of Object)
    "frontend": "..."    # A frontend name (String)
  },
  ...
]

Field "match"

The "match" field consists of zero or more header matches, where each match is composed of the name of the header (Object key) and the regular expression which the header’s value must match (Object value). The header name is case-insensitive.

NB: the regexp is implicitly anchored with ^ and $, i.e. a full string match.

The rule only matches if all header matches have successfully matched against the original, unmodified request, as it existed before any mutation rules. If any of the headers fails to match, then the entire rule is ignored.

Omitting the "match" field, or having "match" set to an empty Object, will cause the rule to match all requests unconditionally.

There are a few special header names:

Field "mutations"

The "mutations" field consists of zero or more mutations that are applied if (and only if) the request matched the current rule. The general structure of a mutation object is as follows:

"mutations": [
  ...
  {
    "type": "...",    # The mutation type (a String; required)
    "header": "...",  # The name of the header to mutate (a String; required for some types)
    "search": "...",  # The regexp to match against the existing value (a String; required)
    "replace": "..."  # The replacement for the field's value (a String; required)
  },
  ...
]

The "type" is one of the following mutation types:

The "header" field is required for the 3 mutation types that deal with headers, and is forbidden otherwise. It is case-insensitive.

NB: the regexp is implicitly anchored with ^ and $, i.e. a full string match.

The replacement string is a template in Go "text/template" format, which is called with . equal to the []string returned from calling FindStringSubmatch on the "search" regexp.

NB: As a convenience, the strings \\0 through \\9 are synonyms for {{ index . N }} in the "replace" string.

Field "frontend"

The "frontend" field consists of the name of a frontend (see Section “Frontends”), or one of a handful of special strings indicating a built-in frontend.

If the "frontend" field is present at all, it means that every request which matches the current rule will stop processing all later rules. No further rules will be processed (first match wins). Conversely, if no "frontend" field is present, then processing continues to the next matching rule.

Here are the special built-in frontends:

The <url> in REDIR:<status>:<url> is a template in Go "text/template" format, which is called with . equal to a *url.URL representing the current request URI, with scheme and host filled in, and with all mutations made up to this point. This means that, if you want to rewrite the URL and then redirect the client to the mutated URL, "frontend": "STATUS:302:" will do nicely.

Client configs

A client config has two parts: a mandatory "target" field, containing a String in target spec format, and an optional "tls" field, containing an Object in TLS client configuration format. The structure is as follows:

{
  "target": "...",  # The target spec (a String; required)
  "tls": {...}      # The TLS client configuration (an Object)
}

Target specs

A target spec is defined similarly to the gRPC Name Syntax: a target spec is a URL-like string which contains an optional “scheme”, an optional “authority”, a mandatory “endpoint”, and an optional “query”.

The meaning of the authority, endpoint, and query depends on the scheme.

The following schemes are supported, both for HTTP and for gRPC:

Scheme "unix"

Complete syntax: "unix:/path/to/socket?balancer=<algo>&serverName=<name>"

The /path/to/socket (of the mandatory endpoint string) is a filesystem path pointing to an AF_UNIX socket.

The optional balancer=<algo> query parameter specifies the balancer algorithm to use. See the “Balancer algorithms” heading.

The optional serverName=<name> query parameter specifies the expected DNSName SAN on the TLS certificate, overriding the default of "localhost". This only has an effect when TLS is in use.

Scheme "ip"

Complete syntax: "ip:<ip:port>[,<ip2:port>...]?balancer=<algo>&serverName=<name>"

The <ip:port>[,<ip2:port>...] (of the mandatory endpoint string) is a comma-separated list of IP addresses and port numbers. The ports are optional, with a default of 80 without TLS or 443 with TLS.

The optional balancer=<algo> query parameter specifies the balancer algorithm to use. See the “Balancer algorithms” heading.

The optional serverName=<name> query parameter specifies the expected DNSName SAN on the TLS certificate, overriding the default of <ip>. This only has an effect when TLS is in use.

Scheme "dns"

Complete syntax: "dns://<server:port>/<domain:port>?balancer=<algo>&pollInterval=<dur>&serverName=<name>"

The <server:port> (of the optional authority string) names a specific DNS server, overriding your OS /etc/resolv.conf settings. If this is present at all, the port is almost always 53, which is the default. Very few people will want to specify this.

The <domain:port> (of the mandatory endpoint string) is the domain name to resolve, plus a named or numbered port. The domain specifies the A/AAAA records to query. The port is optional, with a default of 80 without TLS or 443 with TLS. All hosts must use the same port.

The optional balancer=<algo> query parameter specifies the balancer algorithm to use. See the “Balancer algorithms” heading.

The optional pollInterval=<dur> query parameter specifies how long to cache the DNS query results in memory before making another query. The default is a fairly aggressive "1m" (one minute).

The optional serverName=<name> query parameter specifies the expected DNSName SAN on the TLS certificate, overriding the default of <domain>. This only has an effect when TLS is in use.

Scheme "srv"

Complete syntax: srv://<server:port>/<domain>/<service>?balancer=<algo>&pollInterval=<dur>&serverName=<name>

The <server:port> (of the optional authority string) names a specific DNS server, overriding your OS /etc/resolv.conf settings. If this is present at all, the port is almost always 53, which is the default. Very few people will want to specify this.

The <domain>/<service> (of the mandatory endpoint string) is used to construct the domain name to resolve. Both parts are mandatory. DNS SRV records will be queried at _<service>._tcp.<domain>, followed by lookups of the A/AAAA records of the resulting domain names. Each SRV record specifies its own port, as well as a priority and weight that can be used by a special balancer algorithm.

The optional balancer=<algo> query parameter specifies the balancer algorithm to use. See the “Balancer algorithms” heading.

The optional pollInterval=<dur> query parameter specifies how long to cache the DNS query results in memory before making another query. The default is a fairly aggressive "1m" (one minute).

The optional serverName=<name> query parameter specifies the expected DNSName SAN on the TLS certificate, overriding the default of the domain(s) named by the SRV records. (That is, the default ServerName is the name of the A/AAAA records, not the name of the SRV records.) This only has an effect when TLS is in use.

Scheme "zk"

Complete syntax: zk:///path/to/dir:namedPort?balancer=<algo>

The authority section must be empty.

The mandatory path/to/dir (of the mandatory endpoint string) specifies the path to a ZooKeeper directory containing files, one per host, in either of two JSON syntaxes:

If path/to/dir does not begin with a /, one is automatically added for you.

The optional portName (of the mandatory endpoint string) names a port within the Finagle ServerSet data.

The optional balancer=<algo> query parameter specifies the balancer algorithm to use. See the “Balancer algorithms” heading.

Scheme "etcd"

Complete syntax: etcd:///prefix/string:namedPort?balancer=<algo>

The mandatory prefix/string (of the mandatory endpoint string) specifies the prefix of an Etcd keyspace containing key-value pairs, one per host, in either of two JSON syntaxes:

If prefix/string does not begin with a /, it will not be added for you. Etcd 3.x allows keys whose names don’t start with /, even though the / is conventional. If your prefix/string starts with a /, use four slashes between the scheme and the endpoint, like so: etcd:////path/to/dir.

The prefix/string may end with a /. One will be automatically added for you if it does not already do so.

The optional balancer=<algo> query parameter specifies the balancer algorithm to use. See the “Balancer algorithms” heading.

Balancer algorithms

Balancer "random"

This balancer picks a host at random with uniform probability.

Balancer "roundRobin"

This balancer shuffles the hosts into a random permutation, then cycles through the hosts in that order until the next resolver change. Each host has uniform probability.

Balancer "leastLoaded"

This balancer is not fully implemented yet. It currently behaves similarly to the "random" balancer.

Balancer "srv"

This balancer is only valid for the "srv" target scheme. It respects the SRV records’ priority and weight fields to do weighted, non-uniform random selection. As health checking has not yet been implemented, it will never use any priority tier except the one with lowest numeric value.

Balancer "weightedRandom"

This balancer picks a host at random, but for each host it uses the weight assigned to the host by the resolver in order to determine the probability of selecting that host. Only certain resolvers implement weights.

Balancer "weightedRoundRobin"

This balancer shuffles the hosts into a randomized K-length list, where each host occurs approximately N times such that (N/K) is roughly equal to the normalized value of the host’s resolver-assigned weight. Only certain resolvers implement weights.

TLS client configuration

Some sections, such as "global.etcd" and "frontends", optionally take a "tls" block to specify (1) that TLS should be used, and (2) how to configure it. It has the following structure:

...
"tls": {
  "skipVerify": bool,            # Whether or not to perform any validation whatsoever (Bool)
  "skipVerifyServerName": bool,  # Whether or not to check the ServerName against the server's SANs (Bool)
  "rootCA": "...",               # The path to the trusted root CAs as a concatenated PEM file (String); default is to use the system trusted roots
  "serverName": "...",           # The SNI / SAN DNSName / SAN IPAddress to verify (String); default is to guess
  "commonName": "...",           # The Subject CommonName which we will verify (String); this is rare, you should normally use ServerName
  "clientCert": "...",           # The path to the PEM file containing your client cert (String)
  "clientKey": "..."             # The path to the PEM file containing your client private key (String); default is the value of clientCert
},
...

All fields are optional and have reasonable defaults. The simplest configuration, in which TLS is used with all and only the standard verification steps, and with no mutual TLS (“mTLS”), is as follows:

...
"tls": {},
...