Configuring Emissary-ingress Communications

For Emissary-ingress to do its job of managing network communications for your services, it first needs to know how its own communications should be set up. This is handled by a combination of resources: the Listener, the Host, and the TLSContext.

• Listener: defines where, and how, Emissary-ingress should listen for requests from the network.
• Host: defines which hostnames Emissary-ingress should care about, and how to handle different kinds of requests for those hosts. Hosts can be associated with one or more Listeners.
• TLSContext: defines whether, and how, Emissary-ingress will manage TLS certificates and options. TLSContexts can be associated with one or more Hosts.

Once the basic communications setup is in place, Emissary-ingress Mappings and TCPMappings can be associated with Hosts to actually do routing.

A Note on TLS

TLS can appear intractable if you haven't set up certificates correctly. If you're having trouble with TLS, always check the logs of your Emissary-ingress Pods and look for certificate errors.

Examples / Cookbook

Basic HTTP and HTTPS

A useful configuration is to support either HTTP or HTTPS, in this case on either port 8080 or port 8443. This tends to make it as easy as possible to communicate with the services behind the Emissary-ingress instance. It uses two Listeners and at least one Host.

Listeners:

• Both Listeners use protocol: HTTPS to allow Envoy to inspect incoming connections, determine whether or not TLS is in play, and set X-Forwarded-Proto appropriately. The securityModel then specifies that X-Forwarded-Proto will determine whether requests will be considered secure or insecure.

• The hostBinding shown here will allow any Host in the same namespace as the Listeners to be associated with both Listeners; in turn, that will allow access to that Host's Mappings from either port. For greater control, use a selector instead.

• Note that the Listeners do not specify anything about TLS certificates. The Host handles that; see below.

Host

This example will assume that we expect to be reachable as foo.example.com, and that the foo.example.com certificate is stored in the Kubernetes Secret named foo-secret:

• The tlsSecret specifies the certificate in use for TLS termination.
• The requestPolicy specifies routing HTTPS and redirecting HTTP to HTTPS.
• Since the Host does not specify a selector, only Mappings with a hostname that matches foo.example.com will be associated with this Host.
• Note well that simply defining a TLSContext is not sufficient to terminate TLS: you must define either a Host or an Ingress.
• Note that if no Host is present, but a TLS secret named fallback-secret is available, the system will currently define a Host using fallback-secret. This behavior is subject to change.

HTTP-Only

Another straightforward configuration is to support only HTTP, in this case on port 8080. This uses a single Listener and a single Host:

• Here, we listen only on port 8080, and only for HTTP. HTTPS will be rejected.
• Since requests are only allowed using HTTP, we declare all requests INSECURE by definition.
• The Host specifies routing HTTP, rather than redirecting it.

TLS using ACME (Ambassador Edge Stack only)

This scenario uses ACME to get certificates for foo.example.com and bar.example.com. HTTPS traffic to either host is routed; HTTP traffic to foo.example.com will be redirected to HTTPS, but HTTP traffic to bar.example.com will be rejected outright.

Since this example uses ACME, it is only supported in Ambassador Edge Stack.

For demonstration purposes, we show this example listening for HTTPS on port 9999, using X-Forwarded-Proto.

(Mappings are not shown.)

• Our Listeners will accept HTTPS and HTTP on port 9999, and the protocol used will dictate whether the requests are secure (HTTPS) or insecure (HTTP).
• foo-host defaults to ACME with Let's Encrypt, since acmeProvider.authority is not provided.
• foo-host defaults to redirecting insecure requests, since the default for requestPolicy.insecure.action is Redirect.
• bar-host uses Let's Encrypt as well, but it will reject insecure requests.

If you use ACME for multiple Hosts, add a wildcard Host too. This is required to manage a known issue. This issue will be resolved in a future Ambassador Edge Stack release.

Multiple TLS Certificates

This scenario uses TLS without ACME. Each of our two Hosts uses a distinct TLS certificate. HTTPS traffic to eitherfoo.example.com or bar.example.com is routed, but this time foo.example.com will redirect HTTP requests, while bar.example.com will route them.

Since this example does not use ACME, it is supported in Emissary-ingress as well as Ambassador Edge Stack.

For demonstration purposes, we show this example listening for HTTPS on port 4848, using X-Forwarded-Proto.

• foo-host and bar-host simply reference the tlsSecret to use for termination.
  • If the secret involved contains a wildcard cert, or a cert with multiple SAN, both Hosts could reference the same tlsSecret.
• foo-host relies on the default insecure routing action of Redirect.
• bar-host must explicitly specify routing HTTP.

Using a TLSContext

If you need to share other TLS settings between two Hosts, you can reference a TLSContext as well as the tlsSecret. This is the same as the previous example, but we use a TLSContext to set ALPN information, and we assume that the Secret contains a wildcard cert.

• Note that specifying the tlsSecret is still necessary, even when tlsContext is specified.

ACME With a TLSContext (Ambassador Edge Stack Only)

In Ambassador Edge Stack, you can use a TLSContext with ACME as well. This example is the same as "TLS using ACME", but we use a TLSContext to set ALPN information. Again, ACME is only supported in Ambassador Edge Stack.

• Note that we don't provide the Secret: the ACME client will create it for us.

Using an L7 Load Balancer to Terminate TLS

In this scenario, a layer 7 load balancer ahead of Emissary-ingress will terminate TLS, so Emissary-ingress will always see HTTP with a known good X-Forwarded-Protocol. We'll use that to route HTTPS and redirect HTTP.

• We set l7Depth to 1 to indicate that there's a single trusted L7 load balancer ahead of us.
• We specifically set this Listener to HTTP-only, but we stick with port 8443 just because we expect people setting up TLS at all to expect to use port 8443. (There's nothing special about the port number, pick whatever you like.)
• Our Host does not specify a tlsSecret, so Emissary-ingress will not try to terminate TLS.
• Since the Listener still pays attention to X-Forwarded-Proto, both secure and insecure requests are possible, and we use the Host to route HTTPS and redirect HTTP.

Using a Split L4 Load Balancer to Terminate TLS

Here, we assume that Emissary-ingress is behind a load balancer setup that handles TLS at layer 4:

• Incoming cleartext traffic is forwarded to Emissary-ingress on port 8080.
• Incoming TLS traffic is terminated at the load balancer, then forwarded to Emissary-ingress as cleartext on port 8443.
• This might involve multiple L4 load balancers, but the actual number doesn't matter.
• The actual port numbers we use don't matter either, as long as Emissary-ingress and the load balancer(s) agree on which port is for which traffic.

We're going to route HTTPS for both foo.example.com and bar.example.com, redirect HTTP for foo.example.com, and reject HTTP for bar.example.com.

• Since L4 load balancers cannot set X-Forwarded-Protocol, we don't use it at all here: instead, we dictate that 8080 and 8443 both speak cleartext HTTP, but everything arriving at port 8080 is insecure and everything at port 8443 is secure.

Listening on Multiple Ports

There's no reason you need to use ports 8080 and 8443, or that you're limited to two ports. Here we'll use ports 9001 and 9002 for HTTP, and port 4001 for HTTPS. We'll route traffic irrespective of protocol.

• We can use X-Forwarded-Proto for all our Listeners: the HTTP-only Listeners will set it correctly.
• Each Listener can specify only one port, but there's no hardcoded limit on the number of Listeners you can have.

Using Labels to Associate Hosts and Listeners

In the examples above, the Listeners all associate with any Host in their namespace. In this example, we will use Kubernetes labels to control the association instead.

Here, we'll listen for HTTP to foo.example.com on port 8888, and for either HTTP or HTTPS to bar.example.com on port 9999 (where we'll redirect HTTP to HTTPS). Traffic to baz.example.com will work on both ports, and we'll route HTTP for it rather than redirecting.

• Note the labels on each Host, which the hostBinding on the Listener can reference.
  • Note also that only label selectors are supported at the moment.

Wildcard Hosts and Mappings

In a Mapping, the host is now treated as a glob rather than an exact match, with the goal of vastly reducing the need for host_regex. (The hostname in a Host has always been treated as a glob).

• Note that only prefix and suffix matches are supported, so *.example.com and foo.* are both fine, but foo.*.com will not work -- you'll need to use host_regex if you really need that. (This is an Envoy limitation.)

In this example, we'll accept both HTTP and HTTPS, but:

• Cleartext traffic to any host in lowsec.example.com will be routed.
• Cleartext traffic to any host in normal.example.com will be redirected.
• Any other cleartext traffic will be rejected.

• We'll listen for HTTP or HTTPS on port 8443.
• The three Hosts apply different insecure routing actions depending on the hostname.
• You could also do this with host_regex, but using host with globs will give better performance.
  • Being able to not associate a given Mapping with a given Host when the Mapping's host doesn't match helps a lot when you have many Hosts.
  • Reliably determining if a regex (for the Mapping) matches a glob (for the Host) isn't really possible, so we can't prune host_regex Mappings at all.