Major Changes in Emissary-ingress 2.X

The 2.X family introduces a number of changes to allow Emissary-ingress to more gracefully handle larger installations, reduce global configuration to better handle multitenant or multiorganizational installations, reduce memory footprint, and improve performance. We welcome feedback!! Join us on Slack and let us know what you think.

While Emissary-ingress 2 is functionally compatible with Emissary-ingress 1.14, note that this is a major version change and there are important differences between Emissary-ingress 1.X and Emissary-ingress 3.9.1. For details, read on.

1. Configuration API Version getambassador.io/v3alpha1

Emissary-ingress 2.0 introduced API version getambassador.io/v3alpha1 to allow certain changes in configuration resources that are not backwards compatible with Emissary-ingress 1.X. The most notable example of change is the addition of the mandatory Listener resource; however, there are important changes in Host and Mapping as well.

API version getambassador.io/v3alpha1 replaces x.getambassador.io/v3alpha1 from the 2.0 developer previews. getambassador.io/v3alpha1 may still change as we receive feedback.

2. Kubernetes 1.22 and Structural CRDs

Kubernetes 1.22 requires structural CRDs. This change is primarily meant to support better CRD validation, but it also has the effect that union types are no longer allowed in CRDs: for example, an element that can be either a string or a list of strings is not allowed. Several such elements appeared in the getambassador.io/v2 CRDs, requiring changes. In getambassador.io/v3alpha1:

• ambassador_id must always be a list of strings
• Host.mappingSelector supersedes Host.selector, and controls association between Hosts and Mappings
• Mapping.hostname supersedes Mapping.host and Mapping.host_regex
• Mapping.tls can only be a string
• Mapping.labels always requires maps instead of strings

2. Listeners, Hosts, and Mappings

Emissary-ingress 2.0 introduced the new mandatory Listener CRD, and made some changes to the Host and Mapping resources.

The Listener CRD

The new Listener CRD defines where and how Emissary-ingress should listen for requests from the network, and which Host definitions should be used to process those requests.

Note that Listeners are never created by Emissary-ingress, and must be defined by the user. If you do not define any Listeners, Emissary-ingress will not listen anywhere for connections, and therefore won't do anything useful. It will log a WARNING to this effect.

A Listener specifically defines

• port: a port number on which to listen for new requests;
• protocol and securityModel: the protocol stack and security model to use (e.g. HTTPS using the X-Forwarded-Proto header); and
• hostBinding: how to tell if a given Host should be associated with this Listener:
  • a Listener can choose to consider all Hosts, or only Hosts in the same namespace as the Listener, or
  • a Listener can choose to consider only Hosts with a particular Kubernetes label.

Note that the hostBinding is mandatory. A Listener must specify how to identify the Hosts to associate with the Listener', or the Listener will be rejected. This is intended to help prevent cases where a Listener mistakenly grabs too many Hosts: if you truly need a Listener that associates with all Hosts, the easiest way is to tell the Listener to look for Hosts in all namespaces, with no further selectors, for example:

A Listener that has no associated Hosts will be logged as a WARNING, and will not be included in the Envoy configuration generated by Emissary-ingress.

Note also that there is no limit on how many Listeners may be created, and as such no limit on the number of ports to which a Host may be associated.

Wildcard Hosts No Longer Created

In Emissary-ingress 1.X, Emissary-ingress would make sure that a wildcard Host, with a hostname of "*", was always present. Emissary-ingress 2.X does not force a wildcard Host: if you need the wildcard behavior, you will need to create a Host with a hostname of "*".

Of particular note is that Emissary-ingress will not respond to queries to an IP address unless a wildcard Host is present. If foo.example.com resolves to 10.11.12.13, and the only Host has a hostname of foo.example.com, then:

• requests to http://foo.example.com/ will work, but
• requests to http://10.11.12.13/ will not work.

Adding a Host with a hostname of "*" will allow the second query to work.

Host and Mapping Association

The Host CRD continues to define information about hostnames, TLS certificates, and how to handle requests that are "secure" (using HTTPS) or "insecure" (using HTTP). The Mapping CRD continues to define how to map the URL space to upstream services.

However, as of Emissary-ingress 2.0, a Mapping will not be associated with a Host unless at least one of the following is true:

• The Mapping specifies a hostname attribute that matches the Host in question.

  • Note that a getambassador.io/v2 Mapping has host and host_regex, rather than hostname.
    • A getambassador.io/v3alpha1 Mapping will honor host and host_regex as a transition aid, but host and host_regex are deprecated in favor of hostname.
    • A Mapping that specifies host_regex: true will be associated with all Hosts. This is generally far less desirable than using hostname with a DNS glob.

• The Host specifies a mappingSelector that matches the Mapping's Kubernetes labels.

  • Note that a getambassador.io/v2 Host has a selector, rather than a mappingSelector.
    • A getambassador.io/v3alpha1 Host ignores selector and, instead, looks only at mappingSelector.
    • Where a selector got a default value if not specified, mappingSelector must be explicitly stated.

Without either a hostname match or a label match, the Mapping will not be associated with the Host in question. This is intended to help manage memory consumption with large numbers of Hosts and large numbers of Mappings.

Independent Host Actions

Each Host can specify its requestPolicy.insecure.action independently of any other Host, allowing for HTTP routing as flexible as HTTPS routing.

Host, TLSContext, and TLS Termination

As of Emissary-ingress 2.0, Hosts are required for TLS termination. It is no longer sufficient to create a TLSContext by itself; the Host is required.

The minimal setup for TLS termination is therefore a Kubernetes Secret of type kubernetes.io/tls, and a Host that uses it:

It is not necessary to explicitly state a TLSContext in the Host: setting tlsSecret is enough. Of course, TLSContext is still the ideal way to share TLS configuration between more than one Host. For further examples, see Configuring Emissary-ingress Communications.

Mappings, TCPMappings, and TLS Origination

A getambassador.io/v2 Mapping or TCPMapping could specify tls: true to indicate TLS origination without supplying a certificate. This is not supported in getambassador.io/v3alpha1: instead, use an https:// prefix on the service. In the Mapping, this is straightforward, but there are more details for the TCPMapping when using TLS.

Mappings and labels

The Mapping CRD includes a labels field, used with rate limiting. The syntax of the labels has changed for compatibility with Kubernetes 1.22.

3. Other Changes

Envoy V3 API by Default

By default, Emissary-ingress 2.X will configure Envoy using the V3 Envoy API. In Emissary-ingress 3.9.1, you may switch back to Envoy V2 by setting the AMBASSADOR_ENVOY_API_VERSION environment variable to "V2"; in Emissary-ingress 2.2.0, support for the Envoy V2 API (and the AMBASSADOR_ENVOY_API_VERSION environment variable) will be removed.

More Performant Reconfiguration by Default

In Emissary-ingress 1.X, the environment variable AMBASSADOR_FAST_RECONFIGURE could be used to enable a higher performance implementation of the code Emissary-ingress uses to validate and generate Envoy configuration. In Emissary-ingress 2.X, this higher-performance mode is always enabled.

Changes to the ambassador Module, and the tls Module

It is no longer possible to configure TLS using the tls element of the ambassador Module or using the tls Module. Both of these cases are correctly covered by the TLSContext resource.

With the introduction of the Listener resource, a few settings have moved from the Module to the Listener.

Configuration for the PROXY protocol is part of the Listener resource in Emissary-ingress 2.X, so the use_proxy_protocol element of the ambassador Module is no longer supported. Note that the Listener resource can configure PROXY resource per-Listener, rather than having a single global setting. For further information, see the Listener documentation.

xff_num_trusted_hops has been removed from the Module, and its functionality has been moved to the l7Depth setting in the Listener resource.

TLSContext redirect_cleartext_from and Host insecure.additionalPort

redirect_cleartext_from has been removed from the TLSContext resource; insecure.additionalPort has been removed from the Host CRD. Both of these cases are covered by adding additional Listeners. For further examples, see Configuring Emissary-ingress Communications.

Service Preview No Longer Supported

Service Preview is no longer supported as of Emissary-ingress 2.X, as its use cases are supported by Telepresence.

Edge Policy Console No Longer Supported

The Edge Policy Console has been removed as of Emissary-ingress 2.X, in favor of Ambassador Cloud.

Project CRD No Longer Supported

The Project CRD has been removed as of Emissary-ingress 2.X, in favor of Argo.