Kuma Star

Proxy Template

The proxy template provides configuration options for low-level Envoy resources that Kuma policies do not directly expose.

If you need features that aren’t available as a Kuma policy, open a new issue on GitHub so they can be added to the Kuma roadmap.

A ProxyTemplate policy can provide custom definitions of:

The custom definitions either complement or replace the resources that Kuma generates automatically.

Usage

Kuma uses the following default ProxyTemplate resource for every data plane proxy (kuma-dp) that is added to a Mesh. This resource looks like:

apiVersion: kuma.io/v1alpha1
kind: ProxyTemplate
mesh: default
metadata:
  name: custom-template-1
spec:
  selectors:
    - match:
        kuma.io/service: '*'
  conf:
    # `imports` allows us to reuse the dataplane configuration that Kuma
    # generates automatically and add more customizations on top of it
    imports:
      # `default-proxy` is a reference name for the default
      # data plane proxy configuration generated by Kuma
      - default-proxy

In these examples, note:

  • The selectors object specifies the data plane proxies that are targeted by the ProxyTemplate resource. Values are provided as Kuma tags.
  • The imports object specifies the reusable configuration that Kuma generates automatically. Kuma then extends the imports object with the custom configuration you specify. Possible values:
  • default-proxy - the default configuration for non-ingress data planes.
  • ingress-proxy - the default configuration for zone-ingress proxy.
  • gateway-proxy - the default configuration for mesh gateway.
  • egress-proxy - the default configuration for zone-egress proxy.

You can choose more than one import object.

Modifications

To customize the configuration of data plane proxies, you can combine modifications of any type in one ProxyTemplate. Each modification consists of the following sections:

  • operation - operation applied to the generated config (e.g. add, remove, patch).
  • match - some operations can be applied on matched resources (e.g. remove only resource of given name, patch all outbound resources).
  • value - raw Envoy xDS configuration. Can be partial if operation is patch.

Origin

All resources generated by Kuma are marked with the origin value, so you can match resources. Examples: add new filters but only on inbound listeners, set timeouts on outbound clusters.

Available origins:

  • inbound - resources generated for incoming traffic.
  • outbound - resources generated for outgoing traffic.
  • transparent - resources generated for transparent proxy functionality.
  • prometheus - resources generated when Prometheus metrics are enabled.
  • direct-access - resources generated for Direct Access functionality.
  • ingress - resources generated for Zone Ingress.
  • gateway - resources generated for MeshGateway

Cluster

Modifications that are applied on Clusters resources.

Available operations:

  • add - add a new cluster or replace existing if the name is the same.
  • remove - remove a cluster.
  • patch - patch a part of cluster definition.

Available matchers:

  • name - name of the cluster.
  • origin - origin of the cluster.
apiVersion: kuma.io/v1alpha1
kind: ProxyTemplate
mesh: default
metadata:
  name: custom-template-1
spec:
  selectors:
    - match:
        kuma.io/service: backend_default_svc_80
  conf:
    imports:
      - default-proxy
    modifications:
      - cluster:
          operation: add
          value: |
            name: test-cluster
            connectTimeout: 5s
            type: STATIC
      - cluster:
          operation: patch
          match: # optional: if absent, all clusters will be patched
            name: test-cluster # optional: if absent, all clusters regardless of name will be patched
            origin: inbound # optional: if absent, all clusters regardless of its origin will be patched
          value: | # you can specify only part of cluster definition that will be merged into existing cluster
            connectTimeout: 5s
      - cluster:
          operation: remove
          match: # optional: if absent, all clusters will be removed
            name: test-cluster # optional: if absent, all clusters regardless of name will be removed
            origin: inbound # optional: if absent, all clusters regardless of its origin will be removed

Listener

Modifications that are applied on Listeners resources.

Available operations:

  • add - add a new listener or replace existing if the name is the same.
  • remove - remove a listener.
  • patch - patch a part of listener definition.

Available matchers:

  • name - name of the listener.
  • origin - origin of the listener.
  • tags - tags of inbound or outbound listeners. They match Listener.metadata.filterMetadata[io.kuma.tags] in XDS configuration.
apiVersion: kuma.io/v1alpha1
kind: ProxyTemplate
mesh: default
metadata:
  name: custom-template-1
spec:
  selectors:
    - match:
        kuma.io/service: backend_default_svc_80
  conf:
    imports:
      - default-proxy
    modifications:
      - listener:
          operation: add
          value: |
            name: test-listener
            address:
              socketAddress:
                address: 192.168.0.1
                portValue: 8080
      - listener:
          operation: patch
          match: # optional: if absent, all listeners will be patched
            name: test-listener # optional: if absent, all listeners regardless of name will be patched
            origin: inbound # optional: if absent, all listeners regardless of its origin will be patched
            tags: # optional: if absent, all listeners are matched
              kuma.io/service: backend
          value: | # you can specify only part of listener definition that will be merged into existing listener
            continueOnListenerFiltersTimeout: true
      - listener:
          operation: remove
          match: # optional: if absent, all listeners will be removed
            name: test-listener # optional: if absent, all listeners regardless of name will be removed
            origin: inbound # optional: if absent, all listeners regardless of its origin will be removed

Network Filter

Modifications that are applied on Network Filters that are part of Listeners resource. Modifications are applied on all Filter Chains in the Listener.

Available operations:

  • addFirst - add a new filter as a first filter in Filter Chain.
  • addLast - add a new filter as a last filter in Filter Chain.
  • addAfter - add a new filter after other filter in Filter Chain that is matched using match section.
  • addBefore - add a new filter before other filter in Filter Chain that is matched using match section.
  • patch - patch a matched filter in Filter Chain.
  • remove - remove a filter in Filter Chain.

Available matchers:

  • name - name of the network filter.
  • listenerName - name of the listener.
  • listenerTags - tags of inbound or outbound listeners. They match Listener.metadata.filterMetadata[io.kuma.tags] in XDS configuration.
  • origin - origin of the listener.
apiVersion: kuma.io/v1alpha1
kind: ProxyTemplate
mesh: default
metadata:
  name: custom-template-1
spec:
  selectors:
    - match:
        kuma.io/service: backend_default_svc_80
  conf:
    imports:
      - default-proxy
    modifications:
      - networkFilter:
          operation: addFirst
          match: # optional: if absent, filter will be added to all listeners
            listenerName: inbound:127.0.0.0:80 # optional: if absent, filter will be added to all listeners regardless of name
            listenerTags: # optional: if absent, filter will be added to all listeners regardless of listener tags
              kuma.io/service: backend
            origin: inbound # optional: if absent, filter will be added to all listeners regardless of its origin
          value: |
            name: envoy.filters.network.local_ratelimit
            typedConfig:
              '@type': type.googleapis.com/envoy.extensions.filters.http.local_ratelimit.v3.LocalRateLimit
              statPrefix: rateLimit
              tokenBucket:
                fillInterval: 1s
      - networkFilter:
          operation: addLast
          match: # optional: if absent, filter will be added to all listeners
            listenerName: inbound:127.0.0.0:80 # optional: if absent, filter will be added to all listeners regardless of name
            listenerTags: # optional: if absent, filter will be added to all listeners regardless of listener tags
              kuma.io/service: backend
            origin: inbound # optional: if absent, filter will be added to all listeners regardless of its origin
          value: |
            name: envoy.filters.network.local_ratelimit
            typedConfig:
              '@type': type.googleapis.com/envoy.extensions.filters.http.local_ratelimit.v3.LocalRateLimit
              statPrefix: rateLimit
              tokenBucket:
                fillInterval: 1s
      - networkFilter:
          operation: addBefore
          match:
            name: envoy.filters.network.tcp_proxy # a new filter (Local RateLimit) will be added before existing (TcpProxy). If there is no TcpProxy filter, Local RateLimit won't be added.
            listenerName: inbound:127.0.0.0:80 # optional: if absent, filter will be added to all listeners regardless of name
            listenerTags: # optional: if absent, filter will be added to all listeners regardless of listener tags
              kuma.io/service: backend
            origin: inbound # optional: if absent, filter will be added to all listeners regardless of its origin
          value: |
            name: envoy.filters.network.local_ratelimit
            typedConfig:
              '@type': type.googleapis.com/envoy.extensions.filters.http.local_ratelimit.v3.LocalRateLimit
              statPrefix: rateLimit
              tokenBucket:
                fillInterval: 1s
      - networkFilter:
          operation: addAfter
          match:
            name: envoy.filters.network.tcp_proxy # a new filter (Local RateLimit) will be added after existing (TcpProxy). If there is no TcpProxy filter, Local RateLimit won't be added.
            listenerName: inbound:127.0.0.0:80 # optional: if absent, filter will be added to all listeners regardless of name
            listenerTags: # optional: if absent, filter will be added to all listeners regardless of listener tags
              kuma.io/service: backend
            origin: inbound # optional: if absent, filter will be added to all listeners regardless of its origin
          value: |
            name: envoy.filters.network.local_ratelimit
            typedConfig:
              '@type': type.googleapis.com/envoy.extensions.filters.http.local_ratelimit.v3.LocalRateLimit
              statPrefix: rateLimit
              tokenBucket:
                fillInterval: 1s
      - networkFilter:
          operation: patch
          match:
            name: envoy.filters.network.tcp_proxy 
            listenerName: inbound:127.0.0.0:80 # optional: if absent, filter will be patched within all listeners regardless of name
            listenerTags: # optional: if absent, filter will be patched within all listeners regardless of listener tags
              kuma.io/service: backend
            origin: inbound # optional: if absent, filter will be patched within all listeners regardless of its origin
          value: | # you can specify only part of filter definition that will be merged into existing filter
            name: envoy.filters.network.tcp_proxy
            typedConfig:
              '@type': type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy
              idleTimeout: 10s
      - networkFilter:
          operation: remove
          match: # optional: if absent, all filters from all listeners will be removed
            name: envoy.filters.network.tcp_proxy # optional: if absent, all filters regardless of name will be removed
            listenerName: inbound:127.0.0.0:80 # optional: if absent, all filters regardless of the listener name will be removed
            listenerTags: # optional: if absent, all filters regardless of the listener tags will be removed
              kuma.io/service: backend
            origin: inbound # optional: if absent, all filters regardless of its origin will be removed

Example how to change streamIdleTimeout for MeshGateway:

apiVersion: kuma.io/v1alpha1
kind: ProxyTemplate
mesh: default
metadata:
  name: custom-template-1
spec:
  selectors:
    - match:
        kuma.io/service: '*'
  conf:
    imports:
      - gateway-proxy # default configuration for MeshGateway
    modifications:
      - networkFilter:
          operation: patch
          match:
            name: envoy.filters.network.http_connection_manager
            origin: gateway # you can also specify the name of the listener
          value: |
            name: envoy.filters.network.http_connection_manager
            typedConfig:
              '@type': type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
              streamIdleTimeout: 15s

HTTP Filter

Modifications that are applied on HTTP Filters that are part of Listeners resource. Modifications that Kuma applies on all HTTP Connection Managers in the Listener.

HTTP Filter modifications can only be applied on services configured as HTTP.

Available operations:

  • addFirst - add a new filter as a first filter in HTTP Connection Manager.
  • addLast - add a new filter as a last filter in HTTP Connection Manager.
  • addAfter - add a new filter after other filter in HTTP Connection Manager that is matched using match section.
  • addBefore - add a new filter before other filter in HTTP Connection Manager that is matched using match section.
  • patch - patch a matched filter in HTTP Connection Manager.
  • remove - remove a filter in HTTP Connection Manager.

Available matchers:

  • name - name of the network filter
  • listenerName - name of the listener
  • listenerTags - tags of inbound or outbound listeners. They match Listener.metadata.filterMetadata[io.kuma.tags] in XDS configuration.
  • origin - origin of the listener
apiVersion: kuma.io/v1alpha1
kind: ProxyTemplate
mesh: default
metadata:
  name: custom-template-1
spec:
  selectors:
    - match:
        kuma.io/service: backend_default_svc_80
  conf:
    imports:
      - default-proxy
    modifications:
      - httpFilter:
          operation: addFirst
          match: # optional: if absent, filter will be added to all HTTP Connection Managers
            listenerName: inbound:127.0.0.0:80 # optional: if absent, filter will be added to all listeners regardless of name
            listenerTags: # optional: if absent, filter will be added to all listeners regardless of listener tags
              kuma.io/service: backend
            origin: inbound # optional: if absent, filter will be added to all listeners regardless of its origin
          value: |
            name: envoy.filters.http.gzip
            typedConfig:
              '@type': type.googleapis.com/envoy.extensions.filters.http.gzip.v3.Gzip
              memoryLevel: 9
      - httpFilter:
          operation: addLast
          match: # optional: if absent, filter will be added to all HTTP Connection Managers
            listenerName: inbound:127.0.0.0:80 # optional: if absent, filter will be added to all listeners regardless of name
            listenerTags: # optional: if absent, filter will be added to all listeners regardless of listener tags
              kuma.io/service: backend
            origin: inbound # optional: if absent, filter will be added to all listeners regardless of its origin
          value: |
            name: envoy.filters.http.gzip
            typedConfig:
              '@type': type.googleapis.com/envoy.extensions.filters.http.gzip.v3.Gzip
              memoryLevel: 9
      - httpFilter:
          operation: addBefore
          match:
            name: envoy.filters.http.router # a new filter (Gzip) will be added before existing (Router). If there is no Router filter, Gzip won't be added.
            listenerName: inbound:127.0.0.0:80 # optional: if absent, filter will be added to all listeners regardless of name
            listenerTags: # optional: if absent, filter will be added to all listeners regardless of listener tags
              kuma.io/service: backend
            origin: inbound # optional: if absent, filter will be added to all listeners regardless of its origin
          value: |
            name: envoy.filters.http.gzip
            typedConfig:
              '@type': type.googleapis.com/envoy.extensions.filters.http.gzip.v3.Gzip
              memoryLevel: 9
      - httpFilter:
          operation: addAfter
          match:
            name: envoy.filters.http.router # a new filter (Gzip) will be added after existing (Router). If there is no Router filter, Gzip won't be added.
            listenerName: inbound:127.0.0.0:80 # optional: if absent, filter will be added to all listeners regardless of name
            listenerTags: # optional: if absent, filter will be added to all listeners regardless of listener tags
              kuma.io/service: backend
            origin: inbound # optional: if absent, filter will be added to all listeners regardless of its origin
          value: |
            name: envoy.filters.http.gzip
            typedConfig:
              '@type': type.googleapis.com/envoy.extensions.filters.http.gzip.v3.Gzip
              memoryLevel: 9
      - httpFilter:
          operation: patch
          match:
            name: envoy.filters.http.router 
            listenerName: inbound:127.0.0.0:80 # optional: if absent, filter will be patched within all listeners regardless of name
            listenerTags: # optional: if absent, filter will be patched within all listeners regardless of listener tags
              kuma.io/service: backend
            origin: inbound # optional: if absent, filter will be patched within all listeners regardless of its origin
          value: | # you can specify only part of filter definition that will be merged into existing filter
            name: envoy.filters.http.router 
            typedConfig:
              '@type': type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
              dynamicStats: false
      - httpFilter:
          operation: remove
          match: # optional: if absent, all filters from all listeners will be removed
            name: envoy.filters.http.gzip # optional: if absent, all filters regardless of name will be removed
            listenerName: inbound:127.0.0.0:80 # optional: if absent, all filters regardless of the listener name will be removed
            listenerTags: # optional: if absent, all filters regardless of the listener tags will be removed
              kuma.io/service: backend
            origin: inbound # optional: if absent, all filters regardless of its origin will be removed

VirtualHost

Modifications that are applied on VirtualHost resources.

VirtualHost modifications can only be applied on services configured as HTTP.

Available operations:

  • add - add a new VirtualHost.
  • remove - remove a VirtualHost.
  • patch - patch a part of VirtualHost definition.

Available matchers:

  • name - name of the VirtualHost.
  • origin - origin of the VirtualHost.
  • routeConfigurationName - name of the RouteConfiguration.
apiVersion: kuma.io/v1alpha1
kind: ProxyTemplate
mesh: default
metadata:
  name: custom-template-1
spec:
  selectors:
    - match:
        kuma.io/service: backend_default_svc_80
  conf:
    imports:
      - default-proxy
    modifications:
      - virtualHost:
          operation: add
          value: |
            name: backend
            domains:
            - "*"
            routes:
            - match:
                prefix: /
              route:
                cluster: backend
      - virtualHost:
          operation: patch
          match: # optional: if absent, all listeners will be patched
            name: backend # optional: if absent, all virtual hosts regardless of name will be patched
            origin: inbound # optional: if absent, all virtual hosts regardless of its origin will be patched
            routeConfigurationName: outbound:backend # optional: if absent, all virtual hosts in all route configurations will be patched
          value: | # you can specify only part of virtual host definition that will be merged into existing virtual host
            retryPolicy:
              retryOn: 5xx
              numRetries: 3
      - virtualHost:
          operation: remove
          match: # optional: if absent, all virtual hosts will be removed
            name: test-listener # optional: if absent, all virtual hsots regardless of name will be removed
            origin: inbound # optional: if absent, all virtual hosts regardless of its origin will be removed

How Kuma handles the proxy template

At runtime, whenever kuma-cp generates the configuration for a given data plane proxies, it will proceed as follows:

  1. Kuma searches for all the ProxyTemplates resources that have been defined in the specified Mesh.
  2. It loads in memory the ProxyTemplates resources whose selectors match either an inbound or a gateway definition of any data plane proxies accordingly to the Kuma Tags selected.
  3. Every matching ProxyTemplate is ranked. The ProxyTemplate resource with the highest ranking is used to generate the configuration for the specified data plane proxy (or proxies).
  4. If the ProxyTemplate resource specifies an imports object, these resources are generated first.
  5. If a ProxyTemplate defines a modification object, all modifications are applied, one by one in the order defined in modification section.

Lua filter example

For a more complete example, explore this Lua filter that adds the new x-header: test header to all outgoing HTTP requests to service offers.

apiVersion: kuma.io/v1alpha1
kind: ProxyTemplate
mesh: default
metadata:
  name: backend-lua-filter
spec:
  selectors:
    - match:
        kuma.io/service: backend_default_svc_80
  conf:
    imports:
      - default-proxy # apply modifications on top of resources generated by Kuma
    modifications:
      - httpFilter:
          operation: addBefore
          match:
            name: envoy.filters.http.router
            origin: outbound
            listenerTags:
              kuma.io/service: offers
          value: |
            name: envoy.filters.http.lua
            typedConfig:
              '@type': type.googleapis.com/envoy.extensions.filters.http.lua.v3.Lua
              inline_code: |
                function envoy_on_request(request_handle)
                  request_handle:headers():add("x-header", "test")
                end

Matching

ProxyTemplate is a Dataplane policy. You can use all the tags in the selectors section.

Builtin Gateway support

The Proxy Template policy supports a new gateway-proxy configuration name that can be imported. This generates the Envoy resources for a Kuma Gateway proxy. The origin name for matching template modifications is gateway.

Schema