1 - Client Authentication (v1)

Resource Types

ExecCredential

ExecCredential is used by exec-based plugins to communicate credentials to HTTP transports.

FieldDescription
apiVersion
string
client.authentication.k8s.io/v1
kind
string
ExecCredential
spec [Required]
ExecCredentialSpec
Spec holds information passed to the plugin by the transport.
status
ExecCredentialStatus
Status is filled in by the plugin and holds the credentials that the transport should use to contact the API.

Cluster

Appears in:

Cluster contains information to allow an exec plugin to communicate with the kubernetes cluster being authenticated to.

To ensure that this struct contains everything someone would need to communicate with a kubernetes cluster (just like they would via a kubeconfig), the fields should shadow "k8s.io/client-go/tools/clientcmd/api/v1".Cluster, with the exception of CertificateAuthority, since CA data will always be passed to the plugin as bytes.

FieldDescription
server [Required]
string
Server is the address of the kubernetes cluster (https://hostname:port).
tls-server-name
string
TLSServerName is passed to the server for SNI and is used in the client to check server certificates against. If ServerName is empty, the hostname used to contact the server is used.
insecure-skip-tls-verify
bool
InsecureSkipTLSVerify skips the validity check for the server's certificate. This will make your HTTPS connections insecure.
certificate-authority-data
[]byte
CAData contains PEM-encoded certificate authority certificates. If empty, system roots should be used.
proxy-url
string
ProxyURL is the URL to the proxy to be used for all requests to this cluster.
config
k8s.io/apimachinery/pkg/runtime.RawExtension
Config holds additional config data that is specific to the exec plugin with regards to the cluster being authenticated to.

This data is sourced from the clientcmd Cluster object's extensions[client.authentication.k8s.io/exec] field:

clusters:

  • name: my-cluster cluster: ... extensions:
    • name: client.authentication.k8s.io/exec # reserved extension name for per cluster exec config extension: audience: 06e3fbd18de8 # arbitrary config

In some environments, the user config may be exactly the same across many clusters (i.e. call this exec plugin) minus some details that are specific to each cluster such as the audience. This field allows the per cluster config to be directly specified with the cluster info. Using this field to store secret data is not recommended as one of the prime benefits of exec plugins is that no secrets need to be stored directly in the kubeconfig.

ExecCredentialSpec

Appears in:

ExecCredentialSpec holds request and runtime specific information provided by the transport.

FieldDescription
cluster
Cluster
Cluster contains information to allow an exec plugin to communicate with the kubernetes cluster being authenticated to. Note that Cluster is non-nil only when provideClusterInfo is set to true in the exec provider config (i.e., ExecConfig.ProvideClusterInfo).
interactive [Required]
bool
Interactive declares whether stdin has been passed to this exec plugin.

ExecCredentialStatus

Appears in:

ExecCredentialStatus holds credentials for the transport to use.

Token and ClientKeyData are sensitive fields. This data should only be transmitted in-memory between client and exec plugin process. Exec plugin itself should at least be protected via file permissions.

FieldDescription
expirationTimestamp
meta/v1.Time
ExpirationTimestamp indicates a time when the provided credentials expire.
token [Required]
string
Token is a bearer token used by the client for request authentication.
clientCertificateData [Required]
string
PEM-encoded client TLS certificates (including intermediates, if any).
clientKeyData [Required]
string
PEM-encoded private key for the above certificate.

2 - Client Authentication (v1beta1)

Resource Types

ExecCredential

ExecCredential is used by exec-based plugins to communicate credentials to HTTP transports.

FieldDescription
apiVersion
string
client.authentication.k8s.io/v1beta1
kind
string
ExecCredential
spec [Required]
ExecCredentialSpec
Spec holds information passed to the plugin by the transport.
status
ExecCredentialStatus
Status is filled in by the plugin and holds the credentials that the transport should use to contact the API.

Cluster

Appears in:

Cluster contains information to allow an exec plugin to communicate with the kubernetes cluster being authenticated to.

To ensure that this struct contains everything someone would need to communicate with a kubernetes cluster (just like they would via a kubeconfig), the fields should shadow "k8s.io/client-go/tools/clientcmd/api/v1".Cluster, with the exception of CertificateAuthority, since CA data will always be passed to the plugin as bytes.

FieldDescription
server [Required]
string
Server is the address of the kubernetes cluster (https://hostname:port).
tls-server-name
string
TLSServerName is passed to the server for SNI and is used in the client to check server certificates against. If ServerName is empty, the hostname used to contact the server is used.
insecure-skip-tls-verify
bool
InsecureSkipTLSVerify skips the validity check for the server's certificate. This will make your HTTPS connections insecure.
certificate-authority-data
[]byte
CAData contains PEM-encoded certificate authority certificates. If empty, system roots should be used.
proxy-url
string
ProxyURL is the URL to the proxy to be used for all requests to this cluster.
config
k8s.io/apimachinery/pkg/runtime.RawExtension
Config holds additional config data that is specific to the exec plugin with regards to the cluster being authenticated to.

This data is sourced from the clientcmd Cluster object's extensions[client.authentication.k8s.io/exec] field:

clusters:

  • name: my-cluster cluster: ... extensions:
    • name: client.authentication.k8s.io/exec # reserved extension name for per cluster exec config extension: audience: 06e3fbd18de8 # arbitrary config

In some environments, the user config may be exactly the same across many clusters (i.e. call this exec plugin) minus some details that are specific to each cluster such as the audience. This field allows the per cluster config to be directly specified with the cluster info. Using this field to store secret data is not recommended as one of the prime benefits of exec plugins is that no secrets need to be stored directly in the kubeconfig.

ExecCredentialSpec

Appears in:

ExecCredentialSpec holds request and runtime specific information provided by the transport.

FieldDescription
cluster
Cluster
Cluster contains information to allow an exec plugin to communicate with the kubernetes cluster being authenticated to. Note that Cluster is non-nil only when provideClusterInfo is set to true in the exec provider config (i.e., ExecConfig.ProvideClusterInfo).
interactive [Required]
bool
Interactive declares whether stdin has been passed to this exec plugin.

ExecCredentialStatus

Appears in:

ExecCredentialStatus holds credentials for the transport to use.

Token and ClientKeyData are sensitive fields. This data should only be transmitted in-memory between client and exec plugin process. Exec plugin itself should at least be protected via file permissions.

FieldDescription
expirationTimestamp
meta/v1.Time
ExpirationTimestamp indicates a time when the provided credentials expire.
token [Required]
string
Token is a bearer token used by the client for request authentication.
clientCertificateData [Required]
string
PEM-encoded client TLS certificates (including intermediates, if any).
clientKeyData [Required]
string
PEM-encoded private key for the above certificate.

3 - kube-apiserver Audit Configuration (v1)

Resource Types

Event

Appears in:

Event captures all the information that can be included in an API audit log.

FieldDescription
apiVersion
string
audit.k8s.io/v1
kind
string
Event
level [Required]
Level
AuditLevel at which event was generated
auditID [Required]
k8s.io/apimachinery/pkg/types.UID
Unique audit ID, generated for each request.
stage [Required]
Stage
Stage of the request handling when this event instance was generated.
requestURI [Required]
string
RequestURI is the request URI as sent by the client to a server.
verb [Required]
string
Verb is the kubernetes verb associated with the request. For non-resource requests, this is the lower-cased HTTP method.
user [Required]
authentication/v1.UserInfo
Authenticated user information.
impersonatedUser
authentication/v1.UserInfo
Impersonated user information.
sourceIPs
[]string
Source IPs, from where the request originated and intermediate proxies.
userAgent
string
UserAgent records the user agent string reported by the client. Note that the UserAgent is provided by the client, and must not be trusted.
objectRef
ObjectReference
Object reference this request is targeted at. Does not apply for List-type requests, or non-resource requests.
responseStatus
meta/v1.Status
The response status, populated even when the ResponseObject is not a Status type. For successful responses, this will only include the Code and StatusSuccess. For non-status type error responses, this will be auto-populated with the error Message.
requestObject
k8s.io/apimachinery/pkg/runtime.Unknown
API object from the request, in JSON format. The RequestObject is recorded as-is in the request (possibly re-encoded as JSON), prior to version conversion, defaulting, admission or merging. It is an external versioned object type, and may not be a valid object on its own. Omitted for non-resource requests. Only logged at Request Level and higher.
responseObject
k8s.io/apimachinery/pkg/runtime.Unknown
API object returned in the response, in JSON. The ResponseObject is recorded after conversion to the external type, and serialized as JSON. Omitted for non-resource requests. Only logged at Response Level.
requestReceivedTimestamp
meta/v1.MicroTime
Time the request reached the apiserver.
stageTimestamp
meta/v1.MicroTime
Time the request reached current audit stage.
annotations
map[string]string
Annotations is an unstructured key value map stored with an audit event that may be set by plugins invoked in the request serving chain, including authentication, authorization and admission plugins. Note that these annotations are for the audit event, and do not correspond to the metadata.annotations of the submitted object. Keys should uniquely identify the informing component to avoid name collisions (e.g. podsecuritypolicy.admission.k8s.io/policy). Values should be short. Annotations are included in the Metadata level.

EventList

EventList is a list of audit Events.

FieldDescription
apiVersion
string
audit.k8s.io/v1
kind
string
EventList
metadata
meta/v1.ListMeta
No description provided.
items [Required]
[]Event
No description provided.

Policy

Appears in:

Policy defines the configuration of audit logging, and the rules for how different request categories are logged.

FieldDescription
apiVersion
string
audit.k8s.io/v1
kind
string
Policy
metadata
meta/v1.ObjectMeta
ObjectMeta is included for interoperability with API infrastructure.Refer to the Kubernetes API documentation for the fields of the metadata field.
rules [Required]
[]PolicyRule
Rules specify the audit Level a request should be recorded at. A request may match multiple rules, in which case the FIRST matching rule is used. The default audit level is None, but can be overridden by a catch-all rule at the end of the list. PolicyRules are strictly ordered.
omitStages
[]Stage
OmitStages is a list of stages for which no events are created. Note that this can also be specified per rule in which case the union of both are omitted.
omitManagedFields
bool
OmitManagedFields indicates whether to omit the managed fields of the request and response bodies from being written to the API audit log. This is used as a global default - a value of 'true' will omit the managed fileds, otherwise the managed fields will be included in the API audit log. Note that this can also be specified per rule in which case the value specified in a rule will override the global default.

PolicyList

PolicyList is a list of audit Policies.

FieldDescription
apiVersion
string
audit.k8s.io/v1
kind
string
PolicyList
metadata
meta/v1.ListMeta
No description provided.
items [Required]
[]Policy
No description provided.

GroupResources

Appears in:

GroupResources represents resource kinds in an API group.

FieldDescription
group
string
Group is the name of the API group that contains the resources. The empty string represents the core API group.
resources
[]string
Resources is a list of resources this rule applies to.

For example: 'pods' matches pods. 'pods/log' matches the log subresource of pods. '∗' matches all resources and their subresources. 'pods/∗' matches all subresources of pods. '∗/scale' matches all scale subresources.

If wildcard is present, the validation rule will ensure resources do not overlap with each other.

An empty list implies all resources and subresources in this API groups apply.

resourceNames
[]string
ResourceNames is a list of resource instance names that the policy matches. Using this field requires Resources to be specified. An empty list implies that every instance of the resource is matched.

Level

(Alias of string)

Appears in:

Level defines the amount of information logged during auditing

ObjectReference

Appears in:

ObjectReference contains enough information to let you inspect or modify the referred object.

FieldDescription
resource
string
No description provided.
namespace
string
No description provided.
name
string
No description provided.
uid
k8s.io/apimachinery/pkg/types.UID
No description provided.
apiGroup
string
APIGroup is the name of the API group that contains the referred object. The empty string represents the core API group.
apiVersion
string
APIVersion is the version of the API group that contains the referred object.
resourceVersion
string
No description provided.
subresource
string
No description provided.

PolicyRule

Appears in:

PolicyRule maps requests based off metadata to an audit Level. Requests must match the rules of every field (an intersection of rules).

FieldDescription
level [Required]
Level
The Level that requests matching this rule are recorded at.
users
[]string
The users (by authenticated user name) this rule applies to. An empty list implies every user.
userGroups
[]string
The user groups this rule applies to. A user is considered matching if it is a member of any of the UserGroups. An empty list implies every user group.
verbs
[]string
The verbs that match this rule. An empty list implies every verb.
resources
[]GroupResources
Resources that this rule matches. An empty list implies all kinds in all API groups.
namespaces
[]string
Namespaces that this rule matches. The empty string "" matches non-namespaced resources. An empty list implies every namespace.
nonResourceURLs
[]string
NonResourceURLs is a set of URL paths that should be audited. ∗s are allowed, but only as the full, final step in the path. Examples: "/metrics" - Log requests for apiserver metrics "/healthz∗" - Log all health checks
omitStages
[]Stage
OmitStages is a list of stages for which no events are created. Note that this can also be specified policy wide in which case the union of both are omitted. An empty list means no restrictions will apply.
omitManagedFields
bool
OmitManagedFields indicates whether to omit the managed fields of the request and response bodies from being written to the API audit log. - a value of 'true' will drop the managed fields from the API audit log - a value of 'false' indicates that the managed fileds should be included in the API audit log Note that the value, if specified, in this rule will override the global default If a value is not specified then the global default specified in Policy.OmitManagedFields will stand.

Stage

(Alias of string)

Appears in:

Stage defines the stages in request handling that audit events may be generated.

4 - kube-apiserver Configuration (v1)

Package v1 is the v1 version of the API.

Resource Types

AdmissionConfiguration

AdmissionConfiguration provides versioned configuration for admission controllers.

FieldDescription
apiVersion
string
apiserver.config.k8s.io/v1
kind
string
AdmissionConfiguration
plugins
[]AdmissionPluginConfiguration
Plugins allows specifying a configuration per admission control plugin.

AdmissionPluginConfiguration

Appears in:

AdmissionPluginConfiguration provides the configuration for a single plug-in.

FieldDescription
name [Required]
string
Name is the name of the admission controller. It must match the registered admission plugin name.
path
string
Path is the path to a configuration file that contains the plugin's configuration
configuration
k8s.io/apimachinery/pkg/runtime.Unknown
Configuration is an embedded configuration object to be used as the plugin's configuration. If present, it will be used instead of the path to the configuration file.

5 - kube-apiserver Configuration (v1alpha1)

Package v1alpha1 is the v1alpha1 version of the API.

Resource Types

AdmissionConfiguration

AdmissionConfiguration provides versioned configuration for admission controllers.

FieldDescription
apiVersion
string
apiserver.k8s.io/v1alpha1
kind
string
AdmissionConfiguration
plugins
[]AdmissionPluginConfiguration

Plugins allows specifying a configuration per admission control plugin.

EgressSelectorConfiguration

EgressSelectorConfiguration provides versioned configuration for egress selector clients.

FieldDescription
apiVersion
string
apiserver.k8s.io/v1alpha1
kind
string
EgressSelectorConfiguration
egressSelections [Required]
[]EgressSelection

connectionServices contains a list of egress selection client configurations

TracingConfiguration

TracingConfiguration provides versioned configuration for tracing clients.

FieldDescription
apiVersion
string
apiserver.k8s.io/v1alpha1
kind
string
TracingConfiguration
endpoint
string

Endpoint of the collector that's running on the control-plane node. The APIServer uses the egressType ControlPlane when sending data to the collector. The syntax is defined in https://github.com/grpc/grpc/blob/master/doc/naming.md. Defaults to the otlpgrpc default, localhost:4317 The connection is insecure, and does not support TLS.

samplingRatePerMillion
int32

SamplingRatePerMillion is the number of samples to collect per million spans. Defaults to 0.

AdmissionPluginConfiguration

Appears in:

AdmissionPluginConfiguration provides the configuration for a single plug-in.

FieldDescription
name [Required]
string

Name is the name of the admission controller. It must match the registered admission plugin name.

path
string

Path is the path to a configuration file that contains the plugin's configuration

configuration
k8s.io/apimachinery/pkg/runtime.Unknown

Configuration is an embedded configuration object to be used as the plugin's configuration. If present, it will be used instead of the path to the configuration file.

Connection

Appears in:

Connection provides the configuration for a single egress selection client.

FieldDescription
proxyProtocol [Required]
ProtocolType

Protocol is the protocol used to connect from client to the konnectivity server.

transport
Transport

Transport defines the transport configurations we use to dial to the konnectivity server. This is required if ProxyProtocol is HTTPConnect or GRPC.

EgressSelection

Appears in:

EgressSelection provides the configuration for a single egress selection client.

FieldDescription
name [Required]
string

name is the name of the egress selection. Currently supported values are "controlplane", "master", "etcd" and "cluster" The "master" egress selector is deprecated in favor of "controlplane"

connection [Required]
Connection

connection is the exact information used to configure the egress selection

ProtocolType

(Alias of string)

Appears in:

ProtocolType is a set of valid values for Connection.ProtocolType

TCPTransport

Appears in:

TCPTransport provides the information to connect to konnectivity server via TCP

FieldDescription
url [Required]
string

URL is the location of the konnectivity server to connect to. As an example it might be "https://127.0.0.1:8131"

tlsConfig
TLSConfig

TLSConfig is the config needed to use TLS when connecting to konnectivity server

TLSConfig

Appears in:

TLSConfig provides the authentication information to connect to konnectivity server Only used with TCPTransport

FieldDescription
caBundle
string

caBundle is the file location of the CA to be used to determine trust with the konnectivity server. Must be absent/empty if TCPTransport.URL is prefixed with http:// If absent while TCPTransport.URL is prefixed with https://, default to system trust roots.

clientKey
string

clientKey is the file location of the client key to be used in mtls handshakes with the konnectivity server. Must be absent/empty if TCPTransport.URL is prefixed with http:// Must be configured if TCPTransport.URL is prefixed with https://

clientCert
string

clientCert is the file location of the client certificate to be used in mtls handshakes with the konnectivity server. Must be absent/empty if TCPTransport.URL is prefixed with http:// Must be configured if TCPTransport.URL is prefixed with https://

Transport

Appears in:

Transport defines the transport configurations we use to dial to the konnectivity server

FieldDescription
tcp
TCPTransport

TCP is the TCP configuration for communicating with the konnectivity server via TCP ProxyProtocol of GRPC is not supported with TCP transport at the moment Requires at least one of TCP or UDS to be set

uds
UDSTransport

UDS is the UDS configuration for communicating with the konnectivity server via UDS Requires at least one of TCP or UDS to be set

UDSTransport

Appears in:

UDSTransport provides the information to connect to konnectivity server via UDS

FieldDescription
udsName [Required]
string

UDSName is the name of the unix domain socket to connect to konnectivity server This does not use a unix:// prefix. (Eg: /etc/srv/kubernetes/konnectivity-server/konnectivity-server.socket)

6 - kube-apiserver Encryption Configuration (v1)

Package v1 is the v1 version of the API.

Resource Types

EncryptionConfiguration

EncryptionConfiguration stores the complete configuration for encryption providers.

FieldDescription
apiVersion
string
apiserver.config.k8s.io/v1
kind
string
EncryptionConfiguration
resources [Required]
[]ResourceConfiguration

resources is a list containing resources, and their corresponding encryption providers.

AESConfiguration

Appears in:

AESConfiguration contains the API configuration for an AES transformer.

FieldDescription
keys [Required]
[]Key

keys is a list of keys to be used for creating the AES transformer. Each key has to be 32 bytes long for AES-CBC and 16, 24 or 32 bytes for AES-GCM.

IdentityConfiguration

Appears in:

IdentityConfiguration is an empty struct to allow identity transformer in provider configuration.

KMSConfiguration

Appears in:

KMSConfiguration contains the name, cache size and path to configuration file for a KMS based envelope transformer.

FieldDescription
name [Required]
string

name is the name of the KMS plugin to be used.

cachesize
int32

cachesize is the maximum number of secrets which are cached in memory. The default value is 1000. Set to a negative value to disable caching.

endpoint [Required]
string

endpoint is the gRPC server listening address, for example "unix:///var/run/kms-provider.sock".

timeout
meta/v1.Duration

timeout for gRPC calls to kms-plugin (ex. 5s). The default is 3 seconds.

Key

Appears in:

Key contains name and secret of the provided key for a transformer.

FieldDescription
name [Required]
string

name is the name of the key to be used while storing data to disk.

secret [Required]
string

secret is the actual key, encoded in base64.

ProviderConfiguration

Appears in:

ProviderConfiguration stores the provided configuration for an encryption provider.

FieldDescription
aesgcm [Required]
AESConfiguration

aesgcm is the configuration for the AES-GCM transformer.

aescbc [Required]
AESConfiguration

aescbc is the configuration for the AES-CBC transformer.

secretbox [Required]
SecretboxConfiguration

secretbox is the configuration for the Secretbox based transformer.

identity [Required]
IdentityConfiguration

identity is the (empty) configuration for the identity transformer.

kms [Required]
KMSConfiguration

kms contains the name, cache size and path to configuration file for a KMS based envelope transformer.

ResourceConfiguration

Appears in:

ResourceConfiguration stores per resource configuration.

FieldDescription
resources [Required]
[]string

resources is a list of kubernetes resources which have to be encrypted.

providers [Required]
[]ProviderConfiguration

providers is a list of transformers to be used for reading and writing the resources to disk. eg: aesgcm, aescbc, secretbox, identity.

SecretboxConfiguration

Appears in:

SecretboxConfiguration contains the API configuration for an Secretbox transformer.

FieldDescription
keys [Required]
[]Key

keys is a list of keys to be used for creating the Secretbox transformer. Each key has to be 32 bytes long.

7 - kube-proxy Configuration (v1alpha1)

Resource Types

KubeProxyConfiguration

KubeProxyConfiguration contains everything necessary to configure the Kubernetes proxy server.

FieldDescription
apiVersion
string
kubeproxy.config.k8s.io/v1alpha1
kind
string
KubeProxyConfiguration
featureGates [Required]
map[string]bool
featureGates is a map of feature names to bools that enable or disable alpha/experimental features.
bindAddress [Required]
string
bindAddress is the IP address for the proxy server to serve on (set to 0.0.0.0 for all interfaces)
healthzBindAddress [Required]
string
healthzBindAddress is the IP address and port for the health check server to serve on, defaulting to 0.0.0.0:10256
metricsBindAddress [Required]
string
metricsBindAddress is the IP address and port for the metrics server to serve on, defaulting to 127.0.0.1:10249 (set to 0.0.0.0 for all interfaces)
bindAddressHardFail [Required]
bool
bindAddressHardFail, if true, kube-proxy will treat failure to bind to a port as fatal and exit
enableProfiling [Required]
bool
enableProfiling enables profiling via web interface on /debug/pprof handler. Profiling handlers will be handled by metrics server.
clusterCIDR [Required]
string
clusterCIDR is the CIDR range of the pods in the cluster. It is used to bridge traffic coming from outside of the cluster. If not provided, no off-cluster bridging will be performed.
hostnameOverride [Required]
string
hostnameOverride, if non-empty, will be used as the identity instead of the actual hostname.
clientConnection [Required]
ClientConnectionConfiguration
clientConnection specifies the kubeconfig file and client connection settings for the proxy server to use when communicating with the apiserver.
iptables [Required]
KubeProxyIPTablesConfiguration
iptables contains iptables-related configuration options.
ipvs [Required]
KubeProxyIPVSConfiguration
ipvs contains ipvs-related configuration options.
oomScoreAdj [Required]
int32
oomScoreAdj is the oom-score-adj value for kube-proxy process. Values must be within the range [-1000, 1000]
mode [Required]
ProxyMode
mode specifies which proxy mode to use.
portRange [Required]
string
portRange is the range of host ports (beginPort-endPort, inclusive) that may be consumed in order to proxy service traffic. If unspecified (0-0) then ports will be randomly chosen.
udpIdleTimeout [Required]
meta/v1.Duration
udpIdleTimeout is how long an idle UDP connection will be kept open (e.g. '250ms', '2s'). Must be greater than 0. Only applicable for proxyMode=userspace.
conntrack [Required]
KubeProxyConntrackConfiguration
conntrack contains conntrack-related configuration options.
configSyncPeriod [Required]
meta/v1.Duration
configSyncPeriod is how often configuration from the apiserver is refreshed. Must be greater than 0.
nodePortAddresses [Required]
[]string
nodePortAddresses is the --nodeport-addresses value for kube-proxy process. Values must be valid IP blocks. These values are as a parameter to select the interfaces where nodeport works. In case someone would like to expose a service on localhost for local visit and some other interfaces for particular purpose, a list of IP blocks would do that. If set it to "127.0.0.0/8", kube-proxy will only select the loopback interface for NodePort. If set it to a non-zero IP block, kube-proxy will filter that down to just the IPs that applied to the node. An empty string slice is meant to select all network interfaces.
winkernel [Required]
KubeProxyWinkernelConfiguration
winkernel contains winkernel-related configuration options.
showHiddenMetricsForVersion [Required]
string
ShowHiddenMetricsForVersion is the version for which you want to show hidden metrics.
detectLocalMode [Required]
LocalMode
DetectLocalMode determines mode to use for detecting local traffic, defaults to LocalModeClusterCIDR

KubeProxyConntrackConfiguration

Appears in:

KubeProxyConntrackConfiguration contains conntrack settings for the Kubernetes proxy server.

FieldDescription
maxPerCore [Required]
int32
maxPerCore is the maximum number of NAT connections to track per CPU core (0 to leave the limit as-is and ignore min).
min [Required]
int32
min is the minimum value of connect-tracking records to allocate, regardless of conntrackMaxPerCore (set maxPerCore=0 to leave the limit as-is).
tcpEstablishedTimeout [Required]
meta/v1.Duration
tcpEstablishedTimeout is how long an idle TCP connection will be kept open (e.g. '2s'). Must be greater than 0 to set.
tcpCloseWaitTimeout [Required]
meta/v1.Duration
tcpCloseWaitTimeout is how long an idle conntrack entry in CLOSE_WAIT state will remain in the conntrack table. (e.g. '60s'). Must be greater than 0 to set.

KubeProxyIPTablesConfiguration

Appears in:

KubeProxyIPTablesConfiguration contains iptables-related configuration details for the Kubernetes proxy server.

FieldDescription
masqueradeBit [Required]
int32
masqueradeBit is the bit of the iptables fwmark space to use for SNAT if using the pure iptables proxy mode. Values must be within the range [0, 31].
masqueradeAll [Required]
bool
masqueradeAll tells kube-proxy to SNAT everything if using the pure iptables proxy mode.
syncPeriod [Required]
meta/v1.Duration
syncPeriod is the period that iptables rules are refreshed (e.g. '5s', '1m', '2h22m'). Must be greater than 0.
minSyncPeriod [Required]
meta/v1.Duration
minSyncPeriod is the minimum period that iptables rules are refreshed (e.g. '5s', '1m', '2h22m').

KubeProxyIPVSConfiguration

Appears in:

KubeProxyIPVSConfiguration contains ipvs-related configuration details for the Kubernetes proxy server.

FieldDescription
syncPeriod [Required]
meta/v1.Duration
syncPeriod is the period that ipvs rules are refreshed (e.g. '5s', '1m', '2h22m'). Must be greater than 0.
minSyncPeriod [Required]
meta/v1.Duration
minSyncPeriod is the minimum period that ipvs rules are refreshed (e.g. '5s', '1m', '2h22m').
scheduler [Required]
string
ipvs scheduler
excludeCIDRs [Required]
[]string
excludeCIDRs is a list of CIDR's which the ipvs proxier should not touch when cleaning up ipvs services.
strictARP [Required]
bool
strict ARP configure arp_ignore and arp_announce to avoid answering ARP queries from kube-ipvs0 interface
tcpTimeout [Required]
meta/v1.Duration
tcpTimeout is the timeout value used for idle IPVS TCP sessions. The default value is 0, which preserves the current timeout value on the system.
tcpFinTimeout [Required]
meta/v1.Duration
tcpFinTimeout is the timeout value used for IPVS TCP sessions after receiving a FIN. The default value is 0, which preserves the current timeout value on the system.
udpTimeout [Required]
meta/v1.Duration
udpTimeout is the timeout value used for IPVS UDP packets. The default value is 0, which preserves the current timeout value on the system.

KubeProxyWinkernelConfiguration

Appears in:

KubeProxyWinkernelConfiguration contains Windows/HNS settings for the Kubernetes proxy server.

FieldDescription
networkName [Required]
string
networkName is the name of the network kube-proxy will use to create endpoints and policies
sourceVip [Required]
string
sourceVip is the IP address of the source VIP endoint used for NAT when loadbalancing
enableDSR [Required]
bool
enableDSR tells kube-proxy whether HNS policies should be created with DSR

LocalMode

(Alias of string) Appears in:

LocalMode represents modes to detect local traffic from the node

ProxyMode

(Alias of string) Appears in:

ProxyMode represents modes used by the Kubernetes proxy server.

Currently, three modes of proxy are available in Linux platform: 'userspace' (older, going to be EOL), 'iptables' (newer, faster), 'ipvs'(newest, better in performance and scalability).

Two modes of proxy are available in Windows platform: 'userspace'(older, stable) and 'kernelspace' (newer, faster).

In Linux platform, if proxy mode is blank, use the best-available proxy (currently iptables, but may change in the future). If the iptables proxy is selected, regardless of how, but the system's kernel or iptables versions are insufficient, this always falls back to the userspace proxy. IPVS mode will be enabled when proxy mode is set to 'ipvs', and the fall back path is firstly iptables and then userspace.

In Windows platform, if proxy mode is blank, use the best-available proxy (currently userspace, but may change in the future). If winkernel proxy is selected, regardless of how, but the Windows kernel can't support this mode of proxy, this always falls back to the userspace proxy.

ClientConnectionConfiguration

Appears in:

ClientConnectionConfiguration contains details for constructing a client.

FieldDescription
kubeconfig [Required]
string
kubeconfig is the path to a KubeConfig file.
acceptContentTypes [Required]
string
acceptContentTypes defines the Accept header sent by clients when connecting to a server, overriding the default value of 'application/json'. This field will control all connections to the server used by a particular client.
contentType [Required]
string
contentType is the content type used when sending data to the server from this client.
qps [Required]
float32
qps controls the number of queries per second allowed for this connection.
burst [Required]
int32
burst allows extra queries to accumulate when a client is exceeding its rate.

FormatOptions

Appears in:

FormatOptions contains options for the different logging formats.

FieldDescription
json [Required]
JSONOptions
[Experimental] JSON contains options for logging format "json".

JSONOptions

Appears in:

JSONOptions contains options for logging format "json".

FieldDescription
splitStream [Required]
bool
[Experimental] SplitStream redirects error messages to stderr while info messages go to stdout, with buffering. The default is to write both to stdout, without buffering.
infoBufferSize [Required]
k8s.io/apimachinery/pkg/api/resource.QuantityValue
[Experimental] InfoBufferSize sets the size of the info stream when using split streams. The default is zero, which disables buffering.

VModuleConfiguration

(Alias of []k8s.io/component-base/config/v1alpha1.VModuleItem) Appears in:

VModuleConfiguration is a collection of individual file names or patterns and the corresponding verbosity threshold.

8 - kube-scheduler Configuration (v1beta2)

Resource Types

DefaultPreemptionArgs

DefaultPreemptionArgs holds arguments used to configure the DefaultPreemption plugin.

FieldDescription
apiVersion
string
kubescheduler.config.k8s.io/v1beta2
kind
string
DefaultPreemptionArgs
minCandidateNodesPercentage [Required]
int32
MinCandidateNodesPercentage is the minimum number of candidates to shortlist when dry running preemption as a percentage of number of nodes. Must be in the range [0, 100]. Defaults to 10% of the cluster size if unspecified.
minCandidateNodesAbsolute [Required]
int32
MinCandidateNodesAbsolute is the absolute minimum number of candidates to shortlist. The likely number of candidates enumerated for dry running preemption is given by the formula: numCandidates = max(numNodes ∗ minCandidateNodesPercentage, minCandidateNodesAbsolute) We say "likely" because there are other factors such as PDB violations that play a role in the number of candidates shortlisted. Must be at least 0 nodes. Defaults to 100 nodes if unspecified.

InterPodAffinityArgs

InterPodAffinityArgs holds arguments used to configure the InterPodAffinity plugin.

FieldDescription
apiVersion
string
kubescheduler.config.k8s.io/v1beta2
kind
string
InterPodAffinityArgs
hardPodAffinityWeight [Required]
int32
HardPodAffinityWeight is the scoring weight for existing pods with a matching hard affinity to the incoming pod.

KubeSchedulerConfiguration

KubeSchedulerConfiguration configures a scheduler

FieldDescription
apiVersion
string
kubescheduler.config.k8s.io/v1beta2
kind
string
KubeSchedulerConfiguration
parallelism [Required]
int32
Parallelism defines the amount of parallelism in algorithms for scheduling a Pods. Must be greater than 0. Defaults to 16
leaderElection [Required]
LeaderElectionConfiguration
LeaderElection defines the configuration of leader election client.
clientConnection [Required]
ClientConnectionConfiguration
ClientConnection specifies the kubeconfig file and client connection settings for the proxy server to use when communicating with the apiserver.
healthzBindAddress [Required]
string
Note: Both HealthzBindAddress and MetricsBindAddress fields are deprecated. Only empty address or port 0 is allowed. Anything else will fail validation. HealthzBindAddress is the IP address and port for the health check server to serve on.
metricsBindAddress [Required]
string
MetricsBindAddress is the IP address and port for the metrics server to serve on.
DebuggingConfiguration [Required]
DebuggingConfiguration
(Members of DebuggingConfiguration are embedded into this type.) DebuggingConfiguration holds configuration for Debugging related features TODO: We might wanna make this a substruct like Debugging componentbaseconfigv1alpha1.DebuggingConfiguration
percentageOfNodesToScore [Required]
int32
PercentageOfNodesToScore is the percentage of all nodes that once found feasible for running a pod, the scheduler stops its search for more feasible nodes in the cluster. This helps improve scheduler's performance. Scheduler always tries to find at least "minFeasibleNodesToFind" feasible nodes no matter what the value of this flag is. Example: if the cluster size is 500 nodes and the value of this flag is 30, then scheduler stops finding further feasible nodes once it finds 150 feasible ones. When the value is 0, default percentage (5%--50% based on the size of the cluster) of the nodes will be scored.
podInitialBackoffSeconds [Required]
int64
PodInitialBackoffSeconds is the initial backoff for unschedulable pods. If specified, it must be greater than 0. If this value is null, the default value (1s) will be used.
podMaxBackoffSeconds [Required]
int64
PodMaxBackoffSeconds is the max backoff for unschedulable pods. If specified, it must be greater than podInitialBackoffSeconds. If this value is null, the default value (10s) will be used.
profiles [Required]
[]KubeSchedulerProfile
Profiles are scheduling profiles that kube-scheduler supports. Pods can choose to be scheduled under a particular profile by setting its associated scheduler name. Pods that don't specify any scheduler name are scheduled with the "default-scheduler" profile, if present here.
extenders [Required]
[]Extender
Extenders are the list of scheduler extenders, each holding the values of how to communicate with the extender. These extenders are shared by all scheduler profiles.

NodeAffinityArgs

NodeAffinityArgs holds arguments to configure the NodeAffinity plugin.

FieldDescription
apiVersion
string
kubescheduler.config.k8s.io/v1beta2
kind
string
NodeAffinityArgs
addedAffinity
core/v1.NodeAffinity
AddedAffinity is applied to all Pods additionally to the NodeAffinity specified in the PodSpec. That is, Nodes need to satisfy AddedAffinity AND .spec.NodeAffinity. AddedAffinity is empty by default (all Nodes match). When AddedAffinity is used, some Pods with affinity requirements that match a specific Node (such as Daemonset Pods) might remain unschedulable.

NodeResourcesBalancedAllocationArgs

NodeResourcesBalancedAllocationArgs holds arguments used to configure NodeResourcesBalancedAllocation plugin.

FieldDescription
apiVersion
string
kubescheduler.config.k8s.io/v1beta2
kind
string
NodeResourcesBalancedAllocationArgs
resources [Required]
[]ResourceSpec
Resources to be managed, the default is "cpu" and "memory" if not specified.

NodeResourcesFitArgs

NodeResourcesFitArgs holds arguments used to configure the NodeResourcesFit plugin.

FieldDescription
apiVersion
string
kubescheduler.config.k8s.io/v1beta2
kind
string
NodeResourcesFitArgs
ignoredResources [Required]
[]string
IgnoredResources is the list of resources that NodeResources fit filter should ignore. This doesn't apply to scoring.
ignoredResourceGroups [Required]
[]string
IgnoredResourceGroups defines the list of resource groups that NodeResources fit filter should ignore. e.g. if group is ["example.com"], it will ignore all resource names that begin with "example.com", such as "example.com/aaa" and "example.com/bbb". A resource group name can't contain '/'. This doesn't apply to scoring.
scoringStrategy [Required]
ScoringStrategy
ScoringStrategy selects the node resource scoring strategy. The default strategy is LeastAllocated with an equal "cpu" and "memory" weight.

PodTopologySpreadArgs

PodTopologySpreadArgs holds arguments used to configure the PodTopologySpread plugin.

FieldDescription
apiVersion
string
kubescheduler.config.k8s.io/v1beta2
kind
string
PodTopologySpreadArgs
defaultConstraints
[]core/v1.TopologySpreadConstraint
DefaultConstraints defines topology spread constraints to be applied to Pods that don't define any in `pod.spec.topologySpreadConstraints`. `.defaultConstraints[∗].labelSelectors` must be empty, as they are deduced from the Pod's membership to Services, ReplicationControllers, ReplicaSets or StatefulSets. When not empty, .defaultingType must be "List".
defaultingType
PodTopologySpreadConstraintsDefaulting
DefaultingType determines how .defaultConstraints are deduced. Can be one of "System" or "List".
  • "System": Use kubernetes defined constraints that spread Pods among Nodes and Zones.
  • "List": Use constraints defined in .defaultConstraints.

Defaults to "List" if feature gate DefaultPodTopologySpread is disabled and to "System" if enabled.

VolumeBindingArgs

VolumeBindingArgs holds arguments used to configure the VolumeBinding plugin.

FieldDescription
apiVersion
string
kubescheduler.config.k8s.io/v1beta2
kind
string
VolumeBindingArgs
bindTimeoutSeconds [Required]
int64
BindTimeoutSeconds is the timeout in seconds in volume binding operation. Value must be non-negative integer. The value zero indicates no waiting. If this value is nil, the default value (600) will be used.
shape
[]UtilizationShapePoint
Shape specifies the points defining the score function shape, which is used to score nodes based on the utilization of statically provisioned PVs. The utilization is calculated by dividing the total requested storage of the pod by the total capacity of feasible PVs on each node. Each point contains utilization (ranges from 0 to 100) and its associated score (ranges from 0 to 10). You can turn the priority by specifying different scores for different utilization numbers. The default shape points are: 1) 0 for 0 utilization 2) 10 for 100 utilization All points must be sorted in increasing order by utilization.

Extender

Appears in:

Extender holds the parameters used to communicate with the extender. If a verb is unspecified/empty, it is assumed that the extender chose not to provide that extension.

FieldDescription
urlPrefix [Required]
string
URLPrefix at which the extender is available
filterVerb [Required]
string
Verb for the filter call, empty if not supported. This verb is appended to the URLPrefix when issuing the filter call to extender.
preemptVerb [Required]
string
Verb for the preempt call, empty if not supported. This verb is appended to the URLPrefix when issuing the preempt call to extender.
prioritizeVerb [Required]
string
Verb for the prioritize call, empty if not supported. This verb is appended to the URLPrefix when issuing the prioritize call to extender.
weight [Required]
int64
The numeric multiplier for the node scores that the prioritize call generates. The weight should be a positive integer
bindVerb [Required]
string
Verb for the bind call, empty if not supported. This verb is appended to the URLPrefix when issuing the bind call to extender. If this method is implemented by the extender, it is the extender's responsibility to bind the pod to apiserver. Only one extender can implement this function.
enableHTTPS [Required]
bool
EnableHTTPS specifies whether https should be used to communicate with the extender
tlsConfig [Required]
ExtenderTLSConfig
TLSConfig specifies the transport layer security config
httpTimeout [Required]
meta/v1.Duration
HTTPTimeout specifies the timeout duration for a call to the extender. Filter timeout fails the scheduling of the pod. Prioritize timeout is ignored, k8s/other extenders priorities are used to select the node.
nodeCacheCapable [Required]
bool
NodeCacheCapable specifies that the extender is capable of caching node information, so the scheduler should only send minimal information about the eligible nodes assuming that the extender already cached full details of all nodes in the cluster
managedResources
[]ExtenderManagedResource
ManagedResources is a list of extended resources that are managed by this extender. - A pod will be sent to the extender on the Filter, Prioritize and Bind (if the extender is the binder) phases iff the pod requests at least one of the extended resources in this list. If empty or unspecified, all pods will be sent to this extender. - If IgnoredByScheduler is set to true for a resource, kube-scheduler will skip checking the resource in predicates.
ignorable [Required]
bool
Ignorable specifies if the extender is ignorable, i.e. scheduling should not fail when the extender returns an error or is not reachable.

ExtenderManagedResource

Appears in:

ExtenderManagedResource describes the arguments of extended resources managed by an extender.

FieldDescription
name [Required]
string
Name is the extended resource name.
ignoredByScheduler [Required]
bool
IgnoredByScheduler indicates whether kube-scheduler should ignore this resource when applying predicates.

ExtenderTLSConfig

Appears in:

ExtenderTLSConfig contains settings to enable TLS with extender

FieldDescription
insecure [Required]
bool
Server should be accessed without verifying the TLS certificate. For testing only.
serverName [Required]
string
ServerName is passed to the server for SNI and is used in the client to check server certificates against. If ServerName is empty, the hostname used to contact the server is used.
certFile [Required]
string
Server requires TLS client certificate authentication
keyFile [Required]
string
Server requires TLS client certificate authentication
caFile [Required]
string
Trusted root certificates for server
certData [Required]
[]byte
CertData holds PEM-encoded bytes (typically read from a client certificate file). CertData takes precedence over CertFile
keyData [Required]
[]byte
KeyData holds PEM-encoded bytes (typically read from a client certificate key file). KeyData takes precedence over KeyFile
caData [Required]
[]byte
CAData holds PEM-encoded bytes (typically read from a root certificates bundle). CAData takes precedence over CAFile

KubeSchedulerProfile

Appears in:

KubeSchedulerProfile is a scheduling profile.

FieldDescription
schedulerName [Required]
string
SchedulerName is the name of the scheduler associated to this profile. If SchedulerName matches with the pod's "spec.schedulerName", then the pod is scheduled with this profile.
plugins [Required]
Plugins
Plugins specify the set of plugins that should be enabled or disabled. Enabled plugins are the ones that should be enabled in addition to the default plugins. Disabled plugins are any of the default plugins that should be disabled. When no enabled or disabled plugin is specified for an extension point, default plugins for that extension point will be used if there is any. If a QueueSort plugin is specified, the same QueueSort Plugin and PluginConfig must be specified for all profiles.
pluginConfig [Required]
[]PluginConfig
PluginConfig is an optional set of custom plugin arguments for each plugin. Omitting config args for a plugin is equivalent to using the default config for that plugin.

Plugin

Appears in:

Plugin specifies a plugin name and its weight when applicable. Weight is used only for Score plugins.

FieldDescription
name [Required]
string
Name defines the name of plugin
weight [Required]
int32
Weight defines the weight of plugin, only used for Score plugins.

PluginConfig

Appears in:

PluginConfig specifies arguments that should be passed to a plugin at the time of initialization. A plugin that is invoked at multiple extension points is initialized once. Args can have arbitrary structure. It is up to the plugin to process these Args.

FieldDescription
name [Required]
string
Name defines the name of plugin being configured
args [Required]
k8s.io/apimachinery/pkg/runtime.RawExtension
Args defines the arguments passed to the plugins at the time of initialization. Args can have arbitrary structure.

PluginSet

Appears in:

PluginSet specifies enabled and disabled plugins for an extension point. If an array is empty, missing, or nil, default plugins at that extension point will be used.

FieldDescription
enabled [Required]
[]Plugin
Enabled specifies plugins that should be enabled in addition to default plugins. If the default plugin is also configured in the scheduler config file, the weight of plugin will be overridden accordingly. These are called after default plugins and in the same order specified here.
disabled [Required]
[]Plugin
Disabled specifies default plugins that should be disabled. When all default plugins need to be disabled, an array containing only one "∗" should be provided.

Plugins

Appears in:

Plugins include multiple extension points. When specified, the list of plugins for a particular extension point are the only ones enabled. If an extension point is omitted from the config, then the default set of plugins is used for that extension point. Enabled plugins are called in the order specified here, after default plugins. If they need to be invoked before default plugins, default plugins must be disabled and re-enabled here in desired order.

FieldDescription
queueSort [Required]
PluginSet
QueueSort is a list of plugins that should be invoked when sorting pods in the scheduling queue.
preFilter [Required]
PluginSet
PreFilter is a list of plugins that should be invoked at "PreFilter" extension point of the scheduling framework.
filter [Required]
PluginSet
Filter is a list of plugins that should be invoked when filtering out nodes that cannot run the Pod.
postFilter [Required]
PluginSet
PostFilter is a list of plugins that are invoked after filtering phase, but only when no feasible nodes were found for the pod.
preScore [Required]
PluginSet
PreScore is a list of plugins that are invoked before scoring.
score [Required]
PluginSet
Score is a list of plugins that should be invoked when ranking nodes that have passed the filtering phase.
reserve [Required]
PluginSet
Reserve is a list of plugins invoked when reserving/unreserving resources after a node is assigned to run the pod.
permit [Required]
PluginSet
Permit is a list of plugins that control binding of a Pod. These plugins can prevent or delay binding of a Pod.
preBind [Required]
PluginSet
PreBind is a list of plugins that should be invoked before a pod is bound.
bind [Required]
PluginSet
Bind is a list of plugins that should be invoked at "Bind" extension point of the scheduling framework. The scheduler call these plugins in order. Scheduler skips the rest of these plugins as soon as one returns success.
postBind [Required]
PluginSet
PostBind is a list of plugins that should be invoked after a pod is successfully bound.
multiPoint [Required]
PluginSet
MultiPoint is a simplified config section to enable plugins for all valid extension points.

PodTopologySpreadConstraintsDefaulting

(Alias of string)

Appears in:

PodTopologySpreadConstraintsDefaulting defines how to set default constraints for the PodTopologySpread plugin.

RequestedToCapacityRatioParam

Appears in:

RequestedToCapacityRatioParam define RequestedToCapacityRatio parameters

FieldDescription
shape [Required]
[]UtilizationShapePoint
Shape is a list of points defining the scoring function shape.

ResourceSpec

Appears in:

ResourceSpec represents a single resource.

FieldDescription
name [Required]
string
Name of the resource.
weight [Required]
int64
Weight of the resource.

ScoringStrategy

Appears in:

ScoringStrategy define ScoringStrategyType for node resource plugin

FieldDescription
type [Required]
ScoringStrategyType
Type selects which strategy to run.
resources [Required]
[]ResourceSpec
Resources to consider when scoring. The default resource set includes "cpu" and "memory" with an equal weight. Allowed weights go from 1 to 100. Weight defaults to 1 if not specified or explicitly set to 0.
requestedToCapacityRatio [Required]
RequestedToCapacityRatioParam
Arguments specific to RequestedToCapacityRatio strategy.

ScoringStrategyType

(Alias of string)

Appears in:

ScoringStrategyType the type of scoring strategy used in NodeResourcesFit plugin.

UtilizationShapePoint

Appears in:

UtilizationShapePoint represents single point of priority function shape.

FieldDescription
utilization [Required]
int32
Utilization (x axis). Valid values are 0 to 100. Fully utilized node maps to 100.
score [Required]
int32
Score assigned to given utilization (y axis). Valid values are 0 to 10.

ClientConnectionConfiguration

Appears in:

ClientConnectionConfiguration contains details for constructing a client.

FieldDescription
kubeconfig [Required]
string
kubeconfig is the path to a KubeConfig file.
acceptContentTypes [Required]
string
acceptContentTypes defines the Accept header sent by clients when connecting to a server, overriding the default value of 'application/json'. This field will control all connections to the server used by a particular client.
contentType [Required]
string
contentType is the content type used when sending data to the server from this client.
qps [Required]
float32
qps controls the number of queries per second allowed for this connection.
burst [Required]
int32
burst allows extra queries to accumulate when a client is exceeding its rate.

DebuggingConfiguration

Appears in:

DebuggingConfiguration holds configuration for Debugging related features.

FieldDescription
enableProfiling [Required]
bool
enableProfiling enables profiling via web interface host:port/debug/pprof/
enableContentionProfiling [Required]
bool
enableContentionProfiling enables lock contention profiling, if enableProfiling is true.

FormatOptions

Appears in:

FormatOptions contains options for the different logging formats.

FieldDescription
json [Required]
JSONOptions
[Experimental] JSON contains options for logging format "json".

JSONOptions

Appears in:

JSONOptions contains options for logging format "json".

FieldDescription
splitStream [Required]
bool
[Experimental] SplitStream redirects error messages to stderr while info messages go to stdout, with buffering. The default is to write both to stdout, without buffering.
infoBufferSize [Required]
k8s.io/apimachinery/pkg/api/resource.QuantityValue
[Experimental] InfoBufferSize sets the size of the info stream when using split streams. The default is zero, which disables buffering.

LeaderElectionConfiguration

Appears in:

LeaderElectionConfiguration defines the configuration of leader election clients for components that can run with leader election enabled.

FieldDescription
leaderElect [Required]
bool
leaderElect enables a leader election client to gain leadership before executing the main loop. Enable this when running replicated components for high availability.
leaseDuration [Required]
meta/v1.Duration
leaseDuration is the duration that non-leader candidates will wait after observing a leadership renewal until attempting to acquire leadership of a led but unrenewed leader slot. This is effectively the maximum duration that a leader can be stopped before it is replaced by another candidate. This is only applicable if leader election is enabled.
renewDeadline [Required]
meta/v1.Duration
renewDeadline is the interval between attempts by the acting master to renew a leadership slot before it stops leading. This must be less than or equal to the lease duration. This is only applicable if leader election is enabled.
retryPeriod [Required]
meta/v1.Duration
retryPeriod is the duration the clients should wait between attempting acquisition and renewal of a leadership. This is only applicable if leader election is enabled.
resourceLock [Required]
string
resourceLock indicates the resource object type that will be used to lock during leader election cycles.
resourceName [Required]
string
resourceName indicates the name of resource object that will be used to lock during leader election cycles.
resourceNamespace [Required]
string
resourceName indicates the namespace of resource object that will be used to lock during leader election cycles.

LoggingConfiguration

Appears in:

LoggingConfiguration contains logging options Refer Logs Options for more information.

FieldDescription
format [Required]
string
Format Flag specifies the structure of log messages. default value of format is `text`
flushFrequency [Required]
time.Duration
Maximum number of seconds between log flushes. Ignored if the selected logging backend writes log messages without buffering.
verbosity [Required]
uint32
Verbosity is the threshold that determines which log messages are logged. Default is zero which logs only the most important messages. Higher values enable additional messages. Error messages are always logged.
vmodule [Required]
VModuleConfiguration
VModule overrides the verbosity threshold for individual files. Only supported for "text" log format.
sanitization [Required]
bool
[Experimental] When enabled prevents logging of fields tagged as sensitive (passwords, keys, tokens). Runtime log sanitization may introduce significant computation overhead and therefore should not be enabled in production.`)
options [Required]
FormatOptions
[Experimental] Options holds additional parameters that are specific to the different logging formats. Only the options for the selected format get used, but all of them get validated.

VModuleConfiguration

(Alias of []k8s.io/component-base/config/v1alpha1.VModuleItem)

Appears in:

VModuleConfiguration is a collection of individual file names or patterns and the corresponding verbosity threshold.

9 - kube-scheduler Configuration (v1beta3)

Resource Types

DefaultPreemptionArgs

DefaultPreemptionArgs holds arguments used to configure the DefaultPreemption plugin.

FieldDescription
apiVersion
string
kubescheduler.config.k8s.io/v1beta3
kind
string
DefaultPreemptionArgs
minCandidateNodesPercentage [Required]
int32
MinCandidateNodesPercentage is the minimum number of candidates to shortlist when dry running preemption as a percentage of number of nodes. Must be in the range [0, 100]. Defaults to 10% of the cluster size if unspecified.
minCandidateNodesAbsolute [Required]
int32
MinCandidateNodesAbsolute is the absolute minimum number of candidates to shortlist. The likely number of candidates enumerated for dry running preemption is given by the formula: numCandidates = max(numNodes ∗ minCandidateNodesPercentage, minCandidateNodesAbsolute) We say "likely" because there are other factors such as PDB violations that play a role in the number of candidates shortlisted. Must be at least 0 nodes. Defaults to 100 nodes if unspecified.

InterPodAffinityArgs

InterPodAffinityArgs holds arguments used to configure the InterPodAffinity plugin.

FieldDescription
apiVersion
string
kubescheduler.config.k8s.io/v1beta3
kind
string
InterPodAffinityArgs
hardPodAffinityWeight [Required]
int32
HardPodAffinityWeight is the scoring weight for existing pods with a matching hard affinity to the incoming pod.

KubeSchedulerConfiguration

KubeSchedulerConfiguration configures a scheduler

FieldDescription
apiVersion
string
kubescheduler.config.k8s.io/v1beta3
kind
string
KubeSchedulerConfiguration
parallelism [Required]
int32
Parallelism defines the amount of parallelism in algorithms for scheduling a Pods. Must be greater than 0. Defaults to 16
leaderElection [Required]
LeaderElectionConfiguration
LeaderElection defines the configuration of leader election client.
clientConnection [Required]
ClientConnectionConfiguration
ClientConnection specifies the kubeconfig file and client connection settings for the proxy server to use when communicating with the apiserver.
DebuggingConfiguration [Required]
DebuggingConfiguration
(Members of DebuggingConfiguration are embedded into this type.) DebuggingConfiguration holds configuration for Debugging related features TODO: We might wanna make this a substruct like Debugging componentbaseconfigv1alpha1.DebuggingConfiguration
percentageOfNodesToScore [Required]
int32
PercentageOfNodesToScore is the percentage of all nodes that once found feasible for running a pod, the scheduler stops its search for more feasible nodes in the cluster. This helps improve scheduler's performance. Scheduler always tries to find at least "minFeasibleNodesToFind" feasible nodes no matter what the value of this flag is. Example: if the cluster size is 500 nodes and the value of this flag is 30, then scheduler stops finding further feasible nodes once it finds 150 feasible ones. When the value is 0, default percentage (5%--50% based on the size of the cluster) of the nodes will be scored.
podInitialBackoffSeconds [Required]
int64
PodInitialBackoffSeconds is the initial backoff for unschedulable pods. If specified, it must be greater than 0. If this value is null, the default value (1s) will be used.
podMaxBackoffSeconds [Required]
int64
PodMaxBackoffSeconds is the max backoff for unschedulable pods. If specified, it must be greater than podInitialBackoffSeconds. If this value is null, the default value (10s) will be used.
profiles [Required]
[]KubeSchedulerProfile
Profiles are scheduling profiles that kube-scheduler supports. Pods can choose to be scheduled under a particular profile by setting its associated scheduler name. Pods that don't specify any scheduler name are scheduled with the "default-scheduler" profile, if present here.
extenders [Required]
[]Extender
Extenders are the list of scheduler extenders, each holding the values of how to communicate with the extender. These extenders are shared by all scheduler profiles.

NodeAffinityArgs

NodeAffinityArgs holds arguments to configure the NodeAffinity plugin.

FieldDescription
apiVersion
string
kubescheduler.config.k8s.io/v1beta3
kind
string
NodeAffinityArgs
addedAffinity
core/v1.NodeAffinity
AddedAffinity is applied to all Pods additionally to the NodeAffinity specified in the PodSpec. That is, Nodes need to satisfy AddedAffinity AND .spec.NodeAffinity. AddedAffinity is empty by default (all Nodes match). When AddedAffinity is used, some Pods with affinity requirements that match a specific Node (such as Daemonset Pods) might remain unschedulable.

NodeResourcesBalancedAllocationArgs

NodeResourcesBalancedAllocationArgs holds arguments used to configure NodeResourcesBalancedAllocation plugin.

FieldDescription
apiVersion
string
kubescheduler.config.k8s.io/v1beta3
kind
string
NodeResourcesBalancedAllocationArgs
resources [Required]
[]ResourceSpec
Resources to be managed, the default is "cpu" and "memory" if not specified.

NodeResourcesFitArgs

NodeResourcesFitArgs holds arguments used to configure the NodeResourcesFit plugin.

FieldDescription
apiVersion
string
kubescheduler.config.k8s.io/v1beta3
kind
string
NodeResourcesFitArgs
ignoredResources [Required]
[]string
IgnoredResources is the list of resources that NodeResources fit filter should ignore. This doesn't apply to scoring.
ignoredResourceGroups [Required]
[]string
IgnoredResourceGroups defines the list of resource groups that NodeResources fit filter should ignore. e.g. if group is ["example.com"], it will ignore all resource names that begin with "example.com", such as "example.com/aaa" and "example.com/bbb". A resource group name can't contain '/'. This doesn't apply to scoring.
scoringStrategy [Required]
ScoringStrategy
ScoringStrategy selects the node resource scoring strategy. The default strategy is LeastAllocated with an equal "cpu" and "memory" weight.

PodTopologySpreadArgs

PodTopologySpreadArgs holds arguments used to configure the PodTopologySpread plugin.

FieldDescription
apiVersion
string
kubescheduler.config.k8s.io/v1beta3
kind
string
PodTopologySpreadArgs
defaultConstraints
[]core/v1.TopologySpreadConstraint
DefaultConstraints defines topology spread constraints to be applied to Pods that don't define any in `pod.spec.topologySpreadConstraints`. `.defaultConstraints[∗].labelSelectors` must be empty, as they are deduced from the Pod's membership to Services, ReplicationControllers, ReplicaSets or StatefulSets. When not empty, .defaultingType must be "List".
defaultingType
PodTopologySpreadConstraintsDefaulting
DefaultingType determines how .defaultConstraints are deduced. Can be one of "System" or "List".
  • "System": Use kubernetes defined constraints that spread Pods among Nodes and Zones.
  • "List": Use constraints defined in .defaultConstraints.

Defaults to "List" if feature gate DefaultPodTopologySpread is disabled and to "System" if enabled.

VolumeBindingArgs

VolumeBindingArgs holds arguments used to configure the VolumeBinding plugin.

FieldDescription
apiVersion
string
kubescheduler.config.k8s.io/v1beta3
kind
string
VolumeBindingArgs
bindTimeoutSeconds [Required]
int64
BindTimeoutSeconds is the timeout in seconds in volume binding operation. Value must be non-negative integer. The value zero indicates no waiting. If this value is nil, the default value (600) will be used.
shape
[]UtilizationShapePoint
Shape specifies the points defining the score function shape, which is used to score nodes based on the utilization of statically provisioned PVs. The utilization is calculated by dividing the total requested storage of the pod by the total capacity of feasible PVs on each node. Each point contains utilization (ranges from 0 to 100) and its associated score (ranges from 0 to 10). You can turn the priority by specifying different scores for different utilization numbers. The default shape points are: 1) 0 for 0 utilization 2) 10 for 100 utilization All points must be sorted in increasing order by utilization.

Extender

Appears in:

Extender holds the parameters used to communicate with the extender. If a verb is unspecified/empty, it is assumed that the extender chose not to provide that extension.

FieldDescription
urlPrefix [Required]
string
URLPrefix at which the extender is available
filterVerb [Required]
string
Verb for the filter call, empty if not supported. This verb is appended to the URLPrefix when issuing the filter call to extender.
preemptVerb [Required]
string
Verb for the preempt call, empty if not supported. This verb is appended to the URLPrefix when issuing the preempt call to extender.
prioritizeVerb [Required]
string
Verb for the prioritize call, empty if not supported. This verb is appended to the URLPrefix when issuing the prioritize call to extender.
weight [Required]
int64
The numeric multiplier for the node scores that the prioritize call generates. The weight should be a positive integer
bindVerb [Required]
string
Verb for the bind call, empty if not supported. This verb is appended to the URLPrefix when issuing the bind call to extender. If this method is implemented by the extender, it is the extender's responsibility to bind the pod to apiserver. Only one extender can implement this function.
enableHTTPS [Required]
bool
EnableHTTPS specifies whether https should be used to communicate with the extender
tlsConfig [Required]
ExtenderTLSConfig
TLSConfig specifies the transport layer security config
httpTimeout [Required]
meta/v1.Duration
HTTPTimeout specifies the timeout duration for a call to the extender. Filter timeout fails the scheduling of the pod. Prioritize timeout is ignored, k8s/other extenders priorities are used to select the node.
nodeCacheCapable [Required]
bool
NodeCacheCapable specifies that the extender is capable of caching node information, so the scheduler should only send minimal information about the eligible nodes assuming that the extender already cached full details of all nodes in the cluster
managedResources
[]ExtenderManagedResource
ManagedResources is a list of extended resources that are managed by this extender. - A pod will be sent to the extender on the Filter, Prioritize and Bind (if the extender is the binder) phases iff the pod requests at least one of the extended resources in this list. If empty or unspecified, all pods will be sent to this extender. - If IgnoredByScheduler is set to true for a resource, kube-scheduler will skip checking the resource in predicates.
ignorable [Required]
bool
Ignorable specifies if the extender is ignorable, i.e. scheduling should not fail when the extender returns an error or is not reachable.

ExtenderManagedResource

Appears in:

ExtenderManagedResource describes the arguments of extended resources managed by an extender.

FieldDescription
name [Required]
string
Name is the extended resource name.
ignoredByScheduler [Required]
bool
IgnoredByScheduler indicates whether kube-scheduler should ignore this resource when applying predicates.

ExtenderTLSConfig

Appears in:

ExtenderTLSConfig contains settings to enable TLS with extender

FieldDescription
insecure [Required]
bool
Server should be accessed without verifying the TLS certificate. For testing only.
serverName [Required]
string
ServerName is passed to the server for SNI and is used in the client to check server certificates against. If ServerName is empty, the hostname used to contact the server is used.
certFile [Required]
string
Server requires TLS client certificate authentication
keyFile [Required]
string
Server requires TLS client certificate authentication
caFile [Required]
string
Trusted root certificates for server
certData [Required]
[]byte
CertData holds PEM-encoded bytes (typically read from a client certificate file). CertData takes precedence over CertFile
keyData [Required]
[]byte
KeyData holds PEM-encoded bytes (typically read from a client certificate key file). KeyData takes precedence over KeyFile
caData [Required]
[]byte
CAData holds PEM-encoded bytes (typically read from a root certificates bundle). CAData takes precedence over CAFile

KubeSchedulerProfile

Appears in:

KubeSchedulerProfile is a scheduling profile.

FieldDescription
schedulerName [Required]
string
SchedulerName is the name of the scheduler associated to this profile. If SchedulerName matches with the pod's "spec.schedulerName", then the pod is scheduled with this profile.
plugins [Required]
Plugins
Plugins specify the set of plugins that should be enabled or disabled. Enabled plugins are the ones that should be enabled in addition to the default plugins. Disabled plugins are any of the default plugins that should be disabled. When no enabled or disabled plugin is specified for an extension point, default plugins for that extension point will be used if there is any. If a QueueSort plugin is specified, the same QueueSort Plugin and PluginConfig must be specified for all profiles.
pluginConfig [Required]
[]PluginConfig
PluginConfig is an optional set of custom plugin arguments for each plugin. Omitting config args for a plugin is equivalent to using the default config for that plugin.

Plugin

Appears in:

Plugin specifies a plugin name and its weight when applicable. Weight is used only for Score plugins.

FieldDescription
name [Required]
string
Name defines the name of plugin
weight [Required]
int32
Weight defines the weight of plugin, only used for Score plugins.

PluginConfig

Appears in:

PluginConfig specifies arguments that should be passed to a plugin at the time of initialization. A plugin that is invoked at multiple extension points is initialized once. Args can have arbitrary structure. It is up to the plugin to process these Args.

FieldDescription
name [Required]
string
Name defines the name of plugin being configured
args [Required]
k8s.io/apimachinery/pkg/runtime.RawExtension
Args defines the arguments passed to the plugins at the time of initialization. Args can have arbitrary structure.

PluginSet

Appears in:

PluginSet specifies enabled and disabled plugins for an extension point. If an array is empty, missing, or nil, default plugins at that extension point will be used.

FieldDescription
enabled [Required]
[]Plugin
Enabled specifies plugins that should be enabled in addition to default plugins. If the default plugin is also configured in the scheduler config file, the weight of plugin will be overridden accordingly. These are called after default plugins and in the same order specified here.
disabled [Required]
[]Plugin
Disabled specifies default plugins that should be disabled. When all default plugins need to be disabled, an array containing only one "∗" should be provided.

Plugins

Appears in:

Plugins include multiple extension points. When specified, the list of plugins for a particular extension point are the only ones enabled. If an extension point is omitted from the config, then the default set of plugins is used for that extension point. Enabled plugins are called in the order specified here, after default plugins. If they need to be invoked before default plugins, default plugins must be disabled and re-enabled here in desired order.

FieldDescription
queueSort [Required]
PluginSet
QueueSort is a list of plugins that should be invoked when sorting pods in the scheduling queue.
preFilter [Required]
PluginSet
PreFilter is a list of plugins that should be invoked at "PreFilter" extension point of the scheduling framework.
filter [Required]
PluginSet
Filter is a list of plugins that should be invoked when filtering out nodes that cannot run the Pod.
postFilter [Required]
PluginSet
PostFilter is a list of plugins that are invoked after filtering phase, but only when no feasible nodes were found for the pod.
preScore [Required]
PluginSet
PreScore is a list of plugins that are invoked before scoring.
score [Required]
PluginSet
Score is a list of plugins that should be invoked when ranking nodes that have passed the filtering phase.
reserve [Required]
PluginSet
Reserve is a list of plugins invoked when reserving/unreserving resources after a node is assigned to run the pod.
permit [Required]
PluginSet
Permit is a list of plugins that control binding of a Pod. These plugins can prevent or delay binding of a Pod.
preBind [Required]
PluginSet
PreBind is a list of plugins that should be invoked before a pod is bound.
bind [Required]
PluginSet
Bind is a list of plugins that should be invoked at "Bind" extension point of the scheduling framework. The scheduler call these plugins in order. Scheduler skips the rest of these plugins as soon as one returns success.
postBind [Required]
PluginSet
PostBind is a list of plugins that should be invoked after a pod is successfully bound.
multiPoint [Required]
PluginSet
MultiPoint is a simplified config section to enable plugins for all valid extension points. Plugins enabled through MultiPoint will automatically register for every individual extension point the plugin has implemented. Disabling a plugin through MultiPoint disables that behavior. The same is true for disabling "∗" through MultiPoint (no default plugins will be automatically registered). Plugins can still be disabled through their individual extension points.

In terms of precedence, plugin config follows this basic hierarchy

  1. Specific extension points
  2. Explicitly configured MultiPoint plugins
  3. The set of default plugins, as MultiPoint plugins This implies that a higher precedence plugin will run first and overwrite any settings within MultiPoint. Explicitly user-configured plugins also take a higher precedence over default plugins. Within this hierarchy, an Enabled setting takes precedence over Disabled. For example, if a plugin is set in both multiPoint.Enabled and multiPoint.Disabled, the plugin will be enabled. Similarly, including multiPoint.Disabled = '∗' and multiPoint.Enabled = pluginA will still register that specific plugin through MultiPoint. This follows the same behavior as all other extension point configurations.

PodTopologySpreadConstraintsDefaulting

(Alias of string)

Appears in:

PodTopologySpreadConstraintsDefaulting defines how to set default constraints for the PodTopologySpread plugin.

RequestedToCapacityRatioParam

Appears in:

RequestedToCapacityRatioParam define RequestedToCapacityRatio parameters

FieldDescription
shape [Required]
[]UtilizationShapePoint
Shape is a list of points defining the scoring function shape.

ResourceSpec

Appears in:

ResourceSpec represents a single resource.

FieldDescription
name [Required]
string
Name of the resource.
weight [Required]
int64
Weight of the resource.

ScoringStrategy

Appears in:

ScoringStrategy define ScoringStrategyType for node resource plugin

FieldDescription
type [Required]
ScoringStrategyType
Type selects which strategy to run.
resources [Required]
[]ResourceSpec
Resources to consider when scoring. The default resource set includes "cpu" and "memory" with an equal weight. Allowed weights go from 1 to 100. Weight defaults to 1 if not specified or explicitly set to 0.
requestedToCapacityRatio [Required]
RequestedToCapacityRatioParam
Arguments specific to RequestedToCapacityRatio strategy.

ScoringStrategyType

(Alias of string)

Appears in:

ScoringStrategyType the type of scoring strategy used in NodeResourcesFit plugin.

UtilizationShapePoint

Appears in:

UtilizationShapePoint represents single point of priority function shape.

FieldDescription
utilization [Required]
int32
Utilization (x axis). Valid values are 0 to 100. Fully utilized node maps to 100.
score [Required]
int32
Score assigned to given utilization (y axis). Valid values are 0 to 10.

ClientConnectionConfiguration

Appears in:

ClientConnectionConfiguration contains details for constructing a client.

FieldDescription
kubeconfig [Required]
string
kubeconfig is the path to a KubeConfig file.
acceptContentTypes [Required]
string
acceptContentTypes defines the Accept header sent by clients when connecting to a server, overriding the default value of 'application/json'. This field will control all connections to the server used by a particular client.
contentType [Required]
string
contentType is the content type used when sending data to the server from this client.
qps [Required]
float32
qps controls the number of queries per second allowed for this connection.
burst [Required]
int32
burst allows extra queries to accumulate when a client is exceeding its rate.

DebuggingConfiguration

Appears in:

DebuggingConfiguration holds configuration for Debugging related features.

FieldDescription
enableProfiling [Required]
bool
enableProfiling enables profiling via web interface host:port/debug/pprof/
enableContentionProfiling [Required]
bool
enableContentionProfiling enables lock contention profiling, if enableProfiling is true.

FormatOptions

Appears in:

FormatOptions contains options for the different logging formats.

FieldDescription
json [Required]
JSONOptions
[Experimental] JSON contains options for logging format "json".

JSONOptions

Appears in:

JSONOptions contains options for logging format "json".

FieldDescription
splitStream [Required]
bool
[Experimental] SplitStream redirects error messages to stderr while info messages go to stdout, with buffering. The default is to write both to stdout, without buffering.
infoBufferSize [Required]
k8s.io/apimachinery/pkg/api/resource.QuantityValue
[Experimental] InfoBufferSize sets the size of the info stream when using split streams. The default is zero, which disables buffering.

LeaderElectionConfiguration

Appears in:

LeaderElectionConfiguration defines the configuration of leader election clients for components that can run with leader election enabled.

FieldDescription
leaderElect [Required]
bool
leaderElect enables a leader election client to gain leadership before executing the main loop. Enable this when running replicated components for high availability.
leaseDuration [Required]
meta/v1.Duration
leaseDuration is the duration that non-leader candidates will wait after observing a leadership renewal until attempting to acquire leadership of a led but unrenewed leader slot. This is effectively the maximum duration that a leader can be stopped before it is replaced by another candidate. This is only applicable if leader election is enabled.
renewDeadline [Required]
meta/v1.Duration
renewDeadline is the interval between attempts by the acting master to renew a leadership slot before it stops leading. This must be less than or equal to the lease duration. This is only applicable if leader election is enabled.
retryPeriod [Required]
meta/v1.Duration
retryPeriod is the duration the clients should wait between attempting acquisition and renewal of a leadership. This is only applicable if leader election is enabled.
resourceLock [Required]
string
resourceLock indicates the resource object type that will be used to lock during leader election cycles.
resourceName [Required]
string
resourceName indicates the name of resource object that will be used to lock during leader election cycles.
resourceNamespace [Required]
string
resourceName indicates the namespace of resource object that will be used to lock during leader election cycles.

LoggingConfiguration

Appears in:

LoggingConfiguration contains logging options Refer Logs Options for more information.

FieldDescription
format [Required]
string
Format Flag specifies the structure of log messages. default value of format is `text`
flushFrequency [Required]
time.Duration
Maximum number of seconds between log flushes. Ignored if the selected logging backend writes log messages without buffering.
verbosity [Required]
uint32
Verbosity is the threshold that determines which log messages are logged. Default is zero which logs only the most important messages. Higher values enable additional messages. Error messages are always logged.
vmodule [Required]
VModuleConfiguration
VModule overrides the verbosity threshold for individual files. Only supported for "text" log format.
sanitization [Required]
bool
[Experimental] When enabled prevents logging of fields tagged as sensitive (passwords, keys, tokens). Runtime log sanitization may introduce significant computation overhead and therefore should not be enabled in production.`)
options [Required]
FormatOptions
[Experimental] Options holds additional parameters that are specific to the different logging formats. Only the options for the selected format get used, but all of them get validated.

VModuleConfiguration

(Alias of []k8s.io/component-base/config/v1alpha1.VModuleItem)

Appears in:

VModuleConfiguration is a collection of individual file names or patterns and the corresponding verbosity threshold.

10 - kubeadm Configuration (v1beta2)

Overview

Package v1beta2 defines the v1beta2 version of the kubeadm configuration file format. This version improves on the v1beta1 format by fixing some minor issues and adding a few new fields.

A list of changes since v1beta1:

  • "certificateKey" field is added to InitConfiguration and JoinConfiguration.
  • "ignorePreflightErrors" field is added to the NodeRegistrationOptions.
  • The JSON "omitempty" tag is used in a more places where appropriate.
  • The JSON "omitempty" tag of the "taints" field (inside NodeRegistrationOptions) is removed.

See the Kubernetes 1.15 changelog for further details.

Migration from old kubeadm config versions

Please convert your v1beta1 configuration files to v1beta2 using the "kubeadm config migrate" command of kubeadm v1.15.x (conversion from older releases of kubeadm config files requires older release of kubeadm as well e.g.

  • kubeadm v1.11 should be used to migrate v1alpha1 to v1alpha2; kubeadm v1.12 should be used to translate v1alpha2 to v1alpha3;
  • kubeadm v1.13 or v1.14 should be used to translate v1alpha3 to v1beta1)

Nevertheless, kubeadm v1.15.x will support reading from v1beta1 version of the kubeadm config file format.

Basics

The preferred way to configure kubeadm is to pass an YAML configuration file with the --config option. Some of the configuration options defined in the kubeadm config file are also available as command line flags, but only the most common/simple use case are supported with this approach.

A kubeadm config file could contain multiple configuration types separated using three dashes (---).

kubeadm supports the following configuration types:

apiVersion: kubeadm.k8s.io/v1beta2
kind: InitConfiguration

apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration

apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration

apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration

apiVersion: kubeadm.k8s.io/v1beta2
kind: JoinConfiguration

To print the defaults for "init" and "join" actions use the following commands:

kubeadm config print init-defaults
kubeadm config print join-defaults

The list of configuration types that must be included in a configuration file depends by the action you are performing (init or join) and by the configuration options you are going to use (defaults or advanced customization).

If some configuration types are not provided, or provided only partially, kubeadm will use default values; defaults provided by kubeadm includes also enforcing consistency of values across components when required (e.g. --cluster-cidr flag on controller manager and clusterCIDR on kube-proxy).

Users are always allowed to override default values, with the only exception of a small subset of setting with relevance for security (e.g. enforce authorization-mode Node and RBAC on API server)

If the user provides a configuration types that is not expected for the action you are performing, kubeadm will ignore those types and print a warning.

Kubeadm init configuration types

When executing kubeadm init with the --config option, the following configuration types could be used: InitConfiguration, ClusterConfiguration, KubeProxyConfiguration, KubeletConfiguration, but only one between InitConfiguration and ClusterConfiguration is mandatory.

apiVersion: kubeadm.k8s.io/v1beta2
kind: InitConfiguration
bootstrapTokens:
  ...
nodeRegistration:
  ...

The InitConfiguration type should be used to configure runtime settings, that in case of kubeadm init are the configuration of the bootstrap token and all the setting which are specific to the node where kubeadm is executed, including:

  • nodeRegistration, that holds fields that relate to registering the new node to the cluster; use it to customize the node name, the CRI socket to use or any other settings that should apply to this node only (e.g. the node ip).

  • apiServer, that represents the endpoint of the instance of the API server to be deployed on this node; use it e.g. to customize the API server advertise address.

    apiVersion: kubeadm.k8s.io/v1beta2
    kind: ClusterConfiguration
    networking:
        ...
    etcd:
        ...
    apiServer:
      extraArgs:
        ...
      extraVolumes:
        ...
    ...
    

The ClusterConfiguration type should be used to configure cluster-wide settings, including settings for:

  • Networking, that holds configuration for the networking topology of the cluster; use it e.g. to customize pod subnet or services subnet.

  • Etcd configurations; use it e.g. to customize the local etcd or to configure the API server for using an external etcd cluster.

  • kube-apiserver, kube-scheduler, kube-controller-manager configurations; use it to customize control-plane components by adding customized setting or overriding kubeadm default settings.

    apiVersion: kubeproxy.config.k8s.io/v1alpha1
    kind: KubeProxyConfiguration
      ...
    

The KubeProxyConfiguration type should be used to change the configuration passed to kube-proxy instances deployed in the cluster. If this object is not provided or provided only partially, kubeadm applies defaults.

See https://kubernetes.io/docs/reference/command-line-tools-reference/kube-proxy/ or https://godoc.org/k8s.io/kube-proxy/config/v1alpha1#KubeProxyConfiguration for kube proxy official documentation.

apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
  ...

The KubeletConfiguration type should be used to change the configurations that will be passed to all kubelet instances deployed in the cluster. If this object is not provided or provided only partially, kubeadm applies defaults.

See https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/ or https://godoc.org/k8s.io/kubelet/config/v1beta1#KubeletConfiguration for kubelet official documentation.

Here is a fully populated example of a single YAML file containing multiple configuration types to be used during a kubeadm init run.

apiVersion: kubeadm.k8s.io/v1beta2
kind: InitConfiguration
bootstrapTokens:
  - token: "9a08jv.c0izixklcxtmnze7"
    description: "kubeadm bootstrap token"
    ttl: "24h"
  - token: "783bde.3f89s0fje9f38fhf"
    description: "another bootstrap token"
    usages:
      - authentication
      - signing
    groups:
      - system:bootstrappers:kubeadm:default-node-token
nodeRegistration:
  name: "ec2-10-100-0-1"
  criSocket: "/var/run/dockershim.sock"
  taints:
    - key: "kubeadmNode"
      value: "master"
      effect: "NoSchedule"
  kubeletExtraArgs:
    v: 4
  ignorePreflightErrors:
    - IsPrivilegedUser
localAPIEndpoint:
  advertiseAddress: "10.100.0.1"
  bindPort: 6443
certificateKey: "e6a2eb8581237ab72a4f494f30285ec12a9694d750b9785706a83bfcbbbd2204"
---
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
etcd:
  # one of local or external
local:
  imageRepository: "k8s.gcr.io"
  imageTag: "3.2.24"
  dataDir: "/var/lib/etcd"
  extraArgs:
    listen-client-urls: "http://10.100.0.1:2379"
  serverCertSANs:
    -  "ec2-10-100-0-1.compute-1.amazonaws.com"
  peerCertSANs:
    - "10.100.0.1"
# external:
#   endpoints:
#     - "10.100.0.1:2379"
#     - "10.100.0.2:2379"
#   caFile: "/etcd/kubernetes/pki/etcd/etcd-ca.crt"
#   certFile: "/etcd/kubernetes/pki/etcd/etcd.crt"
#   keyFile: "/etcd/kubernetes/pki/etcd/etcd.key"
networking:
  serviceSubnet: "10.96.0.0/16"
  podSubnet: "10.244.0.0/24"
  dnsDomain: "cluster.local"
kubernetesVersion: "v1.12.0"
controlPlaneEndpoint: "10.100.0.1:6443"
apiServer:
  extraArgs:
    authorization-mode: "Node,RBAC"
  extraVolumes:
    - name: "some-volume"
      hostPath: "/etc/some-path"
      mountPath: "/etc/some-pod-path"
      readOnly: false
      pathType: File
  certSANs:
    - "10.100.1.1"
    - "ec2-10-100-0-1.compute-1.amazonaws.com"
  timeoutForControlPlane: 4m0s
controllerManager:
  extraArgs:
    "node-cidr-mask-size": "20"
  extraVolumes:
    - name: "some-volume"
      hostPath: "/etc/some-path"
      mountPath: "/etc/some-pod-path"
      readOnly: false
      pathType: File
scheduler:
  extraArgs:
    address: "10.100.0.1"
  extraVolumes:
    - name: "some-volume"
      hostPath: "/etc/some-path"
      mountPath: "/etc/some-pod-path"
      readOnly: false
      pathType: File
certificatesDir: "/etc/kubernetes/pki"
imageRepository: "k8s.gcr.io"
useHyperKubeImage: false
clusterName: "example-cluster"
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
# kubelet specific options here
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
# kube-proxy specific options here

Kubeadm join configuration types

When executing kubeadm join with the --config option, the JoinConfiguration type should be provided.

apiVersion: kubeadm.k8s.io/v1beta2
kind: JoinConfiguration
  ...

The JoinConfiguration type should be used to configure runtime settings, that in case of kubeadm join are the discovery method used for accessing the cluster info and all the setting which are specific to the node where kubeadm is executed, including:

  • NodeRegistration, that holds fields that relate to registering the new node to the cluster; use it to customize the node name, the CRI socket to use or any other settings that should apply to this node only (e.g. the node IP).

  • APIEndpoint, that represents the endpoint of the instance of the API server to be eventually deployed on this node.

Resource Types

ClusterConfiguration

ClusterConfiguration contains cluster-wide configuration for a kubeadm cluster

FieldDescription
apiVersion
string
kubeadm.k8s.io/v1beta2
kind
string
ClusterConfiguration
etcd [Required]
Etcd
`etcd` holds configuration for etcd.
networking [Required]
Networking
`networking` holds configuration for the networking topology of the cluster.
kubernetesVersion [Required]
string
`kubernetesVersion` is the target version of the control plane.
controlPlaneEndpoint [Required]
string
`controlPlaneEndpoint` sets a stable IP address or DNS name for the control plane; it can be a valid IP address or a RFC-1123 DNS subdomain, both with optional TCP port. In case the `controlPlaneEndpoint` is not specified, the `advertiseAddress` + `bindPort` are used; in case the `controlPlaneEndpoint` is specified but without a TCP port, the `bindPort` is used. Possible usages are:
  • In a cluster with more than one control plane instances, this field should be assigned the address of the external load balancer in front of the control plane instances.
  • In environments with enforced node recycling, the controlPlaneEndpoint could be used for assigning a stable DNS to the control plane.
apiServer [Required]
APIServer
`apiServer` contains extra settings for the API server.
controllerManager [Required]
ControlPlaneComponent
`controllerManager` contains extra settings for the controller manager.
scheduler [Required]
ControlPlaneComponent
`scheduler` contains extra settings for the scheduler.
dns [Required]
DNS
`dns` defines the options for the DNS add-on installed in the cluster.
certificatesDir [Required]
string
`certificatesDir` specifies where to store or look for all required certificates.
imageRepository [Required]
string
`mageRepository` sets the container registry to pull images from. If empty, `k8s.gcr.io` will be used by default; in case of kubernetes version is a CI build (kubernetes version starts with `ci/`) `gcr.io/k8s-staging-ci-images` is used as a default for control plane components and for kube-proxy, while `k8s.gcr.io` will be used for all the other images.
useHyperKubeImage [Required]
bool
`useHyperKubeImage` controls if hyperkube should be used for Kubernetes components instead of their respective separate images. DEPRECATED: As `hyperkube` is itself deprecated, this fields is too. It will be removed in future kubeadm config versions, kubeadm will print multiple warnings when this set to true, and at some point it may become ignored.
featureGates [Required]
map[string]bool
`featureGates` contains the feature gates enabled by the user.
clusterName [Required]
string
The cluster name.

ClusterStatus

ClusterStatus contains the cluster status. The ClusterStatus will be stored in the kubeadm-config ConfigMap in the cluster, and then updated by kubeadm when additional control plane instance joins or leaves the cluster.

FieldDescription
apiVersion
string
kubeadm.k8s.io/v1beta2
kind
string
ClusterStatus
apiEndpoints [Required]
map[string]github.com/tengqm/kubeconfig/config/kubeadm/v1beta2.APIEndpoint
`apiEndpoints` currently available in the cluster, one for each control plane/API server instance. The key of the map is the IP of the host's default interface.

InitConfiguration

InitConfiguration contains a list of elements that is specific "kubeadm init"-only runtime information.

FieldDescription
apiVersion
string
kubeadm.k8s.io/v1beta2
kind
string
InitConfiguration
bootstrapTokens [Required]
[]BootstrapToken
`bootstrapTokens` is respected at `kubeadm init` time and describes a set of bootstrap tokens to create. This information IS NOT uploaded to the kubeadm cluster ConfigMap, partly because of its sensitive nature.
nodeRegistration [Required]
NodeRegistrationOptions
`nodeRegistration` holds fields that relate to registering the new control-plane node to the cluster.
localAPIEndpoint [Required]
APIEndpoint
`localAPIEndpoint` represents the endpoint of the API server instance that's deployed on this control plane node. In HA setups, this differs from `ClusterConfiguration.controlPlaneEndpoint` in the sense that ControlPlaneEndpoint is the global endpoint for the cluster, which then load-balances the requests to each individual API server. This configuration object lets you customize what IP/DNS name and port the local API server advertises it's accessible on. By default, kubeadm tries to auto-detect the IP of the default interface and use that, but in case that process fails you may set the desired value here.
certificateKey [Required]
string
`certificateKey` sets the key with which certificates and keys are encrypted prior to being uploaded in a secret in the cluster during the `uploadcerts init` phase.

JoinConfiguration

JoinConfiguration contains elements describing a particular node.

FieldDescription
apiVersion
string
kubeadm.k8s.io/v1beta2
kind
string
JoinConfiguration
nodeRegistration [Required]
NodeRegistrationOptions
`nodeRegistration` holds fields that relate to registering the new control-plane node to the cluster
caCertPath [Required]
string
`caCertPath` is the path to the SSL certificate authority used to secure comunications between a node and the control-plane. Defaults to "/etc/kubernetes/pki/ca.crt".
discovery [Required]
Discovery
`discovery` specifies the options for the kubelet to use during the TLS bootstrap process.
controlPlane [Required]
JoinControlPlane
`controlPlane` defines the additional control plane instance to be deployed on the joining node. If nil, no additional control plane instance will be deployed.

APIEndpoint

Appears in:

APIEndpoint struct contains elements of API server instance deployed on a node.

FieldDescription
advertiseAddress [Required]
string
`advertiseAddress` sets the IP address for the API server to advertise.
bindPort [Required]
int32
`bindPort` sets the secure port for the API Server to bind to. Defaults to 6443.

APIServer

Appears in:

APIServer holds settings necessary for API server deployments in the cluster.

FieldDescription
ControlPlaneComponent [Required]
ControlPlaneComponent
(Members of ControlPlaneComponent are embedded into this type.) No description provided.
certSANs [Required]
[]string
`certSANs` sets extra Subject Alternative Names (SANs) for the API Server signing certificate.
timeoutForControlPlane [Required]
meta/v1.Duration
`timeoutForControlPlane` controls the timeout that we wait for the API server to appear.

BootstrapToken

Appears in:

BootstrapToken describes one bootstrap token, stored as a Secret in the cluster

FieldDescription
token [Required]
BootstrapTokenString
`token` is used for establishing bidirectional trust between nodes and control-planes. Used for joining nodes in the cluster.
description [Required]
string
`description` sets a human-friendly message why this token exists and what it's used for, so other administrators can know its purpose.
ttl [Required]
meta/v1.Duration
`ttl` defines the time to live for this token. Defaults to '24h'. `expires` and `ttl` are mutually exclusive.
expires [Required]
meta/v1.Time
`expires` specifies the timestamp when this token expires. Defaults to being set dynamically at runtime based on the `ttl`. `expires` and `ttl` are mutually exclusive.
usages [Required]
[]string
`usages` describes the ways in which this token can be used. Can by default be used for establishing bidirectional trust, but that can be changed here.
groups [Required]
[]string
`groups` specifies the extra groups that this token will authenticate as when/if used for authentication.

BootstrapTokenDiscovery

Appears in:

BootstrapTokenDiscovery is used to set the options for bootstrap token based discovery

FieldDescription
token [Required]
string
`token` is a token used to validate cluster information fetched from the control-plane.
apiServerEndpoint [Required]
string
`apiServerEndpoint` is an IP or domain name to the API server from which information will be fetched.
caCertHashes [Required]
[]string
`caCertHashes` specifies a set of public key pins to verify when token-based discovery is used. The root CA found during discovery must match one of these values. Specifying an empty set disables root CA pinning, which can be unsafe. Each hash is specified as ":", where the only currently supported type is "sha256". This is a hex-encoded SHA-256 hash of the Subject Public Key Info (SPKI) object in DER-encoded ASN.1. These hashes can be calculated using, for example, OpenSSL.
unsafeSkipCAVerification [Required]
bool
`unsafeSkipCAVerification` allows token-based discovery without CA verification via `caCertHashes`. This can weaken the security of kubeadm since other nodes can impersonate the control-plane.

BootstrapTokenString

Appears in:

BootstrapTokenString is a token of the format abcdef.abcdef0123456789 that is used for both validation of the practically of the API server from a joining node's point of view and as an authentication method for the node in the bootstrap phase of "kubeadm join". This token is and should be short-lived

FieldDescription
- [Required]
string
No description provided.
- [Required]
string
No description provided.

ControlPlaneComponent

Appears in:

ControlPlaneComponent holds settings common to control plane component of the cluster

FieldDescription
extraArgs [Required]
map[string]string
`extraArgs` is an extra set of flags to pass to a control plane component. A key in this map is the flag name as it appears on the command line except without leading dash(es).
extraVolumes [Required]
[]HostPathMount
`extraVolumes` is an extra set of host volumes mounted to the control plane component.

DNS

Appears in:

DNS defines the DNS addon that should be used in the cluster

FieldDescription
type [Required]
DNSAddOnType
`type` defines the DNS add-on to be used.
ImageMeta [Required]
ImageMeta
(Members of ImageMeta are embedded into this type.) ImageMeta allows to customize the image used for the DNS component

DNSAddOnType

(Alias of string)

Appears in:

DNSAddOnType defines string identifying DNS add-on types.

Discovery

Appears in:

Discovery specifies the options for the kubelet to use during the TLS Bootstrap process

FieldDescription
bootstrapToken [Required]
BootstrapTokenDiscovery
`bootstrapToken` is used to set the options for bootstrap token based discovery. `bootstrapToken` and `file` are mutually exclusive.
file [Required]
FileDiscovery
`file` is used to specify a file or URL to a kubeconfig file from which to load cluster information. `bootstrapToken` and `file` are mutually exclusive.
tlsBootstrapToken [Required]
string
`tlsBootstrapToken` is a token used for TLS bootstrapping. If `bootstrapToken` is set, this field is defaulted to `.bootstrapToken.token, but can be overridden. If `file` is set, this field ∗∗must be set∗∗ in case the KubeConfigFile does not contain any other authentication information.
timeout [Required]
meta/v1.Duration
`timeout` modifies the discovery timeout.

Etcd

Appears in:

Etcd contains elements describing Etcd configuration.

FieldDescription
local [Required]
LocalEtcd
`local` provides configuration knobs for configuring the local etcd instance. `local` and `external` are mutually exclusive.
external [Required]
ExternalEtcd
`external` describes how to connect to an external etcd cluster. `local` and `external` are mutually exclusive.

ExternalEtcd

Appears in:

ExternalEtcd describes an external etcd cluster. Kubeadm has no knowledge of where certificate files live and they must be supplied.

FieldDescription
endpoints [Required]
[]string
`endpoints` of etcd members.
caFile [Required]
string
`caFile` is an SSL Certificate Authority (CA) file used to secure etcd communication. Required if using a TLS connection.
certFile [Required]
string
`certFile` is an SSL certification file used to secure etcd communication. Required if using a TLS connection.
keyFile [Required]
string
`keyFile` is an SSL key file used to secure etcd communication. Required if using a TLS connection.

FileDiscovery

Appears in:

FileDiscovery is used to specify a file or URL to a kubeconfig file from which to load cluster information

FieldDescription
kubeConfigPath [Required]
string
`kubeConfigPath` is used to specify the actual file path or URL to the kubeconfig file from which to load cluster information.

HostPathMount

Appears in:

HostPathMount contains elements describing volumes that are mounted from the host.

FieldDescription
name [Required]
string
`name` of the volume inside the Pod template.
hostPath [Required]
string
`hostPath` is the path in the host that will be mounted inside the Pod.
mountPath [Required]
string
`mountPath`is the path inside the Pod where hostPath volume will be mounted.
readOnly [Required]
bool
`readOnly` controls write access to the volume.
pathType [Required]
core/v1.HostPathType
`pathType` is the type of the HostPath.

ImageMeta

Appears in:

ImageMeta allows to customize the image used for components that are not originated from the Kubernetes/Kubernetes release process

FieldDescription
imageRepository [Required]
string
`mageRepository` sets the container registry to pull images from. If not set, the `imageRepository` defined in ClusterConfiguration will be used.
imageTag [Required]
string
`imageTag` allows for specifying a tag for the image. In case this value is set, kubeadm does not change automatically the version of the above components during upgrades.

JoinControlPlane

Appears in:

JoinControlPlane contains elements describing an additional control plane instance to be deployed on the joining node.

FieldDescription
localAPIEndpoint [Required]
APIEndpoint
`localAPIEndpoint` represents the endpoint of the API server instance to be deployed on this node.
certificateKey [Required]
string
`certificateKey` is the key that is used for decryption of certificates after they are downloaded from the secret upon joining a new control plane node. The corresponding encryption key is in the InitConfiguration.

LocalEtcd

Appears in:

LocalEtcd describes that kubeadm should run an etcd cluster locally.

FieldDescription
ImageMeta [Required]
ImageMeta
(Members of ImageMeta are embedded into this type.) ImageMeta allows to customize the container used for etcd.
dataDir [Required]
string
`dataDir` is the directory etcd will place its data. Defaults to "/var/lib/etcd".
extraArgs [Required]
map[string]string
`extraArgs` are extra arguments provided to the etcd binary when run inside a static pod. A key in this map is the flag name as it appears on the command line except without leading dash(es).
serverCertSANs [Required]
[]string
`serverCertSANs` sets extra Subject Alternative Names (SANs) for the etcd server signing certificate.
peerCertSANs [Required]
[]string
`peerCertSANs` sets extra Subject Alternative Names (SANs) for the etcd peer signing certificate.

Networking

Appears in:

Networking contains elements describing cluster's networking configuration

FieldDescription
serviceSubnet [Required]
string
`serviceSubnet` is the subnet used by kubernetes Services. Defaults to "10.96.0.0/12".
podSubnet [Required]
string
`podSubnet` is the subnet used by Pods.
dnsDomain [Required]
string
`dnsDomain` is the DNS domain used by kubernetes Services. Defaults to "cluster.local".

NodeRegistrationOptions

Appears in:

NodeRegistrationOptions holds fields that relate to registering a new control-plane or node to the cluster, either via "kubeadm init" or "kubeadm join".

FieldDescription
name [Required]
string
`name` is the `.Metadata.Name` field of the Node API object that will be created in this `kubeadm init` or `kubeadm join` operation. This field is also used in the `CommonName` field of the kubelet's client certificate to the API server. Defaults to the hostname of the node if not provided.
criSocket [Required]
string
`criSocket is used to retrieve container runtime information. This information will be annotated to the Node API object, for later re-use.
taints [Required]
[]core/v1.Taint
`taints` specifies the taints the Node API object should be registered with. If this field is unset, i.e. nil, in the `kubeadm init` process it will be defaulted to `'node-role.kubernetes.io/master=""'`. If you don't want to taint your control-plane node, set this field to an empty list, i.e. `taints: []` in the YAML file. This field is solely used for Node registration.
kubeletExtraArgs [Required]
map[string]string
`kubeletExtraArgs` passes through extra arguments to the kubelet. The arguments here are passed to the kubelet command line via the environment file kubeadm writes at runtime for the kubelet to source. This overrides the generic base-level configuration in the 'kubelet-config-1.X' ConfigMap. Flags have higher priority when parsing. These values are local and specific to the node kubeadm is executing on. A key in this map is the flag name as it appears on the command line except without leading dash(es).
ignorePreflightErrors [Required]
[]string
`ignorePreflightErrors` provides a list of pre-flight errors to be ignored when the current node is registered.

11 - kubeadm Configuration (v1beta3)

Overview

Package v1beta3 defines the v1beta3 version of the kubeadm configuration file format. This version improves on the v1beta2 format by fixing some minor issues and adding a few new fields.

A list of changes since v1beta2:

  • The deprecated "ClusterConfiguration.useHyperKubeImage" field has been removed. Kubeadm no longer supports the hyperkube image.
  • The "ClusterConfiguration.DNS.Type" field has been removed since CoreDNS is the only supported DNS server type by kubeadm.
  • Include "datapolicy" tags on the fields that hold secrets. This would result in the field values to be omitted when API structures are printed with klog.
  • Add "InitConfiguration.SkipPhases", "JoinConfiguration.SkipPhases" to allow skipping a list of phases during kubeadm init/join command execution.
  • Add "InitConfiguration.NodeRegistration.ImagePullPolicy" and "JoinConfiguration.NodeRegistration.ImagePullPolicy" to allow specifying the images pull policy during kubeadm "init" and "join". The value must be one of "Always", "Never" or "IfNotPresent". "IfNotPresent" is the default, which has been the existing behavior prior to this addition.
  • Add "InitConfiguration.Patches.Directory", "JoinConfiguration.Patches.Directory" to allow the user to configure a directory from which to take patches for components deployed by kubeadm.
  • Move the BootstrapToken∗ API and related utilities out of the "kubeadm" API group to a new group "bootstraptoken". The kubeadm API version v1beta3 no longer contains the BootstrapToken∗ structures.

Migration from old kubeadm config versions

  • kubeadm v1.15.x and newer can be used to migrate from v1beta1 to v1beta2.
  • kubeadm v1.22.x and newer no longer support v1beta1 and older APIs, but can be used to migrate v1beta2 to v1beta3.

Basics

The preferred way to configure kubeadm is to pass an YAML configuration file with the --config option. Some of the configuration options defined in the kubeadm config file are also available as command line flags, but only the most common/simple use case are supported with this approach.

A kubeadm config file could contain multiple configuration types separated using three dashes (---).

kubeadm supports the following configuration types:

apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration

apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration

apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration

apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration

apiVersion: kubeadm.k8s.io/v1beta3
kind: JoinConfiguration

To print the defaults for "init" and "join" actions use the following commands:

kubeadm config print init-defaults
kubeadm config print join-defaults

The list of configuration types that must be included in a configuration file depends by the action you are performing (init or join) and by the configuration options you are going to use (defaults or advanced customization).

If some configuration types are not provided, or provided only partially, kubeadm will use default values; defaults provided by kubeadm includes also enforcing consistency of values across components when required (e.g. --cluster-cidr flag on controller manager and clusterCIDR on kube-proxy).

Users are always allowed to override default values, with the only exception of a small subset of setting with relevance for security (e.g. enforce authorization-mode Node and RBAC on api server)

If the user provides a configuration types that is not expected for the action you are performing, kubeadm will ignore those types and print a warning.

Kubeadm init configuration types

When executing kubeadm init with the --config option, the following configuration types could be used: InitConfiguration, ClusterConfiguration, KubeProxyConfiguration, KubeletConfiguration, but only one between InitConfiguration and ClusterConfiguration is mandatory.

apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
bootstrapTokens:
  ...
nodeRegistration:
  ...

The InitConfiguration type should be used to configure runtime settings, that in case of kubeadm init are the configuration of the bootstrap token and all the setting which are specific to the node where kubeadm is executed, including:

  • NodeRegistration, that holds fields that relate to registering the new node to the cluster; use it to customize the node name, the CRI socket to use or any other settings that should apply to this node only (e.g. the node ip).

  • LocalAPIEndpoint, that represents the endpoint of the instance of the API server to be deployed on this node; use it e.g. to customize the API server advertise address.

    apiVersion: kubeadm.k8s.io/v1beta3
    kind: ClusterConfiguration
    networking:
      ...
    etcd:
      ...
    apiServer:
      extraArgs:
        ...
      extraVolumes:
        ...
    ...
    

The ClusterConfiguration type should be used to configure cluster-wide settings, including settings for:

  • Networking, that holds configuration for the networking topology of the cluster; use it e.g. to customize Pod subnet or services subnet.

  • Etcd configurations; use it e.g. to customize the local etcd or to configure the API server for using an external etcd cluster.

  • kube-apiserver, kube-scheduler, kube-controller-manager configurations; use it to customize control-plane components by adding customized setting or overriding kubeadm default settings.

    apiVersion: kubeproxy.config.k8s.io/v1alpha1
    kind: KubeProxyConfiguration
      ...
    

The KubeProxyConfiguration type should be used to change the configuration passed to kube-proxy instances deployed in the cluster. If this object is not provided or provided only partially, kubeadm applies defaults.

See https://kubernetes.io/docs/reference/command-line-tools-reference/kube-proxy/ or https://godoc.org/k8s.io/kube-proxy/config/v1alpha1#KubeProxyConfiguration for kube-proxy official documentation.

apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
  ...

The KubeletConfiguration type should be used to change the configurations that will be passed to all kubelet instances deployed in the cluster. If this object is not provided or provided only partially, kubeadm applies defaults.

See https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/ or https://godoc.org/k8s.io/kubelet/config/v1beta1#KubeletConfiguration for kubelet official documentation.

Here is a fully populated example of a single YAML file containing multiple configuration types to be used during a kubeadm init run.

apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
bootstrapTokens:
- token: "9a08jv.c0izixklcxtmnze7"
  description: "kubeadm bootstrap token"
  ttl: "24h"
- token: "783bde.3f89s0fje9f38fhf"
  description: "another bootstrap token"
  usages:
  - authentication
  - signing
  groups:
  - system:bootstrappers:kubeadm:default-node-token
nodeRegistration:
  name: "ec2-10-100-0-1"
  criSocket: "/var/run/dockershim.sock"
  taints:
  - key: "kubeadmNode"
    value: "master"
    effect: "NoSchedule"
  kubeletExtraArgs:
    v: 4
ignorePreflightErrors:
- IsPrivilegedUser
   imagePullPolicy: "IfNotPresent"
localAPIEndpoint:
  advertiseAddress: "10.100.0.1"
  bindPort: 6443
certificateKey: "e6a2eb8581237ab72a4f494f30285ec12a9694d750b9785706a83bfcbbbd2204"
 skipPhases:
 - addon/kube-proxy
---
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
etcd:
  # one of local or external
  local:
    imageRepository: "k8s.gcr.io"
    imageTag: "3.2.24"
    dataDir: "/var/lib/etcd"
    extraArgs:
      listen-client-urls: "http://10.100.0.1:2379"
    serverCertSANs:
    -  "ec2-10-100-0-1.compute-1.amazonaws.com"
    peerCertSANs:
    - "10.100.0.1"
  # external:
    # endpoints:
    # - "10.100.0.1:2379"
    # - "10.100.0.2:2379"
    # caFile: "/etcd/kubernetes/pki/etcd/etcd-ca.crt"
    # certFile: "/etcd/kubernetes/pki/etcd/etcd.crt"
    # keyFile: "/etcd/kubernetes/pki/etcd/etcd.key"
networking:
  serviceSubnet: "10.96.0.0/16"
  podSubnet: "10.244.0.0/24"
  dnsDomain: "cluster.local"
kubernetesVersion: "v1.21.0"
controlPlaneEndpoint: "10.100.0.1:6443"
apiServer:
  extraArgs:
    authorization-mode: "Node,RBAC"
  extraVolumes:
  - name: "some-volume"
    hostPath: "/etc/some-path"
    mountPath: "/etc/some-pod-path"
    readOnly: false
    pathType: File
  certSANs:
  - "10.100.1.1"
  - "ec2-10-100-0-1.compute-1.amazonaws.com"
  timeoutForControlPlane: 4m0s
controllerManager:
  extraArgs:
    "node-cidr-mask-size": "20"
  extraVolumes:
  - name: "some-volume"
    hostPath: "/etc/some-path"
    mountPath: "/etc/some-pod-path"
    readOnly: false
    pathType: File
scheduler:
  extraArgs:
    address: "10.100.0.1"
  extraVolumes:
  - name: "some-volume"
    hostPath: "/etc/some-path"
    mountPath: "/etc/some-pod-path"
    readOnly: false
    pathType: File
certificatesDir: "/etc/kubernetes/pki"
imageRepository: "k8s.gcr.io"
clusterName: "example-cluster"
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
# kubelet specific options here
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
# kube-proxy specific options here

Kubeadm join configuration types

When executing kubeadm join with the --config option, the JoinConfiguration type should be provided.

apiVersion: kubeadm.k8s.io/v1beta3
kind: JoinConfiguration
  ...

The JoinConfiguration type should be used to configure runtime settings, that in case of kubeadm join are the discovery method used for accessing the cluster info and all the setting which are specific to the node where kubeadm is executed, including:

  • NodeRegistration, that holds fields that relate to registering the new node to the cluster; use it to customize the node name, the CRI socket to use or any other settings that should apply to this node only (e.g. the node ip).

  • APIEndpoint, that represents the endpoint of the instance of the API server to be eventually deployed on this node.

Resource Types

ClusterConfiguration

ClusterConfiguration contains cluster-wide configuration for a kubeadm cluster

FieldDescription
apiVersion
string
kubeadm.k8s.io/v1beta3
kind
string
ClusterConfiguration
etcd
Etcd
`etcd` holds the configuration for etcd.
networking
Networking
`networking` holds configuration for the networking topology of the cluster.
kubernetesVersion
string
`kubernetesVersion` is the target version of the control plane.
controlPlaneEndpoint
string
`controlPlaneEndpoint` sets a stable IP address or DNS name for the control plane. It can be a valid IP address or a RFC-1123 DNS subdomain, both with optional TCP port. In case the `controlPlaneEndpoint` is not specified, the `advertiseAddress` + `bindPort` are used; in case the `controlPlaneEndpoint` is specified but without a TCP port, the `bindPort` is used. Possible usages are:
  • In a cluster with more than one control plane instances, this field should be assigned the address of the external load balancer in front of the control plane instances.
  • In environments with enforced node recycling, the controlPlaneEndpoint could be used for assigning a stable DNS to the control plane.
apiServer
APIServer
`apiServer` contains extra settings for the API server.
controllerManager
ControlPlaneComponent
`controllerManager` contains extra settings for the controller manager.
scheduler
ControlPlaneComponent
`scheduler` contains extra settings for the scheduler.
dns
DNS
`dns` defines the options for the DNS add-on installed in the cluster.
certificatesDir
string
`certificatesDir` specifies where to store or look for all required certificates.
imageRepository
string
`imageRepository` sets the container registry to pull images from. If empty, `k8s.gcr.io` will be used by default. In case of kubernetes version is a CI build (kubernetes version starts with `ci/`) `gcr.io/k8s-staging-ci-images` will be used as a default for control plane components and for kube-proxy, while `k8s.gcr.io` will be used for all the other images.
featureGates
map[string]bool
`featureGates` contains the feature gates enabled by the user.
clusterName
string
The cluster name.

JoinConfiguration

JoinConfiguration contains elements describing a particular node.

FieldDescription
apiVersion
string
kubeadm.k8s.io/v1beta3
kind
string
JoinConfiguration
nodeRegistration
NodeRegistrationOptions
`nodeRegistration` holds fields that relate to registering the new control-plane node to the cluster.
caCertPath
string
`caCertPath` is the path to the SSL certificate authority used to secure comunications between a node and the control-plane. Defaults to "/etc/kubernetes/pki/ca.crt".
discovery [Required]
Discovery
`discovery` specifies the options for the kubelet to use during the TLS bootstrap process.
controlPlane
JoinControlPlane
`controlPlane` defines the additional control plane instance to be deployed on the joining node. If nil, no additional control plane instance will be deployed.
skipPhases
[]string
`skipPhases` is a list of phases to skip during command execution. The list of phases can be obtained with the `kubeadm join --help` command. The flag `--skip-phases` takes precedence over this field.
patches
Patches
`patches` contains options related to applying patches to components deployed by kubeadm during `kubeadm join`.

APIEndpoint

Appears in:

APIEndpoint struct contains elements of API server instance deployed on a node.

FieldDescription
advertiseAddress
string
`advertiseAddress` sets the IP address for the API server to advertise.
bindPort
int32
`bindPorti` sets the secure port for the API Server to bind to. Defaults to 6443.

APIServer

Appears in:

APIServer holds settings necessary for API server deployments in the cluster

FieldDescription
ControlPlaneComponent [Required]
ControlPlaneComponent
(Members of ControlPlaneComponent are embedded into this type.) No description provided.
certSANs
[]string
`certSANs` sets extra Subject Alternative Names (SANs) for the API Server signing certificate.
timeoutForControlPlane
meta/v1.Duration
`timeoutForControlPlane` controls the timeout that we wait for API server to appear.

BootstrapTokenDiscovery

Appears in:

BootstrapTokenDiscovery is used to set the options for bootstrap token based discovery

FieldDescription
token [Required]
string
`token` is a token used to validate cluster information fetched from the control-plane.
apiServerEndpoint
string
`apiServerEndpoint` is an IP or domain name to the API server from which information will be fetched.
caCertHashes
[]string
`caCertHashes` specifies a set of public key pins to verify when token-based discovery is used. The root CA found during discovery must match one of these values. Specifying an empty set disables root CA pinning, which can be unsafe. Each hash is specified as ":", where the only currently supported type is "sha256". This is a hex-encoded SHA-256 hash of the Subject Public Key Info (SPKI) object in DER-encoded ASN.1. These hashes can be calculated using, for example, OpenSSL.
unsafeSkipCAVerification
bool
`unsafeSkipCAVerification` allows token-based discovery without CA verification via `caCertHashes`. This can weaken the security of kubeadm since other nodes can impersonate the control-plane.

ControlPlaneComponent

Appears in:

ControlPlaneComponent holds settings common to control plane component of the cluster

FieldDescription
extraArgs
map[string]string
`extraArgs` is an extra set of flags to pass to the control plane component. A key in this map is the flag name as it appears on the command line except without leading dash(es).
extraVolumes
[]HostPathMount
`extraVolumes` is an extra set of host volumes, mounted to the control plane component.

DNS

Appears in:

DNS defines the DNS addon that should be used in the cluster

FieldDescription
ImageMeta [Required]
ImageMeta
(Members of ImageMeta are embedded into this type.) `imageMeta` allows to customize the image used for the DNS component.

Discovery

Appears in:

Discovery specifies the options for the kubelet to use during the TLS Bootstrap process.

FieldDescription
bootstrapToken
BootstrapTokenDiscovery
`bootstrapToken` is used to set the options for bootstrap token based discovery. `bootstrapToken` and `file` are mutually exclusive.
file
FileDiscovery
`file` is used to specify a file or URL to a kubeconfig file from which to load cluster information. `bootstrapToken` and `file` are mutually exclusive.
tlsBootstrapToken
string
`tlsBootstrapToken` is a token used for TLS bootstrapping. If `bootstrapToken` is set, this field is defaulted to `.bootstrapToken.token`, but can be overridden. If `file` is set, this field ∗∗must be set∗∗ in case the KubeConfigFile does not contain any other authentication information
timeout
meta/v1.Duration
`timeout` modifies the discovery timeout.

Etcd

Appears in:

Etcd contains elements describing Etcd configuration.

FieldDescription
local
LocalEtcd
`local` provides configuration knobs for configuring the local etcd instance. `local` and `external` are mutually exclusive.
external
ExternalEtcd
`external` describes how to connect to an external etcd cluster. `local` and `external` are mutually exclusive.

ExternalEtcd

Appears in:

ExternalEtcd describes an external etcd cluster. Kubeadm has no knowledge of where certificate files live and they must be supplied.

FieldDescription
endpoints [Required]
[]string
`endpoints` contains the list of etcd members.
caFile [Required]
string
`caFile` is an SSL Certificate Authority (CA) file used to secure etcd communication. Required if using a TLS connection.
certFile [Required]
string
`certFile` is an SSL certification file used to secure etcd communication. Required if using a TLS connection.
keyFile [Required]
string
`keyFile` is an SSL key file used to secure etcd communication. Required if using a TLS connection.

FileDiscovery

Appears in:

FileDiscovery is used to specify a file or URL to a kubeconfig file from which to load cluster information.

FieldDescription
kubeConfigPath [Required]
string
`kubeConfigPath` is used to specify the actual file path or URL to the kubeconfig file from which to load cluster information.

HostPathMount

Appears in:

HostPathMount contains elements describing volumes that are mounted from the host.

FieldDescription
name [Required]
string
`name` is the name of the volume inside the Pod template.
hostPath [Required]
string
`hostPath` is the path in the host that will be mounted inside the Pod.
mountPath [Required]
string
`mountPath` is the path inside the Pod where `hostPath` will be mounted.
readOnly
bool
`readOnly` controls write access to the volume.
pathType
core/v1.HostPathType
`pathType` is the type of the `hostPath`.

ImageMeta

Appears in:

ImageMeta allows to customize the image used for components that are not originated from the Kubernetes/Kubernetes release process

FieldDescription
imageRepository
string
`imageRepository` sets the container registry to pull images from. If not set, the `imageRepository` defined in ClusterConfiguration will be used instead.
imageTag
string
`imageTag` allows to specify a tag for the image. In case this value is set, kubeadm does not change automatically the version of the above components during upgrades.

JoinControlPlane

Appears in:

JoinControlPlane contains elements describing an additional control plane instance to be deployed on the joining node.

FieldDescription
localAPIEndpoint
APIEndpoint
`localAPIEndpoint` represents the endpoint of the API server instance to be deployed on this node.
certificateKey
string
`certificateKey` is the key that is used for decryption of certificates after they are downloaded from the secret upon joining a new control plane node. The corresponding encryption key is in the InitConfiguration.

LocalEtcd

Appears in:

LocalEtcd describes that kubeadm should run an etcd cluster locally

FieldDescription
ImageMeta [Required]
ImageMeta
(Members of ImageMeta are embedded into this type.) ImageMeta allows to customize the container used for etcd.
dataDir [Required]
string
`dataDir` is the directory etcd will place its data. Defaults to "/var/lib/etcd".
extraArgs
map[string]string
`extraArgs` are extra arguments provided to the etcd binary when run inside a static Pod. A key in this map is the flag name as it appears on the command line except without leading dash(es).
serverCertSANs
[]string
`serverCertSANs` sets extra Subject Alternative Names (SANs) for the etcd server signing certificate.
peerCertSANs
[]string
`peerCertSANs` sets extra Subject Alternative Names (SANs) for the etcd peer signing certificate.

Networking

Appears in:

Networking contains elements describing cluster's networking configuration

FieldDescription
serviceSubnet
string
`serviceSubnet` is the subnet used by Kubernetes Services. Defaults to "10.96.0.0/12".
podSubnet
string
`podSubnet` is the subnet used by Pods.
dnsDomain
string
`dnsDomain` is the DNS domain used by Kubernetes Services. Defaults to "cluster.local".

NodeRegistrationOptions

Appears in:

NodeRegistrationOptions holds fields that relate to registering a new control-plane or node to the cluster, either via "kubeadm init" or "kubeadm join"

FieldDescription
name
string
`name` is the `.metadata.name` field of the Node API object that will be created in this `kubeadm init` or `kubeadm join` operation. This field is also used in the `CommonName` field of the kubelet's client certificate to the API server. Defaults to the hostname of the node if not provided.
criSocket
string
`criSocket` is used to retrieve container runtime info. This information will be annotated to the Node API object, for later re-use
taints [Required]
[]core/v1.Taint
`tains` specifies the taints the Node API object should be registered with. If this field is unset, i.e. nil, in the `kubeadm init` process it will be defaulted to `taints: ["node-role.kubernetes.io/master:""]`. If you don't want to taint your control-plane node, set this field to an empty slice, i.e. `taints: []` in the YAML file. This field is solely used for Node registration.
kubeletExtraArgs
map[string]string
`kubeletExtraArgs` passes through extra arguments to the kubelet. The arguments here are passed to the kubelet command line via the environment file kubeadm writes at runtime for the kubelet to source. This overrides the generic base-level configuration in the 'kubelet-config-1.X' ConfigMap. Flags have higher priority when parsing. These values are local and specific to the node kubeadm is executing on. A key in this map is the flag name as it appears on the command line except without leading dash(es).
ignorePreflightErrors
[]string
`ignorePreflightErrors` provides a list of pre-flight errors to be ignored when the current node is registered.
imagePullPolicy
core/v1.PullPolicy
`imagePullPolicy` specifies the policy for image pulling during kubeadm "init" and "join" operations. The value of this field must be one of "Always", "IfNotPresent" or "Never". If this field is unset kubeadm will default it to "IfNotPresent", or pull the required images if not present on the host.

Patches

Appears in:

Patches contains options related to applying patches to components deployed by kubeadm.

FieldDescription
directory
string
`directory` is a path to a directory that contains files named "target[suffix][+patchtype].extension". For example, "kube-apiserver0+merge.yaml" or just "etcd.json". "target" can be one of "kube-apiserver", "kube-controller-manager", "kube-scheduler", "etcd". "patchtype" can be one of "strategic" "merge" or "json" and they match the patch formats supported by kubectl. The default "patchtype" is "strategic". "extension" must be either "json" or "yaml". "suffix" is an optional string that can be used to determine which patches are applied first alpha-numerically.

BootstrapToken

Appears in:

BootstrapToken describes one bootstrap token, stored as a Secret in the cluster

FieldDescription
token [Required]
BootstrapTokenString
`token` is used for establishing bidirectional trust between nodes and control-planes. Used for joining nodes in the cluster.
description
string
`description` sets a human-friendly message why this token exists and what it's used for, so other administrators can know its purpose.
ttl
meta/v1.Duration
`ttl` defines the time to live for this token. Defaults to `24h`. `expires` and `ttl` are mutually exclusive.
expires
meta/v1.Time
`expires` specifies the timestamp when this token expires. Defaults to being set dynamically at runtime based on the `ttl`. `expires` and `ttl` are mutually exclusive.
usages
[]string
`usages` describes the ways in which this token can be used. Can by default be used for establishing bidirectional trust, but that can be changed here.
groups
[]string
`groups` specifies the extra groups that this token will authenticate as when/if used for authentication

BootstrapTokenString

Appears in:

BootstrapTokenString is a token of the format abcdef.abcdef0123456789 that is used for both validation of the practically of the API server from a joining node's point of view and as an authentication method for the node in the bootstrap phase of "kubeadm join". This token is and should be short-lived.

FieldDescription
- [Required]
string
No description provided.
- [Required]
string
No description provided.

12 - Kubelet Configuration (v1alpha1)

Resource Types

FormatOptions

Appears in:

FormatOptions contains options for the different logging formats.

FieldDescription
json [Required]
JSONOptions
[Experimental] JSON contains options for logging format "json".

JSONOptions

Appears in:

JSONOptions contains options for logging format "json".

FieldDescription
splitStream [Required]
bool
[Experimental] SplitStream redirects error messages to stderr while info messages go to stdout, with buffering. The default is to write both to stdout, without buffering.
infoBufferSize [Required]
k8s.io/apimachinery/pkg/api/resource.QuantityValue
[Experimental] InfoBufferSize sets the size of the info stream when using split streams. The default is zero, which disables buffering.

VModuleConfiguration

(Alias of []k8s.io/component-base/config/v1alpha1.VModuleItem)

Appears in:

VModuleConfiguration is a collection of individual file names or patterns and the corresponding verbosity threshold.

CredentialProviderConfig

CredentialProviderConfig is the configuration containing information about each exec credential provider. Kubelet reads this configuration from disk and enables each provider as specified by the CredentialProvider type.

FieldDescription
apiVersion
string
kubelet.config.k8s.io/v1alpha1
kind
string
CredentialProviderConfig
providers [Required]
[]CredentialProvider
providers is a list of credential provider plugins that will be enabled by the kubelet. Multiple providers may match against a single image, in which case credentials from all providers will be returned to the kubelet. If multiple providers are called for a single image, the results are combined. If providers return overlapping auth keys, the value from the provider earlier in this list is used.

CredentialProvider

Appears in:

CredentialProvider represents an exec plugin to be invoked by the kubelet. The plugin is only invoked when an image being pulled matches the images handled by the plugin (see matchImages).

FieldDescription
name [Required]
string
name is the required name of the credential provider. It must match the name of the provider executable as seen by the kubelet. The executable must be in the kubelet's bin directory (set by the --image-credential-provider-bin-dir flag).
matchImages [Required]
[]string
matchImages is a required list of strings used to match against images in order to determine if this provider should be invoked. If one of the strings matches the requested image from the kubelet, the plugin will be invoked and given a chance to provide credentials. Images are expected to contain the registry domain and URL path.

Each entry in matchImages is a pattern which can optionally contain a port and a path. Globs can be used in the domain, but not in the port or the path. Globs are supported as subdomains like '∗.k8s.io' or 'k8s.∗.io', and top-level-domains such as 'k8s.∗'. Matching partial subdomains like 'app∗.k8s.io' is also supported. Each glob can only match a single subdomain segment, so ∗.io does not match ∗.k8s.io.

A match exists between an image and a matchImage when all of the below are true:

  • Both contain the same number of domain parts and each part matches.
  • The URL path of an imageMatch must be a prefix of the target image URL path.
  • If the imageMatch contains a port, then the port must match in the image as well.

Example values of matchImages:

  • 123456789.dkr.ecr.us-east-1.amazonaws.com
  • ∗.azurecr.io
  • gcr.io
  • ∗.∗.registry.io
  • registry.io:8080/path
defaultCacheDuration [Required]
meta/v1.Duration
defaultCacheDuration is the default duration the plugin will cache credentials in-memory if a cache duration is not provided in the plugin response. This field is required.
apiVersion [Required]
string
Required input version of the exec CredentialProviderRequest. The returned CredentialProviderResponse MUST use the same encoding version as the input. Current supported values are: - credentialprovider.kubelet.k8s.io/v1alpha1
args
[]string
Arguments to pass to the command when executing it.
env
[]ExecEnvVar
Env defines additional environment variables to expose to the process. These are unioned with the host's environment, as well as variables client-go uses to pass argument to the plugin.

ExecEnvVar

Appears in:

ExecEnvVar is used for setting environment variables when executing an exec-based credential plugin.

FieldDescription
name [Required]
string
No description provided.
value [Required]
string
No description provided.

13 - Kubelet Configuration (v1beta1)

Resource Types

FormatOptions

Appears in:

FormatOptions contains options for the different logging formats.

FieldDescription
json [Required]
JSONOptions
[Experimental] JSON contains options for logging format "json".

JSONOptions

Appears in:

JSONOptions contains options for logging format "json".

FieldDescription
splitStream [Required]
bool
[Experimental] SplitStream redirects error messages to stderr while info messages go to stdout, with buffering. The default is to write both to stdout, without buffering.
infoBufferSize [Required]
k8s.io/apimachinery/pkg/api/resource.QuantityValue
[Experimental] InfoBufferSize sets the size of the info stream when using split streams. The default is zero, which disables buffering.

LoggingConfiguration

Appears in:

LoggingConfiguration contains logging options Refer Logs Options for more information.

FieldDescription
format [Required]
string
Format Flag specifies the structure of log messages. default value of format is `text`
flushFrequency [Required]
time.Duration
Maximum number of seconds between log flushes. Ignored if the selected logging backend writes log messages without buffering.
verbosity [Required]
uint32
Verbosity is the threshold that determines which log messages are logged. Default is zero which logs only the most important messages. Higher values enable additional messages. Error messages are always logged.
vmodule [Required]
VModuleConfiguration
VModule overrides the verbosity threshold for individual files. Only supported for "text" log format.
sanitization [Required]
bool
[Experimental] When enabled prevents logging of fields tagged as sensitive (passwords, keys, tokens). Runtime log sanitization may introduce significant computation overhead and therefore should not be enabled in production.`)
options [Required]
FormatOptions
[Experimental] Options holds additional parameters that are specific to the different logging formats. Only the options for the selected format get used, but all of them get validated.

VModuleConfiguration

(Alias of []k8s.io/component-base/config/v1alpha1.VModuleItem)

Appears in:

VModuleConfiguration is a collection of individual file names or patterns and the corresponding verbosity threshold.

KubeletConfiguration

KubeletConfiguration contains the configuration for the Kubelet

FieldDescription
apiVersion
string
kubelet.config.k8s.io/v1beta1
kind
string
KubeletConfiguration
enableServer [Required]
bool
enableServer enables Kubelet's secured server. Note: Kubelet's insecure port is controlled by the readOnlyPort option. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that it may disrupt components that interact with the Kubelet server. Default: true
staticPodPath
string
staticPodPath is the path to the directory containing local (static) pods to run, or the path to a single static pod file. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that the set of static pods specified at the new path may be different than the ones the Kubelet initially started with, and this may disrupt your node. Default: ""
syncFrequency
meta/v1.Duration
syncFrequency is the max period between synchronizing running containers and config. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that shortening this duration may have a negative performance impact, especially as the number of Pods on the node increases. Alternatively, increasing this duration will result in longer refresh times for ConfigMaps and Secrets. Default: "1m"
fileCheckFrequency
meta/v1.Duration
fileCheckFrequency is the duration between checking config files for new data. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that shortening the duration will cause the Kubelet to reload local Static Pod configurations more frequently, which may have a negative performance impact. Default: "20s"
httpCheckFrequency
meta/v1.Duration
httpCheckFrequency is the duration between checking http for new data. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that shortening the duration will cause the Kubelet to poll staticPodURL more frequently, which may have a negative performance impact. Default: "20s"
staticPodURL
string
staticPodURL is the URL for accessing static pods to run. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that the set of static pods specified at the new URL may be different than the ones the Kubelet initially started with, and this may disrupt your node. Default: ""
staticPodURLHeader
map[string][]string
staticPodURLHeader is a map of slices with HTTP headers to use when accessing the podURL. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that it may disrupt the ability to read the latest set of static pods from StaticPodURL. Default: nil
address
string
address is the IP address for the Kubelet to serve on (set to 0.0.0.0 for all interfaces). If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that it may disrupt components that interact with the Kubelet server. Default: "0.0.0.0"
port
int32
port is the port for the Kubelet to serve on. The port number must be between 1 and 65535, inclusive. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that it may disrupt components that interact with the Kubelet server. Default: 10250
readOnlyPort
int32
readOnlyPort is the read-only port for the Kubelet to serve on with no authentication/authorization. The port number must be between 1 and 65535, inclusive. Setting this field to 0 disables the read-only service. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that it may disrupt components that interact with the Kubelet server. Default: 0 (disabled)
tlsCertFile
string
tlsCertFile is the file containing x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). If tlsCertFile and tlsPrivateKeyFile are not provided, a self-signed certificate and key are generated for the public address and saved to the directory passed to the Kubelet's --cert-dir flag. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that it may disrupt components that interact with the Kubelet server. Default: ""
tlsPrivateKeyFile
string
tlsPrivateKeyFile is the file containing x509 private key matching tlsCertFile. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that it may disrupt components that interact with the Kubelet server. Default: ""
tlsCipherSuites
[]string
tlsCipherSuites is the list of allowed cipher suites for the server. Values are from tls package constants (https://golang.org/pkg/crypto/tls/#pkg-constants). If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that it may disrupt components that interact with the Kubelet server. Default: nil
tlsMinVersion
string
tlsMinVersion is the minimum TLS version supported. Values are from tls package constants (https://golang.org/pkg/crypto/tls/#pkg-constants). If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that it may disrupt components that interact with the Kubelet server. Default: ""
rotateCertificates
bool
rotateCertificates enables client certificate rotation. The Kubelet will request a new certificate from the certificates.k8s.io API. This requires an approver to approve the certificate signing requests. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that disabling it may disrupt the Kubelet's ability to authenticate with the API server after the current certificate expires. Default: false
serverTLSBootstrap
bool
serverTLSBootstrap enables server certificate bootstrap. Instead of self signing a serving certificate, the Kubelet will request a certificate from the 'certificates.k8s.io' API. This requires an approver to approve the certificate signing requests (CSR). The RotateKubeletServerCertificate feature must be enabled when setting this field. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that disabling it will stop the renewal of Kubelet server certificates, which can disrupt components that interact with the Kubelet server in the long term, due to certificate expiration. Default: false
authentication
KubeletAuthentication
authentication specifies how requests to the Kubelet's server are authenticated. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that it may disrupt components that interact with the Kubelet server. Defaults: anonymous: enabled: false webhook: enabled: true cacheTTL: "2m"
authorization
KubeletAuthorization
authorization specifies how requests to the Kubelet's server are authorized. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that it may disrupt components that interact with the Kubelet server. Defaults: mode: Webhook webhook: cacheAuthorizedTTL: "5m" cacheUnauthorizedTTL: "30s"
registryPullQPS
int32
registryPullQPS is the limit of registry pulls per second. The value must not be a negative number. Setting it to 0 means no limit. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that it may impact scalability by changing the amount of traffic produced by image pulls. Default: 5
registryBurst
int32
registryBurst is the maximum size of bursty pulls, temporarily allows pulls to burst to this number, while still not exceeding registryPullQPS. The value must not be a negative number. Only used if registryPullQPS is greater than 0. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that it may impact scalability by changing the amount of traffic produced by image pulls. Default: 10
eventRecordQPS
int32
eventRecordQPS is the maximum event creations per second. If 0, there is no limit enforced. The value cannot be a negative number. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that it may impact scalability by changing the amount of traffic produced by event creations. Default: 5
eventBurst
int32
eventBurst is the maximum size of a burst of event creations, temporarily allows event creations to burst to this number, while still not exceeding eventRecordQPS. This field canot be a negative number and it is only used when eventRecordQPS > 0. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that it may impact scalability by changing the amount of traffic produced by event creations. Default: 10
enableDebuggingHandlers
bool
enableDebuggingHandlers enables server endpoints for log access and local running of containers and commands, including the exec, attach, logs, and portforward features. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that disabling it may disrupt components that interact with the Kubelet server. Default: true
enableContentionProfiling
bool
enableContentionProfiling enables lock contention profiling, if enableDebuggingHandlers is true. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that enabling it may carry a performance impact. Default: false
healthzPort
int32
healthzPort is the port of the localhost healthz endpoint (set to 0 to disable). A valid number is between 1 and 65535. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that it may disrupt components that monitor Kubelet health. Default: 10248
healthzBindAddress
string
healthzBindAddress is the IP address for the healthz server to serve on. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that it may disrupt components that monitor Kubelet health. Default: "127.0.0.1"
oomScoreAdj
int32
oomScoreAdj is The oom-score-adj value for kubelet process. Values must be within the range [-1000, 1000]. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that it may impact the stability of nodes under memory pressure. Default: -999
clusterDomain
string
clusterDomain is the DNS domain for this cluster. If set, kubelet will configure all containers to search this domain in addition to the host's search domains. Dynamic Kubelet Config (deprecated): Dynamically updating this field is not recommended, as it should be kept in sync with the rest of the cluster. Default: ""
clusterDNS
[]string
clusterDNS is a list of IP addresses for the cluster DNS server. If set, kubelet will configure all containers to use this for DNS resolution instead of the host's DNS servers. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that changes will only take effect on Pods created after the update. Draining the node is recommended before changing this field. Default: nil
streamingConnectionIdleTimeout
meta/v1.Duration
streamingConnectionIdleTimeout is the maximum time a streaming connection can be idle before the connection is automatically closed. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that it may impact components that rely on infrequent updates over streaming connections to the Kubelet server. Default: "4h"
nodeStatusUpdateFrequency
meta/v1.Duration
nodeStatusUpdateFrequency is the frequency that kubelet computes node status. If node lease feature is not enabled, it is also the frequency that kubelet posts node status to master. Note: When node lease feature is not enabled, be cautious when changing the constant, it must work with nodeMonitorGracePeriod in nodecontroller. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that it may impact node scalability, and also that the node controller's nodeMonitorGracePeriod must be set to N∗NodeStatusUpdateFrequency, where N is the number of retries before the node controller marks the node unhealthy. Default: "10s"
nodeStatusReportFrequency
meta/v1.Duration
nodeStatusReportFrequency is the frequency that kubelet posts node status to master if node status does not change. Kubelet will ignore this frequency and post node status immediately if any change is detected. It is only used when node lease feature is enabled. nodeStatusReportFrequency's default value is 5m. But if nodeStatusUpdateFrequency is set explicitly, nodeStatusReportFrequency's default value will be set to nodeStatusUpdateFrequency for backward compatibility. Default: "5m"
nodeLeaseDurationSeconds
int32
nodeLeaseDurationSeconds is the duration the Kubelet will set on its corresponding Lease. NodeLease provides an indicator of node health by having the Kubelet create and periodically renew a lease, named after the node, in the kube-node-lease namespace. If the lease expires, the node can be considered unhealthy. The lease is currently renewed every 10s, per KEP-0009. In the future, the lease renewal interval may be set based on the lease duration. The field value must be greater than 0. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that decreasing the duration may reduce tolerance for issues that temporarily prevent the Kubelet from renewing the lease (e.g. a short-lived network issue). Default: 40
imageMinimumGCAge
meta/v1.Duration
imageMinimumGCAge is the minimum age for an unused image before it is garbage collected. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that it may trigger or delay garbage collection, and may change the image overhead on the node. Default: "2m"
imageGCHighThresholdPercent
int32
imageGCHighThresholdPercent is the percent of disk usage after which image garbage collection is always run. The percent is calculated by dividing this field value by 100, so this field must be between 0 and 100, inclusive. When specified, the value must be greater than imageGCLowThresholdPercent. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that it may trigger or delay garbage collection, and may change the image overhead on the node. Default: 85
imageGCLowThresholdPercent
int32
imageGCLowThresholdPercent is the percent of disk usage before which image garbage collection is never run. Lowest disk usage to garbage collect to. The percent is calculated by dividing this field value by 100, so the field value must be between 0 and 100, inclusive. When specified, the value must be less than imageGCHighThresholdPercent. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that it may trigger or delay garbage collection, and may change the image overhead on the node. Default: 80
volumeStatsAggPeriod
meta/v1.Duration
volumeStatsAggPeriod is the frequency for calculating and caching volume disk usage for all pods. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that shortening the period may carry a performance impact. Default: "1m"
kubeletCgroups
string
kubeletCgroups is the absolute name of cgroups to isolate the kubelet in Dynamic Kubelet Config (deprecated): This field should not be updated without a full node reboot. It is safest to keep this value the same as the local config. Default: ""
systemCgroups
string
systemCgroups is absolute name of cgroups in which to place all non-kernel processes that are not already in a container. Empty for no container. Rolling back the flag requires a reboot. The cgroupRoot must be specified if this field is not empty. Dynamic Kubelet Config (deprecated): This field should not be updated without a full node reboot. It is safest to keep this value the same as the local config. Default: ""
cgroupRoot
string
cgroupRoot is the root cgroup to use for pods. This is handled by the container runtime on a best effort basis. Dynamic Kubelet Config (deprecated): This field should not be updated without a full node reboot. It is safest to keep this value the same as the local config. Default: ""
cgroupsPerQOS
bool
cgroupsPerQOS enable QoS based CGroup hierarchy: top level CGroups for QoS classes and all Burstable and BestEffort Pods are brought up under their specific top level QoS CGroup. Dynamic Kubelet Config (deprecated): This field should not be updated without a full node reboot. It is safest to keep this value the same as the local config. Default: true
cgroupDriver
string
cgroupDriver is the driver kubelet uses to manipulate CGroups on the host (cgroupfs or systemd). Dynamic Kubelet Config (deprecated): This field should not be updated without a full node reboot. It is safest to keep this value the same as the local config. Default: "cgroupfs"
cpuManagerPolicy
string
cpuManagerPolicy is the name of the policy to use. Requires the CPUManager feature gate to be enabled. Dynamic Kubelet Config (deprecated): This field should not be updated without a full node reboot. It is safest to keep this value the same as the local config. Default: "None"
cpuManagerPolicyOptions
map[string]string
cpuManagerPolicyOptions is a set of key=value which allows to set extra options to fine tune the behaviour of the cpu manager policies. Requires both the "CPUManager" and "CPUManagerPolicyOptions" feature gates to be enabled. Dynamic Kubelet Config (beta): This field should not be updated without a full node reboot. It is safest to keep this value the same as the local config. Default: nil
cpuManagerReconcilePeriod
meta/v1.Duration
cpuManagerReconcilePeriod is the reconciliation period for the CPU Manager. Requires the CPUManager feature gate to be enabled. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that shortening the period may carry a performance impact. Default: "10s"
memoryManagerPolicy
string
memoryManagerPolicy is the name of the policy to use by memory manager. Requires the MemoryManager feature gate to be enabled. Dynamic Kubelet Config (deprecated): This field should not be updated without a full node reboot. It is safest to keep this value the same as the local config. Default: "none"
topologyManagerPolicy
string
topologyManagerPolicy is the name of the topology manager policy to use. Valid values include:
  • restricted: kubelet only allows pods with optimal NUMA node alignment for requested resources;
  • best-effort: kubelet will favor pods with NUMA alignment of CPU and device resources;
  • none: kubelet has no knowledge of NUMA alignment of a pod's CPU and device resources.
  • single-numa-node: kubelet only allows pods with a single NUMA alignment of CPU and device resources.

Policies other than "none" require the TopologyManager feature gate to be enabled. Dynamic Kubelet Config (deprecated): This field should not be updated without a full node reboot. It is safest to keep this value the same as the local config. Default: "none"

topologyManagerScope
string
topologyManagerScope represents the scope of topology hint generation that topology manager requests and hint providers generate. Valid values include:
  • container: topology policy is applied on a per-container basis.
  • pod: topology policy is applied on a per-pod basis.

"pod" scope requires the TopologyManager feature gate to be enabled. Default: "container"

qosReserved
map[string]string
qosReserved is a set of resource name to percentage pairs that specify the minimum percentage of a resource reserved for exclusive use by the guaranteed QoS tier. Currently supported resources: "memory" Requires the QOSReserved feature gate to be enabled. Dynamic Kubelet Config (deprecated): This field should not be updated without a full node reboot. It is safest to keep this value the same as the local config. Default: nil
runtimeRequestTimeout
meta/v1.Duration
runtimeRequestTimeout is the timeout for all runtime requests except long running requests - pull, logs, exec and attach. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that it may disrupt components that interact with the Kubelet server. Default: "2m"
hairpinMode
string
hairpinMode specifies how the Kubelet should configure the container bridge for hairpin packets. Setting this flag allows endpoints in a Service to loadbalance back to themselves if they should try to access their own Service. Values:
  • "promiscuous-bridge": make the container bridge promiscuous.
  • "hairpin-veth": set the hairpin flag on container veth interfaces.
  • "none": do nothing.

Generally, one must set --hairpin-mode=hairpin-veth to achieve hairpin NAT, because promiscuous-bridge assumes the existence of a container bridge named cbr0. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that it may require a node reboot, depending on the network plugin. Default: "promiscuous-bridge"

maxPods
int32
maxPods is the maximum number of Pods that can run on this Kubelet. The value must be a non-negative integer. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that changes may cause Pods to fail admission on Kubelet restart, and may change the value reported in Node.Status.Capacity[v1.ResourcePods], thus affecting future scheduling decisions. Increasing this value may also decrease performance, as more Pods can be packed into a single node. Default: 110
podCIDR
string
podCIDR is the CIDR to use for pod IP addresses, only used in standalone mode. In cluster mode, this is obtained from the control plane. Dynamic Kubelet Config (deprecated): This field should always be set to the empty default. It should only set for standalone Kubelets, which cannot use Dynamic Kubelet Config. Default: ""
podPidsLimit
int64
podPidsLimit is the maximum number of PIDs in any pod. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that lowering it may prevent container processes from forking after the change. Default: -1
resolvConf
string
resolvConf is the resolver configuration file used as the basis for the container DNS resolution configuration. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that changes will only take effect on Pods created after the update. Draining the node is recommended before changing this field. If set to the empty string, will override the default and effectively disable DNS lookups. Default: "/etc/resolv.conf"
runOnce
bool
runOnce causes the Kubelet to check the API server once for pods, run those in addition to the pods specified by static pod files, and exit. Default: false
cpuCFSQuota
bool
cpuCFSQuota enables CPU CFS quota enforcement for containers that specify CPU limits. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that disabling it may reduce node stability. Default: true
cpuCFSQuotaPeriod
meta/v1.Duration
cpuCFSQuotaPeriod is the CPU CFS quota period value, `cpu.cfs_period_us`. The value must be between 1 us and 1 second, inclusive. Requires the CustomCPUCFSQuotaPeriod feature gate to be enabled. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that limits set for containers will result in different cpu.cfs_quota settings. This will trigger container restarts on the node being reconfigured. Default: "100ms"
nodeStatusMaxImages
int32
nodeStatusMaxImages caps the number of images reported in Node.status.images. The value must be greater than -2. Note: If -1 is specified, no cap will be applied. If 0 is specified, no image is returned. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that different values can be reported on node status. Default: 50
maxOpenFiles
int64
maxOpenFiles is Number of files that can be opened by Kubelet process. The value must be a non-negative number. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that it may impact the ability of the Kubelet to interact with the node's filesystem. Default: 1000000
contentType
string
contentType is contentType of requests sent to apiserver. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that it may impact the ability for the Kubelet to communicate with the API server. If the Kubelet loses contact with the API server due to a change to this field, the change cannot be reverted via dynamic Kubelet config. Default: "application/vnd.kubernetes.protobuf"
kubeAPIQPS
int32
kubeAPIQPS is the QPS to use while talking with kubernetes apiserver. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that it may impact scalability by changing the amount of traffic the Kubelet sends to the API server. Default: 5
kubeAPIBurst
int32
kubeAPIBurst is the burst to allow while talking with kubernetes API server. This field cannot be a negative number. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that it may impact scalability by changing the amount of traffic the Kubelet sends to the API server. Default: 10
serializeImagePulls
bool
serializeImagePulls when enabled, tells the Kubelet to pull images one at a time. We recommend ∗not∗ changing the default value on nodes that run docker daemon with version < 1.9 or an Aufs storage backend. Issue #10959 has more details. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that it may impact the performance of image pulls. Default: true
evictionHard
map[string]string
evictionHard is a map of signal names to quantities that defines hard eviction thresholds. For example: `{"memory.available": "300Mi"}`. To explicitly disable, pass a 0% or 100% threshold on an arbitrary resource. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that it may trigger or delay Pod evictions. Default: memory.available: "100Mi" nodefs.available: "10%" nodefs.inodesFree: "5%" imagefs.available: "15%"
evictionSoft
map[string]string
evictionSoft is a map of signal names to quantities that defines soft eviction thresholds. For example: `{"memory.available": "300Mi"}`. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that it may trigger or delay Pod evictions, and may change the allocatable reported by the node. Default: nil
evictionSoftGracePeriod
map[string]string
evictionSoftGracePeriod is a map of signal names to quantities that defines grace periods for each soft eviction signal. For example: `{"memory.available": "30s"}`. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that it may trigger or delay Pod evictions. Default: nil
evictionPressureTransitionPeriod
meta/v1.Duration
evictionPressureTransitionPeriod is the duration for which the kubelet has to wait before transitioning out of an eviction pressure condition. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that lowering it may decrease the stability of the node when the node is overcommitted. Default: "5m"
evictionMaxPodGracePeriod
int32
evictionMaxPodGracePeriod is the maximum allowed grace period (in seconds) to use when terminating pods in response to a soft eviction threshold being met. This value effectively caps the Pod's terminationGracePeriodSeconds value during soft evictions. Note: Due to issue #64530, the behavior has a bug where this value currently just overrides the grace period during soft eviction, which can increase the grace period from what is set on the Pod. This bug will be fixed in a future release. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that lowering it decreases the amount of time Pods will have to gracefully clean up before being killed during a soft eviction. Default: 0
evictionMinimumReclaim
map[string]string
evictionMinimumReclaim is a map of signal names to quantities that defines minimum reclaims, which describe the minimum amount of a given resource the kubelet will reclaim when performing a pod eviction while that resource is under pressure. For example: `{"imagefs.available": "2Gi"}`. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that it may change how well eviction can manage resource pressure. Default: nil
podsPerCore
int32
podsPerCore is the maximum number of pods per core. Cannot exceed maxPods. The value must be a non-negative integer. If 0, there is no limit on the number of Pods. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that changes may cause Pods to fail admission on Kubelet restart, and may change the value reported in `Node.status.capacity.pods`, thus affecting future scheduling decisions. Increasing this value may also decrease performance, as more Pods can be packed into a single node. Default: 0
enableControllerAttachDetach
bool
enableControllerAttachDetach enables the Attach/Detach controller to manage attachment/detachment of volumes scheduled to this node, and disables kubelet from executing any attach/detach operations. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that changing which component is responsible for volume management on a live node may result in volumes refusing to detach if the node is not drained prior to the update, and if Pods are scheduled to the node before the volumes.kubernetes.io/controller-managed-attach-detach annotation is updated by the Kubelet. In general, it is safest to leave this value set the same as local config. Default: true
protectKernelDefaults
bool
protectKernelDefaults, if true, causes the Kubelet to error if kernel flags are not as it expects. Otherwise the Kubelet will attempt to modify kernel flags to match its expectation. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that enabling it may cause the Kubelet to crash-loop if the Kernel is not configured as Kubelet expects. Default: false
makeIPTablesUtilChains
bool
makeIPTablesUtilChains, if true, causes the Kubelet ensures a set of iptables rules are present on host. These rules will serve as utility rules for various components, e.g. kube-proxy. The rules will be created based on iptablesMasqueradeBit and iptablesDropBit. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that disabling it will prevent the Kubelet from healing locally misconfigured iptables rules. Default: true
iptablesMasqueradeBit
int32
iptablesMasqueradeBit is the bit of the iptables fwmark space to mark for SNAT. Values must be within the range [0, 31]. Must be different from other mark bits. Warning: Please match the value of the corresponding parameter in kube-proxy. TODO: clean up IPTablesMasqueradeBit in kube-proxy. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that it needs to be coordinated with other components, like kube-proxy, and the update will only be effective if MakeIPTablesUtilChains is enabled. Default: 14
iptablesDropBit
int32
iptablesDropBit is the bit of the iptables fwmark space to mark for dropping packets. Values must be within the range [0, 31]. Must be different from other mark bits. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that it needs to be coordinated with other components, like kube-proxy, and the update will only be effective if MakeIPTablesUtilChains is enabled. Default: 15
featureGates
map[string]bool
featureGates is a map of feature names to bools that enable or disable experimental features. This field modifies piecemeal the built-in default values from "k8s.io/kubernetes/pkg/features/kube_features.go". If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider the documentation for the features you are enabling or disabling. While we encourage feature developers to make it possible to dynamically enable and disable features, some changes may require node reboots, and some features may require careful coordination to retroactively disable. Default: nil
failSwapOn
bool
failSwapOn tells the Kubelet to fail to start if swap is enabled on the node. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that setting it to true will cause the Kubelet to crash-loop if swap is enabled. Default: true
memorySwap
MemorySwapConfiguration
memorySwap configures swap memory available to container workloads.
containerLogMaxSize
string
containerLogMaxSize is a quantity defining the maximum size of the container log file before it is rotated. For example: "5Mi" or "256Ki". If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that it may trigger log rotation. Default: "10Mi"
containerLogMaxFiles
int32
containerLogMaxFiles specifies the maximum number of container log files that can be present for a container. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that lowering it may cause log files to be deleted. Default: 5
configMapAndSecretChangeDetectionStrategy
ResourceChangeDetectionStrategy
configMapAndSecretChangeDetectionStrategy is a mode in which ConfigMap and Secret managers are running. Valid values include:
  • Get: kubelet fetches necessary objects directly from the API server;
  • Cache: kubelet uses TTL cache for object fetched from the API server;
  • Watch: kubelet uses watches to observe changes to objects that are in its interest.

Default: "Watch"

systemReserved
map[string]string
systemReserved is a set of ResourceName=ResourceQuantity (e.g. cpu=200m,memory=150G) pairs that describe resources reserved for non-kubernetes components. Currently only cpu and memory are supported. See http://kubernetes.io/docs/user-guide/compute-resources for more detail. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that it may not be possible to increase the reserved resources, because this requires resizing cgroups. Always look for a NodeAllocatableEnforced event after updating this field to ensure that the update was successful. Default: nil
kubeReserved
map[string]string
kubeReserved is a set of ResourceName=ResourceQuantity (e.g. cpu=200m,memory=150G) pairs that describe resources reserved for kubernetes system components. Currently cpu, memory and local storage for root file system are supported. See https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/ for more details. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that it may not be possible to increase the reserved resources, because this requires resizing cgroups. Always look for a NodeAllocatableEnforced event after updating this field to ensure that the update was successful. Default: nil
reservedSystemCPUs [Required]
string
The reservedSystemCPUs option specifies the CPU list reserved for the host level system threads and kubernetes related threads. This provide a "static" CPU list rather than the "dynamic" list by systemReserved and kubeReserved. This option does not support systemReservedCgroup or kubeReservedCgroup.
showHiddenMetricsForVersion
string
showHiddenMetricsForVersion is the previous version for which you want to show hidden metrics. Only the previous minor version is meaningful, other values will not be allowed. The format is `.`, e.g.: `1.16`. The purpose of this format is make sure you have the opportunity to notice if the next release hides additional metrics, rather than being surprised when they are permanently removed in the release after that. Default: ""
systemReservedCgroup
string
systemReservedCgroup helps the kubelet identify absolute name of top level CGroup used to enforce `systemReserved` compute resource reservation for OS system daemons. Refer to [Node Allocatable](https://git.k8s.io/community/contributors/design-proposals/node/node-allocatable.md) doc for more information. Dynamic Kubelet Config (deprecated): This field should not be updated without a full node reboot. It is safest to keep this value the same as the local config. Default: ""
kubeReservedCgroup
string
kubeReservedCgroup helps the kubelet identify absolute name of top level CGroup used to enforce `KubeReserved` compute resource reservation for Kubernetes node system daemons. Refer to [Node Allocatable](https://git.k8s.io/community/contributors/design-proposals/node/node-allocatable.md) doc for more information. Dynamic Kubelet Config (deprecated): This field should not be updated without a full node reboot. It is safest to keep this value the same as the local config. Default: ""
enforceNodeAllocatable
[]string
This flag specifies the various Node Allocatable enforcements that Kubelet needs to perform. This flag accepts a list of options. Acceptable options are `none`, `pods`, `system-reserved` and `kube-reserved`. If `none` is specified, no other options may be specified. When `system-reserved` is in the list, systemReservedCgroup must be specified. When `kube-reserved` is in the list, kubeReservedCgroup must be specified. This field is supported only when `cgroupsPerQOS` is set to true. Refer to [Node Allocatable](https://git.k8s.io/community/contributors/design-proposals/node/node-allocatable.md) for more information. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that removing enforcements may reduce the stability of the node. Alternatively, adding enforcements may reduce the stability of components which were using more than the reserved amount of resources; for example, enforcing kube-reserved may cause Kubelets to OOM if it uses more than the reserved resources, and enforcing system-reserved may cause system daemons to OOM if they use more than the reserved resources. Default: ["pods"]
allowedUnsafeSysctls
[]string
A comma separated whitelist of unsafe sysctls or sysctl patterns (ending in `∗`). Unsafe sysctl groups are `kernel.shm∗`, `kernel.msg∗`, `kernel.sem`, `fs.mqueue.∗`, and `net.∗`. For example: "`kernel.msg∗,net.ipv4.route.min_pmtu`" Default: []
volumePluginDir
string
volumePluginDir is the full path of the directory in which to search for additional third party volume plugins. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that changing the volumePluginDir may disrupt workloads relying on third party volume plugins. Default: "/usr/libexec/kubernetes/kubelet-plugins/volume/exec/"
providerID
string
providerID, if set, sets the unique ID of the instance that an external provider (i.e. cloudprovider) can use to identify a specific node. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that it may impact the ability of the Kubelet to interact with cloud providers. Default: ""
kernelMemcgNotification
bool
kernelMemcgNotification, if set, instructs the the kubelet to integrate with the kernel memcg notification for determining if memory eviction thresholds are exceeded rather than polling. If DynamicKubeletConfig (deprecated; default off) is on, when dynamically updating this field, consider that it may impact the way Kubelet interacts with the kernel. Default: false
logging [Required]
LoggingConfiguration
logging specifies the options of logging. Refer to [Logs Options](https://github.com/kubernetes/component-base/blob/master/logs/options.go) for more information. Default: Format: text
enableSystemLogHandler
bool
enableSystemLogHandler enables system logs via web interface host:port/logs/ Default: true
shutdownGracePeriod
meta/v1.Duration
shutdownGracePeriod specifies the total duration that the node should delay the shutdown and total grace period for pod termination during a node shutdown. Default: "0s"
shutdownGracePeriodCriticalPods
meta/v1.Duration
shutdownGracePeriodCriticalPods specifies the duration used to terminate critical pods during a node shutdown. This should be less than shutdownGracePeriod. For example, if shutdownGracePeriod=30s, and shutdownGracePeriodCriticalPods=10s, during a node shutdown the first 20 seconds would be reserved for gracefully terminating normal pods, and the last 10 seconds would be reserved for terminating critical pods. Default: "0s"
shutdownGracePeriodByPodPriority
[]ShutdownGracePeriodByPodPriority
shutdownGracePeriodByPodPriority specifies the shutdown grace period for Pods based on their associated priority class value. When a shutdown request is received, the Kubelet will initiate shutdown on all pods running on the node with a grace period that depends on the priority of the pod, and then wait for all pods to exit. Each entry in the array represents the graceful shutdown time a pod with a priority class value that lies in the range of that value and the next higher entry in the list when the node is shutting down. For example, to allow critical pods 10s to shutdown, priority>=10000 pods 20s to shutdown, and all remaining pods 30s to shutdown.

shutdownGracePeriodByPodPriority:

  • priority: 2000000000 shutdownGracePeriodSeconds: 10
  • priority: 10000 shutdownGracePeriodSeconds: 20
  • priority: 0 shutdownGracePeriodSeconds: 30

The time the Kubelet will wait before exiting will at most be the maximum of all shutdownGracePeriodSeconds for each priority class range represented on the node. When all pods have exited or reached their grace periods, the Kubelet will release the shutdown inhibit lock. Requires the GracefulNodeShutdown feature gate to be enabled. This configuration must be empty if either ShutdownGracePeriod or ShutdownGracePeriodCriticalPods is set. Default: nil

reservedMemory
[]MemoryReservation
reservedMemory specifies a comma-separated list of memory reservations for NUMA nodes. The parameter makes sense only in the context of the memory manager feature. The memory manager will not allocate reserved memory for container workloads. For example, if you have a NUMA0 with 10Gi of memory and the reservedMemory was specified to reserve 1Gi of memory at NUMA0, the memory manager will assume that only 9Gi is available for allocation. You can specify a different amount of NUMA node and memory types. You can omit this parameter at all, but you should be aware that the amount of reserved memory from all NUMA nodes should be equal to the amount of memory specified by the [node allocatable](https://kubernetes.io/docs/tasks/administer-cluster/reserve-compute-resources/#node-allocatable). If at least one node allocatable parameter has a non-zero value, you will need to specify at least one NUMA node. Also, avoid specifying:
  1. Duplicates, the same NUMA node, and memory type, but with a different value.
  2. zero limits for any memory type.
  3. NUMAs nodes IDs that do not exist under the machine.
  4. memory types except for memory and hugepages-

Default: nil

enableProfilingHandler
bool
enableProfilingHandler enables profiling via web interface host:port/debug/pprof/ Default: true
enableDebugFlagsHandler
bool
enableDebugFlagsHandler enables flags endpoint via web interface host:port/debug/flags/v Default: true
seccompDefault
bool
SeccompDefault enables the use of `RuntimeDefault` as the default seccomp profile for all workloads. This requires the corresponding SeccompDefault feature gate to be enabled as well. Default: false
memoryThrottlingFactor
float64
MemoryThrottlingFactor specifies the factor multiplied by the memory limit or node allocatable memory when setting the cgroupv2 memory.high value to enforce MemoryQoS. Decreasing this factor will set lower high limit for container cgroups and put heavier reclaim pressure while increasing will put less reclaim pressure. See http://kep.k8s.io/2570 for more details. Default: 0.8
registerWithTaints
[]core/v1.Taint
registerWithTaints are an array of taints to add to a node object when the kubelet registers itself. This only takes effect when registerNode is true and upon the initial registration of the node. Default: nil
registerNode
bool
registerNode enables automatic registration with the apiserver. Default: true

SerializedNodeConfigSource

SerializedNodeConfigSource allows us to serialize v1.NodeConfigSource. This type is used internally by the Kubelet for tracking checkpointed dynamic configs. It exists in the kubeletconfig API group because it is classified as a versioned input to the Kubelet.

FieldDescription
apiVersion
string
kubelet.config.k8s.io/v1beta1
kind
string
SerializedNodeConfigSource
source
core/v1.NodeConfigSource
source is the source that we are serializing.

KubeletAnonymousAuthentication

Appears in:

FieldDescription
enabled
bool
enabled allows anonymous requests to the kubelet server. Requests that are not rejected by another authentication method are treated as anonymous requests. Anonymous requests have a username of `system:anonymous`, and a group name of `system:unauthenticated`.

KubeletAuthentication

Appears in:

FieldDescription
x509
KubeletX509Authentication
x509 contains settings related to x509 client certificate authentication.
webhook
KubeletWebhookAuthentication
webhook contains settings related to webhook bearer token authentication.
anonymous
KubeletAnonymousAuthentication
anonymous contains settings related to anonymous authentication.

KubeletAuthorization

Appears in:

FieldDescription
mode
KubeletAuthorizationMode
mode is the authorization mode to apply to requests to the kubelet server. Valid values are `AlwaysAllow` and `Webhook`. Webhook mode uses the SubjectAccessReview API to determine authorization.
webhook
KubeletWebhookAuthorization
webhook contains settings related to Webhook authorization.

KubeletAuthorizationMode

(Alias of string)

Appears in:

KubeletWebhookAuthentication

Appears in:

FieldDescription
enabled
bool
enabled allows bearer token authentication backed by the tokenreviews.authentication.k8s.io API.
cacheTTL
meta/v1.Duration
cacheTTL enables caching of authentication results

KubeletWebhookAuthorization

Appears in:

FieldDescription
cacheAuthorizedTTL
meta/v1.Duration
cacheAuthorizedTTL is the duration to cache 'authorized' responses from the webhook authorizer.
cacheUnauthorizedTTL
meta/v1.Duration
cacheUnauthorizedTTL is the duration to cache 'unauthorized' responses from the webhook authorizer.

KubeletX509Authentication

Appears in:

FieldDescription
clientCAFile
string
clientCAFile is the path to a PEM-encoded certificate bundle. If set, any request presenting a client certificate signed by one of the authorities in the bundle is authenticated with a username corresponding to the CommonName, and groups corresponding to the Organization in the client certificate.

MemoryReservation

Appears in:

MemoryReservation specifies the memory reservation of different types for each NUMA node

FieldDescription
numaNode [Required]
int32
No description provided.
limits [Required]
core/v1.ResourceList
No description provided.

MemorySwapConfiguration

Appears in:

FieldDescription
swapBehavior
string
swapBehavior configures swap memory available to container workloads. May be one of "", "LimitedSwap": workload combined memory and swap usage cannot exceed pod memory limit "UnlimitedSwap": workloads can use unlimited swap, up to the allocatable limit.

ResourceChangeDetectionStrategy

(Alias of string)

Appears in:

ResourceChangeDetectionStrategy denotes a mode in which internal managers (secret, configmap) are discovering object changes.

ShutdownGracePeriodByPodPriority

Appears in:

ShutdownGracePeriodByPodPriority specifies the shutdown grace period for Pods based on their associated priority class value

FieldDescription
priority [Required]
int32
priority is the priority value associated with the shutdown grace period
shutdownGracePeriodSeconds [Required]
int64
shutdownGracePeriodSeconds is the shutdown grace period in seconds

14 - WebhookAdmission Configuration (v1)

Package v1 is the v1 version of the API.

Resource Types

WebhookAdmission

WebhookAdmission provides configuration for the webhook admission controller.

FieldDescription
apiVersion
string
apiserver.config.k8s.io/v1
kind
string
WebhookAdmission
kubeConfigFile [Required]
string
KubeConfigFile is the path to the kubeconfig file.