Skip to content

HyperShift API Reference

Packages:

hypershift.openshift.io/v1beta1

Package v1beta1 contains the HyperShift API.

The HyperShift API enables creating and managing lightweight, flexible, heterogeneous OpenShift clusters at scale.

HyperShift clusters are deployed in a topology which isolates the “control plane” (e.g. etcd, the API server, controller manager, etc.) from the “data plane” (e.g. worker nodes and their kubelets, and the infrastructure on which they run). This enables “hosted control plane as a service” use cases.

HostedCluster

HostedCluster is the primary representation of a HyperShift cluster and encapsulates the control plane and common data plane configuration. Creating a HostedCluster results in a fully functional OpenShift control plane with no attached nodes. To support workloads (e.g. pods), a HostedCluster may have one or more associated NodePool resources.

Field Description
apiVersion string hypershift.openshift.io/v1beta1
kind string HostedCluster
metadata Kubernetes meta/v1.ObjectMeta Refer to the Kubernetes API documentation for the fields of the metadata field.
spec HostedClusterSpec

Spec is the desired behavior of the HostedCluster.



release Release

Release specifies the desired OCP release payload for the hosted cluster.

Updating this field will trigger a rollout of the control plane. The behavior of the rollout will be driven by the ControllerAvailabilityPolicy and InfrastructureAvailabilityPolicy.

clusterID string (Optional)

ClusterID uniquely identifies this cluster. This is expected to be an RFC4122 UUID value (xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx in hexadecimal values). As with a Kubernetes metadata.uid, this ID uniquely identifies this cluster in space and time. This value identifies the cluster in metrics pushed to telemetry and metrics produced by the control plane operators. If a value is not specified, an ID is generated. After initial creation, the value is immutable.

channel string (Optional)

channel is an identifier for explicitly requesting that a non-default set of updates be applied to this cluster. The default channel will be contain stable updates that are appropriate for production clusters.

infraID string (Optional)

InfraID is a globally unique identifier for the cluster. This identifier will be used to associate various cloud resources with the HostedCluster and its associated NodePools.

platform PlatformSpec

Platform specifies the underlying infrastructure provider for the cluster and is used to configure platform specific behavior.

controllerAvailabilityPolicy AvailabilityPolicy (Optional)

ControllerAvailabilityPolicy specifies the availability policy applied to critical control plane components. The default value is SingleReplica.

infrastructureAvailabilityPolicy AvailabilityPolicy (Optional)

InfrastructureAvailabilityPolicy specifies the availability policy applied to infrastructure services which run on cluster nodes. The default value is SingleReplica.

dns DNSSpec

DNS specifies DNS configuration for the cluster.

networking ClusterNetworking

Networking specifies network configuration for the cluster.

autoscaling ClusterAutoscaling (Optional)

Autoscaling specifies auto-scaling behavior that applies to all NodePools associated with the control plane.

etcd EtcdSpec

Etcd specifies configuration for the control plane etcd cluster. The default ManagementType is Managed. Once set, the ManagementType cannot be changed.

services []ServicePublishingStrategyMapping

Services specifies how individual control plane services are published from the hosting cluster of the control plane.

If a given service is not present in this list, it will be exposed publicly by default.

pullSecret Kubernetes core/v1.LocalObjectReference

PullSecret references a pull secret to be injected into the container runtime of all cluster nodes. The secret must have a key named “.dockerconfigjson” whose value is the pull secret JSON.

sshKey Kubernetes core/v1.LocalObjectReference

SSHKey references an SSH key to be injected into all cluster node sshd servers. The secret must have a single key “id_rsa.pub” whose value is the public part of an SSH key.

issuerURL string (Optional)

IssuerURL is an OIDC issuer URL which is used as the issuer in all ServiceAccount tokens generated by the control plane API server. The default value is kubernetes.default.svc, which only works for in-cluster validation.

serviceAccountSigningKey Kubernetes core/v1.LocalObjectReference (Optional)

ServiceAccountSigningKey is a reference to a secret containing the private key used by the service account token issuer. The secret is expected to contain a single key named “key”. If not specified, a service account signing key will be generated automatically for the cluster. When specifying a service account signing key, a IssuerURL must also be specified.

configuration ClusterConfiguration (Optional)

Configuration specifies configuration for individual OCP components in the cluster, represented as embedded resources that correspond to the openshift configuration API.

auditWebhook Kubernetes core/v1.LocalObjectReference (Optional)

AuditWebhook contains metadata for configuring an audit webhook endpoint for a cluster to process cluster audit events. It references a secret that contains the webhook information for the audit webhook endpoint. It is a secret because if the endpoint has mTLS the kubeconfig will contain client keys. The kubeconfig needs to be stored in the secret with a secret key name that corresponds to the constant AuditWebhookKubeconfigKey.

imageContentSources []ImageContentSource (Optional)

ImageContentSources specifies image mirrors that can be used by cluster nodes to pull content.

additionalTrustBundle Kubernetes core/v1.LocalObjectReference (Optional)

AdditionalTrustBundle is a reference to a ConfigMap containing a PEM-encoded X.509 certificate bundle that will be added to the hosted controlplane and nodes

secretEncryption SecretEncryptionSpec (Optional)

SecretEncryption specifies a Kubernetes secret encryption strategy for the control plane.

fips bool (Optional)

FIPS indicates whether this cluster’s nodes will be running in FIPS mode. If set to true, the control plane’s ignition server will be configured to expect that nodes joining the cluster will be FIPS-enabled.

pausedUntil string (Optional)

PausedUntil is a field that can be used to pause reconciliation on a resource. Either a date can be provided in RFC3339 format or a boolean. If a date is provided: reconciliation is paused on the resource until that date. If the boolean true is provided: reconciliation is paused on the resource until the field is removed.

olmCatalogPlacement OLMCatalogPlacement (Optional)

OLMCatalogPlacement specifies the placement of OLM catalog components. By default, this is set to management and OLM catalog components are deployed onto the management cluster. If set to guest, the OLM catalog components will be deployed onto the guest cluster.

nodeSelector map[string]string (Optional)

NodeSelector when specified, must be true for the pods managed by the HostedCluster to be scheduled.

status HostedClusterStatus

Status is the latest observed status of the HostedCluster.

NodePool

NodePool is a scalable set of worker nodes attached to a HostedCluster. NodePool machine architectures are uniform within a given pool, and are independent of the control plane’s underlying machine architecture.

Field Description
apiVersion string hypershift.openshift.io/v1beta1
kind string NodePool
metadata Kubernetes meta/v1.ObjectMeta Refer to the Kubernetes API documentation for the fields of the metadata field.
spec NodePoolSpec

Spec is the desired behavior of the NodePool.



clusterName string

ClusterName is the name of the HostedCluster this NodePool belongs to.

TODO(dan): Should this be a LocalObjectReference?

release Release

Release specifies the OCP release used for the NodePool. This informs the ignition configuration for machines, as well as other platform specific machine properties (e.g. an AMI on the AWS platform).

platform NodePoolPlatform

Platform specifies the underlying infrastructure provider for the NodePool and is used to configure platform specific behavior.

replicas int32 (Optional)

Replicas is the desired number of nodes the pool should maintain. If unset, the default value is 0.

management NodePoolManagement

Management specifies behavior for managing nodes in the pool, such as upgrade strategies and auto-repair behaviors.

autoScaling NodePoolAutoScaling (Optional)

Autoscaling specifies auto-scaling behavior for the NodePool.

config []Kubernetes core/v1.LocalObjectReference

Config is a list of references to ConfigMaps containing serialized MachineConfig resources to be injected into the ignition configurations of nodes in the NodePool. The MachineConfig API schema is defined here:

https://github.com/openshift/machine-config-operator/blob/18963e4f8fe66e8c513ca4b131620760a414997f/pkg/apis/machineconfiguration.openshift.io/v1/types.go#L185

Each ConfigMap must have a single key named “config” whose value is the JSON or YAML of a serialized Resource for machineconfiguration.openshift.io: KubeletConfig ContainerRuntimeConfig MachineConfig or ImageContentSourcePolicy

nodeDrainTimeout Kubernetes meta/v1.Duration (Optional)

NodeDrainTimeout is the total amount of time that the controller will spend on draining a node. The default value is 0, meaning that the node can be drained without any time limitations. NOTE: NodeDrainTimeout is different from kubectl drain --timeout TODO (alberto): Today changing this field will trigger a recreate rolling update, which kind of defeats the purpose of the change. In future we plan to propagate this field in-place. https://github.com/kubernetes-sigs/cluster-api/issues/5880

nodeLabels map[string]string (Optional)

NodeLabels propagates a list of labels to Nodes, only once on creation. Valid values are those in https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#syntax-and-character-set

taints []Taint (Optional)

Taints if specified, propagates a list of taints to Nodes, only once on creation.

pausedUntil string (Optional)

PausedUntil is a field that can be used to pause reconciliation on a resource. Either a date can be provided in RFC3339 format or a boolean. If a date is provided: reconciliation is paused on the resource until that date. If the boolean true is provided: reconciliation is paused on the resource until the field is removed.

tuningConfig []Kubernetes core/v1.LocalObjectReference

TuningConfig is a list of references to ConfigMaps containing serialized Tuned resources to define the tuning configuration to be applied to nodes in the NodePool. The Tuned API is defined here:

https://github.com/openshift/cluster-node-tuning-operator/blob/2c76314fb3cc8f12aef4a0dcd67ddc3677d5b54f/pkg/apis/tuned/v1/tuned_types.go

Each ConfigMap must have a single key named “tuned” whose value is the JSON or YAML of a serialized Tuned.

status NodePoolStatus

Status is the latest observed status of the NodePool.

AESCBCSpec

(Appears on: SecretEncryptionSpec)

AESCBCSpec defines metadata about the AESCBC secret encryption strategy

Field Description
activeKey Kubernetes core/v1.LocalObjectReference

ActiveKey defines the active key used to encrypt new secrets

backupKey Kubernetes core/v1.LocalObjectReference (Optional)

BackupKey defines the old key during the rotation process so previously created secrets can continue to be decrypted until they are all re-encrypted with the active key.

APIServerNetworking

(Appears on: ClusterNetworking)

APIServerNetworking specifies how the APIServer is exposed inside a cluster node.

Field Description
advertiseAddress string

AdvertiseAddress is the address that nodes will use to talk to the API server. This is an address associated with the loopback adapter of each node. If not specified, 172.20.0.1 is used.

port int32

Port is the port at which the APIServer is exposed inside a node. Other pods using host networking cannot listen on this port. If not specified, 6443 is used.

allowedCIDRBlocks []CIDRBlock

AllowedCIDRBlocks is an allow list of CIDR blocks that can access the APIServer If not specified, traffic is allowed from all addresses. This depends on underlying support by the cloud provider for Service LoadBalancerSourceRanges

AWSCloudProviderConfig

(Appears on: AWSPlatformSpec)

AWSCloudProviderConfig specifies AWS networking configuration.

Field Description
subnet AWSResourceReference (Optional)

Subnet is the subnet to use for control plane cloud resources.

zone string (Optional)

Zone is the availability zone where control plane cloud resources are created.

vpc string

VPC is the VPC to use for control plane cloud resources.

AWSEndpointAccessType

(Appears on: AWSPlatformSpec)

AWSEndpointAccessType specifies the publishing scope of cluster endpoints.

Value Description

"Private"

Private endpoint access allows only private API server access and private node communication with the control plane.

"Public"

Public endpoint access allows public API server access and public node communication with the control plane.

"PublicAndPrivate"

PublicAndPrivate endpoint access allows public API server access and private node communication with the control plane.

AWSKMSAuthSpec

(Appears on: AWSKMSSpec)

AWSKMSAuthSpec defines metadata about the management of credentials used to interact and encrypt data via AWS KMS key.

Field Description
awsKms string

The referenced role must have a trust relationship that allows it to be assumed via web identity. https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers_oidc.html. Example: { “Version”: “2012-10-17”, “Statement”: [ { “Effect”: “Allow”, “Principal”: { “Federated”: “{{ .ProviderARN }}” }, “Action”: “sts:AssumeRoleWithWebIdentity”, “Condition”: { “StringEquals”: { “{{ .ProviderName }}:sub”: {{ .ServiceAccounts }} } } } ] }

AWSKMSARN is an ARN value referencing a role appropriate for managing the auth via the AWS KMS key.

The following is an example of a valid policy document:

{ “Version”: “2012-10-17”, “Statement”: [ { “Effect”: “Allow”, “Action”: [ “kms:Encrypt”, “kms:Decrypt”, “kms:ReEncrypt”, “kms:GenerateDataKey”, “kms:DescribeKey” ], “Resource”: %q } ] }

AWSKMSKeyEntry

(Appears on: AWSKMSSpec)

AWSKMSKeyEntry defines metadata to locate the encryption key in AWS

Field Description
arn string

ARN is the Amazon Resource Name for the encryption key

AWSKMSSpec

(Appears on: KMSSpec)

AWSKMSSpec defines metadata about the configuration of the AWS KMS Secret Encryption provider

Field Description
region string

Region contains the AWS region

activeKey AWSKMSKeyEntry

ActiveKey defines the active key used to encrypt new secrets

backupKey AWSKMSKeyEntry (Optional)

BackupKey defines the old key during the rotation process so previously created secrets can continue to be decrypted until they are all re-encrypted with the active key.

auth AWSKMSAuthSpec

Auth defines metadata about the management of credentials used to interact with AWS KMS

AWSNodePoolPlatform

(Appears on: NodePoolPlatform)

AWSNodePoolPlatform specifies the configuration of a NodePool when operating on AWS.

Field Description
instanceType string

InstanceType is an ec2 instance type for node instances (e.g. m5.large).

instanceProfile string

InstanceProfile is the AWS EC2 instance profile, which is a container for an IAM role that the EC2 instance uses.

subnet AWSResourceReference (Optional)

Subnet is the subnet to use for node instances.

ami string (Optional)

AMI is the image id to use for node instances. If unspecified, the default is chosen based on the NodePool release payload image.

securityGroups []AWSResourceReference (Optional)

SecurityGroups is an optional set of security groups to associate with node instances.

rootVolume Volume (Optional)

RootVolume specifies configuration for the root volume of node instances.

resourceTags []AWSResourceTag (Optional)

ResourceTags is an optional list of additional tags to apply to AWS node instances.

These will be merged with HostedCluster scoped tags, and HostedCluster tags take precedence in case of conflicts.

See https://docs.aws.amazon.com/general/latest/gr/aws_tagging.html for information on tagging AWS resources. AWS supports a maximum of 50 tags per resource. OpenShift reserves 25 tags for its use, leaving 25 tags available for the user.

AWSPlatformSpec

(Appears on: PlatformSpec)

AWSPlatformSpec specifies configuration for clusters running on Amazon Web Services.

Field Description
region string

Region is the AWS region in which the cluster resides. This configures the OCP control plane cloud integrations, and is used by NodePool to resolve the correct boot AMI for a given release.

cloudProviderConfig AWSCloudProviderConfig (Optional)

CloudProviderConfig specifies AWS networking configuration for the control plane. This is mainly used for cloud provider controller config: https://github.com/kubernetes/kubernetes/blob/f5be5052e3d0808abb904aebd3218fe4a5c2dd82/staging/src/k8s.io/legacy-cloud-providers/aws/aws.go#L1347-L1364 TODO(dan): should this be named AWSNetworkConfig?

serviceEndpoints []AWSServiceEndpoint (Optional)

ServiceEndpoints specifies optional custom endpoints which will override the default service endpoint of specific AWS Services.

There must be only one ServiceEndpoint for a given service name.

rolesRef AWSRolesRef

RolesRef contains references to various AWS IAM roles required to enable integrations such as OIDC.

resourceTags []AWSResourceTag (Optional)

ResourceTags is a list of additional tags to apply to AWS resources created for the cluster. See https://docs.aws.amazon.com/general/latest/gr/aws_tagging.html for information on tagging AWS resources. AWS supports a maximum of 50 tags per resource. OpenShift reserves 25 tags for its use, leaving 25 tags available for the user.

endpointAccess AWSEndpointAccessType (Optional)

EndpointAccess specifies the publishing scope of cluster endpoints. The default is Public.

additionalAllowedPrincipals []string (Optional)

AdditionalAllowedPrincipals specifies a list of additional allowed principal ARNs to be added to the hosted control plane’s VPC Endpoint Service to enable additional VPC Endpoint connection requests to be automatically accepted. See https://docs.aws.amazon.com/vpc/latest/privatelink/configure-endpoint-service.html for more details around VPC Endpoint Service allowed principals.

AWSPlatformStatus

(Appears on: PlatformStatus)

AWSPlatformStatus contains status specific to the AWS platform

Field Description
defaultWorkerSecurityGroupID string (Optional)

DefaultWorkerSecurityGroupID is the ID of a security group created by the control plane operator. It is used for NodePools that don’t specify a security group.

AWSResourceReference

(Appears on: AWSCloudProviderConfig, AWSNodePoolPlatform)

AWSResourceReference is a reference to a specific AWS resource by ID or filters. Only one of ID or Filters may be specified. Specifying more than one will result in a validation error.

Field Description
id string (Optional)

ID of resource

filters []Filter (Optional)

Filters is a set of key/value pairs used to identify a resource They are applied according to the rules defined by the AWS API: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Using_Filtering.html

AWSResourceTag

(Appears on: AWSNodePoolPlatform, AWSPlatformSpec)

AWSResourceTag is a tag to apply to AWS resources created for the cluster.

Field Description
key string

Key is the key of the tag.

value string

Value is the value of the tag.

Some AWS service do not support empty values. Since tags are added to resources in many services, the length of the tag value must meet the requirements of all services.

AWSRoleCredentials

Field Description
arn string
namespace string
name string

AWSRolesRef

(Appears on: AWSPlatformSpec)

AWSRolesRef contains references to various AWS IAM roles required for operators to make calls against the AWS API.

Field Description
ingressARN string

The referenced role must have a trust relationship that allows it to be assumed via web identity. https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers_oidc.html. Example: { “Version”: “2012-10-17”, “Statement”: [ { “Effect”: “Allow”, “Principal”: { “Federated”: “{{ .ProviderARN }}” }, “Action”: “sts:AssumeRoleWithWebIdentity”, “Condition”: { “StringEquals”: { “{{ .ProviderName }}:sub”: {{ .ServiceAccounts }} } } } ] }

IngressARN is an ARN value referencing a role appropriate for the Ingress Operator.

The following is an example of a valid policy document:

{ “Version”: “2012-10-17”, “Statement”: [ { “Effect”: “Allow”, “Action”: [ “elasticloadbalancing:DescribeLoadBalancers”, “tag:GetResources”, “route53:ListHostedZones” ], “Resource”: “*” }, { “Effect”: “Allow”, “Action”: [ “route53:ChangeResourceRecordSets” ], “Resource”: [ “arn:aws:route53:::PUBLIC_ZONE_ID”, “arn:aws:route53:::PRIVATE_ZONE_ID” ] } ] }

imageRegistryARN string

ImageRegistryARN is an ARN value referencing a role appropriate for the Image Registry Operator.

The following is an example of a valid policy document:

{ “Version”: “2012-10-17”, “Statement”: [ { “Effect”: “Allow”, “Action”: [ “s3:CreateBucket”, “s3:DeleteBucket”, “s3:PutBucketTagging”, “s3:GetBucketTagging”, “s3:PutBucketPublicAccessBlock”, “s3:GetBucketPublicAccessBlock”, “s3:PutEncryptionConfiguration”, “s3:GetEncryptionConfiguration”, “s3:PutLifecycleConfiguration”, “s3:GetLifecycleConfiguration”, “s3:GetBucketLocation”, “s3:ListBucket”, “s3:GetObject”, “s3:PutObject”, “s3:DeleteObject”, “s3:ListBucketMultipartUploads”, “s3:AbortMultipartUpload”, “s3:ListMultipartUploadParts” ], “Resource”: “*” } ] }

storageARN string

StorageARN is an ARN value referencing a role appropriate for the Storage Operator.

The following is an example of a valid policy document:

{ “Version”: “2012-10-17”, “Statement”: [ { “Effect”: “Allow”, “Action”: [ “ec2:AttachVolume”, “ec2:CreateSnapshot”, “ec2:CreateTags”, “ec2:CreateVolume”, “ec2:DeleteSnapshot”, “ec2:DeleteTags”, “ec2:DeleteVolume”, “ec2:DescribeInstances”, “ec2:DescribeSnapshots”, “ec2:DescribeTags”, “ec2:DescribeVolumes”, “ec2:DescribeVolumesModifications”, “ec2:DetachVolume”, “ec2:ModifyVolume” ], “Resource”: “*” } ] }

networkARN string

NetworkARN is an ARN value referencing a role appropriate for the Network Operator.

The following is an example of a valid policy document:

{ “Version”: “2012-10-17”, “Statement”: [ { “Effect”: “Allow”, “Action”: [ “ec2:DescribeInstances”, “ec2:DescribeInstanceStatus”, “ec2:DescribeInstanceTypes”, “ec2:UnassignPrivateIpAddresses”, “ec2:AssignPrivateIpAddresses”, “ec2:UnassignIpv6Addresses”, “ec2:AssignIpv6Addresses”, “ec2:DescribeSubnets”, “ec2:DescribeNetworkInterfaces” ], “Resource”: “*” } ] }

kubeCloudControllerARN string

KubeCloudControllerARN is an ARN value referencing a role appropriate for the KCM/KCC.

The following is an example of a valid policy document:

{ “Version”: “2012-10-17”, “Statement”: [ { “Action”: [ “ec2:DescribeInstances”, “ec2:DescribeImages”, “ec2:DescribeRegions”, “ec2:DescribeRouteTables”, “ec2:DescribeSecurityGroups”, “ec2:DescribeSubnets”, “ec2:DescribeVolumes”, “ec2:CreateSecurityGroup”, “ec2:CreateTags”, “ec2:CreateVolume”, “ec2:ModifyInstanceAttribute”, “ec2:ModifyVolume”, “ec2:AttachVolume”, “ec2:AuthorizeSecurityGroupIngress”, “ec2:CreateRoute”, “ec2:DeleteRoute”, “ec2:DeleteSecurityGroup”, “ec2:DeleteVolume”, “ec2:DetachVolume”, “ec2:RevokeSecurityGroupIngress”, “ec2:DescribeVpcs”, “elasticloadbalancing:AddTags”, “elasticloadbalancing:AttachLoadBalancerToSubnets”, “elasticloadbalancing:ApplySecurityGroupsToLoadBalancer”, “elasticloadbalancing:CreateLoadBalancer”, “elasticloadbalancing:CreateLoadBalancerPolicy”, “elasticloadbalancing:CreateLoadBalancerListeners”, “elasticloadbalancing:ConfigureHealthCheck”, “elasticloadbalancing:DeleteLoadBalancer”, “elasticloadbalancing:DeleteLoadBalancerListeners”, “elasticloadbalancing:DescribeLoadBalancers”, “elasticloadbalancing:DescribeLoadBalancerAttributes”, “elasticloadbalancing:DetachLoadBalancerFromSubnets”, “elasticloadbalancing:DeregisterInstancesFromLoadBalancer”, “elasticloadbalancing:ModifyLoadBalancerAttributes”, “elasticloadbalancing:RegisterInstancesWithLoadBalancer”, “elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer”, “elasticloadbalancing:AddTags”, “elasticloadbalancing:CreateListener”, “elasticloadbalancing:CreateTargetGroup”, “elasticloadbalancing:DeleteListener”, “elasticloadbalancing:DeleteTargetGroup”, “elasticloadbalancing:DescribeListeners”, “elasticloadbalancing:DescribeLoadBalancerPolicies”, “elasticloadbalancing:DescribeTargetGroups”, “elasticloadbalancing:DescribeTargetHealth”, “elasticloadbalancing:ModifyListener”, “elasticloadbalancing:ModifyTargetGroup”, “elasticloadbalancing:RegisterTargets”, “elasticloadbalancing:SetLoadBalancerPoliciesOfListener”, “iam:CreateServiceLinkedRole”, “kms:DescribeKey” ], “Resource”: [ “*” ], “Effect”: “Allow” } ] }

nodePoolManagementARN string

NodePoolManagementARN is an ARN value referencing a role appropriate for the CAPI Controller.

The following is an example of a valid policy document:

{ “Version”: “2012-10-17”, “Statement”: [ { “Action”: [ “ec2:AllocateAddress”, “ec2:AssociateRouteTable”, “ec2:AttachInternetGateway”, “ec2:AuthorizeSecurityGroupIngress”, “ec2:CreateInternetGateway”, “ec2:CreateNatGateway”, “ec2:CreateRoute”, “ec2:CreateRouteTable”, “ec2:CreateSecurityGroup”, “ec2:CreateSubnet”, “ec2:CreateTags”, “ec2:DeleteInternetGateway”, “ec2:DeleteNatGateway”, “ec2:DeleteRouteTable”, “ec2:DeleteSecurityGroup”, “ec2:DeleteSubnet”, “ec2:DeleteTags”, “ec2:DescribeAccountAttributes”, “ec2:DescribeAddresses”, “ec2:DescribeAvailabilityZones”, “ec2:DescribeImages”, “ec2:DescribeInstances”, “ec2:DescribeInternetGateways”, “ec2:DescribeNatGateways”, “ec2:DescribeNetworkInterfaces”, “ec2:DescribeNetworkInterfaceAttribute”, “ec2:DescribeRouteTables”, “ec2:DescribeSecurityGroups”, “ec2:DescribeSubnets”, “ec2:DescribeVpcs”, “ec2:DescribeVpcAttribute”, “ec2:DescribeVolumes”, “ec2:DetachInternetGateway”, “ec2:DisassociateRouteTable”, “ec2:DisassociateAddress”, “ec2:ModifyInstanceAttribute”, “ec2:ModifyNetworkInterfaceAttribute”, “ec2:ModifySubnetAttribute”, “ec2:ReleaseAddress”, “ec2:RevokeSecurityGroupIngress”, “ec2:RunInstances”, “ec2:TerminateInstances”, “tag:GetResources”, “ec2:CreateLaunchTemplate”, “ec2:CreateLaunchTemplateVersion”, “ec2:DescribeLaunchTemplates”, “ec2:DescribeLaunchTemplateVersions”, “ec2:DeleteLaunchTemplate”, “ec2:DeleteLaunchTemplateVersions” ], “Resource”: [ “” ], “Effect”: “Allow” }, { “Condition”: { “StringLike”: { “iam:AWSServiceName”: “elasticloadbalancing.amazonaws.com” } }, “Action”: [ “iam:CreateServiceLinkedRole” ], “Resource”: [ “arn::iam:::role/aws-service-role/elasticloadbalancing.amazonaws.com/AWSServiceRoleForElasticLoadBalancing” ], “Effect”: “Allow” }, { “Action”: [ “iam:PassRole” ], “Resource”: [ “arn::iam:::role/-worker-role” ], “Effect”: “Allow” }, { “Effect”: “Allow”, “Action”: [ “kms:Decrypt”, “kms:Encrypt”, “kms:GenerateDataKey”, “kms:GenerateDataKeyWithoutPlainText”, “kms:DescribeKey” ], “Resource”: “” }, { “Effect”: “Allow”, “Action”: [ “kms:RevokeGrant”, “kms:CreateGrant”, “kms:ListGrants” ], “Resource”: “”, “Condition”: { “Bool”: { “kms:GrantIsForAWSResource”: true } } } ] }

controlPlaneOperatorARN string

ControlPlaneOperatorARN is an ARN value referencing a role appropriate for the Control Plane Operator.

The following is an example of a valid policy document:

{ “Version”: “2012-10-17”, “Statement”: [ { “Effect”: “Allow”, “Action”: [ “ec2:CreateVpcEndpoint”, “ec2:DescribeVpcEndpoints”, “ec2:ModifyVpcEndpoint”, “ec2:DeleteVpcEndpoints”, “ec2:CreateTags”, “route53:ListHostedZones”, “ec2:CreateSecurityGroup”, “ec2:AuthorizeSecurityGroupIngress”, “ec2:AuthorizeSecurityGroupEgress”, “ec2:DeleteSecurityGroup”, “ec2:RevokeSecurityGroupIngress”, “ec2:RevokeSecurityGroupEgress”, “ec2:DescribeSecurityGroups”, “ec2:DescribeVpcs”, ], “Resource”: “*” }, { “Effect”: “Allow”, “Action”: [ “route53:ChangeResourceRecordSets”, “route53:ListResourceRecordSets” ], “Resource”: “arn:aws:route53:::%s” } ] }

AWSServiceEndpoint

(Appears on: AWSPlatformSpec)

AWSServiceEndpoint stores the configuration for services to override existing defaults of AWS Services.

Field Description
name string

Name is the name of the AWS service. This must be provided and cannot be empty.

url string

URL is fully qualified URI with scheme https, that overrides the default generated endpoint for a client. This must be provided and cannot be empty.

AgentNodePoolPlatform

(Appears on: NodePoolPlatform)

AgentNodePoolPlatform specifies the configuration of a NodePool when operating on the Agent platform.

Field Description
agentLabelSelector Kubernetes meta/v1.LabelSelector (Optional)

AgentLabelSelector contains labels that must be set on an Agent in order to be selected for a Machine.

AgentPlatformSpec

(Appears on: PlatformSpec)

AgentPlatformSpec specifies configuration for agent-based installations.

Field Description
agentNamespace string

AgentNamespace is the namespace where to search for Agents for this cluster

AvailabilityPolicy

(Appears on: HostedClusterSpec, HostedControlPlaneSpec)

AvailabilityPolicy specifies a high level availability policy for components.

Value Description

"HighlyAvailable"

HighlyAvailable means components should be resilient to problems across fault boundaries as defined by the component to which the policy is attached. This usually means running critical workloads with 3 replicas and with little or no toleration of disruption of the component.

"SingleReplica"

SingleReplica means components are not expected to be resilient to problems across most fault boundaries associated with high availability. This usually means running critical workloads with just 1 replica and with toleration of full disruption of the component.

AzureNodePoolPlatform

(Appears on: NodePoolPlatform)

Field Description
vmsize string
imageID string (Optional)

ImageID is the id of the image to boot from. If unset, the default image at the location below will be used: subscription/$subscriptionID/resourceGroups/$resourceGroupName/providers/Microsoft.Compute/images/rhcos.x86_64.vhd

diskSizeGB int32 (Optional)
diskStorageAccountType string (Optional)

DiskStorageAccountType is the disk storage account type to use. Valid values are: * Standard_LRS: HDD * StandardSSD_LRS: Standard SSD * Premium_LRS: Premium SDD * UltraSSD_LRS: Ultra SDD

Defaults to Premium_LRS. For more details, visit the Azure documentation: https://docs.microsoft.com/en-us/azure/virtual-machines/disks-types#disk-type-comparison

availabilityZone string (Optional)

AvailabilityZone of the nodepool. Must not be specified for clusters in a location that does not support AvailabilityZone.

AzurePlatformSpec

(Appears on: PlatformSpec)

Field Description
credentials Kubernetes core/v1.LocalObjectReference
location string
resourceGroup string
vnetName string
vnetID string
subnetName string
subscriptionID string
machineIdentityID string
securityGroupName string

CIDRBlock

(Appears on: APIServerNetworking)

ClusterAutoscaling

(Appears on: HostedClusterSpec, HostedControlPlaneSpec)

ClusterAutoscaling specifies auto-scaling behavior that applies to all NodePools associated with a control plane.

Field Description
maxNodesTotal int32

MaxNodesTotal is the maximum allowable number of nodes across all NodePools for a HostedCluster. The autoscaler will not grow the cluster beyond this number.

maxPodGracePeriod int32

MaxPodGracePeriod is the maximum seconds to wait for graceful pod termination before scaling down a NodePool. The default is 600 seconds.

maxNodeProvisionTime string

MaxNodeProvisionTime is the maximum time to wait for node provisioning before considering the provisioning to be unsuccessful, expressed as a Go duration string. The default is 15 minutes.

podPriorityThreshold int32 (Optional)

PodPriorityThreshold enables users to schedule “best-effort” pods, which shouldn’t trigger autoscaler actions, but only run when there are spare resources available. The default is -10.

See the following for more details: https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/FAQ.md#how-does-cluster-autoscaler-work-with-pod-priority-and-preemption

ClusterConfiguration

(Appears on: HostedClusterSpec, HostedControlPlaneSpec)

ClusterConfiguration specifies configuration for individual OCP components in the cluster, represented as embedded resources that correspond to the openshift configuration API.

The API for individual configuration items is at: https://docs.openshift.com/container-platform/4.7/rest_api/config_apis/config-apis-index.html

Field Description
apiServer github.com/openshift/api/config/v1.APIServerSpec (Optional)

APIServer holds configuration (like serving certificates, client CA and CORS domains) shared by all API servers in the system, among them especially kube-apiserver and openshift-apiserver.

authentication github.com/openshift/api/config/v1.AuthenticationSpec (Optional)

Authentication specifies cluster-wide settings for authentication (like OAuth and webhook token authenticators).

featureGate github.com/openshift/api/config/v1.FeatureGateSpec (Optional)

FeatureGate holds cluster-wide information about feature gates.

image github.com/openshift/api/config/v1.ImageSpec (Optional)

Image governs policies related to imagestream imports and runtime configuration for external registries. It allows cluster admins to configure which registries OpenShift is allowed to import images from, extra CA trust bundles for external registries, and policies to block or allow registry hostnames. When exposing OpenShift’s image registry to the public, this also lets cluster admins specify the external hostname.

ingress github.com/openshift/api/config/v1.IngressSpec (Optional)

Ingress holds cluster-wide information about ingress, including the default ingress domain used for routes.

network github.com/openshift/api/config/v1.NetworkSpec (Optional)

Network holds cluster-wide information about the network. It is used to configure the desired network configuration, such as: IP address pools for services/pod IPs, network plugin, etc. Please view network.spec for an explanation on what applies when configuring this resource. TODO (csrwng): Add validation here to exclude changes that conflict with networking settings in the HostedCluster.Spec.Networking field.

oauth github.com/openshift/api/config/v1.OAuthSpec (Optional)

OAuth holds cluster-wide information about OAuth. It is used to configure the integrated OAuth server. This configuration is only honored when the top level Authentication config has type set to IntegratedOAuth.

scheduler github.com/openshift/api/config/v1.SchedulerSpec (Optional)

Scheduler holds cluster-wide config information to run the Kubernetes Scheduler and influence its placement decisions. The canonical name for this config is cluster.

proxy github.com/openshift/api/config/v1.ProxySpec (Optional)

Proxy holds cluster-wide information on how to configure default proxies for the cluster.

ClusterNetworkEntry

(Appears on: ClusterNetworking)

ClusterNetworkEntry is a single IP address block for pod IP blocks. IP blocks are allocated with size 2^HostSubnetLength.

Field Description
cidr github.com/openshift/hypershift/api/util/ipnet.IPNet

CIDR is the IP block address pool.

hostPrefix int32 (Optional)

HostPrefix is the prefix size to allocate to each node from the CIDR. For example, 24 would allocate 2^8=256 adresses to each node. If this field is not used by the plugin, it can be left unset.

ClusterNetworking

(Appears on: HostedClusterSpec, HostedControlPlaneSpec)

ClusterNetworking specifies network configuration for a cluster.

Field Description
machineNetwork []MachineNetworkEntry (Optional)

MachineNetwork is the list of IP address pools for machines.

clusterNetwork []ClusterNetworkEntry

ClusterNetwork is the list of IP address pools for pods.

serviceNetwork []ServiceNetworkEntry (Optional)

ServiceNetwork is the list of IP address pools for services. NOTE: currently only one entry is supported.

networkType NetworkType

NetworkType specifies the SDN provider used for cluster networking.

apiServer APIServerNetworking

APIServer contains advanced network settings for the API server that affect how the APIServer is exposed inside a cluster node.

ClusterVersionStatus

(Appears on: HostedClusterStatus, HostedControlPlaneStatus)

ClusterVersionStatus reports the status of the cluster versioning, including any upgrades that are in progress. The current field will be set to whichever version the cluster is reconciling to, and the conditions array will report whether the update succeeded, is in progress, or is failing.

Field Description
desired github.com/openshift/api/config/v1.Release

desired is the version that the cluster is reconciling towards. If the cluster is not yet fully initialized desired will be set with the information available, which may be an image or a tag.

history []github.com/openshift/api/config/v1.UpdateHistory (Optional)

history contains a list of the most recent versions applied to the cluster. This value may be empty during cluster startup, and then will be updated when a new update is being applied. The newest update is first in the list and it is ordered by recency. Updates in the history have state Completed if the rollout completed - if an update was failing or halfway applied the state will be Partial. Only a limited amount of update history is preserved.

observedGeneration int64

observedGeneration reports which version of the spec is being synced. If this value is not equal to metadata.generation, then the desired and conditions fields may represent a previous version.

availableUpdates []github.com/openshift/api/config/v1.Release

availableUpdates contains updates recommended for this cluster. Updates which appear in conditionalUpdates but not in availableUpdates may expose this cluster to known issues. This list may be empty if no updates are recommended, if the update service is unavailable, or if an invalid channel has been specified.

conditionalUpdates []github.com/openshift/api/config/v1.ConditionalUpdate (Optional)

conditionalUpdates contains the list of updates that may be recommended for this cluster if it meets specific required conditions. Consumers interested in the set of updates that are actually recommended for this cluster should use availableUpdates. This list may be empty if no updates are recommended, if the update service is unavailable, or if an empty or invalid channel has been specified.

ConditionType

Value Description

"AWSDefaultSecurityGroupCreated"

AWSDefaultSecurityGroupCreated indicates whether the default security group for AWS workers has been created. A failure here indicates that NodePools without a security group will be blocked from creating machines.

"EndpointAvailable"

AWSEndpointServiceAvailable indicates whether the AWS Endpoint has been created in the guest VPC

"EndpointServiceAvailable"

AWSEndpointServiceAvailable indicates whether the AWS Endpoint Service has been created for the specified NLB in the management VPC

"CVOScaledDown"

"CloudResourcesDestroyed"

CloudResourcesDestroyed bubbles up the same condition from HCP. It signals if the cloud provider infrastructure created by Kubernetes in the consumer cloud provider account was destroyed. A failure here may require external user intervention to resolve. E.g. cloud provider perms were corrupted. E.g. the guest cluster was broken and kube resource deletion that affects cloud infra like service type load balancer can’t succeed.

"ClusterVersionAvailable"

ClusterVersionAvailable bubbles up Failing configv1.OperatorAvailable from the CVO.

"ClusterVersionFailing"

ClusterVersionFailing bubbles up Failing from the CVO.

"ClusterVersionProgressing"

ClusterVersionProgressing bubbles up configv1.OperatorProgressing from the CVO.

"ClusterVersionReleaseAccepted"

ClusterVersionReleaseAccepted bubbles up Failing ReleaseAccepted from the CVO.

"ClusterVersionSucceeding"

ClusterVersionSucceeding indicates the current status of the desired release version of the HostedCluster as indicated by the Failing condition in the underlying cluster’s ClusterVersion.

"ClusterVersionUpgradeable"

ClusterVersionUpgradeable indicates the Upgradeable condition in the underlying cluster’s ClusterVersion.

"EtcdAvailable"

EtcdAvailable bubbles up the same condition from HCP. It signals if etcd is available. A failure here often means a software bug or a non-stable cluster.

"EtcdSnapshotRestored"

"ExternalDNSReachable"

ExternalDNSReachable bubbles up the same condition from HCP. It signals if the configured external DNS is reachable. A failure here requires external user intervention to resolve. E.g. changing the external DNS domain or making sure the domain is created and registered correctly.

"Available"

HostedClusterAvailable indicates whether the HostedCluster has a healthy control plane. When this is false for too long and there’s no clear indication in the “Reason”, please check the remaining more granular conditions.

"Degraded"

HostedClusterDegraded indicates whether the HostedCluster is encountering an error that may require user intervention to resolve.

"Progressing"

HostedClusterProgressing indicates whether the HostedCluster is attempting an initial deployment or upgrade. When this is false for too long and there’s no clear indication in the “Reason”, please check the remaining more granular conditions.

"Available"

"Degraded"

"IgnitionEndpointAvailable"

IgnitionEndpointAvailable indicates whether the ignition server for the HostedCluster is available to handle ignition requests. A failure here often means a software bug or a non-stable cluster.

"InfrastructureReady"

InfrastructureReady bubbles up the same condition from HCP. It signals if the infrastructure for a control plane to be operational, e.g. load balancers were created successfully. A failure here may require external user intervention to resolve. E.g. hitting quotas on the cloud provider.

"KubeAPIServerAvailable"

KubeAPIServerAvailable bubbles up the same condition from HCP. It signals if the kube API server is available. A failure here often means a software bug or a non-stable cluster.

"PlatformCredentialsFound"

PlatformCredentialsFound indicates that credentials required for the desired platform are valid. A failure here is unlikely to resolve without the changing user input.

"ReconciliationActive"

ReconciliationActive indicates if reconciliation of the HostedCluster is active or paused hostedCluster.spec.pausedUntil.

"ReconciliationSucceeded"

ReconciliationSucceeded indicates if the HostedCluster reconciliation succeeded. A failure here often means a software bug or a non-stable cluster.

"SupportedHostedCluster"

SupportedHostedCluster indicates whether a HostedCluster is supported by the current configuration of the hypershift-operator. e.g. If HostedCluster requests endpointAcess Private but the hypershift-operator is running on a management cluster outside AWS or is not configured with AWS credentials, the HostedCluster is not supported. A failure here is unlikely to resolve without the changing user input.

"UnmanagedEtcdAvailable"

UnmanagedEtcdAvailable indicates whether a user-managed etcd cluster is healthy.

"ValidAWSIdentityProvider"

ValidAWSIdentityProvider indicates if the Identity Provider referenced in the cloud credentials is healthy. E.g. for AWS the idp ARN is referenced in the iam roles. “Version”: “2012-10-17”, “Statement”: [ { “Effect”: “Allow”, “Principal”: { “Federated”: “{{ .ProviderARN }}” }, “Action”: “sts:AssumeRoleWithWebIdentity”, “Condition”: { “StringEquals”: { “{{ .ProviderName }}:sub”: {{ .ServiceAccounts }} } } } ]

A failure here may require external user intervention to resolve.

"ValidAWSKMSConfig"

ValidAWSKMSConfig indicates whether the AWS KMS role and encryption key are valid and operational A failure here indicates that the role or the key are invalid, or the role doesn’t have access to use the key.

"ValidConfiguration"

ValidHostedClusterConfiguration signals if the hostedCluster input is valid and supported by the underlying management cluster. A failure here is unlikely to resolve without the changing user input.

"ValidHostedControlPlaneConfiguration"

ValidHostedControlPlaneConfiguration bubbles up the same condition from HCP. It signals if the hostedControlPlane input is valid and supported by the underlying management cluster. A failure here is unlikely to resolve without the changing user input.

"ValidOIDCConfiguration"

ValidOIDCConfiguration indicates if an AWS cluster’s OIDC condition is detected as invalid. A failure here may require external user intervention to resolve. E.g. oidc was deleted out of band.

"ValidReleaseImage"

ValidReleaseImage indicates if the release image set in the spec is valid for the HostedCluster. For example, this can be set false if the HostedCluster itself attempts an unsupported version before 4.9 or an unsupported upgrade e.g y-stream upgrade before 4.11. A failure here is unlikely to resolve without the changing user input.

DNSSpec

(Appears on: HostedClusterSpec, HostedControlPlaneSpec)

DNSSpec specifies the DNS configuration in the cluster.

Field Description
baseDomain string

BaseDomain is the base domain of the cluster.

baseDomainPrefix string (Optional)

BaseDomainPrefix is the base domain prefix of the cluster. defaults to clusterName if not set. Set it to “” if you don’t want a prefix to be prepended to BaseDomain.

publicZoneID string (Optional)

PublicZoneID is the Hosted Zone ID where all the DNS records that are publicly accessible to the internet exist.

privateZoneID string (Optional)

PrivateZoneID is the Hosted Zone ID where all the DNS records that are only available internally to the cluster exist.

EtcdManagementType

(Appears on: EtcdSpec)

EtcdManagementType is a enum specifying the strategy for managing the cluster’s etcd instance

Value Description

"Managed"

Managed means HyperShift should provision and operator the etcd cluster automatically.

"Unmanaged"

Unmanaged means HyperShift will not provision or manage the etcd cluster, and the user is responsible for doing so.

EtcdSpec

(Appears on: HostedClusterSpec, HostedControlPlaneSpec)

EtcdSpec specifies configuration for a control plane etcd cluster.

Field Description
managementType EtcdManagementType

ManagementType defines how the etcd cluster is managed.

managed ManagedEtcdSpec (Optional)

Managed specifies the behavior of an etcd cluster managed by HyperShift.

unmanaged UnmanagedEtcdSpec (Optional)

Unmanaged specifies configuration which enables the control plane to integrate with an eternally managed etcd cluster.

EtcdTLSConfig

(Appears on: UnmanagedEtcdSpec)

EtcdTLSConfig specifies TLS configuration for HTTPS etcd client endpoints.

Field Description
clientSecret Kubernetes core/v1.LocalObjectReference

ClientSecret refers to a secret for client mTLS authentication with the etcd cluster. It may have the following key/value pairs:

etcd-client-ca.crt: Certificate Authority value
etcd-client.crt: Client certificate value
etcd-client.key: Client certificate key value

Filter

(Appears on: AWSResourceReference)

Filter is a filter used to identify an AWS resource

Field Description
name string

Name of the filter. Filter names are case-sensitive.

values []string

Values includes one or more filter values. Filter values are case-sensitive.

HostedClusterSpec

(Appears on: HostedCluster)

HostedClusterSpec is the desired behavior of a HostedCluster.

Field Description
release Release

Release specifies the desired OCP release payload for the hosted cluster.

Updating this field will trigger a rollout of the control plane. The behavior of the rollout will be driven by the ControllerAvailabilityPolicy and InfrastructureAvailabilityPolicy.

clusterID string (Optional)

ClusterID uniquely identifies this cluster. This is expected to be an RFC4122 UUID value (xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx in hexadecimal values). As with a Kubernetes metadata.uid, this ID uniquely identifies this cluster in space and time. This value identifies the cluster in metrics pushed to telemetry and metrics produced by the control plane operators. If a value is not specified, an ID is generated. After initial creation, the value is immutable.

channel string (Optional)

channel is an identifier for explicitly requesting that a non-default set of updates be applied to this cluster. The default channel will be contain stable updates that are appropriate for production clusters.

infraID string (Optional)

InfraID is a globally unique identifier for the cluster. This identifier will be used to associate various cloud resources with the HostedCluster and its associated NodePools.

platform PlatformSpec

Platform specifies the underlying infrastructure provider for the cluster and is used to configure platform specific behavior.

controllerAvailabilityPolicy AvailabilityPolicy (Optional)

ControllerAvailabilityPolicy specifies the availability policy applied to critical control plane components. The default value is SingleReplica.

infrastructureAvailabilityPolicy AvailabilityPolicy (Optional)

InfrastructureAvailabilityPolicy specifies the availability policy applied to infrastructure services which run on cluster nodes. The default value is SingleReplica.

dns DNSSpec

DNS specifies DNS configuration for the cluster.

networking ClusterNetworking

Networking specifies network configuration for the cluster.

autoscaling ClusterAutoscaling (Optional)

Autoscaling specifies auto-scaling behavior that applies to all NodePools associated with the control plane.

etcd EtcdSpec

Etcd specifies configuration for the control plane etcd cluster. The default ManagementType is Managed. Once set, the ManagementType cannot be changed.

services []ServicePublishingStrategyMapping

Services specifies how individual control plane services are published from the hosting cluster of the control plane.

If a given service is not present in this list, it will be exposed publicly by default.

pullSecret Kubernetes core/v1.LocalObjectReference

PullSecret references a pull secret to be injected into the container runtime of all cluster nodes. The secret must have a key named “.dockerconfigjson” whose value is the pull secret JSON.

sshKey Kubernetes core/v1.LocalObjectReference

SSHKey references an SSH key to be injected into all cluster node sshd servers. The secret must have a single key “id_rsa.pub” whose value is the public part of an SSH key.

issuerURL string (Optional)

IssuerURL is an OIDC issuer URL which is used as the issuer in all ServiceAccount tokens generated by the control plane API server. The default value is kubernetes.default.svc, which only works for in-cluster validation.

serviceAccountSigningKey Kubernetes core/v1.LocalObjectReference (Optional)

ServiceAccountSigningKey is a reference to a secret containing the private key used by the service account token issuer. The secret is expected to contain a single key named “key”. If not specified, a service account signing key will be generated automatically for the cluster. When specifying a service account signing key, a IssuerURL must also be specified.

configuration ClusterConfiguration (Optional)

Configuration specifies configuration for individual OCP components in the cluster, represented as embedded resources that correspond to the openshift configuration API.

auditWebhook Kubernetes core/v1.LocalObjectReference (Optional)

AuditWebhook contains metadata for configuring an audit webhook endpoint for a cluster to process cluster audit events. It references a secret that contains the webhook information for the audit webhook endpoint. It is a secret because if the endpoint has mTLS the kubeconfig will contain client keys. The kubeconfig needs to be stored in the secret with a secret key name that corresponds to the constant AuditWebhookKubeconfigKey.

imageContentSources []ImageContentSource (Optional)

ImageContentSources specifies image mirrors that can be used by cluster nodes to pull content.

additionalTrustBundle Kubernetes core/v1.LocalObjectReference (Optional)

AdditionalTrustBundle is a reference to a ConfigMap containing a PEM-encoded X.509 certificate bundle that will be added to the hosted controlplane and nodes

secretEncryption SecretEncryptionSpec (Optional)

SecretEncryption specifies a Kubernetes secret encryption strategy for the control plane.

fips bool (Optional)

FIPS indicates whether this cluster’s nodes will be running in FIPS mode. If set to true, the control plane’s ignition server will be configured to expect that nodes joining the cluster will be FIPS-enabled.

pausedUntil string (Optional)

PausedUntil is a field that can be used to pause reconciliation on a resource. Either a date can be provided in RFC3339 format or a boolean. If a date is provided: reconciliation is paused on the resource until that date. If the boolean true is provided: reconciliation is paused on the resource until the field is removed.

olmCatalogPlacement OLMCatalogPlacement (Optional)

OLMCatalogPlacement specifies the placement of OLM catalog components. By default, this is set to management and OLM catalog components are deployed onto the management cluster. If set to guest, the OLM catalog components will be deployed onto the guest cluster.

nodeSelector map[string]string (Optional)

NodeSelector when specified, must be true for the pods managed by the HostedCluster to be scheduled.

HostedClusterStatus

(Appears on: HostedCluster)

HostedClusterStatus is the latest observed status of a HostedCluster.

Field Description
version ClusterVersionStatus (Optional)

Version is the status of the release version applied to the HostedCluster.

kubeconfig Kubernetes core/v1.LocalObjectReference (Optional)

KubeConfig is a reference to the secret containing the default kubeconfig for the cluster.

kubeadminPassword Kubernetes core/v1.LocalObjectReference (Optional)

KubeadminPassword is a reference to the secret that contains the initial kubeadmin user password for the guest cluster.

ignitionEndpoint string (Optional)

IgnitionEndpoint is the endpoint injected in the ign config userdata. It exposes the config for instances to become kubernetes nodes.

controlPlaneEndpoint APIEndpoint

ControlPlaneEndpoint contains the endpoint information by which external clients can access the control plane. This is populated after the infrastructure is ready.

oauthCallbackURLTemplate string

OAuthCallbackURLTemplate contains a template for the URL to use as a callback for identity providers. The [identity-provider-name] placeholder must be replaced with the name of an identity provider defined on the HostedCluster. This is populated after the infrastructure is ready.

conditions []Kubernetes meta/v1.Condition (Optional)

Conditions represents the latest available observations of a control plane’s current state.

platform PlatformStatus (Optional)

Platform contains platform-specific status of the HostedCluster

HostedControlPlaneSpec

HostedControlPlaneSpec defines the desired state of HostedControlPlane

Field Description
releaseImage string

ReleaseImage is the release image applied to the hosted control plane.

channel string (Optional)

channel is an identifier for explicitly requesting that a non-default set of updates be applied to this cluster. The default channel will be contain stable updates that are appropriate for production clusters.

pullSecret Kubernetes core/v1.LocalObjectReference
issuerURL string

IssuerURL is an OIDC issuer URL which is used as the issuer in all ServiceAccount tokens generated by the control plane API server. The default value is kubernetes.default.svc, which only works for in-cluster validation.

networking ClusterNetworking (Optional)

Networking specifies network configuration for the cluster. Temporarily optional for backward compatibility, required in future releases.

sshKey Kubernetes core/v1.LocalObjectReference
clusterID string (Optional)

ClusterID is the unique id that identifies the cluster externally. Making it optional here allows us to keep compatibility with previous versions of the control-plane-operator that have no knowledge of this field.

infraID string
platform PlatformSpec
dns DNSSpec
serviceAccountSigningKey Kubernetes core/v1.LocalObjectReference (Optional)

ServiceAccountSigningKey is a reference to a secret containing the private key used by the service account token issuer. The secret is expected to contain a single key named “key”. If not specified, a service account signing key will be generated automatically for the cluster.

controllerAvailabilityPolicy AvailabilityPolicy (Optional)

ControllerAvailabilityPolicy specifies the availability policy applied to critical control plane components. The default value is SingleReplica.

infrastructureAvailabilityPolicy AvailabilityPolicy (Optional)

InfrastructureAvailabilityPolicy specifies the availability policy applied to infrastructure services which run on cluster nodes. The default value is SingleReplica.

fips bool (Optional)

FIPS specifies if the nodes for the cluster will be running in FIPS mode

kubeconfig KubeconfigSecretRef (Optional)

KubeConfig specifies the name and key for the kubeconfig secret

services []ServicePublishingStrategyMapping

Services defines metadata about how control plane services are published in the management cluster.

auditWebhook Kubernetes core/v1.LocalObjectReference (Optional)

AuditWebhook contains metadata for configuring an audit webhook endpoint for a cluster to process cluster audit events. It references a secret that contains the webhook information for the audit webhook endpoint. It is a secret because if the endpoint has MTLS the kubeconfig will contain client keys. This is currently only supported in IBM Cloud. The kubeconfig needs to be stored in the secret with a secret key name that corresponds to the constant AuditWebhookKubeconfigKey.

etcd EtcdSpec

Etcd contains metadata about the etcd cluster the hypershift managed Openshift control plane components use to store data.

configuration ClusterConfiguration

Configuration embeds resources that correspond to the openshift configuration API: https://docs.openshift.com/container-platform/4.7/rest_api/config_apis/config-apis-index.html

imageContentSources []ImageContentSource (Optional)

ImageContentSources lists sources/repositories for the release-image content.

additionalTrustBundle Kubernetes core/v1.LocalObjectReference (Optional)

AdditionalTrustBundle references a ConfigMap containing a PEM-encoded X.509 certificate bundle

secretEncryption SecretEncryptionSpec (Optional)

SecretEncryption contains metadata about the kubernetes secret encryption strategy being used for the cluster when applicable.

pausedUntil string (Optional)

PausedUntil is a field that can be used to pause reconciliation on a resource. Either a date can be provided in RFC3339 format or a boolean. If a date is provided: reconciliation is paused on the resource until that date. If the boolean true is provided: reconciliation is paused on the resource until the field is removed.

olmCatalogPlacement OLMCatalogPlacement (Optional)

OLMCatalogPlacement specifies the placement of OLM catalog components. By default, this is set to management and OLM catalog components are deployed onto the management cluster. If set to guest, the OLM catalog components will be deployed onto the guest cluster.

autoscaling ClusterAutoscaling (Optional)

Autoscaling specifies auto-scaling behavior that applies to all NodePools associated with the control plane.

nodeSelector map[string]string (Optional)

NodeSelector when specified, must be true for the pods managed by the HostedCluster to be scheduled.

HostedControlPlaneStatus

HostedControlPlaneStatus defines the observed state of HostedControlPlane

Field Description
ready bool

Ready denotes that the HostedControlPlane API Server is ready to receive requests This satisfies CAPI contract https://github.com/kubernetes-sigs/cluster-api/blob/cd3a694deac89d5ebeb888307deaa61487207aa0/controllers/cluster_controller_phases.go#L226-L230

initialized bool

Initialized denotes whether or not the control plane has provided a kubeadm-config. Once this condition is marked true, its value is never changed. See the Ready condition for an indication of the current readiness of the cluster’s control plane. This satisfies CAPI contract https://github.com/kubernetes-sigs/cluster-api/blob/cd3a694deac89d5ebeb888307deaa61487207aa0/controllers/cluster_controller_phases.go#L238-L252

externalManagedControlPlane bool

ExternalManagedControlPlane indicates to cluster-api that the control plane is managed by an external service. https://github.com/kubernetes-sigs/cluster-api/blob/65e5385bffd71bf4aad3cf34a537f11b217c7fab/controllers/machine_controller.go#L468

controlPlaneEndpoint APIEndpoint

ControlPlaneEndpoint contains the endpoint information by which external clients can access the control plane. This is populated after the infrastructure is ready.

oauthCallbackURLTemplate string

OAuthCallbackURLTemplate contains a template for the URL to use as a callback for identity providers. The [identity-provider-name] placeholder must be replaced with the name of an identity provider defined on the HostedCluster. This is populated after the infrastructure is ready.

versionStatus ClusterVersionStatus (Optional)

versionStatus is the status of the release version applied by the hosted control plane operator.

version string

Version is the semantic version of the release applied by the hosted control plane operator

Deprecated: Use versionStatus.desired.version instead.

releaseImage string (Optional)

ReleaseImage is the release image applied to the hosted control plane.

Deprecated: Use versionStatus.desired.image instead.

lastReleaseImageTransitionTime Kubernetes meta/v1.Time

lastReleaseImageTransitionTime is the time of the last update to the current releaseImage property.

Deprecated: Use versionStatus.history[0].startedTime instead.

kubeConfig KubeconfigSecretRef

KubeConfig is a reference to the secret containing the default kubeconfig for this control plane.

kubeadminPassword Kubernetes core/v1.LocalObjectReference (Optional)

KubeadminPassword is a reference to the secret containing the initial kubeadmin password for the guest cluster.

conditions []Kubernetes meta/v1.Condition (Optional)

Condition contains details for one aspect of the current state of the HostedControlPlane. Current condition types are: “Available”

platform PlatformStatus (Optional)

Platform contains platform-specific status of the HostedCluster

IBMCloudKMSAuthSpec

(Appears on: IBMCloudKMSSpec)

IBMCloudKMSAuthSpec defines metadata for how authentication is done with IBM Cloud KMS

Field Description
type IBMCloudKMSAuthType

Type defines the IBM Cloud KMS authentication strategy

unmanaged IBMCloudKMSUnmanagedAuthSpec (Optional)

Unmanaged defines the auth metadata the customer provides to interact with IBM Cloud KMS

managed IBMCloudKMSManagedAuthSpec (Optional)

Managed defines metadata around the service to service authentication strategy for the IBM Cloud KMS system (all provider managed).

IBMCloudKMSAuthType

(Appears on: IBMCloudKMSAuthSpec)

IBMCloudKMSAuthType defines the IBM Cloud KMS authentication strategy

Value Description

"Managed"

IBMCloudKMSManagedAuth defines the KMS authentication strategy where the IKS/ROKS platform uses service to service auth to call IBM Cloud KMS APIs (no customer credentials requried)

"Unmanaged"

IBMCloudKMSUnmanagedAuth defines the KMS authentication strategy where a customer supplies IBM Cloud authentication to interact with IBM Cloud KMS APIs

IBMCloudKMSKeyEntry

(Appears on: IBMCloudKMSSpec)

IBMCloudKMSKeyEntry defines metadata for an IBM Cloud KMS encryption key

Field Description
crkID string

CRKID is the customer rook key id

instanceID string

InstanceID is the id for the key protect instance

correlationID string

CorrelationID is an identifier used to track all api call usage from hypershift

url string

URL is the url to call key protect apis over

keyVersion int

KeyVersion is a unique number associated with the key. The number increments whenever a new key is enabled for data encryption.

IBMCloudKMSManagedAuthSpec

(Appears on: IBMCloudKMSAuthSpec)

IBMCloudKMSManagedAuthSpec defines metadata around the service to service authentication strategy for the IBM Cloud KMS system (all provider managed).

IBMCloudKMSSpec

(Appears on: KMSSpec)

IBMCloudKMSSpec defines metadata for the IBM Cloud KMS encryption strategy

Field Description
region string

Region is the IBM Cloud region

auth IBMCloudKMSAuthSpec

Auth defines metadata for how authentication is done with IBM Cloud KMS

keyList []IBMCloudKMSKeyEntry

KeyList defines the list of keys used for data encryption

IBMCloudKMSUnmanagedAuthSpec

(Appears on: IBMCloudKMSAuthSpec)

IBMCloudKMSUnmanagedAuthSpec defines the auth metadata the customer provides to interact with IBM Cloud KMS

Field Description
credentials Kubernetes core/v1.LocalObjectReference

Credentials should reference a secret with a key field of IBMCloudIAMAPIKeySecretKey that contains a apikey to call IBM Cloud KMS APIs

IBMCloudPlatformSpec

(Appears on: NodePoolPlatform, PlatformSpec)

IBMCloudPlatformSpec defines IBMCloud specific settings for components

Field Description
providerType github.com/openshift/api/config/v1.IBMCloudProviderType

ProviderType is a specific supported infrastructure provider within IBM Cloud.

ImageContentSource

(Appears on: HostedClusterSpec, HostedControlPlaneSpec)

ImageContentSource specifies image mirrors that can be used by cluster nodes to pull content. For cluster workloads, if a container image registry host of the pullspec matches Source then one of the Mirrors are substituted as hosts in the pullspec and tried in order to fetch the image.

Field Description
source string

Source is the repository that users refer to, e.g. in image pull specifications.

mirrors []string (Optional)

Mirrors are one or more repositories that may also contain the same images.

InPlaceUpgrade

(Appears on: NodePoolManagement)

InPlaceUpgrade specifies an upgrade strategy which upgrades nodes in-place without any new nodes being created or any old nodes being deleted.

Field Description
maxUnavailable k8s.io/apimachinery/pkg/util/intstr.IntOrString (Optional)

MaxUnavailable is the maximum number of nodes that can be unavailable during the update.

Value can be an absolute number (ex: 5) or a percentage of desired nodes (ex: 10%).

Absolute number is calculated from percentage by rounding down.

Defaults to 1.

Example: when this is set to 30%, a max of 30% of the nodes can be made unschedulable/unavailable immediately when the update starts. Once a set of nodes is updated, more nodes can be made unschedulable for update, ensuring that the total number of nodes schedulable at all times during the update is at least 70% of desired nodes.

KMSProvider

(Appears on: KMSSpec)

KMSProvider defines the supported KMS providers

Value Description

"AWS"

"IBMCloud"

KMSSpec

(Appears on: SecretEncryptionSpec)

KMSSpec defines metadata about the kms secret encryption strategy

Field Description
provider KMSProvider

Provider defines the KMS provider

ibmcloud IBMCloudKMSSpec (Optional)

IBMCloud defines metadata for the IBM Cloud KMS encryption strategy

aws AWSKMSSpec (Optional)

AWS defines metadata about the configuration of the AWS KMS Secret Encryption provider

KubevirtCompute

(Appears on: KubevirtNodePoolPlatform)

KubevirtCompute contains values associated with the virtual compute hardware requested for the VM.

Field Description
memory k8s.io/apimachinery/pkg/api/resource.Quantity (Optional)

Memory represents how much guest memory the VM should have

cores uint32 (Optional)

Cores represents how many cores the guest VM should have

KubevirtDiskImage

(Appears on: KubevirtRootVolume)

KubevirtDiskImage contains values representing where the rhcos image is located

Field Description
containerDiskImage string (Optional)

ContainerDiskImage is a string representing the container image that holds the root disk

KubevirtNodePoolPlatform

(Appears on: NodePoolPlatform)

KubevirtNodePoolPlatform specifies the configuration of a NodePool when operating on KubeVirt platform.

Field Description
rootVolume KubevirtRootVolume

RootVolume represents values associated with the VM volume that will host rhcos

compute KubevirtCompute (Optional)

Compute contains values representing the virtual hardware requested for the VM

KubevirtPersistentVolume

(Appears on: KubevirtVolume)

KubevirtPersistentVolume contains the values involved with provisioning persistent storage for a KubeVirt VM.

Field Description
size k8s.io/apimachinery/pkg/api/resource.Quantity (Optional)

Size is the size of the persistent storage volume

storageClass string (Optional)

StorageClass is the storageClass used for the underlying PVC that hosts the volume

accessModes []PersistentVolumeAccessMode (Optional)

AccessModes is an array that contains the desired Access Modes the root volume should have. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes/#access-modes

KubevirtPlatformSpec

(Appears on: PlatformSpec)

KubevirtPlatformSpec specifies configuration for kubevirt guest cluster installations

Field Description
baseDomainPassthrough bool (Optional)

BaseDomainPassthrough toggles whether or not an automatically generated base domain for the guest cluster should be used that is a subdomain of the management cluster’s *.apps DNS.

For the KubeVirt platform, the basedomain can be autogenerated using the *.apps domain of the management/infra hosting cluster This makes the guest cluster’s base domain a subdomain of the hypershift infra/mgmt cluster’s base domain.

Example: Infra/Mgmt cluster’s DNS Base: example.com Cluster: mgmt-cluster.example.com Apps: *.apps.mgmt-cluster.example.com KubeVirt Guest cluster’s DNS Base: apps.mgmt-cluster.example.com Cluster: guest.apps.mgmt-cluster.example.com Apps: *.apps.guest.apps.mgmt-cluster.example.com

This is possible using OCP wildcard routes

KubevirtRootVolume

(Appears on: KubevirtNodePoolPlatform)

KubevirtRootVolume represents the volume that the rhcos disk will be stored and run from.

Field Description
diskImage KubevirtDiskImage (Optional)

Image represents what rhcos image to use for the node pool

KubevirtVolume KubevirtVolume

(Members of KubevirtVolume are embedded into this type.)

KubevirtVolume represents of type of storage to run the image on

KubevirtVolume

(Appears on: KubevirtRootVolume)

KubevirtVolume represents what kind of storage to use for a KubeVirt VM volume

Field Description
type KubevirtVolumeType (Optional)

Type represents the type of storage to associate with the kubevirt VMs.

persistent KubevirtPersistentVolume (Optional)

Persistent volume type means the VM’s storage is backed by a PVC VMs that use persistent volumes can survive disruption events like restart and eviction This is the default type used when no storage type is defined.

KubevirtVolumeType

(Appears on: KubevirtVolume)

KubevirtVolumeType is a specific supported KubeVirt volumes

Value Description

"Persistent"

KubevirtVolumeTypePersistent represents persistent volume for kubevirt VMs

LoadBalancerPublishingStrategy

(Appears on: ServicePublishingStrategy)

LoadBalancerPublishingStrategy specifies setting used to expose a service as a LoadBalancer.

Field Description
hostname string (Optional)

Hostname is the name of the DNS record that will be created pointing to the LoadBalancer.

MachineNetworkEntry

(Appears on: ClusterNetworking)

MachineNetworkEntry is a single IP address block for node IP blocks.

Field Description
cidr github.com/openshift/hypershift/api/util/ipnet.IPNet

CIDR is the IP block address pool for machines within the cluster.

ManagedEtcdSpec

(Appears on: EtcdSpec)

ManagedEtcdSpec specifies the behavior of an etcd cluster managed by HyperShift.

Field Description
storage ManagedEtcdStorageSpec

Storage specifies how etcd data is persisted.

ManagedEtcdStorageSpec

(Appears on: ManagedEtcdSpec)

ManagedEtcdStorageSpec describes the storage configuration for etcd data.

Field Description
type ManagedEtcdStorageType

Type is the kind of persistent storage implementation to use for etcd.

persistentVolume PersistentVolumeEtcdStorageSpec (Optional)

PersistentVolume is the configuration for PersistentVolume etcd storage. With this implementation, a PersistentVolume will be allocated for every etcd member (either 1 or 3 depending on the HostedCluster control plane availability configuration).

restoreSnapshotURL []string (Optional)

RestoreSnapshotURL allows an optional list of URLs to be provided where an etcd snapshot can be downloaded, for example a pre-signed URL referencing a storage service, one URL per replica. This snapshot will be restored on initial startup, only when the etcd PV is empty.

ManagedEtcdStorageType

(Appears on: ManagedEtcdStorageSpec)

ManagedEtcdStorageType is a storage type for an etcd cluster.

Value Description

"PersistentVolume"

PersistentVolumeEtcdStorage uses PersistentVolumes for etcd storage.

NetworkType

(Appears on: ClusterNetworking)

NetworkType specifies the SDN provider used for cluster networking.

Value Description

"Calico"

Calico specifies Calico as the SDN provider

"OVNKubernetes"

OVNKubernetes specifies OVN as the SDN provider

"OpenShiftSDN"

OpenShiftSDN specifies OpenShiftSDN as the SDN provider

"Other"

Other specifies an undefined SDN provider

NodePoolAutoScaling

(Appears on: NodePoolSpec)

NodePoolAutoScaling specifies auto-scaling behavior for a NodePool.

Field Description
min int32

Min is the minimum number of nodes to maintain in the pool. Must be >= 1.

max int32

Max is the maximum number of nodes allowed in the pool. Must be >= 1.

NodePoolCondition

(Appears on: NodePoolStatus)

We define our own condition type since metav1.Condition has validation for Reason that might be broken by what we bubble up from CAPI. NodePoolCondition defines an observation of NodePool resource operational state.

Field Description
type string

Type of condition in CamelCase or in foo.example.com/CamelCase. Many .condition.type values are consistent across resources like Available, but because arbitrary conditions can be useful (see .node.status.conditions), the ability to deconflict is important.

status Kubernetes core/v1.ConditionStatus

Status of the condition, one of True, False, Unknown.

severity string (Optional)

Severity provides an explicit classification of Reason code, so the users or machines can immediately understand the current situation and act accordingly. The Severity field MUST be set only when Status=False.

lastTransitionTime Kubernetes meta/v1.Time

Last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable.

reason string (Optional)

The reason for the condition’s last transition in CamelCase. The specific API may choose whether or not this field is considered a guaranteed API. This field may not be empty.

message string (Optional)

A human readable message indicating details about the transition. This field may be empty.

observedGeneration int64

NodePoolManagement

(Appears on: NodePoolSpec)

NodePoolManagement specifies behavior for managing nodes in a NodePool, such as upgrade strategies and auto-repair behaviors.

Field Description
upgradeType UpgradeType

UpgradeType specifies the type of strategy for handling upgrades.

replace ReplaceUpgrade

Replace is the configuration for rolling upgrades.

inPlace InPlaceUpgrade

InPlace is the configuration for in-place upgrades.

autoRepair bool (Optional)

AutoRepair specifies whether health checks should be enabled for machines in the NodePool. The default is false.

NodePoolPlatform

(Appears on: NodePoolSpec)

NodePoolPlatform specifies the underlying infrastructure provider for the NodePool and is used to configure platform specific behavior.

Field Description
type PlatformType

Type specifies the platform name.

aws AWSNodePoolPlatform (Optional)

AWS specifies the configuration used when operating on AWS.

ibmcloud IBMCloudPlatformSpec

IBMCloud defines IBMCloud specific settings for components

kubevirt KubevirtNodePoolPlatform (Optional)

Kubevirt specifies the configuration used when operating on KubeVirt platform.

agent AgentNodePoolPlatform (Optional)

Agent specifies the configuration used when using Agent platform.

azure AzureNodePoolPlatform
powervs PowerVSNodePoolPlatform (Optional)

PowerVS specifies the configuration used when using IBMCloud PowerVS platform.

NodePoolSpec

(Appears on: NodePool)

NodePoolSpec is the desired behavior of a NodePool.

Field Description
clusterName string

ClusterName is the name of the HostedCluster this NodePool belongs to.

TODO(dan): Should this be a LocalObjectReference?

release Release

Release specifies the OCP release used for the NodePool. This informs the ignition configuration for machines, as well as other platform specific machine properties (e.g. an AMI on the AWS platform).

platform NodePoolPlatform

Platform specifies the underlying infrastructure provider for the NodePool and is used to configure platform specific behavior.

replicas int32 (Optional)

Replicas is the desired number of nodes the pool should maintain. If unset, the default value is 0.

management NodePoolManagement

Management specifies behavior for managing nodes in the pool, such as upgrade strategies and auto-repair behaviors.

autoScaling NodePoolAutoScaling (Optional)

Autoscaling specifies auto-scaling behavior for the NodePool.

config []Kubernetes core/v1.LocalObjectReference

Config is a list of references to ConfigMaps containing serialized MachineConfig resources to be injected into the ignition configurations of nodes in the NodePool. The MachineConfig API schema is defined here:

https://github.com/openshift/machine-config-operator/blob/18963e4f8fe66e8c513ca4b131620760a414997f/pkg/apis/machineconfiguration.openshift.io/v1/types.go#L185

Each ConfigMap must have a single key named “config” whose value is the JSON or YAML of a serialized Resource for machineconfiguration.openshift.io: KubeletConfig ContainerRuntimeConfig MachineConfig or ImageContentSourcePolicy

nodeDrainTimeout Kubernetes meta/v1.Duration (Optional)

NodeDrainTimeout is the total amount of time that the controller will spend on draining a node. The default value is 0, meaning that the node can be drained without any time limitations. NOTE: NodeDrainTimeout is different from kubectl drain --timeout TODO (alberto): Today changing this field will trigger a recreate rolling update, which kind of defeats the purpose of the change. In future we plan to propagate this field in-place. https://github.com/kubernetes-sigs/cluster-api/issues/5880

nodeLabels map[string]string (Optional)

NodeLabels propagates a list of labels to Nodes, only once on creation. Valid values are those in https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#syntax-and-character-set

taints []Taint (Optional)

Taints if specified, propagates a list of taints to Nodes, only once on creation.

pausedUntil string (Optional)

PausedUntil is a field that can be used to pause reconciliation on a resource. Either a date can be provided in RFC3339 format or a boolean. If a date is provided: reconciliation is paused on the resource until that date. If the boolean true is provided: reconciliation is paused on the resource until the field is removed.

tuningConfig []Kubernetes core/v1.LocalObjectReference

TuningConfig is a list of references to ConfigMaps containing serialized Tuned resources to define the tuning configuration to be applied to nodes in the NodePool. The Tuned API is defined here:

https://github.com/openshift/cluster-node-tuning-operator/blob/2c76314fb3cc8f12aef4a0dcd67ddc3677d5b54f/pkg/apis/tuned/v1/tuned_types.go

Each ConfigMap must have a single key named “tuned” whose value is the JSON or YAML of a serialized Tuned.

NodePoolStatus

(Appears on: NodePool)

NodePoolStatus is the latest observed status of a NodePool.

Field Description
replicas int32 (Optional)

Replicas is the latest observed number of nodes in the pool.

version string

Version is the semantic version of the latest applied release specified by the NodePool.

conditions []NodePoolCondition (Optional)

Conditions represents the latest available observations of the node pool’s current state.

NodePortPublishingStrategy

(Appears on: ServicePublishingStrategy)

NodePortPublishingStrategy specifies a NodePort used to expose a service.

Field Description
address string

Address is the host/ip that the NodePort service is exposed over.

port int32

Port is the port of the NodePort service. If <=0, the port is dynamically assigned when the service is created.

OLMCatalogPlacement

(Appears on: HostedClusterSpec, HostedControlPlaneSpec)

OLMCatalogPlacement is an enum specifying the placement of OLM catalog components.

Value Description

"guest"

GuestOLMCatalogPlacement indicates OLM catalog components will be placed in the guest cluster.

"management"

ManagementOLMCatalogPlacement indicates OLM catalog components will be placed in the management cluster.

PersistentVolumeAccessMode

(Appears on: KubevirtPersistentVolume)

PersistentVolumeEtcdStorageSpec

(Appears on: ManagedEtcdStorageSpec)

PersistentVolumeEtcdStorageSpec is the configuration for PersistentVolume etcd storage.

Field Description
storageClassName string (Optional)

StorageClassName is the StorageClass of the data volume for each etcd member.

See https://kubernetes.io/docs/concepts/storage/persistent-volumes#class-1.

size k8s.io/apimachinery/pkg/api/resource.Quantity (Optional)

Size is the minimum size of the data volume for each etcd member.

PlatformSpec

(Appears on: HostedClusterSpec, HostedControlPlaneSpec)

PlatformSpec specifies the underlying infrastructure provider for the cluster and is used to configure platform specific behavior.

Field Description
type PlatformType

Type is the type of infrastructure provider for the cluster.

aws AWSPlatformSpec (Optional)

AWS specifies configuration for clusters running on Amazon Web Services.

agent AgentPlatformSpec (Optional)

Agent specifies configuration for agent-based installations.

ibmcloud IBMCloudPlatformSpec

IBMCloud defines IBMCloud specific settings for components

azure AzurePlatformSpec

Azure defines azure specific settings

powervs PowerVSPlatformSpec (Optional)

PowerVS specifies configuration for clusters running on IBMCloud Power VS Service. This field is immutable. Once set, It can’t be changed.

kubevirt KubevirtPlatformSpec (Optional)

KubeVirt defines KubeVirt specific settings for cluster components.

PlatformStatus

(Appears on: HostedClusterStatus, HostedControlPlaneStatus)

PlatformStatus contains platform-specific status

Field Description
aws AWSPlatformStatus (Optional)

PlatformType

(Appears on: NodePoolPlatform, PlatformSpec)

PlatformType is a specific supported infrastructure provider.

Value Description

"AWS"

AWSPlatform represents Amazon Web Services infrastructure.

"Agent"

AgentPlatform represents user supplied insfrastructure booted with agents.

"Azure"

AzurePlatform represents Azure infrastructure.

"IBMCloud"

IBMCloudPlatform represents IBM Cloud infrastructure.

"KubeVirt"

KubevirtPlatform represents Kubevirt infrastructure.

"None"

NonePlatform represents user supplied (e.g. bare metal) infrastructure.

"PowerVS"

PowerVSPlatform represents PowerVS infrastructure.

PowerVSNodePoolImageDeletePolicy

(Appears on: PowerVSNodePoolPlatform)

PowerVSNodePoolImageDeletePolicy defines image delete policy to be used for PowerVSNodePoolPlatform

PowerVSNodePoolPlatform

(Appears on: NodePoolPlatform)

PowerVSNodePoolPlatform specifies the configuration of a NodePool when operating on IBMCloud PowerVS platform.

Field Description
systemType string (Optional)

SystemType is the System type used to host the instance. systemType determines the number of cores and memory that is available. Few of the supported SystemTypes are s922,e880,e980. e880 systemType available only in Dallas Datacenters. e980 systemType available in Datacenters except Dallas and Washington. When omitted, this means that the user has no opinion and the platform is left to choose a reasonable default. The current default is s922 which is generally available.

processorType PowerVSNodePoolProcType (Optional)

ProcessorType is the VM instance processor type. It must be set to one of the following values: Dedicated, Capped or Shared.

Dedicated: resources are allocated for a specific client, The hypervisor makes a 1:1 binding of a partition’s processor to a physical processor core. Shared: Shared among other clients. Capped: Shared, but resources do not expand beyond those that are requested, the amount of CPU time is Capped to the value specified for the entitlement.

if the processorType is selected as Dedicated, then Processors value cannot be fractional. When omitted, this means that the user has no opinion and the platform is left to choose a reasonable default. The current default is Shared.

processors k8s.io/apimachinery/pkg/util/intstr.IntOrString (Optional)

Processors is the number of virtual processors in a virtual machine. when the processorType is selected as Dedicated the processors value cannot be fractional. maximum value for the Processors depends on the selected SystemType. when SystemType is set to e880 or e980 maximum Processors value is 143. when SystemType is set to s922 maximum Processors value is 15. minimum value for Processors depends on the selected ProcessorType. when ProcessorType is set as Shared or Capped, The minimum processors is 0.5. when ProcessorType is set as Dedicated, The minimum processors is 1. When omitted, this means that the user has no opinion and the platform is left to choose a reasonable default. The default is set based on the selected ProcessorType. when ProcessorType selected as Dedicated, the default is set to 1. when ProcessorType selected as Shared or Capped, the default is set to 0.5.

memoryGiB int32 (Optional)

MemoryGiB is the size of a virtual machine’s memory, in GiB. maximum value for the MemoryGiB depends on the selected SystemType. when SystemType is set to e880 maximum MemoryGiB value is 7463 GiB. when SystemType is set to e980 maximum MemoryGiB value is 15307 GiB. when SystemType is set to s922 maximum MemoryGiB value is 942 GiB. The minimum memory is 32 GiB.

When omitted, this means the user has no opinion and the platform is left to choose a reasonable default. The current default is 32.

image PowerVSResourceReference (Optional)

Image used for deploying the nodes. If unspecified, the default is chosen based on the NodePool release payload image.

storageType PowerVSNodePoolStorageType (Optional)

StorageType for the image and nodes, this will be ignored if Image is specified. The storage tiers in PowerVS are based on I/O operations per second (IOPS). It means that the performance of your storage volumes is limited to the maximum number of IOPS based on volume size and storage tier. Although, the exact numbers might change over time, the Tier 3 storage is currently set to 3 IOPS/GB, and the Tier 1 storage is currently set to 10 IOPS/GB.

The default is tier1

imageDeletePolicy PowerVSNodePoolImageDeletePolicy (Optional)

ImageDeletePolicy is policy for the image deletion.

delete: delete the image from the infrastructure. retain: delete the image from the openshift but retain in the infrastructure.

The default is delete

PowerVSNodePoolProcType

(Appears on: PowerVSNodePoolPlatform)

PowerVSNodePoolProcType defines processor type to be used for PowerVSNodePoolPlatform

Value Description

"capped"

PowerVSNodePoolCappedProcType defines capped processor type

"dedicated"

PowerVSNodePoolDedicatedProcType defines dedicated processor type

"shared"

PowerVSNodePoolSharedProcType defines shared processor type

PowerVSNodePoolStorageType

(Appears on: PowerVSNodePoolPlatform)

PowerVSNodePoolStorageType defines storage type to be used for PowerVSNodePoolPlatform

PowerVSPlatformSpec

(Appears on: PlatformSpec)

PowerVSPlatformSpec defines IBMCloud PowerVS specific settings for components

Field Description
accountID string

AccountID is the IBMCloud account id. This field is immutable. Once set, It can’t be changed.

cisInstanceCRN string

CISInstanceCRN is the IBMCloud CIS Service Instance’s Cloud Resource Name This field is immutable. Once set, It can’t be changed.

resourceGroup string

ResourceGroup is the IBMCloud Resource Group in which the cluster resides. This field is immutable. Once set, It can’t be changed.

region string

Region is the IBMCloud region in which the cluster resides. This configures the OCP control plane cloud integrations, and is used by NodePool to resolve the correct boot image for a given release. This field is immutable. Once set, It can’t be changed.

zone string

Zone is the availability zone where control plane cloud resources are created. This field is immutable. Once set, It can’t be changed.

subnet PowerVSResourceReference

Subnet is the subnet to use for control plane cloud resources. This field is immutable. Once set, It can’t be changed.

serviceInstanceID string

ServiceInstance is the reference to the Power VS service on which the server instance(VM) will be created. Power VS service is a container for all Power VS instances at a specific geographic region. serviceInstance can be created via IBM Cloud catalog or CLI. ServiceInstanceID is the unique identifier that can be obtained from IBM Cloud UI or IBM Cloud cli.

More detail about Power VS service instance. https://cloud.ibm.com/docs/power-iaas?topic=power-iaas-creating-power-virtual-server

This field is immutable. Once set, It can’t be changed.

vpc PowerVSVPC

VPC specifies IBM Cloud PowerVS Load Balancing configuration for the control plane. This field is immutable. Once set, It can’t be changed.

kubeCloudControllerCreds Kubernetes core/v1.LocalObjectReference

KubeCloudControllerCreds is a reference to a secret containing cloud credentials with permissions matching the cloud controller policy. This field is immutable. Once set, It can’t be changed.

TODO(dan): document the “cloud controller policy”

nodePoolManagementCreds Kubernetes core/v1.LocalObjectReference

NodePoolManagementCreds is a reference to a secret containing cloud credentials with permissions matching the node pool management policy. This field is immutable. Once set, It can’t be changed.

TODO(dan): document the “node pool management policy”

ingressOperatorCloudCreds Kubernetes core/v1.LocalObjectReference

IngressOperatorCloudCreds is a reference to a secret containing ibm cloud credentials for ingress operator to get authenticated with ibm cloud.

storageOperatorCloudCreds Kubernetes core/v1.LocalObjectReference

StorageOperatorCloudCreds is a reference to a secret containing ibm cloud credentials for storage operator to get authenticated with ibm cloud.

PowerVSResourceReference

(Appears on: PowerVSNodePoolPlatform, PowerVSPlatformSpec)

PowerVSResourceReference is a reference to a specific IBMCloud PowerVS resource by ID, or Name. Only one of ID, or Name may be specified. Specifying more than one will result in a validation error.

Field Description
id string (Optional)

ID of resource

name string (Optional)

Name of resource

PowerVSVPC

(Appears on: PowerVSPlatformSpec)

PowerVSVPC specifies IBM Cloud PowerVS LoadBalancer configuration for the control plane.

Field Description
name string

Name for VPC to used for all the service load balancer. This field is immutable. Once set, It can’t be changed.

region string

Region is the IBMCloud region in which VPC gets created, this VPC used for all the ingress traffic into the OCP cluster. This field is immutable. Once set, It can’t be changed.

zone string (Optional)

Zone is the availability zone where load balancer cloud resources are created. This field is immutable. Once set, It can’t be changed.

subnet string (Optional)

Subnet is the subnet to use for load balancer. This field is immutable. Once set, It can’t be changed.

PublishingStrategyType

(Appears on: ServicePublishingStrategy)

PublishingStrategyType defines publishing strategies for services.

Release

(Appears on: HostedClusterSpec, NodePoolSpec)

Release represents the metadata for an OCP release payload image.

Field Description
image string

Image is the image pullspec of an OCP release payload image.

ReplaceUpgrade

(Appears on: NodePoolManagement)

ReplaceUpgrade specifies upgrade behavior that replaces existing nodes according to a given strategy.

Field Description
strategy UpgradeStrategy

Strategy is the node replacement strategy for nodes in the pool.

rollingUpdate RollingUpdate

RollingUpdate specifies a rolling update strategy which upgrades nodes by creating new nodes and deleting the old ones.

RollingUpdate

(Appears on: ReplaceUpgrade)

RollingUpdate specifies a rolling update strategy which upgrades nodes by creating new nodes and deleting the old ones.

Field Description
maxUnavailable k8s.io/apimachinery/pkg/util/intstr.IntOrString (Optional)

MaxUnavailable is the maximum number of nodes that can be unavailable during the update.

Value can be an absolute number (ex: 5) or a percentage of desired nodes (ex: 10%).

Absolute number is calculated from percentage by rounding down.

This can not be 0 if MaxSurge is 0.

Defaults to 0.

Example: when this is set to 30%, old nodes can be deleted down to 70% of desired nodes immediately when the rolling update starts. Once new nodes are ready, more old nodes be deleted, followed by provisioning new nodes, ensuring that the total number of nodes available at all times during the update is at least 70% of desired nodes.

maxSurge k8s.io/apimachinery/pkg/util/intstr.IntOrString (Optional)

MaxSurge is the maximum number of nodes that can be provisioned above the desired number of nodes.

Value can be an absolute number (ex: 5) or a percentage of desired nodes (ex: 10%).

Absolute number is calculated from percentage by rounding up.

This can not be 0 if MaxUnavailable is 0.

Defaults to 1.

Example: when this is set to 30%, new nodes can be provisioned immediately when the rolling update starts, such that the total number of old and new nodes do not exceed 130% of desired nodes. Once old nodes have been deleted, new nodes can be provisioned, ensuring that total number of nodes running at any time during the update is at most 130% of desired nodes.

RoutePublishingStrategy

(Appears on: ServicePublishingStrategy)

RoutePublishingStrategy specifies options for exposing a service as a Route.

Field Description
hostname string (Optional)

Hostname is the name of the DNS record that will be created pointing to the Route.

SecretEncryptionSpec

(Appears on: HostedClusterSpec, HostedControlPlaneSpec)

SecretEncryptionSpec contains metadata about the kubernetes secret encryption strategy being used for the cluster when applicable.

Field Description
type SecretEncryptionType

Type defines the type of kube secret encryption being used

kms KMSSpec (Optional)

KMS defines metadata about the kms secret encryption strategy

aescbc AESCBCSpec (Optional)

AESCBC defines metadata about the AESCBC secret encryption strategy

SecretEncryptionType

(Appears on: SecretEncryptionSpec)

SecretEncryptionType defines the type of kube secret encryption being used.

Value Description

"aescbc"

AESCBC uses AES-CBC with PKCS#7 padding to do secret encryption

"kms"

KMS integrates with a cloud provider’s key management service to do secret encryption

ServiceNetworkEntry

(Appears on: ClusterNetworking)

ServiceNetworkEntry is a single IP address block for the service network.

Field Description
cidr github.com/openshift/hypershift/api/util/ipnet.IPNet

CIDR is the IP block address pool for services within the cluster.

ServicePublishingStrategy

(Appears on: ServicePublishingStrategyMapping)

ServicePublishingStrategy specfies how to publish a ServiceType.

Field Description
type PublishingStrategyType

Type is the publishing strategy used for the service.

nodePort NodePortPublishingStrategy

NodePort configures exposing a service using a NodePort.

loadBalancer LoadBalancerPublishingStrategy

LoadBalancer configures exposing a service using a LoadBalancer.

route RoutePublishingStrategy

Route configures exposing a service using a Route.

ServicePublishingStrategyMapping

(Appears on: HostedClusterSpec, HostedControlPlaneSpec)

ServicePublishingStrategyMapping specifies how individual control plane services are published from the hosting cluster of a control plane.

Field Description
service ServiceType

Service identifies the type of service being published.

servicePublishingStrategy ServicePublishingStrategy

ServicePublishingStrategy specifies how to publish Service.

ServiceType

(Appears on: ServicePublishingStrategyMapping)

ServiceType defines what control plane services can be exposed from the management control plane.

Taint

(Appears on: NodePoolSpec)

Taint is as v1 Core but without TimeAdded. https://github.com/kubernetes/kubernetes/blob/ed8cad1e80d096257921908a52ac69cf1f41a098/staging/src/k8s.io/api/core/v1/types.go#L3037-L3053

Field Description
key string

Required. The taint key to be applied to a node.

value string (Optional)

The taint value corresponding to the taint key.

effect Kubernetes core/v1.TaintEffect

Required. The effect of the taint on pods that do not tolerate the taint. Valid effects are NoSchedule, PreferNoSchedule and NoExecute.

UnmanagedEtcdSpec

(Appears on: EtcdSpec)

UnmanagedEtcdSpec specifies configuration which enables the control plane to integrate with an eternally managed etcd cluster.

Field Description
endpoint string

Endpoint is the full etcd cluster client endpoint URL. For example:

https://etcd-client:2379

If the URL uses an HTTPS scheme, the TLS field is required.

tls EtcdTLSConfig

TLS specifies TLS configuration for HTTPS etcd client endpoints.

UpgradeStrategy

(Appears on: ReplaceUpgrade)

UpgradeStrategy is a specific strategy for upgrading nodes in a NodePool.

Value Description

"OnDelete"

UpgradeStrategyOnDelete replaces old nodes when the deletion of the associated node instances are completed.

"RollingUpdate"

UpgradeStrategyRollingUpdate means use a rolling update for nodes.

UpgradeType

(Appears on: NodePoolManagement)

UpgradeType is a type of high-level upgrade behavior nodes in a NodePool.

Value Description

"InPlace"

UpgradeTypeInPlace is a strategy which replaces nodes in-place with no additional node capacity requirements.

"Replace"

UpgradeTypeReplace is a strategy which replaces nodes using surge node capacity.

Volume

(Appears on: AWSNodePoolPlatform)

Volume specifies the configuration options for node instance storage devices.

Field Description
size int64

Size specifies size (in Gi) of the storage device.

Must be greater than the image snapshot size or 8 (whichever is greater).

type string

Type is the type of the volume.

iops int64 (Optional)

IOPS is the number of IOPS requested for the disk. This is only valid for type io1.

encrypted bool (Optional)

Encrypted is whether the volume should be encrypted or not.

encryptionKey string (Optional)

EncryptionKey is the KMS key to use to encrypt the volume. Can be either a KMS key ID or ARN. If Encrypted is set and this is omitted, the default AWS key will be used. The key must already exist and be accessible by the controller.