Kubernetes cluster
Overview
A Kubernetes cluster is managed Kubernetes whose control plane ECI operates. You provide the nodes your workloads run on through node pools, and use the cluster with kubectl.
This guide is for developers and operators using Kubernetes clusters for the first time. The overall flow is:
- Create a cluster
- Add a node pool
- Connect with kubectl
- If needed, enable load balancers and persistent volumes
Components
| Component | Description |
|---|---|
| Cluster | One Kubernetes cluster, with a name, version, subnet and API endpoint |
| Control plane | The servers that manage the cluster. ECI operates them, as 1 or 3 servers |
| Node pool | A group of identical nodes. A cluster can have several |
| Node | A virtual machine in a node pool. Workloads (pods) run here |
Before you begin
Required permissions
| Task | Permission codes |
|---|---|
| Create a cluster | ACL.Resource.Compute.KubernetesCluster.CREATE |
| Add a node pool | ACL.Resource.Compute.KubernetesNodePool.CREATE |
| Issue a kubeconfig | ACL.Resource.Compute.KubernetesCluster.ACCESS_API, ACL.User.UserAccessToken.CREATE |
The default Owner, Contributor and Support roles can issue kubeconfigs; Reader cannot. Without a permission, the page shows the name and code of the permission you need; pass that code to your administrator. For permission management, see Role-based access control.
Network requirements
- A virtual machine subnet for the cluster.
- Free IPs in the subnet for every control plane node and node. Each node uses one IP.
- The subnet's virtual network must not overlap
10.96.0.0/16or10.244.0.0/16. The cluster uses these two ranges internally for Services and pods.
Pricing
The Kubernetes cluster itself is free of charge. Each resource below that makes up the cluster and its node pools is billed, and quota is checked against these resources.
| Unit | Billed resources |
|---|---|
| Each control plane node | Virtual machine, boot disk, network interface, public IP |
| Each cluster | One endpoint load balancer, one public IP |
| Each node | Virtual machine, boot disk, network interface, public IP |
For example, 3 control plane nodes and 5 nodes use 9 public IPs and 1 load balancer.
Create a cluster
- In Compute > Kubernetes, click Create cluster.
- Fill in the fields below.
- Under Resource composition at the bottom, check the specs, hourly cost and quota.
- Click Create.
| Field | Description |
|---|---|
| Name | The cluster name |
| Kubernetes version | The Kubernetes version of the cluster |
| Virtual network | The virtual network for the cluster |
| Subnet | The subnet, within the chosen virtual network, used by the control plane and nodes |
| Control plane nodes | 1 · Single or 3 · High availability |
| Username / password | The OS account of the control plane nodes |
Input rules:
- Name: 1–32 lowercase letters, digits and hyphens (
-), starting and ending with a letter or digit. - Username: 1–32 characters, starting with a lowercase letter or
_, using only lowercase letters, digits,_and-. - Password: must pass the strength check.
If there is no virtual network or subnet, create one with Create virtual network or Create subnet without leaving the page.
After you submit, an Ongoing work card appears at the top of the cluster detail page and disappears once the control plane and node pools are ready.
The name, subnet, node specs (instance type and disk) and node account cannot change after creation. You can change the node count on the Settings tab, and raise the Kubernetes version with a version upgrade.
Control plane nodes
| Configuration | Behavior |
|---|---|
| 1 · Single | If the server fails, cluster operations stop; running workloads keep running |
| 3 · High availability | Three servers behind a load balancer keep cluster operations running if one fails |
You can switch the control plane between 1 and 3 nodes later on the Settings tab.
Add a node pool
A cluster has no nodes to run workloads on, so add at least one node pool.
- On the cluster detail page, click Add node pool.
- Fill in the fields below and click Create.
| Field | Description |
|---|---|
| Name | The node pool name. Same rules as the cluster name |
| Instance type | The node specs. Choose a GPU type if you need GPU nodes |
| Pricing type | On-demand or reserved |
| Node count | 1–64 |
| Username / password | The nodes' OS account. Same rules as the cluster |
Separate node pools by purpose to keep them easy to manage — for example, a CPU pool for services and a GPU pool for training.
Connect with kubectl
Click Connect under the title of the cluster detail page for a three-step guide.
Install kubectl
If you don't have kubectl, install it following the official Kubernetes install guide. Use a kubectl within one minor version of the cluster.
kubectl version --client
Issue a kubeconfig
- In the Connect dialog, choose an expiry: 30 days, 90 days, 6 months, 12 months, or a custom date.
- Click Issue.
- Click Download to save
<cluster name>-kubeconfig.yaml.
- Once you close the dialog, the issued kubeconfig cannot be shown again. Download or copy it before closing.
- Each issue creates a new access token for your account and embeds it in the kubeconfig. The token carries your account's permissions, so treat it like a password.
- Anyone connecting with the kubeconfig has cluster-admin rights inside the cluster.
- The token is not limited to one cluster; it also works on other clusters you can access in the same organization.
- Do not share it or commit it to a repository.
Check the connection
Run the following in the folder where you saved the kubeconfig. The same command works on any OS and shell.
kubectl --kubeconfig <cluster name>-kubeconfig.yaml get nodes
If the node list appears, you are connected.
NAME STATUS ROLES AGE VERSION
cloud-joomev Ready control-plane 22m v1.33.13
Before you add a node pool, only control plane nodes appear; node pool nodes show ROLES as <none>.
The cluster and context names in the kubeconfig are not the portal's cluster name but cl-<short ID> (e.g. cl-80zrrc). Keep this in mind when combining kubeconfigs of several clusters.
Enable load balancers and persistent volumes
To use LoadBalancer Services and persistent volumes (PVCs), register an access token once after creating the cluster.
Both features are provided by the ECI cloud controller and the block storage driver in the cluster, which manage ECI load balancers and block storage on your behalf and need an access token to call the ECI API. Until a token is registered, a LoadBalancer Service's EXTERNAL-IP stays <pending> and PVCs stay Pending.
Issue a dedicated access token
Issue a token for this cluster only under User management > User access tokens. Sharing a token with other uses is not recommended.
An access token's permissions follow the current role of the user who issued it; if the role changes, tokens already issued change too. The issuing user needs the permissions below.
| Resource | Permissions needed |
|---|---|
| Load balancer | Create · read · delete |
| Public IP | Create · read · delete |
| Block storage | Create · read · update · delete |
| Network interface | Read |
| Subnet | Read |
| Virtual machine | Read |
To keep permissions minimal, create a role with only these permissions, assign it to a user for this purpose, and issue the token as that user. For creating and assigning roles, see Role-based access control.
Permission codes
- Load balancer:
ACL.Resource.Network.LoadBalancer.CREATE,ACL.Resource.Network.LoadBalancer.READ,ACL.Resource.Network.LoadBalancer.DELETE - Public IP:
ACL.Resource.Network.PublicIp.CREATE,ACL.Resource.Network.PublicIp.READ,ACL.Resource.Network.PublicIp.DELETE - Block storage:
ACL.Resource.Storage.BlockStorage.CREATE,ACL.Resource.Storage.BlockStorage.READ,ACL.Resource.Storage.BlockStorage.UPDATE,ACL.Resource.Storage.BlockStorage.DELETE - Network interface:
ACL.Resource.Network.NetworkInterface.READ - Subnet:
ACL.Resource.Network.Subnet.READ - Virtual machine:
ACL.Resource.Compute.VirtualMachine.READ
Register the token in the cluster
Register the token as the eci-cloud-provider-credentials Secret in the eci-system namespace. The key must be ECI_API_TOKEN.
kubectl -n eci-system create secret generic eci-cloud-provider-credentials \
--from-literal=ECI_API_TOKEN=<issued access token>
Both components read the token at startup, so restart them after registering or changing the Secret.
kubectl -n eci-system rollout restart \
ds/eci-cloud-controller-manager deploy/eci-block-csi-controller
Once they restart, pending Services and PVCs are processed automatically.
Renew the token
Access tokens expire. Before that, issue a new token, replace the Secret the same way, and restart both components.
kubectl -n eci-system create secret generic eci-cloud-provider-credentials \
--from-literal=ECI_API_TOKEN=<new access token> \
--dry-run=client -o yaml | kubectl apply -f -
kubectl -n eci-system rollout restart \
ds/eci-cloud-controller-manager deploy/eci-block-csi-controller
When a token expires, existing load balancers and volumes keep working, but new resources and node changes are no longer applied.
Anyone with cluster-admin rights can read this Secret. Manage access to the cluster carefully.
Operating a cluster
Cluster status
| Status | Description |
|---|---|
| Requested | The create request was received |
| Creating | The control plane or nodes are being created |
| Running | The cluster is ready to use |
| Deleting | Deletion is in progress |
| Deleted | Deletion is complete |
During node count changes and version upgrades the status stays Running; follow progress on the Ongoing work card.
Detail page
| Tab | Contents |
|---|---|
| Overview | Cluster info, network, control plane, node pool summary |
| Node pools | The node pool list. Select a node pool to open its detail page |
| Metrics | Per-node CPU, memory, network, storage and GPU metrics |
| Settings | Change control plane nodes, upgrade the version, delete the cluster |
The node pool detail page has the same layout; change its node count (1–64) on the Settings tab.
Changing the node count
Adding nodes needs quota and free subnet IPs for the extra nodes. If quota falls short, the missing items and amounts appear before you save and saving is disabled. Removing nodes is not checked.
Version upgrades
Raise the Kubernetes version of a cluster or node pool under Settings > Version upgrade on its detail page.
- A cluster goes up one minor version at a time (e.g. v1.33 → v1.34).
- Upgrade the cluster first, then its node pools. A node pool can go straight to the cluster's version, but never above it.
- Neither a cluster nor a node pool can be rolled back to a lower version.
- Existing node pools are not upgraded automatically with the cluster. A node pool works normally up to three minor versions below the cluster, but a node pool lower than that blocks the cluster's next upgrade.
- Node pools created after a cluster upgrade start on the cluster's version.
- An upgrade cannot start while a create or change is in progress.
Versions you cannot pick are listed as disabled with the reason. A node pool low enough to block the cluster's next upgrade shows a warning icon in the node pool table and in the version field of its detail page.
Managing access tokens
Tokens created with kubeconfigs are listed under User management > User access tokens; the Kubernetes cluster column shows the cluster each token was issued for.
- When a token expires, issue a new kubeconfig.
- Revoke the tokens of kubeconfigs you no longer use here. A kubeconfig with a revoked token can no longer connect.
For details, see Access tokens.
Resources managed by Kubernetes
Changing resources the cluster created directly in the portal can put them out of sync with the cluster, so some actions are restricted.
- Node virtual machines: shown on the Kubernetes nodes tab of Compute > Virtual machines, without start, stop, edit or delete buttons. Control plane virtual machines are not listed.
- Load balancers: clean up load balancers created for
LoadBalancerServices by deleting the Service in the cluster. - Block storage: volumes created for PVCs are managed by the block storage driver; clean them up by deleting the PVC in the cluster.
Delete
Deletion cannot be undone. Delete the node pools first, then the cluster.
- Delete each node pool at the bottom of its Settings tab. Its nodes and disks are deleted with it, and running workloads stop.
- Delete the cluster at the bottom of its Settings tab. The control plane nodes, load balancer and public IPs are deleted with it.
Troubleshooting
Cannot connect
| Error message | Cause and fix |
|---|---|
Unable to connect to the server | The endpoint is unreachable. Check the firewall on the cluster detail's Network card |
x509: certificate ... | The kubeconfig is damaged or belongs to another cluster. Issue a new kubeconfig |
Unauthorized | The token expired or was revoked, a permission was removed, the account was blocked, or the organization was suspended. Check permissions and account status, then issue a new kubeconfig |
Load balancers or volumes are not created
If EXTERNAL-IP is <pending> or a PVC is Pending, the access token is missing or expired. Register or renew it as described in Enable load balancers and persistent volumes.
Deletion is blocked
- A cluster with node pools left: delete all node pools first. While a node pool is Deleting, wait for it to finish.
- A node pool targeted by a load balancer: delete the
LoadBalancerService in the cluster first. - A subnet used by a cluster: delete the cluster first.
Next steps
- Firewall — access rules for the cluster subnet
- Load balancer — how load balancers work
- Block storage — the volumes PVCs create
- Access tokens — find and revoke kubeconfig tokens
- Role-based access control — granting Kubernetes permissions