Kubectl: Developer tips for the Kubernetes command line
Kubectl, the Kubernetes command-line interface (CLI), has more capabilities than many developers realize. For example, did you know that
kubectl can reach the Kubernetes API while running inside a cluster? You can also use
kubectl to assume different user identities, to select a custom editor to run with the
kubectl edit command, and more.
In this article, I introduce several
kubectl CLI features that will improve your daily workflow. You’ll also learn about the new
kubectl debug command in Kubernetes 1.20.
kubectl needs to locate a configuration file, it checks several places. You are probably familiar with the
$HOME/.kube/ directory, which is the default directory where
kubectl stores its configuration and cache files. You’ve also heard about the
--kubeconfig flag, or
KUBECONFIG environment variable, which is used to pass the location of a configuration file.
Another location that
kubectl checks when loading files is the in-cluster configuration. Not many users know about this option, so I’ll use an example to demonstrate how it works.
Get pods from the container
To start, we’ll run a simple
centos:7 container image:
$ kubectl run centos --stdin --tty --image=centos:7
I passed the
--tty flags to attach to the pod as soon as it is running. We also need a
kubectl binary in the pod:
$ kubectl cp kubectl centos:/bin/
Now, let’s see what happens when we try a
get pods command on the CentoOS 7 container:
$ kubectl get pods Error from server (Forbidden): pods is forbidden: User "system:serviceaccount:soltysh:default" cannot list resource "pods" in API group "" in the namespace "soltysh"
This error says that my user does not have the necessary permissions for this operation. If you look closely at the username (
system:serviceaccount:soltysh:default), you’ll notice it’s actually a default service account assigned to my pod. But where did that come from? To find out, we can increase the verbosity of
kubectl and check the debugging information:
$ kubectl get pods -v=4 I1104 11:51:21.969428 57 merged_client_builder.go:163] Using in-cluster namespace
Notice that I’ve used the in-cluster configuration for this command. The in-cluster configuration checks for a service account token located in
/var/run/secrets/kubernetes.io/serviceaccount/token. It also checks the two environment variables
KUBERNETES_SERVICE_PORT. When it finds all three of these, it knows that it is running inside of a Kubernetes cluster. It then knows that it should read the injected data to talk to the cluster.
Using the view role for read access
Most default Kubernetes clusters with role-based access control (RBAC) turned on have a pre-existing set of cluster roles. Here, we’ll use the
view role, which has read access to all non-escalating resources. To use this role, we need to create a
ClusterRoleBinding, like this:
$ kubectl create clusterrolebinding view-soltysh --clusterrole=view --serviceaccount=soltysh:default
Now, when we re-run the
get pods command, we should be able to view all of the pods running in the current namespace:
$ kubectl get pods NAME READY STATUS RESTARTS AGE centos 1/1 Running 0 18m
The operation succeeded because we used a role that allowed us to view the pods.
Note: If you are wondering where the namespace defaulting happened, check the contents of the
User impersonation with the
Kubernetes has capabilities similar to the
sudo command for Unix. This feature, called user impersonation, lets you invoke any command as a different user. To use this feature in
kubectl, you need to specify the
--as=user flag, where
user is the name of the user you wish to impersonate. Once again, an example will demonstrate the concept.
To set up the demonstration, let’s start as a non-cluster admin user. We will attempt to get the pods from the namespace that we used for the previous example:
$ KUBECONFIG=nonadmin kubectl get pods -n soltysh Error from server (Forbidden): pods is forbidden: User "nonadmin" cannot list resource "pods" in API group "" in the namespace "soltysh"
As expected, we receive an error that we don’t have access to that namespace. Now, let’s try the same command using the
--as=system:admin flag. This lets us impersonate
$ KUBECONFIG=nonadmin kubectl get po -n soltysh --as=system:admin Error from server (Forbidden): users "system:admin" is forbidden: User "nonadmin" cannot impersonate resource "users" in API group "" at the cluster scope
This error tells us that we don’t have the necessary permissions for a user impersonation. Namely, we need to perform the ‘impersonate’ verb on the user attribute, but we currently lack the permissions to do it. How can we access the permissions that we need?
impersonator cluster role
In my cluster, I have an
impersonator cluster role. To use the
impersonate verb, we need to assign this role to the current user. The trick is to invoke the
impersonator operation as a
$ kubectl create clusterrolebinding impersonator-nonadmin --clusterrole=impersonator --user=nonadmin
Now, when we re-run the command as a non-admin user, we can read pods from
$ KUBECONFIG=nonadmin kubectl get po -n soltysh --as=system:admin NAME READY STATUS RESTARTS AGE centos 1/1 Running 0 36m
To confirm that this trick works, let’s see what happens when we try getting the pods without the
$ KUBECONFIG=nonadmin kubectl get po -n soltysh Error from server (Forbidden): pods is forbidden: User "nonadmin" cannot list resource "pods" in API group "" in the namespace "soltysh"
Once again, this command fails. The impersonation only works if we use the
--as flag correctly.
kubectl CLI also has access to the
--as-group flag, which allows you to impersonate a group. Furthermore, you can verify how the command works by using
-v=8 with your
kubectl command. Doing that lets you view all of the headers sent to the cluster.
Specify a custom editor
By default, the
kubectl edit command assumes that you are using
vi (on a Unix-flavored system) or Notepad (in Windows) as your editor. If you prefer a different editor, you can use the
KUBE_EDITOR environment variable to specify it:
$ KUBE_EDITOR="code --wait" kubectl edit po/centos -n soltysh --as=system:admin
In this case, I’m using Visual Studio Code (VS Code). Note that I’ve specified the
--wait flag to ensure that VS Code retains control until all edit operations are complete.
Debug a running application
The last command we’ll look at is
alpha, from Kubernetes 1.19. You can use this command to debug a running application.
kubectl alpha debug command was developed for ephemeral containers but has since transitioned into a full-fledged debugging tool. You can use
kubectl alpha debug to create any of the following:
- Ephemeral containers for debugging your application (assuming the feature is enabled).
- Copies of your running pods with additional tooling to provide insight in case of application failure.
- Pods that you can use to debug your nodes.
kubectl alpha debug command has many more features for you to check out. Additionally, Kubernetes 1.20 promotes this command to beta. If you use the
kubectl CLI with Kubernetes 1.20, you’ll find the
alpha command under
I hope the tips I shared in this article are helpful in your daily work. I will leave you with four takeaways:
kubectlcommand knows how to consume in-cluster configurations to communicate with the cluster that it’s running in. You need to ensure that you have appropriate access rights for the service account assigned to your pod.
--asflag acts like
sudodoes for Unix-based systems. You need to have the appropriate access rights for the impersonate verb.
KUBE_EDITORallows you to choose a different editor for the
- Try the new
kubectl debugcommand for debugging your applications in Kubernetes 1.20 and higher.
If you have questions about these tools or suggestions for improving them, please reach out to me or other SIG-CLI team members. We are the Kubernetes special interest group for the