# Set Up DRA in a Cluster

LLMS index: [llms.txt](/llms.txt)

---

<div class="feature-state-notice feature-stable" title="Feature Gate: DynamicResourceAllocation">
              <span class="feature-state-name">FEATURE STATE:</span> 
              <code>Kubernetes v1.35 [stable]</code>(enabled by default)</div>


<!-- overview -->

This page shows you how to configure _dynamic resource allocation (DRA)_ in a
Kubernetes cluster by enabling API groups and configuring classes of devices.
These instructions are for cluster administrators. 

<!-- body -->

## About DRA {#about-dra}

<p>A Kubernetes feature that lets you request and share resources among Pods.
These resources are often attached
<a class='glossary-tooltip' title='Any resource that&#39;s directly or indirectly attached your cluster&#39;s nodes, like GPUs or circuit boards.' data-bs-toggle='tooltip' data-bs-placement='top' href='/docs/reference/glossary/?all=true#term-device' target='_blank' aria-label='devices'>devices</a> like hardware
accelerators.</p>
<p>With DRA, device drivers and cluster admins define device <em>classes</em> that are
available to <em>claim</em> in workloads. Kubernetes allocates matching devices to
specific claims and places the corresponding Pods on nodes that can access the
allocated devices.</p>

Ensure that you're familiar with how DRA works and with DRA terminology like
<a class='glossary-tooltip' title='A category of devices in the cluster. Users can claim specific devices in a DeviceClass.' data-bs-toggle='tooltip' data-bs-placement='top' href='/docs/concepts/scheduling-eviction/dynamic-resource-allocation/#deviceclass' target='_blank' aria-label='DeviceClasses'>DeviceClasses</a>,
<a class='glossary-tooltip' title='Describes the resources that a workload needs, such as devices. ResourceClaims can request devices from DeviceClasses.' data-bs-toggle='tooltip' data-bs-placement='top' href='/docs/concepts/scheduling-eviction/dynamic-resource-allocation/#resourceclaims-templates' target='_blank' aria-label='ResourceClaims'>ResourceClaims</a>, and
<a class='glossary-tooltip' title='Defines a template for Kubernetes to create ResourceClaims. Used to provide per-Pod or per-PodGroup access to separate, similar resources.' data-bs-toggle='tooltip' data-bs-placement='top' href='/docs/concepts/scheduling-eviction/dynamic-resource-allocation/#resourceclaims-templates' target='_blank' aria-label='ResourceClaimTemplates'>ResourceClaimTemplates</a>.
For details, see
[Dynamic Resource Allocation (DRA)](/docs/concepts/scheduling-eviction/dynamic-resource-allocation/).

<!-- prerequisites -->

## Before you begin

<p>You need to have a Kubernetes cluster, and the kubectl command-line tool must
be configured to communicate with your cluster. It is recommended to run this tutorial on a cluster with at least two nodes that are not acting as control plane hosts. If you do not already have a
cluster, you can create one by using
<a href="https://minikube.sigs.k8s.io/docs/tutorials/multi_node/">minikube</a>
or you can use one of these Kubernetes playgrounds:</p>
<ul>
<li><a href="https://labs.iximiuz.com/playgrounds?category=kubernetes&filter=all">iximiuz Labs</a></li>
<li><a href="https://killercoda.com/playgrounds/scenario/kubernetes">Killercoda</a></li>
<li><a href="https://kodekloud.com/public-playgrounds">KodeKloud</a></li>
</ul>
 
 
 Your Kubernetes server must be at or later than version v1.34.
  <p>To check the version, enter  <code>kubectl version</code>.</p>


* Directly or indirectly attach devices to your cluster. To avoid potential
  issues with drivers, wait until you set up the DRA feature for your
  cluster before you install drivers.

<!-- steps -->

## Optional: enable additional DRA API groups {#enable-dra}

DRA overall is a stable feature in Kubernetes; however, aspects of it may still be alpha or beta.
If you want to use any aspect of DRA that is not yet stable,
and the associated feature relies on a dedicated API kind,
then you must enable the associated alpha or beta API groups.

Some older DRA drivers or workloads might still need the
v1beta1 API from Kubernetes 1.30 or v1beta2 from Kubernetes 1.32.
If and only if support for those is desired, then enable the following
<a class='glossary-tooltip' title='A set of related paths in the Kubernetes API.' data-bs-toggle='tooltip' data-bs-placement='top' href='/docs/concepts/overview/kubernetes-api/#api-groups-and-versioning' target='_blank' aria-label='API groups'>API groups</a>:

    * `resource.k8s.io/v1beta1`
    * `resource.k8s.io/v1beta2`

Alpha features with separate API types need:

   * `resource.k8s.io/v1alpha3`

For more information, see
[Enabling or disabling API groups](/docs/reference/using-api/#enabling-or-disabling).

## Verify that DRA is enabled {#verify}

To verify that the cluster is configured correctly, try to list DeviceClasses:

```shell
kubectl get deviceclasses
```
If the component configuration was correct, the output is similar to the
following:

```
No resources found
```

If DRA isn't correctly configured, the output of the preceding command is
similar to the following:
   
```
error: the server doesn't have a resource type "deviceclasses"
```

For example, this can occur when the resource.k8s.io API group was disabled.
A similar check is applicable to alpha or beta quality top-level types.

Try the following troubleshooting steps:

1. Reconfigure and restart the `kube-apiserver` component.

1. If the complete `.spec.resourceClaims` field gets removed from Pods, or if
   Pods get scheduled without considering the ResourceClaims, then verify
   that the `DynamicResourceAllocation` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/) is not turned off
   for kube-apiserver, kube-controller-manager, kube-scheduler or the kubelet.

## Install device drivers {#install-drivers}

After you enable DRA for your cluster, you can install the drivers for your
attached devices. For instructions, check the documentation of your device
owner or the project that maintains the device drivers. The drivers that you
install must be compatible with DRA.

To verify that your installed drivers are working as expected, list
ResourceSlices in your cluster:

```shell
kubectl get resourceslices
```
The output is similar to the following:

```
NAME                                                  NODE                DRIVER               POOL                             AGE
00000-driver.example.com-cluster-1-node-1-abcde      cluster-1-node-1    driver.example.com   cluster-1-device-pool-1-r1gc     7s
00000-driver.example.com-cluster-1-node-2-fghij      cluster-1-node-2    driver.example.com   cluster-1-device-pool-2-446z     8s
```

Try the following troubleshooting steps:

1. Check the health of the DRA driver and look for error messages about
   publishing ResourceSlices in its log output. The vendor of the driver
   may have further instructions about installation and troubleshooting.

## Create DeviceClasses {#create-deviceclasses}

You can define categories of devices that your application operators can
claim in workloads by creating
<a class='glossary-tooltip' title='A category of devices in the cluster. Users can claim specific devices in a DeviceClass.' data-bs-toggle='tooltip' data-bs-placement='top' href='/docs/concepts/scheduling-eviction/dynamic-resource-allocation/#deviceclass' target='_blank' aria-label='DeviceClasses'>DeviceClasses</a>. Some device
driver providers might also instruct you to create DeviceClasses during driver
installation.

The ResourceSlices that your driver publishes contain information about the
devices that the driver manages, such as capacity, metadata, and attributes. You
can use <a class='glossary-tooltip' title='An expression language that&#39;s designed to be safe for executing user code.' data-bs-toggle='tooltip' data-bs-placement='top' href='https://cel.dev' target='_blank' aria-label='Common Expression Language'>Common Expression Language</a> to filter for properties in your
DeviceClasses, which can make finding devices easier for your workload
operators.

1.  To find the device properties that you can select in DeviceClasses by using
    CEL expressions, get the specification of a ResourceSlice:

    ```shell
    kubectl get resourceslice <resourceslice-name> -o yaml
    ```
    The output is similar to the following:

    ```yaml
    apiVersion: resource.k8s.io/v1
    kind: ResourceSlice
    # lines omitted for clarity
    spec:
      devices:
      - attributes:
          type:
            string: gpu
        capacity:
          memory:
            value: 64Gi
        name: gpu-0
      - attributes:
          type:
            string: gpu
        capacity:
          memory:
            value: 64Gi
        name: gpu-1
      driver: driver.example.com
      nodeName: cluster-1-node-1
    # lines omitted for clarity
    ```
    You can also check the driver provider's documentation for available
    properties and values.

1.  Review the following example DeviceClass manifest, which selects any device
    that's managed by the `driver.example.com` device driver:

    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    <div class="highlight code-sample">
        <div class="copy-code-icon">
        <a href="https://raw.githubusercontent.com/kubernetes/website/main/content/en/examples/dra/deviceclass.yaml" download="dra/deviceclass.yaml"><code>dra/deviceclass.yaml</code>
        </a><img src="/images/copycode.svg" class="icon-copycode" onclick="copyCode('dra-deviceclass-yaml')" title="Copy dra/deviceclass.yaml to clipboard"></img></div>
        <div class="includecode" id="dra-deviceclass-yaml"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl"><span class="nt">apiVersion</span><span class="p">:</span><span class="w"> </span><span class="l">resource.k8s.io/v1</span><span class="w">
    </span></span></span><span class="line"><span class="cl"><span class="nt">kind</span><span class="p">:</span><span class="w"> </span><span class="l">DeviceClass</span><span class="w">
    </span></span></span><span class="line"><span class="cl"><span class="nt">metadata</span><span class="p">:</span><span class="w">
    </span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">example-device-class</span><span class="w">
    </span></span></span><span class="line"><span class="cl"><span class="nt">spec</span><span class="p">:</span><span class="w">
    </span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">selectors</span><span class="p">:</span><span class="w">
    </span></span></span><span class="line"><span class="cl"><span class="w">  </span>- <span class="nt">cel</span><span class="p">:</span><span class="w">
    </span></span></span><span class="line"><span class="cl"><span class="w">      </span><span class="nt">expression</span><span class="p">:</span><span class="w"> </span><span class="p">|-</span><span class="sd">
    </span></span></span><span class="line"><span class="cl"><span class="sd">        device.driver == &#34;driver.example.com&#34;</span><span class="w">
    </span></span></span></code></pre></div></div>
    </div>

1.  Create the DeviceClass in your cluster:

    ```shell
    kubectl apply -f https://k8s.io/examples/dra/deviceclass.yaml
    ```

## Clean up {#clean-up}

To delete the DeviceClass that you created in this task, run the following
command:

```shell
kubectl delete -f https://k8s.io/examples/dra/deviceclass.yaml
```

## What's next

* [Learn more about DRA](/docs/concepts/scheduling-eviction/dynamic-resource-allocation)
* [Allocate Devices to Workloads with DRA](/docs/tasks/configure-pod-container/assign-resources/allocate-devices-dra)
