# Allocate Devices to Workloads with DRA

LLMS index: [llms.txt](/llms.txt)

---

<div class="feature-state-notice feature-stable" title="Feature Gate: DynamicResourceAllocation">
              <span class="feature-state-name">FEATURE STATE:</span> 
              <code>Kubernetes v1.35 [stable]</code>(enabled by default)</div>


<!-- overview -->

This page shows you how to allocate devices to your Pods by using
_dynamic resource allocation (DRA)_. These instructions are for workload
operators. Before reading this page, familiarize yourself with how DRA works and
with DRA terminology like
<a class='glossary-tooltip' title='Describes the resources that a workload needs, such as devices. ResourceClaims can request devices from DeviceClasses.' data-bs-toggle='tooltip' data-bs-placement='top' href='/docs/concepts/scheduling-eviction/dynamic-resource-allocation/#resourceclaims-templates' target='_blank' aria-label='ResourceClaims'>ResourceClaims</a> and
<a class='glossary-tooltip' title='Defines a template for Kubernetes to create ResourceClaims. Used to provide per-Pod or per-PodGroup access to separate, similar resources.' data-bs-toggle='tooltip' data-bs-placement='top' href='/docs/concepts/scheduling-eviction/dynamic-resource-allocation/#resourceclaims-templates' target='_blank' aria-label='ResourceClaimTemplates'>ResourceClaimTemplates</a>.
For more information, see
[Dynamic Resource Allocation (DRA)](/docs/concepts/scheduling-eviction/dynamic-resource-allocation/).

<!-- body -->

## About device allocation with DRA {#about-device-allocation-dra}

As a workload operator, you can _claim_ devices for your workloads by creating
ResourceClaims or ResourceClaimTemplates. When you deploy your workload,
Kubernetes and the device drivers find available devices, allocate them to your
Pods, and place the Pods on nodes that can access those devices.

<!-- prerequisites -->

## Before you begin

<p>You need to have a Kubernetes cluster, and the kubectl command-line tool must
be configured to communicate with your cluster. It is recommended to run this tutorial on a cluster with at least two nodes that are not acting as control plane hosts. If you do not already have a
cluster, you can create one by using
<a href="https://minikube.sigs.k8s.io/docs/tutorials/multi_node/">minikube</a>
or you can use one of these Kubernetes playgrounds:</p>
<ul>
<li><a href="https://labs.iximiuz.com/playgrounds?category=kubernetes&filter=all">iximiuz Labs</a></li>
<li><a href="https://killercoda.com/playgrounds/scenario/kubernetes">Killercoda</a></li>
<li><a href="https://kodekloud.com/public-playgrounds">KodeKloud</a></li>
</ul>
 
 
 Your Kubernetes server must be at or later than version v1.34.
  <p>To check the version, enter  <code>kubectl version</code>.</p>


* Ensure that your cluster admin has set up DRA, attached devices, and installed
  drivers. For more information, see
  [Set Up DRA in a Cluster](/docs/tasks/configure-pod-container/assign-resources/set-up-dra-cluster).

<!-- steps -->

## Identify devices to claim {#identify-devices}

Your cluster administrator or the device drivers create
_<a class='glossary-tooltip' title='A category of devices in the cluster. Users can claim specific devices in a DeviceClass.' data-bs-toggle='tooltip' data-bs-placement='top' href='/docs/concepts/scheduling-eviction/dynamic-resource-allocation/#deviceclass' target='_blank' aria-label='DeviceClasses'>DeviceClasses</a>_ that
define categories of devices. You can claim devices by using
<a class='glossary-tooltip' title='An expression language that&#39;s designed to be safe for executing user code.' data-bs-toggle='tooltip' data-bs-placement='top' href='https://cel.dev' target='_blank' aria-label='Common Expression Language'>Common Expression Language</a> to filter for specific device properties.

Get a list of DeviceClasses in the cluster:

```shell
kubectl get deviceclasses
```
The output is similar to the following:

```
NAME                 AGE
driver.example.com   16m
```
If you get a permission error, you might not have access to get DeviceClasses.
Check with your cluster administrator or with the driver provider for available
device properties.

## Claim resources {#claim-resources}

You can request resources from a DeviceClass by using 
<a class='glossary-tooltip' title='Describes the resources that a workload needs, such as devices. ResourceClaims can request devices from DeviceClasses.' data-bs-toggle='tooltip' data-bs-placement='top' href='/docs/concepts/scheduling-eviction/dynamic-resource-allocation/#resourceclaims-templates' target='_blank' aria-label='ResourceClaims'>ResourceClaims</a>. To
create a ResourceClaim, do one of the following:

* Manually create a ResourceClaim if you want multiple Pods to share access to
  the same devices, or if you want a claim to exist beyond the lifetime of a
  Pod.
* Use a
  <a class='glossary-tooltip' title='Defines a template for Kubernetes to create ResourceClaims. Used to provide per-Pod or per-PodGroup access to separate, similar resources.' data-bs-toggle='tooltip' data-bs-placement='top' href='/docs/concepts/scheduling-eviction/dynamic-resource-allocation/#resourceclaims-templates' target='_blank' aria-label='ResourceClaimTemplate'>ResourceClaimTemplate</a>
  to let Kubernetes generate and manage per-Pod ResourceClaims. Create a
  ResourceClaimTemplate if you want every Pod to have access to separate devices
  that have similar configurations. For example, you might want simultaneous
  access to devices for Pods in a Job that uses
  [parallel execution](/docs/concepts/workloads/controllers/job/#parallel-jobs).

If you directly reference a specific ResourceClaim in a Pod, that ResourceClaim
must already exist in the cluster. If a referenced ResourceClaim doesn't exist,
the Pod remains in a pending state until the ResourceClaim is created. You can
reference an auto-generated ResourceClaim in a Pod, but this isn't recommended
because auto-generated ResourceClaims are bound to the lifetime of the Pod that
triggered the generation.

To create a workload that claims resources, select one of the following options:

<ul class="nav nav-tabs" id="tabs-claim-resources" role="tablist"><li class="nav-item"><a data-bs-toggle="tab" class="nav-link active" href="#tabs-claim-resources-0" role="tab" aria-controls="tabs-claim-resources-0" aria-selected="true">ResourceClaimTemplate</a></li>
	  
		<li class="nav-item"><a data-bs-toggle="tab" class="nav-link" href="#tabs-claim-resources-1" role="tab" aria-controls="tabs-claim-resources-1">ResourceClaim</a></li></ul>

<div class="tab-content" id="tabs-claim-resources-content"><div class="tab-body tab-pane fadeshow active"
        id="tabs-claim-resources-0" role="tabpanel" aria-labelledby="tabs-claim-resources-0-tab" tabindex="claim-resources"><p>Review the following example manifest:</p>
<div class="highlight code-sample">
    <div class="copy-code-icon">
    <a href="https://raw.githubusercontent.com/kubernetes/website/main/content/en/examples/dra/resourceclaimtemplate.yaml" download="dra/resourceclaimtemplate.yaml"><code>dra/resourceclaimtemplate.yaml</code>
    </a><img src="/images/copycode.svg" class="icon-copycode" onclick="copyCode('dra-resourceclaimtemplate-yaml')" title="Copy dra/resourceclaimtemplate.yaml to clipboard"></img></div>
    <div class="includecode" id="dra-resourceclaimtemplate-yaml"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl"><span class="nt">apiVersion</span><span class="p">:</span><span class="w"> </span><span class="l">resource.k8s.io/v1</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="nt">kind</span><span class="p">:</span><span class="w"> </span><span class="l">ResourceClaimTemplate</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="nt">metadata</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">example-resource-claim-template</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="nt">spec</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">spec</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">devices</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span><span class="nt">requests</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">gpu-claim</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">        </span><span class="nt">exactly</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">          </span><span class="nt">deviceClassName</span><span class="p">:</span><span class="w"> </span><span class="l">example-device-class</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">          </span><span class="nt">selectors</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">            </span>- <span class="nt">cel</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">                </span><span class="nt">expression</span><span class="p">:</span><span class="w"> </span><span class="p">|-</span><span class="sd">
</span></span></span><span class="line"><span class="cl"><span class="sd">                  device.attributes[&#34;driver.example.com&#34;].type == &#34;gpu&#34; &amp;&amp;
</span></span></span><span class="line"><span class="cl"><span class="sd">                  device.capacity[&#34;driver.example.com&#34;].memory == quantity(&#34;64Gi&#34;)</span><span class="w">
</span></span></span></code></pre></div></div>
</div>
<p>This manifest creates a ResourceClaimTemplate that requests devices in the
<code>example-device-class</code> DeviceClass that match both of the following parameters:</p>
<ul>
<li>Devices that have a <code>driver.example.com/type</code> attribute with a value of
<code>gpu</code>.</li>
<li>Devices that have <code>64Gi</code> of capacity.</li>
</ul>
<p>To create the ResourceClaimTemplate, run the following command:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-shell" data-lang="shell"><span class="line"><span class="cl">kubectl apply -f https://k8s.io/examples/dra/resourceclaimtemplate.yaml
</span></span></code></pre></div></div><div class="tab-body tab-pane fade"
        id="tabs-claim-resources-1" role="tabpanel" aria-labelledby="tabs-claim-resources-1-tab" tabindex="claim-resources"><p>Review the following example manifest:</p>
<div class="highlight code-sample">
    <div class="copy-code-icon">
    <a href="https://raw.githubusercontent.com/kubernetes/website/main/content/en/examples/dra/resourceclaim.yaml" download="dra/resourceclaim.yaml"><code>dra/resourceclaim.yaml</code>
    </a><img src="/images/copycode.svg" class="icon-copycode" onclick="copyCode('dra-resourceclaim-yaml')" title="Copy dra/resourceclaim.yaml to clipboard"></img></div>
    <div class="includecode" id="dra-resourceclaim-yaml"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl"><span class="nt">apiVersion</span><span class="p">:</span><span class="w"> </span><span class="l">resource.k8s.io/v1</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="nt">kind</span><span class="p">:</span><span class="w"> </span><span class="l">ResourceClaim</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="nt">metadata</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">example-resource-claim</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="nt">spec</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">devices</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">requests</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">single-gpu-claim</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span><span class="nt">exactly</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">        </span><span class="nt">deviceClassName</span><span class="p">:</span><span class="w"> </span><span class="l">example-device-class</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">        </span><span class="nt">allocationMode</span><span class="p">:</span><span class="w"> </span><span class="l">All</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">        </span><span class="nt">selectors</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">        </span>- <span class="nt">cel</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">            </span><span class="nt">expression</span><span class="p">:</span><span class="w"> </span><span class="p">|-</span><span class="sd">
</span></span></span><span class="line"><span class="cl"><span class="sd">              device.attributes[&#34;driver.example.com&#34;].type == &#34;gpu&#34; &amp;&amp;
</span></span></span><span class="line"><span class="cl"><span class="sd">              device.capacity[&#34;driver.example.com&#34;].memory == quantity(&#34;64Gi&#34;)</span><span class="w">
</span></span></span></code></pre></div></div>
</div>
<p>This manifest creates ResourceClaim that requests devices in the
<code>example-device-class</code> DeviceClass that match both of the following parameters:</p>
<ul>
<li>Devices that have a <code>driver.example.com/type</code> attribute with a value of
<code>gpu</code>.</li>
<li>Devices that have <code>64Gi</code> of capacity.</li>
</ul>
<p>To create the ResourceClaim, run the following command:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-shell" data-lang="shell"><span class="line"><span class="cl">kubectl apply -f https://k8s.io/examples/dra/resourceclaim.yaml
</span></span></code></pre></div></div></div>


## Request devices in workloads using DRA {#request-devices-workloads}

To request device allocation, specify a ResourceClaim or a ResourceClaimTemplate
in the `resourceClaims` field of the Pod specification. Then, request a specific
claim by name in the `resources.claims` field of a container in that Pod.
You can specify multiple entries in the `resourceClaims` field and use specific
claims in different containers.

1. Review the following example Job:

   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   <div class="highlight code-sample">
       <div class="copy-code-icon">
       <a href="https://raw.githubusercontent.com/kubernetes/website/main/content/en/examples/dra/dra-example-job.yaml" download="dra/dra-example-job.yaml"><code>dra/dra-example-job.yaml</code>
       </a><img src="/images/copycode.svg" class="icon-copycode" onclick="copyCode('dra-dra-example-job-yaml')" title="Copy dra/dra-example-job.yaml to clipboard"></img></div>
       <div class="includecode" id="dra-dra-example-job-yaml"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl"><span class="nt">apiVersion</span><span class="p">:</span><span class="w"> </span><span class="l">batch/v1</span><span class="w">
   </span></span></span><span class="line"><span class="cl"><span class="nt">kind</span><span class="p">:</span><span class="w"> </span><span class="l">Job</span><span class="w">
   </span></span></span><span class="line"><span class="cl"><span class="nt">metadata</span><span class="p">:</span><span class="w">
   </span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">example-dra-job</span><span class="w">
   </span></span></span><span class="line"><span class="cl"><span class="nt">spec</span><span class="p">:</span><span class="w">
   </span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">completions</span><span class="p">:</span><span class="w"> </span><span class="m">10</span><span class="w">
   </span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">parallelism</span><span class="p">:</span><span class="w"> </span><span class="m">2</span><span class="w">
   </span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">template</span><span class="p">:</span><span class="w">
   </span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">spec</span><span class="p">:</span><span class="w">
   </span></span></span><span class="line"><span class="cl"><span class="w">      </span><span class="nt">restartPolicy</span><span class="p">:</span><span class="w"> </span><span class="l">Never</span><span class="w">
   </span></span></span><span class="line"><span class="cl"><span class="w">      </span><span class="nt">containers</span><span class="p">:</span><span class="w">
   </span></span></span><span class="line"><span class="cl"><span class="w">      </span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">container0</span><span class="w">
   </span></span></span><span class="line"><span class="cl"><span class="w">        </span><span class="nt">image</span><span class="p">:</span><span class="w"> </span><span class="l">ubuntu:24.04</span><span class="w">
   </span></span></span><span class="line"><span class="cl"><span class="w">        </span><span class="nt">command</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">&#34;sleep&#34;</span><span class="p">,</span><span class="w"> </span><span class="s2">&#34;9999&#34;</span><span class="p">]</span><span class="w">
   </span></span></span><span class="line"><span class="cl"><span class="w">        </span><span class="nt">resources</span><span class="p">:</span><span class="w">
   </span></span></span><span class="line"><span class="cl"><span class="w">          </span><span class="nt">claims</span><span class="p">:</span><span class="w">
   </span></span></span><span class="line"><span class="cl"><span class="w">          </span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">separate-gpu-claim</span><span class="w">
   </span></span></span><span class="line"><span class="cl"><span class="w">      </span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">container1</span><span class="w">
   </span></span></span><span class="line"><span class="cl"><span class="w">        </span><span class="nt">image</span><span class="p">:</span><span class="w"> </span><span class="l">ubuntu:24.04</span><span class="w">
   </span></span></span><span class="line"><span class="cl"><span class="w">        </span><span class="nt">command</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">&#34;sleep&#34;</span><span class="p">,</span><span class="w"> </span><span class="s2">&#34;9999&#34;</span><span class="p">]</span><span class="w">
   </span></span></span><span class="line"><span class="cl"><span class="w">        </span><span class="nt">resources</span><span class="p">:</span><span class="w">
   </span></span></span><span class="line"><span class="cl"><span class="w">          </span><span class="nt">claims</span><span class="p">:</span><span class="w">
   </span></span></span><span class="line"><span class="cl"><span class="w">          </span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">shared-gpu-claim</span><span class="w">
   </span></span></span><span class="line"><span class="cl"><span class="w">      </span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">container2</span><span class="w">
   </span></span></span><span class="line"><span class="cl"><span class="w">        </span><span class="nt">image</span><span class="p">:</span><span class="w"> </span><span class="l">ubuntu:24.04</span><span class="w">
   </span></span></span><span class="line"><span class="cl"><span class="w">        </span><span class="nt">command</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">&#34;sleep&#34;</span><span class="p">,</span><span class="w"> </span><span class="s2">&#34;9999&#34;</span><span class="p">]</span><span class="w">
   </span></span></span><span class="line"><span class="cl"><span class="w">        </span><span class="nt">resources</span><span class="p">:</span><span class="w">
   </span></span></span><span class="line"><span class="cl"><span class="w">          </span><span class="nt">claims</span><span class="p">:</span><span class="w">
   </span></span></span><span class="line"><span class="cl"><span class="w">          </span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">shared-gpu-claim</span><span class="w">
   </span></span></span><span class="line"><span class="cl"><span class="w">      </span><span class="nt">resourceClaims</span><span class="p">:</span><span class="w">
   </span></span></span><span class="line"><span class="cl"><span class="w">      </span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">separate-gpu-claim</span><span class="w">
   </span></span></span><span class="line"><span class="cl"><span class="w">        </span><span class="nt">resourceClaimTemplateName</span><span class="p">:</span><span class="w"> </span><span class="l">example-resource-claim-template</span><span class="w">
   </span></span></span><span class="line"><span class="cl"><span class="w">      </span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">shared-gpu-claim</span><span class="w">
   </span></span></span><span class="line"><span class="cl"><span class="w">        </span><span class="nt">resourceClaimName</span><span class="p">:</span><span class="w"> </span><span class="l">example-resource-claim</span><span class="w">
   </span></span></span></code></pre></div></div>
   </div>

   Each Pod in this Job has the following properties:
   
   * Makes a ResourceClaimTemplate named `separate-gpu-claim` and a
     ResourceClaim named `shared-gpu-claim` available to containers.
   * Runs the following containers:
       * `container0` requests the devices from the `separate-gpu-claim` 
         ResourceClaimTemplate. 
       * `container1` and `container2` share access to the devices from the
         `shared-gpu-claim` ResourceClaim.

1. Create the Job: 

   ```shell
   kubectl apply -f https://k8s.io/examples/dra/dra-example-job.yaml
   ```

Try the following troubleshooting steps:

1. When the workload does not start as expected, drill down from Job
   to Pods to ResourceClaims and check the objects
   at each level with `kubectl describe` to see whether there are any
   status fields or events which might explain why the workload is
   not starting.
1. When creating a Pod fails with `must specify one of: resourceClaimName,
   resourceClaimTemplateName`, check that all entries in `pod.spec.resourceClaims`
   have exactly one of those fields set. If they do, then it is possible
   that the cluster has a mutating Pod webhook installed which was built
   against APIs from Kubernetes < 1.32. Work with your cluster administrator
   to check this.

## Clean up {#clean-up}

To delete the Kubernetes objects that you created in this task, follow these
steps:

1.  Delete the example Job:

    ```shell
    kubectl delete -f https://k8s.io/examples/dra/dra-example-job.yaml
    ```

1.  To delete your resource claims, run one of the following commands:

    * Delete the ResourceClaimTemplate:

      ```shell
      kubectl delete -f https://k8s.io/examples/dra/resourceclaimtemplate.yaml
      ```
    * Delete the ResourceClaim:

      ```shell
      kubectl delete -f https://k8s.io/examples/dra/resourceclaim.yaml
      ```

## What's next

* [Learn more about DRA](/docs/concepts/scheduling-eviction/dynamic-resource-allocation)
