# DRA API Objects LLMS index: [llms.txt](/llms.txt) --- This page describes the Kubernetes API kinds that dynamic resource allocation (DRA) uses to categorize, request, and allocate devices. ## DRA terminology {#terminology} DRA uses the following Kubernetes API kinds to provide the core allocation functionality. All of these API kinds are included in the `resource.k8s.io/v1` API group. DeviceClass : Defines a category of devices that can be claimed and how to select specific device attributes in claims. The DeviceClass parameters can match zero or more devices in ResourceSlices. To claim devices from a DeviceClass, ResourceClaims select specific device attributes. ResourceClaim : Describes a request for access to attached resources, such as devices, in the cluster. ResourceClaims provide Pods with access to a specific resource. ResourceClaims can be created by workload operators or generated by Kubernetes based on a ResourceClaimTemplate. ResourceClaimTemplate : Defines a template that Kubernetes uses to create per-Pod ResourceClaims for a workload. ResourceClaimTemplates provide Pods with access to separate, similar resources. Each ResourceClaim that Kubernetes generates from the template is bound to a specific Pod. When the Pod terminates, Kubernetes deletes the corresponding ResourceClaim. ResourceSlice : Represents one or more resources that are attached to nodes, such as devices. Drivers create and manage ResourceSlices in the cluster. When a ResourceClaim is created and used in a Pod, Kubernetes uses ResourceSlices to find nodes that have access to the claimed resources. Kubernetes allocates resources to the ResourceClaim and schedules the Pod onto a node that can access the resources. ### DeviceClass {#deviceclass} A DeviceClass lets cluster admins or device drivers define categories of devices in the cluster. DeviceClasses tell operators what devices they can request and how they can request those devices. You can use [common expression language (CEL)](https://cel.dev) to select devices based on specific attributes. A ResourceClaim that references the DeviceClass can then request specific configurations within the DeviceClass. To create a DeviceClass, see [Set Up DRA in a Cluster](/docs/tasks/configure-pod-container/assign-resources/set-up-dra-cluster). ### ResourceClaims and ResourceClaimTemplates {#resourceclaims-templates} A ResourceClaim defines the resources that a workload needs. Every ResourceClaim has _requests_ that reference a DeviceClass and select devices from that DeviceClass. ResourceClaims can also use _selectors_ to filter for devices that meet specific requirements, and can use _constraints_ to limit the devices that can satisfy a request. ResourceClaims can be created by workload operators or can be generated by Kubernetes based on a ResourceClaimTemplate. A ResourceClaimTemplate defines a template that Kubernetes can use to auto-generate ResourceClaims for Pods. #### Use cases for ResourceClaims and ResourceClaimTemplates {#when-to-use-rc-rct} The method that you use depends on your requirements, as follows: * **ResourceClaim**: you want multiple Pods to share access to specific devices. You manually manage the lifecycle of ResourceClaims that you create. * **ResourceClaimTemplate**: you want Pods to have independent access to separate, similarly-configured devices. Kubernetes generates ResourceClaims from the specification in the ResourceClaimTemplate. The lifetime of each generated ResourceClaim is bound to the lifetime of the corresponding Pod. * [**PodGroup ResourceClaimTemplate**](#workload-resource-claims): you want PodGroups to have independent access to separate, similarly-configured devices that can be shared by their Pods. Kubernetes generates one ResourceClaim for the PodGroup from the specification in the ResourceClaimTemplate. The lifetime of each generated ResourceClaim is bound to the lifetime of the corresponding PodGroup. This requires the [`DRAWorkloadResourceClaims`](/docs/reference/command-line-tools-reference/feature-gates/#DRAWorkloadResourceClaims) feature to be enabled. When you define a workload, you can use Common Expression Language (CEL) to filter for specific device attributes or capacity. The available parameters for filtering depend on the device and the drivers. If you directly reference a specific ResourceClaim in a Pod, that ResourceClaim must already exist in the same namespace as the Pod. If the ResourceClaim doesn't exist in the namespace, the Pod won't schedule. This behavior is similar to how a PersistentVolumeClaim must exist in the same namespace as a Pod that references it. You can reference an auto-generated ResourceClaim in a Pod, but this isn't recommended because auto-generated ResourceClaims are bound to the lifetime of the Pod or PodGroup that triggered the generation. To learn how to claim resources using one of these methods, see [Allocate Devices to Workloads with DRA](/docs/tasks/configure-pod-container/assign-resources/allocate-devices-dra/). #### Prioritized list {#prioritized-list}
Feature state: Stable since Kubernetes v1.36; enabled by default
More information about this feature

This is a stable feature in , and has been since version 1.36. It was first available in the v1.33 release.

You can provide a prioritized list of subrequests for requests in a ResourceClaim or ResourceClaimTemplate. The scheduler will then select the first subrequest that can be allocated. This allows users to specify alternative devices that can be used by the workload if the primary choice is not available. In the example below, the ResourceClaimTemplate requested a device with the color black and the size large. If a device with those attributes is not available, the pod cannot be scheduled. With the prioritized list feature, a second alternative can be specified, which requests two devices with the color white and size small. The large black device will be allocated if it is available. If it is not, but two small white devices are available, the pod will still be able to run. ```yaml apiVersion: resource.k8s.io/v1 kind: ResourceClaimTemplate metadata: name: prioritized-list-claim-template spec: spec: devices: requests: - name: req-0 firstAvailable: - name: large-black deviceClassName: resource.example.com selectors: - cel: expression: |- device.attributes["resource-driver.example.com"].color == "black" && device.attributes["resource-driver.example.com"].size == "large" - name: small-white deviceClassName: resource.example.com selectors: - cel: expression: |- device.attributes["resource-driver.example.com"].color == "white" && device.attributes["resource-driver.example.com"].size == "small" count: 2 ``` If the pod is eligible for multiple nodes in the cluster, the scheduler will use the index of chosen subrequests from any prioritized lists as one of the inputs when it scores each node. So nodes that can allocate devices requested in a higher ranked subrequest are more likely to be chosen than nodes that can only allocate devices for lower ranked subrequests. The decision is made on a per-Pod basis, so if the Pod is a member of a ReplicaSet or similar grouping, you cannot rely on all the members of the group having the same subrequest chosen. Your workload must be able to accommodate this. #### Workload ResourceClaims {#workload-resource-claims}
Feature state: Beta since Kubernetes v1.37; disabled by default
More information about this feature

To use this feature, you (or a cluster administrator) will need to enable the DRAWorkloadResourceClaims feature gate for all relevant components in your cluster.

See Enable Or Disable Feature Gates for more information.

When you organize Pods with the [Workload API](/docs/concepts/workloads/workload-api/), you can reserve ResourceClaims for entire PodGroups instead of individual Pods and generate ResourceClaimTemplates for a PodGroup instead of a single Pod, allowing the Pods within a PodGroup to share access to devices allocated to the generated ResourceClaim. This feature targets two problems: - The ResourceClaim API's `status.reservedFor` list can only contain 256 items. Since kube-scheduler only records individual Pods in that list, only 256 Pods can share a ResourceClaim. By allowing PodGroups to be recorded in `status.reservedFor`, many more than 256 Pods can share a ResourceClaim. - Pods can only share a ResourceClaim when its exact name is known. For complex workloads that replicate _groups_ of Pods, ResourceClaims shared by the Pods in each group need to be created and deleted explicitly when the set of groups scales up and down. By generating ResourceClaims for each PodGroup, a single ResourceClaimTemplate can form the basis for ResourceClaims that are both replicated automatically and shareable among the Pods in a PodGroup. The PodGroup API defines a `spec.resourceClaims` field with the same structure and similar meaning as the `spec.resourceClaims` field in the Pod API: ```yaml apiVersion: scheduling.k8s.io/v1beta1 kind: PodGroup metadata: name: training-group namespace: some-ns spec: ... resourceClaims: - name: pg-claim resourceClaimName: my-pg-claim - name: pg-claim-template resourceClaimTemplateName: my-pg-template ``` Like claims made by Pods, claims for PodGroups defining a `resourceClaimName` refer to a ResourceClaim by name. Claims defining a `resourceClaimTemplateName` refer to a ResourceClaimTemplate which replicates into one ResourceClaim for the entire PodGroup that can be shared amongst its Pods. When a Pod defines a claim with a `name`, `resourceClaimName`, and `resourceClaimTemplateName` that all match one of its PodGroup's `spec.resourceClaims`, then kube-scheduler reserves the ResourceClaim for the PodGroup instead of the Pod. If the Pod's claim does not match one made by its PodGroup, then kube-scheduler reserves the ResourceClaim for the Pod. In either case, reservation is recorded in the ResourceClaim's `status.reservedFor`. PodGroup reservations and the corresponding resource allocation persist in the ResourceClaim until the PodGroup is deleted, even if the group no longer has any Pods. When a Pod claim matching a PodGroup claim defines a `resourceClaimTemplateName`, then one ResourceClaim is generated for the PodGroup. Other Pods in the group defining the same claim will share that generated ResourceClaim instead of prompting a new ResourceClaim to be generated for each Pod. Whether or not a `resourceClaimTemplateName` claim matches a PodGroup claim, the name of the generated ResourceClaim is recorded in the Pod's `status.resourceClaimStatuses`. A matching PodGroup claim for a ResourceClaimTemplate only prompts the creation of a ResourceClaim when the [`DRAWorkloadResourceClaims`](/docs/reference/command-line-tools-reference/feature-gates/#DRAWorkloadResourceClaims) feature is enabled. Instead of creating per-Pod ResourceClaims when the feature is disabled, no ResourceClaim is created to prevent creating spurious per-Pod ResourceClaims during cluster upgrades or rollouts/rollbacks of the feature between `kube-apiserver` and `kube-controller-manager`. ResourceClaims generated from a ResourceClaimTemplate for a PodGroup follow the lifecycle of the PodGroup. The ResourceClaim is first created when both the PodGroup and its ResourceClaimTemplate exist. The ResourceClaim is deleted after the PodGroup has been deleted and the ResourceClaim is no longer reserved. Consider the following example: ```yaml apiVersion: scheduling.k8s.io/v1beta1 kind: PodGroup metadata: name: training-group namespace: some-ns spec: ... resourceClaims: - name: pg-claim resourceClaimName: my-pg-claim - name: pg-claim-template resourceClaimTemplateName: my-pg-template --- apiVersion: v1 kind: Pod metadata: name: training-group-pod-1 namespace: some-ns spec: ... schedulingGroup: podGroupName: training-group resourceClaims: - name: pod-claim resourceClaimName: my-pod-claim - name: pod-claim-template resourceClaimTemplateName: my-pod-template - name: pg-claim resourceClaimName: my-pg-claim - name: pg-claim-template resourceClaimTemplateName: my-pg-template ``` In this example, the `training-group` PodGroup has one Pod named `training-group-pod-1`. The Pod's `pod-claim` and `pod-claim-template` claims do not match any claim made by the PodGroup, so those claims are not affected by the PodGroup: ResourceClaim `my-pod-claim` becomes reserved for the Pod and a ResourceClaim is generated from ResourceClaimTemplate `my-pod-template` and also becomes reserved for the Pod. The `pg-claim` and `pg-claim-template` do match claims made by the PodGroup. ResourceClaim `my-pg-claim` becomes reserved for the PodGroup and a ResourceClaim is generated from ResourceClaimTemplate `my-pg-template` and also becomes reserved for the PodGroup. Associating ResourceClaims with Workload API resources is controlled by the [`DRAWorkloadResourceClaims` feature gate](/docs/reference/command-line-tools-reference/feature-gates/#DRAWorkloadResourceClaims) in the `kube-apiserver`, `kube-controller-manager`, `kube-scheduler`, and `kubelet`. ### ResourceSlice {#resourceslice} Each ResourceSlice represents one or more devices in a pool. The pool is managed by a device driver, which creates and manages ResourceSlices. The resources in a pool might be represented by a single ResourceSlice or span multiple ResourceSlices. ResourceSlices provide useful information to device users and to the scheduler, and are crucial for dynamic resource allocation. Every ResourceSlice must include the following information: * **Resource pool**: a group of one or more resources that the driver manages. The pool can span more than one ResourceSlice. Changes to the resources in a pool must be propagated across all of the ResourceSlices in that pool. The device driver that manages the pool is responsible for ensuring that this propagation happens. * **Devices**: devices in the managed pool. A ResourceSlice can list every device in a pool or a subset of the devices in a pool. The ResourceSlice defines device information like attributes, versions, and capacity. Device users can select devices for allocation by filtering for device information in ResourceClaims or in DeviceClasses. * **Nodes**: the nodes that can access the resources. Drivers can choose which nodes can access the resources, whether that's all of the nodes in the cluster, a single named node, or nodes that have specific node labels. Drivers use a controller to reconcile ResourceSlices in the cluster with the information that the driver has to publish. This controller overwrites any manual changes, such as cluster users creating or modifying ResourceSlices. Consider the following example ResourceSlice: ```yaml apiVersion: resource.k8s.io/v1 kind: ResourceSlice metadata: name: cat-slice spec: driver: "resource-driver.example.com" pool: generation: 1 name: "black-cat-pool" resourceSliceCount: 1 # The allNodes field defines whether any node in the cluster can access the device. allNodes: true devices: - name: "large-black-cat" attributes: color: string: "black" size: string: "large" cat: bool: true ``` This ResourceSlice is managed by the `resource-driver.example.com` driver in the `black-cat-pool` pool. The `allNodes: true` field indicates that any node in the cluster can access the devices. There's one device in the ResourceSlice, named `large-black-cat`, with the following attributes: * `color`: `black` * `size`: `large` * `cat`: `true` A DeviceClass could select this ResourceSlice by using these attributes, and a ResourceClaim could filter for specific devices in that DeviceClass. #### Naming and prioritization {#resourceslice-naming-and-prioritization} The order in which the Kubernetes scheduler evaluates devices for allocation is determined by the lexicographical sorting of ResourceSlice and resource pool names. The scheduler uses a first-fit strategy, meaning it selects the first available device that satisfies the claim's requirements. This allows the priority of resource allocation to be influenced by the names assigned to pools and ResourceSlices. Note that pools without [binding conditions](/docs/concepts/resource-management/dynamic-resource-allocation/dra-features/#device-binding-conditions) are always evaluated before those with binding conditions, regardless of their names. For drivers built using the `k8s.io/dynamic-resources/kubeletplugin` Go package or the ResourceSlice controller from that module, these components automatically handle ResourceSlice naming to ensure they are evaluated in the order specified by the driver. ## Admin access {#admin-access}
Feature state: Stable since Kubernetes v1.36; enabled by default
More information about this feature

This is a stable feature in , and has been since version 1.36. It was first available in the v1.32 release.

You can mark a request in a ResourceClaim or ResourceClaimTemplate as having privileged features for maintenance and troubleshooting tasks. A request with admin access grants access to in-use devices and may enable additional permissions when making the device available in a container: ```yaml apiVersion: resource.k8s.io/v1 kind: ResourceClaimTemplate metadata: name: large-black-cat-claim-template spec: spec: devices: requests: - name: req-0 exactly: deviceClassName: resource.example.com allocationMode: All adminAccess: true ``` Admin access is a privileged mode and should not be granted to regular users in multi-tenant clusters. Only users authorized to create ResourceClaim or ResourceClaimTemplate objects in namespaces labeled with `resource.kubernetes.io/admin-access: "true"` (case-sensitive) can use the `adminAccess` field. This ensures that non-admin users cannot misuse the feature. Admin access is controlled by the [`DRAAdminAccess` feature gate](/docs/reference/command-line-tools-reference/feature-gates/#DRAAdminAccess) in the `kube-apiserver`, `kube-scheduler`, and `kubelet`. ## List type attributes {#list-type-attributes}
Feature state: Alpha since Kubernetes v1.36; disabled by default
More information about this feature

To use this feature, you (or a cluster administrator) will need to enable the DRAListTypeAttributes feature gate for all relevant components in your cluster.

See Enable Or Disable Feature Gates for more information.

This feature improves the ResourceSlice API, allowing DRA drivers to specify list values for device attributes instead of only scalars. This is useful for modeling more complex internal node topologies, for example when a CPU has adjacency to multiple PCIe roots. For ResourceClaim authors (end users), this means that the `matchAttribute` and `distinctAttribute` work better for these cases. - `matchAttribute` — the two attributes must have a *non-empty list intersection*, rather than be identical (scalar values are treated as single-item lists). This just means that if one driver publishes a single value for, say, the PCIe root, and another driver publishes a list, the constraint is met as long as the single value appears somewhere in the list. - `distinctAttribute` — the attribute values must be *pairwise-disjoint* (no value shared between any two devices) To help ResourceClaim authors use attributes that may be lists inside CEL expressions, this feature also introduces an `includes()` CEL function. ``` # Scalar attribute (backward compatible) # assume: device.attributes["dra.example.com"].model = "model-a" device.attributes["dra.example.com"].model.includes("model-a") # true device.attributes["dra.example.com"].model.includes("model-b") # false # List-type attribute (requires DRAListTypeAttributes) # assume: device.attributes["dra.example.com"].supported-models= ["model-a", "model-b"] device.attributes["dra.example.com"].supported-models.includes("model-a") # true device.attributes["dra.example.com"].supported-models.includes("model-c") # false ``` ### Details for DRA Driver Authors By default, each `DeviceAttribute` holds exactly one scalar value: a boolean, an integer, a string, or a semantic version string. The `DRAListTypeAttributes` feature gate extends `DeviceAttribute` with four list-type fields, allowing a device to advertise multiple values for a single attribute: - **`bools`** — a list of boolean values - **`ints`** — a list of 64-bit integer values - **`strings`** — a list of strings (each at most 64 characters) - **`versions`** — a list of semantic version strings per semver.org spec 2.0.0 (each at most 64 characters) The total number of individual attribute values per device (scalar fields plus all list elements combined) is limited to **48**. When any device in a ResourceSlice uses this feature or other advanced features such as taints, the ResourceSlice will be limited to at most **64** devices. use list-type attributes or other advanced features such as taints. Here is an example of a device advertising multiple supported models using a list-type string attribute: ```yaml kind: ResourceSlice apiVersion: resource.k8s.io/v1 metadata: name: example-resourceslice spec: nodeName: worker-1 pool: name: pool generation: 1 resourceSliceCount: 1 driver: dra.example.com devices: - name: gpu-0 attributes: dra.example.com/supported-models: strings: - model-a - model-b ``` List type attributes is controlled by the [`DRAListTypeAttributes` feature gate](/docs/reference/command-line-tools-reference/feature-gates/#DRAListTypeAttributes) in the `kube-apiserver` and `kube-scheduler`. ## Derived attributes {#derived-attributes}
Feature state: Alpha since Kubernetes v1.37; disabled by default
More information about this feature

To use this feature, you (or a cluster administrator) will need to enable the DRADerivedAttributes feature gate for all relevant components in your cluster.

See Enable Or Disable Feature Gates for more information.

The `matchAttribute` and `distinctAttribute` constraints normally require devices to publish attributes under the exact same name. If a GPU driver publishes `pcie_locality` and a NIC driver publishes `pcie_root` (or embeds the same information in a string like `numa0-pcie1`), the scheduler has no way to recognize that these represent the same thing, so devices from the two drivers can't be co-located without first agreeing on a shared attribute name. `derivedAttributes` lets you bridge this gap inline, without waiting for drivers to standardize on shared attribute names. Add one or more `derivedAttributes` entries to a request, under `.spec.devices.requests[].exactly` or `.spec.devices.requests[].firstAvailable[]`. Each entry defines a CEL expression that the scheduler evaluates against every candidate device for that request. The result becomes a virtual attribute that can be referenced from a `matchAttribute` or `distinctAttribute` constraint exactly like a driver-provided device attribute. ```yaml apiVersion: resource.k8s.io/v1 kind: ResourceClaim metadata: name: gpu-nic-numa-alignment spec: devices: requests: - name: gpu exactly: deviceClassName: gpu.example.com count: 1 derivedAttributes: - name: derived/numa expression: device.attributes["gpu.example.com"].numa - name: nic exactly: deviceClassName: nic.example.com count: 1 derivedAttributes: - name: derived/numa expression: device.attributes["nic.example.com"].numaNode constraints: - requests: ["gpu", "nic"] matchAttribute: derived/numa ``` In this example, the `gpu` and `nic` drivers publish topology information under different attribute names (`numa` and `numaNode`). Each request computes a common `derived/numa` value from its own device's attributes, and the `matchAttribute` constraint aligns the two requests on that virtual attribute, even though the underlying drivers never agreed on a shared attribute name. A few things to know about `derivedAttributes`: - **Naming**: `name` must be a DNS subdomain followed by a `/` and a C identifier, the same format used for driver-provided attribute names (for example, `example.com/numaNode` or `derived/numaNode`). If the name matches a driver-provided attribute already published by the driver, the derived attribute's value shadows the driver-provided one for constraint matching. Use a domain prefix that no driver would use, such as `derived/`, if you want to avoid shadowing unintentionally. You can define up to 32 derived attributes per request. - **Must be used by a constraint**: every derived attribute must be referenced by at least one `matchAttribute` or `distinctAttribute` constraint that applies to the request (or subrequest) that defines it. Otherwise the ResourceClaim fails validation. - **Evaluation scope and order**: `expression` is evaluated once per candidate device, after the request's own CEL selectors (`.selectors[].cel`) have already filtered that device. As a result, derived attributes can't be referenced from selector expressions, and aren't exposed through `device.attributes` in the CEL environment. - **Return type**: `expression` must evaluate to a scalar (`string`, `int`, `bool`, or a semantic version) or, when the `DRAListTypeAttributes` feature gate is also enabled, a list of one of those scalar types. - **Cost limits**: each expression has a maximum length and a limit on its estimated CEL evaluation cost. On top of that, the combined estimated cost of all `derivedAttributes` expressions in a ResourceClaim is also capped, to bound the total overhead added to a single scheduling attempt. The ResourceClaim is rejected if any of these limits are exceeded. - **Runtime errors abort scheduling**: if evaluating an expression fails for a candidate device, for example because it references an attribute that device doesn't have, the scheduler aborts allocation and the Pod fails to schedule, rather than silently skipping that device. Write expressions defensively, for example by checking that an attribute exists before reading it. Derived attributes is controlled by the [`DRADerivedAttributes` feature gate](/docs/reference/command-line-tools-reference/feature-gates/#DRADerivedAttributes) in the `kube-apiserver` and `kube-scheduler`. For a list of standard device attributes that DRA drivers can publish, see the [Standard Device Attributes](/docs/reference/node/dra-standard-device-attributes/) reference. ## Extended resource allocation by DRA {#extended-resource}
Feature state: Stable since Kubernetes v1.37; enabled by default
More information about this feature

This is a stable feature in , and has been since version 1.37. It was first available in the v1.34 release.

You can provide an extended resource name for a DeviceClass. The scheduler will then select the devices matching the class for the extended resource requests. This allows users to continue using extended resource requests in a pod to request either extended resources provided by device plugin, or DRA devices. The same extended resource can be provided either by device plugin, or DRA on one single cluster node. The same extended resource can be provided by device plugin on some nodes, and DRA on other nodes in the same cluster. In the example below, the DeviceClass is given an extendedResourceName `example.com/gpu`. If a pod requested for the extended resource `example.com/gpu: 2`, it can be scheduled to a node with two or more devices matching the DeviceClass. ```yaml apiVersion: resource.k8s.io/v1 kind: DeviceClass metadata: name: gpu.example.com spec: selectors: - cel: expression: device.driver == 'gpu.example.com' && device.attributes['gpu.example.com'].type == 'gpu' extendedResourceName: example.com/gpu ``` In addition, users can use a special extended resource to allocate devices without having to explicitly create a ResourceClaim. Using the extended resource name prefix `deviceclass.resource.kubernetes.io/` and the DeviceClass name. This works for any DeviceClass, even if it does not specify an extended resource name. The resulting ResourceClaim will contain a request for an `ExactCount` of the specified number of devices of that DeviceClass. Extended resource allocation by DRA is controlled by the [`DRAExtendedResource` feature gate](/docs/reference/command-line-tools-reference/feature-gates/#DRAExtendedResource) in the `kube-apiserver`, `kube-scheduler`, `kube-controller-manager`, and `kubelet`. For a hands-on walkthrough of requesting extended resources, see [Assign Extended Resources to a Container](/docs/tasks/configure-pod-container/extended-resource/).