Tools→YARN Resource Calculator

YARN Resource Calculator

Compute production-ready YARN, MapReduce, and Spark configuration for your CDP cluster

RAM per node (GB)

CPUs per node

Cores per CPU

HyperThreading: On

HDDs per node

Disk size (TB)

Configuration Output

# NodeManager

yarn.nodemanager.resource.cpu-vcores = 44

yarn.nodemanager.resource.memory-mb = 250880

yarn.nodemanager.vmem-check-enabled = false

# Scheduler

yarn.scheduler.minimum-allocation-vcores = 1

yarn.scheduler.maximum-allocation-vcores = 44

yarn.scheduler.minimum-allocation-mb = 1024

yarn.scheduler.maximum-allocation-mb = 250880

# MapReduce

mapreduce.map.memory.mb = 1024

mapreduce.map.cpu.vcores = 1

mapreduce.map.java.opts = -Xmx819m

mapreduce.reduce.memory.mb = 1024

mapreduce.reduce.cpu.vcores = 1

mapreduce.reduce.java.opts = -Xmx819m

yarn.app.mapreduce.am.resource.mb = 1024

yarn.app.mapreduce.am.resource.cpu-vcores = 1

yarn.app.mapreduce.am.command-opts = -Xmx819m

mapreduce.task.io.sort.mb = 409

mapreduce.job.heap.memory-mb.ratio = 0.8

# Spark Executor

--num-executors = 109

--executor-cores = 4

--executor-memory = 20g

spark.yarn.executor.memoryOverhead = 2280m

--driver-memory = 4g

--driver-cores = 1

spark.dynamicAllocation.enabled = true

Sanity Checks

9 pass1 warn

Scheduler max memory >= min memory

Max 250880 MB >= Min 1024 MB

Scheduler max vcores >= min vcores

Max 44 >= Min 1

NodeManager memory >= scheduler max

NM 250880 MB >= Scheduler max 250880 MB

NodeManager vcores >= scheduler max

NM 44 >= Scheduler max 44

Min container size >= 1024 MB

Min 1024 MB is adequate

Heap ratio between 0.75 and 0.9

Heap ratio 0.8 is in recommended range

Spark executor fits in NodeManager

Request 22807 MB <= Available 250880 MB

Spark executor cores <= 5

4 cores is optimal

Spark overhead >= 384 MB

Overhead 2280 MB >= Spark minimum 384 MB

vmem-check should be disabled

Recommend yarn.nodemanager.vmem-check-enabled = false (unstable, platform-dependent)

Master YARN tuning in the CDP Administration course

How YARN Resource Calculation Works

YARN resource calculation starts with your worker node hardware — RAM, CPUs, and disk — then subtracts resources reserved for system services (OS, Cloudera Manager Agent, DataNode, NodeManager). The remaining resources become available for YARN containers.

The calculator then sizes containers for MapReduce (map tasks, reduce tasks, ApplicationMaster) and Spark executors. Each configuration value maps directly to a Cloudera Manager property that you can copy and apply to your cluster.

Sanity checks validate that your configuration is internally consistent — for example, that the scheduler maximum allocation doesn't exceed NodeManager capacity, and that Spark executor memory plus overhead fits within a single node.