.TH "GCLOUD_DATAPROC_CLUSTERS_CREATE" 1



.SH "NAME"
.HP
gcloud dataproc clusters create \- create a cluster



.SH "SYNOPSIS"
.HP
\f5gcloud dataproc clusters create\fR (\fICLUSTER\fR\ :\ \fB\-\-region\fR=\fIREGION\fR) [\fB\-\-action\-on\-failed\-primary\-workers\fR=\fIACTION_ON_FAILED_PRIMARY_WORKERS\fR] [\fB\-\-async\fR] [\fB\-\-autoscaling\-policy\fR=\fIAUTOSCALING_POLICY\fR] [\fB\-\-bucket\fR=\fIBUCKET\fR] [\fB\-\-cluster\-type\fR=\fITYPE\fR] [\fB\-\-confidential\-compute\fR] [\fB\-\-dataproc\-metastore\fR=\fIDATAPROC_METASTORE\fR] [\fB\-\-delete\-max\-idle\fR=\fIDELETE_MAX_IDLE\fR] [\fB\-\-driver\-pool\-accelerator\fR=[\fItype\fR=\fITYPE\fR,[\fIcount\fR=\fICOUNT\fR],...]] [\fB\-\-driver\-pool\-boot\-disk\-size\fR=\fIDRIVER_POOL_BOOT_DISK_SIZE\fR] [\fB\-\-driver\-pool\-boot\-disk\-type\fR=\fIDRIVER_POOL_BOOT_DISK_TYPE\fR] [\fB\-\-driver\-pool\-id\fR=\fIDRIVER_POOL_ID\fR] [\fB\-\-driver\-pool\-local\-ssd\-interface\fR=\fIDRIVER_POOL_LOCAL_SSD_INTERFACE\fR] [\fB\-\-driver\-pool\-machine\-type\fR=\fIDRIVER_POOL_MACHINE_TYPE\fR] [\fB\-\-driver\-pool\-min\-cpu\-platform\fR=\fIPLATFORM\fR] [\fB\-\-driver\-pool\-size\fR=\fIDRIVER_POOL_SIZE\fR] [\fB\-\-enable\-component\-gateway\fR] [\fB\-\-initialization\-action\-timeout\fR=\fITIMEOUT\fR;\ default="10m"] [\fB\-\-initialization\-actions\fR=\fICLOUD_STORAGE_URI\fR,[...]] [\fB\-\-labels\fR=[\fIKEY\fR=\fIVALUE\fR,...]] [\fB\-\-master\-accelerator\fR=[\fItype\fR=\fITYPE\fR,[\fIcount\fR=\fICOUNT\fR],...]] [\fB\-\-master\-boot\-disk\-provisioned\-iops\fR=\fIMASTER_BOOT_DISK_PROVISIONED_IOPS\fR] [\fB\-\-master\-boot\-disk\-provisioned\-throughput\fR=\fIMASTER_BOOT_DISK_PROVISIONED_THROUGHPUT\fR] [\fB\-\-master\-boot\-disk\-size\fR=\fIMASTER_BOOT_DISK_SIZE\fR] [\fB\-\-master\-boot\-disk\-type\fR=\fIMASTER_BOOT_DISK_TYPE\fR] [\fB\-\-master\-local\-ssd\-interface\fR=\fIMASTER_LOCAL_SSD_INTERFACE\fR] [\fB\-\-master\-machine\-type\fR=\fIMASTER_MACHINE_TYPE\fR] [\fB\-\-master\-min\-cpu\-platform\fR=\fIPLATFORM\fR] [\fB\-\-min\-secondary\-worker\-fraction\fR=\fIMIN_SECONDARY_WORKER_FRACTION\fR] [\fB\-\-node\-group\fR=\fINODE_GROUP\fR] [\fB\-\-num\-driver\-pool\-local\-ssds\fR=\fINUM_DRIVER_POOL_LOCAL_SSDS\fR] [\fB\-\-num\-master\-local\-ssds\fR=\fINUM_MASTER_LOCAL_SSDS\fR] [\fB\-\-num\-masters\fR=\fINUM_MASTERS\fR] [\fB\-\-num\-secondary\-worker\-local\-ssds\fR=\fINUM_SECONDARY_WORKER_LOCAL_SSDS\fR] [\fB\-\-num\-worker\-local\-ssds\fR=\fINUM_WORKER_LOCAL_SSDS\fR] [\fB\-\-optional\-components\fR=[\fICOMPONENT\fR,...]] [\fB\-\-private\-ipv6\-google\-access\-type\fR=\fIPRIVATE_IPV6_GOOGLE_ACCESS_TYPE\fR] [\fB\-\-properties\fR=[\fIPREFIX\fR:\fIPROPERTY\fR=\fIVALUE\fR,...]] [\fB\-\-secondary\-worker\-accelerator\fR=[\fItype\fR=\fITYPE\fR,[\fIcount\fR=\fICOUNT\fR],...]] [\fB\-\-secondary\-worker\-boot\-disk\-size\fR=\fISECONDARY_WORKER_BOOT_DISK_SIZE\fR] [\fB\-\-secondary\-worker\-boot\-disk\-type\fR=\fISECONDARY_WORKER_BOOT_DISK_TYPE\fR] [\fB\-\-secondary\-worker\-local\-ssd\-interface\fR=\fISECONDARY_WORKER_LOCAL_SSD_INTERFACE\fR] [\fB\-\-secondary\-worker\-machine\-types\fR=\fItype\fR=\fIMACHINE_TYPE\fR[,\fItype\fR=\fIMACHINE_TYPE\fR...][,\fIrank\fR=\fIRANK\fR]] [\fB\-\-secondary\-worker\-standard\-capacity\-base\fR=\fISECONDARY_WORKER_STANDARD_CAPACITY_BASE\fR] [\fB\-\-secondary\-worker\-standard\-capacity\-percent\-above\-base\fR=\fISECONDARY_WORKER_STANDARD_CAPACITY_PERCENT_ABOVE_BASE\fR] [\fB\-\-shielded\-integrity\-monitoring\fR] [\fB\-\-shielded\-secure\-boot\fR] [\fB\-\-shielded\-vtpm\fR] [\fB\-\-stop\-max\-idle\fR=\fISTOP_MAX_IDLE\fR] [\fB\-\-temp\-bucket\fR=\fITEMP_BUCKET\fR] [\fB\-\-tier\fR=\fITIER\fR] [\fB\-\-worker\-accelerator\fR=[\fItype\fR=\fITYPE\fR,[\fIcount\fR=\fICOUNT\fR],...]] [\fB\-\-worker\-boot\-disk\-provisioned\-iops\fR=\fIWORKER_BOOT_DISK_PROVISIONED_IOPS\fR] [\fB\-\-worker\-boot\-disk\-provisioned\-throughput\fR=\fIWORKER_BOOT_DISK_PROVISIONED_THROUGHPUT\fR] [\fB\-\-worker\-boot\-disk\-size\fR=\fIWORKER_BOOT_DISK_SIZE\fR] [\fB\-\-worker\-boot\-disk\-type\fR=\fIWORKER_BOOT_DISK_TYPE\fR] [\fB\-\-worker\-local\-ssd\-interface\fR=\fIWORKER_LOCAL_SSD_INTERFACE\fR] [\fB\-\-worker\-min\-cpu\-platform\fR=\fIPLATFORM\fR] [\fB\-\-zone\fR=\fIZONE\fR,\ \fB\-z\fR\ \fIZONE\fR] [\fB\-\-delete\-expiration\-time\fR=\fIDELETE_EXPIRATION_TIME\fR\ |\ \fB\-\-delete\-max\-age\fR=\fIDELETE_MAX_AGE\fR] [\fB\-\-gce\-pd\-kms\-key\fR=\fIGCE_PD_KMS_KEY\fR\ :\ \fB\-\-gce\-pd\-kms\-key\-keyring\fR=\fIGCE_PD_KMS_KEY_KEYRING\fR\ \fB\-\-gce\-pd\-kms\-key\-location\fR=\fIGCE_PD_KMS_KEY_LOCATION\fR\ \fB\-\-gce\-pd\-kms\-key\-project\fR=\fIGCE_PD_KMS_KEY_PROJECT\fR] [\fB\-\-identity\-config\-file\fR=\fIIDENTITY_CONFIG_FILE\fR\ |\ \fB\-\-secure\-multi\-tenancy\-user\-mapping\fR=\fISECURE_MULTI_TENANCY_USER_MAPPING\fR] [\fB\-\-image\fR=\fIIMAGE\fR\ |\ \fB\-\-image\-version\fR=\fIVERSION\fR] [\fB\-\-kerberos\-config\-file\fR=\fIKERBEROS_CONFIG_FILE\fR\ |\ \fB\-\-enable\-kerberos\fR\ \fB\-\-kerberos\-root\-principal\-password\-uri\fR=\fIKERBEROS_ROOT_PRINCIPAL_PASSWORD_URI\fR\ [\fB\-\-kerberos\-kms\-key\fR=\fIKERBEROS_KMS_KEY\fR\ :\ \fB\-\-kerberos\-kms\-key\-keyring\fR=\fIKERBEROS_KMS_KEY_KEYRING\fR\ \fB\-\-kerberos\-kms\-key\-location\fR=\fIKERBEROS_KMS_KEY_LOCATION\fR\ \fB\-\-kerberos\-kms\-key\-project\fR=\fIKERBEROS_KMS_KEY_PROJECT\fR]] [\fB\-\-kms\-key\fR=\fIKMS_KEY\fR\ :\ \fB\-\-kms\-keyring\fR=\fIKMS_KEYRING\fR\ \fB\-\-kms\-location\fR=\fIKMS_LOCATION\fR\ \fB\-\-kms\-project\fR=\fIKMS_PROJECT\fR] [\fB\-\-metadata\fR=\fIKEY\fR=\fIVALUE\fR,[\fIKEY\fR=\fIVALUE\fR,...]\ \fB\-\-resource\-manager\-tags\fR=\fIKEY\fR=\fIVALUE\fR,[\fIKEY\fR=\fIVALUE\fR,...]\ \fB\-\-scopes\fR=\fISCOPE\fR,[\fISCOPE\fR,...]\ \fB\-\-service\-account\fR=\fISERVICE_ACCOUNT\fR\ \fB\-\-tags\fR=\fITAG\fR,[\fITAG\fR,...]\ \fB\-\-network\fR=\fINETWORK\fR\ |\ \fB\-\-subnet\fR=\fISUBNET\fR\ \fB\-\-reservation\fR=\fIRESERVATION\fR\ \fB\-\-reservation\-affinity\fR=\fIRESERVATION_AFFINITY\fR;\ default="any"] [[\fB\-\-metric\-sources\fR=[\fIMETRIC_SOURCE\fR,...]\ :\ \fB\-\-metric\-overrides\fR=[\fIMETRIC_SOURCE\fR:\fIINSTANCE\fR:\fIGROUP\fR:\fIMETRIC\fR,...]\ |\ \fB\-\-metric\-overrides\-file\fR=\fIMETRIC_OVERRIDES_FILE\fR]] [\fB\-\-no\-address\fR\ |\ \fB\-\-public\-ip\-address\fR] [\fB\-\-single\-node\fR\ |\ \fB\-\-min\-num\-workers\fR=\fIMIN_NUM_WORKERS\fR\ \fB\-\-num\-secondary\-workers\fR=\fINUM_SECONDARY_WORKERS\fR\ \fB\-\-num\-workers\fR=\fINUM_WORKERS\fR\ \fB\-\-secondary\-worker\-type\fR=\fITYPE\fR;\ default="preemptible"] [\fB\-\-stop\-expiration\-time\fR=\fISTOP_EXPIRATION_TIME\fR\ |\ \fB\-\-stop\-max\-age\fR=\fISTOP_MAX_AGE\fR] [\fB\-\-worker\-machine\-type\fR=\fIWORKER_MACHINE_TYPE\fR\ |\ \fB\-\-worker\-machine\-types\fR=\fItype\fR=\fIMACHINE_TYPE\fR[,\fItype\fR=\fIMACHINE_TYPE\fR...][,\fIrank\fR=\fIRANK\fR]] [\fIGCLOUD_WIDE_FLAG\ ...\fR]



.SH "DESCRIPTION"

Create a cluster.



.SH "EXAMPLES"

To create a cluster, run:

.RS 2m
$ gcloud dataproc clusters create my\-cluster \-\-region=us\-central1
.RE



.SH "POSITIONAL ARGUMENTS"

.RS 2m
.TP 2m

Cluster resource \- The name of the cluster to create. The arguments in this
group can be used to specify the attributes of this resource. (NOTE) Some
attributes are not given arguments in this group but can be set in other ways.

To set the \f5project\fR attribute:
.RS 2m
.IP "\(em" 2m
provide the argument \f5cluster\fR on the command line with a fully specified
name;
.IP "\(em" 2m
provide the argument \f5\-\-project\fR on the command line;
.IP "\(em" 2m
set the property \f5core/project\fR.
.RE
.sp

This must be specified.


.RS 2m
.TP 2m
\fICLUSTER\fR

ID of the cluster or fully qualified identifier for the cluster.

To set the \f5cluster\fR attribute:
.RS 2m
.IP "\(bu" 2m
provide the argument \f5cluster\fR on the command line.
.RE
.sp

This positional argument must be specified if any of the other arguments in this
group are specified.

.TP 2m
\fB\-\-region\fR=\fIREGION\fR

Dataproc region for the cluster. Each Dataproc region constitutes an independent
resource namespace constrained to deploying instances into Compute Engine zones
inside the region. Overrides the default \f5dataproc/region\fR property value
for this command invocation.

To set the \f5region\fR attribute:
.RS 2m
.IP "\(bu" 2m
provide the argument \f5cluster\fR on the command line with a fully specified
name;
.IP "\(bu" 2m
provide the argument \f5\-\-region\fR on the command line;
.IP "\(bu" 2m
set the property \f5dataproc/region\fR.
.RE
.sp


.RE
.RE
.sp

.SH "FLAGS"

.RS 2m
.TP 2m
\fB\-\-action\-on\-failed\-primary\-workers\fR=\fIACTION_ON_FAILED_PRIMARY_WORKERS\fR

Failure action to take when primary workers fail during cluster creation.
\fIACTION_ON_FAILED_PRIMARY_WORKERS\fR must be one of:

.RS 2m
.TP 2m
\fBDELETE\fR
delete the failed primary workers
.TP 2m
\fBFAILURE_ACTION_UNSPECIFIED\fR
failure action is not specified
.TP 2m
\fBNO_ACTION\fR
take no action
.RE
.sp


.TP 2m
\fB\-\-async\fR

Return immediately, without waiting for the operation in progress to complete.

.TP 2m
\fB\-\-autoscaling\-policy\fR=\fIAUTOSCALING_POLICY\fR

ID of the autoscaling policy or fully qualified identifier for the autoscaling
policy.

To set the \f5autoscaling_policy\fR attribute:
.RS 2m
.IP "\(em" 2m
provide the argument \f5\-\-autoscaling\-policy\fR on the command line.
.RE
.sp

.TP 2m
\fB\-\-bucket\fR=\fIBUCKET\fR

The Google Cloud Storage bucket to use by default to stage job dependencies,
miscellaneous config files, and job driver console output when using this
cluster.

.TP 2m
\fB\-\-cluster\-type\fR=\fITYPE\fR

The type of cluster. \fITYPE\fR must be one of: \fBstandard\fR,
\fBsingle\-node\fR, \fBzero\-scale\fR.

.TP 2m
\fB\-\-confidential\-compute\fR

Enables Confidential VM. See
https://cloud.google.com/compute/confidential\-vm/docs for more information.
Note that Confidential VM can only be enabled when the machine types are N2D
(https://cloud.google.com/compute/docs/machine\-types#n2d_machine_types) and the
image is SEV Compatible.

.TP 2m
\fB\-\-dataproc\-metastore\fR=\fIDATAPROC_METASTORE\fR

Specify the name of a Dataproc Metastore service to be used as an external
metastore in the format:
"projects/{project\-id}/locations/{region}/services/{service\-name}".

.TP 2m
\fB\-\-delete\-max\-idle\fR=\fIDELETE_MAX_IDLE\fR

The duration after the last job completes to auto\-delete the cluster, such as
"2h" or "1d". See $ gcloud topic datetimes for information on duration formats.

.TP 2m
\fB\-\-driver\-pool\-accelerator\fR=[\fItype\fR=\fITYPE\fR,[\fIcount\fR=\fICOUNT\fR],...]

Attaches accelerators, such as GPUs, to the driver\-pool instance(s).

.RS 2m
.TP 2m
\fBtype\fR
The specific type of accelerator to attach to the instances, such as
\f5nvidia\-tesla\-t4\fR for NVIDIA T4. Use \f5gcloud compute accelerator\-types
list\fR to display available accelerator types.

.TP 2m
\fBcount\fR
The number of accelerators to attach to each instance. The default value is 1.

.RE
.sp
.TP 2m
\fB\-\-driver\-pool\-boot\-disk\-size\fR=\fIDRIVER_POOL_BOOT_DISK_SIZE\fR

The size of the boot disk. The value must be a whole number followed by a size
unit of \f5\fIKB\fR\fR for kilobyte, \f5\fIMB\fR\fR for megabyte, \f5\fIGB\fR\fR
for gigabyte, or \f5\fITB\fR\fR for terabyte. For example, \f5\fI10GB\fR\fR will
produce a 10 gigabyte disk. The minimum size a boot disk can have is 10 GB. Disk
size must be a multiple of 1 GB.

.TP 2m
\fB\-\-driver\-pool\-boot\-disk\-type\fR=\fIDRIVER_POOL_BOOT_DISK_TYPE\fR

The type of the boot disk. The value must be \f5pd\-balanced\fR, \f5pd\-ssd\fR,
or \f5pd\-standard\fR.

.TP 2m
\fB\-\-driver\-pool\-id\fR=\fIDRIVER_POOL_ID\fR

Custom identifier for the DRIVER Node Group being created. If not provided, a
random string is generated.

.TP 2m
\fB\-\-driver\-pool\-local\-ssd\-interface\fR=\fIDRIVER_POOL_LOCAL_SSD_INTERFACE\fR

Interface to use to attach local SSDs to cluster driver pool node(s).

.TP 2m
\fB\-\-driver\-pool\-machine\-type\fR=\fIDRIVER_POOL_MACHINE_TYPE\fR

The type of machine to use for the cluster driver pool nodes. Defaults to
server\-specified.

.TP 2m
\fB\-\-driver\-pool\-min\-cpu\-platform\fR=\fIPLATFORM\fR

When specified, the VM is scheduled on the host with a specified CPU
architecture or a more recent CPU platform that's available in that zone. To
list available CPU platforms in a zone, run:

.RS 2m
$ gcloud compute zones describe ZONE
.RE

CPU platform selection may not be available in a zone. Zones that support CPU
platform selection provide an \f5availableCpuPlatforms\fR field, which contains
the list of available CPU platforms in the zone (see Availability of CPU
platforms for more information).

.TP 2m
\fB\-\-driver\-pool\-size\fR=\fIDRIVER_POOL_SIZE\fR

The size of the cluster driver pool.

.TP 2m
\fB\-\-enable\-component\-gateway\fR

Enable access to the web UIs of selected components on the cluster through the
component gateway.

.TP 2m
\fB\-\-initialization\-action\-timeout\fR=\fITIMEOUT\fR; default="10m"

The maximum duration of each initialization action. See $ gcloud topic datetimes
for information on duration formats.

.TP 2m
\fB\-\-initialization\-actions\fR=\fICLOUD_STORAGE_URI\fR,[...]

A list of Google Cloud Storage URIs of executables to run on each node in the
cluster.

.TP 2m
\fB\-\-labels\fR=[\fIKEY\fR=\fIVALUE\fR,...]

List of label KEY=VALUE pairs to add.

Keys must start with a lowercase character and contain only hyphens (\f5\-\fR),
underscores (\f5_\fR), lowercase characters, and numbers. Values must contain
only hyphens (\f5\-\fR), underscores (\f5_\fR), lowercase characters, and
numbers.

.TP 2m
\fB\-\-master\-accelerator\fR=[\fItype\fR=\fITYPE\fR,[\fIcount\fR=\fICOUNT\fR],...]

Attaches accelerators, such as GPUs, to the master instance(s).

.RS 2m
.TP 2m
\fBtype\fR
The specific type of accelerator to attach to the instances, such as
\f5nvidia\-tesla\-t4\fR for NVIDIA T4. Use \f5gcloud compute accelerator\-types
list\fR to display available accelerator types.

.TP 2m
\fBcount\fR
The number of accelerators to attach to each instance. The default value is 1.

.RE
.sp
.TP 2m
\fB\-\-master\-boot\-disk\-provisioned\-iops\fR=\fIMASTER_BOOT_DISK_PROVISIONED_IOPS\fR

Indicates the IOPS (https://cloud.google.com/compute/docs/disks/hyperdisks#iops)
to provision for the disk. This sets the limit for disk I/O operations per
second. This is only supported if the bootdisk type is hyperdisk\-balanced
(https://cloud.google.com/compute/docs/disks/hyperdisks).

.TP 2m
\fB\-\-master\-boot\-disk\-provisioned\-throughput\fR=\fIMASTER_BOOT_DISK_PROVISIONED_THROUGHPUT\fR

Indicates the throughput
(https://cloud.google.com/compute/docs/disks/hyperdisks#throughput) to provision
for the disk. This sets the limit for throughput in MiB per second. This is only
supported if the bootdisk type is hyperdisk\-balanced
(https://cloud.google.com/compute/docs/disks/hyperdisks).

.TP 2m
\fB\-\-master\-boot\-disk\-size\fR=\fIMASTER_BOOT_DISK_SIZE\fR

The size of the boot disk. The value must be a whole number followed by a size
unit of \f5\fIKB\fR\fR for kilobyte, \f5\fIMB\fR\fR for megabyte, \f5\fIGB\fR\fR
for gigabyte, or \f5\fITB\fR\fR for terabyte. For example, \f5\fI10GB\fR\fR will
produce a 10 gigabyte disk. The minimum size a boot disk can have is 10 GB. Disk
size must be a multiple of 1 GB.

.TP 2m
\fB\-\-master\-boot\-disk\-type\fR=\fIMASTER_BOOT_DISK_TYPE\fR

The type of the boot disk. The value must be \f5pd\-balanced\fR, \f5pd\-ssd\fR,
or \f5pd\-standard\fR.

.TP 2m
\fB\-\-master\-local\-ssd\-interface\fR=\fIMASTER_LOCAL_SSD_INTERFACE\fR

Interface to use to attach local SSDs to master node(s) in a cluster.

.TP 2m
\fB\-\-master\-machine\-type\fR=\fIMASTER_MACHINE_TYPE\fR

The type of machine to use for the master. Defaults to server\-specified.

.TP 2m
\fB\-\-master\-min\-cpu\-platform\fR=\fIPLATFORM\fR

When specified, the VM is scheduled on the host with a specified CPU
architecture or a more recent CPU platform that's available in that zone. To
list available CPU platforms in a zone, run:

.RS 2m
$ gcloud compute zones describe ZONE
.RE

CPU platform selection may not be available in a zone. Zones that support CPU
platform selection provide an \f5availableCpuPlatforms\fR field, which contains
the list of available CPU platforms in the zone (see Availability of CPU
platforms for more information).

.TP 2m
\fB\-\-min\-secondary\-worker\-fraction\fR=\fIMIN_SECONDARY_WORKER_FRACTION\fR

Minimum fraction of secondary worker nodes required to create the cluster. If it
is not met, cluster creation will fail. Must be a decimal value between 0 and 1.
The number of required secondary workers is calculated by
ceil(min\-secondary\-worker\-fraction * num_secondary_workers). Defaults to
0.0001.

.TP 2m
\fB\-\-node\-group\fR=\fINODE_GROUP\fR

The name of the sole\-tenant node group to create the cluster on. Can be a short
name ("node\-group\-name") or in the format
"projects/{project\-id}/zones/{zone}/nodeGroups/{node\-group\-name}".

.TP 2m
\fB\-\-num\-driver\-pool\-local\-ssds\fR=\fINUM_DRIVER_POOL_LOCAL_SSDS\fR

The number of local SSDs to attach to each cluster driver pool node.

.TP 2m
\fB\-\-num\-master\-local\-ssds\fR=\fINUM_MASTER_LOCAL_SSDS\fR

The number of local SSDs to attach to the master in a cluster.

.TP 2m
\fB\-\-num\-masters\fR=\fINUM_MASTERS\fR

The number of master nodes in the cluster.


.TS
tab(	);
lB lB
l l.
Number of Masters	Cluster Mode
1	Standard
3	High Availability
.TE

.TP 2m
\fB\-\-num\-secondary\-worker\-local\-ssds\fR=\fINUM_SECONDARY_WORKER_LOCAL_SSDS\fR

The number of local SSDs to attach to each preemptible worker in a cluster.

.TP 2m
\fB\-\-num\-worker\-local\-ssds\fR=\fINUM_WORKER_LOCAL_SSDS\fR

The number of local SSDs to attach to each worker in a cluster.

.TP 2m
\fB\-\-optional\-components\fR=[\fICOMPONENT\fR,...]

List of optional components to be installed on cluster machines.

The following page documents the optional components that can be installed:
https://cloud.google.com/dataproc/docs/concepts/configuring\-clusters/optional\-components.

.TP 2m
\fB\-\-private\-ipv6\-google\-access\-type\fR=\fIPRIVATE_IPV6_GOOGLE_ACCESS_TYPE\fR

The private IPv6 Google access type for the cluster.
\fIPRIVATE_IPV6_GOOGLE_ACCESS_TYPE\fR must be one of: \fBinherit\-subnetwork\fR,
\fBoutbound\fR, \fBbidirectional\fR.

.TP 2m
\fB\-\-properties\fR=[\fIPREFIX\fR:\fIPROPERTY\fR=\fIVALUE\fR,...]

Specifies configuration properties for installed packages, such as Hadoop and
Spark.

Properties are mapped to configuration files by specifying a prefix, such as
"core:io.serializations". The following are supported prefixes and their
mappings:


.TS
tab(	);
lB lB lB
l l l.
Prefix	File	Purpose of file
capacity-scheduler	capacity-scheduler.xml	Hadoop YARN Capacity Scheduler configuration
core	core-site.xml	Hadoop general configuration
distcp	distcp-default.xml	Hadoop Distributed Copy configuration
hadoop-env	hadoop-env.sh	Hadoop specific environment variables
hdfs	hdfs-site.xml	Hadoop HDFS configuration
hive	hive-site.xml	Hive configuration
mapred	mapred-site.xml	Hadoop MapReduce configuration
mapred-env	mapred-env.sh	Hadoop MapReduce specific environment variables
pig	pig.properties	Pig configuration
spark	spark-defaults.conf	Spark configuration
spark-env	spark-env.sh	Spark specific environment variables
yarn	yarn-site.xml	Hadoop YARN configuration
yarn-env	yarn-env.sh	Hadoop YARN specific environment variables
.TE

See
https://cloud.google.com/dataproc/docs/concepts/configuring\-clusters/cluster\-properties
for more information.

.TP 2m
\fB\-\-secondary\-worker\-accelerator\fR=[\fItype\fR=\fITYPE\fR,[\fIcount\fR=\fICOUNT\fR],...]

Attaches accelerators, such as GPUs, to the secondary\-worker instance(s).

.RS 2m
.TP 2m
\fBtype\fR
The specific type of accelerator to attach to the instances, such as
\f5nvidia\-tesla\-t4\fR for NVIDIA T4. Use \f5gcloud compute accelerator\-types
list\fR to display available accelerator types.

.TP 2m
\fBcount\fR
The number of accelerators to attach to each instance. The default value is 1.

.RE
.sp
.TP 2m
\fB\-\-secondary\-worker\-boot\-disk\-size\fR=\fISECONDARY_WORKER_BOOT_DISK_SIZE\fR

The size of the boot disk. The value must be a whole number followed by a size
unit of \f5\fIKB\fR\fR for kilobyte, \f5\fIMB\fR\fR for megabyte, \f5\fIGB\fR\fR
for gigabyte, or \f5\fITB\fR\fR for terabyte. For example, \f5\fI10GB\fR\fR will
produce a 10 gigabyte disk. The minimum size a boot disk can have is 10 GB. Disk
size must be a multiple of 1 GB.

.TP 2m
\fB\-\-secondary\-worker\-boot\-disk\-type\fR=\fISECONDARY_WORKER_BOOT_DISK_TYPE\fR

The type of the boot disk. The value must be \f5pd\-balanced\fR, \f5pd\-ssd\fR,
or \f5pd\-standard\fR.

.TP 2m
\fB\-\-secondary\-worker\-local\-ssd\-interface\fR=\fISECONDARY_WORKER_LOCAL_SSD_INTERFACE\fR

Interface to use to attach local SSDs to each secondary worker in a cluster.

.TP 2m
\fB\-\-secondary\-worker\-machine\-types\fR=\fItype\fR=\fIMACHINE_TYPE\fR[,\fItype\fR=\fIMACHINE_TYPE\fR...][,\fIrank\fR=\fIRANK\fR]

Types of machines with optional rank for secondary workers to use. Defaults to
server\-specified.eg.
\-\-secondary\-worker\-machine\-types="type=e2\-standard\-8,type=t2d\-standard\-8,rank=0"

.TP 2m
\fB\-\-secondary\-worker\-standard\-capacity\-base\fR=\fISECONDARY_WORKER_STANDARD_CAPACITY_BASE\fR

This flag sets the base number of Standard VMs to use for secondary workers
(https://cloud.google.com/dataproc/docs/concepts/compute/secondary\-vms#preemptible_and_non\-preemptible_secondary_workers).
Dataproc will create only standard VMs until it reaches this number, then it
will mix Spot and Standard VMs according to
\f5\fISECONDARY_WORKER_STANDARD_CAPACITY_PERCENT_ABOVE_BASE\fR\fR.

.TP 2m
\fB\-\-secondary\-worker\-standard\-capacity\-percent\-above\-base\fR=\fISECONDARY_WORKER_STANDARD_CAPACITY_PERCENT_ABOVE_BASE\fR

When combining Standard and Spot VMs for secondary\-workers
(https://cloud.google.com/dataproc/docs/concepts/compute/secondary\-vms#preemptible_and_non\-preemptible_secondary_workers)
once the number of Standard VMs specified by
\f5\fISECONDARY_WORKER_STANDARD_CAPACITY_BASE\fR\fR has been used, this flag
specifies the percentage of the total number of additional Standard VMs
secondary workers will use. Spot VMs will be used for the remaining percentage.

.TP 2m
\fB\-\-shielded\-integrity\-monitoring\fR

Enables monitoring and attestation of the boot integrity of the cluster's VMs.
vTPM (virtual Trusted Platform Module) must also be enabled. A TPM is a hardware
module that can be used for different security operations, such as remote
attestation, encryption, and sealing of keys.

.TP 2m
\fB\-\-shielded\-secure\-boot\fR

The cluster's VMs will boot with secure boot enabled.

.TP 2m
\fB\-\-shielded\-vtpm\fR

The cluster's VMs will boot with the TPM (Trusted Platform Module) enabled. A
TPM is a hardware module that can be used for different security operations,
such as remote attestation, encryption, and sealing of keys.

.TP 2m
\fB\-\-stop\-max\-idle\fR=\fISTOP_MAX_IDLE\fR

The duration after the last job completes to auto\-stop the cluster, such as
"2h" or "1d". See $ gcloud topic datetimes for information on duration formats.

.TP 2m
\fB\-\-temp\-bucket\fR=\fITEMP_BUCKET\fR

The Google Cloud Storage bucket to use by default to store ephemeral cluster and
jobs data, such as Spark and MapReduce history files.

.TP 2m
\fB\-\-tier\fR=\fITIER\fR

Cluster tier. \fITIER\fR must be one of: \fBpremium\fR, \fBstandard\fR.

.TP 2m
\fB\-\-worker\-accelerator\fR=[\fItype\fR=\fITYPE\fR,[\fIcount\fR=\fICOUNT\fR],...]

Attaches accelerators, such as GPUs, to the worker instance(s).

.RS 2m
.TP 2m
\fBtype\fR
The specific type of accelerator to attach to the instances, such as
\f5nvidia\-tesla\-t4\fR for NVIDIA T4. Use \f5gcloud compute accelerator\-types
list\fR to display available accelerator types.

.TP 2m
\fBcount\fR
The number of accelerators to attach to each instance. The default value is 1.

.RE
.sp
.TP 2m
\fB\-\-worker\-boot\-disk\-provisioned\-iops\fR=\fIWORKER_BOOT_DISK_PROVISIONED_IOPS\fR

Indicates the IOPS (https://cloud.google.com/compute/docs/disks/hyperdisks#iops)
to provision for the disk. This sets the limit for disk I/O operations per
second. This is only supported if the bootdisk type is hyperdisk\-balanced
(https://cloud.google.com/compute/docs/disks/hyperdisks).

.TP 2m
\fB\-\-worker\-boot\-disk\-provisioned\-throughput\fR=\fIWORKER_BOOT_DISK_PROVISIONED_THROUGHPUT\fR

Indicates the throughput
(https://cloud.google.com/compute/docs/disks/hyperdisks#throughput) to provision
for the disk. This sets the limit for throughput in MiB per second. This is only
supported if the bootdisk type is hyperdisk\-balanced
(https://cloud.google.com/compute/docs/disks/hyperdisks).

.TP 2m
\fB\-\-worker\-boot\-disk\-size\fR=\fIWORKER_BOOT_DISK_SIZE\fR

The size of the boot disk. The value must be a whole number followed by a size
unit of \f5\fIKB\fR\fR for kilobyte, \f5\fIMB\fR\fR for megabyte, \f5\fIGB\fR\fR
for gigabyte, or \f5\fITB\fR\fR for terabyte. For example, \f5\fI10GB\fR\fR will
produce a 10 gigabyte disk. The minimum size a boot disk can have is 10 GB. Disk
size must be a multiple of 1 GB.

.TP 2m
\fB\-\-worker\-boot\-disk\-type\fR=\fIWORKER_BOOT_DISK_TYPE\fR

The type of the boot disk. The value must be \f5pd\-balanced\fR, \f5pd\-ssd\fR,
or \f5pd\-standard\fR.

.TP 2m
\fB\-\-worker\-local\-ssd\-interface\fR=\fIWORKER_LOCAL_SSD_INTERFACE\fR

Interface to use to attach local SSDs to each worker in a cluster.

.TP 2m
\fB\-\-worker\-min\-cpu\-platform\fR=\fIPLATFORM\fR

When specified, the VM is scheduled on the host with a specified CPU
architecture or a more recent CPU platform that's available in that zone. To
list available CPU platforms in a zone, run:

.RS 2m
$ gcloud compute zones describe ZONE
.RE

CPU platform selection may not be available in a zone. Zones that support CPU
platform selection provide an \f5availableCpuPlatforms\fR field, which contains
the list of available CPU platforms in the zone (see Availability of CPU
platforms for more information).

.TP 2m
\fB\-\-zone\fR=\fIZONE\fR, \fB\-z\fR \fIZONE\fR

The compute zone (e.g. us\-central1\-a) for the cluster. If empty and \-\-region
is set to a value other than \f5global\fR, the server will pick a zone in the
region. Overrides the default \fBcompute/zone\fR property value for this command
invocation.

.TP 2m

At most one of these can be specified:


.RS 2m
.TP 2m
\fB\-\-delete\-expiration\-time\fR=\fIDELETE_EXPIRATION_TIME\fR

The time when the cluster will be auto\-deleted, such as
"2017\-08\-29T18:52:51.142Z." See $ gcloud topic datetimes for information on
time formats.

.TP 2m
\fB\-\-delete\-max\-age\fR=\fIDELETE_MAX_AGE\fR

The lifespan of the cluster, with auto\-deletion upon completion, such as "2h"
or "1d". See $ gcloud topic datetimes for information on duration formats.

.RE
.sp
.TP 2m

Key resource \- The Cloud KMS (Key Management Service) cryptokey that will be
used to protect the cluster. The 'Compute Engine Service Agent' service account
must hold permission 'Cloud KMS CryptoKey Encrypter/Decrypter'. The arguments in
this group can be used to specify the attributes of this resource.


.RS 2m
.TP 2m
\fB\-\-gce\-pd\-kms\-key\fR=\fIGCE_PD_KMS_KEY\fR

ID of the key or fully qualified identifier for the key.

To set the \f5kms\-key\fR attribute:
.RS 2m
.IP "\(bu" 2m
provide the argument \f5\-\-gce\-pd\-kms\-key\fR on the command line.
.RE
.sp

This flag argument must be specified if any of the other arguments in this group
are specified.

.TP 2m
\fB\-\-gce\-pd\-kms\-key\-keyring\fR=\fIGCE_PD_KMS_KEY_KEYRING\fR

The KMS keyring of the key.

To set the \f5kms\-keyring\fR attribute:
.RS 2m
.IP "\(bu" 2m
provide the argument \f5\-\-gce\-pd\-kms\-key\fR on the command line with a
fully specified name;
.IP "\(bu" 2m
provide the argument \f5\-\-gce\-pd\-kms\-key\-keyring\fR on the command line.
.RE
.sp

.TP 2m
\fB\-\-gce\-pd\-kms\-key\-location\fR=\fIGCE_PD_KMS_KEY_LOCATION\fR

The Google Cloud location for the key.

To set the \f5kms\-location\fR attribute:
.RS 2m
.IP "\(bu" 2m
provide the argument \f5\-\-gce\-pd\-kms\-key\fR on the command line with a
fully specified name;
.IP "\(bu" 2m
provide the argument \f5\-\-gce\-pd\-kms\-key\-location\fR on the command line.
.RE
.sp

.TP 2m
\fB\-\-gce\-pd\-kms\-key\-project\fR=\fIGCE_PD_KMS_KEY_PROJECT\fR

The Google Cloud project for the key.

To set the \f5kms\-project\fR attribute:
.RS 2m
.IP "\(bu" 2m
provide the argument \f5\-\-gce\-pd\-kms\-key\fR on the command line with a
fully specified name;
.IP "\(bu" 2m
provide the argument \f5\-\-gce\-pd\-kms\-key\-project\fR on the command line;
.IP "\(bu" 2m
set the property \f5core/project\fR.
.RE
.sp

.RE
.sp
.TP 2m

Specifying these flags will enable Secure Multi\-Tenancy for the cluster.

At most one of these can be specified:


.RS 2m
.TP 2m
\fB\-\-identity\-config\-file\fR=\fIIDENTITY_CONFIG_FILE\fR

Path to a YAML (or JSON) file containing the configuration for Secure
Multi\-Tenancy on the cluster. The path can be a Cloud Storage URL (Example:
\'gs://path/to/file') or a local file system path. If you pass "\-" as the value
of the flag the file content will be read from stdin.

The YAML file is formatted as follows:

.RS 2m
  # Required. The mapping from user accounts to service accounts.
  user_service_account_mapping:
    bob@company.com: service\-account\-bob@project.iam.gserviceaccount.com
    alice@company.com: service\-account\-alice@project.iam.gserviceaccount.com
.RE

.TP 2m
\fB\-\-secure\-multi\-tenancy\-user\-mapping\fR=\fISECURE_MULTI_TENANCY_USER_MAPPING\fR

A string of user\-to\-service\-account mappings. Mappings are separated by
commas, and each mapping takes the form of "user\-account:service\-account".
Example:
"bob@company.com:service\-account\-bob@project.iam.gserviceaccount.com,alice@company.com:service\-account\-alice@project.iam.gserviceaccount.com".

.RE
.sp
.TP 2m

At most one of these can be specified:


.RS 2m
.TP 2m
\fB\-\-image\fR=\fIIMAGE\fR

The custom image used to create the cluster. It can be the image name, the image
URI, or the image family URI, which selects the latest image from the family.

.TP 2m
\fB\-\-image\-version\fR=\fIVERSION\fR

The image version to use for the cluster. Defaults to the latest version.

.RE
.sp
.TP 2m

Specifying these flags will enable Kerberos for the cluster.

At most one of these can be specified:


.RS 2m
.TP 2m
\fB\-\-kerberos\-config\-file\fR=\fIKERBEROS_CONFIG_FILE\fR

Path to a YAML (or JSON) file containing the configuration for Kerberos on the
cluster. If you pass \f5\-\fR as the value of the flag the file content will be
read from stdin.

The YAML file is formatted as follows:

.RS 2m
  # Optional. Flag to indicate whether to Kerberize the cluster.
  # The default value is true.
  enable_kerberos: true
.RE

.RS 2m
  # Optional. The Google Cloud Storage URI of a KMS encrypted file
  # containing the root principal password.
  root_principal_password_uri: gs://bucket/password.encrypted
.RE

.RS 2m
  # Optional. The URI of the Cloud KMS key used to encrypt
  # sensitive files.
  kms_key_uri:
    projects/myproject/locations/global/keyRings/mykeyring/cryptoKeys/my\-key
.RE

.RS 2m
  # Configuration of SSL encryption. If specified, all sub\-fields
  # are required. Otherwise, Dataproc will provide a self\-signed
  # certificate and generate the passwords.
  ssl:
    # Optional. The Google Cloud Storage URI of the keystore file.
    keystore_uri: gs://bucket/keystore.jks
.RE

.RS 2m
    # Optional. The Google Cloud Storage URI of a KMS encrypted
    # file containing the password to the keystore.
    keystore_password_uri: gs://bucket/keystore_password.encrypted
.RE

.RS 2m
    # Optional. The Google Cloud Storage URI of a KMS encrypted
    # file containing the password to the user provided key.
    key_password_uri: gs://bucket/key_password.encrypted
.RE

.RS 2m
    # Optional. The Google Cloud Storage URI of the truststore
    # file.
    truststore_uri: gs://bucket/truststore.jks
.RE

.RS 2m
    # Optional. The Google Cloud Storage URI of a KMS encrypted
    # file containing the password to the user provided
    # truststore.
    truststore_password_uri:
      gs://bucket/truststore_password.encrypted
.RE

.RS 2m
  # Configuration of cross realm trust.
  cross_realm_trust:
    # Optional. The remote realm the Dataproc on\-cluster KDC will
    # trust, should the user enable cross realm trust.
    realm: REMOTE.REALM
.RE

.RS 2m
    # Optional. The KDC (IP or hostname) for the remote trusted
    # realm in a cross realm trust relationship.
    kdc: kdc.remote.realm
.RE

.RS 2m
    # Optional. The admin server (IP or hostname) for the remote
    # trusted realm in a cross realm trust relationship.
    admin_server: admin\-server.remote.realm
.RE

.RS 2m
    # Optional. The Google Cloud Storage URI of a KMS encrypted
    # file containing the shared password between the on\-cluster
    # Kerberos realm and the remote trusted realm, in a cross
    # realm trust relationship.
    shared_password_uri:
      gs://bucket/cross\-realm.password.encrypted
.RE

.RS 2m
  # Optional. The Google Cloud Storage URI of a KMS encrypted file
  # containing the master key of the KDC database.
  kdc_db_key_uri: gs://bucket/kdc_db_key.encrypted
.RE

.RS 2m
  # Optional. The lifetime of the ticket granting ticket, in
  # hours. If not specified, or user specifies 0, then default
  # value 10 will be used.
  tgt_lifetime_hours: 1
.RE

.RS 2m
  # Optional. The name of the Kerberos realm. If not specified,
  # the uppercased domain name of the cluster will be used.
  realm: REALM.NAME
.RE

.TP 2m
\fB\-\-enable\-kerberos\fR

Enable Kerberos on the cluster.

.TP 2m
\fB\-\-kerberos\-root\-principal\-password\-uri\fR=\fIKERBEROS_ROOT_PRINCIPAL_PASSWORD_URI\fR

Google Cloud Storage URI of a KMS encrypted file containing the root principal
password. Must be a Cloud Storage URL beginning with 'gs://'.

.TP 2m

Key resource \- The Cloud KMS (Key Management Service) cryptokey that will be
used to protect the password. The 'Compute Engine Service Agent' service account
must hold permission 'Cloud KMS CryptoKey Encrypter/Decrypter'. The arguments in
this group can be used to specify the attributes of this resource.


.RS 2m
.TP 2m
\fB\-\-kerberos\-kms\-key\fR=\fIKERBEROS_KMS_KEY\fR

ID of the key or fully qualified identifier for the key.

To set the \f5kms\-key\fR attribute:
.RS 2m
.IP "\(em" 2m
provide the argument \f5\-\-kerberos\-kms\-key\fR on the command line.
.RE
.sp

This flag argument must be specified if any of the other arguments in this group
are specified.

.TP 2m
\fB\-\-kerberos\-kms\-key\-keyring\fR=\fIKERBEROS_KMS_KEY_KEYRING\fR

The KMS keyring of the key.

To set the \f5kms\-keyring\fR attribute:
.RS 2m
.IP "\(em" 2m
provide the argument \f5\-\-kerberos\-kms\-key\fR on the command line with a
fully specified name;
.IP "\(em" 2m
provide the argument \f5\-\-kerberos\-kms\-key\-keyring\fR on the command line.
.RE
.sp

.TP 2m
\fB\-\-kerberos\-kms\-key\-location\fR=\fIKERBEROS_KMS_KEY_LOCATION\fR

The Google Cloud location for the key.

To set the \f5kms\-location\fR attribute:
.RS 2m
.IP "\(em" 2m
provide the argument \f5\-\-kerberos\-kms\-key\fR on the command line with a
fully specified name;
.IP "\(em" 2m
provide the argument \f5\-\-kerberos\-kms\-key\-location\fR on the command line.
.RE
.sp

.TP 2m
\fB\-\-kerberos\-kms\-key\-project\fR=\fIKERBEROS_KMS_KEY_PROJECT\fR

The Google Cloud project for the key.

To set the \f5kms\-project\fR attribute:
.RS 2m
.IP "\(em" 2m
provide the argument \f5\-\-kerberos\-kms\-key\fR on the command line with a
fully specified name;
.IP "\(em" 2m
provide the argument \f5\-\-kerberos\-kms\-key\-project\fR on the command line;
.IP "\(em" 2m
set the property \f5core/project\fR.
.RE
.sp

.RE
.RE
.sp
.TP 2m

Key resource \- The Cloud KMS (Key Management Service) cryptokey that will be
used to protect the cluster. The 'Compute Engine Service Agent' service account
must hold permission 'Cloud KMS CryptoKey Encrypter/Decrypter'. The arguments in
this group can be used to specify the attributes of this resource.


.RS 2m
.TP 2m
\fB\-\-kms\-key\fR=\fIKMS_KEY\fR

ID of the key or fully qualified identifier for the key.

To set the \f5kms\-key\fR attribute:
.RS 2m
.IP "\(bu" 2m
provide the argument \f5\-\-kms\-key\fR on the command line.
.RE
.sp

This flag argument must be specified if any of the other arguments in this group
are specified.

.TP 2m
\fB\-\-kms\-keyring\fR=\fIKMS_KEYRING\fR

The KMS keyring of the key.

To set the \f5kms\-keyring\fR attribute:
.RS 2m
.IP "\(bu" 2m
provide the argument \f5\-\-kms\-key\fR on the command line with a fully
specified name;
.IP "\(bu" 2m
provide the argument \f5\-\-kms\-keyring\fR on the command line.
.RE
.sp

.TP 2m
\fB\-\-kms\-location\fR=\fIKMS_LOCATION\fR

The Google Cloud location for the key.

To set the \f5kms\-location\fR attribute:
.RS 2m
.IP "\(bu" 2m
provide the argument \f5\-\-kms\-key\fR on the command line with a fully
specified name;
.IP "\(bu" 2m
provide the argument \f5\-\-kms\-location\fR on the command line.
.RE
.sp

.TP 2m
\fB\-\-kms\-project\fR=\fIKMS_PROJECT\fR

The Google Cloud project for the key.

To set the \f5kms\-project\fR attribute:
.RS 2m
.IP "\(bu" 2m
provide the argument \f5\-\-kms\-key\fR on the command line with a fully
specified name;
.IP "\(bu" 2m
provide the argument \f5\-\-kms\-project\fR on the command line;
.IP "\(bu" 2m
set the property \f5core/project\fR.
.RE
.sp

.RE
.sp
.TP 2m

Compute Engine options for Dataproc clusters.



.RS 2m
.TP 2m
\fB\-\-metadata\fR=\fIKEY\fR=\fIVALUE\fR,[\fIKEY\fR=\fIVALUE\fR,...]

Metadata to be made available to the guest operating system running on the
instances

.TP 2m
\fB\-\-resource\-manager\-tags\fR=\fIKEY\fR=\fIVALUE\fR,[\fIKEY\fR=\fIVALUE\fR,...]

Specifies a list of resource manager tags to apply to each cluster node (master
and worker nodes).

.TP 2m
\fB\-\-scopes\fR=\fISCOPE\fR,[\fISCOPE\fR,...]

Specifies scopes for the node instances. Multiple SCOPEs can be specified,
separated by commas. Examples:

.RS 2m
$ gcloud dataproc clusters create example\-cluster \e
    \-\-scopes https://www.googleapis.com/auth/bigtable.admin
.RE

.RS 2m
$ gcloud dataproc clusters create example\-cluster \e
    \-\-scopes sqlservice,bigquery
.RE

The following \fBminimum scopes\fR are necessary for the cluster to function
properly and are always added, even if not explicitly specified:

.RS 2m
https://www.googleapis.com/auth/devstorage.read_write
https://www.googleapis.com/auth/logging.write
.RE

If the \f5\-\-scopes\fR flag is not specified, the following \fBdefault
scopes\fR are also included:

.RS 2m
https://www.googleapis.com/auth/bigquery
https://www.googleapis.com/auth/bigtable.admin.table
https://www.googleapis.com/auth/bigtable.data
https://www.googleapis.com/auth/devstorage.full_control
.RE

If you want to enable all scopes use the 'cloud\-platform' scope.

SCOPE can be either the full URI of the scope or an alias. \fBDefault\fR scopes
are assigned to all instances. Available aliases are:


.TS
tab(	);
lB lB
l l.
Alias	URI
bigquery	https://www.googleapis.com/auth/bigquery
cloud-platform	https://www.googleapis.com/auth/cloud-platform
cloud-source-repos	https://www.googleapis.com/auth/source.full_control
cloud-source-repos-ro	https://www.googleapis.com/auth/source.read_only
compute-ro	https://www.googleapis.com/auth/compute.readonly
compute-rw	https://www.googleapis.com/auth/compute
datastore	https://www.googleapis.com/auth/datastore
default	https://www.googleapis.com/auth/devstorage.read_only
	https://www.googleapis.com/auth/logging.write
	https://www.googleapis.com/auth/monitoring.write
	https://www.googleapis.com/auth/pubsub
	https://www.googleapis.com/auth/service.management.readonly
	https://www.googleapis.com/auth/servicecontrol
	https://www.googleapis.com/auth/trace.append
gke-default	https://www.googleapis.com/auth/devstorage.read_only
	https://www.googleapis.com/auth/logging.write
	https://www.googleapis.com/auth/monitoring
	https://www.googleapis.com/auth/service.management.readonly
	https://www.googleapis.com/auth/servicecontrol
	https://www.googleapis.com/auth/trace.append
logging-write	https://www.googleapis.com/auth/logging.write
monitoring	https://www.googleapis.com/auth/monitoring
monitoring-read	https://www.googleapis.com/auth/monitoring.read
monitoring-write	https://www.googleapis.com/auth/monitoring.write
pubsub	https://www.googleapis.com/auth/pubsub
service-control	https://www.googleapis.com/auth/servicecontrol
service-management	https://www.googleapis.com/auth/service.management.readonly
sql (deprecated)	https://www.googleapis.com/auth/sqlservice
sql-admin	https://www.googleapis.com/auth/sqlservice.admin
storage-full	https://www.googleapis.com/auth/devstorage.full_control
storage-ro	https://www.googleapis.com/auth/devstorage.read_only
storage-rw	https://www.googleapis.com/auth/devstorage.read_write
taskqueue	https://www.googleapis.com/auth/taskqueue
trace	https://www.googleapis.com/auth/trace.append
userinfo-email	https://www.googleapis.com/auth/userinfo.email
.TE

DEPRECATION WARNING: https://www.googleapis.com/auth/sqlservice account scope
and \f5sql\fR alias do not provide SQL instance management capabilities and have
been deprecated. Please, use https://www.googleapis.com/auth/sqlservice.admin or
\f5sql\-admin\fR to manage your Google SQL Service instances.

.TP 2m
\fB\-\-service\-account\fR=\fISERVICE_ACCOUNT\fR

The Google Cloud IAM service account to be authenticated as.

.TP 2m
\fB\-\-tags\fR=\fITAG\fR,[\fITAG\fR,...]

Specifies a list of tags to apply to the instance. These tags allow network
firewall rules and routes to be applied to specified VM instances. See \fBgcloud
compute firewall\-rules create\fR(1) for more details.

To read more about configuring network tags, read this guide:
https://cloud.google.com/vpc/docs/add\-remove\-network\-tags

To list instances with their respective status and tags, run:

.RS 2m
$ gcloud compute instances list \e
    \-\-format='table(name,status,tags.list())'
.RE

To list instances tagged with a specific tag, \f5tag1\fR, run:

.RS 2m
$ gcloud compute instances list \-\-filter='tags:tag1'
.RE

.TP 2m

At most one of these can be specified:


.RS 2m
.TP 2m
\fB\-\-network\fR=\fINETWORK\fR

The Compute Engine network that the VM instances of the cluster will be part of.
This is mutually exclusive with \-\-subnet. If neither is specified, this
defaults to the "default" network.

.TP 2m
\fB\-\-subnet\fR=\fISUBNET\fR

Specifies the subnet that the cluster will be part of. This is mutally exclusive
with \-\-network.

.RE
.sp
.TP 2m

Specifies the reservation for the instance.


.RS 2m
.TP 2m
\fB\-\-reservation\fR=\fIRESERVATION\fR

The name of the reservation, required when
\f5\-\-reservation\-affinity=specific\fR.

.TP 2m
\fB\-\-reservation\-affinity\fR=\fIRESERVATION_AFFINITY\fR; default="any"

The type of reservation for the instance. \fIRESERVATION_AFFINITY\fR must be one
of: \fBany\fR, \fBnone\fR, \fBspecific\fR.

.RE
.RE
.sp
.TP 2m
\fB\-\-metric\-sources\fR=[\fIMETRIC_SOURCE\fR,...]

Specifies a list of cluster Metric Sources
(https://cloud.google.com/dataproc/docs/guides/monitoring#available_oss_metrics)
to collect custom metrics. \fIMETRIC_SOURCE\fR must be one of: \fBFLINK\fR,
\fBHDFS\fR, \fBHIVEMETASTORE\fR, \fBHIVESERVER2\fR,
\fBMONITORING_AGENT_DEFAULTS\fR, \fBSPARK\fR, \fBSPARK_HISTORY_SERVER\fR,
\fBYARN\fR.

.TP 2m

At most one of these can be specified:


.RS 2m
.TP 2m
\fB\-\-metric\-overrides\fR=[\fIMETRIC_SOURCE\fR:\fIINSTANCE\fR:\fIGROUP\fR:\fIMETRIC\fR,...]

List of metrics that override the default metrics enabled for the metric
sources. Any of the available OSS metrics
(https://cloud.google.com/dataproc/docs/guides/monitoring#available_oss_metrics)
and all Spark metrics, can be listed for collection as a metric override.
Override metric values are case sensitive, and must be provided, if appropriate,
in CamelCase format, for example:

\fBsparkHistoryServer:JVM:Memory:NonHeapMemoryUsage.committed\fR
\fBhiveserver2:JVM:Memory:NonHeapMemoryUsage.used\fR

Only the specified overridden metrics will be collected from a given metric
source. For example, if one or more \fBspark:executive\fR metrics are listed as
metric overrides, other \fBSPARK\fR metrics will not be collected. The
collection of default OSS metrics from other metric sources is unaffected. For
example, if both \fBSPARK\fR and \fBYARN\fR metric sources are enabled, and
overrides are provided for Spark metrics only, all default YARN metrics will be
collected.

The source of the specified metric override must be enabled. For example, if one
or more \fBspark:driver\fR metrics are provided as metric overrides, the spark
metric source must be enabled (\fB\-\-metric\-sources=spark\fR).

.TP 2m
\fB\-\-metric\-overrides\-file\fR=\fIMETRIC_OVERRIDES_FILE\fR

Path to a file containing list of Metrics that override the default metrics
enabled for the metric sources. The path can be a Cloud Storage URL (example:
\f5gs://path/to/file\fR) or a local file system path.

.RE
.sp
.TP 2m

At most one of these can be specified:


.RS 2m
.TP 2m
\fB\-\-no\-address\fR

If provided, the instances in the cluster will not be assigned external IP
addresses.

If omitted, then the Dataproc service will apply a default policy to determine
if each instance in the cluster gets an external IP address or not.

Note: Dataproc VMs need access to the Dataproc API. This can be achieved without
external IP addresses using Private Google Access
(https://cloud.google.com/compute/docs/private\-google\-access).

.TP 2m
\fB\-\-public\-ip\-address\fR

If provided, cluster instances are assigned external IP addresses.

If omitted, the Dataproc service applies a default policy to determine whether
or not each instance in the cluster gets an external IP address.

Note: Dataproc VMs need access to the Dataproc API. This can be achieved without
external IP addresses using Private Google Access
(https://cloud.google.com/compute/docs/private\-google\-access).

.RE
.sp
.TP 2m

At most one of these can be specified:


.RS 2m
.TP 2m
\fB\-\-single\-node\fR

Create a single node cluster.

A single node cluster has all master and worker components. It cannot have any
separate worker nodes. If this flag is not specified, a cluster with separate
workers is created.

.TP 2m

Multi\-node cluster flags


.RS 2m
.TP 2m
\fB\-\-min\-num\-workers\fR=\fIMIN_NUM_WORKERS\fR

Minimum number of primary worker nodes to provision for cluster creation to
succeed.

.TP 2m
\fB\-\-num\-secondary\-workers\fR=\fINUM_SECONDARY_WORKERS\fR

The number of secondary worker nodes in the cluster.

.TP 2m
\fB\-\-num\-workers\fR=\fINUM_WORKERS\fR

The number of worker nodes in the cluster. Defaults to server\-specified.

.TP 2m
\fB\-\-secondary\-worker\-type\fR=\fITYPE\fR; default="preemptible"

The type of the secondary worker group. \fITYPE\fR must be one of:
\fBpreemptible\fR, \fBnon\-preemptible\fR, \fBspot\fR.

.RE
.RE
.sp
.TP 2m

At most one of these can be specified:


.RS 2m
.TP 2m
\fB\-\-stop\-expiration\-time\fR=\fISTOP_EXPIRATION_TIME\fR

The time when the cluster will be auto\-stopped, such as
"2017\-08\-29T18:52:51.142Z." See $ gcloud topic datetimes for information on
time formats.

.TP 2m
\fB\-\-stop\-max\-age\fR=\fISTOP_MAX_AGE\fR

The lifespan of the cluster, with auto\-stop upon completion, such as "2h" or
"1d". See $ gcloud topic datetimes for information on duration formats.

.RE
.sp
.TP 2m

At most one of these can be specified:


.RS 2m
.TP 2m
\fB\-\-worker\-machine\-type\fR=\fIWORKER_MACHINE_TYPE\fR

The type of machine to use for primary workers. Defaults to server\-specified.

.TP 2m
\fB\-\-worker\-machine\-types\fR=\fItype\fR=\fIMACHINE_TYPE\fR[,\fItype\fR=\fIMACHINE_TYPE\fR...][,\fIrank\fR=\fIRANK\fR]

Machine types
(https://cloud.google.com/dataproc/docs/concepts/compute/supported\-machine\-types)
for primary worker nodes to use with optional rank. A lower rank number is given
higher preference. Based on availablilty, Dataproc tries to create primary
worker VMs using the worker machine type with the lowest rank, and then tries to
use machine types with higher ranks as necessary. Machine types with the same
rank are given the same preference. Example use:
\-\-worker\-machine\-types="type=e2\-standard\-8,type=n2\-standard\-8,rank=0".
For more information, see Dataproc Flexible VMs
(https://cloud.google.com/dataproc/docs/concepts/configuring\-clusters/flexible\-vms)


.RE
.RE
.sp

.SH "GCLOUD WIDE FLAGS"

These flags are available to all commands: \-\-access\-token\-file, \-\-account,
\-\-billing\-project, \-\-configuration, \-\-flags\-file, \-\-flatten,
\-\-format, \-\-help, \-\-impersonate\-service\-account, \-\-log\-http,
\-\-project, \-\-quiet, \-\-trace\-token, \-\-user\-output\-enabled,
\-\-verbosity.

Run \fB$ gcloud help\fR for details.



.SH "NOTES"

These variants are also available:

.RS 2m
$ gcloud alpha dataproc clusters create
.RE

.RS 2m
$ gcloud beta dataproc clusters create
.RE