.TH "GCLOUD_ALPHA_DATAPROC_CLUSTERS_GKE_CREATE" 1



.SH "NAME"
.HP
gcloud alpha dataproc clusters gke create \- create a GKE\-based virtual cluster



.SH "SYNOPSIS"
.HP
\f5gcloud alpha dataproc clusters gke create\fR (\fICLUSTER\fR\ :\ \fB\-\-region\fR=\fIREGION\fR) \fB\-\-spark\-engine\-version\fR=\fISPARK_ENGINE_VERSION\fR (\fB\-\-gke\-cluster\fR=\fIGKE_CLUSTER\fR\ :\ \fB\-\-gke\-cluster\-location\fR=\fIGKE_CLUSTER_LOCATION\fR) [\fB\-\-async\fR] [\fB\-\-namespace\fR=\fINAMESPACE\fR] [\fB\-\-pools\fR=[\fIKEY\fR=\fIVALUE\fR[;\fIVALUE\fR],...]] [\fB\-\-properties\fR=[\fIPREFIX\fR:\fIPROPERTY\fR=\fIVALUE\fR,...]] [\fB\-\-setup\-workload\-identity\fR] [\fB\-\-staging\-bucket\fR=\fISTAGING_BUCKET\fR] [\fB\-\-history\-server\-cluster\fR=\fIHISTORY_SERVER_CLUSTER\fR\ :\ \fB\-\-history\-server\-cluster\-region\fR=\fIHISTORY_SERVER_CLUSTER_REGION\fR] [\fB\-\-metastore\-service\fR=\fIMETASTORE_SERVICE\fR\ :\ \fB\-\-metastore\-service\-location\fR=\fIMETASTORE_SERVICE_LOCATION\fR] [\fIGCLOUD_WIDE_FLAG\ ...\fR]



.SH "DESCRIPTION"

\fB(ALPHA)\fR Create a GKE\-based virtual cluster.



.SH "EXAMPLES"

Create a Dataproc on GKE cluster in us\-central1 on a GKE cluster in the same
project and region with default values:

.RS 2m
$ gcloud alpha dataproc clusters gke create my\-cluster \e
    \-\-region=us\-central1 \-\-gke\-cluster=my\-gke\-cluster \e
    \-\-spark\-engine\-version=latest \-\-pools='name=dp,roles=default'
.RE

Create a Dataproc on GKE cluster in us\-central1 on a GKE cluster in the same
project and zone us\-central1\-f with default values:

.RS 2m
$ gcloud alpha dataproc clusters gke create my\-cluster \e
    \-\-region=us\-central1 \-\-gke\-cluster=my\-gke\-cluster \e
    \-\-gke\-cluster\-location=us\-central1\-f \e
    \-\-spark\-engine\-version=3.1 \-\-pools='name=dp,roles=default'
.RE

Create a Dataproc on GKE cluster in us\-central1 with machine type
\'e2\-standard\-4', autoscaling 5\-15 nodes per zone.

.RS 2m
$ gcloud alpha dataproc clusters gke create my\-cluster \e
    \-\-region='us\-central1' \e
    \-\-gke\-cluster='projects/my\-project/locations/us\-central1/cluster\e
s/my\-gke\-cluster' \-\-spark\-engine\-version=dataproc\-1.5 \e
    \-\-pools='name=dp\-default,roles=default,machineType=e2\-standard\-4\e
,min=5,max=15'
.RE

Create a Dataproc on GKE cluster in us\-central1 with two distinct node pools.

.RS 2m
$ gcloud alpha dataproc clusters gke create my\-cluster \e
    \-\-region='us\-central1' \-\-gke\-cluster='my\-gke\-cluster' \e
    \-\-spark\-engine\-version='dataproc\-2.0' \e
    \-\-pools='name=dp\-default,roles=default,machineType=e2\-standard\-4\e
' \e
    \-\-pools='name=workers,roles=spark\-drivers;spark\-executors,machin\e
eType=n2\-standard\-8
.RE



.SH "POSITIONAL ARGUMENTS"

.RS 2m
.TP 2m

Cluster resource \- The name of the cluster to create. The arguments in this
group can be used to specify the attributes of this resource. (NOTE) Some
attributes are not given arguments in this group but can be set in other ways.

To set the \f5project\fR attribute:
.RS 2m
.IP "\(em" 2m
provide the argument \f5cluster\fR on the command line with a fully specified
name;
.IP "\(em" 2m
provide the argument \f5\-\-project\fR on the command line;
.IP "\(em" 2m
set the property \f5core/project\fR.
.RE
.sp

This must be specified.


.RS 2m
.TP 2m
\fICLUSTER\fR

ID of the cluster or fully qualified identifier for the cluster.

To set the \f5cluster\fR attribute:
.RS 2m
.IP "\(bu" 2m
provide the argument \f5cluster\fR on the command line.
.RE
.sp

This positional argument must be specified if any of the other arguments in this
group are specified.

.TP 2m
\fB\-\-region\fR=\fIREGION\fR

Dataproc region for the cluster. Each Dataproc region constitutes an independent
resource namespace constrained to deploying instances into Compute Engine zones
inside the region. Overrides the default \f5dataproc/region\fR property value
for this command invocation.

To set the \f5region\fR attribute:
.RS 2m
.IP "\(bu" 2m
provide the argument \f5cluster\fR on the command line with a fully specified
name;
.IP "\(bu" 2m
provide the argument \f5\-\-region\fR on the command line;
.IP "\(bu" 2m
set the property \f5dataproc/region\fR.
.RE
.sp


.RE
.RE
.sp

.SH "REQUIRED FLAGS"

.RS 2m
.TP 2m
\fB\-\-spark\-engine\-version\fR=\fISPARK_ENGINE_VERSION\fR

The version of the Spark engine to run on this cluster.

.TP 2m

Gke cluster resource \- The GKE cluster to install the Dataproc cluster on. The
arguments in this group can be used to specify the attributes of this resource.
(NOTE) Some attributes are not given arguments in this group but can be set in
other ways.

To set the \f5project\fR attribute:
.RS 2m
.IP "\(em" 2m
provide the argument \f5\-\-gke\-cluster\fR on the command line with a fully
specified name;
.IP "\(em" 2m
provide the argument \f5\-\-project\fR on the command line;
.IP "\(em" 2m
set the property \f5core/project\fR.
.RE
.sp

This must be specified.


.RS 2m
.TP 2m
\fB\-\-gke\-cluster\fR=\fIGKE_CLUSTER\fR

ID of the gke\-cluster or fully qualified identifier for the gke\-cluster.

To set the \f5gke\-cluster\fR attribute:
.RS 2m
.IP "\(bu" 2m
provide the argument \f5\-\-gke\-cluster\fR on the command line.
.RE
.sp

This flag argument must be specified if any of the other arguments in this group
are specified.

.TP 2m
\fB\-\-gke\-cluster\-location\fR=\fIGKE_CLUSTER_LOCATION\fR

GKE region for the gke\-cluster.

To set the \f5gke\-cluster\-location\fR attribute:
.RS 2m
.IP "\(bu" 2m
provide the argument \f5\-\-gke\-cluster\fR on the command line with a fully
specified name;
.IP "\(bu" 2m
provide the argument \f5\-\-gke\-cluster\-location\fR on the command line;
.IP "\(bu" 2m
provide the argument \f5\-\-region\fR on the command line;
.IP "\(bu" 2m
set the property \f5dataproc/region\fR.
.RE
.sp


.RE
.RE
.sp

.SH "OPTIONAL FLAGS"

.RS 2m
.TP 2m
\fB\-\-async\fR

Return immediately, without waiting for the operation in progress to complete.

.TP 2m
\fB\-\-namespace\fR=\fINAMESPACE\fR

The name of the Kubernetes namespace to deploy Dataproc system components in.
This namespace does not need to exist.

.TP 2m
\fB\-\-pools\fR=[\fIKEY\fR=\fIVALUE\fR[;\fIVALUE\fR],...]

Each \f5\-\-pools\fR flag represents a GKE node pool associated with the virtual
cluster. It is a comma\-separated list in the form \f5KEY=VALUE[;VALUE]\fR,
where certain keys may have multiple values.

The following KEYs must be specified:

.RS 2m
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
KEY    Type             Example                  Description
\-\-\-\-\-\- \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
name   string           `my\-node\-pool`          Name of the node pool.
.RE

.RS 2m
roles  repeated string  `default;spark\-driver`  Roles that each node pool will perform.
                                                [One Pool must have DEFAULT role] Valid values are
                                                `default`, `controller`, `spark\-driver`, `spark\-executor`.
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
.RE

The following KEYs may be specified:

.RS 2m
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
KEY                Type             Example                                       Description
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
machineType        string           `n1\-standard\-8`                               Compute Engine machine type to use.
.RE

.RS 2m
preemptible        boolean          `false`                                       If true, then this node pool uses preemptible VMs.
                                                                                  This Must be `false` for a node pool with the CONTROLLER role or
                                                                                  for a node pool with the DEFAULT role in no node pool has the CONTROLLER role.
.RE

.RS 2m
localSsdCount      int              `2`                                           The number of local SSDs to attach to each node.
.RE

.RS 2m
localNvmeSsdCount  int              `2`                                           The number of local NVMe SSDs to attach to each node.
.RE

.RS 2m
accelerator        repeated string  `nvidia\-tesla\-a100=1`                         Accelerators to attach to each node, in NODE=COUNT format.
.RE

.RS 2m
minCpuPlatform     string           `Intel Skylake`                               Minimum CPU platform for each node.
.RE

.RS 2m
bootDiskKmsKey     string           `projects/project\-id/locations/us\-central1    The Customer Managed Encryption Key (CMEK) used to encrypt
                                    /keyRings/keyRing\-name/cryptoKeys/key\-name`   the boot disk attached to each node in the node pool.
.RE

.RS 2m
locations          repeated string  `us\-west1\-a;us\-west1\-c`                       Zones within the location of the GKE cluster.
                                                                                  All `\-\-pools` flags for a Dataproc cluster must have identical locations.
.RE

.RS 2m
min                int              `0`                                           Minimum number of nodes per zone that this node pool can scale down to.
.RE

.RS 2m
max                int              `10`                                          Maximum number of nodes per zone that this node pool can scale up to.
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
.RE

.TP 2m
\fB\-\-properties\fR=[\fIPREFIX\fR:\fIPROPERTY\fR=\fIVALUE\fR,...]

Specifies configuration properties for installed packages, such as Spark.
Properties are mapped to configuration files by specifying a prefix, such as
"core:io.serializations".

.TP 2m
\fB\-\-setup\-workload\-identity\fR

Sets up the GKE Workload Identity for your Dataproc on GKE cluster. Note that
running this requires elevated permissions as it will manipulate IAM policies on
the Google Service Accounts that will be used by your Dataproc on GKE cluster.

.TP 2m
\fB\-\-staging\-bucket\fR=\fISTAGING_BUCKET\fR

The Cloud Storage bucket to use to stage job dependencies, miscellaneous config
files, and job driver console output when using this cluster.

.TP 2m

History server cluster resource \- A Dataproc Cluster created as a History
Server, see https://cloud.google.com/dataproc/docs/concepts/jobs/history\-server
The arguments in this group can be used to specify the attributes of this
resource. (NOTE) Some attributes are not given arguments in this group but can
be set in other ways.

To set the \f5project\fR attribute:
.RS 2m
.IP "\(em" 2m
provide the argument \f5\-\-history\-server\-cluster\fR on the command line with
a fully specified name;
.IP "\(em" 2m
provide the argument \f5\-\-project\fR on the command line;
.IP "\(em" 2m
set the property \f5core/project\fR.
.RE
.sp


.RS 2m
.TP 2m
\fB\-\-history\-server\-cluster\fR=\fIHISTORY_SERVER_CLUSTER\fR

ID of the history\-server\-cluster or fully qualified identifier for the
history\-server\-cluster.

To set the \f5history\-server\-cluster\fR attribute:
.RS 2m
.IP "\(bu" 2m
provide the argument \f5\-\-history\-server\-cluster\fR on the command line.
.RE
.sp

This flag argument must be specified if any of the other arguments in this group
are specified.

.TP 2m
\fB\-\-history\-server\-cluster\-region\fR=\fIHISTORY_SERVER_CLUSTER_REGION\fR

Compute Engine region for the history\-server\-cluster. It must be the same
region as the Dataproc cluster that is being created.

To set the \f5history\-server\-cluster\-region\fR attribute:
.RS 2m
.IP "\(bu" 2m
provide the argument \f5\-\-history\-server\-cluster\fR on the command line with
a fully specified name;
.IP "\(bu" 2m
provide the argument \f5\-\-history\-server\-cluster\-region\fR on the command
line;
.IP "\(bu" 2m
provide the argument \f5\-\-region\fR on the command line;
.IP "\(bu" 2m
set the property \f5dataproc/region\fR.
.RE
.sp

.RE
.sp
.TP 2m

Metastore service resource \- Dataproc Metastore Service to be used as an
external metastore. The arguments in this group can be used to specify the
attributes of this resource. (NOTE) Some attributes are not given arguments in
this group but can be set in other ways.

To set the \f5project\fR attribute:
.RS 2m
.IP "\(em" 2m
provide the argument \f5\-\-metastore\-service\fR on the command line with a
fully specified name;
.IP "\(em" 2m
provide the argument \f5\-\-project\fR on the command line;
.IP "\(em" 2m
set the property \f5core/project\fR.
.RE
.sp


.RS 2m
.TP 2m
\fB\-\-metastore\-service\fR=\fIMETASTORE_SERVICE\fR

ID of the metastore\-service or fully qualified identifier for the
metastore\-service.

To set the \f5metastore\-service\fR attribute:
.RS 2m
.IP "\(bu" 2m
provide the argument \f5\-\-metastore\-service\fR on the command line.
.RE
.sp

This flag argument must be specified if any of the other arguments in this group
are specified.

.TP 2m
\fB\-\-metastore\-service\-location\fR=\fIMETASTORE_SERVICE_LOCATION\fR

Dataproc Metastore location for the metastore\-service.

To set the \f5metastore\-service\-location\fR attribute:
.RS 2m
.IP "\(bu" 2m
provide the argument \f5\-\-metastore\-service\fR on the command line with a
fully specified name;
.IP "\(bu" 2m
provide the argument \f5\-\-metastore\-service\-location\fR on the command line;
.IP "\(bu" 2m
provide the argument \f5\-\-region\fR on the command line;
.IP "\(bu" 2m
set the property \f5dataproc/region\fR.
.RE
.sp


.RE
.RE
.sp

.SH "GCLOUD WIDE FLAGS"

These flags are available to all commands: \-\-access\-token\-file, \-\-account,
\-\-billing\-project, \-\-configuration, \-\-flags\-file, \-\-flatten,
\-\-format, \-\-help, \-\-impersonate\-service\-account, \-\-log\-http,
\-\-project, \-\-quiet, \-\-trace\-token, \-\-user\-output\-enabled,
\-\-verbosity.

Run \fB$ gcloud help\fR for details.



.SH "NOTES"

This command is currently in alpha and might change without notice. If this
command fails with API permission errors despite specifying the correct project,
you might be trying to access an API with an invitation\-only early access
allowlist. These variants are also available:

.RS 2m
$ gcloud dataproc clusters gke create
.RE

.RS 2m
$ gcloud beta dataproc clusters gke create
.RE