.TH "GCLOUD_BETA_DATAFLOW_FLEX\-TEMPLATE_BUILD" 1



.SH "NAME"
.HP
gcloud beta dataflow flex\-template build \- builds a flex template file from the specified parameters



.SH "SYNOPSIS"
.HP
\f5gcloud beta dataflow flex\-template build\fR \fITEMPLATE_FILE_GCS_PATH\fR \fB\-\-sdk\-language\fR=\fISDK_LANGUAGE\fR (\fB\-\-image\fR=\fIIMAGE\fR\ |\ \fB\-\-env\fR=[\fIENV\fR,...]\ \fB\-\-flex\-template\-base\-image\fR=\fIFLEX_TEMPLATE_BASE_IMAGE\fR\ \fB\-\-image\-gcr\-path\fR=\fIIMAGE_GCR_PATH\fR\ (\fB\-\-go\-binary\-path\fR=\fIGO_BINARY_PATH\fR\ |\ \fB\-\-jar\fR=[\fIJAR\fR,...]\ |\ \fB\-\-py\-path\fR=[\fIPY_PATH\fR,...])\ |\ [\fB\-\-yaml\-pipeline\-path\fR=\fIYAML_PIPELINE_PATH\fR\ :\ \fB\-\-yaml\-image\fR=\fIYAML_IMAGE\fR]) [\fB\-\-additional\-experiments\fR=[\fIADDITIONAL_EXPERIMENTS\fR,...]] [\fB\-\-additional\-user\-labels\fR=[\fIADDITIONAL_USER_LABELS\fR,...]] [\fB\-\-cloud\-build\-service\-account\fR=\fICLOUD_BUILD_SERVICE_ACCOUNT\fR] [\fB\-\-dataflow\-kms\-key\fR=\fIDATAFLOW_KMS_KEY\fR] [\fB\-\-disable\-public\-ips\fR] [\fB\-\-enable\-streaming\-engine\fR] [\fB\-\-gcs\-log\-dir\fR=\fIGCS_LOG_DIR\fR] [\fB\-\-image\-repository\-cert\-path\fR=\fIIMAGE_REPOSITORY_CERT_PATH\fR] [\fB\-\-image\-repository\-password\-secret\-id\fR=\fIIMAGE_REPOSITORY_PASSWORD_SECRET_ID\fR] [\fB\-\-image\-repository\-username\-secret\-id\fR=\fIIMAGE_REPOSITORY_USERNAME_SECRET_ID\fR] [\fB\-\-max\-workers\fR=\fIMAX_WORKERS\fR] [\fB\-\-metadata\-file\fR=\fIPATH_TO_FILE\fR] [\fB\-\-network\fR=\fINETWORK\fR] [\fB\-\-num\-workers\fR=\fINUM_WORKERS\fR] [\fB\-\-print\-only\fR] [\fB\-\-service\-account\-email\fR=\fISERVICE_ACCOUNT_EMAIL\fR] [\fB\-\-staging\-location\fR=\fISTAGING_LOCATION\fR] [\fB\-\-subnetwork\fR=\fISUBNETWORK\fR] [\fB\-\-temp\-location\fR=\fITEMP_LOCATION\fR] [\fB\-\-worker\-machine\-type\fR=\fIWORKER_MACHINE_TYPE\fR] [\fB\-\-worker\-region\fR=\fIWORKER_REGION\fR\ |\ \fB\-\-worker\-zone\fR=\fIWORKER_ZONE\fR] [\fIGCLOUD_WIDE_FLAG\ ...\fR]



.SH "DESCRIPTION"

\fB(BETA)\fR Builds a flex template file from the specified parameters.



.SH "EXAMPLES"

To build and store a flex template JSON file, run:

.RS 2m
$ gcloud beta dataflow flex\-template build \e
    gs://template\-file\-gcs\-path \-\-image=gcr://image\-path \e
    \-\-metadata\-file=/local/path/to/metadata.json \-\-sdk\-language=JAVA
.RE

If using prebuilt template image from private registry, run:

.RS 2m
$ gcloud beta dataflow flex\-template build \e
    gs://template\-file\-gcs\-path \e
    \-\-image=private.registry.com:3000/image\-path \e
    \-\-image\-repository\-username\-secret\-id="projects/test\-project/sec\e
rets/username\-secret" \e
    \-\-image\-repository\-password\-secret\-id="projects/test\-project/sec\e
rets/password\-secret/versions/latest" \e
    \-\-metadata\-file=metadata.json \-\-sdk\-language=JAVA
.RE

To build the template image and flex template JSON file, run:

.RS 2m
$ gcloud beta dataflow flex\-template build \e
    gs://template\-file\-gcs\-path \e
    \-\-image\-gcr\-path=gcr://path\-to\-store\-image \e
    \-\-jar=path/to/pipeline.jar \-\-jar=path/to/dependency.jar \e
    \-\-env=FLEX_TEMPLATE_JAVA_MAIN_CLASS=classpath \e
    \-\-flex\-template\-base\-image=JAVA11 \e
    \-\-metadata\-file=/local/path/to/metadata.json \-\-sdk\-language=JAVA
.RE



.SH "POSITIONAL ARGUMENTS"

.RS 2m
.TP 2m
\fITEMPLATE_FILE_GCS_PATH\fR

The Google Cloud Storage location of the flex template file.Overrides if file
already exists.


.RE
.sp

.SH "REQUIRED FLAGS"

.RS 2m
.TP 2m
\fB\-\-sdk\-language\fR=\fISDK_LANGUAGE\fR

SDK language of the flex template job. \fISDK_LANGUAGE\fR must be one of:
\fBJAVA\fR, \fBPYTHON\fR, \fBGO\fR, \fBYAML\fR.

.TP 2m

Exactly one of these must be specified:


.RS 2m
.TP 2m
\fB\-\-image\fR=\fIIMAGE\fR

Path to the any image registry location of the prebuilt flex template image.

.TP 2m
\fB\-\-env\fR=[\fIENV\fR,...]

Environment variables to create for the Dockerfile. You can pass them as a comma
separated list or repeat individually with \-\-env flag. Ex: \-\-env="A=B,C=D"
or \-\-env A=B, \-\-env C=D.When you reference files/dir in env variables,
please specify relative path to the paths passed via \-\-py\-path.Ex: if you
pass. \-\-py\-path="path/pipleline/" then set
FLEX_TEMPLATE_PYTHON_PY_FILE="pipeline/pipeline.py" You can find the list of
supported environment variables in this link.
https://cloud.google.com/dataflow/docs/guides/templates/configuring\-flex\-templates#setting_required_dockerfile_environment_variables.

This flag argument must be specified if any of the other arguments in this group
are specified.

.TP 2m
\fB\-\-flex\-template\-base\-image\fR=\fIFLEX_TEMPLATE_BASE_IMAGE\fR

Flex template base image to be used while building the container image. Allowed
choices are allowed labels (JAVA11, JAVA17, JAVA21, PYTHON3, GO), supported
distroless images (JAVA11_DISTROLESS, JAVA17_DISTROLESS, JAVA21_DISTROLESS,
GO_DISTROLESS), or full gcr.io path of the specific version of the base image.
For labels, we use the latest base image version to build the container. You can
also provide a specific version from this link
https://gcr.io/dataflow\-templates\-base/

This flag argument must be specified if any of the other arguments in this group
are specified.

.TP 2m
\fB\-\-image\-gcr\-path\fR=\fIIMAGE_GCR_PATH\fR

The Google Container Registry or Google Artifact Registry location to store the
flex template image to be built.

This flag argument must be specified if any of the other arguments in this group
are specified.

.TP 2m

Exactly one of these must be specified:


.RS 2m
.TP 2m
\fB\-\-go\-binary\-path\fR=\fIGO_BINARY_PATH\fR

Local path to your compiled dataflow pipeline Go binary. The binary should be
compiled to run on the target worker architecture (usually linux\-amd64). See
https://beam.apache.org/documentation/sdks/go\-cross\-compilation/ for more
information.

.TP 2m
\fB\-\-jar\fR=[\fIJAR\fR,...]

Local path to your dataflow pipeline jar file and all their dependent jar files
required for the flex template classpath. You can pass them as a comma separated
list or repeat individually with \-\-jar flag. Ex: \-\-jar="code.jar,dep.jar" or
\-\-jar code.jar, \-\-jar dep.jar.

.TP 2m
\fB\-\-py\-path\fR=[\fIPY_PATH\fR,...]

Local path to your dataflow pipeline python files and all their dependent files
required for the flex template classpath. You can pass them as a comma separated
list or repeat individually with \-\-py\-path flag. Ex:
\-\-py\-path="path/pipleline/,path/dependency/" or \-\-py\-path path/pipleline/,
\-\-py\-path path/dependency/.

.RE
.sp
.TP 2m
\fB\-\-yaml\-pipeline\-path\fR=\fIYAML_PIPELINE_PATH\fR

Local path to your YAML pipeline file. Use a full or relative path to a local
file containing the value of yaml_pipeline_path.

This flag argument must be specified if any of the other arguments in this group
are specified.

.TP 2m
\fB\-\-yaml\-image\fR=\fIYAML_IMAGE\fR

Path to the any image registry location of the prebuilt yaml template image.


.RE
.RE
.sp

.SH "OPTIONAL FLAGS"

.RS 2m
.TP 2m
\fB\-\-additional\-experiments\fR=[\fIADDITIONAL_EXPERIMENTS\fR,...]

Default experiments to pass to the job.

.TP 2m
\fB\-\-additional\-user\-labels\fR=[\fIADDITIONAL_USER_LABELS\fR,...]

Default user labels to pass to the job. Example:
\-\-additional\-user\-labels='{"key1":"value1"}'

.TP 2m
\fB\-\-cloud\-build\-service\-account\fR=\fICLOUD_BUILD_SERVICE_ACCOUNT\fR

Service account to run the Cloud Build in the format
projects/{project}/serviceAccounts/{service_account}. Ensure that the account
you are using to run 'gcloud dataflow flex\-template build' has
\'ServiceAccountUser' role on the specified Cloud Build service account you
provide with the \-\-cloud\-build\-service\-account flag. The specified service
account must have required permissions to build the image. If the specified
service account is in a project that is different from the project where you are
starting builds, see
https://cloud.google.com/build/docs/securing\-builds/configure\-user\-specified\-service\-accounts#cross\-project_set_up
to grant the necessary access.

.TP 2m
\fB\-\-dataflow\-kms\-key\fR=\fIDATAFLOW_KMS_KEY\fR

Default Cloud KMS key to protect the job resources.

.TP 2m
\fB\-\-disable\-public\-ips\fR

Cloud Dataflow workers must not use public IP addresses. Overrides the default
\fBdataflow/disable_public_ips\fR property value for this command invocation.

.TP 2m
\fB\-\-enable\-streaming\-engine\fR

Enable Streaming Engine for the streaming job by default. Overrides the default
\fBdataflow/enable_streaming_engine\fR property value for this command
invocation.

.TP 2m
\fB\-\-gcs\-log\-dir\fR=\fIGCS_LOG_DIR\fR

Google Cloud Storage directory to save build logs.(Must be a URL beginning with
\'gs://'.)

.TP 2m
\fB\-\-image\-repository\-cert\-path\fR=\fIIMAGE_REPOSITORY_CERT_PATH\fR

The full URL to self\-signed certificate of private registry in Cloud Storage.
For example, gs://mybucket/mycerts/selfsigned.crt. The certificate provided in
Cloud Storage must be DER\-encoded and may be supplied in binary or printable
(Base64) encoding. If the certificate is provided in Base64 encoding, it must be
bounded at the beginning by \-\-\-\-\-BEGIN CERTIFICATE\-\-\-\-\-, and must be
bounded at the end by \-\-\-\-\-END CERTIFICATE\-\-\-\-\-. If this parameter is
provided, the docker daemon in the template launcher will be instructed to trust
that certificate.

.TP 2m
\fB\-\-image\-repository\-password\-secret\-id\fR=\fIIMAGE_REPOSITORY_PASSWORD_SECRET_ID\fR

Secret Manager secret id for the password to authenticate to private registry.
Should be in the format
projects/{project}/secrets/{secret}/versions/{secret_version} or
projects/{project}/secrets/{secret}. If the version is not provided latest
version will be used.

.TP 2m
\fB\-\-image\-repository\-username\-secret\-id\fR=\fIIMAGE_REPOSITORY_USERNAME_SECRET_ID\fR

Secret Manager secret id for the username to authenticate to private registry.
Should be in the format
projects/{project}/secrets/{secret}/versions/{secret_version} or
projects/{project}/secrets/{secret}. If the version is not provided latest
version will be used.

.TP 2m
\fB\-\-max\-workers\fR=\fIMAX_WORKERS\fR

Default maximum number of workers to run.

.TP 2m
\fB\-\-metadata\-file\fR=\fIPATH_TO_FILE\fR

Local path to the metadata json file for the flex template. Use a full or
relative path to a local file containing the value of metadata_file.

.TP 2m
\fB\-\-network\fR=\fINETWORK\fR

Default Compute Engine network for launching instances to run your pipeline.

.TP 2m
\fB\-\-num\-workers\fR=\fINUM_WORKERS\fR

Initial number of workers to use by default.

.TP 2m
\fB\-\-print\-only\fR

Prints the container spec to stdout. Does not save in Google Cloud Storage.
Overrides the default \fBdataflow/print_only\fR property value for this command
invocation.

.TP 2m
\fB\-\-service\-account\-email\fR=\fISERVICE_ACCOUNT_EMAIL\fR

Default service account to run the workers as.

.TP 2m
\fB\-\-staging\-location\fR=\fISTAGING_LOCATION\fR

Default Google Cloud Storage location to stage local files.(Must be a URL
beginning with 'gs://'.)

.TP 2m
\fB\-\-subnetwork\fR=\fISUBNETWORK\fR

Default Compute Engine subnetwork for launching instances to run your pipeline.

.TP 2m
\fB\-\-temp\-location\fR=\fITEMP_LOCATION\fR

Default Google Cloud Storage location to stage temporary files. If not set,
defaults to the value for \-\-staging\-location.(Must be a URL beginning with
\'gs://'.)

.TP 2m
\fB\-\-worker\-machine\-type\fR=\fIWORKER_MACHINE_TYPE\fR

Default type of machine to use for workers. Defaults to server\-specified.

.TP 2m

At most one of these can be specified:


.RS 2m
.TP 2m
\fB\-\-worker\-region\fR=\fIWORKER_REGION\fR

Default region to run the workers in.

.TP 2m
\fB\-\-worker\-zone\fR=\fIWORKER_ZONE\fR

Default zone to run the workers in.


.RE
.RE
.sp

.SH "GCLOUD WIDE FLAGS"

These flags are available to all commands: \-\-access\-token\-file, \-\-account,
\-\-billing\-project, \-\-configuration, \-\-flags\-file, \-\-flatten,
\-\-format, \-\-help, \-\-impersonate\-service\-account, \-\-log\-http,
\-\-project, \-\-quiet, \-\-trace\-token, \-\-user\-output\-enabled,
\-\-verbosity.

Run \fB$ gcloud help\fR for details.



.SH "NOTES"

This command is currently in beta and might change without notice. This variant
is also available:

.RS 2m
$ gcloud dataflow flex\-template build
.RE