.TH "GCLOUD_ALPHA_STORAGE_RSYNC" 1



.SH "NAME"
.HP
gcloud alpha storage rsync \- synchronize content of two buckets/directories



.SH "SYNOPSIS"
.HP
\f5gcloud alpha storage rsync\fR \fISOURCE\fR \fIDESTINATION\fR [\fB\-\-additional\-headers\fR=\fIHEADER\fR=\fIVALUE\fR] [\fB\-\-canned\-acl\fR=\fIPREDEFINED_ACL\fR,\ \fB\-\-predefined\-acl\fR=\fIPREDEFINED_ACL\fR,\ \fB\-a\fR\ \fIPREDEFINED_ACL\fR] [\fB\-\-checksums\-only\fR] [\fB\-\-no\-clobber\fR,\ \fB\-n\fR] [\fB\-\-content\-md5\fR=\fIMD5_DIGEST\fR] [\fB\-\-continue\-on\-error\fR,\ \fB\-c\fR] [\fB\-\-delete\-unmatched\-destination\-objects\fR] [\fB\-\-dry\-run\fR] [\fB\-\-exclude\fR=[\fIREGEX\fR,...],\ \fB\-x\fR\ [\fIREGEX\fR,...]] [\fB\-\-gzip\-in\-flight\fR=[\fIFILE_EXTENSIONS\fR,...],\ \fB\-j\fR\ [\fIFILE_EXTENSIONS\fR,...]] [\fB\-\-gzip\-in\-flight\-all\fR,\ \fB\-J\fR] [\fB\-\-no\-ignore\-symlinks\fR] [\fB\-\-include\-managed\-folders\fR] [\fB\-\-preserve\-posix\fR,\ \fB\-P\fR] [\fB\-\-recursive\fR,\ \fB\-R\fR,\ \fB\-r\fR] [\fB\-\-skip\-if\-dest\-has\-newer\-mtime\fR,\ \fB\-u\fR] [\fB\-\-skip\-unsupported\fR,\ \fB\-U\fR] [\fB\-\-decryption\-keys\fR=[\fIDECRYPTION_KEY\fR,...]\ \fB\-\-encryption\-key\fR=\fIENCRYPTION_KEY\fR] [\fB\-\-cache\-control\fR=\fICACHE_CONTROL\fR\ \fB\-\-content\-disposition\fR=\fICONTENT_DISPOSITION\fR\ \fB\-\-content\-encoding\fR=\fICONTENT_ENCODING\fR\ \fB\-\-content\-language\fR=\fICONTENT_LANGUAGE\fR\ \fB\-\-content\-type\fR=\fICONTENT_TYPE\fR\ \fB\-\-custom\-time\fR=\fICUSTOM_TIME\fR\ \fB\-\-clear\-custom\-contexts\fR\ |\ \fB\-\-custom\-contexts\fR=[\fICUSTOM_CONTEXTS_KEYS_AND_VALUES\fR,...]\ |\ \fB\-\-custom\-contexts\-file\fR=\fICUSTOM_CONTEXTS_FILE\fR\ |\ \fB\-\-remove\-custom\-contexts\fR=[\fICUSTOM_CONTEXTS_KEYS\fR,...]\ \fB\-\-update\-custom\-contexts\fR=[\fICUSTOM_CONTEXTS_KEYS_AND_VALUES\fR,...]\ \fB\-\-clear\-custom\-metadata\fR\ |\ \fB\-\-custom\-metadata\fR=[\fICUSTOM_METADATA_KEYS_AND_VALUES\fR,...]\ |\ \fB\-\-remove\-custom\-metadata\fR=[\fIMETADATA_KEYS\fR,...]\ \fB\-\-update\-custom\-metadata\fR=[\fICUSTOM_METADATA_KEYS_AND_VALUES\fR,...]] [\fB\-\-if\-generation\-match\fR=\fIGENERATION\fR\ \fB\-\-if\-metageneration\-match\fR=\fIMETAGENERATION\fR] [\fIGCLOUD_WIDE_FLAG\ ...\fR]



.SH "DESCRIPTION"

\fB(ALPHA)\fR \fBgcloud alpha storage rsync\fR copies to and updates objects at
\f5DESTINATION\fR to match \f5SOURCE\fR. \f5SOURCE\fR must specify a directory,
bucket, or bucket subdirectory. \fBgcloud alpha storage rsync\fR does not copy
empty directory trees, since storage providers use a flat namespace
(https://cloud.google.com/storage/docs/folders).

Note, shells (like bash, zsh) sometimes attempt to expand wildcards in ways that
can be surprising. Also, attempting to copy files whose names contain wildcard
characters can result in problems.

If synchronizing a large amount of data between clouds you might consider
setting up a Google Compute Engine account and running \fBgcloud alpha storage
rsync\fR there. Since \fBgcloud alpha storage rsync\fR cross\-provider data
transfers flow through the machine where \fBgcloud alpha storage rsync\fR is
running, doing this can make your transfer run significantly faster than on your
local workstation.



.SH "EXAMPLES"

To sync the contents of the local directory \f5data\fR to the bucket
gs://my\-bucket/data:

.RS 2m
$ gcloud alpha storage rsync data gs://my\-bucket/data
.RE

To recurse into directories use \f5\-\-recursive\fR:

.RS 2m
$ gcloud alpha storage rsync data gs://my\-bucket/data \-\-recursive
.RE

To make the local directory \f5my\-data\fR the same as the contents of
gs://mybucket/data and delete objects in the local directory that are not in
gs://mybucket/data:

.RS 2m
$ gcloud alpha storage rsync gs://mybucket/data my\-data \e
    \-\-recursive \-\-delete\-unmatched\-destination\-objects
.RE

To make the contents of gs://mybucket2 the same as gs://mybucket1 and delete
objects in gs://mybucket2 that are not in gs://mybucket1:

.RS 2m
$ gcloud alpha storage rsync gs://mybucket1 gs://mybucket2 \e
    \-\-recursive \-\-delete\-unmatched\-destination\-objects
.RE

To copy all objects from \f5dir1\fR into \f5dir2\fR and delete all objects in
\f5dir2\fR which are not in \f5dir1\fR:

.RS 2m
$ gcloud alpha storage rsync dir1 dir2 \-\-recursive \- \e
    \-\-delete\-unmatched\-destination\-objects
.RE

To mirror your objects across cloud providers:

.RS 2m
$ gcloud alpha storage rsync gs://my\-gs\-bucket s3://my\-s3\-bucket \e
    \-\-recursive \-\-delete\-unmatched\-destination\-objects
.RE

To apply gzip compression to only uploaded image files in \f5dir\fR:

.RS 2m
$ gcloud alpha storage rsync dir gs://my\-bucket/data \e
    \-\-gzip\-in\-flight=jpeg,jpg,gif,png
.RE

To skip the file \f5dir/data1/a.txt\fR:

.RS 2m
$ gcloud alpha storage rsync dir gs://my\-bucket \e
    \-\-exclude="data./.*\e.txt$"
.RE

To skip all .txt and .jpg files:

.RS 2m
$ gcloud alpha storage rsync dir gs://my\-bucket \e
    \-\-exclude=".*\e.txt$|.*\e.jpg$"
.RE

The contexts are preserved when using rsync to sync to Google Cloud Storage.
However, the following command would clear all custom contexts from the
destination object while syncing to the destination bucket.

.RS 2m
$ gcloud alpha storage rsync gs://my\-bucket/ \e
    gs://destination\-bucket/ \-\-clear\-custom\-contexts
.RE



.SH "POSITIONAL ARGUMENTS"

.RS 2m
.TP 2m
\fISOURCE\fR

The source container path.

.TP 2m
\fIDESTINATION\fR

The destination container path.


.RE
.sp

.SH "FLAGS"

.RS 2m
.TP 2m
\fB\-\-additional\-headers\fR=\fIHEADER\fR=\fIVALUE\fR

Includes arbitrary headers in storage API calls. Accepts a comma separated list
of key=value pairs, e.g. \f5header1=value1,header2=value2\fR. Overrides the
default \fBstorage/additional_headers\fR property value for this command
invocation.

.TP 2m
\fB\-\-canned\-acl\fR=\fIPREDEFINED_ACL\fR, \fB\-\-predefined\-acl\fR=\fIPREDEFINED_ACL\fR, \fB\-a\fR \fIPREDEFINED_ACL\fR

Applies predefined, or "canned," ACLs to a resource. See docs for a list of
predefined ACL constants:
https://cloud.google.com/storage/docs/access\-control/lists#predefined\-acl

.TP 2m
\fB\-\-checksums\-only\fR

When comparing objects with matching names at the source and destination, skip
modification time check and compare object hashes. Normally, hashes are only
compared if modification times are not available.

.TP 2m
\fB\-\-no\-clobber\fR, \fB\-n\fR

Do not overwrite existing files or objects at the destination. Skipped items
will be printed. This option may perform an additional GET request for cloud
objects before attempting an upload.

.TP 2m
\fB\-\-content\-md5\fR=\fIMD5_DIGEST\fR

Manually specified MD5 hash digest for the contents of an uploaded file. This
flag cannot be used when uploading multiple files. The custom digest is used by
the cloud provider for validation.

.TP 2m
\fB\-\-continue\-on\-error\fR, \fB\-c\fR

If any operations are unsuccessful, the command will exit with a non\-zero exit
status after completing the remaining operations. This flag takes effect only in
sequential execution mode (i.e. processor and thread count are set to 1).
Parallelism is default.

.TP 2m
\fB\-\-delete\-unmatched\-destination\-objects\fR

Delete extra files under DESTINATION not found under SOURCE. By default extra
files are not deleted. Managed folders are not affected by this flag.

Note: this option can delete data quickly if you specify the wrong source and
destination combination.

.TP 2m
\fB\-\-dry\-run\fR

Print what operations rsync would perform without actually executing them.

.TP 2m
\fB\-\-exclude\fR=[\fIREGEX\fR,...], \fB\-x\fR [\fIREGEX\fR,...]

Exclude objects matching regex pattern from rsync.

Note that this is a Python regular expression, not a pure wildcard pattern. For
example, matching a string ending in "abc" is \f5.*abc$\fR rather than
\f5*abc\fR. Also note that the exclude path is relative, as opposed to absolute
(similar to Linux \f5rsync\fR and \f5tar\fR exclude options).

For the Windows cmd.exe command line interpreter, use \f5^\fR as an escape
character instead of \f5\e\fR and escape the \f5|\fR character. When using
Windows PowerShell, use \f5'\fR instead of \f5"\fR and surround the \f5|\fR
character with \f5"\fR.

.TP 2m
\fB\-\-gzip\-in\-flight\fR=[\fIFILE_EXTENSIONS\fR,...], \fB\-j\fR [\fIFILE_EXTENSIONS\fR,...]

Applies gzip transport encoding to any file upload whose extension matches the
input extension list. This is useful when uploading files with compressible
content such as .js, .css, or .html files. This also saves network bandwidth
while leaving the data uncompressed in Cloud Storage.

When you specify the \f5\-\-gzip\-in\-flight\fR option, files being uploaded are
compressed in\-memory and on\-the\-wire only. Both the local files and Cloud
Storage objects remain uncompressed. The uploaded objects retain the
\f5Content\-Type\fR and name of the original files.

.TP 2m
\fB\-\-gzip\-in\-flight\-all\fR, \fB\-J\fR

Applies gzip transport encoding to file uploads. This option works like the
\f5\-\-gzip\-in\-flight\fR option described above, but it applies to all
uploaded files, regardless of extension.

CAUTION: If some of the source files don't compress well, such as binary data,
using this option may result in longer uploads.

.TP 2m
\fB\-\-ignore\-symlinks\fR

Ignore file symlinks instead of copying what they point to. Enabled by default,
use \fB\-\-no\-ignore\-symlinks\fR to disable.

.TP 2m
\fB\-\-include\-managed\-folders\fR

Includes managed folders in command operations. For transfers, gcloud storage
will set up managed folders in the destination with the same IAM policy bindings
as the source. Managed folders are only included with recursive cloud\-to\-cloud
transfers.

.TP 2m
\fB\-\-preserve\-posix\fR, \fB\-P\fR

Causes POSIX attributes to be preserved when objects are copied. With this
feature enabled, gcloud storage will copy several fields provided by the stat
command: access time, modification time, owner UID, owner group GID, and the
mode (permissions) of the file.

For uploads, these attributes are read off of local files and stored in the
cloud as custom metadata. For downloads, custom cloud metadata is set as POSIX
attributes on files after they are downloaded.

On Windows, this flag will only set and restore access time and modification
time because Windows doesn't have a notion of POSIX UID, GID, and mode.

.TP 2m
\fB\-\-recursive\fR, \fB\-R\fR, \fB\-r\fR

Recursively copy the contents of any directories that match the source path
expression.

.TP 2m
\fB\-\-skip\-if\-dest\-has\-newer\-mtime\fR, \fB\-u\fR

Skip operating on destination object if it has a newer modification time than
the source.

.TP 2m
\fB\-\-skip\-unsupported\fR, \fB\-U\fR

Skip objects with unsupported object types.


.RE
.sp

.SH "ENCRYPTION FLAGS"

.RS 2m
.TP 2m
\fB\-\-decryption\-keys\fR=[\fIDECRYPTION_KEY\fR,...]

A comma\-separated list of customer\-supplied encryption keys (RFC 4648 section
4 base64\-encoded AES256 strings) that will be used to decrypt Cloud Storage
objects. Data encrypted with a customer\-managed encryption key (CMEK) is
decrypted automatically, so CMEKs do not need to be listed here.

.TP 2m
\fB\-\-encryption\-key\fR=\fIENCRYPTION_KEY\fR

The encryption key to use for encrypting target objects. The specified
encryption key can be a customer\-supplied encryption key (An RFC 4648 section 4
base64\-encoded AES256 string), or a customer\-managed encryption key of the
form
\f5projects/{project}/locations/{location}/keyRings/{key\-ring}/cryptoKeys/{crypto\-key}\fR.
The specified key also acts as a decryption key, which is useful when copying or
moving encrypted data to a new location. Using this flag in an \f5objects
update\fR command triggers a rewrite of target objects.


.RE
.sp

.SH "OBJECT METADATA FLAGS"

.RS 2m
.TP 2m
\fB\-\-cache\-control\fR=\fICACHE_CONTROL\fR

How caches should handle requests and responses.

.TP 2m
\fB\-\-content\-disposition\fR=\fICONTENT_DISPOSITION\fR

How content should be displayed.

.TP 2m
\fB\-\-content\-encoding\fR=\fICONTENT_ENCODING\fR

How content is encoded (e.g. \f5\fIgzip\fR\fR).

.TP 2m
\fB\-\-content\-language\fR=\fICONTENT_LANGUAGE\fR

Content's language (e.g. \f5\fIen\fR\fR signifies "English").

.TP 2m
\fB\-\-content\-type\fR=\fICONTENT_TYPE\fR

Type of data contained in the object (e.g. \f5\fItext/html\fR\fR).

.TP 2m
\fB\-\-custom\-time\fR=\fICUSTOM_TIME\fR

Custom time for Cloud Storage objects in RFC 3339 format.

.TP 2m

Group that allow users to handle object contexts.

At most one of these can be specified:


.RS 2m
.TP 2m
\fB\-\-clear\-custom\-contexts\fR

Clears all custom contexts on objects.

.TP 2m
\fB\-\-custom\-contexts\fR=[\fICUSTOM_CONTEXTS_KEYS_AND_VALUES\fR,...]

Sets custom contexts on objects. The existing custom contexts (if any) would be
overwritten.

.TP 2m
\fB\-\-custom\-contexts\-file\fR=\fICUSTOM_CONTEXTS_FILE\fR

Path to a local JSON or YAML file containing custom contexts one wants to set on
an object. For example:

1. The following JSON document shows two key value pairs, i.e. (key1, value1)
and (key2, value2):

.RS 2m
  {
    "key1": {"value": "value1"},
    "key2": {"value": "value2"}
  }
.RE

2. The following YAML document shows two key value pairs, i.e. (key1, value1)
and (key2, value2):

.RS 2m
  key1:
    value: value1
  key2:
    value: value2
.RE

Note: Currently object contexts only supports string format for values.

.TP 2m

Flags that preserve the existing contexts on the object, and can be specified
together. However they cannot be specified with
\f5\-\-clear\-custom\-contexts\fR, \f5\-\-custom\-contexts\fR or
\f5\-\-custom\-contexts\-file\fR. If \f5\-\-update\-custom\-contexts\fR and
\f5\-\-remove\-custom\-contexts\fR are specified together, the
\f5\-\-remove\-custom\-contexts\fR would be applied first on object.


.RS 2m
.TP 2m
\fB\-\-remove\-custom\-contexts\fR=[\fICUSTOM_CONTEXTS_KEYS\fR,...]

Removes the custom contexts on the object, if an entry is not found, it would be
ignored.

.TP 2m
\fB\-\-update\-custom\-contexts\fR=[\fICUSTOM_CONTEXTS_KEYS_AND_VALUES\fR,...]

Updates the custom contexts on the object, if an entry is found, it would be
overwritten, otherwise the entry would be added.

.RE
.RE
.sp
.TP 2m

At most one of these can be specified:


.RS 2m
.TP 2m
\fB\-\-clear\-custom\-metadata\fR

Clears all custom metadata on objects. When used with \f5\-\-preserve\-posix\fR,
POSIX attributes will still be stored in custom metadata.

.TP 2m
\fB\-\-custom\-metadata\fR=[\fICUSTOM_METADATA_KEYS_AND_VALUES\fR,...]

Sets custom metadata on objects. When used with \f5\-\-preserve\-posix\fR, POSIX
attributes are also stored in custom metadata.

.TP 2m

Flags that preserve unspecified existing metadata cannot be used with
\f5\-\-custom\-metadata\fR or \f5\-\-clear\-custom\-metadata\fR, but can be
specified together:


.RS 2m
.TP 2m
\fB\-\-remove\-custom\-metadata\fR=[\fIMETADATA_KEYS\fR,...]

Removes individual custom metadata keys from objects. This flag can be used with
\f5\-\-update\-custom\-metadata\fR. When used with \f5\-\-preserve\-posix\fR,
POSIX attributes specified by this flag are not preserved.

.TP 2m
\fB\-\-update\-custom\-metadata\fR=[\fICUSTOM_METADATA_KEYS_AND_VALUES\fR,...]

Adds or sets individual custom metadata key value pairs on objects. Existing
custom metadata not specified with this flag is not changed. This flag can be
used with \f5\-\-remove\-custom\-metadata\fR. When keys overlap with those
provided by \f5\-\-preserve\-posix\fR, values specified by this flag are used.


.RE
.RE
.RE
.sp

.SH "PRECONDITION FLAGS"

.RS 2m
.TP 2m
\fB\-\-if\-generation\-match\fR=\fIGENERATION\fR

Execute only if the generation matches the generation of the requested object.

.TP 2m
\fB\-\-if\-metageneration\-match\fR=\fIMETAGENERATION\fR

Execute only if the metageneration matches the metageneration of the requested
object.


.RE
.sp

.SH "GCLOUD WIDE FLAGS"

These flags are available to all commands: \-\-access\-token\-file, \-\-account,
\-\-billing\-project, \-\-configuration, \-\-flags\-file, \-\-flatten,
\-\-format, \-\-help, \-\-impersonate\-service\-account, \-\-log\-http,
\-\-project, \-\-quiet, \-\-trace\-token, \-\-user\-output\-enabled,
\-\-verbosity.

Run \fB$ gcloud help\fR for details.



.SH "NOTES"

This command is currently in alpha and might change without notice. If this
command fails with API permission errors despite specifying the correct project,
you might be trying to access an API with an invitation\-only early access
allowlist. This variant is also available:

.RS 2m
$ gcloud storage rsync
.RE