It will skip files that already exist in the destination. Rclone's copy command creates copies of files between a source and a destination (e.g., from CARC systems to your Google Drive storage). To delete a file in the remote ProjectDocs folder: rclone deletefile "google-drive:ProjectDocs/test.txt" Copying files To create a new subfolder in ProjectDocs: rclone mkdir "google-drive:ProjectDocs/Test" Rclone can be used to list, create, or delete files on the remote connection, similar to using commands to view local files.įor example, to view the files in a ProjectDocs folder on your "google-drive" remote: rclone ls "google-drive:ProjectDocs" For example: rclone ls "google-drive:" Navigating remote cloud storage For example, using rsync: rsync ~/.config/rclone/nf copied to the correct location, you can then test the connection from CARC systems by listing files in the remote. Note the file location and then copy the file to your home directory on CARC systems. Keep this "google-drive" remote?: Enter "y" if all the information was entered correctly.Įnter "q" to exit the main rclone config menu.įind the config file location by running the following command: rclone config file You should then see a configuration complete message and a copy of the configuration settings. Log in to your Google Account if needed and authorize the rclone app for access.Ĭonfigure this as a Shared Drive (Team Drive)?: Enter "n" if it is a personal drive. Your browser should then open a new page. Service_account_file: Leave this blank by pressing "Enter" to use interactive login. Scope: Choose "1" to allow access to all files. It is recommended to create your own client ID instead. Rclone's default client ID is shared by all users of rclone, possibly resulting in slow performance. Leave both prompts blank by pressing "Enter" to accept the default values. Look for Google Drive and enter the associated number.Ĭlient_id and client_secret: The next prompts are Google Application Client ID client_id and Google Application Client Secret client_secret. Storage: After choosing a name, the next prompt should be a long numbered menu with different storage solutions. Name: Name the remote something informative, e.g., "google-drive". Enter "n" to create a new remote and follow the prompts. Once installed, enter rclone config to configure a new remote. See instructions on the Rclone downloads page. See the official docs here.įirst install Rclone on your local computer. This process requires installing Rclone on your local computer, configuring the remote on your local computer, and then copying the resulting configuration file to your CARC home directory. The following example shows how to set up a Google Drive remote connection. See more information for each provider by consulting the official Rclone docs. There may be a different set of steps to follow depending on the storage provider. It will then prompt you to choose some configuration settings and then a method to grant permission for rclone to access your cloud storage account. Rclone will then prompt you one-by-one to enter details about the new remote like the following: You should see the following three options in your terminal: No remotes found - make a new one n) New remote s) Set configuration password q) Quit config n/s/q>Įnter "n" to continue with a new remote setup. To begin the process of creating a new remote connection, use the command rclone config. Usage of each cloud service requires the setup of a remote connection, or "remote", to that service. A list of current commands can be found here. To use rclone, first load the corresponding module: module load gcc/11.3.0 rcloneĪfter loading the rclone module, all rclone commands will become accessible. On CARC clusters, rclone is available as a software module. You can find instructions for this in the Getting Started with Discovery or Getting Started with Endeavour user guides. Loading the rclone moduleīegin by logging in. Alternative backup programs like Borg, Kopia, or Restic may be better solutions in those cases. Please note that rclone transfers may be slow for directories with large numbers of files. This is useful for creating synced backups, for example. Rclone allows users to copy or sync files from local storage to a cloud storage provider like Google Drive, Dropbox, OneDrive, etc (or vice versa). Rclone is an open-source, command-line utility for managing files in cloud storage.
0 Comments
Leave a Reply. |