Rclone is a command-line program to manage files on cloud storage. It is a feature-rich alternative to cloud vendors' web storage interfaces. Over 70 cloud storage products support rclone including S3 object stores, business & consumer file storage services, as well as standard transfer protocols. !!! warning "Be Mindful of Sync Type" The `bisync` command is meant to keep multiple locations synced with eachother, while in contrast the `sync` command forces the source to overwrite the destination. If you just want to dump the source into the destination on top of existing data, use the `copy` command within rclone. ## Usage Overview There is a lot to keep in mind when using rclone, primarily with the `sync` command. You can find more information in the [Official Documentation](https://rclone.org/commands/) ## rClone `bisync` Implementation Perform bidirectional synchronization between two paths. Bisync provides a bidirectional cloud sync solution in rclone. It retains the `locationA` and `locationB` filesystem listings from the prior run. On each successive run it will: - List files on `locationA` and `locationB`, and check for changes on each side. Changes include `New`, `Newer`, `Older`, and `Deleted files`. - Propagate changes on Path1 to Path2, and vice-versa. ### Example Usage The following commands illustrate how to use bisync to synchronize a local folder and a remote folder (assumed to be Google Drive). The `--drive-skip-gdocs` flag simply does not sync Google Drive specific documents back to the local folder, such as `*.gsheet`, `*.gdoc`, etc. The `--resilient` flag means that if there are network interruptions, rclone will attempt to recover on its own automatically to resume where it last left off in the sync. === "Initial Sync" You only run this command the first time you sync the local and remote locations. It builds a database between the two locations to effectively "index" every file locally and remotely, so that subsequent syncs can track deletions, additions, and modifications in both locations. ```powershell .\rclone.exe bisync "$locationA" "$locationB" --create-empty-src-dirs --compare size,modtime,checksum --resilient --log-level ERROR --drive-skip-gdocs --fix-case --resync ``` === "Subsequent Syncs" You run this command after the initial sync to check for changes, and push those changes to both locations. ```powershell .\rclone.exe bisync "$locationA" "$locationB" --create-empty-src-dirs --compare size,modtime,checksum --resilient --log-level ERROR --drive-skip-gdocs --fix-case --force ``` ??? example "Self-Testing Powershell Script" You may not be 100% confident in the behavior of rclone, especially with something such as the `bisync` command in rclone. If that is the case, you can run (and modify to your liking) the following powershell script to validate it behaves as you expect it would. ``` powershell # Change Directory to rClone Folder CD "C:\Users\nicole.rappe.BUNNY-LAB\Desktop\rclone-v1.68.1-windows-amd64" # Clear the console clear # Define paths for LOCATION_A and LOCATION_B $locationA = "D:\RCLONE_TESTING_LAB\LOCATION_A" $locationB = "D:\RCLONE_TESTING_LAB\LOCATION_B" $canaryFileA = "$locationA\CANARY.TXT" $canaryFileB = "$locationB\CANARY.TXT" $dummyFileA = "$locationA\DUMMY.TXT" $dummyFileB = "$locationB\DUMMY.TXT" # Step 1: Add dummy files to LOCATION_A and LOCATION_B to avoid empty directory errors Write-Host "Adding dummy files to LOCATION_A and LOCATION_B to avoid empty directory sync errors..." "DUMMY_DATA" | Out-File -FilePath $dummyFileA -Encoding UTF8 "DUMMY_DATA" | Out-File -FilePath $dummyFileB -Encoding UTF8 # Step 2: Create CANARY file in LOCATION_A Write-Host "Creating CANARY in LOCATION_A..." "ORIGINAL_DATA" | Out-File -FilePath $canaryFileA -Encoding UTF8 # Write original data to canary file # Step 3: Run the initial bisync to sync LOCATION_A and LOCATION_B Write-Host "Bi-Syncing LOCATION_A and LOCATION_B..." .\rclone.exe bisync "$locationA" "$locationB" --create-empty-src-dirs --compare size,modtime,checksum --resilient --log-level ERROR --drive-skip-gdocs --fix-case --resync # Step 4: Check if the CANARY file exists in LOCATION_B if (Test-Path $canaryFileB) { Write-Host "CANARY file found in LOCATION_B." } else { Write-Host "ERROR: CANARY file not found in LOCATION_B." exit 1 # Exit script if file is not found } # Step 5: Overwrite data in the CANARY file in LOCATION_B Write-Host "Updating CANARY file in LOCATION_B with 'UPDATED_DATA'..." "UPDATED_DATA" | Out-File -FilePath $canaryFileB -Encoding UTF8 # Write updated data to canary file # Step 6: Run bisync again to sync changes from LOCATION_B back to LOCATION_A Write-Host "Bi-Syncing LOCATION_B back to LOCATION_A..." .\rclone.exe bisync "$locationA" "$locationB" --create-empty-src-dirs --compare size,modtime,checksum --resilient --log-level ERROR --drive-skip-gdocs --fix-case --force # Step 7: Verify that UPDATED_DATA is now in the CANARY file in LOCATION_A $canaryContentA = Get-Content -Path $canaryFileA -Encoding UTF8 if ($canaryContentA -eq "UPDATED_DATA") { Write-Host "UPDATED_DATA confirmed in LOCATION_A's CANARY file." } else { Write-Host "ERROR: UPDATED_DATA not found in LOCATION_A's CANARY file." exit 1 # Exit script if content is not as expected } # Step 8: Delete the CANARY file in LOCATION_A Write-Host "Deleting CANARY file from LOCATION_A..." Remove-Item -Path $canaryFileA -Force -ErrorAction SilentlyContinue # Step 9: Run bisync again to propagate deletion to LOCATION_B Write-Host "Bi-Syncing LOCATION_A and LOCATION_B to remove CANARY file from LOCATION_B..." .\rclone.exe bisync "$locationA" "$locationB" --create-empty-src-dirs --compare size,modtime,checksum --resilient --log-level ERROR --drive-skip-gdocs --fix-case --force # Step 10: Confirm CANARY file no longer exists in LOCATION_B if (-not (Test-Path $canaryFileB)) { Write-Host "CANARY file successfully deleted from LOCATION_B." } else { Write-Host "ERROR: CANARY file still exists in LOCATION_B." exit 1 # Exit script if deletion did not propagate } # Step 11: Clean up dummy files in LOCATION_A and LOCATION_B Write-Host "Cleaning up dummy files from LOCATION_A and LOCATION_B..." Remove-Item -Path $dummyFileA -Force -ErrorAction SilentlyContinue Remove-Item -Path $dummyFileB -Force -ErrorAction SilentlyContinue Write-Host "All steps completed successfully!" ``` !!! example "Example Output" ``` Adding dummy files to LOCATION_A and LOCATION_B to avoid empty directory sync errors... Creating CANARY in LOCATION_A... Bi-Syncing LOCATION_A and LOCATION_B... CANARY file found in LOCATION_B. Updating CANARY file in LOCATION_B with 'UPDATED_DATA'... Bi-Syncing LOCATION_B back to LOCATION_A... UPDATED_DATA confirmed in LOCATION_A's CANARY file. Deleting CANARY file from LOCATION_A... Bi-Syncing LOCATION_A and LOCATION_B to remove CANARY file from LOCATION_B... CANARY file successfully deleted from LOCATION_B. Cleaning up dummy files from LOCATION_A and LOCATION_B... All steps completed successfully! ```