bunny-lab/docs
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update Scripts/Powershell/rClone.md

Browse Source
This commit is contained in:
Nicole Rappe
2024-12-05 00:44:28 -07:00
parent 5b3ed40f60
commit 38af91e84f
1 changed files with 8 additions and 99 deletions
Download Patch File Download Diff File Expand all files Collapse all files

107
Scripts/Powershell/rClone.md
View File

@ -16,110 +16,19 @@ Perform bidirectional synchronization between two paths. Bisync provides a bidi
The following commands illustrate how to use bisync to synchronize a local folder and a remote folder (assumed to be Google Drive). The `--drive-skip-gdocs` flag simply does not sync Google Drive specific documents back to the local folder, such as `*.gsheet`, `*.gdoc`, etc. The `--resilient` flag means that if there are network interruptions, rclone will attempt to recover on its own automatically to resume where it last left off in the sync.
=== "Initial Sync"
You only run this command the first time you sync the local and remote locations. It builds a database between the two locations to effectively "index" every file locally and remotely, so that subsequent syncs can track deletions, additions, and modifications in both locations.
We want to first sync down any files that are from the remote location (Google Drive/Remote Folder/Network Share/etc) and overwrite any local files with the newer files. This ONLY overwrites local files that are older than the remote files, but if the local files are newer, they are left alone. `--update` = (Skip files that are newer on the destination) > This allows us to ensure that the newest changes on Google Drive are pulled down before performing a bisync.
```powershell
.\rclone.exe bisync "$locationA" "$locationB" --create-empty-src-dirs --compare size,modtime,checksum --resilient --log-level ERROR --drive-skip-gdocs --fix-case --resync
.\rclone.exe sync "Remote" "Local" --update --log-level INFO --drive-skip-gdocs --create-empty-src-dirs --progress
```
=== "Subsequent Syncs"
You run this command after the initial sync to check for changes, and push those changes to both locations.
At this point, the local directory has the newest remote version of all of the files that exist in both locations, so if anyone made changes to a file in Google Drive, and those changes are newer than the local files, it overwrites the local files, but if the local files were newer, they were left alone. This second command performs the first and all subsequent bisyncs, with conflict resolution, meaning:
- If the remote file was newer, it deletes the older local file and overwrites it with the newer remote file,
- If the local file was newer, it deletes the older remote file and overwrites it with the newer local file
```powershell
.\rclone.exe bisync "$locationA" "$locationB" --create-empty-src-dirs --compare size,modtime,checksum --resilient --log-level ERROR --drive-skip-gdocs --fix-case --force
.\rclone.exe bisync "Local" "Remote" --create-empty-src-dirs --conflict-resolve newer --conflict-loser delete --compare size,modtime,checksum --resilient --log-level ERROR --drive-skip-gdocs --fix-case --force --progress
```
??? example "Self-Testing Powershell Script"
You may not be 100% confident in the behavior of rclone, especially with something such as the `bisync` command in rclone. If that is the case, you can run (and modify to your liking) the following powershell script to validate it behaves as you expect it would.
``` powershell
# Change Directory to rClone Folder
CD "C:\Users\nicole.rappe.BUNNY-LAB\Desktop\rclone-v1.68.1-windows-amd64"
# Clear the console
clear
# Define paths for LOCATION_A and LOCATION_B
$locationA = "D:\RCLONE_TESTING_LAB\LOCATION_A"
$locationB = "D:\RCLONE_TESTING_LAB\LOCATION_B"
$canaryFileA = "$locationA\CANARY.TXT"
$canaryFileB = "$locationB\CANARY.TXT"
$dummyFileA = "$locationA\DUMMY.TXT"
$dummyFileB = "$locationB\DUMMY.TXT"
# Step 1: Add dummy files to LOCATION_A and LOCATION_B to avoid empty directory errors
Write-Host "Adding dummy files to LOCATION_A and LOCATION_B to avoid empty directory sync errors..."
"DUMMY_DATA" | Out-File -FilePath $dummyFileA -Encoding UTF8
"DUMMY_DATA" | Out-File -FilePath $dummyFileB -Encoding UTF8
# Step 2: Create CANARY file in LOCATION_A
Write-Host "Creating CANARY in LOCATION_A..."
"ORIGINAL_DATA" | Out-File -FilePath $canaryFileA -Encoding UTF8 # Write original data to canary file
# Step 3: Run the initial bisync to sync LOCATION_A and LOCATION_B
Write-Host "Bi-Syncing LOCATION_A and LOCATION_B..."
.\rclone.exe bisync "$locationA" "$locationB" --create-empty-src-dirs --compare size,modtime,checksum --resilient --log-level ERROR --drive-skip-gdocs --fix-case --resync
# Step 4: Check if the CANARY file exists in LOCATION_B
if (Test-Path $canaryFileB) {
Write-Host "CANARY file found in LOCATION_B."
} else {
Write-Host "ERROR: CANARY file not found in LOCATION_B."
exit 1 # Exit script if file is not found
}
# Step 5: Overwrite data in the CANARY file in LOCATION_B
Write-Host "Updating CANARY file in LOCATION_B with 'UPDATED_DATA'..."
"UPDATED_DATA" | Out-File -FilePath $canaryFileB -Encoding UTF8 # Write updated data to canary file
# Step 6: Run bisync again to sync changes from LOCATION_B back to LOCATION_A
Write-Host "Bi-Syncing LOCATION_B back to LOCATION_A..."
.\rclone.exe bisync "$locationA" "$locationB" --create-empty-src-dirs --compare size,modtime,checksum --resilient --log-level ERROR --drive-skip-gdocs --fix-case --force
# Step 7: Verify that UPDATED_DATA is now in the CANARY file in LOCATION_A
$canaryContentA = Get-Content -Path $canaryFileA -Encoding UTF8
if ($canaryContentA -eq "UPDATED_DATA") {
Write-Host "UPDATED_DATA confirmed in LOCATION_A's CANARY file."
} else {
Write-Host "ERROR: UPDATED_DATA not found in LOCATION_A's CANARY file."
exit 1 # Exit script if content is not as expected
}
# Step 8: Delete the CANARY file in LOCATION_A
Write-Host "Deleting CANARY file from LOCATION_A..."
Remove-Item -Path $canaryFileA -Force -ErrorAction SilentlyContinue
# Step 9: Run bisync again to propagate deletion to LOCATION_B
Write-Host "Bi-Syncing LOCATION_A and LOCATION_B to remove CANARY file from LOCATION_B..."
.\rclone.exe bisync "$locationA" "$locationB" --create-empty-src-dirs --compare size,modtime,checksum --resilient --log-level ERROR --drive-skip-gdocs --fix-case --force
# Step 10: Confirm CANARY file no longer exists in LOCATION_B
if (-not (Test-Path $canaryFileB)) {
Write-Host "CANARY file successfully deleted from LOCATION_B."
} else {
Write-Host "ERROR: CANARY file still exists in LOCATION_B."
exit 1 # Exit script if deletion did not propagate
}
# Step 11: Clean up dummy files in LOCATION_A and LOCATION_B
Write-Host "Cleaning up dummy files from LOCATION_A and LOCATION_B..."
Remove-Item -Path $dummyFileA -Force -ErrorAction SilentlyContinue
Remove-Item -Path $dummyFileB -Force -ErrorAction SilentlyContinue
Write-Host "All steps completed successfully!"
```
!!! example "Example Output"
```
Adding dummy files to LOCATION_A and LOCATION_B to avoid empty directory sync errors...
Creating CANARY in LOCATION_A...
Bi-Syncing LOCATION_A and LOCATION_B...
CANARY file found in LOCATION_B.
Updating CANARY file in LOCATION_B with 'UPDATED_DATA'...
Bi-Syncing LOCATION_B back to LOCATION_A...
UPDATED_DATA confirmed in LOCATION_A's CANARY file.
Deleting CANARY file from LOCATION_A...
Bi-Syncing LOCATION_A and LOCATION_B to remove CANARY file from LOCATION_B...
CANARY file successfully deleted from LOCATION_B.
Cleaning up dummy files from LOCATION_A and LOCATION_B...
All steps completed successfully!
```
!!! info
The `--conflict-resolve newer` flag is how the bisync determines how to declare a "winner" and a "loser", the winner being the newer file, and the loser being the older file. The `--conflict-loser delete` is the action to perform to the older file when a conflict is found in either direction.