nicole
79132d8012
24 KiB
tags
| tags | ||||
|---|---|---|---|---|
|
Purpose
This document explains the practical differences between the rclone copy, sync, check, and bisync commands and provides a safety-first workflow for configuring and recovering a bidirectional synchronization pair. The examples use PowerShell and assume that one side of the synchronization is a Google Drive remote.
Rclone is a command-line file-management program that supports cloud storage providers, object-storage platforms, local filesystems, and standard transfer protocols.
!!! info "Version Context"
This document was reviewed against rclone v1.75.0. Older releases may not support every bisync flag documented here, and future versions may change some recovery behavior.
!!! danger "Rclone Can Delete or Overwrite Data"
The sync and bisync commands can delete, replace, rename, or propagate the deletion of files. Always confirm the source and destination paths, maintain a separate backup or snapshot, and preview unfamiliar operations with --dry-run before allowing them to modify data.
A dry run is a preview and is not a replacement for a backup.
Command Behavior
The correct command depends on the intended relationship between the source and destination.
| Command | Primary Behavior | Deletes Destination-Only Files | Direction |
|---|---|---|---|
copy |
Adds or updates source files at the destination while retaining unrelated destination files | No | One-way |
sync |
Makes the destination match the source | Yes | One-way |
check |
Compares files without modifying either side | No | Read-only |
bisync |
Detects and propagates changes made on either side by comparing the current state against prior listings | Yes | Two-way |
Use copy for Additive Transfers
Use copy when you need to add or update files without deleting files that already exist only at the destination.
& $Rclone copy "Source" "Destination" --update --dry-run --verbose
The --update flag skips a source file when the corresponding destination file has a newer modification time. It does not create version history, and it does not prevent an older destination file from being replaced by a newer source file.
After reviewing the dry-run output, repeat the operation without --dry-run:
& $Rclone copy "Source" "Destination" --update --verbose
Use sync Only for Intentional Mirroring
Use sync when the destination must become a one-way mirror of the source.
!!! danger "sync Deletes Destination-Only Files"
The sync command removes files from the destination when they do not exist in the source. This is true even when --update is present.
The `--update` flag only prevents a newer destination file from being replaced by an older source file. It does not convert `sync` into an additive operation and does not protect destination-only files from deletion.
Preview the operation first:
& $Rclone sync "Source" "Destination" --dry-run --verbose
Only remove --dry-run after confirming that every proposed copy, replacement, and deletion is intentional:
& $Rclone sync "Source" "Destination" --verbose
Use check for Read-Only Comparison
The check command compares files on both sides without copying, replacing, or deleting them.
& $Rclone check "Source" "Destination" --combined "rclone-comparison.txt" --log-level INFO --log-file "rclone-check.log"
The combined report uses the following symbols:
| Symbol | Meaning |
|---|---|
= |
The file exists on both sides and matches |
+ |
The file exists only in the source |
- |
The file exists only in the destination |
* |
The same path exists on both sides, but the files differ |
! |
The file could not be read or compared |
The check command compares files but does not report missing empty directories.
Understand Bisync State
Bisync is a stateful two-way synchronization command. It retains listings of Path1 and Path2 from the prior successful run and compares those listings against the current state during the next run.
Bisync does not maintain a file-version archive or historical copies of every changed file. The listings record synchronization state and metadata needed to determine whether a file is new, changed, or deleted relative to the previous run.
!!! warning "Protect the Bisync Work Directory"
Use a stable, explicit --workdir for every run of a given bisync pair. Do not change the work directory, reverse the order of Path1 and Path2, delete the listing files, or run the same pair with different filters without understanding that the existing state may no longer be valid.
On Windows, bisync otherwise stores its state beneath the profile of the account running rclone. This can cause scheduled and interactive executions to use different state directories when they run as different accounts.
Define the Example Environment
Replace the example values before running the commands in this document.
$Rclone = "C:\Path\To\rclone.exe"
$LocalPath = "C:\Path\To\Local"
$RemotePath = "GoogleDrive:Path/To/Remote"
$WorkDir = "C:\Path\To\BisyncState"
$LogDir = "C:\Path\To\Logs"
$FilterFile = "C:\Path\To\bisync-filters.txt"
Create the state and log directories:
New-Item -Path $WorkDir,$LogDir -ItemType Directory -Force | Out-Null
Create a consistent filters file:
- *.lnk
The *.lnk rule excludes Windows shortcut files at any depth beneath the synchronization root.
!!! warning "Filter Changes Require a Reviewed Rebaseline" Bisync compares current listings against prior listings. Adding, removing, or changing a filter can make previously tracked files disappear from the new listings and appear to have been deleted.
Treat filter changes as a state-changing event. Stop scheduled runs, review the new scope, and perform a dry-run resync before committing the change.
Validate the Paths
Confirm the installed rclone version:
& $Rclone version
Confirm that the local root already exists:
Test-Path -LiteralPath $LocalPath
Expected result:
True
Confirm that the configured remote path is reachable:
& $Rclone lsd $RemotePath
!!! warning "Do Not Automatically Create an Unexpected Root" Stop if either path is missing or points to an unexpected location. Automatically creating an empty root can conceal a path, authentication, drive-mount, or configuration failure and can cause bisync to interpret an entire data set as deleted.
Configure the Access Check
The --check-access flag verifies that matching files named RCLONE_TEST are visible in the same relative locations on both sides. This provides additional protection against an unavailable mount, incorrect remote root, or incomplete listing being interpreted as mass deletion.
Create the access-check file at the root of the local path and copy it to the root of the remote path:
$AccessFile = Join-Path $LocalPath "RCLONE_TEST"
New-Item -Path $AccessFile -ItemType File -Force | Out-Null
& $Rclone copyto $AccessFile "$RemotePath/RCLONE_TEST"
Confirm that the file appears on both sides:
& $Rclone lsf $LocalPath --include "RCLONE_TEST"
& $Rclone lsf $RemotePath --include "RCLONE_TEST"
Expected result from both commands:
RCLONE_TEST
!!! warning "Do Not Delete the Access-Check File"
The RCLONE_TEST file must remain visible on both sides while --check-access is enabled. Bisync will abort when the file cannot be found.
Audit the Existing Data
Before establishing or rebuilding the bisync state, compare the current file sets without changing either side:
& $Rclone check $LocalPath $RemotePath --filter-from $FilterFile --drive-skip-gdocs --combined (Join-Path $LogDir "pre-bisync-check.txt") --log-level INFO --log-file (Join-Path $LogDir "pre-bisync-check.log")
Review all source-only, destination-only, differing, and unreadable files before proceeding. Determine whether the expected recovery is to merge both sides, prefer one authoritative side, or preserve selected files manually.
Initialize a Bisync Pair
A new bisync pair requires an initial resync to establish its Path1 and Path2 listings. A preliminary sync --update operation is not required and can unnecessarily delete destination-only files.
!!! danger "Resync Is Not a Database-Only Repair" A resync reconciles the actual contents of both paths while rebuilding the listings.
Files that exist on only one side are copied to the other side. When the same relative path contains different files on both sides, `--resync-mode` selects one version as the winner and overwrites the other version.
The conflict flags used during normal bisync runs do not apply during a resync. A resync does not rename the losing version into a conflict copy.
Select the Resync Policy
Choose the resync policy according to which data is authoritative:
| Mode | Use When |
|---|---|
path1 |
Path1 is authoritative and must win every same-path difference |
path2 |
Path2 is authoritative and must win every same-path difference |
newer |
Modification times are trustworthy and the newest same-path version should win |
larger |
File size is a more reliable winner than modification time |
A bare --resync is equivalent to --resync-mode path1. Do not use a bare --resync unless Path1 is intentionally authoritative.
The examples below use --resync-mode newer. This is appropriate only when both sides provide trustworthy modification times and the intended policy is to preserve the newer same-path file.
Preview the Initial Resync
Run the initial resync as a dry run:
& $Rclone bisync $LocalPath $RemotePath --workdir $WorkDir --resync-mode newer --create-empty-src-dirs --compare size,modtime,checksum --check-access --max-delete 0 --filters-file $FilterFile --drive-skip-gdocs --fix-case --dry-run --verbose --log-file (Join-Path $LogDir "bisync-resync-dry-run.log")
Review the complete log for:
- Files copied from Path1 to Path2
- Files copied from Path2 to Path1
- Same-path files that would be replaced
- Unexpected paths
- Access or checksum errors
- Proposed deletion behavior
!!! note "Dry-Run Deletion Messages" A bisync dry run may display confusing deletion messages because the simulated copy that would normally precede the deletion did not actually occur. Review the complete sequence rather than evaluating an isolated deletion line.
Do not dismiss an unexpected deletion unless the log clearly shows that it is a dry-run artifact associated with a preceding simulated copy.
The --max-delete 0 setting blocks ordinary file deletions during this recovery run. It does not prevent a same-path losing version from being overwritten during resync, so a separate backup or snapshot remains required.
Perform the Initial Resync
After reviewing and approving the dry-run output, repeat the command without --dry-run:
& $Rclone bisync $LocalPath $RemotePath --workdir $WorkDir --resync-mode newer --create-empty-src-dirs --compare size,modtime,checksum --check-access --max-delete 0 --filters-file $FilterFile --drive-skip-gdocs --fix-case --verbose --log-file (Join-Path $LogDir "bisync-resync-live.log")
A successful run should end with:
Bisync successful
Validate the Reconciled Data
Compare both sides after the live resync:
& $Rclone check $LocalPath $RemotePath --filter-from $FilterFile --drive-skip-gdocs --combined (Join-Path $LogDir "post-resync-check.txt") --log-level INFO --log-file (Join-Path $LogDir "post-resync-check.log")
The expected result is:
0 differences found
Run Normal Bisync Operations
After the baseline has been established, all normal runs must omit --resync and --resync-mode.
Preview the first normal run:
& $Rclone bisync $LocalPath $RemotePath --workdir $WorkDir --create-empty-src-dirs --conflict-resolve newer --conflict-loser num --compare size,modtime,checksum --resilient --recover --max-lock 2m --check-access --max-delete 10 --filters-file $FilterFile --drive-skip-gdocs --fix-case --dry-run --log-level INFO --log-file (Join-Path $LogDir "bisync-normal-dry-run.log")
A healthy unchanged pair should report:
No changes found
Updating listings
Bisync successful
After reviewing the dry run, perform one controlled live run:
& $Rclone bisync $LocalPath $RemotePath --workdir $WorkDir --create-empty-src-dirs --conflict-resolve newer --conflict-loser num --compare size,modtime,checksum --resilient --recover --max-lock 2m --check-access --max-delete 10 --filters-file $FilterFile --drive-skip-gdocs --fix-case --log-level INFO --log-file (Join-Path $LogDir "bisync.log")
This normal command can be placed into the scheduled automation after it has completed successfully under the same Windows account and execution context that will run the scheduled task.
Understand the Normal Bisync Flags
| Flag | Behavior |
|---|---|
--workdir |
Stores the prior Path1 and Path2 listings in an explicit, stable directory |
--create-empty-src-dirs |
Synchronizes the creation and deletion of empty directories |
--compare size,modtime,checksum |
Uses size, modification time, and checksums when determining file state |
--conflict-resolve newer |
Prefers the newer file when the same file changed independently on both sides since the prior run |
--conflict-loser num |
Preserves the losing conflict version under a numbered .conflictN name |
--resilient |
Allows certain less-serious errors to be retried during future runs rather than immediately requiring resync |
--recover |
Uses a backup listing to recover from some interrupted or ungracefully terminated runs |
--max-lock 2m |
Allows a stale bisync lock to expire after two minutes |
--check-access |
Aborts when matching RCLONE_TEST files cannot be found on both sides |
--max-delete 10 |
Aborts when more than 10 percent of tracked files appear deleted on either side |
--filters-file |
Applies one consistent set of exclusions to both bisync listings |
--drive-skip-gdocs |
Makes native Google Docs, Sheets, Slides, and other Google-native documents invisible to rclone |
--fix-case |
Allows supported case-only filename corrections between filesystems |
--log-level INFO |
Records normal operations, changes, warnings, and errors rather than errors alone |
Understand Conflict Resolution
A bisync conflict occurs when the same relative file is new or changed on both sides compared with the prior successful run and the current file contents are not identical.
The --conflict-resolve newer option does not mean that rclone blindly compares every file and always keeps whichever modification time is newest. It applies specifically to a detected two-sided conflict.
With --conflict-loser num, the winner retains the original filename and the losing version is preserved with a numbered conflict suffix, such as:
Document.docx
Document.docx.conflict1
This is safer than --conflict-loser delete, which permanently removes the losing conflict version.
Understand the Deletion Limit
The --max-delete 10 value is a conservative example rather than a universal requirement. Select a threshold appropriate for the size of the data set and the organization's normal deletion patterns.
A large folder rename may appear as many deletions and many new files and can exceed the configured threshold.
!!! danger "Do Not Use --force as a Recurring Option"
The --force flag bypasses the --max-delete safety check. Do not include it in a normal scheduled command.
When a legitimate change exceeds the threshold, stop the scheduled operation, review the proposed changes with `--dry-run`, and use a one-time explicitly approved threshold instead of permanently disabling the protection.
Understand Google-Native Documents
The --drive-skip-gdocs flag removes native Google documents from rclone listings. These objects are not downloaded, exported, compared, or synchronized while the flag is enabled.
This flag does not merely ignore local files with extensions such as .gdoc or .gsheet. It makes the corresponding native Google Drive objects effectively invisible to rclone.
Remove this flag only when Google-native documents must be exported and synchronized and the desired export formats have been deliberately configured. Changing this behavior on an established bisync pair requires a reviewed rebaseline because it changes which objects appear in the listings.
Repair a Broken Bisync Pair
A request for --resync does not automatically mean that the data is damaged. Bisync may be unable to locate or trust its prior state because the work directory changed, the command used different paths or filters, the execution account changed, or a prior run ended with a critical error.
Stop Automated Runs
Stop all scheduled or looping bisync processes before investigating or repairing the state. Confirm that another rclone process is not modifying either path or the bisync listings.
Review the Logs
Search the most recent log for the first critical error rather than relying only on the final resync message.
Get-Content (Join-Path $LogDir "bisync.log") -Tail 300
Look for:
- Authentication or remote-access errors
- Missing-path errors
- Missing or invalid listing files
- Access-check failures
- Excessive deletion warnings
- File-copy or file-move failures
- Lock-file errors
- Filter changes
Bisync abortedBisync critical error
Verify the Existing Configuration
Confirm that the repair command uses:
- The original Path1 and Path2 in the original order
- The original
--workdir - The original filter rules
- The intended rclone configuration file
- The correct remote account and storage location
- The same Google Docs behavior
- The same comparison settings
List the current bisync state directory:
Get-ChildItem -LiteralPath $WorkDir
The directory should contain Path1 and Path2 .lst files for the configured pair.
Test Remote and Local Access
Confirm that both roots are present and readable:
Test-Path -LiteralPath $LocalPath
& $Rclone lsd $RemotePath
Confirm that the access-check file remains visible:
& $Rclone lsf $LocalPath --include "RCLONE_TEST"
& $Rclone lsf $RemotePath --include "RCLONE_TEST"
Compare the Current Data
Run a read-only comparison before deciding to resync:
& $Rclone check $LocalPath $RemotePath --filter-from $FilterFile --drive-skip-gdocs --combined (Join-Path $LogDir "repair-comparison.txt") --log-level INFO --log-file (Join-Path $LogDir "repair-check.log")
Review whether one-sided files are expected, whether one side is authoritative, and whether same-path differences should be resolved manually.
Attempt Normal Recovery When Appropriate
When the prior run was only interrupted and the normal command already uses --recover, a normal controlled run may recover without requiring a resync.
Run it first as a dry run:
& $Rclone bisync $LocalPath $RemotePath --workdir $WorkDir --create-empty-src-dirs --conflict-resolve newer --conflict-loser num --compare size,modtime,checksum --resilient --recover --max-lock 2m --check-access --max-delete 10 --filters-file $FilterFile --drive-skip-gdocs --fix-case --dry-run --log-level INFO --log-file (Join-Path $LogDir "bisync-recovery-dry-run.log")
Proceed with a live normal run only when the output is understood and expected.
Rebuild the State Only When Required
Use a resync only when:
- The pair has never been initialized
- The listings are missing or cannot be trusted
- A critical bisync error explicitly requires a resync
- The path scope or filter rules intentionally changed
- You are deliberately establishing a new authoritative baseline
Select the correct --resync-mode, perform a dry run, review every proposed change, and then follow the initialization and validation sequence documented above.
!!! danger "Do Not Describe Resync as Non-Destructive" Resync can overwrite a same-path file and can restore files that were intentionally deleted by copying one-sided files back to the other side.
The `--conflict-resolve` and `--conflict-loser` flags are ignored during resync. Maintain an independent backup and do not proceed until the chosen resync policy is understood.
Validate Bidirectional Synchronization
After the initial setup or a repair, validate both directions with disposable files.
- Create a test text file beneath the local root.
- Run a normal bisync and confirm that the file appears remotely.
- Create a second test text file beneath the remote root.
- Run another normal bisync and confirm that the file appears locally.
- Delete both test files.
- Run another normal bisync and confirm that the deletions propagate as intended.
- Confirm that the final run ends with
Bisync successful. - Run
rclone checkand confirm that no file differences remain.
Remove all test artifacts when validation is complete.
Troubleshooting
Google Drive Reports Shared Drive Not Found
An error such as 404: Shared Drive not found normally means that the authenticated Google account cannot access the configured Shared Drive ID.
Verify:
- The correct Google account completed OAuth
- The account still has access to the Shared Drive
- The configured Shared Drive ID is correct
- The Shared Drive was not deleted and recreated under a new ID
Do not replace the drive ID with another visible Shared Drive merely because authentication succeeded.
Bisync Cannot Find Its Listings
Confirm that the command uses the original --workdir and that the scheduled task runs under the expected account.
Do not perform an immediate resync when the actual problem is that rclone is looking in the wrong state directory.
The Deletion Limit Was Exceeded
Stop the operation and determine why so many files appear deleted. Common causes include:
- An unavailable local mount
- An inaccessible remote path
- An incorrect synchronization root
- A large folder rename
- Changed filter rules
- An actual mass deletion
Do not use --force until the reported deletions have been independently reviewed and approved.
Conflict Files Are Appearing
Files ending in .conflict1, .conflict2, or another numbered suffix indicate that both sides changed independently and bisync preserved the losing version.
Review the contents, retain the correct version, and remove the obsolete conflict copy after confirming that it is no longer needed.
Google Docs Are Missing
Native Google Docs are intentionally absent when --drive-skip-gdocs is enabled. Remove or change this behavior only as a deliberate configuration change with a reviewed resync.
Dry Run Appears to Delete a Newly Copied File
Review the complete dry-run sequence. Bisync can display an apparent deletion because the preceding simulated copy did not actually create the file on the other side.
Do not ignore unrelated or unexplained deletion messages.
Google Drive Reports Duplicate Objects
Google Drive can contain multiple objects with the same name in one folder. List duplicate names with:
& $Rclone dedupe list $RemotePath
Use the interactive resolver only after reviewing the file sizes, modification times, and hashes:
& $Rclone dedupe interactive $RemotePath
When the correct version cannot be determined confidently, rename and preserve both objects rather than deleting one automatically.