Syncthing replaces Dropbox / GDrive

Updated 04-Sep-2024

• See also the local file sync tool FreeFileSync for handy local drive copying.

Google Drive (GDrive) and other cloud storage alternatives such as Dropbox and Microsoft OneDrive all have the serious drawback of keeping one's information in a third party cloud repository. Privacy and security are generally compromised this way, even when paying for storage (as opposed to having an advertising model, which is worse in many ways).

The challenge is to have an equally robust service that can effectively, and efficiently (regarding resource requirements) sychronize files across multiple devices. Remain on our own devices. And remain open source. In our case we have three different operating systems on four devices to support:

• Android
• ChromeOS (which runs Android apps)
• Debian Linux stable and testing

Options such as OwnCloud don't work because of the high overhead needed to get the services to run, in terms of memory and processing on a server.

Syncthing for File Synchronization

File synchronization is not backup, though with versioning there is a sort of backup-lite going on. Syncthing versioning doesn't really work as expected by the name, so that is really backup-very-lite.

Syncthing is available from repositories and directly from Github. There are ports and other configuration issues to enable for routing. There is also an Android app, so that is what will be used on Android and ChromeOS.

Install Syncthing on Debian stable

Note: for Debian testing there is no need to add a repository.

See:

Add PGP key

curl -s https://syncthing.net/release-key.txt | sudo apt-key add -

Add the "stable" channel to your APT sources:

echo "deb https://apt.syncthing.net/ syncthing stable" | sudo tee /etc/apt/sources.list.d/syncthing.list

Update repositories

sudo apt update

Create the syncthing user

adduser syncthing

Install syncthing

sudo apt install syncthing

Edit the .xml file

sudo nano ~/.config/syncthing/config.xml

Change the 127.0.0.1 address to 0.0.0.0 to enable access from anywhere

Allow Ports on Debian

Had to punch holes through networking best to tidily add rules using UFW

• https://11.22.33.44:8384/ (server)
• http://127.0.0.1:8384/ (workstation)

such as:

ufw allow syncthing
ufw allow syncthing-gui
ufw reload

Install Syncthing on Android

For Android and ChromeOS devices, install Syncthing for Android from the Play Store.

Note that the best approach is to share out something like /syncthing-photos as a one-way share, and root that to the photos directory (either the local device or MicroSD card), and then also create an inbound two-way sync for a subset of critical documents.

For devices with more storage, additional inbound two-way shares could be added to provide greater access to one's documents.

For example on an Android device with a MicroSD card, the photos are shared under /storage/CE6A-1402/DCIM/Camera. Note that the best approach is then a manual copy/paste of the images to a better archived directory structure that is then synced to a backup server, and a manual image deletion on the Android device, on a monthly basis.

For an inbound, two-way sync to the Android device for critical (editable) files, the directory is: /storage/emulated/0/Download. Note that this is the same directory that is used in ChromeOS for inbound, two-way shares.

Configure Syncthing to Turn on Automatically

For Debian/LXDE the simple approach is to add an item to the autostart menu

• Menu > Preferences > Default Applications for LXSession > Autostart

@/usr/bin/syncthing -no-browser

See also Starting Syncthing Automatically

Android Syncthing*

The Android syncthing app has a toggle to support auto-on functionality as needed.

Some Issues with Synchronizing

The main thing is to think out one's synchronization policies and plans. One-way synchronization, two-way sinchronization, master and slave device replication, etc. There are many options. Some files one will want to keep everywhere, with version control. Other files one will want only in one or two locations (large files/repositories).

The best approach is to partition into folders so that different folders contain different content that will be sychronized differently. Some examples:

• Images/Photos folder on a mobile device
  • Should be synchronized but also allow for repository of more images on a backup location.
  • Workflow: sync mobile folder to desktop. On desktop, move images to a second folder (removing them from mobile via synchronization), and then have the second folder synchronized to a server. That server folder can have SFTP for remote access and also provide two-way synchronization back to the desktop for things such as editing images that are on a web server.

It is important to have a manual workflow as well (or semi-automated) so that things are easier to manage.

Synchronization vs. SFTP

Synchronization is useful, but is not a replacement for SFTP which should be seen as on-demand push/pull. For example, a large repository can be synchronized between two larger-capacity devices (e.g, Debian server and Debian workstation), but also allow access via SFTP for smaller-capacity devices (ChromeOS/Android).

Syncthing Autostart

For a desktop, add to Startup Applications

/usr/bin/syncthing  -no-browser -home="/home/user/.config/syncthing"

For a server, create a service, or something like that. First enable and start a server service using the username. It will create the appropriate ~/.config/syncthing files. Note the @syncthing is the account name to be used.

systemctl enable [email protected]
systemctl start [email protected]

Syncthing troubleshooting

• If says can't find folder, then create a .stfolder folder in the root of the specific shared directory
• If get an inode error (especially on Android) turn off Watch for Changes in > WEB GUI > SHARE > Advanced

Syncthing Version Control (very-lite)

There are many different version control options for Syncthing, depending on what versioning scheme and how much storage will be set aside for it. Basically, however, this version of version control/backup is total crap since it only makes a backup on synching to another machine (e.g., if the source is overwritten by a downstream sync, then the source keeps a copy of the original before overwriting. This means if working on a single original source there is no version control/backup.

Basically, the most reasonable option for version control, in this situation of a severe lack, is probably an automated git versioning.

• Making git auto-commit
• Gitwatch

Value of Syncthing for the Enduser

Ultimately, Syncthing lets the enduser take full control over their data on their devices in terms of files that are synchronized with other devices. Along with SFTP on a server, and possibly something like AWS S3 and Glacier, it appears to provide a useful protocol, gui admin console, and applications that can do everything that GDrive/Dropbox/OneDrive offer in terms of synchronization. Since disk space is already something that can be managed at the level of S3/Glacier and local devices, it provide a key element in a resource-efficient, open source package.

Real World Replacement of Dropbox

I've got syncthing running on four devices with the main desktop (Debian testing) as the primary repository. Nearly all of the reposed data is syncronized to a Debian stable server running on AWS lightsail. This is a web server with 80gb of storage ($20/month), and my data consists of approximately 90,000 files, 10,000 directories, and 40gb of storage. The third device is my Android phone which syncs out photos, and inbound syncs a lightweight directory of editable files, which allow for reading/editing those files on-the-go. The fourth device is a Chromebook convertible which has the least storage of all devices, but nevertheless is able to handily inbound sync about 5gb of data and allows for most of the working directories on active projects.