Overview

Duplicati uses block-level deduplication to minimize storage use and upload bandwidth. Instead of saving whole files for each backup version, Duplicati breaks data into small blocks, identifies duplicates, and stores each unique block only once. This method ensures that even minor file changes reuse existing data, greatly improving efficiency.

The Block-Based Storage Format

When Duplicati processes a backup source, each file is divided into fixed-size or variable-size blocks (depending on configuration). Each block is hashed using a cryptographic hash (SHA-256 by default). These hashes act as unique fingerprints.

• New blocks are compressed and encrypted before upload.

• Duplicate blocks are referenced by their existing hash, avoiding re-upload.

These blocks are grouped into volumes for transfer to the backend storage, and a local database tracks mappings between files, blocks, and backup versions.

Deduplication Across Versions

Because Duplicati stores data by block rather than by file, deduplication automatically spans all backups:

• If a file is unchanged, its block hashes match existing entries.

• If a file changes partially, only modified blocks generate new uploads.

• If two different files share identical content segments, those blocks are stored once.

This block-level granularity allows incremental backups that are small, even when large files change slightly.

Metadata and Verification

Duplicati maintains a local database that indexes blocks, hashes, and file metadata. During verification, hashes ensure integrity. The same mechanism detects corruption or tampering in remote storage.

The combination of compression, encryption, and block-level deduplication results in secure and efficient storage compatible with a wide range of backend systems.

Advantages

• Reduced storage footprint through data reuse

• Lower upload bandwidth for repeated backups

• Faster incremental operations

• Consistent integrity checking via hashes

Summary

Duplicati’s block-level deduplication is the core of its storage efficiency. By storing data as small, hashed blocks, it avoids duplication between versions and across files while preserving security and integrity. This architecture allows users to keep long-term encrypted backups with minimal growth in storage size.

Want to try out Duplicati? Go to the download page!

More details

There are many more details on the forum. You can read more on the block-based backup deduplication, the backup process or how the restore process works.