Performance tuning for slow PBS backups

Sometimes backups take longer than you expect. First incremental after a full backup took 3 hours? Restore of a 50 GB VM dragged on for 4 hours? Here are the typical bottlenecks and what to do about them.

Bottleneck 1 — your PVE’s internet connection

The most common factor. Symptoms:

• Backup sits on INFO: read X% (...) ... at the same value for a long time
• Restore crawls

Self-test: measure your connection’s upload — that’s what limits backup speed, not the download. On the PVE host you can use speedtest-cli (package of the same name), or a browser speed test from a machine on the same network.

For perspective: at 50 Mbit/s upload, ~6 MB/s go through the line — a 100 GB full backup takes around 4.5 to 5 hours at that rate. In practice usually longer, because of TCP overhead, encryption overhead etc.

What helps:

• Upgrade the connection (fiber, business DSL).
• Extend the backup window (run it at night).
• Enable a bandwidth limit so the backup doesn’t kill other services during the day.
• Back up only the important VMs (instead of everything).

Bottleneck 2 — compression CPU on the PVE

PBS compresses data while sending (zstd by default). If your PVE is weak (Atom CPU, older Xeon), the CPU can be the bottleneck.

Test: open htop in an SSH session during the backup and check whether proxmox-backup-client pushes the CPU to 100%.

What helps:

• Switch compression to lzo (faster, less compression) or gzip at a low level.
• Reduce the CPU quota of other VMs during the backup.
• Run several backups in parallel (CPU per backup drops, but total throughput rises — until I/O becomes the bottleneck).

In the GUI: Datacenter → Backup → Job → Edit → Compression.

Bottleneck 3 — local disk performance on the PVE

PBS has to read the data from the PVE storage, hash it, compress it, send it. If your local storage is slow (old HDD, overloaded RAID), even 10 Gbit is useless.

Test: iostat -x 5 during the backup — check %util of the backup source disk.

What helps:

• Disk defragmentation (in VMs too, depending on the filesystem).
• Storage migration to NVMe/SSD.
• Spread VMs across less loaded storage (sometimes a backup stalls because another VM is writing hard at that moment).

Bottleneck 4 — verify or GC running in parallel

If a large verify job or a garbage collection is running on our PBS, the datastore’s disk I/O is under load — backups write more slowly.

On our side, verify and GC run at night between 3 and 6 a.m. If your backups are slow during that window, this could be the cause.

What helps:

• Set your backup schedule to 22:00 or 02:00, before our verify window.
• For repeated performance problems: open a ticket with task IDs and we’ll look at our datastore load.

Bottleneck 5 — incremental mode isn’t working

If a backup that should be incremental suddenly transfers everything again, that points to:

• The local VM disk image was recreated (e.g. after a disk migration)
• The snapshot index was discarded
• VM snapshots were deleted/created, which breaks the dirty-bitmap state

Test: search the backup log for using fast-incremental. If it’s missing, it’s a full backup.

What helps:

• Don’t touch VM snapshots manually before the backup.
• After migrating to new storage: accept one full backup, after that incremental works again.

Bottleneck 6 — TLS overhead with many small files

For VMs with millions of small files and correspondingly many small chunks, TLS handshake overhead can become noticeable.

What helps:

• Build on a smaller number of well-packed disk images (instead of 100 small ones).
• Measure the theoretical performance with proxmox-backup-client benchmark — if that’s well below your backup throughput, the network isn’t the problem.

Restore performance specifically

Restore is often slower than backup, because:

• PBS has to collect chunks from the deduplicated pool — if a VM shares many “foreign” chunks, that’s random I/O.
• Your PVE storage has to write data (often slower than reading).

What helps:

• Enable live restore for large VMs — the VM starts immediately, the data transfer runs in the background.
• Move the restore to quieter times.
• For truly critical restores: open a ticket, we can pre-warm caches on the PBS side.

PBS server-side performance

We run PBS on hardware built for backup workloads:

• Supermicro servers with large SAS HDDs behind hardware RAID controllers as RAID 10
• RAID 10 delivers high I/O for backup and restore and survives individual disk failures
• At least 64 GB RAM, often more
• 10 Gbit connectivity per datastore server

If you feel the performance isn’t right — open a ticket with:

• Task ID of the slow backup
• Expected vs. measured transfer rate
• Time when the problem occurred

We’ll check load, disk health stats and storage latencies on the server side.

Frequently asked questions

My first backup is very slow, all the following ones are fast — normal? Yes. The first backup is full, all subsequent ones are incremental. That’s by design.

How many TB per hour should realistically go through? On symmetric 1 Gbit: ~400 GB/h net. On 100 Mbit: ~40 GB/h. On 50 Mbit home DSL: ~20 GB/h.

My backup shows 80% done but keeps going forever — why? Sometimes the last few percent are “verification at the end” (PBS checks the index) — that can take a few minutes without the percentage moving. Check the logs.

Does it matter whether I have one or several backup jobs? Performance-wise, barely. What matters is the order — if everything is scheduled for 02:00, everything runs in parallel. With 5 VMs at once, things get tight.

Why is GC sometimes slow? GC reads all chunks (unreferenced ones included). On a 5 TB datastore that’s a lot of chunks and a lot of random I/O. RAID 10 spreads the load across several disks, but it still takes a while on large datastores. For a recurring performance problem: open a ticket.

What if my VM is too big for a single backup window? The backup job keeps running past the window until it finishes; it only blocks the next job start. If that’s chronic: shrink the VM, split it into two VMs, or split the VM into sub-disks.

What’s next

• Configuring a backup job in PVE
• Understanding storage usage
• Garbage collection