New Backup Format - Size comparison on the bucket vs legacy
What's your experience on the size comparison with using the new backup format relate to using the legacy mode?
The legacy backup gives a bit of overhead, but at least keep forever.
How does the keep forever for the new format is working for all forum members?
I was wondering how much GB is a set of data using the legacy format, vs the new backup format after 1 year in terms of data consumption, because on Amazon we pay per GB or TB.
Having to have many full backups, makes me wonder how much should I quote when I try to close a new customer as well.
Thanks for the help.
We use legacy format for file backups. It is incremental forever. No need to keep multiple "full" copies such as is required with the new format.
We use the new format exclusively for Disaster Recovery VHDx and Image backups. Now that we can keep a single version of the image/VHDx in the cloud, it has worked out great. We do daily incremental and weeky synthetic fulls to Backblaze which does not have a minimum renention period.
It makes zero sense to go with the new format for file backups unless you are required to keep say an Annual full, a Quarterly ful, etc. We have no such requirement.
Thanks for sharing Steve. I was thinking on rolling out the new backup format for specific customers who have a lot of files on his file server.
On Amazon S3, we pay for each file uploaded to the cloud, there are customers who have like 6.000.000 files that a complete restore would probably take weeks in the legacy, but with the new format would be much faster.
I'm trying to find a way to make the new backup format works, in a way that doesn't have too much overhead but still would allow to have a faster restore.
I didn't figure it out yet.
I was thinking maybe to have a reduced retention such as 3 months, or running the new format incremental forever, but still I'm not convinced on what is the best course of action.
We design our systems to have a hypervisor and a separate VHDx disk for the DC and two vhdx disks for the File server- one with the OS (C:) and the other with the data (D:).
Using the new format, we do nightly incrementals and weekly synthetic fulls of all VHDx on a given server to BackBlaze B2 ( not BB S3 as it does not support synthetic backups.) BackBlaze is half the cost of the cheapest (viable) Amazon price.
We also do legacy format file backups locally, and to a separate Cloud provider ( Google Nearline or Wasabi).
So a complete system restore in a disaster situation requires only that we restore the VHDx files, which is a lot faster than restoring the files individually. A synthetic full backup of a 2TB data vhdx file can be completed in 12- 15 hours each weekend - depending on your upload speed and the data change rate. The incrementals run in five hours tops each night.
So I suggest local ( one year retention- not guaranteed) and cloud legacy individual file backups ( 90 day retention guaranteed) for operational recovery, and new format VHDx/ Image backups for DR. We keep only the most recent full VHDx files and any subsequent incrementals. Each week we start a new set and the previous week’s set gets purged.
What are your retention needs? Once we know that, we might have an easier time recommending settings for retention when using the new backup format. You will see much better restore performance with large numbers of files and you do get synthetic fulls with Amazon S3 with your file backups, as well as client-side deduplication, and backup verification built-in.
Thanks for sharing your schedule Steve, gave some insights for the process.
David, our requirement in this scenario is mostly for file backups.
We would like to keep 1 year retention of files and at least 3 versions.
Today, we run legacy format with 1 year delete local and keep at least 3 version.
We have one customer with 6 million files, which would take forever to restore the entire dataset.
So, we though on using the new format.
But as per I'm seeing, a always incremental approach wouldn't work, and run monthly fulls, would consume something like what... 10x more GB on the storage rather than the legacy format, due to the monthly full backup.
I wanted to use the new format because of the deduplication and much reduced recovery time, fetching the packages rather then millions of small files is much faster.
I know there is no rule of thumb and it depend on requirements, but are you having traction for file-based backups with this new format?
Thanks and regards,
Sign in or register to add a comment.
Add a Comment
Welcome to MSP360 Forum!
MSP360 Managed Products
Managed Backup - General
Managed Backup Windows
Managed Backup Mac
Managed Backup Linux
Managed Backup SQL Server
Managed Backup Exchange
Managed Backup Microsoft 365
Managed Backup G Workspace
Backup for Linux
Backup SQL Server
Connect Free/Pro (Remote Desktop)
"Product total size limit exceeded" for small backup to empty bucket
Cloudberry Explorer pro (S3 browser) not listing folders completely in DELL ECS S3 bucket
Internal error (500) when trying to copy big-size object from bucket to another bucket
FAQ - Managed Backup - Customization
Terms of Service
Useful Hints and Tips
Created with PlushForums
© 2023 MSP360 Forum