Retention Policy Problem with V7
  • Steve Putnam
    36
    Been using Version 7.2 with Backblaze with the intent of doing a synthetic full image and VHDx Files each weekend.
    The retention period is set for 5 days. I would expect that once a new Synthetic full is completed that the previous weekend's backup would be purged - since it has been over five days since the last backup.
    That is not what is happening.
    It is saying that for last weeks version - "this will be purged in 5 days" - after completeing the new weekly backup.
    So it appears that the previous week backup does not get purged until 5 days after the next backup runs. Not what I expected/want.
    In this scenario, there will always be TWO backups in the Cloud, even though I want to keep only ONE backup. Previously it worked fine. I did monthly Image cloud backups with a retention of one week. The prior month's backup would get purged since obviously it was done a month ago, so I ended up with one full backup.

    I have sent this to support, but they do not seem to understand the question or think this is normal expected behavior. I am telling you that this is NOT expected behavior.
    I want to keep a single Full backup in Cloud Storage.
    Steve Putnam Options
  • Steve Putnam
    36
    See the screenshot below - taken today 11/7.
    First generation from 10/30- 10/31 says it will be deleted in two days.
    Yet the retention period is only 3 days and the job runs weekly so the first generation should have been deleted at the completion of the second generation on 11/6.
    Now if the data would actually get purged without having to wait until the next week's plan runs it would be ok. But it appears that there is no way to prevent there always being two generations in Cloud storage - doubling my cost.
    aws4_request&X-Amz-Date=20211107T164433Z&X-Amz-Expires=604791&X-Amz-Signature=1381381a811600480a17869921bd390db8b68d4e94dd702ee6d167b156c0c092&X-Amz-SignedHeaders=host&response-content-disposition=inline
    Steve Putnam Options
  • Steve Putnam
    36
    So I finally got the answer from support:

    You are absolutely right about new backup format retention. In new backup format, retention is the period of time a backup generation is retained once it has been replaced. This means in your Full Backup Only configuration you will always have two Full Backups. It is not currently possible to delete your previous Full Backup when you perform your next Full Backup.

    This is different from how retention worked in legacy backup format. Legacy format retention period was based on backup date.

    I'm going to submit a feature request on your behalf.
    Steve Putnam Options
  • Steve Putnam
    36
    This is becoming a large problem. While there is absolutely no reason to need to keep two fulls in Cloud storage of our Image backups, there is no way to keep only the latest one. So our storage costs have doubled from 17TB to 35TB each month. That equates to an additional $85/month, which might not seem like a lot, but we are a small MSP, and did not anticipate this added expense.
    Have not heard anything back about whether this will be remedied in a future release, so am posting here to hopefully get an update.
    Steve Putnam Options
  • David Gugick
    118
    ↪Steve Putnam
    how did you avoid having multiple full backups in cloud storage with the old backup format? Even with the version-based retention set to something like 14 days with fulls running every 14 days you would still have two in there until the first full backup set could be deleted. How are you using retention using the new backup format?
    David Gugick Options
  • Steve Putnam
    36
    Very simple. Used to do full backups monthly with 3 day retention. Once newest full backup completed, the previous full got purged. Worked great.
    Steve Putnam Options
  • David Gugick
    118
    ↪Steve Putnam
    So, no incrementals then, correct? If so, I understand, but likely not a common use case. But I see how we could mimic that same behavior with the new format. As I recall I've already spoken to the team when we first discussed. I'll follow up with them on Monday.
    David Gugick Options
  • Steve Putnam
    36
    Thanks. No need for incrementals, as fhese are Disaster Recovery images or VHDx files. A month or week old image is fine to get someone backup,and running, using these as a base plus daily file based backups
    Steve Putnam Options
  • David Gugick
    118
    ↪Steve Putnam
    I checked with engineering, and the changes are already in the planning stage and should be released with the 7.4.1 agent release. No exact dates for release yet. I will let you know when I have a date. The next release is 7.4.0 - 7.4.1 follows.
    David Gugick Options
  • Steve Putnam
    36
    Thanks David, for following up on this. FYI, I did discover that if I delete the oldest of the two generations and uncheck "Enable Full Consistency Check" box, that the synthetic full runs just fine with only a warning that some data is missing from storage.
    Not worth the time and effort to do that every week for sure.
    Steve Putnam Options
  • Steve Putnam
    36
    David,
    Any chance this issue was addressed in the latest release 7.5? Really tired of spending $80+ per month to keep two full image/ VHDx copies in the cloud when we really only need one.
    Steve Putnam Options
  • Alexander Negrash
    32
    ↪Steve Putnam
    Hi Steve! This has been addressed in version 7.5. We have also updated our help article to reflect the updated retention logic. https://help.mspbackups.com/backup/about/retention-policy
    Alexander Negrash Options
  • Steve Putnam
    36
    I updated the server to 7.5 and made sure that my retention was set to 1 day (as it had been). I forced a synthetic Full Image Backup.
    I expected that since it has been four days since the last Full backup, that the four day old Full would get purged, but that did not happen.
    It is behaving the same as the prior version - It always keeps two generations, where I was expecting to now only have to keep one.
    See attached file for a screenshot of the Backup storage showing that both generations are still there.
    Attachment
    PUT5 (9K)
    Steve Putnam Options
  • Alexander Negrash
    32
    ↪Steve Putnam
    Steve, I have escalated this to the development team.
    Alexander Negrash Options
  • Steve Putnam
    36
    Any word from the development as to whether this can be fixed?
    Steve Putnam Options
  • David Gugick
    118
    ↪Steve Putnam
    Looks like this is addressed in 7.5.1. No release date yet, but it should be soon. Support should reach out from the support case. In any case, the changes to 7.5.0 should have deleted the original set the next day (1 day delayed from what was expected), so your issue should have been temporary. Needless to say, it has been addressed internally.
    David Gugick Options
  • Alexander Negrash
    32
    ↪Steve Putnam

    Steve 7.5.1 with this bug fix has been released.
    Alexander Negrash Options
  • Steve Putnam
    36
    Thank you. I will test tonight.
    Steve Putnam Options
  • taareni9
    0
    I expected that since it has been four days since the last Full backup, that the four day old Full would get purged, but that did not happen.
    taareni9 Options
  • David Gugick
    118
    ↪taareni9
    What's your retention set to? Retention is about how much backup data you need to keep. So if it's set to 3 days and you are running a Full + 2 Incremental backups and then repeating the process, on the fourth day the initial full is 4 days old, but you are keeping 3 days, and that full needs to be kept until it can safely be deleted while still maintaining the 3 days of retention.
    David Gugick Options
bold
italic
underline
strike
code
quote
ulist
image
url
mention
reveal
youtube
tweet
Add a Comment