Couldn't agree more. Logged in this morning with 147 failed backups. Yet, no info or alerts from MSP360 team. It's ridiculous that I have to come to the forums to discover it's not just us having the problem. When we utilize support, it's logs, high level logs, and some more logs, then also install this 3rd party software utility and send us more logs. We're ending up far too involved and spending too much time troubleshooting your product that's not working.

What's the point of having https://status.msp360.com/ when it shows all systems as operational and that's clearly not the case. Hasn't even been updated since Oct 4th.

Every month or two, there's a major bug or "known issue" yet nobody is ever made aware of this. Is it really asking too much for you to send out a simple email or keep up to date with your status page? You are seriously dropping the ball on some basic features and we're becoming increasingly frustrated with the amount of babysitting required.

Thanks, I appreciate it David.

Thanks David. We weren't manually running it, we just had in the scheduling to run a full backup once per month on a specified day/time each month. Attaching image for reference. As you can see, we didn't need to set a daily schedule for block level, since when the NAS backup would chain into the Cloud backup, it would run a block level automatically.

Yes, your interpretation is correct. We are using the legacy backup format.

Our implementation worked because the chained cloud backup would run an incremental daily, and we then have a schedule record for the cloud backup to run a synthetic full image on the 1st Saturday or Sunday of each month at a specified off hour. That way it wasn't an endless chain of incremental backups and we were getting a full cloud image in once per month.

If I'm not mistaken, hybrid backup doesn't support synthetic full correct?

Unchaining the backups would put us back in the position of having to time the completion of the NAS backups to ensure they're not overlapping with the cloud backup.

Please understand when we started using MSP360 (it was Cloudberry back then), hybrid backup didn't exist. Nearly all of our 300+ setups are identical to the described implementation and therefore we're really in a tough spot here. Thanks.

David, we have continued to work with support, and still don't have a fix at this point. I feel a fair question to ask is if settings of the legacy backup format are being modified, why isn't this change listed in the version release notes? Attached you can see the chain backup options for the legacy format from 2 build versions of the backup agent. The build before the issues started for us, and the build after. On the latest builds, it now defaults to "use settings of the current plan" which never existed as an option before within chain settings. If I'd seen this change listed the release notes, I wouldn't have deployed the update.

We chain our daily incremental cloud image backups to run after completion our Full image NAS backups. This way we don't have to set a start time for the incremental cloud image backup to begin since the NAS backup completion time can vary, and only 1 image backup can run at a time. We have run full daily image NAS backups that chain into daily incremental image cloud backups for years, and now all of sudden we can't?

This essentially means that if I want to chain a cloud backup to run after a NAS backup, and the NAS backup is a full image backup, the cloud backup must now be a full image backup as well? I want to ensure I'm not misinterpreting this. Thanks.

We experienced this same issue as well. We have hundreds of Endpoints and I was unable to find a reason while this was happening on a handful, but not all across the board. In addition to this, we had approximately 20 servers that were affected in which they started performing full image backups to cloud daily instead of incremental image backups.

Don't mean to dig at you guys, but this isn't the 1st, or even 2nd time something like this has happened to us. It's making me extremely hesitant to push out agent updates unless absolutely necessary going forward. We end up having to scramble to find a workaround. In this case the only solution was to manually downgrade each affected machine to the previous agent version. Can you PLEASE bring up the possibility of having the ability to choose the deployed version of the backup agent to the dev team??? At the very least, being able to rollback to the previous version would be invaluable for situations like this.


As it is currently, once you update there's no going back without disabling auto-updates. Then you still have to manually login into each endpoint and install the previous version. It's extremely time consuming, and not to mention as far as I'm aware, there's no repository to download previous agent versions unless you happen to have the installer stored somewhere locally, which thankfully in this case, I did.

Couldn't agree more. We have no problem setting up new client's on the updated backup format going forward, but forcing widescale retroactive adoption and placing the burden onto your customers seems to be strongarm tactic that doesn't sit well with us. You guys are selling almost exclusively to MSP's now, not individual businesses. You must understand that a complete systematic change like these has huge impact on us. We have hundreds of terabytes of data that would need to be reuploaded to cloud storage to accommodate the new format. Also, we'll be the ones to incur the additional cloud storage costs if switching to the new format since it would be a different backup plan, and therefore separate storage. We've been with you guys for a few years now, and there really seems to be a disconnect between the company and your clientele. From our perspective, there's a high level of focus placed on "what's the next new feature" vs. making your existing software bulletproof and gradually improving it over time.

Unrelated, but having a "current bugs" or "known issues" tab in the management console would be immensely helpful. I also saw the "backup client's out of date" bug this morning, and end up having to drill down into the forums to find information regarding this, or I have to call your support just for them to tell me it's a known issue, it's just doesn't seem very efficient for you or us.

That's all well and good for new clients, but we have nearly 800 backup plans at this point in time running on the "legacy" backup format. I agree with Steve Putnam in re-uploading hundreds of clients data is hardly a viable option. When is this supposed to be enforced? This is something that would take us an unbelievable amount of man hours to complete as the change would need to be individually handled on each machine which is just crazy to even think of.

Also, in the new backup format, there's no option to run full only backups? you have to first enable incremental backups to allow for full image backups, but then have no option for exclusively running a full image backup.

Thanks David!

Yeah, I've given up on this issue being fixed at this point. It's been over a year and to this day, I'm still manually modifying text documents on every new setup we do.

Hi Ivan,

I noticed you mentioned it downloads "a part of the backup data required to start a test Windows Hyper-V VM" Could you elaborate on that and exactly what data would be getting restored? Thanks.

I called into phone support and they confirmed there's currently a large scale issue that's being investigated.

I wanted to express our disappointment that there was no information or notification sent out regarding something as significant as this. I currently have 147+ overdue client backups from last night as a direct result of this issue.

A system status page, or an alert in the management console would go a long way in keeping customers in the loop and reassuring us that you're on top of things. An explanation of what happened and what's being done to mitigate something like this from occurring in the future feels like an appropriate response in this situation.

We love your guys product when it works as intended, but lately it seems we're getting into an ugly pattern of new issues popping up and never being properly adressed.

Anton, following up as this issue is still occuring. Really hoping for a fix to resolve this. Today is the 1st of the month and we're having the same problems as we were previously. I've had a ticket open for a month and it was escalated to the developers, but I haven't heard anything back. Thanks.

Hi Matt,
It's been quite some time, hope all is well. I just noticed this morning that version 6.3 has been released. I checked the release notes, but did not see this mentioned. Can you please confirm if this has been resolved in the lastest version?

Guys, Still waiting for an update on this. We're busier than ever and this is adding additional time and frustration to each new setup we're deploying. Please address this.

Guys,

Hoping for an update on this as it's been well over a month. We've been taking on and converting a lot of client's lately to this setup with Cloudberry, and I'm growing tired of the manual workaround that I'm having to use each time. Also while it does work, I've noticed that when using the manual workaround correction you have to modify the file twice to get the changes to actually stick.

thanks,

Hi Anton,

Very cool to see Powershell module. However, I'm not sure what you've referenced is what I'm trying to accomplish. Our problem is chained backups running full backups daily instead of block level backups daily.

We run as follows. NAS backup full daily, then chained cloud backup block level daily. Then the cloud backup needs to run a synthetic full once a month.

The method you've described here is just to force a chained backup to run as full correct?

Thanks,

Just wanted to update. I was assisted by support who was able to provide a workaround for the issue. The fix requires manually modifying the backup configuration file from the backup agent on the server, and changing values for 3 different lines of code.

While I truly appreciate the help and temporary workaround, I'd just like to reiterate the importance of a timely fix for this. Having to login to each server and manually update a .txt configuration file for each client's backup plan is very tedious, and a time waste for our limited resources.

Since the tech was able to fix it, I'm assuming you're aware of the specifics and where the problem is occurring? If not, I'd be happy to show you exactly where the issue is happening.

Thanks.

Logs sent.

Sergey, Thanks for that. I've just checked one of the machines with this issue and can confirm that <ForceFullNextPlan> is already set to false.