Backups are in a proprietary format and files need to be restored using the product. What is your particular use case that you need the backup folders to be in a specific format; presumably one that matches the source folders? If you browse backups using the product's Backup Storage tab, then you do not see the "$" in the actual folder names that contain them. If you need to access backed up files from other products / utilities, you may want to try using the Simple Backup mode - it has minimal options for retention, but it does leave the files / folder in backup storage in their native format (no "$" or compression / encryption).

Where would I find "Simple Backup mode" in the web interface? I am using Ubuntu Server 20.04

That's a Windows-only option. So back to my original question: What's your use case that the backup format is important?

To have access to backup files without having to go into each folder to grab files

Are you saying you need to access files from other utilities without having to restore them first with CloudBerry Backup?