Forever Forward Incemental Configuration

[al3x]

One example of a setup might be: If there is a full only, create a diff.  If the last backup is a diff create an inc.  If the last 5 were incs, create a diff.  If there are three diffs and the last one includes 5 incs, start a new full.  There are many possibilities, of course.

Yeah, but you see the problem with adding so many rules, right? You'd have to find a way to show that in the GUI without being way too confusing. And we already have the schedules that also allow a lot of options. So, I don't know. It would be a completely new ruleset on top of the existing scheduler. But it would still not allow manual backups based on calendar days for example.

My point is, a manual option on top of the schedule wouldn't be a complete new ruleset, but rather a small addon to allow more dynamic manual backups.

My Christmas wish is therefore actually similar to Gork’s:
A backup chain consisting of a full backup followed by, for example, exactly 5 incremental backups that belong only to that full backup,
and then starting the whole process over again with a new full backup and 5 incremental backups, and so on.

With retention, you could, for example, decide how many of these complete chains you want to keep.
A solution like this could be started at any time with a single click (or scheduled); it would never require a decision about when or which
type of backup needs to be made, and files would never need to be merged 

That would work with my schedule "extension" idea:

You would for example create a daily inc. schedule and tick "Perform a full backup after a specified number of backups".
Then just pause the schedule and enable the "Full" image retention policy with for example 3 backups and it will keep 3 chains.

But maybe let's wait what @admin thinks about all of those ideas.👍

Cheers,
al3x

[Bespoken]

My Christmas wish is therefore actually similar to Gork’s:
A backup chain consisting of a full backup followed by, for example, exactly 5 incremental backups that belong only to that full backup,
and then starting the whole process over again with a new full backup and 5 incremental backups, and so on.

Maybe Christmas is here.

First do an initial Full Backup manually, then setup a schedule:
- select Incremental to be run daily or weekly (choose days)
- select Perform a full backup after a specified number of backups <5>
- select retention policy for Full (2+), and Incremental (5, 10, etc)

Would that work for you?

[al3x]

Maybe Christmas is here.

First do an initial Full Backup manually, then setup a schedule:
- select Incremental to be run daily or weekly (choose days)
- select Perform a full backup after a specified number of backups <5>
- select retention policy for Full (2+), and Incremental (5, 10, etc)

Would that work for you?

Unfortunately, not yet

First off, you would have to not choose a retention policy for incrementals, otherwise it will start merging your incrementals after 5 runs but still not perform a new full backup.

Secondly, if you manually click on "Backup", the current logic will still only perform the predefined backup method forever without respecting the "Perform a full backup after..." option.
That is because it's part of a schedule and not considered for manual runs.

So if you want to have some more logic for manual backups, that would still need changes in HBS. But it's still time till Christmas. 

[Bespoken]

So along with the other enhancement requests there is also the need for a setting of "Do not merge Incrementals". this should be doable.

[admin]

@All,

Thank you all for the in-depth discussion.

I understand that the ultimate request is for both manual and scheduled backups to be able to generate cyclic backup sub-chains in the form of "1 full backup + x incremental backups" according to the configuration, and for the image retention policy to be based on backup sub-chains rather than on image types (full, incremental, and differential). We will need some time to discuss the feasibility of this approach internally.

I believe the most challenging issue is how to present this in the user interface. The existing backup scheduling policy and image retention policy will certainly need to be retained, and layering an additional "chain-state-based ruleset" on top of them could easily make the interface complex and difficult to understand.

You are all welcome to continue adding other aspects that should be considered, or to share your own thoughts on the issues mentioned above. We will carefully evaluate the feasibility and keep you updated as we make progress.

Have a nice day!

Best regards!

[Bespoken]

I believe the most challenging issue is how to present this in the user interface. The existing backup scheduling policy and image retention policy will certainly need to be retained, and layering an additional "chain-state-based ruleset" on top of them could easily make the interface complex and difficult to understand.

You are all welcome to continue adding other aspects that should be considered, or to share your own thoughts on the issues mentioned above. We will carefully evaluate the feasibility and keep you updated as we make progress.

Here is a proposal of an addition to the schedule option, and a new facility to control manual backups.

Changes to Schedule screen
- add "Do not merge incremental backups"
   
This allows the existing incremental backups to remain unchanged and old ones to drop off when the number to keep is exceeded.

New Manual Cyclic Backup screen
When this setting is enabled, clicking a Backup on the main backup window would use the rules defined here.
   
- The Manual Cyclic Backup control can be turned On or Off at any time
- the Retention Policy is similar to those in the Schedule settings, except the count for differential and incremental refer to the chain and not the total for this backup
- some items are mutually exclusive within the Cyclic Backup selection, ie. specifying the number of diff/inc backups to trigger a full, and if a diff or inc is started after a full

[al3x]

Changes to Schedule screen
- add "Do not merge incremental backups"

This allows the existing incremental backups to remain unchanged and old ones to drop off when the number to keep is exceeded.

That's basically the feature of the image retention policy for incremental backups. Incremental images will be merged if you set this retention policy. If you don't set this, it won't merge images. You can't "drop off" old inc. backups because newer ones rely on it, so you have to create a new diff/full image or merge old ones.

New Manual Cyclic Backup screen
When this setting is enabled, clicking a Backup on the main backup window would use the rules defined here.

The main problem with these is, they only cover some of the settings, e.g. you have a dedicated option just for a new month (what about week, special calendar days or something else etc.). So I personally think those would overcomplicate the options a lot.

[Bespoken]

Changes to Schedule screen
- add "Do not merge incremental backups"

This allows the existing incremental backups to remain unchanged and old ones to drop off when the number to keep is exceeded.

That's basically the feature of the image retention policy for incremental backups. Incremental images will be merged if you set this retention policy. If you don't set this, it won't merge images. You can't "drop off" old inc. backups because newer ones rely on it, so you have to create a new diff/full image or merge old ones.

This is an override to the retention policy when a merge is never wanted, and old incremental backups are to be automatically deleted.

[al3x]

This is an override to the retention policy when a merge is never wanted, and old incremental backups are to be automatically deleted.

That's technically not possible, sorry to say. You can't just delete old incremental backups, newer ones rely on the full chain including the old incremental backups. So you have to merge them or create a new diff/full backup.

[Bespoken]

This is an override to the retention policy when a merge is never wanted, and old incremental backups are to be automatically deleted.

That's technically not possible, sorry to say. You can't just delete old incremental backups, newer ones rely on the full chain including the old incremental backups. So you have to merge them or create a new diff/full backup.

Then the setting/config must then allow all incremental backups in that chain to be auto deleted if there are one or more new full backups. The objective is to avoid manual activity and merging.