Hosting Provided By

Re: [GENERAL] "Resurrected" data files - problem?

From: Simon Riggs <simon(at)2ndquadrant.com>
Date: Fri Nov 09 2007 - 17:17:11 EST

On Fri, 2007-11-09 at 10:59 -0500, Tom Lane wrote:
> Simon Riggs <simon@2ndquadrant.com> writes:
> > On Fri, 2007-11-09 at 10:28 +0100, Albe Laurenz wrote:
> >> I think that understanding is finally dawning here.
> >>
> >> The problem you see is that the backup software might decide
> >> that the file has not been changed, skip it and go on backing
> >> up other files, but the file can still be modified before
> >> pg_stop_backup(), correct?
>
> > Correct.
>
> Surely that's nonsense --- otherwise a time-extended base backup
> could not work either.
>
> What is required of the filesystem backup process is that each 8K page
> of each file be restored to a state that it had at some time between
> pg_start_backup and pg_stop_backup. The exact time can be different for
> different pages. I don't see a reason to think that a base+incremental
> backup method can't meet that requirement.

Hmm, OK, I do think we can improve on what I said before. What I said was safe though, not nonsense, plus I think the difference between the two is a hair's breadth in practice. Most of the time we're taking about excluding historical data partitions, designed specifically to minimise backup windows and data maintenance.

The question is which timestamps do you compare in order to arrive at the maximal set of files that don't need to be backed up twice?

I'm assuming we're talking about starting recovery at the second pg_start_backup(). Any file that changes during the first backup cannot be a candidate, so timestamp before first backup must equal timestamp before second backup. So we can effectively reduce things to just 2 timestamps, not 4 as I had originally said.

So we have:

pg_start_backup()
timestamp1
full backup
pg_stop_backup()

...

pg_start_backup() --- WAL chain starts here timestamp2
incremental backup
pg_stop_backup()

Do you need help?

Any file for which ts1 == ts2 can be skipped during the incremental backup.

-- 
  Simon Riggs
  2ndQuadrant  
http://www.2ndQuadrant.com


---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings

Received on Fri Nov 9 17:19:36 2007

This message: [ Message body ]
Next message: Coarr, Matt: "[GENERAL] any way to query for current connections to db?"
Previous message: Todd A. Cook: "[GENERAL] Is "query" a reserved word in 8.3 plpgsql?"
In reply to: Tom Lane: "Re: [GENERAL] "Resurrected" data files - problem?"
Next in thread: Albe Laurenz: "Re: [GENERAL] "Resurrected" data files - problem?"

Contemporary messages sorted: [ By Date ] [ By Thread ] [ By Subject ] [ By Author ] [ By messages with attachments ]

This archive was generated by hypermail 2.1.8 : Mon Jun 16 2008 - 19:38:31 EDT