HiveBrain v1.2.0
Get Started
← Back to all entries
patternsqlModerate

Backing up & restoring 10-20 SQL Server databases to a ~synchronous state?

Submitted by: @import:stackexchange-dba··
0
Viewed 0 times
databasessynchronoussqlbackingrestoringstateserver

Problem

I need to backup up 10-20 SQL Server 2008 R2 databases with sizes between 10-50 GB, while they are online and used simultaneously by a single enterprise app. I also need to restore them to a state that is largely synchronized across all databases (I can afford up to a few seconds of desync between databases). The purpose is to capture production data for QA/DEV environments.

I would strongly like to not demand databases run in full recovery and to come up with a backup method that is dedicated to capturing data for QA environments and remains independent of a main backup process which is not under my control.

For my customers, it will take 1-2 hours to capture 20 full backups at ~30 GB each. This makes taking full backups sequentially unacceptable as the databases would be too desynchronized when running in simple recovery.

I'm looking for an idea better than these:

IDEA 1: SAN-level snapshot of VM disks. xcopy MDFs/LDFs from snapshot.

Once the copied files are attached to a different server instance, its recovery process should produce consistent databases that are snapshot pretty much simultaneously.

Googling around convinced me this is a bad idea, at least because I may get desync vs. master/msdb/etc.

IDEA 2: Orchestrate a complex backup & sync-restore across all databases

This requires me demanding databases run in full recovery, which I don't want. Start parallel backups for all databases well before the deadline (T0). Once T0 is reached, backup all logs (should take at most a few minutes). Take the resulting myriad of backups and try to restore them & roll logs forward/back to obtain a somewhat consistent state across databases, relative to T0.

This requires a lot of planning & scripting to have it used reliably so I would go to great lengths to avoid it.

Am I missing some other solution?

P.S.1: I would've loved being able to use db snapshots. The idea was to initiate a snapshot on each db (which should be over in seconds), then fully backup each o

Solution

I need to backup up 10-20 SQL Server dbs used simultaneously by a single enterprise app, while they are online, in such a way as to restore them to a state that is largely synchronized across all dbs

What you are looking for is a consistent backup across all your customer databases, you should use FULL backups along with Marked Transactions (emphasis in bold added):


When you make related updates to two or more databases, related databases, you can use transaction marks to recover them to a logically consistent point. However, this recovery loses any transaction that is committed after the mark that was used as the recovery point. Marking transactions is suitable only when you are testing related databases or when you are willing to lose recently committed transactions.

Make sure that you take adhoc transaction log backup with COPY_ONLY, else your recovery will be a pain, since any adhoc transaction log backup without COPY_ONLY will break the log chain. As a precaution, you can restrict users to use only COPY_ONLY backups.


I need a solution for SQL Server versions 2008 R2 and later. Db sizes is up to 50 GB per db and the time to backup all of them is likely over 1-2 hours.

Marked transactions will work for your situation. The only thing to make parallel backups is to STRIPE them, but then you end up making sure that you don't lose your stripes of backup. To make them faster, you can play with BUFFERCOUNT and MAXTRANSFERSIZE.

You should use backup compression as well as enable Instant file initialization.

Refer to

  • Using Marked Transactions (Full Recovery Model)



  • How to increase SQL Database Full Backup speed using compression and Solid State Disks ?

Context

StackExchange Database Administrators Q#117734, answer score: 12

Revisions (0)

No revisions yet.