patternsqlMinor
Can PostgreSQL databases be attached/detached on the fly?
Viewed 0 times
postgresqlcanthedatabasesattacheddetachedfly
Problem
Is it possible to attach/detach databases to PostgreSQL on the fly, i.e. while the server is running without having to restart the service? I can't find anything related to this in the documentation.
What I am trying to achieve is to have the user be able to select different data directories from an archive location and then have them loaded into PostgreSQL, and later removed when the user is finished.
What I am trying to achieve is to have the user be able to select different data directories from an archive location and then have them loaded into PostgreSQL, and later removed when the user is finished.
Solution
No, short answer is no. This has been asked at the PostgreSQL development mailing list at 2011: Detach/attach database
and the many replies in summary said "yes, this is feasible to implement but quite complicated to do so.":
You would have to do quite some surgery because of oids from shared tables. I don't think thats easily doable.
Andres Freund
The main reason this doesn't work is XID management.
It's barely possible you could make it work if you first locked all
other sessions out of the DB and then froze every XID in the database,
but that's a sufficiently heavyweight operation to make it of dubious
value.
...
Tom Lane
I have to admit I've thought about this from time to time, and it
would be pretty cool. I was initially thinking that it wouldn't be
that difficult to do this on a per-database level, because if you
slurp up a whole database then by definition you're also including the
system catalogs, which means that you have the pg_class, pg_attribute,
and pg_type entries that are necessary to interpret the table
contents. If you do anything more fine-grained (per-tablespace,
per-table, or whatever) then things get much more complex, but at the
database level you only need to worry about interactions with other
globals: tablespace and role definitions. And we could probably write
code to grovel through the system catalogs for a newly "mounted"
database and do search and replace on the appropriate columns, to map
from the old OIDs to the new ones. It wouldn't be simple, but I think
it could be done.
But Tom's point about XIDs and LSNs seems like it kind of puts a
bullet through the heart of the whole idea. Now, before you can move
the database (or table, or whatever) between clusters, you've got to
rewrite all the data files to freeze XIDs and, I don't know, zero out
LSNs, or something. And if you're going to rewrite all the data, then
you've pretty much lost all the benefit of doing this in the first
place. In fact, it might end up being slower than a dump and
restore; even an uncompressed dump will be smaller than the on-disk
footprint of the original database, and many dumps compress quite
well.
Robert Haas
What about having database-level XIDs rather than cluster-level? Is that
remotely feasible?
Thom Brown
Maybe. You'd need a set separate set for shared catalogs, too. It
seems like a heck of a lot of work, though, especially since (IME,
anyway) most people only really one run one database per cluster.
Robert Haas
Leaving aside all the internal implementation details / problems that would make this possible, the last comment has something that may be helpful to you:
Run a different cluster per database. You would probably need to implement
something of an infrastructure so when a user wants to "attach" a database, it would start up the related cluster and stop it when the user wants to "detach" it.
and the many replies in summary said "yes, this is feasible to implement but quite complicated to do so.":
You would have to do quite some surgery because of oids from shared tables. I don't think thats easily doable.
Andres Freund
The main reason this doesn't work is XID management.
It's barely possible you could make it work if you first locked all
other sessions out of the DB and then froze every XID in the database,
but that's a sufficiently heavyweight operation to make it of dubious
value.
...
Tom Lane
I have to admit I've thought about this from time to time, and it
would be pretty cool. I was initially thinking that it wouldn't be
that difficult to do this on a per-database level, because if you
slurp up a whole database then by definition you're also including the
system catalogs, which means that you have the pg_class, pg_attribute,
and pg_type entries that are necessary to interpret the table
contents. If you do anything more fine-grained (per-tablespace,
per-table, or whatever) then things get much more complex, but at the
database level you only need to worry about interactions with other
globals: tablespace and role definitions. And we could probably write
code to grovel through the system catalogs for a newly "mounted"
database and do search and replace on the appropriate columns, to map
from the old OIDs to the new ones. It wouldn't be simple, but I think
it could be done.
But Tom's point about XIDs and LSNs seems like it kind of puts a
bullet through the heart of the whole idea. Now, before you can move
the database (or table, or whatever) between clusters, you've got to
rewrite all the data files to freeze XIDs and, I don't know, zero out
LSNs, or something. And if you're going to rewrite all the data, then
you've pretty much lost all the benefit of doing this in the first
place. In fact, it might end up being slower than a dump and
restore; even an uncompressed dump will be smaller than the on-disk
footprint of the original database, and many dumps compress quite
well.
Robert Haas
What about having database-level XIDs rather than cluster-level? Is that
remotely feasible?
Thom Brown
Maybe. You'd need a set separate set for shared catalogs, too. It
seems like a heck of a lot of work, though, especially since (IME,
anyway) most people only really one run one database per cluster.
Robert Haas
Leaving aside all the internal implementation details / problems that would make this possible, the last comment has something that may be helpful to you:
Run a different cluster per database. You would probably need to implement
something of an infrastructure so when a user wants to "attach" a database, it would start up the related cluster and stop it when the user wants to "detach" it.
Context
StackExchange Database Administrators Q#151839, answer score: 4
Revisions (0)
No revisions yet.