Join Log in
Score:1
avatar Server

Is it possible to migrate between two physical disks without (much) downtime?

de flag
Christopher Schultz

I have a server with a medium-sized database on a disk which is filling-up. There is no LVM or RAID or anything like that in play at this point. I already have the new disk installed in the server.

Is it possible to migrate the data to the new physical disk with minimal or no downtime? We have performed a copy-speed-test from the old drive to the new one and it's going to be a few hours to accomplish. Not strictly speaking a deal-breaker, but I'd like to do better if possible.

I have a few ideas, and I'd like to know what the feasibility of each might be.

  1. Migrate to LVM. If it's possible to migrate a bare partition to an LVM one (is it!?), then it's a simple pvmove to move to the new disk without any downtime at all. I'm kicking myself for not having used LVM at the time. :/

  2. Use dm-raid to mirror from the existing disk to the new disk, wait for sync, then break the raid and toss the old disk. This requires only enough downtime to re-mount the filesystem from the physical device to the device-mapper device. The "problem" here, then is that I would have a RAID layer I wasn't using after all is said and done. I also don't achieve the flexibility of LVM using this setup.

Another option is setting up a new server (or even just a new service on the same server) and use the replication capabilities of the database (PostgreSQL in this case), but that seems like much more work than necessary.

89
0 + 10
lvm
Mat avatar
cn flag
Mat
Alternative to replication would be to set up new tablespaces and move objects from old to new. (But I don't know Postgres enough - might incur a lot of locking.)
0
Reply
djdomi avatar
za flag
djdomi
databases should not be directly copied as long they are beeing used, i would suggest a downtime on a weekend and dd the complete harddisk and extend it afterwards. remember that in case of no raid and backup your are really silly and get a greater impact as this
0
Reply
de flag
Christopher Schultz
@djdomi I wasn't suggesting copying the data out from underneath a running database. I'm specifically looking for under-filesystem options, here. My initial read/write test just used `cp -r` to see how long it would take. We considered using `rsync` twice (once online, once afterward), but decided that the read-speed of the old drive is the bottleneck and we'd need to read 100% of both the source and the target in order to perform a "differential" rsync during the downtime, so there is no actual benefit.
0
Reply
de flag
Christopher Schultz
@Mat Moving objects from one tablespace to another might work, but the application on top of the db doesn't support anything like that and I don't know PostgreSQL well enough to migrate in-use data like that.
0
Reply
A.B avatar
cl flag
A.B
I'll put back this link I hesitated to put, because it requires a lot of elbow grease to get it right: https://www.kernel.org/doc/html/latest/admin-guide/device-mapper/dm-clone.html . caveat: [Updating on-disk metadata](https://www.kernel.org/doc/html/latest/admin-guide/device-mapper/dm-clone.html#updating-on-disk-metadata). Would just require a short downtime to switch the underlying device of the FS. With more elbow grease, the target could be an LVM PV/LV prepared in advance.Then once everything is migrated, 2nd small downtime to adjust again.
0
Reply
de flag
Christopher Schultz
@A.B `dm-clone` looks like what I'm looking for. Can you confirm that I'd (1) stop my service/db, remount the existing fs readonly, create the dm-clone from old->new device, re-mount so things look like they did before, then (2) restart the services and let hydration complete, then (3) at any point after hydration has completed, I can remove the old device and the dm-clone fs, and switch to just using the new one?
0
Reply
A.B avatar
cl flag
A.B
you wouldn't remount the fs readonly. You'd prepare the new block at the target with a link (using clone features) to the old block device and would then directly mount the fs at the new place. hydration can work during database operations. You can probably manage to have the target use the exact blocks of a prepared-in-advance LV, so you later have LVM features available
0
Reply
de flag
Christopher Schultz
@A.B I was asking about read-only because I wouldn't want the source filesystem to be modified during the active copy-operation. I suppose it could be entirely unmounted, actually, since dm-clone will be working at the block level, anyway.
0
Reply
de flag
Christopher Schultz
For an LVM-target, I can specify the block-offset of the target to avoid overwriting the LVM header on the target device? That sound interesting indeed. Theoretically, though, I could also clone the source to the target where the target is an LVM logical volume already, since the source device itself contains nothing but the filesystem and the logical volume would be similar.
0
Reply
A.B avatar
cl flag
A.B
Yes that was implied: source is read-only once nothing uses it. To avoid manipulation errors, you can force a block device to be readonly with `blockdev --setro`. And yes you can choose the LV itself to be the destination part of the dm clone.
0
Reply
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.