Difference between revisions of "Cloning disks or partitions using dd"

From Notes_Wiki
m
m
 
(4 intermediate revisions by the same user not shown)
Line 1: Line 1:
=Cloning disks or partitions using dd=
[[Main Page|Home]] > [[CentOS]] > [[CentOS 6.x]] > [[Backup tools]] > [[Dd]] > [[Cloning disks or partitions using dd]]


We can backup entire hard-disk or partition or clone hard-disks using dd and nc command combinations. To clone hard-disk from one machine to another we can use:
We can backup entire hard-disk or partition or clone hard-disks using dd and nc command combinations. To clone hard-disk from one machine to another we can use:
Line 9: Line 9:
#Start data receiver using:
#Start data receiver using:
#:<pre>
#:<pre>
#::nc -l -p 9000 | gunzip -4 | dd of=/dev/sda bs=40000000
#::nc -l -p 9000 | dd of=/dev/sda bs=40000000
#:</pre>
#:</pre>


Line 19: Line 19:
#Start data sender using:
#Start data sender using:
#:<pre>
#:<pre>
#:: dd if=/dev/sda bs=40000000 | gzip -4 | nc <IP_of_destination> 9000
#:: dd if=/dev/sda bs=40000000 | nc <IP_of_destination> 9000
#:</pre>
#:</pre>


Note:  
Note:  
*The default compression level used by gzip is 6. Any compression level higher than that will cause too much CPU usage and very less network usage. Hence use compression less than or equal to 6. Compression of 4 used above is good enough for 100mbps networks, especially if same source is serving three destinations and source has dual core CPU with good RAM. On gigabit networks one may want to avoid compression or reduce it to -1 or -2.
*The above process can also be used for copying VM images from one machine to another machine. Copying sparse VM images would cause ballooning on destination. Then we can use '<tt>cp --sparse=always</tt>' to reduce the ballooned image to normal size if required. It should be noted that this finishes faster than <tt>rsync</tt> with '<tt>-Sz</tt>' combinations
*The above process can also be used for copying VM images from one machine to another machine. Copying sparse VM images would cause ballooning on destination. Then we can use '<tt>cp --sparse=always</tt>' to reduce the ballooned image to normal size if required. It should be noted that this finishes faster than <tt>rsync</tt> with '<tt>-Sz</tt>' combinations
* If network is slow (Internet or 100mbps) then use "gzip -4" at source and use "gunzip -4" at destination.




Line 30: Line 30:


==Seeing progress==
==Seeing progress==
To see progress while copying use option '<tt>status=progress</tt>'.  For example:
<pre>
dd if=/dev/cdrom of=/opt/cd1.iso status=progress
</pre>
In this case progress is continuously displayed on the machine where dd is run with status=progress option.


To see progress of how much data is copied we can use:
 
Older way to see progress of how much data is copied was:
<pre>
<pre>
ps -C dd
ps -C dd
while true; do kill -USR1 <process_id>; sleep 1; done
while true; do kill -USR1 <process_id>; sleep 1; done
</pre>
</pre>
on any of source and destination. In case source is serving multiple destinations than preferably do it on destination.
on any of source and destination. In case source is serving multiple destinations than preferably do it on destination.




Line 44: Line 50:


When using same source to copy data to three destinations parallely, try to start source to send data to all three soon one after other. If there is large gap while trying to send same data to three destinations the data read from hard-disk into RAM cache may get replaced and hence require re-reading of hard-disk. This may lead to hard-disk becoming bottle-neck and not network. It was observed that copying to more than three destinations at same time does not works well and causes machines to hang for some unknown reason.
When using same source to copy data to three destinations parallely, try to start source to send data to all three soon one after other. If there is large gap while trying to send same data to three destinations the data read from hard-disk into RAM cache may get replaced and hence require re-reading of hard-disk. This may lead to hard-disk becoming bottle-neck and not network. It was observed that copying to more than three destinations at same time does not works well and causes machines to hang for some unknown reason.
[[Main Page|Home]] > [[CentOS]] > [[CentOS 6.x]] > [[Backup tools]] > [[Dd]] > [[Cloning disks or partitions using dd]]

Latest revision as of 03:02, 26 February 2024

Home > CentOS > CentOS 6.x > Backup tools > Dd > Cloning disks or partitions using dd

We can backup entire hard-disk or partition or clone hard-disks using dd and nc command combinations. To clone hard-disk from one machine to another we can use:

On destination machine:

  1. Boot using Linux live CD
  2. If Live CD is cached in RAM then use 'eject /dev/scd0' to eject cd
  3. Use 'dhclient eth0' to get IP from DHCP server. Note that IP address obtained.
  4. Start data receiver using:
    nc -l -p 9000 | dd of=/dev/sda bs=40000000


On source machine:

  1. Boot using Linux live CD
  2. If Live CD is cached in RAM then use 'eject /dev/scd0' to eject cd
  3. Use 'dhclient eth0' to get IP from DHCP server. Note that IP address obtained.
  4. Start data sender using:
    dd if=/dev/sda bs=40000000 | nc <IP_of_destination> 9000

Note:

  • The above process can also be used for copying VM images from one machine to another machine. Copying sparse VM images would cause ballooning on destination. Then we can use 'cp --sparse=always' to reduce the ballooned image to normal size if required. It should be noted that this finishes faster than rsync with '-Sz' combinations
  • If network is slow (Internet or 100mbps) then use "gzip -4" at source and use "gunzip -4" at destination.



Seeing progress

To see progress while copying use option 'status=progress'. For example:

dd if=/dev/cdrom of=/opt/cd1.iso status=progress

In this case progress is continuously displayed on the machine where dd is run with status=progress option.


Older way to see progress of how much data is copied was:

ps -C dd
while true; do kill -USR1 <process_id>; sleep 1; done

on any of source and destination. In case source is serving multiple destinations than preferably do it on destination.


Performance

It is better to use udpcast in case you want to copy same source to more than one destinations

When using same source to copy data to three destinations parallely, try to start source to send data to all three soon one after other. If there is large gap while trying to send same data to three destinations the data read from hard-disk into RAM cache may get replaced and hence require re-reading of hard-disk. This may lead to hard-disk becoming bottle-neck and not network. It was observed that copying to more than three destinations at same time does not works well and causes machines to hang for some unknown reason.




Home > CentOS > CentOS 6.x > Backup tools > Dd > Cloning disks or partitions using dd