Difference between revisions of "Securely wipe disk"

From ArchWiki
Jump to: navigation, search
(Moving headings around, pls give me 24 hours ;))
(Moving content to Select a program: WIP)
Line 45: Line 45:
 
{{Note|You can retrieve progress of the dd command with this command: {{ic|kill -USR1 $(pidof dd)}}}}
 
{{Note|You can retrieve progress of the dd command with this command: {{ic|kill -USR1 $(pidof dd)}}}}
  
=== Badblocks ===
+
=== pattern write test ===
 
+
#badblocks -c 10240 -wsvt random /dev/<drive>
+
 
+
Where {{ic|/dev/<drive>}} is the drive to be encrypted.
+
 
+
 
{{Note|The {{ic|badblocks}} command overwrites the drive at a much faster rate by generating data that is not truly random.}}
 
{{Note|The {{ic|badblocks}} command overwrites the drive at a much faster rate by generating data that is not truly random.}}
  
See also [[badblocks]].
+
See also [[#Badblocks]].
 
+
=== shred ===
+
{{Box BLUE|From [http://en.wikipedia.org/wiki/Shred_%28Unix%29 Wikipedia]:|'''shred''' is a Unix command that can be used to securely delete files and devices so that they can be recovered only with great difficulty with specialised hardware, if at all. It is a part of GNU Core Utilities.}}
+
 
+
'''shred''' uses three passes, writing pseudo-random data to the harddrive each pass. This can be reduced or increased.
+
 
+
# shred -v /dev/<drive>
+
 
+
Where {{ic|/dev/<drive>}} is the drive to be encrypted. This invokes shred with default settings, displaying the progress to stdout.
+
+
# shred --verbose --random-source=/dev/urandom -n1 /dev/<drive>
+
 
+
Invokes shred telling it to only do one pass, with entropy from /dev/urandom. See [http://www.gnu.org/software/coreutils/manual/coreutils.html#shred-invocation GNU Coreutils] for more information.
+
 
+
=== dcfldd ===
+
There are a variety of applications that securely wipe a disk like {{Pkg|shred}} and {{Pkg|dd}}. Alternatively, {{Pkg|dcfldd}} is an enhanced version of dd with features useful for forensics and security. It accepts most of dd's parameters and includes status output. The last stable version of dcfldd was released on December 19, 2006.<sup>[http://dcfldd.sourceforge.net/]</sup>
+
  
 
=== Performance ===
 
=== Performance ===
Line 96: Line 75:
 
As a matter of course the best wiping practice is to never write unencrypted data.
 
As a matter of course the best wiping practice is to never write unencrypted data.
  
=== Select a program ===
+
== Select a program ==
==== coreutils ====
+
=== coreutils ===
 +
Official documentation for dd and shred is linked to under [[#See also]].
 +
 
 +
==== dd ====
 +
{{Wikipedia|Dd_(Unix)}}
 +
{{Note|cp does the same as dd without any operands but is not designed for more versatile disk wiping procedures.}}
 +
 
 
===== Checking progress of dd while running =====
 
===== Checking progress of dd while running =====
 
By default, there is no output of dd until the task has finished.  With kill and the "USR1"-Signal you can force status output without actually killing the program. Open up a 2nd root terminal and issue the following command:
 
By default, there is no output of dd until the task has finished.  With kill and the "USR1"-Signal you can force status output without actually killing the program. Open up a 2nd root terminal and issue the following command:
Line 113: Line 98:
 
  634388480 bytes (634 MB) copied, 8.17097 s, 77.6 MB/s
 
  634388480 bytes (634 MB) copied, 8.17097 s, 77.6 MB/s
  
==== dd-spin-offs ====
+
==== shred ====
 +
{{Box BLUE|From [http://en.wikipedia.org/wiki/Shred_%28Unix%29 Wikipedia]:|Shred is a Unix command that can be used to securely delete files and devices so that they can be recovered only with great difficulty with specialised hardware, if at all.}}
 +
 
 +
Shred uses three passes, writing pseudo-random data to the harddrive each pass. This can be reduced or increased.
 +
 
 +
# shred -v /dev/<drive>
 +
 
 +
Where {{ic|/dev/<drive>}} is the drive to be encrypted. This invokes shred with default settings, displaying the progress to stdout.
 +
 +
# shred --verbose --random-source=/dev/urandom -n1 /dev/<drive>
 +
 
 +
Invokes shred telling it to only do one pass, with entropy from /dev/urandom.
 +
 
 +
=== dd-spin-offs ===
 +
{{Moveto|Basic file operations|Did you ever want to write an article about dd and Co? Then just go ahead.}}
 +
 
 +
==== dcfldd ====
 +
{{Pkg|dcfldd}} is an enhanced version of dd with features useful for forensics and security. It accepts most of dd's parameters and includes status output. The last stable version of dcfldd was released on December 19, 2006.<sup>[http://dcfldd.sourceforge.net/]</sup>
 +
 
 +
==== ddrescue ====
 +
GNU {{Pkg|ddrescue}} is a data recovery tool. It's capable of ignoring read errors what is a useless feature for disk wiping in almost any case.
 +
[http://www.gnu.org/software/ddrescue/manual/ddrescue_manual.html GNU ddrescue Manual]
 +
 
 +
=== Badblocks ===
 +
{{Accuracy|Badblocks per default does test 64 KB in 1 KB blocks at a time. Here it's suggested up to now to test 10 MB in 1 KB blocks. Is something wrong with the general aim of the [[#Block size]] section to align blocksize to physical geometry and write it block by block? Or is this aimed at no-HDD Storage?}}
 +
 
 +
#badblocks -c 10240 -wsvt random /dev/<drive>
 +
 
 +
Where {{ic|/dev/<drive>}} is the drive to be encrypted.
 +
 
 +
{{Note|[[S.M.A.R.T.]] (Self-Monitoring, Analysis, and Reporting Technology) is featured in almost every HDD still in use nowadays.}}
  
 
== Select a target ==
 
== Select a target ==
Line 173: Line 188:
  
 
== See also ==
 
== See also ==
* [http://www.linuxquestions.org/questions/linux-newbie-8/learn-the-dd-command-362506/ Learn the DD command]
+
* [http://www.gnu.org/software/coreutils/manual/coreutils.html#Basic-operations GNU Coreutils Manpage on Basic operations]. Official documentation for dd and shred.
 +
 
 +
* [http://www.linuxquestions.org/questions/linux-newbie-8/learn-the-dd-command-362506/ Learn the DD command]. - linuxquestions.org

Revision as of 20:27, 26 September 2012

Template:Article summary start Template:Article summary text Template:Article summary heading Template:Article summary wiki Template:Article summary wiki Template:Article summary wiki Template:Article summary wiki Template:Article summary end

Wiping a disk is done by writing new data over every single bit.

Note: References to "disks" in this article also apply to loopback devices.

Common use cases

Wipe all data left on the device

There may be (possibly unencrypted) data left on the device and you want to protect against simple Forensic Investigation that would be possible with i.e. File Recovery-Software.

If you are not going to set up block device encryption but just want to roughly wipe everything from the disk you could consider using /dev/zero or simple patterns instead of a cryptographically strong random number generator. (Referred to as RNG in this article from now on.) This allows to wipe big disks with maximum performance and is meant to provide a level of data erasure not allowing to reconstruct data with normal system functions like standard ATA commands.

Nevertheless it is possible you might consider prefering the RNG-Method due to Security concerns. This is covered up in the Section about #Preparations for block device encryption.

Also read the section on the possibility of #Data remanence if you want to take wiping serious.

Preparations for block device encryption

If you want to prepare your drive to securely set up Block device encryption inside the wiped area afterwards you really should use random data.

Warning: If Block device encryption is mapped on a partition that contains anything else than random/encrypted data, disclosure of usage patterns on the encrypted drive is possible and weakens the encryption the kind of it does for filesystem-level-encryption. Do never use /dev/zero, simple patterns (badblocks, eg.) or other unrandom data before setting up Block device encryption if you are serious about it!

Select a data source for overwriting

Template:Moveto There are three sources of random data commonly used for securely overwriting hard disk partitions; /dev/urandom, badblocks, and shred.

dd and /dev/urandom

#dd if=/dev/urandom of=/dev/<drive> bs=4096

Where /dev/<drive> is the drive to be encrypted.

Note: Using /dev/urandom will take a long time to completely overwrite a drive with "random" data. In the strictest sense, /dev/urandom is less random than /dev/random; however, /dev/random uses the kernel entropy pool and will halt overwriting until more input entropy once this pool has been exhausted. This makes the use of /dev/random for overwriting hard disks impractical.
Note: Users may also find that /dev/urandom takes an excessively long time on large drives of several hundred gigabytes or more (more than twenty-four hours). Frandom offers a faster alternative.
Note: You can retrieve progress of the dd command with this command: kill -USR1 $(pidof dd)

pattern write test

Note: The badblocks command overwrites the drive at a much faster rate by generating data that is not truly random.

See also #Badblocks.

Performance

Note: Everything regarding Benchmarking disk wipes should get merged there.

For Data that is not truely random your disk's writing speed should be the only limiting factor. If you need random data performance may heavily depend on what you choose as source of randomness.

/dev/random

The Kernel built-in RNG /dev/random provides you the same quality random data you would use for keygeneration, but can be nearly impractical to use at least for wiping current HDD capacitys. What makes disk wiping take so long with is to wait for it to gather enough true entropy. In an entropy starved situation (e.g. remote server) this might never end while doing search operations on large directories or if your at your desktop running a first person shooter can speed up things a lot.

You can always compare /proc/sys/kernel/random/entropy_avail against /proc/sys/kernel/random/poolsize to keep an eye on your entropy pool.

Pseudorandom Data

A Good Compromise between Performance and Security might be the use of a pseudorandom number generator (like Frandom) or a cryptographically secure pseudorandom number generator like Yarrow (FreeBSD/OS-X) or Fortuna (the intended successor of Yarrow)

Tip: The cryptsetup FAQ mentions a very simple procedure to use an existing dm-crypt-Volume to act as a simple PRNG and wipe all free space accesible trough the underlying partition. It is also claimed to protect against disclosure of usage patterns. dd if=/dev/zero of=/dev/mapper/luks Thats it! This will wipe all data written to your dm-crypt Volume.

Conclusion

If you want to wipe sensitive data you can use anything matching your needs.

If you want to setup block device encryption afterwards you should always wipe at least with Pseudorandom data.

As a matter of course the best wiping practice is to never write unencrypted data.

Select a program

coreutils

Official documentation for dd and shred is linked to under #See also.

dd

Template:Wikipedia

Note: cp does the same as dd without any operands but is not designed for more versatile disk wiping procedures.
Checking progress of dd while running

By default, there is no output of dd until the task has finished. With kill and the "USR1"-Signal you can force status output without actually killing the program. Open up a 2nd root terminal and issue the following command:

# killall -USR1 dd
Note: This will affect all other running dd-processes as well.

Or:

# kill -USR1 <PID_OF_dd_COMMAND>

For example:

# kill -USR1 $(pidof dd)

This causes the terminal in which dd is running to output the progress at the time the command was run. For example:

605+0 records in
605+0 records out
634388480 bytes (634 MB) copied, 8.17097 s, 77.6 MB/s

shred

From Wikipedia: Shred is a Unix command that can be used to securely delete files and devices so that they can be recovered only with great difficulty with specialised hardware, if at all.

Shred uses three passes, writing pseudo-random data to the harddrive each pass. This can be reduced or increased.

# shred -v /dev/<drive>

Where /dev/<drive> is the drive to be encrypted. This invokes shred with default settings, displaying the progress to stdout.

# shred --verbose --random-source=/dev/urandom -n1 /dev/<drive>

Invokes shred telling it to only do one pass, with entropy from /dev/urandom.

dd-spin-offs

Template:Moveto

dcfldd

dcfldd is an enhanced version of dd with features useful for forensics and security. It accepts most of dd's parameters and includes status output. The last stable version of dcfldd was released on December 19, 2006.[1]

ddrescue

GNU ddrescue is a data recovery tool. It's capable of ignoring read errors what is a useless feature for disk wiping in almost any case. GNU ddrescue Manual

Badblocks

Tango-inaccurate.pngThe factual accuracy of this article or section is disputed.Tango-inaccurate.png

Reason: Badblocks per default does test 64 KB in 1 KB blocks at a time. Here it's suggested up to now to test 10 MB in 1 KB blocks. Is something wrong with the general aim of the #Block size section to align blocksize to physical geometry and write it block by block? Or is this aimed at no-HDD Storage? (Discuss in Talk:Securely wipe disk#)
#badblocks -c 10240 -wsvt random /dev/<drive>

Where /dev/<drive> is the drive to be encrypted.

Note: S.M.A.R.T. (Self-Monitoring, Analysis, and Reporting Technology) is featured in almost every HDD still in use nowadays.

Select a target

Note: Fdisk will not work on GPT formatted devices. Use gdisk instead.

Use fdisk to locate all read/write devices. This will include USB drives if the user can access them. List the partition tables:

# fdisk -l

Check the output for lines that start with devices such as /dev/sda. For example:

Disk /dev/sdc: 4063 MB, 4063232000 bytes
125 heads, 62 sectors/track, 1024 cylinders
Units = cylinders of 7750 * 512 = 3968000 bytes
Disk identifier: 0x00000000

In the preceding example the USB thumb drive is listed as /dev/sdc.

Block size

Template:Wikipedia If you have a Advanced Format hard drive it is recommended that you specify a block size larger than the default 512 bytes. To speed up the overwriting process choose a block size matching your drive's physical geometry by appending the block size option to the dd command (i.e. bs=4096).

To quickly find the block size of the device issue the following command:

# dumpe2fs -h /dev/sdX | grep 'Block size:'

For more information read How to Find the Block Size on The Linux Information Project and the Block size section in the core GNU utilites manual.

Overwrite the disk

Warning: There is no confirmation regarding the sanity of this command so repeatedly check that the correct drive or partition has been targeted. Make certain that the of=... option points to the target drive and not to a system disk.

Zero-fill the disk by writing a zero byte to every addressable location on the disk using the /dev/zero stream.

# dcfldd if=/dev/zero of=/dev/sdX bs=4096

or the /dev/random stream:

# dcfldd if=/dev/urandom of=/dev/sdX bs=4096

The process is finished when dcfldd reports, No space left on device. For example:

18944 blocks (75776Mb) written.dcfldd:: No space left on device

Data remanence

Tango-view-fullscreen.pngThis article or section needs expansion.Tango-view-fullscreen.png

Reason: This section is too dependent on links to Wikipedia. Links to diverse and high quality resources should be added. (Discuss in Talk:Securely wipe disk#)
Template:Wikipedia

The residual representation of data may remain even after attempts have been made to remove or erase the data.

Residual data may be removed by writing random data to the disk or with more than one iteration. However, more than one iteration may not significantly decrease the ability to reconstruct the data of hard disk drives. For more information see Secure deletion: a single overwrite will do it or Overwriting Hard Drive Data: The Great Wiping Controversy.

If the data can be located on the disk and you can confirm that it has never been copied anywhere else, a random number generator provides a quick and thorough alternative.

Residual magnetism

Wiped hard disk drives and other magnetic storage can get disassembled in a cleanroom and then analyzed with equipment like a magnetic force microscope. This may allow the overwritten data to be reconstructed by analyzing the measured residual magnetics.

This method of data recovery for current HDD's is largely theoretical and would require substantial financial resources. Nevertheless degaussing is still practiced.

Old magnetic storage

Securely wiping old magnetic storage (e.g. floppy disks, magnetic tape) is much harder due to much lower memory storage density. Many iterations with random data might be needed to wipe any sensitive data. To ensure that data has been completely erased most resources advise physical destruction.

Flash memory

Like older magnetic storage, flash memory can be difficult to wipe because of wear leveling and transparent compression. For more information see Reliably Erasing Data From Flash-Based Solid State Drives.

Filesystem, operation system, programs

The operating system, executing programs or journaling file systems may copy your unencrypted data throughout the block device. However, this should only be relevant in conjunction with one of the above, because you are writing to plain disks.

See also