Difference between revisions of "Securely wipe disk"

From ArchWiki
Jump to: navigation, search
(Overwrite the disk: change 4MB in 4KB-sector size / remove Template:Accuracy / link to disk sector and Advanced Format instead of moved(?) Wikipedia:Block size#Block size)
m (Kernel built-in RNG: typo)
(47 intermediate revisions by 3 users not shown)
Line 5: Line 5:
 
{{Article summary heading|Related}}
 
{{Article summary heading|Related}}
 
{{Article summary wiki|File Recovery}}
 
{{Article summary wiki|File Recovery}}
 +
{{Article summary wiki|Benchmarking disk wipes}}
 +
{{Article summary wiki|Frandom}}
 
{{Article summary wiki|Disk Encryption#Preparing the disk}}
 
{{Article summary wiki|Disk Encryption#Preparing the disk}}
 
{{Article summary wiki|dm-crypt with LUKS}}
 
{{Article summary wiki|dm-crypt with LUKS}}
{{Article summary wiki|Frandom}}
 
 
{{Article summary end}}
 
{{Article summary end}}
  
Wiping a disk is done by simply writing new data over every single bit.
+
Wiping a disk is done by writing new data over every single bit.
  
{{Tip|Where refered to "disks" in this article you can of course apply the same procedure for your loopback-devices or anything else.}}
+
{{Note|References to "disks" in this article also apply to loopback devices.}}
  
As mentioned in the [[Disk_Encryption#Preparing_the_disk|Disk Encryption]]-Article there might be different scenarios for why you want to wipe a disk.
+
== Common use cases ==
 +
=== Wipe all data left on the device ===
 +
The most common usecase for completely and irrevocably wiping a device will be when the device it going to be given away or also maybe sold. There may be (unencrypted) data left on the device and you want to protect against simple forensic investigation that is mere child's play with for example [[File Recovery]] software.
  
'''Wipe all data left on the device.'''
+
If you want to quickly wipe everything from the disk /dev/zero or simple patterns allow maximum performance while adequate randomness can be advantageous in some cases that should be covered up in [[#Data remanence]].
There may be (possibly unencrypted) data left on the device and you want to protect against simple Forensic Investigation that would be possible with i.e. [[File Recovery]]-Software.
+
  
'''Preparation for block device encryption.'''
+
Every overwritten bit means to provide a level of data erasure not allowing recovery with normal system functions (like standard ATA/SCSI commands) and hardware interfaces. Any file recovery software mentioned above then would need to be specialized on proprietary storage-hardware features.
You want to prepare your drive to securely set up [[Disk_Encryption#Block_device_encryption|Block device encryption]] inside the wiped area afterwards.
+
  
== Introduction  ==
+
In case of a HDD data recreation will not be possible without at least undocumented drive commands or fiddling about the device’s controller or firmware to make them read out for example reallocated sectors (bad blocks that [[S.M.A.R.T.]] retired from use).
=== Wipe all data left on the device ===
+
If you are not going to set up block device encryption but just want to roughly wipe everything from the disk you could consider using /dev/zero or simple patterns instead of a cryptographically strong [[Wikipedia:Random_number_generator|random number generator]]. (Referred to as RNG in this article from now on.) This allows to wipe big disks with maximum performance.
+
 
+
However you might consider prefering the RNG-Method due to Security concerns. This is covered up in the Section about [[#Preparations for block device encryption]].
+
  
Also read the section on the possibility of [[#Data remanence]] if you want to take wiping serious.
+
There are different wiping issues with different physical storage technologys, most notably all Flash memory based devices and older magnetic storage (old HDD's, floppy disks, tape).
  
 
=== Preparations for block device encryption ===
 
=== Preparations for block device encryption ===
{{Warning|If Block device encryption is mapped on a partition that contains anything else than random/encrypted data, disclosure of usage patterns on the encrypted drive is possible and weakens the encryption the kind of it does for filesystem-level-encryption. Do never use /dev/zero, simple patterns (badblocks, eg.) or other unrandomn data before setting up Block device encryption if you are serious about it!}}
+
If you want to prepare your drive to securely set up [[Disk Encryption#Block device encryption]] inside the wiped area afterwards you really should use [[#Random data]] generated by a trusted cryptographically strong random number generator (referred to as RNG in this article from now on).
  
==== Performance ====
+
{{Wikipedia|Random number generation}}
The Kernel built-in RNG [[Wikipedia:/dev/random|/dev/random]] provides you the same quality random data you would use for keygeneration, but can be nearly impractical to use at least for wiping current HDD capacitys.
+
What makes disk wiping take so long with is to wait for it to [[Wikipedia:Hardware_random_number_generator#Using_observed_events|gather enough true entropy]].
+
In an entropy starved situation (e.g. remote server) this might never end while doing search operations on large directories or if your at your desktop running a first person shooter can speed up things a lot.
+
  
You can always compare {{ic|/proc/sys/kernel/random/entropy_avail}} against {{ic|/proc/sys/kernel/random/poolsize}} to keep an eye on your entropy pool.
+
{{Warning|If Block device encryption is mapped on a partition that contains anything else than random/encrypted data, disclosure of usage patterns on the encrypted drive is possible and weakens the encryption being comparable with filesystem-level-encryption. Do never use /dev/zero, simple patterns (badblocks, eg.) or other unrandom data before setting up Block device encryption if you are serious about it!}}
  
{{Note|Everything regarding [[Benchmarking disk wipes]] should be merged there}}
+
== Select a data source for overwriting ==
  
==== Pseudorandom Data ====
+
As just said If you want to wipe sensitive data you can use anything matching your needs.
A Good Compromise between Performance and Security might be the use of a [[Wikipedia:Pseudorandom_number_generator|pseudorandom number generator]] (like [[Frandom]]) or a [[Wikipedia:Cryptographically_secure_pseudorandom_number_generator|cryptographically secure pseudorandom number generator]]
+
like [[Wikipedia:Yarrow_algorithm|Yarrow]] (FreeBSD/OS-X) or [[Wikipedia:Fortuna_(PRNG)|Fortuna]] (the intended successor of Yarrow)
+
  
{{Tip|The [http://code.google.com/p/cryptsetup/wiki/FrequentlyAskedQuestions cryptsetup FAQ] mentions a very simple procedure to use an existing dm-crypt-Volume to act as a simple PRNG and wipe all free space accesible trough the underlying partition. It is also claimed to protect against disclosure of usage patterns. {{ic|1= dd if=/dev/zero of=/dev/'''mapper'''/luks}} Thats it! Obviously this will wipe all data written to your dm-crypt Volume.}}
+
If you want to setup block device encryption afterwards you should always wipe at least with Pseudorandom data.
  
=== Conclusion ===
+
For data that is not truly random your disk's writing speed should be the only limiting factor. If you need random data, the required system performance to generate it may extremely depend on what you choose as source of entropy.
If you want to wipe sensitive data you can use anything matching your needs.
+
  
If you want to setup block device encryption you should always wipe at least with Pseudorandom data.
+
{{Note|Everything regarding [[Benchmarking disk wipes]] should get merged there.}}
  
As a matter of course the best wiping practice is to never write unencrypted data.
+
=== Unrandom data ===
 +
Overwriting with {{ic|/dev/zero}} or simple patterns is considered secure in most resources. In the case of current HDD's it should be sufficient for fast disk wipes.
  
== Select a target ==
+
{{Warning|A drive that is abnormally fast in writing patterns or zeroing could be doing transparent compression. It is obviously presumable not all blocks get wiped this way. Some [[#Flash memory]] devices do "feature" that.}}
{{Note|Fdisk will not work on [[GUID Partition Table|GPT]] formatted devices. Use {{Pkg|gdisk}} instead.}}
+
Use fdisk to locate all read/write devices. This will include USB drives if the user can access them. List the partition tables:
+
# fdisk -l
+
  
Check the output for lines that start with devices such as {{ic|/dev/sda}}. For example:
+
==== Pattern write test ====
Disk /dev/sdc: 4063 MB, 4063232000 bytes
+
[[#Badblocks]] can write simple patterns to every block of a device and then read and check them searching for damaged areas (just like memtest86* does with memory).
125 heads, 62 sectors/track, 1024 cylinders
+
Units = cylinders of 7750 * 512 = 3968000 bytes
+
Disk identifier: 0x00000000
+
In the preceding example the USB thumb drive is listed as {{ic|/dev/sdc}}.
+
+
== Overwrite the disk ==
+
If you have a modern hard drive it is recommended that you specify a block size larger than the default 512 bytes. To speed up the overwriting process choose a block size matching your drive's physical geometry by appending the block size option to the dd command (i.e. {{ic|<nowiki>bs=4096</nowiki>}} for [[Wikipedia:Advanced Format|Advanced Format]]-disks). For more information read the [[Wikipedia:Disk_sector|disk sector]] Article on Wikipedia.
+
  
{{Accuracy|Does dumpe2fs recognize 512e-discs? (4K physical sectors on the drive media with 512 byte logical emulation)}}
+
As the pattern is written to every accesible block this effectively wipes the device.
To quickly find the block size of the device issue the following command:
+
# dumpe2fs -h /dev/sdX | grep 'Block size:'
+
For more information read [http://www.linfo.org/get_block_size.html How to Find the Block Size] on The Linux Information Project.
+
  
{{warning|There is no confirmation regarding the sanity of this command so '''repeatedly check''' that the correct drive or partition has been targeted. Make certain that the {{ic|<nowiki>of=...</nowiki>}} option points to the target drive and not to a system disk.}}
+
=== Random data ===
 +
{{Note|Data that is hard to compress (random data) will get written slower, if the drive logic mentioned in the [[#Unrandom data]] warning tries compressing it. This should not lead to [[#Data remanence]] though. As maximum write-speed is not the performance-bottleneck it can get completely neglected while wiping disks with random data.}}
  
Zero-fill the disk by writing a zero byte to every addressable location on the disk using the [[Wikipedia:/dev/zero|/dev/zero]] stream.
+
==== Kernel built-in RNG ====
 +
The kernel build in RNG's /dev/(u)[[random]] are highly recommended for producing reliable random data providing the same security level that is used for the creation of cryptographic keys.
  
# dcfldd if=/dev/zero of=/dev/sdX bs=4096
+
; /dev/random: uses an entropy pool of 4096 bits (512 Bytes) to generate random data and stops when the pool is exhausted until it get's (slowly) refilled. /dev/random is absolutely not designed for wiping entire HDD's.
 +
; /dev/urandom: reuses entropy and although not suited for the most crucial cryptographic purposes, for example the generation of longterm keys, its quality should be sufficient for a paranoid disk wipe, [[#Preparations for block device encryption|preparing for block device encryption]], wiping LUKS keyslots, wiping single files and many other purposes.
  
or the [[Wikipedia:/dev/random|/dev/random]] stream:
+
For much better performance consider using a true [[random#Pseudorandom_number_generator|pseudorandom number generator]].
# dcfldd if=/dev/urandom of=/dev/sdX bs=4096
+
 
 +
== Select a program ==
 +
{{ic|/dev/<drive>}} is the drive to be encrypted.
 +
 
 +
=== Coreutils ===
 +
{{Merge|Core_Utilities|Basic file operations are '''not specific to disk wiping!''' Unrelated stuff in this section should get merged and then deleted and replaced with a link to [[Core Utilities]]. Did you ever want to write an article about dd and Co? Then just go ahead.}}
  
The process is finished when dcfldd reports, {{ic|No space left on device}}. For example:
+
Official documentation for dd and shred is linked to under [[#See also]].
18944 blocks (75776Mb) written.dcfldd:: No space left on device
+
  
=== Common tools ===
+
==== Dd ====
There are a variety of applications that can be used to wipe disks like [[Wikipedia:Dd_(Unix)#Disk_wipe|dd]] or [[Wikipedia:Shred_(Unix)|shred]] (both are in {{Pkg|util-linux}}).
+
{{Wikipedia|Dd_(Unix)}}
 +
{{Note|cp does the same as dd without any operands but is not designed for more versatile disk wiping procedures.}}
  
==== Checking progress of dd while running ====
+
===== Checking progress of dd while running =====
By default, there is no output of dd until the task has finished.  With kill and the "USR1"-Signal you can force status output without actually killing the program. Simply open up a 2nd root terminal and issue the following command:
+
By default, there is no output of dd until the task has finished.  With kill and the "USR1"-Signal you can force status output without actually killing the program. Open up a 2nd root terminal and issue the following command:
 
  # killall -USR1 dd
 
  # killall -USR1 dd
{{Note|Obviously this will affect all other running dd-processes as well.}}
+
{{Note|This will affect all other running dd-processes as well.}}
 
Or:
 
Or:
 
  # kill -USR1 <PID_OF_dd_COMMAND>
 
  # kill -USR1 <PID_OF_dd_COMMAND>
Line 105: Line 92:
 
  634388480 bytes (634 MB) copied, 8.17097 s, 77.6 MB/s
 
  634388480 bytes (634 MB) copied, 8.17097 s, 77.6 MB/s
  
==== Dcfldd ====
+
===== Dd spin-offs =====
[http://dcfldd.sourceforge.net/ dcfldd] is an enhanced version of dd with features useful for forensics and security. It accepts most of dd's parameters and includes status output. Install {{Pkg|dcfldd}} from the [[official repositories]].
+
Other dd alike programs feature periodical status output like i.e. a simple progress bar.
{{Note|Dcfldd's source has been untouched since 19.12.2006.}}
+
 
 +
'''dcfldd'''
 +
 
 +
{{Pkg|dcfldd}} is an enhanced version of dd with features useful for forensics and security. It accepts most of dd's parameters and includes status output. The last stable version of dcfldd was released on December 19, 2006.<sup>[http://dcfldd.sourceforge.net/]</sup>
 +
 
 +
'''ddrescue'''
 +
 
 +
GNU {{Pkg|ddrescue}} is a data recovery tool. It's capable of ignoring read errors what is a useless feature for disk wiping in almost any case.
 +
[http://www.gnu.org/software/ddrescue/manual/ddrescue_manual.html GNU ddrescue Manual]
 +
 
 +
==== shred ====
 +
{{Box BLUE|From [http://en.wikipedia.org/wiki/Shred_%28Unix%29 Wikipedia]:|Shred is a Unix command that can be used to securely delete files and devices so that they can be recovered only with great difficulty with specialised hardware, if at all.}}
 +
 
 +
Shred uses three passes, writing pseudo-random data to the harddrive each pass. This can be reduced or increased.
 +
 
 +
# shred -v /dev/<drive>
 +
 
 +
This invokes shred with default settings, displaying the progress to stdout.
 +
 +
# shred --verbose --random-source=/dev/urandom -n1 /dev/<drive>
 +
 
 +
Invokes shred telling it to only do one pass, with entropy from /dev/urandom.
 +
 
 +
=== Badblocks ===
 +
For letting [[Badblocks#read-write_Test|badblocks perform a disk wipe]] a destructive read-write test has to be done.
 +
 
 +
# badblocks -c 10240 -wsv /dev/<drive>
 +
 
 +
== Select a target ==
 +
{{Note|Fdisk will not work on [[GUID Partition Table|GPT]] formatted devices. Use {{Pkg|gdisk}} instead.}}
 +
Use fdisk to locate all read/write devices the user has read acess to.
 +
 
 +
Check the output for lines that start with devices such as {{ic|/dev/sdX}}.
 +
 
 +
This is an example for a HDD formatted to boot a linux system:
 +
 
 +
{{hc|# fdisk -l|<nowiki>Disk /dev/sda: 250.1 GB, 250059350016 bytes, 488397168 sectors
 +
Units = sectors of 1 * 512 = 512 bytes
 +
Sector size (logical/physical): 512 bytes / 512 bytes
 +
I/O size (minimum/optimal): 512 bytes / 512 bytes
 +
Disk identifier: 0x00ff784a
 +
 
 +
  Device Boot      Start        End      Blocks  Id  System
 +
/dev/sda1  *        2048      206847      102400  83  Linux
 +
/dev/sda2          206848  488397167  244095160  83  Linux</nowiki>}}
 +
 
 +
Or the Arch Install Medium written to a 4GB USB thumb drive:
 +
 
 +
{{hc|# fdisk -l|<nowiki>Disk /dev/sdb: 4075 MB, 4075290624 bytes, 7959552 sectors
 +
Units = sectors of 1 * 512 = 512 bytes
 +
Sector size (logical/physical): 512 bytes / 512 bytes
 +
I/O size (minimum/optimal): 512 bytes / 512 bytes
 +
Disk identifier: 0x526e236e
 +
 
 +
  Device Boot      Start        End      Blocks  Id  System
 +
/dev/sdb1  *          0      802815      401408  17  Hidden HPFS/NTFS</nowiki>}}
 +
 
 +
=== Block size ===
 +
{{Wikipedia|Dd (Unix)#Block size}}
 +
If you have a [[Wikipedia:Advanced Format|Advanced Format]] hard drive it is recommended that you specify a block size larger than the default 512 bytes. To speed up the overwriting process choose a block size matching your drive's physical geometry by appending the block size option to the dd command (i.e. {{ic|<nowiki>bs=4096</nowiki>}} for 4KB).
 +
 
 +
Fdisk prints physical and logical sector size for every disk.
 +
 
 +
Alternatively sysfs does expose information:
 +
/sys/block/sdX/queue/physical_block_size
 +
/sys/block/sdX/queue/logical_block_size
 +
/sys/block/sdX/alignment_offset
 +
 
 +
== Overwrite the disk ==
 +
{{warning|There is no confirmation regarding the sanity of this command so '''repeatedly check''' that the correct drive or partition has been targeted. Make certain that the {{ic|<nowiki>of=...</nowiki>}} option points to the target drive and not to a system disk.}}
 +
 
 +
Zero-fill the disk by writing a zero byte to every addressable location on the disk using the [[Wikipedia:/dev/zero|/dev/zero]] stream.
 +
 
 +
# dd if=/dev/zero of=/dev/sdX bs=4096
 +
 
 +
or the [[Wikipedia:/dev/random|/dev/random]] stream:
 +
# dd if=/dev/urandom of=/dev/sdX bs=4096
 +
 
 +
The process is finished when dd reports, {{ic|No space left on device}}:
 +
dd: writing to ‘/dev/sdb’: No space left on device
 +
7959553+0 records in
 +
7959552+0 records out
 +
4075290624 bytes (4.1 GB) copied, 1247.7 s, 3.3 MB/s
  
 
== Data remanence ==
 
== Data remanence ==
[[Wikipedia:Data_remanence|Data remanence]] after wiping is possible for a bunch of reasons.
+
{{Expansion|This section is too dependent on links to Wikipedia. Links to diverse and high quality resources should be added.}}
{{Expansion|Reliable good resources should be linked instead of that much Wikipedia Articles.}}
+
  
===== residual magnetism =====
+
{{Wikipedia|Data remanence}}
Wiped HDD and other magnetic storage can get disassembled in a cleanroom and then analyzed with HighTech Equipment like a [[Wikipedia:Magnetic_force_microscope|magnetic force microscope]] to guess on the overwritten data by computing around the measured [[Wikipedia:Remanence|residual magnetics]].
+
The residual representation of data may remain even after attempts have been made to remove or erase the data.
  
{{Note|This is more kind of theoretical for current HDD's at the moment and assumes an attacker extremely heavy on financial resources. Nevertheless [[Wikipedia:Degaussing#Degaussing_magnetic_data_storage_media|degaussing]] is still practiced.}}
+
Residual data may get wiped by writing (random) data to the disk with a single or even more than one iteration. However, more than one iteration may not significantly decrease the possibility to reconstruct the data of hard disk drives. For more information see [http://www.h-online.com/news/Secure-deletion-a-single-overwrite-will-do-it--/112432 Secure deletion: a single overwrite will do it - The H Security].
To seriously make sure you wiped it out for ever you nevertheless might want to use random data and if you think you will really be able to sleep better feel free to give it more than one iteration.
+
  
{{Note|In case your disk is a ordinary HDD going trough more than one iteration may not significantly decrease the ability to reconstruct the data. (Resources: [http://www.h-online.com/news/Secure-deletion-a-single-overwrite-will-do-it--/112432 Secure deletion: a single overwrite will do it], [http://www.springerlink.com/content/408263ql11460147/ Overwriting Hard Drive Data: The Great Wiping Controversy]).}}
+
=== Random data ===
 +
If the data can get exactly located on the disk and was never copied anywhere else, wiping with random data can be thoroughgoing and impressively quick as long there is enough entropy in the pool.
  
{{Tip|If you know exactly what you want to wipe and where it's located on the disk and that it has never been copied anywhere else you can do very secure wiping in precious few seconds with any RNG.}}
+
A good example is cryptsetup using /dev/urandom for [[Dm-crypt_with_LUKS#Wipe_LUKS_keyslots|wiping the LUKS keyslots]].
  
===== older magnetic storage =====
+
=== Hardware specific issues ===
Secure wiping of any "older" magnetic storage (especially Floppys, Magnetic Tape, ...) is much harder due to much lower [[Wikipedia:Memory storage density|memory storage density]]. Many iterations with random data might be needed to wipe any sensitive data ever copied there. Most resources even advise physical destruction in addition if you want to be really sure!
+
==== Flash memory ====
 +
[[Wikipedia:Write amplification]] and other characteristics make Flash memory a stubborn target for reliable wiping.
 +
As there is a lot of transparent abstraction in between data as seen by a device's controller chip and the operating system sight data is never overwritten in place and wiping particular blocks or files is not reliable.
  
===== Flash-Storage =====
+
Other "features" like transparent compression (all SandForce SSD's) can compress your /dev/zero or pattern stream so if wiping is fast beyond belief this might be the case.
But not only old Media can be hard to wipe. Flash-Storage behaves very unforgiving as well due to [[Wikipedia:Wear_leveling|wear leveling]], transparent compression, ...
+
  
'''Resources:'''
+
Disassembling Flash memory devices, unsoldering the chips and analyzing data content without the controller in between is feasible without difficulty using [http://www.flash-extractor.com/manual/reader_models/ simple hardware]. Data recovery companys do it for cheap money.
  
[http://www.usenix.org/events/fast11/tech/full_papers/Wei.pdf Reliably Erasing Data From Flash-Based Solid State Drives]
+
For more information see: [http://www.usenix.org/events/fast11/tech/full_papers/Wei.pdf Reliably Erasing Data From Flash-Based Solid State Drives].
  
===== Filesystem, OS, Programms =====
+
==== Residual magnetism ====
The OS, executed Programs or a (journaling) filesystem also can do a lot to copy and spread your unencrypted data over the block device, but because you are writing to plain disks this should only be relevant in conjunction with one of the above.
+
Wiped hard disk drives and other magnetic storage can get disassembled in a cleanroom and then analyzed with equipment like a [[Wikipedia:Magnetic force microscope|magnetic force microscope]]. This may allow the overwritten data to be reconstructed by analyzing the measured [[Wikipedia:Remanence|residual magnetics]].
 +
 
 +
This method of data recovery for current HDD's is largely theoretical and would require substantial financial resources. Nevertheless [[Wikipedia:Degaussing#Degaussing magnetic data storage media|degaussing]] is still a practiced countermeasure.
 +
 
 +
==== Old magnetic storage ====
 +
Securely wiping old magnetic storage (e.g. floppy disks, magnetic tape) is much harder due to much lower [[Wikipedia:Memory storage density|memory storage density]]. Many iterations with random data might be needed to wipe any sensitive data. To ensure that data has been completely erased most resources advise physical destruction.
 +
 
 +
==== Operating system, programs and filesystem ====
 +
{{Note|This is not specific to any hardware obviously.}}
 +
The operating system, executed programs or [[Wikipedia:Journaling file system|journaling file system]]s may copy your unencrypted data throughout the block device. When writing to plain disks this should only be relevant in conjunction with one of the above.
  
 
== See also ==
 
== See also ==
* [http://www.linuxquestions.org/questions/linux-newbie-8/learn-the-dd-command-362506/ Learn the DD command]
+
* [http://www.gnu.org/software/coreutils/manual/coreutils.html#Basic-operations GNU Coreutils Manpage on Basic operations]. Official documentation for dd and shred.
 +
 
 +
* [http://www.linuxquestions.org/questions/linux-newbie-8/learn-the-dd-command-362506/ Learn the DD command]. - linuxquestions.org

Revision as of 21:16, 2 October 2012

Summary help replacing me
Wipe all traces left from (un-)encrypted data and/or prepare for block device encryption
Related
File Recovery
Benchmarking disk wipes
Frandom
Disk Encryption#Preparing the disk
dm-crypt with LUKS

Wiping a disk is done by writing new data over every single bit.

Note: References to "disks" in this article also apply to loopback devices.

Common use cases

Wipe all data left on the device

The most common usecase for completely and irrevocably wiping a device will be when the device it going to be given away or also maybe sold. There may be (unencrypted) data left on the device and you want to protect against simple forensic investigation that is mere child's play with for example File Recovery software.

If you want to quickly wipe everything from the disk /dev/zero or simple patterns allow maximum performance while adequate randomness can be advantageous in some cases that should be covered up in #Data remanence.

Every overwritten bit means to provide a level of data erasure not allowing recovery with normal system functions (like standard ATA/SCSI commands) and hardware interfaces. Any file recovery software mentioned above then would need to be specialized on proprietary storage-hardware features.

In case of a HDD data recreation will not be possible without at least undocumented drive commands or fiddling about the device’s controller or firmware to make them read out for example reallocated sectors (bad blocks that S.M.A.R.T. retired from use).

There are different wiping issues with different physical storage technologys, most notably all Flash memory based devices and older magnetic storage (old HDD's, floppy disks, tape).

Preparations for block device encryption

If you want to prepare your drive to securely set up Disk Encryption#Block device encryption inside the wiped area afterwards you really should use #Random data generated by a trusted cryptographically strong random number generator (referred to as RNG in this article from now on).

Template:Wikipedia

Warning: If Block device encryption is mapped on a partition that contains anything else than random/encrypted data, disclosure of usage patterns on the encrypted drive is possible and weakens the encryption being comparable with filesystem-level-encryption. Do never use /dev/zero, simple patterns (badblocks, eg.) or other unrandom data before setting up Block device encryption if you are serious about it!

Select a data source for overwriting

As just said If you want to wipe sensitive data you can use anything matching your needs.

If you want to setup block device encryption afterwards you should always wipe at least with Pseudorandom data.

For data that is not truly random your disk's writing speed should be the only limiting factor. If you need random data, the required system performance to generate it may extremely depend on what you choose as source of entropy.

Note: Everything regarding Benchmarking disk wipes should get merged there.

Unrandom data

Overwriting with /dev/zero or simple patterns is considered secure in most resources. In the case of current HDD's it should be sufficient for fast disk wipes.

Warning: A drive that is abnormally fast in writing patterns or zeroing could be doing transparent compression. It is obviously presumable not all blocks get wiped this way. Some #Flash memory devices do "feature" that.

Pattern write test

#Badblocks can write simple patterns to every block of a device and then read and check them searching for damaged areas (just like memtest86* does with memory).

As the pattern is written to every accesible block this effectively wipes the device.

Random data

Note: Data that is hard to compress (random data) will get written slower, if the drive logic mentioned in the #Unrandom data warning tries compressing it. This should not lead to #Data remanence though. As maximum write-speed is not the performance-bottleneck it can get completely neglected while wiping disks with random data.

Kernel built-in RNG

The kernel build in RNG's /dev/(u)random are highly recommended for producing reliable random data providing the same security level that is used for the creation of cryptographic keys.

/dev/random
uses an entropy pool of 4096 bits (512 Bytes) to generate random data and stops when the pool is exhausted until it get's (slowly) refilled. /dev/random is absolutely not designed for wiping entire HDD's.
/dev/urandom
reuses entropy and although not suited for the most crucial cryptographic purposes, for example the generation of longterm keys, its quality should be sufficient for a paranoid disk wipe, preparing for block device encryption, wiping LUKS keyslots, wiping single files and many other purposes.

For much better performance consider using a true pseudorandom number generator.

Select a program

/dev/<drive> is the drive to be encrypted.

Coreutils

Merge-arrows-2.pngThis article or section is a candidate for merging with Core_Utilities.Merge-arrows-2.png

Notes: Basic file operations are not specific to disk wiping! Unrelated stuff in this section should get merged and then deleted and replaced with a link to Core Utilities. Did you ever want to write an article about dd and Co? Then just go ahead. (Discuss in Talk:Securely wipe disk#)

Official documentation for dd and shred is linked to under #See also.

Dd

Template:Wikipedia

Note: cp does the same as dd without any operands but is not designed for more versatile disk wiping procedures.
Checking progress of dd while running

By default, there is no output of dd until the task has finished. With kill and the "USR1"-Signal you can force status output without actually killing the program. Open up a 2nd root terminal and issue the following command:

# killall -USR1 dd
Note: This will affect all other running dd-processes as well.

Or:

# kill -USR1 <PID_OF_dd_COMMAND>

For example:

# kill -USR1 $(pidof dd)

This causes the terminal in which dd is running to output the progress at the time the command was run. For example:

605+0 records in
605+0 records out
634388480 bytes (634 MB) copied, 8.17097 s, 77.6 MB/s
Dd spin-offs

Other dd alike programs feature periodical status output like i.e. a simple progress bar.

dcfldd

dcfldd is an enhanced version of dd with features useful for forensics and security. It accepts most of dd's parameters and includes status output. The last stable version of dcfldd was released on December 19, 2006.[1]

ddrescue

GNU ddrescue is a data recovery tool. It's capable of ignoring read errors what is a useless feature for disk wiping in almost any case. GNU ddrescue Manual

shred

From Wikipedia: Shred is a Unix command that can be used to securely delete files and devices so that they can be recovered only with great difficulty with specialised hardware, if at all.

Shred uses three passes, writing pseudo-random data to the harddrive each pass. This can be reduced or increased.

# shred -v /dev/<drive>

This invokes shred with default settings, displaying the progress to stdout.

# shred --verbose --random-source=/dev/urandom -n1 /dev/<drive>

Invokes shred telling it to only do one pass, with entropy from /dev/urandom.

Badblocks

For letting badblocks perform a disk wipe a destructive read-write test has to be done.

# badblocks -c 10240 -wsv /dev/<drive>

Select a target

Note: Fdisk will not work on GPT formatted devices. Use gdisk instead.

Use fdisk to locate all read/write devices the user has read acess to.

Check the output for lines that start with devices such as /dev/sdX.

This is an example for a HDD formatted to boot a linux system:

# fdisk -l
Disk /dev/sda: 250.1 GB, 250059350016 bytes, 488397168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00ff784a

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048      206847      102400   83  Linux
/dev/sda2          206848   488397167   244095160   83  Linux

Or the Arch Install Medium written to a 4GB USB thumb drive:

# fdisk -l
Disk /dev/sdb: 4075 MB, 4075290624 bytes, 7959552 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x526e236e

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *           0      802815      401408   17  Hidden HPFS/NTFS

Block size

Template:Wikipedia If you have a Advanced Format hard drive it is recommended that you specify a block size larger than the default 512 bytes. To speed up the overwriting process choose a block size matching your drive's physical geometry by appending the block size option to the dd command (i.e. bs=4096 for 4KB).

Fdisk prints physical and logical sector size for every disk.

Alternatively sysfs does expose information:

/sys/block/sdX/queue/physical_block_size
/sys/block/sdX/queue/logical_block_size
/sys/block/sdX/alignment_offset

Overwrite the disk

Warning: There is no confirmation regarding the sanity of this command so repeatedly check that the correct drive or partition has been targeted. Make certain that the of=... option points to the target drive and not to a system disk.

Zero-fill the disk by writing a zero byte to every addressable location on the disk using the /dev/zero stream.

# dd if=/dev/zero of=/dev/sdX bs=4096

or the /dev/random stream:

# dd if=/dev/urandom of=/dev/sdX bs=4096

The process is finished when dd reports, No space left on device:

dd: writing to ‘/dev/sdb’: No space left on device
7959553+0 records in
7959552+0 records out
4075290624 bytes (4.1 GB) copied, 1247.7 s, 3.3 MB/s

Data remanence

Tango-view-fullscreen.pngThis article or section needs expansion.Tango-view-fullscreen.png

Reason: This section is too dependent on links to Wikipedia. Links to diverse and high quality resources should be added. (Discuss in Talk:Securely wipe disk#)
Template:Wikipedia

The residual representation of data may remain even after attempts have been made to remove or erase the data.

Residual data may get wiped by writing (random) data to the disk with a single or even more than one iteration. However, more than one iteration may not significantly decrease the possibility to reconstruct the data of hard disk drives. For more information see Secure deletion: a single overwrite will do it - The H Security.

Random data

If the data can get exactly located on the disk and was never copied anywhere else, wiping with random data can be thoroughgoing and impressively quick as long there is enough entropy in the pool.

A good example is cryptsetup using /dev/urandom for wiping the LUKS keyslots.

Hardware specific issues

Flash memory

Wikipedia:Write amplification and other characteristics make Flash memory a stubborn target for reliable wiping. As there is a lot of transparent abstraction in between data as seen by a device's controller chip and the operating system sight data is never overwritten in place and wiping particular blocks or files is not reliable.

Other "features" like transparent compression (all SandForce SSD's) can compress your /dev/zero or pattern stream so if wiping is fast beyond belief this might be the case.

Disassembling Flash memory devices, unsoldering the chips and analyzing data content without the controller in between is feasible without difficulty using simple hardware. Data recovery companys do it for cheap money.

For more information see: Reliably Erasing Data From Flash-Based Solid State Drives.

Residual magnetism

Wiped hard disk drives and other magnetic storage can get disassembled in a cleanroom and then analyzed with equipment like a magnetic force microscope. This may allow the overwritten data to be reconstructed by analyzing the measured residual magnetics.

This method of data recovery for current HDD's is largely theoretical and would require substantial financial resources. Nevertheless degaussing is still a practiced countermeasure.

Old magnetic storage

Securely wiping old magnetic storage (e.g. floppy disks, magnetic tape) is much harder due to much lower memory storage density. Many iterations with random data might be needed to wipe any sensitive data. To ensure that data has been completely erased most resources advise physical destruction.

Operating system, programs and filesystem

Note: This is not specific to any hardware obviously.

The operating system, executed programs or journaling file systems may copy your unencrypted data throughout the block device. When writing to plain disks this should only be relevant in conjunction with one of the above.

See also