1. Secure hard disk sanitization
1.1
Introduction
1.2
Physical Destruction
1.3
Degaussing
1.4
Overwriting
1.4.1
Official overwriting standards
1.4.2
Bruce Schneier's 7-pass method
1.4.3
Peter Gutmann's 35-pass method
1.4.4
Roy Pfitzner's 33-random-pass method
1.4.5
Recommended Software
1.4.6
Systemic Problems with Overwriting: Write Caches
1.4.7
Security risk through Journaling File Systems
1.5
Executive Summary
1. Sanitizing Hard Disks
1.1 Introduction
Basically, there are 3 techniques for sanitizing hard disks:
i) Physical destruction
ii) Degaussing
iii) Overwriting the disk's data
In my opinion, only a properly carried out physical destruction
(smelting/pulverizing) of the platters
(=disks) of a hard disk guarantees 100% secure deletion of data from
HDDs.
To my knowledge, there is no scientific data that would indicate how
strong the magnetic
field would have to be in order to irrecoverably delete the
contents of a hard disk. This and the missing opportunity for most
people to generate strong magnetic fields in the range of 1 or several Tesla,
leads me to argue against Degaussing as a means of secure data removal.
Overwriting can be a higly secure method for HDD cleansing, but only if
the proper method with sufficient overwrite passes is employed.
According to Pfitzner R., 2003, at least 33 overwrite passes
with pseudo-random
data are required to irrecoverably destroy the contents of a hard disk.
Even intelligence
agencies with all their modern forensic lab equipment would
then be unable to recover meaningful data.
1.2 Physical destruction
Smelting
or pulverizing the platters of a hard drive seem to be the best ways to
get rid of your data forever. Complete abrasion of the surface of the
platters seems to be reliable too. For now, I don't know how many mm
you would have to grind.
Just breaking the platters in a few pieces is probably not enough. Dean
Devera (The
Difficulty of Data Annihilation from Disk Drives: or Exnihilation Made
Easy) remarks:
"It is exceedingly difficult, but not
impossible if we're dealing with relatively few pieces. Once
reassembled, high-powered magnetic microscopy could then be turned to
the media surface. [...] But the possibility of platter reconstruction
exists, however minute." (Devera D, 2001)
Speaking from my own experience, the platters of (modern) hard disks
are extremely robust, almost unbreakable with manual force.
Furthermore, the screws of the hard disk case are screwed so tightly,
that it takes minutes to open the case and free the platters. If the
RIAA is already knocking on your door, forget about physical
destruction because you simply don't have enough time.
If you can generate temperatures
that are high but not sufficient to smelt the platters, you might
consider the Curie
point. The Curie point is the temperature above which a ferromagnetic
material looses its ferromagnetic properties. It then becomes purely paramagnetic.
As far as I know, the platters of modern hard disks usually consist of aluminium
or
aluminium alloys (some are made of glass substrates).
Aluminium itself is paramagnetic, therefore thin iron oxide
or cobalt
layers are applied in order to get ferromagnetic abilities. The melting
point for aluminium is 933.47 Kelvin
(= 660.32°Celsius).
The Curie point for Cobalt lies at 1388 K and those for the various
iron oxides usually slightly below the Curie point for pure Iron
which is 1043 K.
Now the problem is that we don't really know what temperature we need
to reach the Curie point for the material our platter is made of.
Probably we don't even know what material/alloys we have to deal with.
E.g., if the platter consists of aluminium with a cobalt layer, you
would need at least 1388 K to paramagnetize the cobalt layer, but at
that temperature the Aluminium should be melting already.
In conclusion,
if you want to cook your hard disk platters, generate a temperature
that exceeds the melting point for the material you're handling. If you
cannot establish this, go with pulverization.
That physical destruction does have to be applied properly shows the
following report from the Computer
Crime Research Center:
"The detailed examination made it possible
to restore most files that the criminal tried to damage through
physical destruction of the computer hard disk. The technically correct
investigation materials including expert examination results allowed
proving guilty of Mr. F. and institute criminal proceedings against
him."
[Source]
1.3 Degaussing
As noted earlier, I don't know any scientific studies that
would tell us of what magnitude the magnetic field we're putting our
hard disk in should be, so that all data is irrecoverably destroyed.
Some people say that degaussing with a magnetic field strength
that is about 5 times the coercivity makes a recovery uneconomical (but
not impossible). (Dawson et al. 2003)
The German Federal Office for Information Security (BSI)
states that degaussers are principally not suited for securely
sanitizing hard disks. (BSI 1999)
>From what I know about hard disk technology and the
corresponding coercivity,
at least 1 or 2 Tesla are required for degaussing, but that's just my
speculation.
In addition to that, the overwhelming majority of users simply
does not have access to strong magnetic fields, except people in a Nuclear
Medicine or Physics Department.
Charles
Preston (who attended the FBI
National Academy and wrote the article "The Data Dilemma"
Security Management, February 1995) once answered a question on Privacy
Digest 4.06. Here's some short cutout:
"Degaussing (strong magnetic fields that
destroy patterns on the media) with a very strong magnetic wand or
strong degausser will make the data very expensive and difficult to
recover.
A report from the Institute
for Defense Analyses from several years ago stated that with
enough processing power and time, data could be recovered almost
regardless of the method used to erase it. The same report gave a rule
of thumb about the necessary strength of magnetic fields used to erase
data. If this holds true for newer media like high-density diskettes
and DAT
drives, it may be impossible to adequately erase this media, including
hard drives, with current degaussers."
1.4 Overwriting
The goal of overwriting is, to reorientate the magnetic
domains as often as possible, such that the remaining residual magnetism
does not allow any conclusions about the original data.
If you overwrite a 1 with a 1, the magnetic
flux density is higher (e.g. 1.05) than overwriting a 0 with
a 1 (e.g. 0.95). Through the use of Scanning
Probe or Magnetic
Force Microscopes (SPMs, MFMs) it is possible to recover data
that has been overwritten several times, especially if the overwrite
patterns are predefined and not random.
Overwriting is, if properly applied, a thoroughly secure way
of erasing data from hard disks (for secure methods -> Pfitzner
33-random-pass method).
Warning: There may occur errors in a hard
drive. AFAIK these bad blocks (a.k.a. clusters)/sectors/tracks
are then mapped and locked by the drive. I.e. no regular software will
then be able to access these bad sectors and information in these will
NOT be overwritten by what pattern ever. In some cases a
low-level-format with vendor specific boot media may also format bad
sectors, but I don't know that for sure.
To check for bad sectors on your hard drive, download the appropriate
diagnostics boot media from the manufacturer's site and let it scan
your drive for bad sectors.
1.4.1 Official overwriting standards
There are several different overwriting patterns, proposed by various
intelligence, the military
and government organisations.
It is unlikely that a government publishes a method that will protect
you from its law
enforcement and intelligence agencies at the date of
publication. E.g. the DoD 5220.22-M
pattern (3 passes) is not approved for sanitizing media that contains
secret or top-secret information by the DoD itself.
Renowned security expert Peter Gutmann
explains:
"The ... problem with official data
destruction standards is that the information in them may be partially
inaccurate in an attempt to fool opposing intelligence agencies (which
is probably why a great many guidelines on sanitizing media are classified)."
(Gutmann P, 1996)
Furthermore, most of these official overwriting standards
contain static elements, which means that a lot of passes consist of
overwriting with Ones or Zeroes or Complements. Later on, we will learn
that the best strategy is to apply a scrubbing with random
data to the drive.
Not recommended patterns include:
-US NAVSO
P-5239-26 (RLL) (3 passes)
-US NAVSO
P-5239-26 (MFM) (3 passes)
-US DoD
5220.22-M (8-306./E) (3 passes)
-US DoD
5220.22-M (8-306./E,C and E) (7 passes)
-US
AR 380-19 (3 passes)
-ACSI
33 605 (2 pass * n) ( Australia)
-BSI
GSHB M2.167 (3 passes) (Germany)
Comment: The original
NISPOM (National Industrial Security Program Operating Manual
= DoD 5220.22 M) was first published in 1995. When you look at Chapter 8
section 306, you can see the "Clearing and Sanitization Matrix" which
prescribes the well-known 3-pass pattern for hard drives. So far, so
good. Since then NISPOM has been changed twice: in July 1997 and
February 2001. After the 2001 change, the recommended 3-pass pattern
vanished. The updated NISPOM now only contains a general
description (8.301) of what Clearing(a) and Sanitization(b)
mean. The same holds true for the Army Regulation 380-19. The version
from August, 1st 1990 describes a 3-pass pattern for hard
disks, whereas the updated
version from February, 27th 1998 does not contain information
on specific overwriting patterns any more.
This might indicate that the progress in hard disk technology with
increasing densities
causes considerable problems to those trying to recover deleted
information. Maybe progress in computer
forensics is much slower than required to fill the gap.
1.4.2 Bruce Schneier's 7-pass method
Bruce Schneier
in his book "Applied Cryptography" (1996):
"Most commercial programs that claim to
implement the DoD standard overwrite three times: first with all ones,
then with all zeros, and finally with a repeating one-zero pattern.
Given my general level of paranoia, I recommend overwriting a deleted
file seven times: the first time with all ones, the second time with
all zeros, and five times with a cryptographically
securepseudo-random
sequence." (Schneier B, 1996)
First, it must be criticized that Schneier doesn't explain why he
recommends the described 7-pass pattern. "Given my general level of paranoia"
is all we get to know.
Second, he himself admits:
"Recent developments at the National Institute of Standards
and Technology with electron-tunnelling microscopes suggest
even that might not be enough. Honestly, if your data is sufficiently
valuable, assume that it is impossible to erase data completely off
magnetic media. Burn or shred the media; it's cheaper to buy media new
than to lose your secrets." (Schneier B, 1996)
1.4.3 Peter Gutmann's 35-pass method
Here is Peter Gutmann's statement in the updated paper Secure
Deletion of Data from Magnetic and Solid-State Memory:
Epilogue
In the
time since this paper was published, some people have treated the
35-pass overwrite technique described in it more as a kind of voodoo
incantation to banish evil spirits than the result of a technical
analysis of drive encoding techniques. As a result, they advocate
applying the voodoo to PRML and EPRML drives even though it will have
no more effect than a simple scrubbing with random data. In fact
performing the full 35-pass overwrite is pointless for any drive since
it targets a blend of scenarios involving all types of (normally-used)
encoding technology, which covers everything back to 30+-year-old MFM
methods (if you don't understand that statement, re-read the paper). If
you're using a drive which uses encoding technology X, you only need to
perform the passes specific to X, and you never need to perform all 35
passes.
For any modern PRML/EPRML drive, a few passes of random scrubbing is
the best you can do. As the paper says, "A good scrubbing with random
data will do about as well as can be expected". This was true in 1996,
and is still true now.
Looking at
this from the other point of view, with the ever-increasing data
density on disk platters and a corresponding reduction in feature size
and use of exotic techniques to record data on the medium, it's
unlikely that anything can be recovered from any recent drive except
perhaps one or two levels via basic error-cancelling techniques. In
particular the the drives in use at the time that this paper was
originally written have mostly fallen out of use, so the methods that
applied specifically to the older, lower-density technology don't apply
any more. Conversely, with modern high-density drives, even if you've
got 10KB of sensitive data on a drive and can't erase it with 100%
certainty, the chances of an adversary being able to find the erased
traces of that 10KB in 80GB of other erased traces are close to zero.
I.e. overwriting with the Gutmann pattern makes only sense if you have
a hard drive that uses MFM/RLL
encoding. In general, MFM was used on most drives before IDE.
The initial RLL encodings were used sinced the late 1980s but are no
longer in use today. Meanwhile, many new and complex encoding
techniques have evolved, such as PRML, EPRML, EEPRML, Trellis and MTR
codes.
Therefore it is impossible to create a perfect wipe pattern for all
(modern) hard drives. As Gutmann said, the best you can do today is
scrubbing with random data.
1.4.4 Roy Pfitzner's 33-random-pass method
In 2003, Dipl-Ing. Roy Pfitzner, a German IT security expert, wrote an
explosive paper about the secure removal of data:
Pfitzner, R.: Sicheres Löschen
von Dateien – Standards, Löschtools, Empfehlungen.
Der Landesbeauftragte für den
Datenschutz und für das Recht auf Akteneinsicht Brandenburg,
Internes Arbeitspapier, 2003.
This translates to:
Pfitzner, R.: Secure Deletion of Files -
Standards, Erasure Tools, Recommendations.
The Federal Commissioner for Privacy
Protection and the Right for access records Brandenburg,
internal working paper, 2003.
As far as I know, Mr Pfitzner then worked for the Commissioner for
Privacy Protection of the federal state Brandenburg. In 2003
(or 2004), he asked to be transferrred to a superior rank within the
Department of Strategic Planning and Innovation in the Department of the
Interior of the Federal State Brandenburg. (The Spiegel news
report said that he also worked for Interior when he wrote the paper.)
Until today, his paper is classified and thus not available to the
general public. Fortunately there are 2 documents that give us at least
some information from his paper.
The first document
is a report by German news magazine Der Spiegel in
December 2003 (52/2003).
According to the journalists, Pfitzner said that data could be
retrieved even if it was overwritten 20 times and that one would have
to overwrite more than 30 times with random data to achieve a security
level that would defy the capabilities of law enforcement and
intelligence agencies.
The second document
is an orientation guide on secure data removal:
Original title:
Orientierungshilfe
„Sicheres Löschen magnetischer
Datenträger“
Grundlagen,
Werkzeuge und Empfehlungen aus Sicht des Datenschutzes
Translation:
Orientation guide "Secure Sanitization
of magnetic data storage media"
Basics, Tools and Recommendations for
Data Privacy Purposes
It has been published in October 2004 by the working committee
"Technical and Organizational Issues in Privacy Protection" of the
Conference of Commissioners for Privacy Protection of the federal
states of Germany and the national Commissioner.
This orientation guide is a review of methods for the secure deletion
of data from hard disks and other media. Most of the information is
based on Pfitzner's classified paper (it's cited).
According to the guide, the classified Pfitzner paper says:
- 33 overwrite passes
with random data are sufficient such that with a
probability of 0,99 every magnetic domain gets re-orientated at least
twice -> Very
High security level
- 7 overwrite passes
with random data are sufficient such that with a
probability of 0,99 every magnetic domain gets re-orientated at least
once -> Medium
security level
Maybe Pfitzner, who then and now works for the state government, had
access to classified papers and resources that allowed him to evaluate
modern recovery techniques and their limits.
The orientation guide encompasses an evaluation of other methods as
well, probably this assessment is from Pfitzners classified paper:
| Method |
Number
of overwrite passes |
Use of random numbers |
Consideration
of different
encoding techniques |
Protection against
extensive
laboratory analyses |
| Single Pass 0 or 1 |
1 |
no |
no |
very low |
| BSI GSHB |
4 till 6 |
no |
no |
low |
| VS IT-Richtlinien |
7 |
no |
no |
low |
| DoD 5220.22-M |
3 |
yes (1x) |
no |
low |
| DoD 5220.22-M ECE |
7 |
yes (3x) |
no |
medium |
| Gutmann |
35 |
yes (8x) |
yes (27x) |
very high |
| Pfitzner |
33 |
yes (33x) |
no |
very high |
Note: Remember that the lack of considering different disk encoding
techniques is not a disadvantage with modern hard drives -> also
see 1.4.3
Additionally, here's a statement from David H. Schultz ("Beyond
Fingerprints") who writes in a foot note:
"xxii Until recently computer
forensic and data recovery specialists agreed that data overwritten a
total of nine times would ensure the data could not be recovered. With
the development of Magnetic Force Microscopy, however, data can be
recovered even after the hard disk has been overwritten a dozen or more
times. The ability to detect each "layer" becomes progressively more
difficult to recover the older it is. Michael Overly, Overly on
Electronic Evidence in California, p. 2-25. (Eagan, MN: West Group,
1999)."
1.4.5 Recommended Software
Software for Overwriting should meet the following criteria:
- support for overwriting with
pseudo-random data
- number of overwrite passes should be
freely determined by the user
- Open Source
Although the importance of an open source code for sanitization
purposes is not so important as it is with cryptographic
software, I would not recommend software that is closed source. There
is always the possibility that it contains a flaw or (intentionally
implemented) security weakness.
If you perform a single overwrite pass with Ones or Zeroes, you can
easily check with an Hex editor after the erasure procedure. But with
multiple passes with pseudo-random data it's much harder to tell
whether the program performed all passes as you told it to do or not.
The orientation guide "Secure Sanitization of magnetic data storage
media" lists the following software:
- Eraser
(WinNT/2000/XP, Win9x) by Sami
Tolvanen, Garrett
Trant, Bill Johnson
Hint: Eraser does not automatically recognize which files/folders may
contain sensitive information.
The very valuable tool CCleaner
is able to detect the location of many sensitive files:
You can then create new "Tasks" within Eraser to wipe those sensitive
areas that are recognized by CCleaner. You will have to use CCleaner
for Registry cleaning, remember to use a Registry compacting tool like
RegCompact.NET afterwards. Another recommended tool is MRUblaster.
- DBAN (boot and
nuke program) by Darik
Horn
- Wipe (Linux)
by Tom Vier
- dd:
Unix/Linux
command. Note that this only properly works when erasing complete partitions/hard
disks. E.g.:
root@localhost:~> dd if=/dev/random of=/dev/hda1
There is also a number of commercial counter-forensic privacy tools
like Evidence Eliminator, East-Tec Eraser (also advertised as
Cyberscrub), Window Washer etc. These programs often claim to erase all
information about the users's computer usage, like records of created
documents, visited websites, viewed image files, downloaded files,
installed and executed programs and so on.
In June 2005, the first scientific evaluation of wiping tools was
published by Matthew Geiger and Lorrie Faith Cranor:
Counter-Forensic Privacy Tools - A Forensic Evaluation.
The results were devastating and clearly showed that all examined
programs failed to completely wipe all sensitive information (more
footprint icons indicate greater exposure):
In 2002, Kurt Seifried demonstrated that commercial products often miss
information stored in alternate data-streams within the NTFS: Kurt
Seifried Security Advisory 003 (KSSA-003)
1.4.6 Systemic Problems with Overwriting: Write Caches
One problem of overwriting that has not been addressed so far concerns
the caches of hard disks. These are used for buffering read/write
commands. Operating Systems may also use some sort of caching/buffering
for better performance. AFAIK Windows XP uses a write cache by
default.
In the worst case, if you tell a wiping program like Eraser to
overwrite a file 33 times, the disk cache may buffer the first 32
overwrite passes and only perform the last one.
You can and should deactivate Write Caching by selecting Properties of
your Hard disk within the Device Manager. If you cannot do this within
windows then you have to set it in the BIOS or use special software
from the HDD manufacturer.
According to Darik Horn, "DBAN
makes long writes to ensure that hardware buffers are written through."
DBAN optionally can verify all passes, which takes about double the
time than without verifying.
If you're not sure whether the write cache has been properly
deactivated, you can do the following:
Start your PC and boot DBAN, then perform a 1-pass overwrite with
pseudo-random data. When finished, reset your PC, boot DBAN and repeat
the 1-pass overwrite. You will have to do that 33 times to reach the
very high security level of the Pfitzner Method.
1.4.7 Security risk through Journaling File Systems
Many modern operating
systems such as Windows XP (NTFS), Mac OS X ( HFS Plus ), and GNU/Linux
with a kernel version greater than 2.4 (Ext3, JFS, ReiserFS, and XFS)
have the ability to use a journaling filesystem that makes complete
erasure of data unlikely. Journaling filesystems are used to increase
the integrity of data in case of failures. To accomplish this, the
filesystems keep meta data and logs in various places known to the
filesystem; most filesystems can also journal all data, but turn this
functionality off by default. The meta data and logs will not be
securely wiped with a file wiping tool. To increase performance, these
filesystems will often arrange I/O commands in an efficient manner and
may continuously move data around the disk to prevent the need for
operations similar to Windows scandisk. The performance enhancing
capabilities of the filesystems makes wiping files hard because the
data may only be wiped in its present location, leaving unwiped blocks
of the data in other locations on the hard disk. Also, the filesystem
may not execute all requests of a redundant I/O command.
File Wipe from Wikipedia
Currently I don't know whether it is possible to turn off any
journaling features of NTFS.
What you can and should do in any case is disabling Window's own System
Restore.
Solutions:
1. store your data on a partition that employs a non-journaling file
system, e.g. FAT32 -> problem: FAT32 doesn't have the security
features and other benefits from a modern FS like NTFS
2. store as much data as possible on encrypted partitions/file
containers and wipe the root partition regularly
1.5 Executive Summary
1. Both physical destruction and overwriting can provide a very high
security level, given that both methods are done properly. Degaussing
as a means for hard disk sanitization is not recommended.
2. In order to achieve a proper physical destruction, complete smelting
or pulverization of the platters have to be accomplished.
3. In order to gain a very high security level with overwriting, the
Pfitzner Method has to be applied, i.e. overwriting with 33 passes of
pseudo-random data. If only a medium security level is needed, 7 passes
with pseudo-random data are sufficient. Software recovery tools can be
thwarted with 1 pass pseudo-random data. Official overwriting
standards, Bruce Schneier's 7-pass and Peter Gutmann's 35-pass method
are not recommended. However, the Gutmann method may make sense with
very old drives (drives that use MFM or RLL encoding).
If the hard disk contains bad sectors, a secure sanitization may be
impossible if sensitive information is stored in these bad sectors. It
is recommended to scan the drive for bad sectors with vendor diagnostic
boot media.
4. Make sure that all write caches of your drive / OS are deactivated.
Also see -> 1.4.6
5. The higher the density of a hard drive, the harder the task of
recovery is -> modern hard drives with extremely high densities
are more easily sanitized than older hard disks.
6. In most cases, it is not possible to remove all traces of computer
activity. Usually some information is always missed. Sensitive
information is often found through:
- swap file
- hibernation
file
- Windows system restore
- Windows registry
- creation of temporary files by the OS
(operating system) or applications
- creation of working and backup copies
(e.g. .bak/.sik ending) of files by the OS or applications
- traces of sensitive files within OS
files
- files that are manually encrypted and
not properly erased subsequently
- metadata
of file systems: file name, time stamps for creation/change
of files, access rights
- alternate data streams (ADS) of NTFS:
ADS allow to secretly store information bound to a specific file. You can find links to helpful tools in the
Wikipedia article about Fork
(filesystem).
- journals of journaling
file systems -> also see 1.4.7
- RAID systems
- external backups: unencrypted data on
CDs/DVDs, external HDDs, floppies etc.
Therefore it is highly recommended to completely erase whole partitions
and not only single files/folders.
7. Because of 6., the recommended software tool for disk sanitization
is DBAN by Darik Horn
8. Because a secure disk sanitization is time consuming, it is highly
recommended to use an open source on-the-fly
encryption program for as much data as possible. Using the
Pfitzner Method for encrypted data may pose an impossible to crack task
for the next decades.
9. Take into account that a post-mortem analysis of your phyiscal
memory (RAM) may yield sensitive information as well. Recommended
Reading: Data
Remanence in Semiconductor Devices by Peter Gutmann (yeah,
Gutmann strikes again :D
References:
Bundesamt
für Sicherheit in der Informationstechnik: Wiederaufbereiten
von VS-Datenträgern. Hinweisblatt Nr. 11 zur Umsetzung von
§ 12 der VS IT-Richtlinien des Bundesministerium des Innern.
21.11.1999
Dawson
M, Forgie C, Davis J and Tauber S, Data Recovery, CSSE 592/492
Computer Forensics
May 7th, 2003
Devera
D: The Difficulty of Data Annihilation from Disk Drives: or
Exnihilation Made Easy. (December, 11th 2001)
Garfinkel
S and Shelat A: Remembrance of Data passed: A disk sanitization study.
IEEE Security and Privacy, January/February 2003. pp 17-27
Gutmann P: Secure Deletion of Data from Magnetic and Solid-State Memory.
Proceedings of The Sixth USENIX Security Symposium (1996). pp 77-90
Gutmann
P: Secure Deletion of Data from Magnetic and Solid-State Memory.
updated paper
Gutmann
P: Data Remanence in Semiconductor Devices. Proceedings of
the 10th USENIX Security Symposium (2001). pp 39-54.
Schneier
B: Applied Cryptography. John Wiley & Sons, Inc.
(1996). 2nd ed.
