https://wiki.archlinux.org/api.php?action=feedcontributions&user=MaBeef&feedformat=atomArchWiki - User contributions [en]2024-03-28T19:03:11ZUser contributionsMediaWiki 1.41.0https://wiki.archlinux.org/index.php?title=Identify_damaged_files&diff=210767Identify damaged files2012-06-21T01:09:02Z<p>MaBeef: </p>
<hr />
<div>[[Category:Storage]]<br />
This article gives details on how to find out which file owns a given block. The main purpose for doing so is finding out which file was damaged in the event a storage device develops any bad blocks (that way you'll know if you lost anything important).<br />
<br />
For most of these commands you will have to be either root or a user that has direct read access to the drive you are checking (being a member of the disk group should be enough). As usual, a current backup is always a good idea, especially if imminent drive failure is suspected.[[S.M.A.R.T.]] can help with determining that.<br />
<br />
Right now this article is only written for JFS and EXT filesystems.<br />
<br />
==Finding Bad Blocks==<br />
Just use the [[badblocks]] command. There are a few scan modes supported by it. There's read-only mode (default) which is the least accurate. There is the destructive write-mode (-w option) which is the most accurate but takes longer and will (obviously) destroy all data on the drive, thus making it quite useless for matching blocks up to files. There is finally the non-destructive read-write mode which is probably as accurate as the destructive mode, the only real downside of which is it's probably the slowest. However, if a drive is known to be failing then read-only mode is probably still the safest.<br />
<br />
To do a verbose (-v option), read-only scan, run one of these commands (with x being the drive letter and y being partition number you want to scan):<br />
<br />
Whole disk scan:<br />
badblocks -v /dev/sdx<br />
<br />
Single partition scan:<br />
badblocks -v /dev/sdxy<br />
<br />
The downside to scanning the drive as a whole is that each filesystem is going to start its block count relative to the partition it's on. This means that if you have a bad block that happens to be on, let's say, the second partition, and that partition starts on block 1000, then you will have to subtract 1000 from your block number in order to get the number you want. So if a scan from the beginning of the disk results in block number 1005 being bad, then block 5 of the second partition is what you'll actually be using.<br />
<br />
Otherwise, if you've found bad blocks after doing a full scan, you can simply figure out which partitions they're on, and rescan those in order to get the block number, rather than do any block math.<br />
<br />
Another thing to note is that badblocks defaults to 1024 byte blocks so you will either have to change the default size with the -b option in order to match your filesystem or manually convert the block number(s) later.<br />
<br />
If you need to figure out where your partitions start and end run fdisk (older versions might have defaulted to cylinders, not sure. If so the -u option will change the default unit to sectors). Make sure to note the block size fdisk is using so that you can convert the block counts to match your scan.<br />
<br />
fdisk -l /dev/sdx<br />
<br />
255 heads, 63 sectors/track, 19457 cylinders, total 312581808 sectors<br />
'''Units = sectors of 1 * 512 = 512 bytes'''<br />
Sector size (logical/physical): 512 bytes / 512 bytes<br />
I/O size (minimum/optimal): 512 bytes / 512 bytes<br />
Disk identifier: 0x00000000<br />
<br />
After all this, you should have the block number(s) of your bad block(s), relative to the partition they exist on.<br />
<br />
==Debug the Filesystem (JFS)==<br />
jfs_debugfs will give you access to all the low level structures within any JFS filesystem. Other filesystems such as the EXT filesystems have similar tools. It is probably a good idea to umount any filesystem before you run this on them. To use it just run:<br />
<br />
jfs_debugfs /dev/sdxy<br />
<br />
This puts you into a command console. The first thing you should note is your aggregate block size. This is (presumably) the block size the filesystem is using. JFS seems to default to 4096 bytes.<br />
<br />
If you did not run badblocks using the block size that your filesystem is using then you will need to convert your block number(s) to match it (remember to use the block number(s) relative to the partition they're on).<br />
<br />
i.e. block number 100 with a block size of 1024 bytes becomes block number 25 at 4096 bytes. The formula is:<br />
<br />
(original block number) / ((filesystem block size) / (badblocks block size))<br />
<br />
Now the entire point of running this program (for the purpose of this article) is to get the inode number. To do this run the command:<br />
<br />
d ''blocknumber'' 0 i<br />
<br />
The syntax is the d command for display, the block number, the offset (just set it to 0), and the display format i for inode.<br />
<br />
''Note'': If you get an error then that means the block is not allocated and is being used as free space. In that case this is a good thing as it means nothing important was damaged.<br />
<br />
The decimal number that di_number is set to is the one we want. From here you type x to exit out of the display mode. Repeat the display command for each bad block that you have and note all of their inode numbers. For more info on the inode such as permissions and filetype type:<br />
<br />
i ''inodenumber''<br />
<br />
When you have all the inode numbers type q to quit.<br />
<br />
==Find Damaged Files (JFS/Universal)==<br />
<br />
Finally to find the damaged file you can simply use the gnu find utility. Mount your filesystem and run:<br />
<br />
find / -inum ''inodenumber''<br />
<br />
Substitute "/" for the mountpoint of the filesystem that the inode belongs to. If you search root and have more than one filesystem mounted (who doesn't?) you can find multiple files with the same inode number on different filesystems, plus find will take significantly longer. Remember, an inode is only unique to the filesystem that it's in.<br />
<br />
==Debug the Filesystem (EXT(2/3/4))==<br />
tune2fs will give you access to all the low level structures within any EXT filesystem. It is probably a good idea to umount any filesystem before you run this on them.<br />
<br />
The first thing we want to do is get the block size from the filessystem in question. Just run:<br />
<br />
tune2fs -l /dev/sdxy | grep Block<br />
Block count: 29119820<br />
'''Block size: 4096'''<br />
<br />
In this case 4096 is the block size being used (it appears to be the default).<br />
<br />
If you did not run badblocks using the block size that your filesystem is using then you will need to convert your block number(s) to match it (remember to use the block number(s) relative to the partition they're on).<br />
<br />
i.e. block number 100 with a block size of 1024 bytes becomes block number 25 at 4096 bytes. The formula is:<br />
<br />
(original block number) / ((filesystem block size) / (badblocks block size))<br />
<br />
Now the entire point of running this program (for the purpose of this article) is to get the inode number. To do this run the command:<br />
<br />
debugfs<br />
<br />
Then in the debugfs console, use the open command on the EXT partition containing the bad sector:<br />
<br />
debugfs: open /dev/sdxy<br />
<br />
Finally, use the testb command to get information about the block in question (in this example block 1000):<br />
<br />
debugfs: testb ''inodenumber''<br />
<br />
<br />
''Note'': If debugfs says that block isn't in use then that means the block is not allocated and is being used as free space. In that case this is a good thing as it means nothing important was damaged.<br />
<br />
If the block is in use then run this command to get the inode number<br />
<br />
icheck ''inodenumber''<br />
<br />
This will return two numbers. The block number and the inode number.<br />
<br />
==Find Damaged Files (EXT(2/3/4))==<br />
Use the inode number (second number from the icheck command) with the ncheck command:<br />
<br />
ncheck ''inodenumber''<br />
<br />
debugfs will give you the full pathname to the file using the bad block. Now you will know what was actually damaged.<br />
<br />
If the inode number is very small and ncheck fails to return a path then it's probably the journal itself that is damaged. To delete the journal simply run this command on the partition:<br />
<br />
tune2fs -O ^has_journal /dev/sdxy<br />
<br />
Run the testb command again from the debugfs console on the bad block and it should be no longer marked as used if it was indeed used by the journal. To build a new journal run:<br />
<br />
tune2fs -j /dev/sdxy<br />
<br />
==Force the disk to reallocate bad block==<br />
First you'll want to see how many badblocks the harddrive is aware of through the smartctl command:<br />
<br />
smartctl -t long /dev/sdx [wait until test completes, then]<br />
smartctl -l selftest /dev/sdx<br />
<br />
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE<br />
'''5 Reallocated_Sector_Ct 0x0033 100 100 005 Pre-fail Always - 0'''<br />
'''196 Reallocated_Event_Count 0x0032 100 100 000 Old_age Always - 0'''<br />
197 Current_Pending_Sector 0x0022 100 100 000 Old_age Always - 0<br />
'''198 Offline_Uncorrectable 0x0008 100 100 000 Old_age Offline - 0'''<br />
<br />
<br />
To make the harddrive transparently map out the badblock with a spare good sector you will have to simply write zeros to the bad block using the dd command as root. Remember that with this command you have to work with the same block size as your filesystem and the block as to be relative to the partition the filesystem is on and NOT the harddrive as a whole:<br />
<br />
dd if=/dev/zero of=/dev/sdxy bs=4096 count=1 seek=2269012<br />
sync<br />
<br />
You can see if the harddrive did indeed map out an additional bad sector by checking with the smartctl command and seeing if the reallocated sector or event count went up:<br />
<br />
smartctl -A /dev/sdx<br />
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE<br />
'''5 Reallocated_Sector_Ct 0x0033 100 100 005 Pre-fail Always - 1'''<br />
'''196 Reallocated_Event_Count 0x0032 100 100 000 Old_age Always - 1'''<br />
197 Current_Pending_Sector 0x0022 100 100 000 Old_age Always - 0<br />
'''198 Offline_Uncorrectable 0x0008 100 100 000 Old_age Offline - 1'''<br />
<br />
To get Offline_Uncorrectable to go back to 0 you need to run a SMART long test and a selftest:<br />
<br />
smartctl -t long /dev/sdx [wait until test completes, then]<br />
smartctl -l selftest /dev/sdx<br />
<br />
==See also==<br />
<br />
[http://smartmontools.sourceforge.net/badblockhowto.html EXT2/3 badblocks howto]</div>MaBeefhttps://wiki.archlinux.org/index.php?title=Identify_damaged_files&diff=210766Identify damaged files2012-06-21T01:00:43Z<p>MaBeef: </p>
<hr />
<div>[[Category:Storage]]<br />
This article gives details on how to find out which file owns a given block. The main purpose for doing so is finding out which file was damaged in the event a storage device develops any bad blocks (that way you'll know if you lost anything important).<br />
<br />
For most of these commands you will have to be either root or a user that has direct read access to the drive you are checking (being a member of the disk group should be enough). As usual, a current backup is always a good idea, especially if imminent drive failure is suspected.[[S.M.A.R.T.]] can help with determining that.<br />
<br />
Right now this article is only written for JFS and EXT filesystems.<br />
<br />
==Finding Bad Blocks==<br />
Just use the [[badblocks]] command. There are a few scan modes supported by it. There's read-only mode (default) which is the least accurate. There is the destructive write-mode (-w option) which is the most accurate but takes longer and will (obviously) destroy all data on the drive, thus making it quite useless for matching blocks up to files. There is finally the non-destructive read-write mode which is probably as accurate as the destructive mode, the only real downside of which is it's probably the slowest. However, if a drive is known to be failing then read-only mode is probably still the safest.<br />
<br />
To do a verbose (-v option), read-only scan, run one of these commands (with x being the drive letter and y being partition number you want to scan):<br />
<br />
Whole disk scan:<br />
badblocks -v /dev/sdx<br />
<br />
Single partition scan:<br />
badblocks -v /dev/sdxy<br />
<br />
The downside to scanning the drive as a whole is that each filesystem is going to start its block count relative to the partition it's on. This means that if you have a bad block that happens to be on, let's say, the second partition, and that partition starts on block 1000, then you will have to subtract 1000 from your block number in order to get the number you want. So if a scan from the beginning of the disk results in block number 1005 being bad, then block 5 of the second partition is what you'll actually be using.<br />
<br />
Otherwise, if you've found bad blocks after doing a full scan, you can simply figure out which partitions they're on, and rescan those in order to get the block number, rather than do any block math.<br />
<br />
Another thing to note is that badblocks defaults to 1024 byte blocks so you will either have to change the default size with the -b option in order to match your filesystem or manually convert the block number(s) later.<br />
<br />
If you need to figure out where your partitions start and end run fdisk (older versions might have defaulted to cylinders, not sure. If so the -u option will change the default unit to sectors). Make sure to note the block size fdisk is using so that you can convert the block counts to match your scan.<br />
<br />
fdisk -l /dev/sdx<br />
<br />
255 heads, 63 sectors/track, 19457 cylinders, total 312581808 sectors<br />
'''Units = sectors of 1 * 512 = 512 bytes'''<br />
Sector size (logical/physical): 512 bytes / 512 bytes<br />
I/O size (minimum/optimal): 512 bytes / 512 bytes<br />
Disk identifier: 0x00000000<br />
<br />
After all this, you should have the block number(s) of your bad block(s), relative to the partition they exist on.<br />
<br />
==Debug the Filesystem (JFS)==<br />
jfs_debugfs will give you access to all the low level structures within any JFS filesystem. Other filesystems such as the EXT filesystems have similar tools. It is probably a good idea to umount any filesystem before you run this on them. To use it just run:<br />
<br />
jfs_debugfs /dev/sdxy<br />
<br />
This puts you into a command console. The first thing you should note is your aggregate block size. This is (presumably) the block size the filesystem is using. JFS seems to default to 4096 bytes. If you did not run badblocks using the block size that your filesystem is using then you will need to convert your block number(s) to match it (remember to use the block number(s) relative to the partition it's on).<br />
<br />
i.e. block number 100 with a block size of 1024 bytes becomes block number 25 at 4096 bytes. The formula is:<br />
<br />
(original block number) / ((filesystem block size) / (badblocks block size))<br />
<br />
Now the entire point of running this program (for the purpose of this article) is to get the inode number. To do this run the command:<br />
<br />
d ''blocknumber'' 0 i<br />
<br />
The syntax is the d command for display, the block number, the offset (just set it to 0), and the display format i for inode.<br />
<br />
''Note'': If you get an error then that means the block is not allocated and is being used as free space. In this case this is a good thing as it means nothing important was damaged by that sector.<br />
<br />
The decimal number that di_number is set to is the one we want. From here you type x to exit out of the display mode. Repeat the display command for each bad block that you have and note all of their inode numbers. For more info on the inode such as permissions and filetype type:<br />
<br />
i ''inodenumber''<br />
<br />
When you have all the inode numbers type q to quit then skip ahead to the "Debug the Filesystem" section.<br />
<br />
==Find Damaged Files (JFS)==<br />
<br />
Finally to find the damaged file you can simply use the gnu find utility. Mount your filesystem and run:<br />
<br />
find / -inum ''inodenumber''<br />
<br />
Substitute "/" for the mountpoint of the filesystem that the inode belongs to. If you search root and have more than one filesystem mounted (who doesn't?) you can find multiple files with the same inode number on different filesystems. Remember, an inode is only unique to the filesystem that it's in.<br />
<br />
==Debug the Filesystem (EXT(2/3/4))==<br />
tune2fs will give you access to all the low level structures within any EXT filesystem. It is probably a good idea to umount any filesystem before you run this on them.<br />
<br />
The first thing we want to do is get the block size from the filessystem in question. Just run (probably as root):<br />
<br />
tune2fs -l /dev/sdxy | grep Block<br />
Block count: 29119820<br />
'''Block size: 4096'''<br />
<br />
In this case 4096 is the block size being used (it appears to be the default). If you did not run badblocks using block size that your filesystem is using then you will need to convert your block numbers to match it (i.e. block number 100 with a block size of 1024 bytes becomes block number 25 at 4096 bytes).<br />
<br />
Now the entire point of running this program for the purpose of this article is to get the inode number. To do this run the command (as root):<br />
<br />
debugfs<br />
<br />
Then in the debugfs console, use the open command on the EXT partition containing the bad sector:<br />
<br />
debugfs: open /dev/sdxy<br />
<br />
Finally, use the testb command to get information about the block in question (in this example block 1000):<br />
<br />
debugfs: testb 1000<br />
<br />
If debugfs says that block isn't in use then that means the block is not allocated and is being used as free space. In this case this is a good thing as it means nothing important was damaged by that sector.<br />
<br />
If the block is in use then run this command to get the inode number<br />
<br />
icheck 1000<br />
<br />
This will return two numbers. The block number and the inode number.<br />
<br />
==Find Damaged Files (EXT(2/3/4))==<br />
Use the inode number (second number from the icheck command) with the ncheck command:<br />
<br />
ncheck 12345<br />
<br />
debugfs will give you the full pathname to the file using the bad block. Now you will know what was actually damaged.<br />
<br />
If the inode number is very small and ncheck fails to return a path then it's probably the journal itself that is damaged. To delete the journal simply run this command on the partition:<br />
<br />
tune2fs -O ^has_journal /dev/sdxy<br />
<br />
Run the testb command again from the debugfs console on the bad block and it should be no longer marked as used if it was indeed used by the journal. To build a new journal run:<br />
<br />
tune2fs -j /dev/sdxy<br />
<br />
==Force the disk to reallocate bad block==<br />
First you'll want to see how many badblocks the harddrive is aware of through the smartctl command:<br />
<br />
smartctl -t long /dev/sdx [wait until test completes, then]<br />
smartctl -l selftest /dev/sdx<br />
<br />
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE<br />
'''5 Reallocated_Sector_Ct 0x0033 100 100 005 Pre-fail Always - 0'''<br />
'''196 Reallocated_Event_Count 0x0032 100 100 000 Old_age Always - 0'''<br />
197 Current_Pending_Sector 0x0022 100 100 000 Old_age Always - 0<br />
'''198 Offline_Uncorrectable 0x0008 100 100 000 Old_age Offline - 0'''<br />
<br />
<br />
To make the harddrive transparently map out the badblock with a spare good sector you will have to simply write zeros to the bad block using the dd command as root. Remember that with this command you have to work with the same block size as your filesystem and the block as to be relative to the partition the filesystem is on and NOT the harddrive as a whole:<br />
<br />
dd if=/dev/zero of=/dev/sdxy bs=4096 count=1 seek=2269012<br />
sync<br />
<br />
You can see if the harddrive did indeed map out an additional bad sector by checking with the smartctl command and seeing if the reallocated sector or event count went up:<br />
<br />
smartctl -A /dev/sdx<br />
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE<br />
'''5 Reallocated_Sector_Ct 0x0033 100 100 005 Pre-fail Always - 1'''<br />
'''196 Reallocated_Event_Count 0x0032 100 100 000 Old_age Always - 1'''<br />
197 Current_Pending_Sector 0x0022 100 100 000 Old_age Always - 0<br />
'''198 Offline_Uncorrectable 0x0008 100 100 000 Old_age Offline - 1'''<br />
<br />
To get Offline_Uncorrectable to go back to 0 you need to run a SMART long test and a selftest:<br />
<br />
smartctl -t long /dev/sdx [wait until test completes, then]<br />
smartctl -l selftest /dev/sdx<br />
<br />
==See also==<br />
<br />
[http://smartmontools.sourceforge.net/badblockhowto.html EXT2/3 badblocks howto]</div>MaBeefhttps://wiki.archlinux.org/index.php?title=Identify_damaged_files&diff=210765Identify damaged files2012-06-21T00:57:58Z<p>MaBeef: </p>
<hr />
<div>[[Category:Storage]]<br />
This article gives details on how to find out which file owns a given block. The main purpose for doing so is finding out which file was damaged in the event a storage device develops any bad blocks (that way you'll know if you lost anything important).<br />
<br />
For most of these commands you will have to be either root or a user that has direct read access to the drive you are checking (being a member of the disk group should be enough). As usual, a current backup is always a good idea, especially if imminent drive failure is suspected.[[S.M.A.R.T.]] can help with determining that.<br />
<br />
Right now this article is only written for JFS and EXT filesystems.<br />
<br />
==Finding Bad Blocks==<br />
Just use the [[badblocks]] command. There are a few scan modes supported by it. There's read-only mode (default) which is the least accurate. There is the destructive write-mode (-w option) which is the most accurate but takes longer and will (obviously) destroy all data on the drive, thus making it quite useless for matching blocks up to files. There is finally the non-destructive read-write mode which is probably as accurate as the destructive mode, the only real downside of which is it's probably the slowest. However, if a drive is known to be failing then read-only mode is probably still the safest.<br />
<br />
To do a verbose (-v option), read-only scan, run one of these commands (with x being the drive letter and y being partition number you want to scan):<br />
<br />
Whole disk scan:<br />
badblocks -v /dev/sdx<br />
<br />
Single partition scan:<br />
badblocks -v /dev/sdxy<br />
<br />
The downside to scanning the drive as a whole is that each filesystem is going to start its block count relative to the partition it's on. This means that if you have a bad block that happens to be on, let's say, the second partition, and that partition starts on block 1000, then you will have to subtract 1000 from your block number in order to get the number you want. So if a scan from the beginning of the disk results in block number 1005 being bad, then block 5 of the second partition is what you'll actually be using.<br />
<br />
Otherwise, if you've found bad blocks after doing a full scan, you can simply figure out which partitions they're on, and rescan those in order to get the block number, rather than do any block math.<br />
<br />
Another thing to note is that badblocks defaults to 1024 byte blocks so you will either have to change the default size with the -b option in order to match your filesystem or manually convert the block number(s) later.<br />
<br />
If you need to figure out where your partitions start and end run fdisk (older versions might have defaulted to cylinders, not sure. If so the -u option will change the default unit to sectors). Make sure to note the block size fdisk is using so that you can convert the block counts to match your scan.<br />
<br />
fdisk -l /dev/sdx<br />
<br />
255 heads, 63 sectors/track, 19457 cylinders, total 312581808 sectors<br />
'''Units = sectors of 1 * 512 = 512 bytes'''<br />
Sector size (logical/physical): 512 bytes / 512 bytes<br />
I/O size (minimum/optimal): 512 bytes / 512 bytes<br />
Disk identifier: 0x00000000<br />
<br />
After all this, you should have the block number(s) of your bad block(s), relative to the partition they exist on.<br />
<br />
==Debug the Filesystem (JFS)==<br />
jfs_debugfs will give you access to all the low level structures within any JFS filesystem. Other filesystems such as the EXT filesystems have similar tools. It is probably a good idea to umount any filesystem before you run this on them. To use it just run:<br />
<br />
jfs_debugfs /dev/sdxy<br />
<br />
This puts you into a command console. The first thing you should note is your aggregate block size. This is (presumably) the block size the filesystem is using. JFS seems to default to 4096 bytes. If you did not run badblocks using the block size that your filesystem is using then you will need to convert your block number(s) to match it (remember to use the block number(s) relative to the partition it's on).<br />
<br />
i.e. block number 100 with a block size of 1024 bytes becomes block number 25 at 4096 bytes. The formula is:<br />
<br />
(original block number) / ((filesystem block size) / (badblocks block size))<br />
<br />
Now the entire point of running this program (for the purpose of this article) is to get the inode number. To do this run the command:<br />
<br />
d ''blocknumber'' 0 i<br />
<br />
The syntax is the d command for display, the block number, the offset (just set it to 0), and the display format i for inode.<br />
<br />
''Note'': If you get an error then that means the block is not allocated and is being used as free space. In this case this is a good thing as it means nothing important was damaged by that sector.<br />
<br />
The decimal number that di_number is set to is the one we want. From here you type x to exit out of the display mode. Repeat the display command for each bad block that you have and note all of their inode numbers. For more info on the inode such as permissions and filetype type:<br />
<br />
i ''inodenumber''<br />
<br />
i of course stands for inode.<br />
<br />
When you have all the inode numbers type q to quit.<br />
<br />
==Find Damaged Files (JFS)==<br />
<br />
Finally to find the damaged file you can simply use the gnu find utility. Mount your filesystem and run:<br />
<br />
find / -inum ''inodenumber''<br />
<br />
Substitute "/" for the mountpoint of the filesystem that the inode belongs to. If you search root and have more than one filesystem mounted (who doesn't?) you can find multiple files with the same inode number on different filesystems. Remember, an inode is only unique to the filesystem that it's in.<br />
<br />
==Debug the Filesystem (EXT(2/3/4))==<br />
tune2fs will give you access to all the low level structures within any EXT filesystem. It is probably a good idea to umount any filesystem before you run this on them.<br />
<br />
The first thing we want to do is get the block size from the filessystem in question. Just run (probably as root):<br />
<br />
tune2fs -l /dev/sdxy | grep Block<br />
Block count: 29119820<br />
'''Block size: 4096'''<br />
<br />
In this case 4096 is the block size being used (it appears to be the default). If you did not run badblocks using block size that your filesystem is using then you will need to convert your block numbers to match it (i.e. block number 100 with a block size of 1024 bytes becomes block number 25 at 4096 bytes).<br />
<br />
Now the entire point of running this program for the purpose of this article is to get the inode number. To do this run the command (as root):<br />
<br />
debugfs<br />
<br />
Then in the debugfs console, use the open command on the EXT partition containing the bad sector:<br />
<br />
debugfs: open /dev/sdxy<br />
<br />
Finally, use the testb command to get information about the block in question (in this example block 1000):<br />
<br />
debugfs: testb 1000<br />
<br />
If debugfs says that block isn't in use then that means the block is not allocated and is being used as free space. In this case this is a good thing as it means nothing important was damaged by that sector.<br />
<br />
If the block is in use then run this command to get the inode number<br />
<br />
icheck 1000<br />
<br />
This will return two numbers. The block number and the inode number.<br />
<br />
==Find Damaged Files (EXT(2/3/4))==<br />
Use the inode number (second number from the icheck command) with the ncheck command:<br />
<br />
ncheck 12345<br />
<br />
debugfs will give you the full pathname to the file using the bad block. Now you will know what was actually damaged.<br />
<br />
If the inode number is very small and ncheck fails to return a path then it's probably the journal itself that is damaged. To delete the journal simply run this command on the partition:<br />
<br />
tune2fs -O ^has_journal /dev/sdxy<br />
<br />
Run the testb command again from the debugfs console on the bad block and it should be no longer marked as used if it was indeed used by the journal. To build a new journal run:<br />
<br />
tune2fs -j /dev/sdxy<br />
<br />
==Force the disk to reallocate bad block==<br />
First you'll want to see how many badblocks the harddrive is aware of through the smartctl command:<br />
<br />
smartctl -t long /dev/sdx [wait until test completes, then]<br />
smartctl -l selftest /dev/sdx<br />
<br />
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE<br />
'''5 Reallocated_Sector_Ct 0x0033 100 100 005 Pre-fail Always - 0'''<br />
'''196 Reallocated_Event_Count 0x0032 100 100 000 Old_age Always - 0'''<br />
197 Current_Pending_Sector 0x0022 100 100 000 Old_age Always - 0<br />
'''198 Offline_Uncorrectable 0x0008 100 100 000 Old_age Offline - 0'''<br />
<br />
<br />
To make the harddrive transparently map out the badblock with a spare good sector you will have to simply write zeros to the bad block using the dd command as root. Remember that with this command you have to work with the same block size as your filesystem and the block as to be relative to the partition the filesystem is on and NOT the harddrive as a whole:<br />
<br />
dd if=/dev/zero of=/dev/sdxy bs=4096 count=1 seek=2269012<br />
sync<br />
<br />
You can see if the harddrive did indeed map out an additional bad sector by checking with the smartctl command and seeing if the reallocated sector or event count went up:<br />
<br />
smartctl -A /dev/sdx<br />
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE<br />
'''5 Reallocated_Sector_Ct 0x0033 100 100 005 Pre-fail Always - 1'''<br />
'''196 Reallocated_Event_Count 0x0032 100 100 000 Old_age Always - 1'''<br />
197 Current_Pending_Sector 0x0022 100 100 000 Old_age Always - 0<br />
'''198 Offline_Uncorrectable 0x0008 100 100 000 Old_age Offline - 1'''<br />
<br />
To get Offline_Uncorrectable to go back to 0 you need to run a SMART long test and a selftest:<br />
<br />
smartctl -t long /dev/sdx [wait until test completes, then]<br />
smartctl -l selftest /dev/sdx<br />
<br />
==See also==<br />
<br />
[http://smartmontools.sourceforge.net/badblockhowto.html EXT2/3 badblocks howto]</div>MaBeefhttps://wiki.archlinux.org/index.php?title=Identify_damaged_files&diff=210763Identify damaged files2012-06-21T00:51:30Z<p>MaBeef: </p>
<hr />
<div>[[Category:Storage]]<br />
This article gives details on how to find out which file owns a given block. The main purpose for doing so is finding out which file was damaged in the event a storage device develops any bad blocks (that way you'll know if you lost anything important).<br />
<br />
For most of these commands you will have to be either root or a user that has direct read access to the drive you are checking (being a member of the disk group should be enough). As usual, a current backup is always a good idea, especially if imminent drive failure is suspected.[https://wiki.archlinux.org/index.php/SMART S.M.A.R.T.] can help with determining that.<br />
<br />
Right now this article is only written for JFS and EXT filesystems.<br />
<br />
==Finding Bad Blocks==<br />
Just use the [[badblocks]] command. There are a few scan modes supported by it. There's read-only mode (default) which is the least accurate. There is the destructive write-mode (-w option) which is the most accurate but takes longer and will (obviously) destroy all data on the drive, thus making it quite useless for matching blocks up to files. There is finally the non-destructive read-write mode which is probably as accurate as the destructive mode, the only real downside of which is it's probably the slowest. However, if a drive is known to be failing then read-only mode is probably still the safest.<br />
<br />
To do a verbose (-v option), read-only scan, run one of these commands (with x being the drive letter and y being partition number you want to scan):<br />
<br />
Whole disk scan:<br />
badblocks -v /dev/sdx<br />
<br />
Single partition scan:<br />
badblocks -v /dev/sdxy<br />
<br />
The downside to scanning the drive as a whole is that each filesystem is going to start its block count relative to the partition it's on. This means that if you have a bad block that happens to be on, let's say, the second partition, and that partition starts on block 1000, then you will have to subtract 1000 from your block number in order to get the number you want. So if a scan from the beginning of the disk results in block number 1005 being bad, then block 5 of the second partition is what you'll actually be using.<br />
<br />
Otherwise, if you've found bad blocks after doing a full scan, you can simply figure out which partitions they're on, and rescan those in order to get the block number, rather than do any block math.<br />
<br />
Another thing to note is that badblocks defaults to 1024 byte blocks so you will either have to change the default size with the -b option in order to match your filesystem or manually convert the block number(s) later.<br />
<br />
If you need to figure out where your partitions start and end run fdisk (older versions might have defaulted to cylinders, not sure. If so the -u option will change the default unit to sectors). Make sure to note the block size fdisk is using so that you can convert the block counts to match your scan.<br />
<br />
fdisk -l /dev/sdx<br />
<br />
255 heads, 63 sectors/track, 19457 cylinders, total 312581808 sectors<br />
'''Units = sectors of 1 * 512 = 512 bytes'''<br />
Sector size (logical/physical): 512 bytes / 512 bytes<br />
I/O size (minimum/optimal): 512 bytes / 512 bytes<br />
Disk identifier: 0x00000000<br />
<br />
After all this, you should have the block number(s) of your bad block(s), relative to the partition they exist on.<br />
<br />
==Debug the Filesystem (JFS)==<br />
jfs_debugfs will give you access to all the low level structures within any JFS filesystem. Other filesystems such as the EXT filesystems have similar tools. It is probably a good idea to umount any filesystem before you run this on them. To use it just run (as root):<br />
<br />
jfs_debugfs /dev/sdxy<br />
<br />
This puts you into a command console. The first thing you should note is your aggregate block size. This is (presumably) the block size the filesystem is using. JFS seems to default to 4096 bytes. If you did not run badblocks using block size that your filesystem is using then you will need to convert your block numbers to match it (i.e. block number 100 with a block size of 1024 bytes becomes block number 25 at 4096 bytes).<br />
<br />
Now the entire point of running this program for the purpose of this article is to get the inode number. To do this run the command:<br />
<br />
d ''blocknumber'' 0 i<br />
<br />
The syntax is the d command for display, the block number, the offset (just set it to 0), and the display format i for inode.<br />
<br />
''Note'': If you get an error then that means the block is not allocated and is being used as free space. In this case this is a good thing as it means nothing important was damaged by that sector.<br />
<br />
The decimal number that di_number is set to is the one we want. From here you type x to exit out of the display mode. Repeat the display command for each bad block that you have and note all of their inode numbers. For more info on the inode such as permissions and filetype type:<br />
<br />
i ''inodenumber''<br />
<br />
i of course stands for inode.<br />
<br />
When you have all the inode numbers type q to quit.<br />
<br />
==Find Damaged Files (JFS)==<br />
<br />
Finally to find the damaged file you can simply use the gnu find utility. Mount your filesystem and run:<br />
<br />
find / -inum ''inodenumber''<br />
<br />
Substitute "/" for the mountpoint of the filesystem that the inode belongs to. If you search root and have more than one filesystem mounted (who doesn't?) you can find multiple files with the same inode number on different filesystems. Remember, an inode is only unique to the filesystem that it's in.<br />
<br />
==Debug the Filesystem (EXT(2/3/4))==<br />
tune2fs will give you access to all the low level structures within any EXT filesystem. It is probably a good idea to umount any filesystem before you run this on them.<br />
<br />
The first thing we want to do is get the block size from the filessystem in question. Just run (probably as root):<br />
<br />
tune2fs -l /dev/sdxy | grep Block<br />
Block count: 29119820<br />
'''Block size: 4096'''<br />
<br />
In this case 4096 is the block size being used (it appears to be the default). If you did not run badblocks using block size that your filesystem is using then you will need to convert your block numbers to match it (i.e. block number 100 with a block size of 1024 bytes becomes block number 25 at 4096 bytes).<br />
<br />
Now the entire point of running this program for the purpose of this article is to get the inode number. To do this run the command (as root):<br />
<br />
debugfs<br />
<br />
Then in the debugfs console, use the open command on the EXT partition containing the bad sector:<br />
<br />
debugfs: open /dev/sdxy<br />
<br />
Finally, use the testb command to get information about the block in question (in this example block 1000):<br />
<br />
debugfs: testb 1000<br />
<br />
If debugfs says that block isn't in use then that means the block is not allocated and is being used as free space. In this case this is a good thing as it means nothing important was damaged by that sector.<br />
<br />
If the block is in use then run this command to get the inode number<br />
<br />
icheck 1000<br />
<br />
This will return two numbers. The block number and the inode number.<br />
<br />
==Find Damaged Files (EXT(2/3/4))==<br />
Use the inode number (second number from the icheck command) with the ncheck command:<br />
<br />
ncheck 12345<br />
<br />
debugfs will give you the full pathname to the file using the bad block. Now you will know what was actually damaged.<br />
<br />
If the inode number is very small and ncheck fails to return a path then it's probably the journal itself that is damaged. To delete the journal simply run this command on the partition:<br />
<br />
tune2fs -O ^has_journal /dev/sdxy<br />
<br />
Run the testb command again from the debugfs console on the bad block and it should be no longer marked as used if it was indeed used by the journal. To build a new journal run:<br />
<br />
tune2fs -j /dev/sdxy<br />
<br />
==Force the disk to reallocate bad block==<br />
First you'll want to see how many badblocks the harddrive is aware of through the smartctl command:<br />
<br />
smartctl -t long /dev/sdx [wait until test completes, then]<br />
smartctl -l selftest /dev/sdx<br />
<br />
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE<br />
'''5 Reallocated_Sector_Ct 0x0033 100 100 005 Pre-fail Always - 0'''<br />
'''196 Reallocated_Event_Count 0x0032 100 100 000 Old_age Always - 0'''<br />
197 Current_Pending_Sector 0x0022 100 100 000 Old_age Always - 0<br />
'''198 Offline_Uncorrectable 0x0008 100 100 000 Old_age Offline - 0'''<br />
<br />
<br />
To make the harddrive transparently map out the badblock with a spare good sector you will have to simply write zeros to the bad block using the dd command as root. Remember that with this command you have to work with the same block size as your filesystem and the block as to be relative to the partition the filesystem is on and NOT the harddrive as a whole:<br />
<br />
dd if=/dev/zero of=/dev/sdxy bs=4096 count=1 seek=2269012<br />
sync<br />
<br />
You can see if the harddrive did indeed map out an additional bad sector by checking with the smartctl command and seeing if the reallocated sector or event count went up:<br />
<br />
smartctl -A /dev/sdx<br />
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE<br />
'''5 Reallocated_Sector_Ct 0x0033 100 100 005 Pre-fail Always - 1'''<br />
'''196 Reallocated_Event_Count 0x0032 100 100 000 Old_age Always - 1'''<br />
197 Current_Pending_Sector 0x0022 100 100 000 Old_age Always - 0<br />
'''198 Offline_Uncorrectable 0x0008 100 100 000 Old_age Offline - 1'''<br />
<br />
To get Offline_Uncorrectable to go back to 0 you need to run a SMART long test and a selftest:<br />
<br />
smartctl -t long /dev/sdx [wait until test completes, then]<br />
smartctl -l selftest /dev/sdx<br />
<br />
==See also==<br />
<br />
[http://smartmontools.sourceforge.net/badblockhowto.html EXT2/3 badblocks howto]</div>MaBeefhttps://wiki.archlinux.org/index.php?title=Identify_damaged_files&diff=210761Identify damaged files2012-06-21T00:50:20Z<p>MaBeef: More rephrasing, added example output to fdisk</p>
<hr />
<div>[[Category:Storage]]<br />
This article gives details on how to find out which file owns a given block. The main purpose for doing so is finding out which file was damaged in the event a storage device develops any bad blocks (that way you'll know if you lost anything important).<br />
<br />
For most of these commands you will have to be either root or a user that has direct read access to the drive you are checking (being a member of the disk group should be enough). As usual, a current backup is always a good idea, especially if imminent drive failure is suspected.[https://wiki.archlinux.org/index.php/SMART S.M.A.R.T.] can help with determining that.<br />
<br />
Right now this article is only written for JFS and EXT filesystems.<br />
<br />
==Finding Bad Blocks==<br />
Just use the [[badblocks]] command. There are a few scan modes supported by it. There's read-only mode (default) which is the least accurate. There is the destructive write-mode (-w option) which is the most accurate but takes longer and will (obviously) destroy all data on the drive, thus making it quite useless for matching blocks up to files. There is finally the non-destructive read-write mode which is probably as accurate as the destructive mode, the only real downside of which is it's probably the slowest. However, if a drive is known to be failing then read-only mode is probably still the safest.<br />
<br />
To do a verbose (-v option), read-only scan, run one of these commands (with x being the drive letter and y being partition number you want to scan):<br />
<br />
Whole disk scan:<br />
badblocks -v /dev/sdx<br />
<br />
Single partition scan:<br />
badblocks -v /dev/sdxy<br />
<br />
The downside to scanning the drive as a whole is that each filesystem is going to start its block count relative to the partition it's on. This means that if you have a bad block that happens to be on, let's say, the second partition, and that partition starts on block 1000, then you will have to subtract 1000 from your block number in order to get the number you want. So if a scan from the beginning of the disk results in block number 1005 being bad, then block 5 of the second partition is what you'll actually be using.<br />
<br />
Otherwise, if you've found bad blocks after doing a full scan, you can simply figure out which partitions they're on, and rescan those in order to get the block number, rather than do any block math.<br />
<br />
Another thing to note is that badblocks defaults to 1024 byte blocks so you will either have to change the default size with the -b option in order to match your filesystem or manually convert the block number(s) later.<br />
<br />
If you need to figure out where your partitions start and end run fdisk (older versions might have defaulted to cylinders, not sure. If so the -u option will change the default unit to sectors). Make sure to note the block size fdisk is using so that you can convert the block counts to match your scan.<br />
<br />
fdisk -l /dev/sdx<br />
<br />
255 heads, 63 sectors/track, 19457 cylinders, total 312581808 sectors<br />
'''Units = sectors of 1 * 512 = 512 bytes'''<br />
Sector size (logical/physical): 512 bytes / 512 bytes<br />
I/O size (minimum/optimal): 512 bytes / 512 bytes<br />
Disk identifier: 0x00000000<br />
<br />
<br />
==Debug the Filesystem (JFS)==<br />
jfs_debugfs will give you access to all the low level structures within any JFS filesystem. Other filesystems such as the EXT filesystems have similar tools. It is probably a good idea to umount any filesystem before you run this on them. To use it just run (as root):<br />
<br />
jfs_debugfs /dev/sdxy<br />
<br />
This puts you into a command console. The first thing you should note is your aggregate block size. This is (presumably) the block size the filesystem is using. JFS seems to default to 4096 bytes. If you did not run badblocks using block size that your filesystem is using then you will need to convert your block numbers to match it (i.e. block number 100 with a block size of 1024 bytes becomes block number 25 at 4096 bytes).<br />
<br />
Now the entire point of running this program for the purpose of this article is to get the inode number. To do this run the command:<br />
<br />
d ''blocknumber'' 0 i<br />
<br />
The syntax is the d command for display, the block number, the offset (just set it to 0), and the display format i for inode.<br />
<br />
''Note'': If you get an error then that means the block is not allocated and is being used as free space. In this case this is a good thing as it means nothing important was damaged by that sector.<br />
<br />
The decimal number that di_number is set to is the one we want. From here you type x to exit out of the display mode. Repeat the display command for each bad block that you have and note all of their inode numbers. For more info on the inode such as permissions and filetype type:<br />
<br />
i ''inodenumber''<br />
<br />
i of course stands for inode.<br />
<br />
When you have all the inode numbers type q to quit.<br />
<br />
==Find Damaged Files (JFS)==<br />
<br />
Finally to find the damaged file you can simply use the gnu find utility. Mount your filesystem and run:<br />
<br />
find / -inum ''inodenumber''<br />
<br />
Substitute "/" for the mountpoint of the filesystem that the inode belongs to. If you search root and have more than one filesystem mounted (who doesn't?) you can find multiple files with the same inode number on different filesystems. Remember, an inode is only unique to the filesystem that it's in.<br />
<br />
==Debug the Filesystem (EXT(2/3/4))==<br />
tune2fs will give you access to all the low level structures within any EXT filesystem. It is probably a good idea to umount any filesystem before you run this on them.<br />
<br />
The first thing we want to do is get the block size from the filessystem in question. Just run (probably as root):<br />
<br />
tune2fs -l /dev/sdxy | grep Block<br />
Block count: 29119820<br />
'''Block size: 4096'''<br />
<br />
In this case 4096 is the block size being used (it appears to be the default). If you did not run badblocks using block size that your filesystem is using then you will need to convert your block numbers to match it (i.e. block number 100 with a block size of 1024 bytes becomes block number 25 at 4096 bytes).<br />
<br />
Now the entire point of running this program for the purpose of this article is to get the inode number. To do this run the command (as root):<br />
<br />
debugfs<br />
<br />
Then in the debugfs console, use the open command on the EXT partition containing the bad sector:<br />
<br />
debugfs: open /dev/sdxy<br />
<br />
Finally, use the testb command to get information about the block in question (in this example block 1000):<br />
<br />
debugfs: testb 1000<br />
<br />
If debugfs says that block isn't in use then that means the block is not allocated and is being used as free space. In this case this is a good thing as it means nothing important was damaged by that sector.<br />
<br />
If the block is in use then run this command to get the inode number<br />
<br />
icheck 1000<br />
<br />
This will return two numbers. The block number and the inode number.<br />
<br />
==Find Damaged Files (EXT(2/3/4))==<br />
Use the inode number (second number from the icheck command) with the ncheck command:<br />
<br />
ncheck 12345<br />
<br />
debugfs will give you the full pathname to the file using the bad block. Now you will know what was actually damaged.<br />
<br />
If the inode number is very small and ncheck fails to return a path then it's probably the journal itself that is damaged. To delete the journal simply run this command on the partition:<br />
<br />
tune2fs -O ^has_journal /dev/sdxy<br />
<br />
Run the testb command again from the debugfs console on the bad block and it should be no longer marked as used if it was indeed used by the journal. To build a new journal run:<br />
<br />
tune2fs -j /dev/sdxy<br />
<br />
==Force the disk to reallocate bad block==<br />
First you'll want to see how many badblocks the harddrive is aware of through the smartctl command:<br />
<br />
smartctl -t long /dev/sdx [wait until test completes, then]<br />
smartctl -l selftest /dev/sdx<br />
<br />
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE<br />
'''5 Reallocated_Sector_Ct 0x0033 100 100 005 Pre-fail Always - 0'''<br />
'''196 Reallocated_Event_Count 0x0032 100 100 000 Old_age Always - 0'''<br />
197 Current_Pending_Sector 0x0022 100 100 000 Old_age Always - 0<br />
'''198 Offline_Uncorrectable 0x0008 100 100 000 Old_age Offline - 0'''<br />
<br />
<br />
To make the harddrive transparently map out the badblock with a spare good sector you will have to simply write zeros to the bad block using the dd command as root. Remember that with this command you have to work with the same block size as your filesystem and the block as to be relative to the partition the filesystem is on and NOT the harddrive as a whole:<br />
<br />
dd if=/dev/zero of=/dev/sdxy bs=4096 count=1 seek=2269012<br />
sync<br />
<br />
You can see if the harddrive did indeed map out an additional bad sector by checking with the smartctl command and seeing if the reallocated sector or event count went up:<br />
<br />
smartctl -A /dev/sdx<br />
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE<br />
'''5 Reallocated_Sector_Ct 0x0033 100 100 005 Pre-fail Always - 1'''<br />
'''196 Reallocated_Event_Count 0x0032 100 100 000 Old_age Always - 1'''<br />
197 Current_Pending_Sector 0x0022 100 100 000 Old_age Always - 0<br />
'''198 Offline_Uncorrectable 0x0008 100 100 000 Old_age Offline - 1'''<br />
<br />
To get Offline_Uncorrectable to go back to 0 you need to run a SMART long test and a selftest:<br />
<br />
smartctl -t long /dev/sdx [wait until test completes, then]<br />
smartctl -l selftest /dev/sdx<br />
<br />
==See also==<br />
<br />
[http://smartmontools.sourceforge.net/badblockhowto.html EXT2/3 badblocks howto]</div>MaBeefhttps://wiki.archlinux.org/index.php?title=Identify_damaged_files&diff=210759Identify damaged files2012-06-21T00:44:42Z<p>MaBeef: </p>
<hr />
<div>[[Category:Storage]]<br />
This article gives details on how to find out which file owns a given block. The main purpose for doing so is finding out which file was damaged in the event a storage device develops any bad blocks (that way you'll know if you lost anything important).<br />
<br />
For most of these commands you will have to be either root or a user that has direct read access to the drive you are checking (being a member of the disk group should be enough). As usual, a current backup is always a good idea, especially if imminent drive failure is suspected.[https://wiki.archlinux.org/index.php/SMART S.M.A.R.T.] can help with determining that.<br />
<br />
Right now this article is only written for JFS and EXT filesystems.<br />
<br />
==Finding Bad Blocks==<br />
Just use the [[badblocks]] command. There are a few scan modes supported by it. There's read-only mode (default) which is the least accurate. There is the destructive write-mode (-w option) which is the most accurate but takes longer and will (obviously) destroy all data on the drive, thus making it quite useless for matching blocks up to files. There is finally the non-destructive read-write mode which is probably as accurate as the destructive mode, the only real downside of which is it's probably the slowest. However, if a drive is known to be failing then read-only mode is probably still the safest.<br />
<br />
To do a verbose (-v option), read-only scan run this command (with x being the drive letter and y being partition number you want to scan):<br />
<br />
Whole disk scan:<br />
badblocks -v /dev/sdx<br />
<br />
or<br />
Single partition scan:<br />
badblocks -v /dev/sdxy<br />
<br />
The downside to scanning the drive as a whole is that each filesystem is going to start its block count relative to the partition it's on. This means that if you have a bad block on let's say the second partition, and that partition starts on block 1000, then you will have to subtract 1000 from your block number in order to get the number you want. So if scan from the beginning of the disk results in block number 1005 being bad, then block 5 of the second partition is what you'll actually be using.<br />
<br />
The alternative is if you've scanned the drive as a whole and have found bad blocks, figure out the partitions they fall on then scan those, using the new block numbers that result from that.<br />
<br />
Another thing to note is that badblocks defaults to 1024 byte blocks so you will either have to change the default size with the -b option to match your filesystem or manually convert the block number(s) later.<br />
<br />
If you need to figure out where your partitions start and end run fdisk (older versions might have defaulted to cylinders, not sure. If so the -u option will change the default unit to sectors). Make sure to note the block size fdisk is using so that you can convert the block counts to match your scan.<br />
<br />
fdisk -l /dev/sdx<br />
<br />
==Debug the Filesystem (JFS)==<br />
jfs_debugfs will give you access to all the low level structures within any JFS filesystem. Other filesystems such as the EXT filesystems have similar tools. It is probably a good idea to umount any filesystem before you run this on them. To use it just run (as root):<br />
<br />
jfs_debugfs /dev/sdxy<br />
<br />
This puts you into a command console. The first thing you should note is your aggregate block size. This is (presumably) the block size the filesystem is using. JFS seems to default to 4096 bytes. If you did not run badblocks using block size that your filesystem is using then you will need to convert your block numbers to match it (i.e. block number 100 with a block size of 1024 bytes becomes block number 25 at 4096 bytes).<br />
<br />
Now the entire point of running this program for the purpose of this article is to get the inode number. To do this run the command:<br />
<br />
d ''blocknumber'' 0 i<br />
<br />
The syntax is the d command for display, the block number, the offset (just set it to 0), and the display format i for inode.<br />
<br />
''Note'': If you get an error then that means the block is not allocated and is being used as free space. In this case this is a good thing as it means nothing important was damaged by that sector.<br />
<br />
The decimal number that di_number is set to is the one we want. From here you type x to exit out of the display mode. Repeat the display command for each bad block that you have and note all of their inode numbers. For more info on the inode such as permissions and filetype type:<br />
<br />
i ''inodenumber''<br />
<br />
i of course stands for inode.<br />
<br />
When you have all the inode numbers type q to quit.<br />
<br />
==Find Damaged Files (JFS)==<br />
<br />
Finally to find the damaged file you can simply use the gnu find utility. Mount your filesystem and run:<br />
<br />
find / -inum ''inodenumber''<br />
<br />
Substitute "/" for the mountpoint of the filesystem that the inode belongs to. If you search root and have more than one filesystem mounted (who doesn't?) you can find multiple files with the same inode number on different filesystems. Remember, an inode is only unique to the filesystem that it's in.<br />
<br />
==Debug the Filesystem (EXT(2/3/4))==<br />
tune2fs will give you access to all the low level structures within any EXT filesystem. It is probably a good idea to umount any filesystem before you run this on them.<br />
<br />
The first thing we want to do is get the block size from the filessystem in question. Just run (probably as root):<br />
<br />
tune2fs -l /dev/sdxy | grep Block<br />
Block count: 29119820<br />
'''Block size: 4096'''<br />
<br />
In this case 4096 is the block size being used (it appears to be the default). If you did not run badblocks using block size that your filesystem is using then you will need to convert your block numbers to match it (i.e. block number 100 with a block size of 1024 bytes becomes block number 25 at 4096 bytes).<br />
<br />
Now the entire point of running this program for the purpose of this article is to get the inode number. To do this run the command (as root):<br />
<br />
debugfs<br />
<br />
Then in the debugfs console, use the open command on the EXT partition containing the bad sector:<br />
<br />
debugfs: open /dev/sdxy<br />
<br />
Finally, use the testb command to get information about the block in question (in this example block 1000):<br />
<br />
debugfs: testb 1000<br />
<br />
If debugfs says that block isn't in use then that means the block is not allocated and is being used as free space. In this case this is a good thing as it means nothing important was damaged by that sector.<br />
<br />
If the block is in use then run this command to get the inode number<br />
<br />
icheck 1000<br />
<br />
This will return two numbers. The block number and the inode number.<br />
<br />
==Find Damaged Files (EXT(2/3/4))==<br />
Use the inode number (second number from the icheck command) with the ncheck command:<br />
<br />
ncheck 12345<br />
<br />
debugfs will give you the full pathname to the file using the bad block. Now you will know what was actually damaged.<br />
<br />
If the inode number is very small and ncheck fails to return a path then it's probably the journal itself that is damaged. To delete the journal simply run this command on the partition:<br />
<br />
tune2fs -O ^has_journal /dev/sdxy<br />
<br />
Run the testb command again from the debugfs console on the bad block and it should be no longer marked as used if it was indeed used by the journal. To build a new journal run:<br />
<br />
tune2fs -j /dev/sdxy<br />
<br />
==Force the disk to reallocate bad block==<br />
First you'll want to see how many badblocks the harddrive is aware of through the smartctl command:<br />
<br />
smartctl -t long /dev/sdx [wait until test completes, then]<br />
smartctl -l selftest /dev/sdx<br />
<br />
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE<br />
'''5 Reallocated_Sector_Ct 0x0033 100 100 005 Pre-fail Always - 0'''<br />
'''196 Reallocated_Event_Count 0x0032 100 100 000 Old_age Always - 0'''<br />
197 Current_Pending_Sector 0x0022 100 100 000 Old_age Always - 0<br />
'''198 Offline_Uncorrectable 0x0008 100 100 000 Old_age Offline - 0'''<br />
<br />
<br />
To make the harddrive transparently map out the badblock with a spare good sector you will have to simply write zeros to the bad block using the dd command as root. Remember that with this command you have to work with the same block size as your filesystem and the block as to be relative to the partition the filesystem is on and NOT the harddrive as a whole:<br />
<br />
dd if=/dev/zero of=/dev/sdxy bs=4096 count=1 seek=2269012<br />
sync<br />
<br />
You can see if the harddrive did indeed map out an additional bad sector by checking with the smartctl command and seeing if the reallocated sector or event count went up:<br />
<br />
smartctl -A /dev/sdx<br />
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE<br />
'''5 Reallocated_Sector_Ct 0x0033 100 100 005 Pre-fail Always - 1'''<br />
'''196 Reallocated_Event_Count 0x0032 100 100 000 Old_age Always - 1'''<br />
197 Current_Pending_Sector 0x0022 100 100 000 Old_age Always - 0<br />
'''198 Offline_Uncorrectable 0x0008 100 100 000 Old_age Offline - 1'''<br />
<br />
To get Offline_Uncorrectable to go back to 0 you need to run a SMART long test and a selftest:<br />
<br />
smartctl -t long /dev/sdx [wait until test completes, then]<br />
smartctl -l selftest /dev/sdx<br />
<br />
==See also==<br />
<br />
[http://smartmontools.sourceforge.net/badblockhowto.html EXT2/3 badblocks howto]</div>MaBeefhttps://wiki.archlinux.org/index.php?title=Identify_damaged_files&diff=210758Identify damaged files2012-06-21T00:43:35Z<p>MaBeef: clearer phrasing</p>
<hr />
<div>[[Category:Storage]]<br />
This article gives details on how to find out which file owns a given block. The main purpose for doing so is finding out which file was damaged in the event a storage device develops any bad blocks (that way you'll know if you lost anything important).<br />
<br />
For most of these commands you will have to be either root or a user that has direct read access to the drive you are checking (being a member of the disk group should be enough). As usual, a current backup is always a good idea, especially if imminent drive failure is suspected.[https://wiki.archlinux.org/index.php/SMART S.M.A.R.T.] can help with determining that.<br />
<br />
Right now this article is only written for JFS and EXT filesystems.<br />
<br />
==Finding Bad Blocks==<br />
Just use the [[badblocks]] command. There are a few scan modes supported by it. There's read-only mode (default) which is the least accurate. There is the destructive write-mode (-w option) which is the most accurate but takes longer and will (obviously) destroy all data on the drive, thus making it quite useless for matching blocks up to files. There is finally the non-destructive read-write mode which is probably as accurate as the destructive mode, the only real downside of which is it's probably the slowest. However, if a drive is known to be failing then read-only mode is probably still the safest.<br />
<br />
To do a verbose (-v option), read-only scan run this command (with x being the drive letter and y being partition number you want to scan):<br />
<br />
badblocks -v /dev/sdx<br />
<br />
or<br />
<br />
badblocks -v /dev/sdxy<br />
<br />
The downside to scanning the drive as a whole is that each filesystem is going to start its block count relative to the partition it's on. This means that if you have a bad block on let's say the second partition, and that partition starts on block 1000, then you will have to subtract 1000 from your block number in order to get the number you want. So if scan from the beginning of the disk results in block number 1005 being bad, then block 5 of the second partition is what you'll actually be using.<br />
<br />
The alternative is if you've scanned the drive as a whole and have found bad blocks, figure out the partitions they fall on then scan those, using the new block numbers that result from that.<br />
<br />
Another thing to note is that badblocks defaults to 1024 byte blocks so you will either have to change the default size with the -b option to match your filesystem or manually convert the block number(s) later.<br />
<br />
If you need to figure out where your partitions start and end run fdisk (older versions might have defaulted to cylinders, not sure. If so the -u option will change the default unit to sectors). Make sure to note the block size fdisk is using so that you can convert the block counts to match your scan.<br />
<br />
fdisk -l /dev/sdx<br />
<br />
==Debug the Filesystem (JFS)==<br />
jfs_debugfs will give you access to all the low level structures within any JFS filesystem. Other filesystems such as the EXT filesystems have similar tools. It is probably a good idea to umount any filesystem before you run this on them. To use it just run (as root):<br />
<br />
jfs_debugfs /dev/sdxy<br />
<br />
This puts you into a command console. The first thing you should note is your aggregate block size. This is (presumably) the block size the filesystem is using. JFS seems to default to 4096 bytes. If you did not run badblocks using block size that your filesystem is using then you will need to convert your block numbers to match it (i.e. block number 100 with a block size of 1024 bytes becomes block number 25 at 4096 bytes).<br />
<br />
Now the entire point of running this program for the purpose of this article is to get the inode number. To do this run the command:<br />
<br />
d ''blocknumber'' 0 i<br />
<br />
The syntax is the d command for display, the block number, the offset (just set it to 0), and the display format i for inode.<br />
<br />
''Note'': If you get an error then that means the block is not allocated and is being used as free space. In this case this is a good thing as it means nothing important was damaged by that sector.<br />
<br />
The decimal number that di_number is set to is the one we want. From here you type x to exit out of the display mode. Repeat the display command for each bad block that you have and note all of their inode numbers. For more info on the inode such as permissions and filetype type:<br />
<br />
i ''inodenumber''<br />
<br />
i of course stands for inode.<br />
<br />
When you have all the inode numbers type q to quit.<br />
<br />
==Find Damaged Files (JFS)==<br />
<br />
Finally to find the damaged file you can simply use the gnu find utility. Mount your filesystem and run:<br />
<br />
find / -inum ''inodenumber''<br />
<br />
Substitute "/" for the mountpoint of the filesystem that the inode belongs to. If you search root and have more than one filesystem mounted (who doesn't?) you can find multiple files with the same inode number on different filesystems. Remember, an inode is only unique to the filesystem that it's in.<br />
<br />
==Debug the Filesystem (EXT(2/3/4))==<br />
tune2fs will give you access to all the low level structures within any EXT filesystem. It is probably a good idea to umount any filesystem before you run this on them.<br />
<br />
The first thing we want to do is get the block size from the filessystem in question. Just run (probably as root):<br />
<br />
tune2fs -l /dev/sdxy | grep Block<br />
Block count: 29119820<br />
'''Block size: 4096'''<br />
<br />
In this case 4096 is the block size being used (it appears to be the default). If you did not run badblocks using block size that your filesystem is using then you will need to convert your block numbers to match it (i.e. block number 100 with a block size of 1024 bytes becomes block number 25 at 4096 bytes).<br />
<br />
Now the entire point of running this program for the purpose of this article is to get the inode number. To do this run the command (as root):<br />
<br />
debugfs<br />
<br />
Then in the debugfs console, use the open command on the EXT partition containing the bad sector:<br />
<br />
debugfs: open /dev/sdxy<br />
<br />
Finally, use the testb command to get information about the block in question (in this example block 1000):<br />
<br />
debugfs: testb 1000<br />
<br />
If debugfs says that block isn't in use then that means the block is not allocated and is being used as free space. In this case this is a good thing as it means nothing important was damaged by that sector.<br />
<br />
If the block is in use then run this command to get the inode number<br />
<br />
icheck 1000<br />
<br />
This will return two numbers. The block number and the inode number.<br />
<br />
==Find Damaged Files (EXT(2/3/4))==<br />
Use the inode number (second number from the icheck command) with the ncheck command:<br />
<br />
ncheck 12345<br />
<br />
debugfs will give you the full pathname to the file using the bad block. Now you will know what was actually damaged.<br />
<br />
If the inode number is very small and ncheck fails to return a path then it's probably the journal itself that is damaged. To delete the journal simply run this command on the partition:<br />
<br />
tune2fs -O ^has_journal /dev/sdxy<br />
<br />
Run the testb command again from the debugfs console on the bad block and it should be no longer marked as used if it was indeed used by the journal. To build a new journal run:<br />
<br />
tune2fs -j /dev/sdxy<br />
<br />
==Force the disk to reallocate bad block==<br />
First you'll want to see how many badblocks the harddrive is aware of through the smartctl command:<br />
<br />
smartctl -t long /dev/sdx [wait until test completes, then]<br />
smartctl -l selftest /dev/sdx<br />
<br />
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE<br />
'''5 Reallocated_Sector_Ct 0x0033 100 100 005 Pre-fail Always - 0'''<br />
'''196 Reallocated_Event_Count 0x0032 100 100 000 Old_age Always - 0'''<br />
197 Current_Pending_Sector 0x0022 100 100 000 Old_age Always - 0<br />
'''198 Offline_Uncorrectable 0x0008 100 100 000 Old_age Offline - 0'''<br />
<br />
<br />
To make the harddrive transparently map out the badblock with a spare good sector you will have to simply write zeros to the bad block using the dd command as root. Remember that with this command you have to work with the same block size as your filesystem and the block as to be relative to the partition the filesystem is on and NOT the harddrive as a whole:<br />
<br />
dd if=/dev/zero of=/dev/sdxy bs=4096 count=1 seek=2269012<br />
sync<br />
<br />
You can see if the harddrive did indeed map out an additional bad sector by checking with the smartctl command and seeing if the reallocated sector or event count went up:<br />
<br />
smartctl -A /dev/sdx<br />
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE<br />
'''5 Reallocated_Sector_Ct 0x0033 100 100 005 Pre-fail Always - 1'''<br />
'''196 Reallocated_Event_Count 0x0032 100 100 000 Old_age Always - 1'''<br />
197 Current_Pending_Sector 0x0022 100 100 000 Old_age Always - 0<br />
'''198 Offline_Uncorrectable 0x0008 100 100 000 Old_age Offline - 1'''<br />
<br />
To get Offline_Uncorrectable to go back to 0 you need to run a SMART long test and a selftest:<br />
<br />
smartctl -t long /dev/sdx [wait until test completes, then]<br />
smartctl -l selftest /dev/sdx<br />
<br />
==See also==<br />
<br />
[http://smartmontools.sourceforge.net/badblockhowto.html EXT2/3 badblocks howto]</div>MaBeefhttps://wiki.archlinux.org/index.php?title=Identify_damaged_files&diff=210757Identify damaged files2012-06-21T00:41:09Z<p>MaBeef: </p>
<hr />
<div>[[Category:Storage]]<br />
This article gives details on how to find out which file owns a given block. The main purpose for doing so is finding out which file was damaged in the event a storage device develops any bad blocks (that way you'll know if you lost anything important).<br />
<br />
For most of these commands you will have to be either root or a user that has direct read access to the drive you are checking (being a member of the disk group should be enough). As usual, a current backup is always a good idea, especially if imminent drive failure is suspected.[https://wiki.archlinux.org/index.php/SMART S.M.A.R.T.] can help with determining that.<br />
<br />
Right now this article is only written for JFS and EXT filesystems.<br />
<br />
==Finding Bad Blocks==<br />
Just use the [[badblocks]] command. There are a few scan modes supported by it. There's read-only mode (default) which is the least accurate. There is the destructive write-mode (-w option) which is the most accurate but takes longer and will (obviously) destroy all data on the drive, thus making it quite useless for matching blocks up to files. There is finally the non-destructive read-write mode which is probably as accurate as the destructive mode, the only real downside of which is it's probably the slowest. However, if a drive is known to be failing then read-only mode is probably still the safest.<br />
<br />
To do a verbose (-v option), read-only scan run this command (with x being the drive letter and y being partition number you want to scan):<br />
<br />
badblocks -v /dev/sdx<br />
<br />
or<br />
<br />
badblocks -v /dev/sdxy<br />
<br />
The downside to scanning the drive as a whole is that each filesystem is going to start its block count relative to the partition it's on. This means that if you have a bad block on let's say the second partition and the second partition starts on block 1000, then you have to subtract 1000 from your block number to get the number you want. So if block number 1005 is bad on that partition, then block 5 is what you'll actually be using.<br />
<br />
The alternative is if you've scanned the drive as a whole and have found bad blocks, figure out the partitions they fall on then scan those, using the new block numbers that result from that.<br />
<br />
Another thing to note is that badblocks defaults to 1024 byte blocks so you will either have to change the default size with the -b option to match your filesystem or manually convert the block number(s) later.<br />
<br />
If you need to figure out where your partitions start and end run fdisk (older versions might have defaulted to cylinders, not sure. If so the -u option will change the default unit to sectors). Make sure to note the block size fdisk is using so that you can convert the block counts to match your scan.<br />
<br />
fdisk -l /dev/sdx<br />
<br />
==Debug the Filesystem (JFS)==<br />
jfs_debugfs will give you access to all the low level structures within any JFS filesystem. Other filesystems such as the EXT filesystems have similar tools. It is probably a good idea to umount any filesystem before you run this on them. To use it just run (as root):<br />
<br />
jfs_debugfs /dev/sdxy<br />
<br />
This puts you into a command console. The first thing you should note is your aggregate block size. This is (presumably) the block size the filesystem is using. JFS seems to default to 4096 bytes. If you did not run badblocks using block size that your filesystem is using then you will need to convert your block numbers to match it (i.e. block number 100 with a block size of 1024 bytes becomes block number 25 at 4096 bytes).<br />
<br />
Now the entire point of running this program for the purpose of this article is to get the inode number. To do this run the command:<br />
<br />
d ''blocknumber'' 0 i<br />
<br />
The syntax is the d command for display, the block number, the offset (just set it to 0), and the display format i for inode.<br />
<br />
''Note'': If you get an error then that means the block is not allocated and is being used as free space. In this case this is a good thing as it means nothing important was damaged by that sector.<br />
<br />
The decimal number that di_number is set to is the one we want. From here you type x to exit out of the display mode. Repeat the display command for each bad block that you have and note all of their inode numbers. For more info on the inode such as permissions and filetype type:<br />
<br />
i ''inodenumber''<br />
<br />
i of course stands for inode.<br />
<br />
When you have all the inode numbers type q to quit.<br />
<br />
==Find Damaged Files (JFS)==<br />
<br />
Finally to find the damaged file you can simply use the gnu find utility. Mount your filesystem and run:<br />
<br />
find / -inum ''inodenumber''<br />
<br />
Substitute "/" for the mountpoint of the filesystem that the inode belongs to. If you search root and have more than one filesystem mounted (who doesn't?) you can find multiple files with the same inode number on different filesystems. Remember, an inode is only unique to the filesystem that it's in.<br />
<br />
==Debug the Filesystem (EXT(2/3/4))==<br />
tune2fs will give you access to all the low level structures within any EXT filesystem. It is probably a good idea to umount any filesystem before you run this on them.<br />
<br />
The first thing we want to do is get the block size from the filessystem in question. Just run (probably as root):<br />
<br />
tune2fs -l /dev/sdxy | grep Block<br />
Block count: 29119820<br />
'''Block size: 4096'''<br />
<br />
In this case 4096 is the block size being used (it appears to be the default). If you did not run badblocks using block size that your filesystem is using then you will need to convert your block numbers to match it (i.e. block number 100 with a block size of 1024 bytes becomes block number 25 at 4096 bytes).<br />
<br />
Now the entire point of running this program for the purpose of this article is to get the inode number. To do this run the command (as root):<br />
<br />
debugfs<br />
<br />
Then in the debugfs console, use the open command on the EXT partition containing the bad sector:<br />
<br />
debugfs: open /dev/sdxy<br />
<br />
Finally, use the testb command to get information about the block in question (in this example block 1000):<br />
<br />
debugfs: testb 1000<br />
<br />
If debugfs says that block isn't in use then that means the block is not allocated and is being used as free space. In this case this is a good thing as it means nothing important was damaged by that sector.<br />
<br />
If the block is in use then run this command to get the inode number<br />
<br />
icheck 1000<br />
<br />
This will return two numbers. The block number and the inode number.<br />
<br />
==Find Damaged Files (EXT(2/3/4))==<br />
Use the inode number (second number from the icheck command) with the ncheck command:<br />
<br />
ncheck 12345<br />
<br />
debugfs will give you the full pathname to the file using the bad block. Now you will know what was actually damaged.<br />
<br />
If the inode number is very small and ncheck fails to return a path then it's probably the journal itself that is damaged. To delete the journal simply run this command on the partition:<br />
<br />
tune2fs -O ^has_journal /dev/sdxy<br />
<br />
Run the testb command again from the debugfs console on the bad block and it should be no longer marked as used if it was indeed used by the journal. To build a new journal run:<br />
<br />
tune2fs -j /dev/sdxy<br />
<br />
==Force the disk to reallocate bad block==<br />
First you'll want to see how many badblocks the harddrive is aware of through the smartctl command:<br />
<br />
smartctl -t long /dev/sdx [wait until test completes, then]<br />
smartctl -l selftest /dev/sdx<br />
<br />
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE<br />
'''5 Reallocated_Sector_Ct 0x0033 100 100 005 Pre-fail Always - 0'''<br />
'''196 Reallocated_Event_Count 0x0032 100 100 000 Old_age Always - 0'''<br />
197 Current_Pending_Sector 0x0022 100 100 000 Old_age Always - 0<br />
'''198 Offline_Uncorrectable 0x0008 100 100 000 Old_age Offline - 0'''<br />
<br />
<br />
To make the harddrive transparently map out the badblock with a spare good sector you will have to simply write zeros to the bad block using the dd command as root. Remember that with this command you have to work with the same block size as your filesystem and the block as to be relative to the partition the filesystem is on and NOT the harddrive as a whole:<br />
<br />
dd if=/dev/zero of=/dev/sdxy bs=4096 count=1 seek=2269012<br />
sync<br />
<br />
You can see if the harddrive did indeed map out an additional bad sector by checking with the smartctl command and seeing if the reallocated sector or event count went up:<br />
<br />
smartctl -A /dev/sdx<br />
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE<br />
'''5 Reallocated_Sector_Ct 0x0033 100 100 005 Pre-fail Always - 1'''<br />
'''196 Reallocated_Event_Count 0x0032 100 100 000 Old_age Always - 1'''<br />
197 Current_Pending_Sector 0x0022 100 100 000 Old_age Always - 0<br />
'''198 Offline_Uncorrectable 0x0008 100 100 000 Old_age Offline - 1'''<br />
<br />
To get Offline_Uncorrectable to go back to 0 you need to run a SMART long test and a selftest:<br />
<br />
smartctl -t long /dev/sdx [wait until test completes, then]<br />
smartctl -l selftest /dev/sdx<br />
<br />
==See also==<br />
<br />
[http://smartmontools.sourceforge.net/badblockhowto.html EXT2/3 badblocks howto]</div>MaBeefhttps://wiki.archlinux.org/index.php?title=Identify_damaged_files&diff=210756Identify damaged files2012-06-21T00:39:37Z<p>MaBeef: added a comma</p>
<hr />
<div>[[Category:Storage]]<br />
This article gives details on how to find out which file owns a given block. The main purpose for doing so is finding out which file was damaged in the event a storage device develops any bad blocks (that way you'll know if you lost anything important).<br />
<br />
For most of these commands you will have to be either root or a user that has direct read access to the drive you are checking (being a member of the disk group should be enough). As usual, a current backup is always a good idea, especially if imminent drive failure is suspected.[https://wiki.archlinux.org/index.php/SMART S.M.A.R.T.] can help with determining that.<br />
<br />
Right now this article is only written for JFS and EXT filesystems.<br />
<br />
==Finding Bad Blocks==<br />
Just use the [[badblocks]] command. There are a few scan modes supported by it. There's read-only mode (default) which is the least accurate. There is the destructive write-mode (-w option) which is the most accurate but takes longer and will (obviously) destroy all data on the drive, thus making it quite useless for matching blocks up to files. There is finally the non-destructive read-write mode which is probably as accurate as the destructive mode, the only real downside of which is it's probably the slowest. However, if a drive is known to be failing then read-only mode is probably still the safest.<br />
<br />
To do a verbose, read-only scan run this command (with x being the drive letter and y being partition number you want to scan):<br />
<br />
badblocks -v /dev/sdx<br />
<br />
or<br />
<br />
badblocks -v /dev/sdxy<br />
<br />
The downside to scanning the drive as a whole is that each filesystem is going to start its block count relative to the partition it's on. This means that if you have a bad block on let's say the second partition and the second partition starts on block 1000, then you have to subtract 1000 from your block number to get the number you want. So if block number 1005 is bad on that partition, then block 5 is what you'll actually be using.<br />
<br />
The alternative is if you've scanned the drive as a whole and have found bad blocks, figure out the partitions they fall on then scan those, using the new block numbers that result from that.<br />
<br />
Another thing to note is that badblocks defaults to 1024 byte blocks so you will either have to change the default size with the -b option to match your filesystem or manually convert the block number(s) later.<br />
<br />
If you need to figure out where your partitions start and end run fdisk (older versions might have defaulted to cylinders, not sure. If so the -u option will change the default unit to sectors). Make sure to note the block size fdisk is using so that you can convert the block counts to match your scan.<br />
<br />
fdisk -l /dev/sdx<br />
<br />
==Debug the Filesystem (JFS)==<br />
jfs_debugfs will give you access to all the low level structures within any JFS filesystem. Other filesystems such as the EXT filesystems have similar tools. It is probably a good idea to umount any filesystem before you run this on them. To use it just run (as root):<br />
<br />
jfs_debugfs /dev/sdxy<br />
<br />
This puts you into a command console. The first thing you should note is your aggregate block size. This is (presumably) the block size the filesystem is using. JFS seems to default to 4096 bytes. If you did not run badblocks using block size that your filesystem is using then you will need to convert your block numbers to match it (i.e. block number 100 with a block size of 1024 bytes becomes block number 25 at 4096 bytes).<br />
<br />
Now the entire point of running this program for the purpose of this article is to get the inode number. To do this run the command:<br />
<br />
d ''blocknumber'' 0 i<br />
<br />
The syntax is the d command for display, the block number, the offset (just set it to 0), and the display format i for inode.<br />
<br />
''Note'': If you get an error then that means the block is not allocated and is being used as free space. In this case this is a good thing as it means nothing important was damaged by that sector.<br />
<br />
The decimal number that di_number is set to is the one we want. From here you type x to exit out of the display mode. Repeat the display command for each bad block that you have and note all of their inode numbers. For more info on the inode such as permissions and filetype type:<br />
<br />
i ''inodenumber''<br />
<br />
i of course stands for inode.<br />
<br />
When you have all the inode numbers type q to quit.<br />
<br />
==Find Damaged Files (JFS)==<br />
<br />
Finally to find the damaged file you can simply use the gnu find utility. Mount your filesystem and run:<br />
<br />
find / -inum ''inodenumber''<br />
<br />
Substitute "/" for the mountpoint of the filesystem that the inode belongs to. If you search root and have more than one filesystem mounted (who doesn't?) you can find multiple files with the same inode number on different filesystems. Remember, an inode is only unique to the filesystem that it's in.<br />
<br />
==Debug the Filesystem (EXT(2/3/4))==<br />
tune2fs will give you access to all the low level structures within any EXT filesystem. It is probably a good idea to umount any filesystem before you run this on them.<br />
<br />
The first thing we want to do is get the block size from the filessystem in question. Just run (probably as root):<br />
<br />
tune2fs -l /dev/sdxy | grep Block<br />
Block count: 29119820<br />
'''Block size: 4096'''<br />
<br />
In this case 4096 is the block size being used (it appears to be the default). If you did not run badblocks using block size that your filesystem is using then you will need to convert your block numbers to match it (i.e. block number 100 with a block size of 1024 bytes becomes block number 25 at 4096 bytes).<br />
<br />
Now the entire point of running this program for the purpose of this article is to get the inode number. To do this run the command (as root):<br />
<br />
debugfs<br />
<br />
Then in the debugfs console, use the open command on the EXT partition containing the bad sector:<br />
<br />
debugfs: open /dev/sdxy<br />
<br />
Finally, use the testb command to get information about the block in question (in this example block 1000):<br />
<br />
debugfs: testb 1000<br />
<br />
If debugfs says that block isn't in use then that means the block is not allocated and is being used as free space. In this case this is a good thing as it means nothing important was damaged by that sector.<br />
<br />
If the block is in use then run this command to get the inode number<br />
<br />
icheck 1000<br />
<br />
This will return two numbers. The block number and the inode number.<br />
<br />
==Find Damaged Files (EXT(2/3/4))==<br />
Use the inode number (second number from the icheck command) with the ncheck command:<br />
<br />
ncheck 12345<br />
<br />
debugfs will give you the full pathname to the file using the bad block. Now you will know what was actually damaged.<br />
<br />
If the inode number is very small and ncheck fails to return a path then it's probably the journal itself that is damaged. To delete the journal simply run this command on the partition:<br />
<br />
tune2fs -O ^has_journal /dev/sdxy<br />
<br />
Run the testb command again from the debugfs console on the bad block and it should be no longer marked as used if it was indeed used by the journal. To build a new journal run:<br />
<br />
tune2fs -j /dev/sdxy<br />
<br />
==Force the disk to reallocate bad block==<br />
First you'll want to see how many badblocks the harddrive is aware of through the smartctl command:<br />
<br />
smartctl -t long /dev/sdx [wait until test completes, then]<br />
smartctl -l selftest /dev/sdx<br />
<br />
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE<br />
'''5 Reallocated_Sector_Ct 0x0033 100 100 005 Pre-fail Always - 0'''<br />
'''196 Reallocated_Event_Count 0x0032 100 100 000 Old_age Always - 0'''<br />
197 Current_Pending_Sector 0x0022 100 100 000 Old_age Always - 0<br />
'''198 Offline_Uncorrectable 0x0008 100 100 000 Old_age Offline - 0'''<br />
<br />
<br />
To make the harddrive transparently map out the badblock with a spare good sector you will have to simply write zeros to the bad block using the dd command as root. Remember that with this command you have to work with the same block size as your filesystem and the block as to be relative to the partition the filesystem is on and NOT the harddrive as a whole:<br />
<br />
dd if=/dev/zero of=/dev/sdxy bs=4096 count=1 seek=2269012<br />
sync<br />
<br />
You can see if the harddrive did indeed map out an additional bad sector by checking with the smartctl command and seeing if the reallocated sector or event count went up:<br />
<br />
smartctl -A /dev/sdx<br />
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE<br />
'''5 Reallocated_Sector_Ct 0x0033 100 100 005 Pre-fail Always - 1'''<br />
'''196 Reallocated_Event_Count 0x0032 100 100 000 Old_age Always - 1'''<br />
197 Current_Pending_Sector 0x0022 100 100 000 Old_age Always - 0<br />
'''198 Offline_Uncorrectable 0x0008 100 100 000 Old_age Offline - 1'''<br />
<br />
To get Offline_Uncorrectable to go back to 0 you need to run a SMART long test and a selftest:<br />
<br />
smartctl -t long /dev/sdx [wait until test completes, then]<br />
smartctl -l selftest /dev/sdx<br />
<br />
==See also==<br />
<br />
[http://smartmontools.sourceforge.net/badblockhowto.html EXT2/3 badblocks howto]</div>MaBeefhttps://wiki.archlinux.org/index.php?title=Identify_damaged_files&diff=210755Identify damaged files2012-06-21T00:38:49Z<p>MaBeef: Fixed some device names</p>
<hr />
<div>[[Category:Storage]]<br />
This article gives details on how to find out which file owns a given block. The main purpose for doing so is finding out which file was damaged in the event a storage device develops any bad blocks (that way you'll know if you lost anything important).<br />
<br />
For most of these commands you will have to be either root or a user that has direct read access to the drive you are checking (being a member of the disk group should be enough). As usual, a current backup is always a good idea, especially if imminent drive failure is suspected.[https://wiki.archlinux.org/index.php/SMART S.M.A.R.T.] can help with determining that.<br />
<br />
Right now this article is only written for JFS and EXT filesystems.<br />
<br />
==Finding Bad Blocks==<br />
Just use the [[badblocks]] command. There are a few scan modes supported by it. There's read-only mode (default) which is the least accurate. There is the destructive write-mode (-w option) which is the most accurate but takes longer and will (obviously) destroy all data on the drive, thus making it quite useless for matching blocks up to files. There is finally the non-destructive read-write mode which is probably as accurate as the destructive mode, the only real downside of which is it's probably the slowest. However, if a drive is known to be failing then read-only mode is probably still the safest.<br />
<br />
To do a verbose read-only scan run this command (with x being the drive letter and y being partition number you want to scan):<br />
<br />
badblocks -v /dev/sdx<br />
<br />
or<br />
<br />
badblocks -v /dev/sdxy<br />
<br />
The downside to scanning the drive as a whole is that each filesystem is going to start its block count relative to the partition it's on. This means that if you have a bad block on let's say the second partition and the second partition starts on block 1000, then you have to subtract 1000 from your block number to get the number you want. So if block number 1005 is bad on that partition, then block 5 is what you'll actually be using.<br />
<br />
The alternative is if you've scanned the drive as a whole and have found bad blocks, figure out the partitions they fall on then scan those, using the new block numbers that result from that.<br />
<br />
Another thing to note is that badblocks defaults to 1024 byte blocks so you will either have to change the default size with the -b option to match your filesystem or manually convert the block number(s) later.<br />
<br />
If you need to figure out where your partitions start and end run fdisk (older versions might have defaulted to cylinders, not sure. If so the -u option will change the default unit to sectors). Make sure to note the block size fdisk is using so that you can convert the block counts to match your scan.<br />
<br />
fdisk -l /dev/sdx<br />
<br />
==Debug the Filesystem (JFS)==<br />
jfs_debugfs will give you access to all the low level structures within any JFS filesystem. Other filesystems such as the EXT filesystems have similar tools. It is probably a good idea to umount any filesystem before you run this on them. To use it just run (as root):<br />
<br />
jfs_debugfs /dev/sdxy<br />
<br />
This puts you into a command console. The first thing you should note is your aggregate block size. This is (presumably) the block size the filesystem is using. JFS seems to default to 4096 bytes. If you did not run badblocks using block size that your filesystem is using then you will need to convert your block numbers to match it (i.e. block number 100 with a block size of 1024 bytes becomes block number 25 at 4096 bytes).<br />
<br />
Now the entire point of running this program for the purpose of this article is to get the inode number. To do this run the command:<br />
<br />
d ''blocknumber'' 0 i<br />
<br />
The syntax is the d command for display, the block number, the offset (just set it to 0), and the display format i for inode.<br />
<br />
''Note'': If you get an error then that means the block is not allocated and is being used as free space. In this case this is a good thing as it means nothing important was damaged by that sector.<br />
<br />
The decimal number that di_number is set to is the one we want. From here you type x to exit out of the display mode. Repeat the display command for each bad block that you have and note all of their inode numbers. For more info on the inode such as permissions and filetype type:<br />
<br />
i ''inodenumber''<br />
<br />
i of course stands for inode.<br />
<br />
When you have all the inode numbers type q to quit.<br />
<br />
==Find Damaged Files (JFS)==<br />
<br />
Finally to find the damaged file you can simply use the gnu find utility. Mount your filesystem and run:<br />
<br />
find / -inum ''inodenumber''<br />
<br />
Substitute "/" for the mountpoint of the filesystem that the inode belongs to. If you search root and have more than one filesystem mounted (who doesn't?) you can find multiple files with the same inode number on different filesystems. Remember, an inode is only unique to the filesystem that it's in.<br />
<br />
==Debug the Filesystem (EXT(2/3/4))==<br />
tune2fs will give you access to all the low level structures within any EXT filesystem. It is probably a good idea to umount any filesystem before you run this on them.<br />
<br />
The first thing we want to do is get the block size from the filessystem in question. Just run (probably as root):<br />
<br />
tune2fs -l /dev/sdxy | grep Block<br />
Block count: 29119820<br />
'''Block size: 4096'''<br />
<br />
In this case 4096 is the block size being used (it appears to be the default). If you did not run badblocks using block size that your filesystem is using then you will need to convert your block numbers to match it (i.e. block number 100 with a block size of 1024 bytes becomes block number 25 at 4096 bytes).<br />
<br />
Now the entire point of running this program for the purpose of this article is to get the inode number. To do this run the command (as root):<br />
<br />
debugfs<br />
<br />
Then in the debugfs console, use the open command on the EXT partition containing the bad sector:<br />
<br />
debugfs: open /dev/sdxy<br />
<br />
Finally, use the testb command to get information about the block in question (in this example block 1000):<br />
<br />
debugfs: testb 1000<br />
<br />
If debugfs says that block isn't in use then that means the block is not allocated and is being used as free space. In this case this is a good thing as it means nothing important was damaged by that sector.<br />
<br />
If the block is in use then run this command to get the inode number<br />
<br />
icheck 1000<br />
<br />
This will return two numbers. The block number and the inode number.<br />
<br />
==Find Damaged Files (EXT(2/3/4))==<br />
Use the inode number (second number from the icheck command) with the ncheck command:<br />
<br />
ncheck 12345<br />
<br />
debugfs will give you the full pathname to the file using the bad block. Now you will know what was actually damaged.<br />
<br />
If the inode number is very small and ncheck fails to return a path then it's probably the journal itself that is damaged. To delete the journal simply run this command on the partition:<br />
<br />
tune2fs -O ^has_journal /dev/sdxy<br />
<br />
Run the testb command again from the debugfs console on the bad block and it should be no longer marked as used if it was indeed used by the journal. To build a new journal run:<br />
<br />
tune2fs -j /dev/sdxy<br />
<br />
==Force the disk to reallocate bad block==<br />
First you'll want to see how many badblocks the harddrive is aware of through the smartctl command:<br />
<br />
smartctl -t long /dev/sdx [wait until test completes, then]<br />
smartctl -l selftest /dev/sdx<br />
<br />
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE<br />
'''5 Reallocated_Sector_Ct 0x0033 100 100 005 Pre-fail Always - 0'''<br />
'''196 Reallocated_Event_Count 0x0032 100 100 000 Old_age Always - 0'''<br />
197 Current_Pending_Sector 0x0022 100 100 000 Old_age Always - 0<br />
'''198 Offline_Uncorrectable 0x0008 100 100 000 Old_age Offline - 0'''<br />
<br />
<br />
To make the harddrive transparently map out the badblock with a spare good sector you will have to simply write zeros to the bad block using the dd command as root. Remember that with this command you have to work with the same block size as your filesystem and the block as to be relative to the partition the filesystem is on and NOT the harddrive as a whole:<br />
<br />
dd if=/dev/zero of=/dev/sdxy bs=4096 count=1 seek=2269012<br />
sync<br />
<br />
You can see if the harddrive did indeed map out an additional bad sector by checking with the smartctl command and seeing if the reallocated sector or event count went up:<br />
<br />
smartctl -A /dev/sdx<br />
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE<br />
'''5 Reallocated_Sector_Ct 0x0033 100 100 005 Pre-fail Always - 1'''<br />
'''196 Reallocated_Event_Count 0x0032 100 100 000 Old_age Always - 1'''<br />
197 Current_Pending_Sector 0x0022 100 100 000 Old_age Always - 0<br />
'''198 Offline_Uncorrectable 0x0008 100 100 000 Old_age Offline - 1'''<br />
<br />
To get Offline_Uncorrectable to go back to 0 you need to run a SMART long test and a selftest:<br />
<br />
smartctl -t long /dev/sdx [wait until test completes, then]<br />
smartctl -l selftest /dev/sdx<br />
<br />
==See also==<br />
<br />
[http://smartmontools.sourceforge.net/badblockhowto.html EXT2/3 badblocks howto]</div>MaBeefhttps://wiki.archlinux.org/index.php?title=Identify_damaged_files&diff=210754Identify damaged files2012-06-21T00:27:44Z<p>MaBeef: More intro changes</p>
<hr />
<div>[[Category:Storage]]<br />
This article gives details on how to find out which file owns a given block. The main purpose for doing so is finding out which file was damaged in the event a storage device develops any bad blocks (that way you'll know if you lost anything important).<br />
<br />
For most of these commands you will have to be either root or a user that has direct read access to the drive you are checking (being a member of the disk group should be enough). As usual, a current backup is always a good idea, especially if imminent drive failure is suspected.[https://wiki.archlinux.org/index.php/SMART S.M.A.R.T.] can help with determining that.<br />
<br />
Right now this article is only written for JFS and EXT filesystems.<br />
<br />
==Finding Bad Blocks==<br />
Just use the [[badblocks]] command. There are a few scan modes supported by it. There's read-only mode (default) which is the least accurate. There is the destructive write-mode (-w option) which is the most accurate but takes longer and will (obviously) destroy all data on the drive, thus making it quite useless for matching blocks up to files. There is finally the non-destructive read-write mode which is probably as accurate as the destructive mode, the only real downside of which is it's probably the slowest. However, if a drive is known to be failing then read-only mode is probably still the safest.<br />
<br />
To do a verbose read-only scan run this command (with x being the drive letter and y being partition number you want to scan):<br />
<br />
badblocks -v /dev/sdx<br />
<br />
or<br />
<br />
badblocks -v /dev/sdxy<br />
<br />
The downside to scanning the drive as a whole is that each filesystem is going to start its block count relative to the partition it's on. This means that if you have a bad block on let's say the second partition and the second partition starts on block 1000, then you have to subtract 1000 from your block number to get the number you want. So if block number 1005 is bad on that partition, then block 5 is what you'll actually be using.<br />
<br />
The alternative is if you've scanned the drive as a whole and have found bad blocks, figure out the partitions they fall on then scan those, using the new block numbers that result from that.<br />
<br />
Another thing to note is that badblocks defaults to 1024 byte blocks so you will either have to change the default size with the -b option to match your filesystem or manually convert the block number(s) later.<br />
<br />
If you need to figure out where your partitions start and end run fdisk (older versions might have defaulted to cylinders, not sure. If so the -u option will change the default unit to sectors). Make sure to note the block size fdisk is using so that you can convert the block counts to match your scan.<br />
<br />
fdisk -l /dev/sdx<br />
<br />
==Debug the Filesystem (JFS)==<br />
jfs_debugfs will give you access to all the low level structures within any JFS filesystem. Other filesystems such as the EXT filesystems have similar tools. It is probably a good idea to umount any filesystem before you run this on them. To use it just run (as root):<br />
<br />
jfs_debugfs /dev/sdxy<br />
<br />
This puts you into a command console. The first thing you should note is your aggregate block size. This is (presumably) the block size the filesystem is using. JFS seems to default to 4096 bytes. If you did not run badblocks using block size that your filesystem is using then you will need to convert your block numbers to match it (i.e. block number 100 with a block size of 1024 bytes becomes block number 25 at 4096 bytes).<br />
<br />
Now the entire point of running this program for the purpose of this article is to get the inode number. To do this run the command:<br />
<br />
d ''blocknumber'' 0 i<br />
<br />
The syntax is the d command for display, the block number, the offset (just set it to 0), and the display format i for inode.<br />
<br />
''Note'': If you get an error then that means the block is not allocated and is being used as free space. In this case this is a good thing as it means nothing important was damaged by that sector.<br />
<br />
The decimal number that di_number is set to is the one we want. From here you type x to exit out of the display mode. Repeat the display command for each bad block that you have and note all of their inode numbers. For more info on the inode such as permissions and filetype type:<br />
<br />
i ''inodenumber''<br />
<br />
i of course stands for inode.<br />
<br />
When you have all the inode numbers type q to quit.<br />
<br />
==Find Damaged Files (JFS)==<br />
<br />
Finally to find the damaged file you can simply use the gnu find utility. Mount your filesystem and run:<br />
<br />
find / -inum ''inodenumber''<br />
<br />
Substitute "/" for the mountpoint of the filesystem that the inode belongs to. If you search root and have more than one filesystem mounted (who doesn't?) you can find multiple files with the same inode number on different filesystems. Remember, an inode is only unique to the filesystem that it's in.<br />
<br />
==Debug the Filesystem (EXT(2/3/4))==<br />
tune2fs will give you access to all the low level structures within any EXT filesystem. It is probably a good idea to umount any filesystem before you run this on them.<br />
<br />
The first thing we want to do is get the block size from the filessystem in question. Just run (probably as root):<br />
<br />
tune2fs -l /dev/hda3 | grep Block<br />
Block count: 29119820<br />
'''Block size: 4096'''<br />
<br />
In this case 4096 is the block size being used (it appears to be the default). If you did not run badblocks using block size that your filesystem is using then you will need to convert your block numbers to match it (i.e. block number 100 with a block size of 1024 bytes becomes block number 25 at 4096 bytes).<br />
<br />
Now the entire point of running this program for the purpose of this article is to get the inode number. To do this run the command (as root):<br />
<br />
debugfs<br />
<br />
Then in the debugfs console, use the open command on the EXT partition containing the bad sector:<br />
<br />
debugfs: open /dev/sdxy<br />
<br />
Finally, use the testb command to get information about the block in question (in this example block 1000):<br />
<br />
debugfs: testb 1000<br />
<br />
If debugfs says that block isn't in use then that means the block is not allocated and is being used as free space. In this case this is a good thing as it means nothing important was damaged by that sector.<br />
<br />
If the block is in use then run this command to get the inode number<br />
<br />
icheck 1000<br />
<br />
This will return two numbers. The block number and the inode number.<br />
<br />
==Find Damaged Files (EXT(2/3/4))==<br />
Use the inode number (second number from the icheck command) with the ncheck command:<br />
<br />
ncheck 12345<br />
<br />
debugfs will give you the full pathname to the file using the bad block. Now you will know what was actually damaged.<br />
<br />
If the inode number is very small and ncheck fails to return a path then it's probably the journal itself that is damaged. To delete the journal simply run this command on the partition:<br />
<br />
tune2fs -O ^has_journal /dev/sdxy<br />
<br />
Run the testb command again from the debugfs console on the bad block and it should be no longer marked as used if it was indeed used by the journal. To build a new journal run:<br />
<br />
tune2fs -j /dev/sdxy<br />
<br />
==Force the disk to reallocate bad block==<br />
First you'll want to see how many badblocks the harddrive is aware of through the smartctl command:<br />
<br />
smartctl -t long /dev/hda [wait until test completes, then]<br />
smartctl -l selftest /dev/hda<br />
<br />
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE<br />
'''5 Reallocated_Sector_Ct 0x0033 100 100 005 Pre-fail Always - 0'''<br />
'''196 Reallocated_Event_Count 0x0032 100 100 000 Old_age Always - 0'''<br />
197 Current_Pending_Sector 0x0022 100 100 000 Old_age Always - 0<br />
'''198 Offline_Uncorrectable 0x0008 100 100 000 Old_age Offline - 0'''<br />
<br />
<br />
To make the harddrive transparently map out the badblock with a spare good sector you will have to simply write zeros to the bad block using the dd command as root. Remember that with this command you have to work with the same block size as your filesystem and the block as to be relative to the partition the filesystem is on and NOT the harddrive as a whole:<br />
<br />
dd if=/dev/zero of=/dev/hda3 bs=4096 count=1 seek=2269012<br />
sync<br />
<br />
You can see if the harddrive did indeed map out an additional bad sector by checking with the smartctl command and seeing if the reallocated sector or event count went up:<br />
<br />
smartctl -A /dev/hda<br />
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE<br />
'''5 Reallocated_Sector_Ct 0x0033 100 100 005 Pre-fail Always - 1'''<br />
'''196 Reallocated_Event_Count 0x0032 100 100 000 Old_age Always - 1'''<br />
197 Current_Pending_Sector 0x0022 100 100 000 Old_age Always - 0<br />
'''198 Offline_Uncorrectable 0x0008 100 100 000 Old_age Offline - 1'''<br />
<br />
To get Offline_Uncorrectable to go back to 0 you need to run a SMART long test and a selftest:<br />
<br />
smartctl -t long /dev/hda [wait until test completes, then]<br />
smartctl -l selftest /dev/hda<br />
<br />
==See also==<br />
<br />
[http://smartmontools.sourceforge.net/badblockhowto.html EXT2/3 badblocks howto]</div>MaBeefhttps://wiki.archlinux.org/index.php?title=Identify_damaged_files&diff=210753Identify damaged files2012-06-21T00:24:46Z<p>MaBeef: Change in intro</p>
<hr />
<div>[[Category:Storage]]<br />
This article gives details on how to find out which file owns a given block. The main purpose for doing so is finding out which file was damaged in the event a storage device develops any bad blocks (that way you'll know if you lost anything important).<br />
<br />
For these commands you will have to either be root or a user that has direct read access to the drive you are scanning. As usual, a current backup is always a good idea, especially if imminent drive failure is suspected.<br />
<br />
Right now this is only written for JFS and EXT filesystems (I'd like to also figure this out for XFS -MaBeef).<br />
<br />
==Finding Bad Blocks==<br />
Just use the [[badblocks]] command. There are a few scan modes supported by it. There's read-only mode (default) which is the least accurate. There is the destructive write-mode (-w option) which is the most accurate but takes longer and will (obviously) destroy all data on the drive, thus making it quite useless for matching blocks up to files. There is finally the non-destructive read-write mode which is probably as accurate as the destructive mode, the only real downside of which is it's probably the slowest. However, if a drive is known to be failing then read-only mode is probably still the safest.<br />
<br />
To do a verbose read-only scan run this command (with x being the drive letter and y being partition number you want to scan):<br />
<br />
badblocks -v /dev/sdx<br />
<br />
or<br />
<br />
badblocks -v /dev/sdxy<br />
<br />
The downside to scanning the drive as a whole is that each filesystem is going to start its block count relative to the partition it's on. This means that if you have a bad block on let's say the second partition and the second partition starts on block 1000, then you have to subtract 1000 from your block number to get the number you want. So if block number 1005 is bad on that partition, then block 5 is what you'll actually be using.<br />
<br />
The alternative is if you've scanned the drive as a whole and have found bad blocks, figure out the partitions they fall on then scan those, using the new block numbers that result from that.<br />
<br />
Another thing to note is that badblocks defaults to 1024 byte blocks so you will either have to change the default size with the -b option to match your filesystem or manually convert the block number(s) later.<br />
<br />
If you need to figure out where your partitions start and end run fdisk (older versions might have defaulted to cylinders, not sure. If so the -u option will change the default unit to sectors). Make sure to note the block size fdisk is using so that you can convert the block counts to match your scan.<br />
<br />
fdisk -l /dev/sdx<br />
<br />
==Debug the Filesystem (JFS)==<br />
jfs_debugfs will give you access to all the low level structures within any JFS filesystem. Other filesystems such as the EXT filesystems have similar tools. It is probably a good idea to umount any filesystem before you run this on them. To use it just run (as root):<br />
<br />
jfs_debugfs /dev/sdxy<br />
<br />
This puts you into a command console. The first thing you should note is your aggregate block size. This is (presumably) the block size the filesystem is using. JFS seems to default to 4096 bytes. If you did not run badblocks using block size that your filesystem is using then you will need to convert your block numbers to match it (i.e. block number 100 with a block size of 1024 bytes becomes block number 25 at 4096 bytes).<br />
<br />
Now the entire point of running this program for the purpose of this article is to get the inode number. To do this run the command:<br />
<br />
d ''blocknumber'' 0 i<br />
<br />
The syntax is the d command for display, the block number, the offset (just set it to 0), and the display format i for inode.<br />
<br />
''Note'': If you get an error then that means the block is not allocated and is being used as free space. In this case this is a good thing as it means nothing important was damaged by that sector.<br />
<br />
The decimal number that di_number is set to is the one we want. From here you type x to exit out of the display mode. Repeat the display command for each bad block that you have and note all of their inode numbers. For more info on the inode such as permissions and filetype type:<br />
<br />
i ''inodenumber''<br />
<br />
i of course stands for inode.<br />
<br />
When you have all the inode numbers type q to quit.<br />
<br />
==Find Damaged Files (JFS)==<br />
<br />
Finally to find the damaged file you can simply use the gnu find utility. Mount your filesystem and run:<br />
<br />
find / -inum ''inodenumber''<br />
<br />
Substitute "/" for the mountpoint of the filesystem that the inode belongs to. If you search root and have more than one filesystem mounted (who doesn't?) you can find multiple files with the same inode number on different filesystems. Remember, an inode is only unique to the filesystem that it's in.<br />
<br />
==Debug the Filesystem (EXT(2/3/4))==<br />
tune2fs will give you access to all the low level structures within any EXT filesystem. It is probably a good idea to umount any filesystem before you run this on them.<br />
<br />
The first thing we want to do is get the block size from the filessystem in question. Just run (probably as root):<br />
<br />
tune2fs -l /dev/hda3 | grep Block<br />
Block count: 29119820<br />
'''Block size: 4096'''<br />
<br />
In this case 4096 is the block size being used (it appears to be the default). If you did not run badblocks using block size that your filesystem is using then you will need to convert your block numbers to match it (i.e. block number 100 with a block size of 1024 bytes becomes block number 25 at 4096 bytes).<br />
<br />
Now the entire point of running this program for the purpose of this article is to get the inode number. To do this run the command (as root):<br />
<br />
debugfs<br />
<br />
Then in the debugfs console, use the open command on the EXT partition containing the bad sector:<br />
<br />
debugfs: open /dev/sdxy<br />
<br />
Finally, use the testb command to get information about the block in question (in this example block 1000):<br />
<br />
debugfs: testb 1000<br />
<br />
If debugfs says that block isn't in use then that means the block is not allocated and is being used as free space. In this case this is a good thing as it means nothing important was damaged by that sector.<br />
<br />
If the block is in use then run this command to get the inode number<br />
<br />
icheck 1000<br />
<br />
This will return two numbers. The block number and the inode number.<br />
<br />
==Find Damaged Files (EXT(2/3/4))==<br />
Use the inode number (second number from the icheck command) with the ncheck command:<br />
<br />
ncheck 12345<br />
<br />
debugfs will give you the full pathname to the file using the bad block. Now you will know what was actually damaged.<br />
<br />
If the inode number is very small and ncheck fails to return a path then it's probably the journal itself that is damaged. To delete the journal simply run this command on the partition:<br />
<br />
tune2fs -O ^has_journal /dev/sdxy<br />
<br />
Run the testb command again from the debugfs console on the bad block and it should be no longer marked as used if it was indeed used by the journal. To build a new journal run:<br />
<br />
tune2fs -j /dev/sdxy<br />
<br />
==Force the disk to reallocate bad block==<br />
First you'll want to see how many badblocks the harddrive is aware of through the smartctl command:<br />
<br />
smartctl -t long /dev/hda [wait until test completes, then]<br />
smartctl -l selftest /dev/hda<br />
<br />
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE<br />
'''5 Reallocated_Sector_Ct 0x0033 100 100 005 Pre-fail Always - 0'''<br />
'''196 Reallocated_Event_Count 0x0032 100 100 000 Old_age Always - 0'''<br />
197 Current_Pending_Sector 0x0022 100 100 000 Old_age Always - 0<br />
'''198 Offline_Uncorrectable 0x0008 100 100 000 Old_age Offline - 0'''<br />
<br />
<br />
To make the harddrive transparently map out the badblock with a spare good sector you will have to simply write zeros to the bad block using the dd command as root. Remember that with this command you have to work with the same block size as your filesystem and the block as to be relative to the partition the filesystem is on and NOT the harddrive as a whole:<br />
<br />
dd if=/dev/zero of=/dev/hda3 bs=4096 count=1 seek=2269012<br />
sync<br />
<br />
You can see if the harddrive did indeed map out an additional bad sector by checking with the smartctl command and seeing if the reallocated sector or event count went up:<br />
<br />
smartctl -A /dev/hda<br />
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE<br />
'''5 Reallocated_Sector_Ct 0x0033 100 100 005 Pre-fail Always - 1'''<br />
'''196 Reallocated_Event_Count 0x0032 100 100 000 Old_age Always - 1'''<br />
197 Current_Pending_Sector 0x0022 100 100 000 Old_age Always - 0<br />
'''198 Offline_Uncorrectable 0x0008 100 100 000 Old_age Offline - 1'''<br />
<br />
To get Offline_Uncorrectable to go back to 0 you need to run a SMART long test and a selftest:<br />
<br />
smartctl -t long /dev/hda [wait until test completes, then]<br />
smartctl -l selftest /dev/hda<br />
<br />
==See also==<br />
<br />
[http://smartmontools.sourceforge.net/badblockhowto.html EXT2/3 badblocks howto]</div>MaBeefhttps://wiki.archlinux.org/index.php?title=Identify_damaged_files&diff=203315Identify damaged files2012-05-31T07:08:52Z<p>MaBeef: Made the smartctl output line up better</p>
<hr />
<div>[[Category:Storage]]<br />
{{i18n|Find File that Owns a Given Block}}<br />
<br />
This article gives details on how to find out which file owns a given block. The main purpose for doing so is finding out which file was damaged in the event a storage device develops any bad blocks.<br />
<br />
For these commands you will have to either be root or a user that has direct read access to the drive you are scanning. As usual, a current backup is always a good idea, especially if imminent drive failure is suspected.<br />
<br />
Right now this is only written for JFS and EXT filesystems (I'd like to also figure this out for XFS -MaBeef).<br />
<br />
==Finding Bad Blocks==<br />
Just use the [[badblocks]] command. There are a few scan modes supported by it. There's read-only mode (default) which is the least accurate. There is the destructive write-mode (-w option) which is the most accurate but takes longer and will (obviously) destroy all data on the drive, thus making it quite useless for matching blocks up to files. There is finally the non-destructive read-write mode which is probably as accurate as the destructive mode, the only real downside of which is it's probably the slowest. However, if a drive is known to be failing then read-only mode is probably still the safest.<br />
<br />
To do a verbose read-only scan run this command (with x being the drive letter and y being partition number you want to scan):<br />
<br />
badblocks -v /dev/sdx<br />
<br />
or<br />
<br />
badblocks -v /dev/sdxy<br />
<br />
The downside to scanning the drive as a whole is that each filesystem is going to start its block count relative to the partition it's on. This means that if you have a bad block on let's say the second partition and the second partition starts on block 1000, then you have to subtract 1000 from your block number to get the number you want. So if block number 1005 is bad on that partition, then block 5 is what you'll actually be using.<br />
<br />
The alternative is if you've scanned the drive as a whole and have found bad blocks, figure out the partitions they fall on then scan those, using the new block numbers that result from that.<br />
<br />
Another thing to note is that badblocks defaults to 1024 byte blocks so you will either have to change the default size with the -b option to match your filesystem or manually convert the block number(s) later.<br />
<br />
If you need to figure out where your partitions start and end run fdisk (older versions might have defaulted to cylinders, not sure. If so the -u option will change the default unit to sectors). Make sure to note the block size fdisk is using so that you can convert the block counts to match your scan.<br />
<br />
fdisk -l /dev/sdx<br />
<br />
==Debug the Filesystem (JFS)==<br />
jfs_debugfs will give you access to all the low level structures within any JFS filesystem. Other filesystems such as the EXT filesystems have similar tools. It is probably a good idea to umount any filesystem before you run this on them. To use it just run (as root):<br />
<br />
jfs_debugfs /dev/sdxy<br />
<br />
This puts you into a command console. The first thing you should note is your aggregate block size. This is (presumably) the block size the filesystem is using. JFS seems to default to 4096 bytes. If you did not run badblocks using block size that your filesystem is using then you will need to convert your block numbers to match it (i.e. block number 100 with a block size of 1024 bytes becomes block number 25 at 4096 bytes).<br />
<br />
Now the entire point of running this program for the purpose of this article is to get the inode number. To do this run the command:<br />
<br />
d ''blocknumber'' 0 i<br />
<br />
The syntax is the d command for display, the block number, the offset (just set it to 0), and the display format i for inode.<br />
<br />
''Note'': If you get an error then that means the block is not allocated and is being used as free space. In this case this is a good thing as it means nothing important was damaged by that sector.<br />
<br />
The decimal number that di_number is set to is the one we want. From here you type x to exit out of the display mode. Repeat the display command for each bad block that you have and note all of their inode numbers. For more info on the inode such as permissions and filetype type:<br />
<br />
i ''inodenumber''<br />
<br />
i of course stands for inode.<br />
<br />
When you have all the inode numbers type q to quit.<br />
<br />
==Find Damaged Files (JFS)==<br />
<br />
Finally to find the damaged file you can simply use the gnu find utility. Mount your filesystem and run:<br />
<br />
find / -inum ''inodenumber''<br />
<br />
Substitute "/" for the mountpoint of the filesystem that the inode belongs to. If you search root and have more than one filesystem mounted (who doesn't?) you can find multiple files with the same inode number on different filesystems. Remember, an inode is only unique to the filesystem that it's in.<br />
<br />
==Debug the Filesystem (EXT(2/3/4))==<br />
tune2fs will give you access to all the low level structures within any EXT filesystem. It is probably a good idea to umount any filesystem before you run this on them.<br />
<br />
The first thing we want to do is get the block size from the filessystem in question. Just run (probably as root):<br />
<br />
tune2fs -l /dev/hda3 | grep Block<br />
Block count: 29119820<br />
'''Block size: 4096'''<br />
<br />
In this case 4096 is the block size being used (it appears to be the default). If you did not run badblocks using block size that your filesystem is using then you will need to convert your block numbers to match it (i.e. block number 100 with a block size of 1024 bytes becomes block number 25 at 4096 bytes).<br />
<br />
Now the entire point of running this program for the purpose of this article is to get the inode number. To do this run the command (as root):<br />
<br />
debugfs<br />
<br />
Then in the debugfs console, use the open command on the EXT partition containing the bad sector:<br />
<br />
debugfs: open /dev/sdxy<br />
<br />
Finally, use the testb command to get information about the block in question (in this example block 1000):<br />
<br />
debugfs: testb 1000<br />
<br />
If debugfs says that block isn't in use then that means the block is not allocated and is being used as free space. In this case this is a good thing as it means nothing important was damaged by that sector.<br />
<br />
If the block is in use then run this command to get the inode number<br />
<br />
icheck 1000<br />
<br />
This will return two numbers. The block number and the inode number.<br />
<br />
==Find Damaged Files (EXT(2/3/4))==<br />
Use the inode number (second number from the icheck command) with the ncheck command:<br />
<br />
ncheck 12345<br />
<br />
debugfs will give you the full pathname to the file using the bad block. Now you will know what was actually damaged.<br />
<br />
If the inode number is very small and ncheck fails to return a path then it's probably the journal itself that is damaged. To delete the journal simply run this command on the partition:<br />
<br />
tune2fs -O ^has_journal /dev/sdxy<br />
<br />
Run the testb command again from the debugfs console on the bad block and it should be no longer marked as used if it was indeed used by the journal. To build a new journal run:<br />
<br />
tune2fs -j /dev/sdxy<br />
<br />
==Force the disk to reallocate bad block==<br />
First you'll want to see how many badblocks the harddrive is aware of through the smartctl command:<br />
<br />
smartctl -t long /dev/hda [wait until test completes, then]<br />
smartctl -l selftest /dev/hda<br />
<br />
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE<br />
'''5 Reallocated_Sector_Ct 0x0033 100 100 005 Pre-fail Always - 0'''<br />
'''196 Reallocated_Event_Count 0x0032 100 100 000 Old_age Always - 0'''<br />
197 Current_Pending_Sector 0x0022 100 100 000 Old_age Always - 0<br />
'''198 Offline_Uncorrectable 0x0008 100 100 000 Old_age Offline - 0'''<br />
<br />
<br />
To make the harddrive transparently map out the badblock with a spare good sector you will have to simply write zeros to the bad block using the dd command as root. Remember that with this command you have to work with the same block size as your filesystem and the block as to be relative to the partition the filesystem is on and NOT the harddrive as a whole:<br />
<br />
dd if=/dev/zero of=/dev/hda3 bs=4096 count=1 seek=2269012<br />
sync<br />
<br />
You can see if the harddrive did indeed map out an additional bad sector by checking with the smartctl command and seeing if the reallocated sector or event count went up:<br />
<br />
smartctl -A /dev/hda<br />
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE<br />
'''5 Reallocated_Sector_Ct 0x0033 100 100 005 Pre-fail Always - 1'''<br />
'''196 Reallocated_Event_Count 0x0032 100 100 000 Old_age Always - 1'''<br />
197 Current_Pending_Sector 0x0022 100 100 000 Old_age Always - 0<br />
'''198 Offline_Uncorrectable 0x0008 100 100 000 Old_age Offline - 1'''<br />
<br />
To get Offline_Uncorrectable to go back to 0 you need to run a SMART long test and a selftest:<br />
<br />
smartctl -t long /dev/hda [wait until test completes, then]<br />
smartctl -l selftest /dev/hda<br />
<br />
==See also==<br />
<br />
[http://smartmontools.sourceforge.net/badblockhowto.html EXT2/3 badblocks howto]</div>MaBeefhttps://wiki.archlinux.org/index.php?title=Identify_damaged_files&diff=203314Identify damaged files2012-05-31T07:06:47Z<p>MaBeef: </p>
<hr />
<div>[[Category:Storage]]<br />
{{i18n|Find File that Owns a Given Block}}<br />
<br />
This article gives details on how to find out which file owns a given block. The main purpose for doing so is finding out which file was damaged in the event a storage device develops any bad blocks.<br />
<br />
For these commands you will have to either be root or a user that has direct read access to the drive you are scanning. As usual, a current backup is always a good idea, especially if imminent drive failure is suspected.<br />
<br />
Right now this is only written for JFS and EXT filesystems (I'd like to also figure this out for XFS -MaBeef).<br />
<br />
==Finding Bad Blocks==<br />
Just use the [[badblocks]] command. There are a few scan modes supported by it. There's read-only mode (default) which is the least accurate. There is the destructive write-mode (-w option) which is the most accurate but takes longer and will (obviously) destroy all data on the drive, thus making it quite useless for matching blocks up to files. There is finally the non-destructive read-write mode which is probably as accurate as the destructive mode, the only real downside of which is it's probably the slowest. However, if a drive is known to be failing then read-only mode is probably still the safest.<br />
<br />
To do a verbose read-only scan run this command (with x being the drive letter and y being partition number you want to scan):<br />
<br />
badblocks -v /dev/sdx<br />
<br />
or<br />
<br />
badblocks -v /dev/sdxy<br />
<br />
The downside to scanning the drive as a whole is that each filesystem is going to start its block count relative to the partition it's on. This means that if you have a bad block on let's say the second partition and the second partition starts on block 1000, then you have to subtract 1000 from your block number to get the number you want. So if block number 1005 is bad on that partition, then block 5 is what you'll actually be using.<br />
<br />
The alternative is if you've scanned the drive as a whole and have found bad blocks, figure out the partitions they fall on then scan those, using the new block numbers that result from that.<br />
<br />
Another thing to note is that badblocks defaults to 1024 byte blocks so you will either have to change the default size with the -b option to match your filesystem or manually convert the block number(s) later.<br />
<br />
If you need to figure out where your partitions start and end run fdisk (older versions might have defaulted to cylinders, not sure. If so the -u option will change the default unit to sectors). Make sure to note the block size fdisk is using so that you can convert the block counts to match your scan.<br />
<br />
fdisk -l /dev/sdx<br />
<br />
==Debug the Filesystem (JFS)==<br />
jfs_debugfs will give you access to all the low level structures within any JFS filesystem. Other filesystems such as the EXT filesystems have similar tools. It is probably a good idea to umount any filesystem before you run this on them. To use it just run (as root):<br />
<br />
jfs_debugfs /dev/sdxy<br />
<br />
This puts you into a command console. The first thing you should note is your aggregate block size. This is (presumably) the block size the filesystem is using. JFS seems to default to 4096 bytes. If you did not run badblocks using block size that your filesystem is using then you will need to convert your block numbers to match it (i.e. block number 100 with a block size of 1024 bytes becomes block number 25 at 4096 bytes).<br />
<br />
Now the entire point of running this program for the purpose of this article is to get the inode number. To do this run the command:<br />
<br />
d ''blocknumber'' 0 i<br />
<br />
The syntax is the d command for display, the block number, the offset (just set it to 0), and the display format i for inode.<br />
<br />
''Note'': If you get an error then that means the block is not allocated and is being used as free space. In this case this is a good thing as it means nothing important was damaged by that sector.<br />
<br />
The decimal number that di_number is set to is the one we want. From here you type x to exit out of the display mode. Repeat the display command for each bad block that you have and note all of their inode numbers. For more info on the inode such as permissions and filetype type:<br />
<br />
i ''inodenumber''<br />
<br />
i of course stands for inode.<br />
<br />
When you have all the inode numbers type q to quit.<br />
<br />
==Find Damaged Files (JFS)==<br />
<br />
Finally to find the damaged file you can simply use the gnu find utility. Mount your filesystem and run:<br />
<br />
find / -inum ''inodenumber''<br />
<br />
Substitute "/" for the mountpoint of the filesystem that the inode belongs to. If you search root and have more than one filesystem mounted (who doesn't?) you can find multiple files with the same inode number on different filesystems. Remember, an inode is only unique to the filesystem that it's in.<br />
<br />
==Debug the Filesystem (EXT(2/3/4))==<br />
tune2fs will give you access to all the low level structures within any EXT filesystem. It is probably a good idea to umount any filesystem before you run this on them.<br />
<br />
The first thing we want to do is get the block size from the filessystem in question. Just run (probably as root):<br />
<br />
tune2fs -l /dev/hda3 | grep Block<br />
Block count: 29119820<br />
'''Block size: 4096'''<br />
<br />
In this case 4096 is the block size being used (it appears to be the default). If you did not run badblocks using block size that your filesystem is using then you will need to convert your block numbers to match it (i.e. block number 100 with a block size of 1024 bytes becomes block number 25 at 4096 bytes).<br />
<br />
Now the entire point of running this program for the purpose of this article is to get the inode number. To do this run the command (as root):<br />
<br />
debugfs<br />
<br />
Then in the debugfs console, use the open command on the EXT partition containing the bad sector:<br />
<br />
debugfs: open /dev/sdxy<br />
<br />
Finally, use the testb command to get information about the block in question (in this example block 1000):<br />
<br />
debugfs: testb 1000<br />
<br />
If debugfs says that block isn't in use then that means the block is not allocated and is being used as free space. In this case this is a good thing as it means nothing important was damaged by that sector.<br />
<br />
If the block is in use then run this command to get the inode number<br />
<br />
icheck 1000<br />
<br />
This will return two numbers. The block number and the inode number.<br />
<br />
==Find Damaged Files (EXT(2/3/4))==<br />
Use the inode number (second number from the icheck command) with the ncheck command:<br />
<br />
ncheck 12345<br />
<br />
debugfs will give you the full pathname to the file using the bad block. Now you will know what was actually damaged.<br />
<br />
If the inode number is very small and ncheck fails to return a path then it's probably the journal itself that is damaged. To delete the journal simply run this command on the partition:<br />
<br />
tune2fs -O ^has_journal /dev/sdxy<br />
<br />
Run the testb command again from the debugfs console on the bad block and it should be no longer marked as used if it was indeed used by the journal. To build a new journal run:<br />
<br />
tune2fs -j /dev/sdxy<br />
<br />
==Force the disk to reallocate bad block==<br />
First you'll want to see how many badblocks the harddrive is aware of through the smartctl command:<br />
<br />
smartctl -t long /dev/hda [wait until test completes, then]<br />
smartctl -l selftest /dev/hda<br />
<br />
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE<br />
'''5 Reallocated_Sector_Ct 0x0033 100 100 005 Pre-fail Always - 0'''<br />
'''196 Reallocated_Event_Count 0x0032 100 100 000 Old_age Always - 0'''<br />
197 Current_Pending_Sector 0x0022 100 100 000 Old_age Always - 0<br />
'''198 Offline_Uncorrectable 0x0008 100 100 000 Old_age Offline - 0'''<br />
<br />
<br />
To make the harddrive transparently map out the badblock with a spare good sector you will have to simply write zeros to the bad block using the dd command as root. Remember that with this command you have to work with the same block size as your filesystem and the block as to be relative to the partition the filesystem is on and NOT the harddrive as a whole:<br />
<br />
dd if=/dev/zero of=/dev/hda3 bs=4096 count=1 seek=2269012<br />
sync<br />
<br />
You can see if the harddrive did indeed map out an additional bad sector by checking with the smartctl command and seeing if the reallocated sector or event count went up:<br />
<br />
smartctl -A /dev/hda<br />
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE<br />
'''5 Reallocated_Sector_Ct 0x0033 100 100 005 Pre-fail Always - 1'''<br />
'''196 Reallocated_Event_Count 0x0032 100 100 000 Old_age Always - 1'''<br />
197 Current_Pending_Sector 0x0022 100 100 000 Old_age Always - 0<br />
'''198 Offline_Uncorrectable 0x0008 100 100 000 Old_age Offline - 1'''<br />
<br />
To get Offline_Uncorrectable to go back to 0 you need to run a SMART long test and a selftest:<br />
<br />
smartctl -t long /dev/hda [wait until test completes, then]<br />
smartctl -l selftest /dev/hda<br />
<br />
==See also==<br />
<br />
[http://smartmontools.sourceforge.net/badblockhowto.html EXT2/3 badblocks howto]</div>MaBeefhttps://wiki.archlinux.org/index.php?title=Identify_damaged_files&diff=203311Identify damaged files2012-05-31T05:13:02Z<p>MaBeef: </p>
<hr />
<div>[[Category:Storage]]<br />
{{i18n|Find File that Owns a Given Block}}<br />
<br />
This article gives details on how to find out which file owns a given block. The main purpose for doing so is finding out which file was damaged in the event a storage device develops any bad blocks.<br />
<br />
For these commands you will have to either be root or a user that has direct read access to the drive you are scanning. As usual, a current backup is always a good idea, especially if imminent drive failure is suspected.<br />
<br />
Right now this is only written for JFS and EXT filesystems (I'd like to also figure this out for XFS -MaBeef).<br />
<br />
==Finding Bad Blocks==<br />
Just use the [[badblocks]] command. There are a few scan modes supported by it. There's read-only mode (default) which is the least accurate. There is the destructive write-mode (-w option) which is the most accurate but takes longer and will (obviously) destroy all data on the drive, thus making it quite useless for matching blocks up to files. There is finally the non-destructive read-write mode which is probably as accurate as the destructive mode, the only real downside of which is it's probably the slowest. However, if a drive is known to be failing then read-only mode is probably still the safest.<br />
<br />
To do a verbose read-only scan run this command (with x being the drive letter and y being partition number you want to scan):<br />
<br />
badblocks -v /dev/sdx<br />
<br />
or<br />
<br />
badblocks -v /dev/sdxy<br />
<br />
The downside to scanning the drive as a whole is that each filesystem is going to start its block count relative to the partition it's on. This means that if you have a bad block on let's say the second partition and the second partition starts on block 1000, then you have to subtract 1000 from your block number to get the number you want. So if block number 1005 is bad on that partition, then block 5 is what you'll actually be using.<br />
<br />
The alternative is if you've scanned the drive as a whole and have found bad blocks, figure out the partitions they fall on then scan those, using the new block numbers that result from that.<br />
<br />
Another thing to note is that badblocks defaults to 1024 byte blocks so you will either have to change the default size with the -b option to match your filesystem or manually convert the block number(s) later.<br />
<br />
If you need to figure out where your partitions start and end run fdisk (older versions might have defaulted to cylinders, not sure. If so the -u option will change the default unit to sectors). Make sure to note the block size fdisk is using so that you can convert the block counts to match your scan.<br />
<br />
fdisk -l /dev/sdx<br />
<br />
==Debug the Filesystem (JFS)==<br />
jfs_debugfs will give you access to all the low level structures within any JFS filesystem. Other filesystems such as the EXT filesystems have similar tools. It is probably a good idea to umount any filesystem before you run this on them. To use it just run (as root):<br />
<br />
jfs_debugfs /dev/sdxy<br />
<br />
This puts you into a command console. The first thing you should note is your aggregate block size. This is (presumably) the block size the filesystem is using. JFS seems to default to 4096 bytes. If you did not run badblocks using block size that your filesystem is using then you will need to convert your block numbers to match it (i.e. block number 100 with a block size of 1024 bytes becomes block number 25 at 4096 bytes).<br />
<br />
Now the entire point of running this program for the purpose of this article is to get the inode number. To do this run the command:<br />
<br />
d ''blocknumber'' 0 i<br />
<br />
The syntax is the d command for display, the block number, the offset (just set it to 0), and the display format i for inode.<br />
<br />
''Note'': If you get an error then that means the block is not allocated and is being used as free space. In this case this is a good thing as it means nothing important was damaged by that sector.<br />
<br />
The decimal number that di_number is set to is the one we want. From here you type x to exit out of the display mode. Repeat the display command for each bad block that you have and note all of their inode numbers. For more info on the inode such as permissions and filetype type:<br />
<br />
i ''inodenumber''<br />
<br />
i of course stands for inode.<br />
<br />
When you have all the inode numbers type q to quit.<br />
<br />
==Find Damaged Files (JFS)==<br />
<br />
Finally to find the damaged file you can simply use the gnu find utility. Mount your filesystem and run:<br />
<br />
find / -inum ''inodenumber''<br />
<br />
Substitute "/" for the mountpoint of the filesystem that the inode belongs to. If you search root and have more than one filesystem mounted (who doesn't?) you can find multiple files with the same inode number on different filesystems. Remember, an inode is only unique to the filesystem that it's in.<br />
<br />
==Debug the Filesystem (EXT(2/3/4))==<br />
tune2fs will give you access to all the low level structures within any EXT filesystem. It is probably a good idea to umount any filesystem before you run this on them.<br />
<br />
The first thing we want to do is get the block size from the filessystem in question. Just run (probably as root):<br />
<br />
tune2fs -l /dev/hda3 | grep Block<br />
Block count: 29119820<br />
'''Block size: 4096'''<br />
<br />
In this case 4096 is the block size being used (it appears to be the default). If you did not run badblocks using block size that your filesystem is using then you will need to convert your block numbers to match it (i.e. block number 100 with a block size of 1024 bytes becomes block number 25 at 4096 bytes).<br />
<br />
Now the entire point of running this program for the purpose of this article is to get the inode number. To do this run the command (as root):<br />
<br />
debugfs<br />
<br />
Then in the debugfs console, use the open command on the EXT partition containing the bad sector:<br />
<br />
debugfs: open /dev/sdxy<br />
<br />
Finally, use the testb command to get information about the block in question (in this example block 1000):<br />
<br />
debugfs: testb 1000<br />
<br />
If debugfs says that block isn't in use then that means the block is not allocated and is being used as free space. In this case this is a good thing as it means nothing important was damaged by that sector.<br />
<br />
If the block is in use then run this command to get the inode number<br />
<br />
icheck 1000<br />
<br />
This will return two numbers. The block number and the inode number.<br />
<br />
==Find Damaged Files (EXT2/3/4)==<br />
Use the inode number (second number from the icheck command) with the ncheck command:<br />
<br />
ncheck 12345<br />
<br />
debugfs will give you the full pathname to the file using the bad block. Now you will know what was actually damaged.<br />
<br />
If the inode number is very small and ncheck fails to return a path then it's probably the journal itself that is damaged. To delete the journal simply run this command on the partition:<br />
<br />
tune2fs -O ^has_journal /dev/sdxy<br />
<br />
Run the testb command again from the debugfs console on the bad block and it should be no longer marked as used if it was indeed used by the journal. To build a new journal run:<br />
<br />
tune2fs -j /dev/sdxy<br />
<br />
==Force the disk to reallocate bad block==<br />
First you'll want to see how many badblocks the harddrive is aware of through the smartctl command:<br />
<br />
smartctl -t long /dev/hda [wait until test completes, then]<br />
smartctl -l selftest /dev/hda<br />
<br />
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE<br />
'''5 Reallocated_Sector_Ct 0x0033 100 100 005 Pre-fail Always - 0'''<br />
'''196 Reallocated_Event_Count 0x0032 100 100 000 Old_age Always - 0'''<br />
197 Current_Pending_Sector 0x0022 100 100 000 Old_age Always - 0<br />
'''198 Offline_Uncorrectable 0x0008 100 100 000 Old_age Offline - 0'''<br />
<br />
<br />
To make the harddrive transparently map out the badblock with a spare good sector you will have to simply write zeros to the bad block using the dd command as root. Remember that with this command you have to work with the same block size as your filesystem and the block as to be relative to the partition the filesystem is on and NOT the harddrive as a whole:<br />
<br />
dd if=/dev/zero of=/dev/hda3 bs=4096 count=1 seek=2269012<br />
sync<br />
<br />
You can see if the harddrive did indeed map out an additional bad sector by checking with the smartctl command and seeing if the reallocated sector or event count went up:<br />
<br />
smartctl -A /dev/hda<br />
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE<br />
'''5 Reallocated_Sector_Ct 0x0033 100 100 005 Pre-fail Always - 1'''<br />
'''196 Reallocated_Event_Count 0x0032 100 100 000 Old_age Always - 1'''<br />
197 Current_Pending_Sector 0x0022 100 100 000 Old_age Always - 0<br />
'''198 Offline_Uncorrectable 0x0008 100 100 000 Old_age Offline - 1'''<br />
<br />
To get Offline_Uncorrectable to go back to 0 you need to run a SMART long test and a selftest:<br />
<br />
smartctl -t long /dev/hda [wait until test completes, then]<br />
smartctl -l selftest /dev/hda<br />
<br />
==See also==<br />
<br />
[http://smartmontools.sourceforge.net/badblockhowto.html EXT2/3 badblocks howto]</div>MaBeefhttps://wiki.archlinux.org/index.php?title=Identify_damaged_files&diff=203310Identify damaged files2012-05-31T05:10:33Z<p>MaBeef: Finally got around to updating this.</p>
<hr />
<div>[[Category:Storage]]<br />
{{i18n|Find File that Owns a Given Block}}<br />
<br />
This article gives details on how to find out which file owns a given block. The main purpose for doing so is finding out which file was damaged in the event a storage device develops any bad blocks.<br />
<br />
For these commands you will have to either be root or a user that has direct read access to the drive you are scanning. As usual, a current backup is always a good idea, especially if imminent drive failure is suspected.<br />
<br />
==Finding Bad Blocks==<br />
Just use the [[badblocks]] command. There are a few scan modes supported by it. There's read-only mode (default) which is the least accurate. There is the destructive write-mode (-w option) which is the most accurate but takes longer and will (obviously) destroy all data on the drive, thus making it quite useless for matching blocks up to files. There is finally the non-destructive read-write mode which is probably as accurate as the destructive mode, the only real downside of which is it's probably the slowest. However, if a drive is known to be failing then read-only mode is probably still the safest.<br />
<br />
To do a verbose read-only scan run this command (with x being the drive letter and y being partition number you want to scan):<br />
<br />
badblocks -v /dev/sdx<br />
<br />
or<br />
<br />
badblocks -v /dev/sdxy<br />
<br />
The downside to scanning the drive as a whole is that each filesystem is going to start its block count relative to the partition it's on. This means that if you have a bad block on let's say the second partition and the second partition starts on block 1000, then you have to subtract 1000 from your block number to get the number you want. So if block number 1005 is bad on that partition, then block 5 is what you'll actually be using.<br />
<br />
The alternative is if you've scanned the drive as a whole and have found bad blocks, figure out the partitions they fall on then scan those, using the new block numbers that result from that.<br />
<br />
Another thing to note is that badblocks defaults to 1024 byte blocks so you will either have to change the default size with the -b option to match your filesystem or manually convert the block number(s) later.<br />
<br />
If you need to figure out where your partitions start and end run fdisk (older versions might have defaulted to cylinders, not sure. If so the -u option will change the default unit to sectors). Make sure to note the block size fdisk is using so that you can convert the block counts to match your scan.<br />
<br />
fdisk -l /dev/sdx<br />
<br />
==Debug the Filesystem (JFS)==<br />
jfs_debugfs will give you access to all the low level structures within any JFS filesystem. Other filesystems such as the EXT filesystems have similar tools. It is probably a good idea to umount any filesystem before you run this on them. To use it just run (as root):<br />
<br />
jfs_debugfs /dev/sdxy<br />
<br />
This puts you into a command console. The first thing you should note is your aggregate block size. This is (presumably) the block size the filesystem is using. JFS seems to default to 4096 bytes. If you did not run badblocks using block size that your filesystem is using then you will need to convert your block numbers to match it (i.e. block number 100 with a block size of 1024 bytes becomes block number 25 at 4096 bytes).<br />
<br />
Now the entire point of running this program for the purpose of this article is to get the inode number. To do this run the command:<br />
<br />
d ''blocknumber'' 0 i<br />
<br />
The syntax is the d command for display, the block number, the offset (just set it to 0), and the display format i for inode.<br />
<br />
''Note'': If you get an error then that means the block is not allocated and is being used as free space. In this case this is a good thing as it means nothing important was damaged by that sector.<br />
<br />
The decimal number that di_number is set to is the one we want. From here you type x to exit out of the display mode. Repeat the display command for each bad block that you have and note all of their inode numbers. For more info on the inode such as permissions and filetype type:<br />
<br />
i ''inodenumber''<br />
<br />
i of course stands for inode.<br />
<br />
When you have all the inode numbers type q to quit.<br />
<br />
==Find Damaged Files (JFS)==<br />
<br />
Finally to find the damaged file you can simply use the gnu find utility. Mount your filesystem and run:<br />
<br />
find / -inum ''inodenumber''<br />
<br />
Substitute "/" for the mountpoint of the filesystem that the inode belongs to. If you search root and have more than one filesystem mounted (who doesn't?) you can find multiple files with the same inode number on different filesystems. Remember, an inode is only unique to the filesystem that it's in.<br />
<br />
==Debug the Filesystem (EXT(2/3/4))==<br />
tune2fs will give you access to all the low level structures within any EXT filesystem. It is probably a good idea to umount any filesystem before you run this on them.<br />
<br />
The first thing we want to do is get the block size from the filessystem in question. Just run (probably as root):<br />
<br />
tune2fs -l /dev/hda3 | grep Block<br />
Block count: 29119820<br />
'''Block size: 4096'''<br />
<br />
In this case 4096 is the block size being used (it appears to be the default). If you did not run badblocks using block size that your filesystem is using then you will need to convert your block numbers to match it (i.e. block number 100 with a block size of 1024 bytes becomes block number 25 at 4096 bytes).<br />
<br />
Now the entire point of running this program for the purpose of this article is to get the inode number. To do this run the command (as root):<br />
<br />
debugfs<br />
<br />
Then in the debugfs console, use the open command on the EXT partition containing the bad sector:<br />
<br />
debugfs: open /dev/sdxy<br />
<br />
Finally, use the testb command to get information about the block in question (in this example block 1000):<br />
<br />
debugfs: testb 1000<br />
<br />
If debugfs says that block isn't in use then that means the block is not allocated and is being used as free space. In this case this is a good thing as it means nothing important was damaged by that sector.<br />
<br />
If the block is in use then run this command to get the inode number<br />
<br />
icheck 1000<br />
<br />
This will return two numbers. The block number and the inode number.<br />
<br />
==Find Damaged Files (EXT2/3/4)==<br />
Use the inode number (second number from the icheck command) with the ncheck command:<br />
<br />
ncheck 12345<br />
<br />
debugfs will give you the full pathname to the file using the bad block. Now you will know what was actually damaged.<br />
<br />
If the inode number is very small and ncheck fails to return a path then it's probably the journal itself that is damaged. To delete the journal simply run this command on the partition:<br />
<br />
tune2fs -O ^has_journal /dev/sdxy<br />
<br />
Run the testb command again from the debugfs console on the bad block and it should be no longer marked as used if it was indeed used by the journal. To build a new journal run:<br />
<br />
tune2fs -j /dev/sdxy<br />
<br />
==Force the disk to reallocate bad block==<br />
First you'll want to see how many badblocks the harddrive is aware of through the smartctl command:<br />
<br />
smartctl -t long /dev/hda [wait until test completes, then]<br />
smartctl -l selftest /dev/hda<br />
<br />
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE<br />
'''5 Reallocated_Sector_Ct 0x0033 100 100 005 Pre-fail Always - 0'''<br />
'''196 Reallocated_Event_Count 0x0032 100 100 000 Old_age Always - 0'''<br />
197 Current_Pending_Sector 0x0022 100 100 000 Old_age Always - 0<br />
'''198 Offline_Uncorrectable 0x0008 100 100 000 Old_age Offline - 0'''<br />
<br />
<br />
To make the harddrive transparently map out the badblock with a spare good sector you will have to simply write zeros to the bad block using the dd command as root. Remember that with this command you have to work with the same block size as your filesystem and the block as to be relative to the partition the filesystem is on and NOT the harddrive as a whole:<br />
<br />
dd if=/dev/zero of=/dev/hda3 bs=4096 count=1 seek=2269012<br />
sync<br />
<br />
You can see if the harddrive did indeed map out an additional bad sector by checking with the smartctl command and seeing if the reallocated sector or event count went up:<br />
<br />
smartctl -A /dev/hda<br />
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE<br />
'''5 Reallocated_Sector_Ct 0x0033 100 100 005 Pre-fail Always - 1'''<br />
'''196 Reallocated_Event_Count 0x0032 100 100 000 Old_age Always - 1'''<br />
197 Current_Pending_Sector 0x0022 100 100 000 Old_age Always - 0<br />
'''198 Offline_Uncorrectable 0x0008 100 100 000 Old_age Offline - 1'''<br />
<br />
To get Offline_Uncorrectable to go back to 0 you need to run a SMART long test and a selftest:<br />
<br />
smartctl -t long /dev/hda [wait until test completes, then]<br />
smartctl -l selftest /dev/hda<br />
<br />
==See also==<br />
<br />
[http://smartmontools.sourceforge.net/badblockhowto.html EXT2/3 badblocks howto]</div>MaBeefhttps://wiki.archlinux.org/index.php?title=Identify_damaged_files&diff=121189Identify damaged files2010-11-12T20:38:20Z<p>MaBeef: </p>
<hr />
<div>[[Category:HOWTOs (English)]]<br />
[[Category:Storage (English)]]<br />
<br />
This article gives details on how to find out which file owns a given block. The main purpose for doing so is finding out which file was damaged in the event a storage device develops any bad blocks. Currently, JFS will be the only filesystem covered by this article but there should be similar tools available for other filesystems. (I'm planning on adding an EXT guide soon -MaBeef)<br />
<br />
For these commands you will have to either be root or a user that has direct read access to the drive you are scanning. As usual, a current backup is always a good idea, especially if imminent drive failure is suspected.<br />
<br />
==Finding Bad Blocks==<br />
Just use the badblocks command. There are a few scan modes supported by it. There's read-only mode (default) which is the least accurate. There is the destructive write-mode (-w option) which is the most accurate but takes longer and will (obviously) destroy all data on the drive, thus making it quite useless for matching blocks up to files. There is finally the non-destructive read-write mode which is probably as accurate as the destructive mode, the only real downside of which is it's probably the slowest. However, if a drive is known to be failing then read-only mode is probably still the safest.<br />
<br />
To do a verbose read-only scan run this command (with x being the drive letter and y being partition number you want to scan):<br />
<br />
badblocks -v /dev/sdx<br />
<br />
or<br />
<br />
badblocks -v /dev/sdxy<br />
<br />
The downside to scanning the drive as a whole is that each filesystem is going to start its block count relative to the partition it's on. This means that if you have a bad block on let's say the second partition and the second partition starts on block 1000, then you have to subtract 1000 from your block number to get the number you want. The alternative is if you've scanned the drive as a whole and have found bad blocks, figure out the partitions they fall on then scan those, using the new block numbers that result from that.<br />
<br />
Another thing to note is that badblocks defaults to 1024 byte blocks so you will either have to change the default size with the -b option to match your filesystem or manually convert the block number(s) later.<br />
<br />
If you need to figure out where your partitions start and end run fdisk (older versions might have defaulted to cylinders, not sure. If so the -u option will change the default unit to sectors). Make sure to note the block size fdisk is using so that you can convert the block counts to match your scan.<br />
<br />
fdisk -l /dev/sdx<br />
<br />
==Debug the Filesystem==<br />
jfs_debugfs will give you access to all the low level structures within any JFS filesystem. Other filesystems such as the EXT filesystems have similar tools. It is probably a good idea to umount any filesystem before you run this on them. To use it just run:<br />
<br />
jfs_debugfs /dev/sdxy<br />
<br />
This puts you into a command console. The first thing you should note is your aggregate block size. This is (presumably) the block size the filesystem is using. JFS seems to default to 4096 bytes. If you did not run badblocks using block size that your filesystem is using then you will need to convert your block numbers to match it (i.e. block number 100 with a block size of 1024 bytes becomes block number 25 at 4096 bytes).<br />
<br />
Now the entire point of running this program for the purpose of this article is to get the inode number. To do this run the command:<br />
<br />
d ''blocknumber'' 0 i<br />
<br />
The syntax is the d command for display, the block number, the offset (just set it to 0), and the display format i for inode.<br />
<br />
''Note'': If you get an error then that probably means the block is not allocated and is being used as free space. In this case this is a good thing as it means nothing important was damaged by that sector. You can fill the partition up using dd to try to force the drive to map out the bad sector.<br />
<br />
The decimal number that di_number is set to is the one we want. From here you type x to exit out of the display mode. Repeat the display command for each bad block that you have and note all of their inode numbers. For more info on the inode such as permissions and filetype type:<br />
<br />
i ''inodenumber''<br />
<br />
i of course stands for inode.<br />
<br />
When you have all the inode numbers type q to quit.<br />
<br />
==Find Damaged Files==<br />
<br />
Finally to find the damaged file you can simply use the gnu find utility. Mount your filesystem and run:<br />
<br />
find / -inum ''inodenumber''<br />
<br />
Substitute "/" for the mountpoint of the filesystem that the inode belongs to. If you search root and have more than one filesystem mounted (who doesn't?) you can find multiple files with the same inode number on different filesystems. Remember, an inode is only unique to the filesystem that it's in.</div>MaBeefhttps://wiki.archlinux.org/index.php?title=Find_File_that_Owns_a_Block&diff=120148Find File that Owns a Block2010-10-28T05:11:34Z<p>MaBeef: Redirected page to Find File that Owns a Given Block</p>
<hr />
<div>#REDIRECT [[Find File that Owns a Given Block]]</div>MaBeefhttps://wiki.archlinux.org/index.php?title=Find_File_that_Owns_a_Block&diff=120147Find File that Owns a Block2010-10-28T05:11:20Z<p>MaBeef: </p>
<hr />
<div>{{db-g7}}<br />
#REDIRECT [[Find File that Owns a Given Block]]</div>MaBeefhttps://wiki.archlinux.org/index.php?title=Find_File_that_Owns_a_Block&diff=120146Find File that Owns a Block2010-10-28T05:10:20Z<p>MaBeef: moved Find File that Owns a Block to Find File that Owns a Given Block</p>
<hr />
<div>#REDIRECT [[Find File that Owns a Given Block]]</div>MaBeefhttps://wiki.archlinux.org/index.php?title=Identify_damaged_files&diff=120145Identify damaged files2010-10-28T05:10:20Z<p>MaBeef: moved Find File that Owns a Block to Find File that Owns a Given Block</p>
<hr />
<div>[[Category:HOWTOs (English)]]<br />
[[Category:Storage (English)]]<br />
<br />
This article gives details on how to find out which file owns a given block. The main purpose for doing so is finding out which file was damaged in the event a storage device develops any bad blocks. Currently, JFS will be the only filesystem covered by this article but there should be similar tools available for other filesystems.<br />
<br />
For these commands you will have to either be root or a user that has direct read access to the drive you are scanning. As usual, a current backup is always a good idea, especially if imminent drive failure is suspected.<br />
<br />
==Finding Bad Blocks==<br />
Just use the badblocks command. There are a few scan modes supported by it. There's read-only mode (default) which is the least accurate. There is the destructive write-mode (-w option) which is the most accurate but takes longer and will (obviously) destroy all data on the drive, thus making it quite useless for matching blocks up to files. There is finally the non-destructive read-write mode which is probably as accurate as the destructive mode, the only real downside of which is it's probably the slowest. However, if a drive is known to be failing then read-only mode is probably still the safest.<br />
<br />
To do a verbose read-only scan run this command (with x being the drive letter and y being partition number you want to scan):<br />
<br />
badblocks -v /dev/sdx<br />
<br />
or<br />
<br />
badblocks -v /dev/sdxy<br />
<br />
The downside to scanning the drive as a whole is that each filesystem is going to start its block count relative to the partition it's on. This means that if you have a bad block on let's say the second partition and the second partition starts on block 1000, then you have to subtract 1000 from your block number to get the number you want. The alternative is if you've scanned the drive as a whole and have found bad blocks, figure out the partitions they fall on then scan those, using the new block numbers that result from that.<br />
<br />
Another thing to note is that badblocks defaults to 1024 byte blocks so you will either have to change the default size with the -b option to match your filesystem or manually convert the block number(s) later.<br />
<br />
If you need to figure out where your partitions start and end run fdisk (older versions might have defaulted to cylinders, not sure. If so the -u option will change the default unit to sectors). Make sure to note the block size fdisk is using so that you can convert the block counts to match your scan.<br />
<br />
fdisk -l /dev/sdx<br />
<br />
==Debug the Filesystem==<br />
jfs_debugfs will give you access to all the low level structures within any JFS filesystem. Other filesystems such as the EXT filesystems have similar tools. It is probably a good idea to umount any filesystem before you run this on them. To use it just run:<br />
<br />
jfs_debugfs /dev/sdxy<br />
<br />
This puts you into a command console. The first thing you should note is your aggregate block size. This is (presumably) the block size the filesystem is using. JFS seems to default to 4096 bytes. If you did not run badblocks using block size that your filesystem is using then you will need to convert your block numbers to match it (i.e. block number 100 with a block size of 1024 bytes becomes block number 25 at 4096 bytes).<br />
<br />
Now the entire point of running this program for the purpose of this article is to get the inode number. To do this run the command:<br />
<br />
d ''blocknumber'' 0 i<br />
<br />
The syntax is the d command for display, the block number, the offset (just set it to 0), and the display format i for inode.<br />
<br />
The decimal number that di_number is set to is the one we want. From here you type x to exit out of the display mode. Repeat the display command for each bad block that you have and note all of their inode numbers. For more info on the inode such as permissions and filetype type:<br />
<br />
i ''inodenumber''<br />
<br />
i of course stands for inode.<br />
<br />
When you have all the inode numbers type q to quit.<br />
<br />
==Find Damaged Files==<br />
<br />
Finally to find the damaged file you can simply use the gnu find utility. Mount your filesystem and run:<br />
<br />
find / -inum ''inodenumber''<br />
<br />
Substitute "/" for the mountpoint of the filesystem that the inode belongs to. If you search root and have more than one filesystem mounted (who doesn't?) you can find multiple files with the same inode number on different filesystems. Remember, an inode is only unique to the filesystem that it's in.</div>MaBeefhttps://wiki.archlinux.org/index.php?title=Identify_damaged_files&diff=120144Identify damaged files2010-10-28T05:05:19Z<p>MaBeef: Created page with "Category:HOWTOs (English) Category:Storage (English) This article gives details on how to find out which file owns a given block. The main purpose for doing so is findin..."</p>
<hr />
<div>[[Category:HOWTOs (English)]]<br />
[[Category:Storage (English)]]<br />
<br />
This article gives details on how to find out which file owns a given block. The main purpose for doing so is finding out which file was damaged in the event a storage device develops any bad blocks. Currently, JFS will be the only filesystem covered by this article but there should be similar tools available for other filesystems.<br />
<br />
For these commands you will have to either be root or a user that has direct read access to the drive you are scanning. As usual, a current backup is always a good idea, especially if imminent drive failure is suspected.<br />
<br />
==Finding Bad Blocks==<br />
Just use the badblocks command. There are a few scan modes supported by it. There's read-only mode (default) which is the least accurate. There is the destructive write-mode (-w option) which is the most accurate but takes longer and will (obviously) destroy all data on the drive, thus making it quite useless for matching blocks up to files. There is finally the non-destructive read-write mode which is probably as accurate as the destructive mode, the only real downside of which is it's probably the slowest. However, if a drive is known to be failing then read-only mode is probably still the safest.<br />
<br />
To do a verbose read-only scan run this command (with x being the drive letter and y being partition number you want to scan):<br />
<br />
badblocks -v /dev/sdx<br />
<br />
or<br />
<br />
badblocks -v /dev/sdxy<br />
<br />
The downside to scanning the drive as a whole is that each filesystem is going to start its block count relative to the partition it's on. This means that if you have a bad block on let's say the second partition and the second partition starts on block 1000, then you have to subtract 1000 from your block number to get the number you want. The alternative is if you've scanned the drive as a whole and have found bad blocks, figure out the partitions they fall on then scan those, using the new block numbers that result from that.<br />
<br />
Another thing to note is that badblocks defaults to 1024 byte blocks so you will either have to change the default size with the -b option to match your filesystem or manually convert the block number(s) later.<br />
<br />
If you need to figure out where your partitions start and end run fdisk (older versions might have defaulted to cylinders, not sure. If so the -u option will change the default unit to sectors). Make sure to note the block size fdisk is using so that you can convert the block counts to match your scan.<br />
<br />
fdisk -l /dev/sdx<br />
<br />
==Debug the Filesystem==<br />
jfs_debugfs will give you access to all the low level structures within any JFS filesystem. Other filesystems such as the EXT filesystems have similar tools. It is probably a good idea to umount any filesystem before you run this on them. To use it just run:<br />
<br />
jfs_debugfs /dev/sdxy<br />
<br />
This puts you into a command console. The first thing you should note is your aggregate block size. This is (presumably) the block size the filesystem is using. JFS seems to default to 4096 bytes. If you did not run badblocks using block size that your filesystem is using then you will need to convert your block numbers to match it (i.e. block number 100 with a block size of 1024 bytes becomes block number 25 at 4096 bytes).<br />
<br />
Now the entire point of running this program for the purpose of this article is to get the inode number. To do this run the command:<br />
<br />
d ''blocknumber'' 0 i<br />
<br />
The syntax is the d command for display, the block number, the offset (just set it to 0), and the display format i for inode.<br />
<br />
The decimal number that di_number is set to is the one we want. From here you type x to exit out of the display mode. Repeat the display command for each bad block that you have and note all of their inode numbers. For more info on the inode such as permissions and filetype type:<br />
<br />
i ''inodenumber''<br />
<br />
i of course stands for inode.<br />
<br />
When you have all the inode numbers type q to quit.<br />
<br />
==Find Damaged Files==<br />
<br />
Finally to find the damaged file you can simply use the gnu find utility. Mount your filesystem and run:<br />
<br />
find / -inum ''inodenumber''<br />
<br />
Substitute "/" for the mountpoint of the filesystem that the inode belongs to. If you search root and have more than one filesystem mounted (who doesn't?) you can find multiple files with the same inode number on different filesystems. Remember, an inode is only unique to the filesystem that it's in.</div>MaBeefhttps://wiki.archlinux.org/index.php?title=SFTP_chroot&diff=88067SFTP chroot2009-12-18T10:34:26Z<p>MaBeef: </p>
<hr />
<div>[[Category:Networking (English)]]<br />
[[Category:Security (English)]]<br />
[[Category:HOWTOs (English)]]<br />
<br />
== Introduction ==<br />
OpenSSH 4.9+ includes a built-in chroot for sftp, but requires a few tweaks to the normal install.<br />
<br />
==Installation==<br />
This package is available in the core repository. To install it, run <br />
# pacman -S openssh<br />
<br />
==Configuration==<br />
In /etc/ssh/sshd_config, modify the Subsystem line for sftp:<br />
Subsystem sftp internal-sftp<br />
<br />
At the end of the file, add something similar to the following for a group:<br />
Match Group sftpusers<br />
ChrootDirectory %h<br />
ForceCommand internal-sftp<br />
AllowTcpForwarding no<br />
<br />
Or for a user:<br />
Match User username<br />
ChrootDirectory %h<br />
ForceCommand internal-sftp<br />
<br />
The %h represents the users home directory.<br />
<br />
Restart sshd:<br />
# /etc/rc.d/sshd restart<br />
<br />
===Adding new chrooted users===<br />
If using the group method above, ensure all sftp users are put in the appropriate group, i.e.:<br />
usermod -g sftpusers<br />
<br />
Also, set their shell to /bin/false to prevent a normal ssh login:<br />
usermod -s /bin/false<br />
<br />
Note that since this is only for sftp, a proper chroot environment with a shell and /dev/* doesn't need to be created.<br />
<br />
Their chroot will be the same as their home directory. The permissions are not the same as a normal home, though. Their home directory must be owned as root and not writable by another user or group. This includes the path leading to the directory. My recommendation is to use /usr/local/chroot as a root and build the home directories under that.<br />
<br />
===Testing your chroot===<br />
# ssh username@localhost<br />
<br />
should refuse the connection or fail on login. The response varies, possibly due to the version of OpenSSH used.<br />
<br />
# sftp username@localhost<br />
<br />
should place you in the chroot'd environment. If not, ensure that your /etc/hosts.allow file contains the following line or one similar to allow ssh connections:<br />
<br />
sshd: ALL<br />
<br />
sshd will also reject sftp connections to accounts that are set to chroot into any directory that has ownership/permissions that sshd doesn't consider secure. sshd's apparently strict ownership/permissions requirements dictate that every directory in the chroot path must be owned by root and only writable for the owner. So, for example, if the chroot environment is in a user's home directory both /home and /home/username must be owned by root (group ownership doesn't seem to matter) and have permissions along the lines of 700 or 755 (775 won't work for example since it's group writable).<br />
<br />
<br />
= Links & References =<br />
*[http://www.minstrel.org.uk/papers/sftp/ http://www.minstrel.org.uk/papers/sftp/builtin/]</div>MaBeefhttps://wiki.archlinux.org/index.php?title=SFTP_chroot&diff=88065SFTP chroot2009-12-18T10:33:34Z<p>MaBeef: </p>
<hr />
<div>[[Category:Networking (English)]]<br />
[[Category:Security (English)]]<br />
[[Category:HOWTOs (English)]]<br />
<br />
== Introduction ==<br />
OpenSSH 4.9+ includes a built-in chroot for sftp, but requires a few tweaks to the normal install.<br />
<br />
==Installation==<br />
This package is available in the core repository. To install it, run <br />
# pacman -S openssh<br />
<br />
==Configuration==<br />
In /etc/ssh/sshd_config, modify the Subsystem line for sftp:<br />
Subsystem sftp internal-sftp<br />
<br />
At the end of the file, add something similar to the following for a group:<br />
Match Group sftpusers<br />
ChrootDirectory %h<br />
ForceCommand internal-sftp<br />
AllowTcpForwarding no<br />
<br />
Or for a user:<br />
Match User username<br />
ChrootDirectory %h<br />
ForceCommand internal-sftp<br />
<br />
The %h represents the users home directory.<br />
<br />
Restart sshd:<br />
# /etc/rc.d/sshd restart<br />
<br />
===Adding new chrooted users===<br />
If using the group method above, ensure all sftp users are put in the appropriate group, i.e.:<br />
usermod -g sftpusers<br />
<br />
Also, set their shell to /bin/false to prevent a normal ssh login:<br />
usermod -s /bin/false<br />
<br />
Note that since this is only for sftp, a proper chroot environment with a shell and /dev/* doesn't need to be created.<br />
<br />
Their chroot will be the same as their home directory. The permissions are not the same as a normal home, though. Their home directory must be owned as root and not writable by another user or group. This includes the path leading to the directory. My recommendation is to use /usr/local/chroot as a root and build the home directories under that.<br />
<br />
===Testing your chroot===<br />
# ssh username@localhost<br />
<br />
should refuse the connection or fail on login. The response varies, possibly due to the version of OpenSSH used.<br />
<br />
# sftp username@localhost<br />
<br />
should place you in the chroot'd environment. If not, ensure that your /etc/hosts.allow file contains the following line or one similar to allow ssh connections:<br />
<br />
sshd: ALL<br />
<br />
sshd will also reject sftp connections to accounts that are set to chroot into any directory that has ownership/permissions that sshd doesn't consider secure. sshd's apparently strict ownership/permissions requirements dictate that every directory in the chroot path must be owned by root and only writable for the owner. So, for example, if the chroot environment is in a user's home directory both /home and /home/username must be owned by root (group ownership doesn't seem to matter) and permissions along the lines of 700 or 755 (775 won't work for example since it's group writable).<br />
<br />
<br />
= Links & References =<br />
*[http://www.minstrel.org.uk/papers/sftp/ http://www.minstrel.org.uk/papers/sftp/builtin/]</div>MaBeefhttps://wiki.archlinux.org/index.php?title=Talk:SFTP_chroot&diff=88064Talk:SFTP chroot2009-12-18T10:31:14Z<p>MaBeef: Created page with 'I added a note to the bottom about ownership/permissions issues sshd can give you when you're setting it to chroot. I followed this guide and ran into a problem where no matter w…'</p>
<hr />
<div>I added a note to the bottom about ownership/permissions issues sshd can give you when you're setting it to chroot. I followed this guide and ran into a problem where no matter what I seemed to do sshd would keep rejecting sftp connections. Turns out it won't allow you to chroot to directories that don't have what it considers secure permissions.<br />
<br />
This is my first edit on this wiki btw. [[User:MaBeef|MaBeef]] 05:31, 18 December 2009 (EST)</div>MaBeefhttps://wiki.archlinux.org/index.php?title=SFTP_chroot&diff=88062SFTP chroot2009-12-18T10:25:59Z<p>MaBeef: Added notes about ownership/permissions that sshd requires to allow the chroot</p>
<hr />
<div>[[Category:Networking (English)]]<br />
[[Category:Security (English)]]<br />
[[Category:HOWTOs (English)]]<br />
<br />
== Introduction ==<br />
OpenSSH 4.9+ includes a built-in chroot for sftp, but requires a few tweaks to the normal install.<br />
<br />
==Installation==<br />
This package is available in the core repository. To install it, run <br />
# pacman -S openssh<br />
<br />
==Configuration==<br />
In /etc/ssh/sshd_config, modify the Subsystem line for sftp:<br />
Subsystem sftp internal-sftp<br />
<br />
At the end of the file, add something similar to the following for a group:<br />
Match Group sftpusers<br />
ChrootDirectory %h<br />
ForceCommand internal-sftp<br />
AllowTcpForwarding no<br />
<br />
Or for a user:<br />
Match User username<br />
ChrootDirectory %h<br />
ForceCommand internal-sftp<br />
<br />
The %h represents the users home directory.<br />
<br />
Restart sshd:<br />
# /etc/rc.d/sshd restart<br />
<br />
===Adding new chrooted users===<br />
If using the group method above, ensure all sftp users are put in the appropriate group, i.e.:<br />
usermod -g sftpusers<br />
<br />
Also, set their shell to /bin/false to prevent a normal ssh login:<br />
usermod -s /bin/false<br />
<br />
Note that since this is only for sftp, a proper chroot environment with a shell and /dev/* doesn't need to be created.<br />
<br />
Their chroot will be the same as their home directory. The permissions are not the same as a normal home, though. Their home directory must be owned as root and not writable by another user or group. This includes the path leading to the directory. My recommendation is to use /usr/local/chroot as a root and build the home directories under that.<br />
<br />
===Testing your chroot===<br />
# ssh username@localhost<br />
<br />
should refuse the connection or fail on login. The response varies, possibly due to the version of OpenSSH used.<br />
<br />
# sftp username@localhost<br />
<br />
should place you in the chroot'd environment. If not, ensure that your /etc/hosts.allow file contains the following line or one similar to allow ssh connections:<br />
<br />
sshd: ALL<br />
<br />
sshd will also reject sftp connections to accounts that are set to chroot into any directory that has ownership/permissions that sshd doesn't consider secure. sshd apparently has some strict ownership/permission requirements because every directory in the chroot path must be owned by root and only writable for the owner. So, for example, if the chroot environment is in a user's home directory both /home and /home/username must be owned by root (group ownership doesn't seem to matter) and permissions along the lines of 700 or 755 (775 won't work for example since it's group writable).<br />
<br />
<br />
= Links & References =<br />
*[http://www.minstrel.org.uk/papers/sftp/ http://www.minstrel.org.uk/papers/sftp/builtin/]</div>MaBeef