10: GPTs; the FAT filesystem
DRAFT
Announcement
GPTs
Modern PCs and Macs use the GUID Partition Table (GPT) standard on fixed hard disks. The GPT is a more sophisticated system that parallels the MBR in many ways but fixes many of its problems.
A GPT-formatted disk is laid out as follows:
(Image copyright © 2007 Wikimedia Commons and used under Attribution-ShareAlike 2.5 Generic (CC BY-SA 2.5).)
Note that the GPT is duplicated at the start and end of disk, and supposed to be kept in sync. This is a backup to mitigate data corruption.
Also note that there’s a “protective MBR”. What’s that about? If you put a GPT-formatted disk into a MBR-only computer, it contains dummy values (a single maximum size partition of type 0xEE) so that the disk shows up as formatted. Hey, what were those bytes on my Mac’s “MBR”? Remember, the first partition starts at 0x1BE:
hexdump -Cv -n 512 disk-image.dd
00000000 33 c0 8e d0 bc 00 7c 8e c0 8e d8 be 00 7c bf 00 |3.....|......|..|
00000010 06 b9 00 02 fc f3 a4 50 68 1c 06 cb fb b9 04 00 |.......Ph.......|
00000020 bd be 07 80 7e 00 00 7c 0b 0f 85 0e 01 83 c5 10 |....~..|........|
00000030 e2 f1 cd 18 88 56 00 55 c6 46 11 05 c6 46 10 00 |.....V.U.F...F..|
00000040 b4 41 bb aa 55 cd 13 5d 72 0f 81 fb 55 aa 75 09 |.A..U..]r...U.u.|
00000050 f7 c1 01 00 74 03 fe 46 10 66 60 80 7e 10 00 74 |....t..F.f`.~..t|
00000060 26 66 68 00 00 00 00 66 ff 76 08 68 00 00 68 00 |&fh....f.v.h..h.|
00000070 7c 68 01 00 68 10 00 b4 42 8a 56 00 8b f4 cd 13 ||h..h...B.V.....|
00000080 9f 83 c4 10 9e eb 14 b8 01 02 bb 00 7c 8a 56 00 |............|.V.|
00000090 8a 76 01 8a 4e 02 8a 6e 03 cd 13 66 61 73 1c fe |.v..N..n...fas..|
000000a0 4e 11 75 0c 80 7e 00 80 0f 84 8a 00 b2 80 eb 84 |N.u..~..........|
000000b0 55 32 e4 8a 56 00 cd 13 5d eb 9e 81 3e fe 7d 55 |U2..V...]...>.}U|
000000c0 aa 75 6e ff 76 00 e8 8d 00 75 17 fa b0 d1 e6 64 |.un.v....u.....d|
000000d0 e8 83 00 b0 df e6 60 e8 7c 00 b0 ff e6 64 e8 75 |......`.|....d.u|
000000e0 00 fb b8 00 bb cd 1a 66 23 c0 75 3b 66 81 fb 54 |.......f#.u;f..T|
000000f0 43 50 41 75 32 81 f9 02 01 72 2c 66 68 07 bb 00 |CPAu2....r,fh...|
00000100 00 66 68 00 02 00 00 66 68 08 00 00 00 66 53 66 |.fh....fh....fSf|
00000110 53 66 55 66 68 00 00 00 00 66 68 00 7c 00 00 66 |SfUfh....fh.|..f|
00000120 61 68 00 00 07 cd 1a 5a 32 f6 ea 00 7c 00 00 cd |ah.....Z2...|...|
00000130 18 a0 b7 07 eb 08 a0 b6 07 eb 03 a0 b5 07 32 e4 |..............2.|
00000140 05 00 07 8b f0 ac 3c 00 74 09 bb 07 00 b4 0e cd |......<.t.......|
00000150 10 eb f2 f4 eb fd 2b c9 e4 64 eb 00 24 02 e0 f8 |......+..d..$...|
00000160 24 02 c3 49 6e 76 61 6c 69 64 20 70 61 72 74 69 |$..Invalid parti|
00000170 74 69 6f 6e 20 74 61 62 6c 65 00 45 72 72 6f 72 |tion table.Error|
00000180 20 6c 6f 61 64 69 6e 67 20 6f 70 65 72 61 74 69 | loading operati|
00000190 6e 67 20 73 79 73 74 65 6d 00 4d 69 73 73 69 6e |ng system.Missin|
000001a0 67 20 6f 70 65 72 61 74 69 6e 67 20 73 79 73 74 |g operating syst|
000001b0 65 6d 00 00 00 63 7b 9a 7d 6e 00 00 00 00 00 fe |em...c{.}n......|
000001c0 ff ff ee fe ff ff 01 00 00 00 a3 70 3d 3a 00 00 |...........p=:..|
000001d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000001e0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000001f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 55 aa |..............U.|
00 fe ff ff *ee* fe ff ff 01 00 00 00 a3 70 3d 3a
ee
! Well, there you go then.
Where’s the actual GPT data? It starts in a header in the next sector.
https://en.wikipedia.org/wiki/GUID_Partition_Table#Partition_table_header_.28LBA_1.29
dd if=disk-image.dd bs=512 count=1 skip=1 | hexdump -Cv
1+0 records in
1+0 records out
512 bytes transferred in 0.000016 secs (31580642 bytes/sec)
00000000 45 46 49 20 50 41 52 54 00 00 01 00 5c 00 00 00 |EFI PART....\...|
00000010 7f 95 63 62 00 00 00 00 01 00 00 00 00 00 00 00 |..cb............|
00000020 a3 70 3d 3a 00 00 00 00 22 00 00 00 00 00 00 00 |.p=:....".......|
00000030 82 70 3d 3a 00 00 00 00 a8 4d 00 00 93 38 00 00 |.p=:.....M...8..|
00000040 3d 11 00 00 f3 62 00 00 02 00 00 00 00 00 00 00 |=....b..........|
00000050 80 00 00 00 80 00 00 00 81 d5 7a 4e 00 00 00 00 |..........zN....|
00000060 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000070 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000090 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000000a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000000b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000000c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000000d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000000e0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000000f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000100 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000110 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000120 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000130 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000140 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000150 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000160 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000170 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000180 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000190 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000001a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000001b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000001c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000001d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000001e0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000001f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000200
Note, for example, we have the right header (EFI PART
), and that the partition entries (partition table) offset is located at byte 0x48 and is equal to 2 (that is, it’s at the next sector on the disk). There’s also a 4-byte value for the number of entries and the size of each entry at 0x50 and 0x54 (very much like the IFD). In this image, there are 128 entries (little endian: 08 00 00 00
) each 128 bytes long (which is typical).
Let’s pull out the next sector to see the first few entries:
dd if=disk-image.dd bs=512 count=1 skip=2 | hexdump -Cv
1+0 records in
1+0 records out
512 bytes transferred in 0.000478 secs (1071599 bytes/sec)
00000000 28 73 2a c1 1f f8 d2 11 ba 4b 00 a0 c9 3e c9 3b |(s*......K...>.;|
00000010 e3 22 00 00 4b 2b 00 00 1e 43 00 00 72 0b 00 00 |."..K+...C..r...|
00000020 28 00 00 00 00 00 00 00 27 40 06 00 00 00 00 00 |(.......'@......|
00000030 00 00 00 00 00 00 00 00 45 00 46 00 49 00 20 00 |........E.F.I. .|
00000040 73 00 79 00 73 00 74 00 65 00 6d 00 20 00 70 00 |s.y.s.t.e.m. .p.|
00000050 61 00 72 00 74 00 69 00 74 00 69 00 6f 00 6e 00 |a.r.t.i.t.i.o.n.|
00000060 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000070 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000080 72 6f 74 53 67 61 aa 11 aa 11 00 30 65 43 ec ac |rotSga.....0eC..|
00000090 41 5a 00 00 e1 30 00 00 8a 74 00 00 df 60 00 00 |AZ...0...t...`..|
000000a0 28 40 06 00 00 00 00 00 7f 9a e1 22 00 00 00 00 |(@........."....|
000000b0 00 00 00 00 00 00 00 00 49 00 72 00 6f 00 6e 00 |........I.r.o.n.|
000000c0 00 00 6d 00 65 00 72 00 00 00 00 00 00 00 00 00 |..m.e.r.........|
000000d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000000e0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000000f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000100 74 6f 6f 42 00 00 aa 11 aa 11 00 30 65 43 ec ac |tooB.......0eC..|
00000110 40 0b 46 a0 6e 7e 24 42 91 02 75 57 32 ab c9 dd |@.F.n~$B..uW2...|
00000120 80 9a e1 22 00 00 00 00 9f f9 f4 22 00 00 00 00 |..."......."....|
00000130 00 00 00 00 00 00 02 00 52 00 65 00 63 00 6f 00 |........R.e.c.o.|
00000140 76 00 65 00 72 00 79 00 20 00 48 00 44 00 00 00 |v.e.r.y. .H.D...|
00000150 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000160 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000170 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000180 00 53 46 48 00 00 aa 11 aa 11 00 30 65 43 ec ac |.SFH.......0eC..|
00000190 c0 34 9b 30 09 0b fb 41 95 02 8b 95 10 1f 4e 41 |.4.0...A......NA|
000001a0 00 00 f5 22 00 00 00 00 7f 70 39 3a 00 00 00 00 |...".....p9:....|
000001b0 00 00 00 00 00 00 00 00 41 00 70 00 70 00 6c 00 |........A.p.p.l.|
000001c0 65 00 5f 00 48 00 46 00 53 00 5f 00 55 00 6e 00 |e._.H.F.S._.U.n.|
000001d0 74 00 69 00 74 00 6c 00 65 00 64 00 5f 00 32 00 |t.i.t.l.e.d._.2.|
000001e0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000001f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000200
Entries look like: https://en.wikipedia.org/wiki/GUID_Partition_Table#Partition_entries
Let’s look at the first few:
00000000 28 73 2a c1 1f f8 d2 11 ba 4b 00 a0 c9 3e c9 3b |(s*......K...>.;|
00000010 e3 22 00 00 4b 2b 00 00 1e 43 00 00 72 0b 00 00 |."..K+...C..r...|
00000020 28 00 00 00 00 00 00 00 27 40 06 00 00 00 00 00 |(.......'@......|
00000030 00 00 00 00 00 00 00 00 45 00 46 00 49 00 20 00 |........E.F.I. .|
00000040 73 00 79 00 73 00 74 00 65 00 6d 00 20 00 70 00 |s.y.s.t.e.m. .p.|
00000050 61 00 72 00 74 00 69 00 74 00 69 00 6f 00 6e 00 |a.r.t.i.t.i.o.n.|
00000060 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000070 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
Each entry starts with a 16-byte partition type GUID. This one’s is: 28 73 2a c1 1f f8 d2 11 ba 4b 00 a0 c9 3e c9 3b
. Hmm, it doesn’t seem to be in the list at https://en.wikipedia.org/wiki/GUID_Partition_Table#Partition_type_GUIDs. What might it be? Turns out it’s: an EFI System Partition (spoiler: the name is stored as UTF-16LE text!). https://en.wikipedia.org/wiki/EFI_system_partition. Also, the entry really is there – look at it and consider endianness.
Bootable drives in UEFI systems put the bootloader and associated utilities on this partition, which is FAT-like (though actually standardized as part of UEFI, so not exactly FAT). Different OSes put different stuff here; it’s a standardized, extensible way to get more than the 446 bytes the MBR provided for a full-featured boot loader (and whatever else your OS provider wants to give you).
At 0x10, the next 16 bytes are a unique-to-this-partition-on-this-computer GUID.
At 0x20, there are 8 bytes (LE) telling you the first LBA of this partition, then 8 bytes telling you the last (inclusive).
struct.unpack('<QQ', bytes.fromhex('28 00 00 00 00 00 00 00 27 40 06 00 00 00 00 00'))
# => (40, 409639)
Again, does that match what mmls
told us?
mmls /dev/rdisk0
GUID Partition Table (EFI)
Offset Sector: 0
Units are in 512-byte sectors
Slot Start End Length Description
000: Meta 0000000000 0000000000 0000000001 Safety Table
001: ------- 0000000000 0000000039 0000000040 Unallocated
002: Meta 0000000001 0000000001 0000000001 GPT Header
003: Meta 0000000002 0000000033 0000000032 Partition Table
004: 000 0000000040 0000409639 0000409600 EFI system partition
005: 001 0000409640 0585210495 0584800856 Iron
006: 002 0585210496 0586480031 0001269536 Recovery HD
007: ------- 0586480032 0586481663 0000001632 Unallocated
008: 003 0586481664 0976842879 0390361216 Apple_HFS_Untitled_2
009: ------- 0976842880 0977105059 0000262180 Unallocated
Yup, exactly right. Then 8 bytes of “attribute flags,” with only the first three bits being universally defined so far, and the last 16 defined by each partition type. Finally at 0x38 the partition name, in 72 bytes of UTF-16LE encoded text.
The next one looks like:
00000080 72 6f 74 53 67 61 aa 11 aa 11 00 30 65 43 ec ac |rotSga.....0eC..|
00000090 41 5a 00 00 e1 30 00 00 8a 74 00 00 df 60 00 00 |AZ...0...t...`..|
000000a0 28 40 06 00 00 00 00 00 7f 9a e1 22 00 00 00 00 |(@........."....|
000000b0 00 00 00 00 00 00 00 00 49 00 72 00 6f 00 6e 00 |........I.r.o.n.|
000000c0 00 00 6d 00 65 00 72 00 00 00 00 00 00 00 00 00 |..m.e.r.........|
000000d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000000e0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000000f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
This is an Apple partition, and you can see it’s named “Iron”, which is my Mac’s primary partition’s name.
If we kept going, you’d see that unused entries are zeroed out (that is, they consist entirely of bytes set to the value 00
).
So you get the idea here, anyway. GPTs are a fairly straightforward binary data structure to store hard disk partition information. They’re slightly more complex than MBRs, but they don’t require the contortions of MBRs (which is a legacy format originally designed for computers and disks over 30 years ago!).
So that’s it for lecture content on volumes and partitions, though you will probably want to Read the Book, supplemented with more recent web resources, for the fine details of MBRs and GPTs.
Filesystems
Filesystems are provided by operating systems to programs (and ultimately, to users). They provide a mechanism to store data in a structured way, usually organized around “files” and “directories”.
The filesystem doesn’t worry about the internal structure of files (like the JPEGs we’ve dealt with); it treats them as blobs of bytes to be stored and retrieved when requested.
The filesystem instead manages the files that are stored and the space in which they are stored. Minimally, a file system must be able to store, retrieve, and remove data from a partition corresponding to a particular file. To do so, it must keep track of:
- free and allocated space within the partition that it is responsible for;
- the location of each file within the partition
- the user-visible name of each file (the filename)
- the user-visible location of each file (the path to the file)
Carrier refers to these items as essential and therefore “more trustworthy” in some sense than nonessential items. The reasoning here is that the above must be correct, or the filesystem isn’t fulfilling its basic purpose.
Filesystems might also track many other items of metadata including:
- creation, modification, or last-accessed time
- ownership information
- permissions (readable, writeable, executable, by who?)
Carrier provides a broad overview of the analysis techniques you might use at an abstract level in Chapter 8; we’re going to jump right into the details of the FAT filesystem and see them as we go along.
FAT
The File Allocation Table (FAT) filesystem is simple (maybe too simple, in some ways) and extraordinarily widespread. While to doesn’t see much use on new computers, it was the domininant filesystem on PCs from the early 80s through the late 90s / early 00s.
Being simple and nearly ubiquitious, virtually every OS (not just Microsoft’s) had and have full support for it.
It lives on as the filesystem on virtually every USB thumb drive and SD-card (either FAT16, FAT32, or “exFAT”), and as a variant in the EFI System partition on every bootable GPT volume.
There are several variants you might encounter: FAT16, FAT32, and exFAT, with the latter two more common now (as FAT16 cannot support partitions larger than 2GB, and even “small” USB thumbdrives are generally larger than this nowadays).
FAT allocates storage in units called “clusters”; a cluster is the minimum allocable unit on a FAT filesystem. Clusters are a number of sectors; the number must be a power of two.
FAT layout
FAT manages files, directories, and free space using two data structures:
- The directory entry, which contains the file (or directory) name, size, starting address, and other metadata; and
- The file allocation table, which has two functions. First, it tracks the allocation status of clusters (that is, is part of a file stored at a given cluster?). Second, if a file is split across more than one cluster, it lets you find the clusters after the first (the first’s address being stored in the directory entry).
At a high level, FAT filesystems have three parts:
- the reserved area, which stores essential data the describes the remainder of the filesystem,
- the FAT area, where one or more (typically two) copies of the FAT structures are stored, and
- the data area, where directory entries and file data are stored.
Let’s look at each of these in turn.
Reserved area
Let’s do these on the reserved area of my USB key. Remember, we can use mmls
to view the partition map, or parse the MBR (or GPT) ourselves.
mmls /dev/rdisk4
Password:
DOS Partition Table
Offset Sector: 0
Units are in 512-byte sectors
Slot Start End Length Description
000: Meta 0000000000 0000000000 0000000001 Primary Table (#0)
001: ------- 0000000000 0000000001 0000000002 Unallocated
002: 000:000 0000000002 0003911679 0003911678 DOS FAT16 (0x06)```
So the FAT partition begins on sector 2 of the drive. Let’s extract it and view it:
dd if=/dev/rdisk4 bs=512 skip=2 count=1 | hexdump -Cv
1+0 records in
1+0 records out
512 bytes transferred in 0.000573 secs (893668 bytes/sec)
00000000 eb 3c 90 6d 6b 66 73 2e 66 61 74 00 02 40 01 00 |.<.mkfs.fat..@..|
00000010 02 00 04 00 00 f8 00 01 3e 00 3e 00 02 00 00 00 |........>.>.....|
00000020 fe af 3b 00 80 00 29 08 47 d5 4b 4d 41 52 43 27 |..;...).G.KMARC'|
00000030 73 20 55 53 42 20 46 41 54 31 36 20 20 20 0e 1f |s USB FAT16 ..|
00000040 be 5b 7c ac 22 c0 74 0b 56 b4 0e bb 07 00 cd 10 |.[|.".t.V.......|
00000050 5e eb f0 32 e4 cd 16 cd 19 eb fe 54 68 69 73 20 |^..2.......This |
00000060 69 73 20 6e 6f 74 20 61 20 62 6f 6f 74 61 62 6c |is not a bootabl|
00000070 65 20 64 69 73 6b 2e 20 20 50 6c 65 61 73 65 20 |e disk. Please |
00000080 69 6e 73 65 72 74 20 61 20 62 6f 6f 74 61 62 6c |insert a bootabl|
00000090 65 20 66 6c 6f 70 70 79 20 61 6e 64 0d 0a 70 72 |e floppy and..pr|
000000a0 65 73 73 20 61 6e 79 20 6b 65 79 20 74 6f 20 74 |ess any key to t|
000000b0 72 79 20 61 67 61 69 6e 20 2e 2e 2e 20 0d 0a 00 |ry again ... ...|
000000c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000000d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000000e0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000000f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000100 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000110 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000120 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000130 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000140 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000150 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000160 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000170 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000180 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000190 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000001a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000001b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000001c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000001d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000001e0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000001f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 55 aa |..............U.|
00000200
Note that like in the MBR, multibyte values in the FAT boot sector are little-endian.
0x0, 3 bytes: The reserved area starts with machine code that (in older bootable FAT partitions) tells the CPU how far ahead to JMP to the next part of the boot loader.
0x3, 8 bytes: Then 8 bytes of ASCII-coded OEM. “mkfs.fat”
0xB, 2 bytes: bytes per sector “00 02” -> 512
0xD, 1 byte: sectors per cluster “40” -> 64 (so a cluster is 64 * 512 = 32KB on this USB key)
0xE, 2 bytes: size of the reserved area, in sectors “01 00” -> 1
0x10, 1 byte: # FATs “02”
0x11 2 bytes: (0 in FAT32) (in FAT16: max # files in root directory) “00 04” -> 1024
0x13 2 bytes: # sectors; if doesn’t fit in 2 bytes, should be zero “00 00 “
0x15 1 byte: media type (0xf8 fixed disk, 0xf0 removable) “f8”
0x16 2 bytes: (in FAT32: 0) (in FAT16: size in sectors of the FAT) “00 01” -> 256
0x18 2 bytes: sectors per track “3e 00”
0x1A 2 bytes: # heads “3e 00”
0x1C 4 bytes: # sectors before start of partition (from beginning of disk, or extended partition) “02 00 00 00” -> 2
0x20 4 bytes: # sectors in file system (will be zero if bytes at 0x13 are non-zero, and vice versa) “FE AF 3B 00” -> 3911678
0x24 1 byte: BIOS INT13h drive number (see https://en.wikipedia.org/wiki/INT_13H; not used these days)
0x25 1 byte: not used
0x26 1 byte: signature for validity of next entries; 0x29 if valid.
0x27 4 bytes: volume (partition) serial number
0x2b 11 bytes: volume (partition) label, ASCII
0x36 8 bytes: FS type label in ASCII – usually “FAT16”
0x3e–0x1FE: unused by FAT; sometimes contains boot code (it might be what’s jumped to by the JMP instructions)
0x1FE (2 bytes): signature value (55 AA)
This boot sector is the only thing in its reserved area; the reserved area in FAT32 might contain a few more things (including a FSINFO structure) but we’ll skip those details here.
We can use fsstat
to view this information via Sleuthkit. We have to tell it where the filesystem if we access the volume directly, as it doesn’t do partition table parsing itself:
fsstat -o 2 /dev/rdisk4
FILE SYSTEM INFORMATION
--------------------------------------------
File System Type: FAT16
OEM Name: mkfs.fat
Volume ID: 0x4bd54708
Volume Label (Boot Sector): MARC's USB
Volume Label (Root Directory): MARC's USB
File System Type Label: FAT16
Sectors before file system: 2
File System Layout (in sectors)
Total Range: 0 - 3911677
* Reserved: 0 - 0
** Boot Sector: 0
* FAT 0: 1 - 256
* FAT 1: 257 - 512
* Data Area: 513 - 3911677
** Root Directory: 513 - 576
** Cluster Area: 577 - 3911616
** Non-clustered: 3911617 - 3911677
METADATA INFORMATION
--------------------------------------------
Range: 2 - 62578646
Root Directory: 2
CONTENT INFORMATION
--------------------------------------------
Sector Size: 512
Cluster Size: 32768
Total Cluster Range: 2 - 61111
FAT CONTENTS (in sectors)
--------------------------------------------
577-640 (64) -> EOF
641-704 (64) -> EOF
705-768 (64) -> EOF
...more output...
Immediately following the reserved area (whose size is described above) comes the FATs themselves, with no header or footer.
File Allocation Tables
The FATs come next. In our example, there are 2 of them, each 256 sectors long. In FAT16, they are a sequence of 16-bit entries, one per cluster in the data area.
Entries corresponding to unallocated clusters store the value zero.
Entries corresponding to file/directories that are the last (including only) cluster allocated to the file contain any value greater than or equal to 0xFFF8 (again, little endian).
Entries containing 0xFFF7 indicate a bad sector (modern hard drives usually handle this, though it might matter if someone was attempting to hide data).
Other values tell you the next cluster of a file (or directory), given the current cluster.
Weirdly, the address of the first cluster in the data area is 2. And so the first two entries of the FAT don’t correspond to clusters; they are usually used store a copy of the media type, and the “dirty status” — whether the disk was ejected properly and/or the OS was shut down before poweroff.
Let’s look at the start of our FAT:
dd if=/dev/rdisk4 bs=512 skip=3 count=1 |hexdump -Cv
1+0 records in
1+0 records out
512 bytes transferred in 0.000652 secs (785473 bytes/sec)
00000000 f8 ff ff 7f ff ff ff ff ff ff ff ff ff ff ff ff |................|
00000010 ff ff ff ff 00 00 ff ff 00 00 ff ff ff ff ff ff |................|
00000020 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |................|
00000030 00 00 1a 00 ff ff ff ff ff ff ff ff ff ff ff ff |................|
00000040 ff ff ff ff ff ff 24 00 ff ff 26 00 ff ff 28 00 |......$...&...(.|
00000050 29 00 2a 00 ff ff 2c 00 2d 00 2e 00 ff ff 30 00 |).*...,.-.....0.|
00000060 ff ff 00 00 ff ff ff ff ff ff ff ff ff ff ff ff |................|
00000070 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |................|
00000080 ff ff ff ff ff ff 00 00 ff ff ff ff ff ff ff ff |................|
00000090 ff ff ff ff ff ff ff ff 00 00 4e 00 ff ff 00 00 |..........N.....|
000000a0 00 00 ff ff ff ff 00 00 00 00 ff ff 00 00 00 00 |................|
000000b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000000c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000000d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000000e0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000000f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000100 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000110 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000120 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000130 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000140 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000150 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000160 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000170 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000180 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000190 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000001a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000001b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000001c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000001d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000001e0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000001f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000200
fsstat
shows us some of this too, but note all TSK tools use sectors instead of clusters, so the offsets differ.
fsstat -o 2 /dev/rdisk4
...other stuff comes first...
FAT CONTENTS (in sectors)
--------------------------------------------
577-640 (64) -> EOF
641-704 (64) -> EOF
705-768 (64) -> EOF
769-832 (64) -> EOF
833-896 (64) -> EOF
897-960 (64) -> EOF
961-1024 (64) -> EOF
1025-1088 (64) -> EOF
1153-1216 (64) -> EOF
1281-1344 (64) -> EOF
1345-1408 (64) -> EOF
1409-1472 (64) -> EOF
1473-1536 (64) -> EOF
1537-1600 (64) -> EOF
1601-1664 (64) -> EOF
1665-1728 (64) -> EOF
1729-1792 (64) -> EOF
1793-1856 (64) -> EOF
1857-1920 (64) -> EOF
1921-1984 (64) -> EOF
2049-2176 (128) -> EOF
2177-2240 (64) -> EOF
2241-2304 (64) -> EOF
2305-2368 (64) -> EOF
2369-2432 (64) -> EOF
2433-2496 (64) -> EOF
2497-2560 (64) -> EOF
2561-2624 (64) -> EOF
2625-2688 (64) -> EOF
2689-2816 (128) -> EOF
2817-2944 (128) -> EOF
2945-3200 (256) -> EOF
3201-3456 (256) -> EOF
3457-3584 (128) -> EOF
3649-3712 (64) -> EOF
3713-3776 (64) -> EOF
3777-3840 (64) -> EOF
3841-3904 (64) -> EOF
3905-3968 (64) -> EOF
3969-4032 (64) -> EOF
4033-4096 (64) -> EOF
4097-4160 (64) -> EOF
4161-4224 (64) -> EOF
4225-4288 (64) -> EOF
4289-4352 (64) -> EOF
4353-4416 (64) -> EOF
4417-4480 (64) -> EOF
4481-4544 (64) -> EOF
4545-4608 (64) -> EOF
4609-4672 (64) -> EOF
4673-4736 (64) -> EOF
4801-4864 (64) -> EOF
4865-4928 (64) -> EOF
4929-4992 (64) -> EOF
4993-5056 (64) -> EOF
5057-5120 (64) -> EOF
5121-5184 (64) -> EOF
5185-5248 (64) -> EOF
5249-5312 (64) -> EOF
5377-5504 (128) -> EOF
5633-5696 (64) -> EOF
5697-5760 (64) -> EOF
5889-5952 (64) -> EOF
More on this Thursday…