SD Card Access for a Arty A7: Part 7

Foreword

In the previous post we managed to read a sector of data from an SD Card.

In this post w will continue our journey and see if we can read a file from a FAT32 formatted partition. To be able to read a file from a partition will form an important part of being able to boot an Amiga core on a Arty A7, and thus to load a Boot ROM and disk image from SD Card.

There is quite number of technical details involved to read a file from a FAT32 partition. Writing the functionality for this right from the start in 6502 Assembly language is quite a daunting task.

To make our lives easier we will start to write the functionality in a High Level language. As my current knowledge of FAT32 is rather limited, experimenting with such a partition in a High Level Language will get one quickly up to speed.

Once we know how to read a file from a FAT32 partition, we can write 6502 assembly for this in a future post.

About MBR and FAT32

Before we look in detail how to read a file from an SD Card, let us start with some basic terminology.

Firstly, the storage of any SD Card is divided into many blocks, where each block is 512 bytes in size. The choice of 512 bytes per block is actually rooted in the history of Personal computers where almost any Floppy Disk Drive or Hard Drive had a basic block size of 512 bytes.

To address blocks, all the blocks are numbered consecutively starting at block 0, and going up to the maximum number of blocks the device supports.

Block zero is called the MBR or master boot record. This is also where the history of the IBM PC kicks in again and I think is probably still relevant today. When an IBM PC boots up, it looks for machine language program at block zero, to start the booting of the system. Hence the name Master Boot Record.

The MBR has some other purpose as well, which is to store one or more partition entries, so you can be able to create more than one volume on the same device. We briefly encountered this in the previous post where we saw data at location 0x1be of block 0. This data was in actual fact a partition entry.

Also, from the previous post, you will remember that apart from the partition entry data, all the other bytes of that block were zero. So, although we call block zero the MBR, the SD Cards you use today most probably will not contain machine code in that block.

Now back to the partition entry. A partition entry gives us the block number of the sector of the partition in question. This first sector of the partition is called again, surprise, a boot record! The name is again because of the legacy of the IBM PC.

The boot record also contain some data to hold of the File allocation table and to be able to read the contents of a file.

We will have a look at a typical boot record in the next section.

Looking at the boot record

To look at the boot record, we first need find the block number of the it, via the MBR. In the previous post I made a screenshot of the MBR from the SD Card I was playing with, which contained the partition entry.

Here is the screenshot again, but with the bytes of the partition entry highlighted:

As you can see the partition entry starts at 0x1be and is 16 bytes in size. To get the meaning of the bytes, we look at the following link:

https://en.wikipedia.org/wiki/Master_boot_record#PTE

Two pieces of useful information for us, in the last 8 bytes of the entry:

• offset 8: LBA of first absolute sector in the partition (4 bytes)
• offset C: Number of sectors in partition.

For interest sake, let us start by checking if we can calculate the size of the partition. The bytes at offset set are as follows: EB 0B 76 00

These bytes are in little endian order, so we need to reverse them: 00 76 0B EB. This gives us the number of sectors in hexadecimal. In decimal this is 7,736,299. To verify if this number is correct, we need to convert number of sectors to bytes, that is multiplying by 512:

3960,985,088

With the thousand separators, we get more or less to 4GB, which is the size of the SD Card I am using.

Next, let us determine the staring block of the FAT32 partition. The data bytes for this 15 20 00 00. Swop this around because of little endianness:

00 00 20 15

So, the starting block is 2015 in hex, which is, 8213 in decimal. Now, let us write a program for reading this sector from the SD Card dump and display it.

The program language I am going to use is Java. This just the computer language I use every day in my work, so for this reason I am going to use it.

First we need to open the dump:

RandomAccessFile fis = RandomAccessFile("dump.sdcard","r");

With instances of RandomAccessFile, we can easily jump around within different positions in the file, which is what will need for this exercise of attempting to read a file from a FAT32 partition in a dump file.

Now, using instances of RandomAccessFile, as well as any other classes which allows you to read from a file in Java, to surround it with a try-catch block to handle IOExceptions. However, to keep the conversation focussed, I will not go into details of exception handling in Java.

Next, let us add some code for seeking to the boot sector and reading it:

        fis.seek(8213 * 512);
        byte[] buf = new byte[512];
        fis.read(buf);
        var dataString = new String(buf, StandardCharsets.US_ASCII);
        System.out.println(dataString);

Obviously we need to multiply 8213 by 512, because the seek method wants the position in bytes. I then reaad the sector into a byte buffer and then convert it to a String to see if there is any interesting human readable properties. When printing the String, we see the following:

Interesting snippets, we see like MSDOS5.0 and BLACKBERRY FAT32. This is more or less strings we expect from a Bootsector formatted with FAT32. 

Some of you might be confused with the word BLACBERRY in the output. Well, I used a Blackberry Phone about 10 years ago. When I received the phone it became bundled with an SD Card, which I am using for this exercise.

This confirms that we found the correct sector as the boot sector. In the next section we go further looking into the info stored in this sector.

A deeper look into the Boot sector

To make sense of the info stored in the boot sector, the following resources is of great help:

https://wiki.osdev.org/FAT

As you can see, there is quite a number of sectors stored in Boot sector. We will only be needing a handful of these:

        sectorsPerFat = FatBrowser.readFourBytes(buf, 36);
        numFat = buf[16];
        numReserved = buf[14];
        sectorsPerCluster = buf[13];
        rootCluster = buf[44];
        dataStart = numReserved + numFat * sectorsPerFat;

Let us dissect this snippet of code a bit. buf is the byte buffer we read in the previous section, containing the boot sector. FatBrowser.readFourBytes() is a pseudo function for taking four bytes starting at position 36 of buf, and forming a number.

Now, with FAT32, which we use here, the available storage is divided into multiple clusters. Usually each cluster is more than one sector in size. In fact, with the SD Card I use in this post, each cluster is 64 sectors in size, indicated by the variable sectorsPerCluster.

With a cluster size of 64 sectors, it means that the size allocated per file will be at least 64 sectors in size and multiples of it. The FAT keeps track where the different parts of a file are on a disk and also works with clusters.

Having talked a lot about FATs and Clusters, let us see how these are arranged on the partition:

For the numbers in the diagram, I have used my SD Card as an example. The number in your case might differ.

In my case the partition starts with 9 reserved sector, of which the boot record is the first reserved sector.

Following the Reserved sectors comes the actual FAT, with one or more copies. The total number of FATs is indicated by byte 16 of the Boot record, indicated by the variable name numFat in the code snippet above. The actual size of 945 sectors of the FAT I got from the variable sectorsPerFAT.

The clusters that the FAT refers to lives in the Data Area. In my case, the Data Area begins at sector 9 + 945 + 945 = 1899.

Now, one may be tempted to say the first cluster in the Data Area is cluster number zero. This is, however, not the case with FAT, where the first cluster is numbered 2. The reason for this is because in the FAT table entries zero and one are reserved. The significance of this 2 is that any cluster number you obtain from the system, you need to subtract two to get the real cluster slot number within the Data Area.

The rootCluster in the code snippet above is no exception to the rule. In my case rootCluster is 2, meaning you will find the root directory at Slot 0 in the Data Area, e.g. right at the beginning of the Data Area.

The root directory contains actual file entries which we are interested in, which we will try to read in the next section.

Looking into the root directory

Let us have look at the how the first sector of the root directory looks like. The following Java snippet will read this sector and display it:

        fis.seek((8213 + 9 + 945+ 945) * 512);
        byte[] buf = new byte[512];
        fis.read(buf);
        var dataString = new String(buf, StandardCharsets.US_ASCII);
        for (int i = 0; i < 16; i++) {
            System.out.println(dataString.substring(0, 32));
            dataString = dataString.substring(32);
        }

The number we use in the seek I used as derived from the previous section, with 8213 the start of my FAT32 partition.

From the link I presented earlier on from osdev.org, I know each file entry in the root directory is 32 bytes, so I only print 32 bytes per line. The output I got from this program, looks like this:

Each line starts with something that looks like a filename. Some lines have characters which are separated by some whitespace. These characters are in actual fact Unicode characters which have two bytes per character.

These filenames with Unicode characters are actually entries for long filenames. For what we want to do, it is best to ignore long file entries, and just focus on the non-Unicode lines, like RECORD~1, BLACKB~1, MUSIC and so on. I will illustrate in a moment how we can filter out the long file entries.

Reading a file

Being able to view root directory entries, let us see if we can the contents of file. To do this, we first need to find the file entry in the root directory.

To do this, let us start by writing a code snippet for listing the filenames in the root directory, excluding the long file entries. To determine if an entry is a long file entry, we need to look at byte 11 of the file entry. From the link from OSDev.org, the bits in the attribute byte has the following meaning:

• READ_ONLY=0x01
• HIDDEN=0x02
• SYSTEM=0x04 
• VOLUME_ID=0x08 
• DIRECTORY=0x10 
• ARCHIVE=0x20 
• LFN=READ_ONLY|HIDDEN|SYSTEM|VOLUME_ID

For our purposes, we want to skip entries where the attribute is 15, resulting in the following code snippet:

        for (int i = 0; i < 16; i++) {
            int beginFileEntry = i * 32;
            if (buf[beginFileEntry + 11] == 15) {
                continue;
            }
            System.out.println(new String(buf, beginFileEntry, 11));
        }

This result in the following output:

This looks a lot cleaner than our previous attempt. One that looks strange in this output, however, is the entry preceded by the question mark. This is a deleted file, which we should also remove from our result. To skip past deleted files, we can just continue the loop as well if the first character of a filename is 0xE5.

Also, all the file entries we see here are directory entries. We need to search some more blocks in the is root directory cluster, to find some useful files to look at. So, we add an outer loop to our existing loop for reading the next block to look at:

        for (int j = 0; j < 63; j++) {
            fis.read(buf);
            for (int i = 0; i < 16; i++) {
                int beginFileEntry = i * 32;
                if (buf[beginFileEntry + 11] == 15) {
                    continue;
                }
                if ((buf[beginFileEntry] & 0xff) == 0xE5) {
                    continue;
                }
                System.out.println(new String(buf, beginFileEntry, 11));
            }
        }

This time we are seeing some more interesting stuff:

Here we see a couple of individual files, like WMPINFO.XML, CONTACTS.VCF, TEST.TXT and so on. Remember, these filenames are in 8.3 format. First 8 characters are the filename, followed by an extension of 3 characters. If the filename is less than 8 characters, you will space padding between filename and extension in output.

From these file entries, let us see if we can output the content of TEST.TXT. We modify our loops, so it will output the file entry slot within the current sector we are busy with:

        int foundSlot = -1;
        outerLoop:
        for (int j = 0; j < 63; j++) {
            fis.read(buf);
            for (int i = 0; i < 16; i++) {
                int beginFileEntry = i * 32;
                if (buf[beginFileEntry + 11] == 15) {
                    continue;
                }
                if ((buf[beginFileEntry] & 0xff) == 0xE5) {
                    continue;
                }
                if (new String(buf, beginFileEntry, 11).equalsIgnoreCase("TEST    TXT")) {
                    foundSlot = i * 32;
                    break outerLoop;
                }
            }
        }

Now, from the resulting file entry, we know the following:

• bytes 20 + 21: High 16 bits of first cluster number for file
• bytes 26 + 27: low 16 bits of first cluster number for file

To calculate the first cluster number of the file, we need to do some bit manipulation:

        int cluster = (buf[foundSlot + 21] & 0xff) >> 24 | (buf[foundSlot + 20] & 0xff) >> 16
                | (buf[foundSlot + 27] & 0xff) >> 8 | (buf[foundSlot + 26] & 0xff) >> 0;

Finally, we read the first sector of the file in question as follows:

        fis.seek((8213 + 9 + 945+ 945 + (cluster - 2) * 64) * 512);
        fis.read(buf);
        System.out.println(new String(buf));

In the seek command we basically start off again with calculation to find beginning of Data area, and adding to it the cluster number converted to a sector count. This is the output I get:

Ok, I admit I created this file beforehand and copied it to the SD Card, before making an image of it, which I used in this post😁

With all this done, I think we covered the basics of locating and reading a file from a FAT32 partition.

In Summary

In this post we continued our journey to find out how to read a file from an SD Card. We wrote some snippets of code to gradually explore how a FAT32 partition works and ended off by successfully reading file from such a partition.

In the next post we will redo the exercise, but by writing the code in 6502 Assembly language.

Until next time!