Rendering Linux Console to VGA screen: Part 1

Foreword

In the previous post we managed to boot Linux from an SDCARD on the Zybo board without any manual intervention.

This is one step closer for our Zybo board in running standalone, but our Zybo board still needs to be attached to a PC in order for us to view the output of the Linux console.

In this post and in the next one, we will see if we can display the Linux console output on a VGA screen that is attached to the Zybo board. Once we have achieved this goal we would be able to say that our Zybo board can truly operate on its own without the help of a PC.

Overview

The following diagram gives an overview of what we want to achieve in this post and the next one:

So, in the end we want to have a module implemented in FPGA that continously read pixel data from SDRAM and output it to a VGA screen.

Such an area in SDRAM is referred to a as a framebuffer in the Linux world and is accompanied by a Framebuffer driver.

We ultimately want the output of the Linux console to be rendered as bitmap images to the framebuffer so that it can be displayed on the VGA screen.

In this post we will be dealing with the Linux part of the equation, which will be developing the Framebuffer driver and see how it can be integrated with the Linux console.

In the next post we will deal with the FPGA part of the equation.

Linux Console interaction with Framebuffer Subsystem

We start off by investigating how the Linux Console interact with the Framebuffer subsystem.

Firstly, let us have a closer look at a very simple framebuffer driver. For this purpose we will be using the simple framebuffer which is available within the Linux source code, located in the path drivers/video/fbdev/simplefb.c.

When a framebuffer device has loaded successfully in Linux, an entry under /dev will be created of the form fbX, where X is 0 for the first framebuffer device.

However, if you look through the source of the file simplefb.c, you will not really find any evidence where the node fbX is created under the /dev node.

A clue to this question is found within the method simplefb_probe at the following method call:

...
ret = register_framebuffer(info);
...

The declaration of this function can be traced to the file drivers/video/fbdev/core/fbmem.c. This function eventually calls the function do_register_framebuffer and this where the entry gets created under /dev:

...
 fb_info->dev = device_create(fb_class, fb_info->device,
         MKDEV(FB_MAJOR, i), NULL, "fb%d", i);
...

So far so good, but how do you get a Linux cosole to use this framebuffer device to render its contents?

To accomplish this, there is some glue logic happening within the file drivers/video/console/fbcon.c. To explain what is going on in this file, there is a very nice text file Documentation/fb/fbcon.txt. The key in this document is load scenario #3:

3. Driver is compiled as a module, fbcon is compiled statically
You get your standard console.  Once the driver is loaded with
'modprobe xxxfb', fbcon automatically takes over the console with
the possible exception of using the fbcon=map:n option.

So, some nice magic is happening in the background. Provided fbcon is compiled statically into the kernel, as soon as you load a framebuffer driver as a module, everything should just work!

Building and Installing the Simple Framebuffer as a module

The Simple Framebuffer driver mentioned in the previous section is a perfect match for as framebuffer driver for the purposes of this post.

So, in this section we will be building this driver as a module, so we can use it later on.

Firstly, please ensure that your environment is setup, as described in this post.

Create an empty directory and copy the following from the Linux source tree to it: drivers/video/fbdev/simplefb.c

Within this new directory, you also need to create file called Makefile, with the following content:

obj-m += simplefb.c

all:
 make -C <path to linux source> M=<path to folder containing simplefb.c> modules

clean:
 make -C <path to linux source> M=<path to folder containing simplefb.c> clean

Substitute the square brackets with the paths that are applicable in your situation.

You can now build the module with the following command:

make ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi-

After the build process is completed, a new file should have been created called simplefb.ko, in the same folder as the source file.

You will need to copy this file to ramdisk that you are using to boot Linux on the Zybo (Within any home directory will do). There is a number of steps involved for changing the contents of a RAMDisk, and is covered in this resource: https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/18842473/Build+and+Modify+a+Rootfs

In order to load the simplefb module, we also need to make some modifications to the device tree. This involves adding the following node to zynq-7000.dtsi:

 framebuffer0: framebuffer@1d385000 {
  compatible = "simple-framebuffer";
  reg = <0x1d385000 (1600 * 1200 * 2)>;
  width = <1600>;
  height = <1200>;
  stride = <(1600 * 2)>;
  format = "r5g6b5";
 };

This node corresponds more or less as describe in the document Documentation/devicetree/bindings/video/simple-framebuffer.txt

With this change you will need to recompile the device tree and copy it to the SD Card that you use to boot the Zybo board into Linux.

Initial attempt to load new Module

With our modified device tree and ramdisk copied to the SDCard, let us boot the Zybo board into Linux.

To load the module, issue the following command:

insmod /home/default/simplefb.ko

This assumes that you have copied this kernel module file to /home/default.

In my initial attempt my kernel module failed to load successfully. I was present with error code -12, which is an out of memory error.

To troubleshoot this issue, I added a number of printk statements (e.g. the kernel's version of printf) to get an idea of where the module actually falls over. Eventually I found the snippet that was causing the issue:

 info->screen_base = ioremap_wc(info->fix.smem_start,
           info->fix.smem_len);

The problem appear to be related to memory management.

In the next section we will have a closer look into memory management with this driver.

Memory management within Simplefb

Let us start by finding out what the function ioremap_wc does.

In effect ioremap_wc is a variant of the ioremap function, which is described here. The following sentence quoted from this resource, summarise the purpose of this function:

A successful call to ioremap() returns a kernel virtual address corresponding to start of the requested physical address range.

This makes kind of sense within the context of a framebuffer driver. When display hardware access SDRAM, it can only work with physical memory addresses. Virtual addresses is simply meaningless to display hardware.

The Linux kernel and User programs, however, only deals with Virtual address space. In the majority of cases the kernel/user program can't easily know what the physical address is for a given virtual address.

For this purpose we need the ioremap_wc function. We have a reserved area in RAM for the Framebuffer, and this function just translate this physical address range into a virtual one so that the Linux kernel can work with it.

Next, let us have a look at how the paramteres to the function ioremap_wc are derived:

...
mem = platform_get_resource(pdev, IORESOURCE_MEM, 0);
...
info->fix.smem_start = mem->start;
info->fix.smem_len = resource_size(mem);
...
info->screen_base = ioremap_wc(info->fix.smem_start,
           info->fix.smem_len);
...

The values for the resource IORESOURCE_MEM is retrieved from the framebuffer node in the device tree in the following attribute:

 framebuffer0: framebuffer@1d385000 {
...
  reg = <0x1d385000 (1600 * 1200 * 2)>;
...
 };

The first number is the starting address for the framebuffer, followed by the size of the framebuffer.

If we convert these numbers to MegaBytes, the framebuffer starts at around 467MB and is about 3MB in size.

As the Zybo board contains 512MB with the Linux Kernel using most of this, one can clearly see that the memory range of the framebuffer conflict with the memory range of the Linux Kernel.

One can solve this issue by moving the framebuffer to the top of memory and lowering the upper limit of memory that Linux can reserve for its own use.

To move the framebuffer to the top of RAM, we need to change the framebuffer node in the device tree as follows:

 framebuffer0: framebuffer@1d385000 {
...
  reg = <0x1fb00000 (1600 * 1200 * 2)>;
...
 };

This will effectively move the framebuffer to address 507MB.

To set the top of usable Linux memory, we need to supply a parameter to the Linux Kernel when it boots up. We set this as a persistable variable within UBoot, with the following commands at the uboot console:

setenv bootargs "mem=500M"
saveenv

If you now restart the Zybo board, you will find that the same insmod command will load the frambuffer successfully.

Loading simplefb automatically at startup

At this point the loading of the framebuffer is a manual process. It would be nice if we can automatically load this module at startup.

If I do an Internet search on how to autoload a module on startup in Linux, I basically get two suggestions:

• Adding the module to the file /etc/modules (for use with SysVinit init system)
• Adding the module to a conf file under /etc/modules-load.d (for use with systemd init system.)

In short, neither of the above methods methods work on the Linux booting on my Zybo board.

To get some insight into the matter, let us have a closer look into how the above two methods are implemented on different Linux distributions.

It turns out that each Linux distribution can only implement one of the methods and is based on the init system it support. An init system is basically the system that Linux invoke after it has started up.

After Linux has started up, it will look for executable called init in the /sbin directory.

In Ubuntu 15 and subsequent versions, for instance, /sbin/init will point to the executable /lib/systemd/systemd. These versions clearly uses the systemd init system, which will invoke scripts under etc/modules-load.d.

Ubuntu versions before Ubuntu 15 utilises the SysVinit system, which will automatically load modules within the file /etc/modules.

It is kind of easier to figure out how systemd invoke scripts under etc/modules-load.d than it is to figure out how SystemVinit loads the modules specified in /etc/modules.

To get an idea how SytemVinit does this, we have to boot into a pre-Ubuntu 15 version. On my PC I am currently running Ubuntu 16.04, so I need to jump through a couple of extra hoops to test this.

I downloaded a Ubuntu 13 iso, after which I used Virtualbox to boot this image. It is just quicker this way, than to first write this image to a USB stick or a CD and then booting from it.

With this ISO bootup within Virtualbox, we are prompted whether we want to install or just try out ubuntu. I opted for the quick option, try Ubuntu.

With UBuntu 13.04 booted and having opened up a Terminal Window, we ask ourselves: what should we be looking for? Doing some further reading on the Internet, I found that we should look out for a file called /etc/init/kmod.conf.

This file indeed exists on a Ubuntu 13.04 installation and opening it up reveals something very interesting:

We have found where the module autoloading from /etc/modules happen!

Let us now check if we can find something similar in our Linux installation on the Zybo board.

Firstly, let us check to what /sbin/init is mapped to. This is actually something a bit different than we expected:

lrwxrwxrwx    1 root     root            14 Nov 27  2012 init -> ../bin/busybox

On the other hand, however, this link to init is something you will find on most embedded systems.

Busybox is very light, but sacrifice some functionality.

The Busybox init utility read the /etc/inittab file that explains what to do at various events.

The Linux installation on the Zybo board, have a inittab file that looks as follows:

::sysinit:/etc/init.d/rcS

# /bin/ash
#
# Start an askfirst shell on the serial ports
# and the tty0 for when the video is being used

ttyPS0::respawn:-/bin/ash
tty0::respawn:-/bin/ash

# What to do when restarting the init process

::restart:/sbin/init

# What to do before rebooting

::shutdown:/bin/umount -a -r

From this snippet we see that when Linux starts up the script /etc/init.d/rcS will get executed.

The quickest way get our module to load at startup, is to add it to this rcS script.  The resulting rcS script will something like the following:

#!/bin/sh

echo "Starting rcS..."
insmod /lib/modules/3.6.0/simplefb.ko
echo "++ Mounting filesystem"
mkdir -p /dev/pts
mkdir -p /dev/i2c
mount -a
...

This is enough configuration for our kernel module to load at startup.

Testing results

With our module that loads automatically at startup, it would be interesting to see if the Linux console does in fact write to the framebuffer.

A quick way to check this is to use the XDB console to dump memory from the framebuffer region to a file.

I gave that a try and got something sensible back:

In this exercise a have also attached a USB keyboard and blindly started typing, which in this case was to change into the directory /etc and listing contents.

So, we know our Framebuffer works and a USB keyboard also works out of the box!

In Summary

In this post we integerated the simpleframebuffer into the Linux distrubition running on the Zybo board.

We confirmed that the Linux console indeed writes to the Framebuffer.

In the next post we will be developing the FPGA part of the design that will render the contents of the framebuffer to the VGA screen.

Till next time!