Building Linux Kernel

1) Obtain Linux Source code
——-> 1.1) Obtain as a single file

Kernel is hosted at kernel.org. You can download the latest kernel from
ftp://ftp.kernel.org/pub/linux/kernel/

$ wget -c https://www.kernel.org/pub/linux/kernel/v3.x/linux-3.13.5.tar.xz

——-> 1.2) Clone using git

$ git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git linux-git

2) Configure
——-> 2.1) Old Config

$ make oldconfig

——-> 2.2) Custom Config

$ make menuconfig

3) Building

$ make -j4

4) Installing

# make modules_install install

Linux executable without main() Function | Write a C Executable program without main() function

Very Short version :

1) Create the program without main() function.

			
rajkumar.r@17:56:58:~/workspace/raj/workouts/ex_without_main$ cat exe_wo_main.c 
#include 
#include 
const char my_interp[] __attribute__((section(".interp"))) = "/lib/ld-linux.so.2";
int fn_wo_main() {
	printf("This is a function without main\n");
	exit(0);
}
rajkumar.r@17:57:02:~/workspace/raj/workouts/ex_without_main$

2) Compile it as a Shared object

gcc -shared -Wl,-e,fn_wo_main exe_wo_main.c -o exe_wo_main.so

3) Run it!!

			
rajkumar.r@17:57:36:~/workspace/raj/workouts/ex_without_main$ ./exe_wo_main.so 
This is a function without main
rajkumar.r@17:58:04:~/workspace/raj/workouts/ex_without_main$

Longer Version with Explanation :

Generally, all of the C programs starts at main() function. This is a standard defined fro C programming. When we look at the internals of this, we can understand how the system actually works in the background. C is a compiled language, which means, the source code is converted into the executable, before the execution starts. The general C program compilation process goes through the following steps, which is well documented in several other websites.

C Source code
Preprocessor – Intermediate files
Compiler – Object files
Linker – Linked Object file [ Executable file ]
Loader – Loads the executable and executes.

For our goal, we have to understand the linking process. We write our code starting from main(). But before the main(), there are several things happens in the background. Like, setting up of the environment, fetching input, configuring console etc. These are beautifully abstracted by GCC and the host system which hides those process from the user.

Here, we will unravel a small step of the long and complex process. The entry point of the execution. When the source code is preprocessed and compiled, the object codes are generated. After this, the standard and user built object codes are linked to generate the executable. The linking is done by the linker script which directs how to create the binary executable. This is where, the entry point of the executable is defined and configured as main().

Before main(), there happens few configurations. One among them is, configuration of dynamic loader, which loads the required symbols at run time. Usually in Linux, it is /lib/ld-linux.so.2, which inturn links to the corresponding loader provided by the compiler collection. In my case, it seems to be as below.

rajkumar.r@17:59:50:~/workspace/raj/workouts/ex_without_main$ ls -l /lib/ld-linux.so.2 
lrwxrwxrwx 1 root root 25 Jan 28  2013 /lib/ld-linux.so.2 -> i386-linux-gnu/ld-2.15.so

GCC Compiler is a very powerful and sophisticated compiler. GCC also offers a lot of control to the developers. One among several wonderful option is, -Wl,option. According to gcc man page,

-Wl,option
	   Pass option as an option to the linker.  If option contains commas, it is split
	   into multiple options at the commas.  You can use this syntax to pass an argument
	   to the option.  For example, -Wl,-Map,output.map passes -Map output.map to the
	   linker.  When using the GNU linker, you can also get the same effect with 
	   -Wl,-Map=output.map

From ld manual @t http://ftp.gnu.org/pub/old-gnu/Manuals/ld-2.9.1/html_node/ld_24.html,

The linker command language includes a command specifically for defining the first executable instruction in an output file (its entry point). Its argument is a symbol name:

ENTRY(symbol)

Like symbol assignments, the ENTRY command may be placed either as an independent command in the command file, or among the section definitions within the SECTIONS command--whatever makes the most sense for your layout.

ENTRY is only one of several ways of choosing the entry point. You may indicate it in any of the following ways (shown in descending order of priority: methods higher in the list override methods lower down).

	the `-e' entry command-line option;
	the ENTRY(symbol) command in a linker control script;
	the value of the symbol start, if present;
	the address of the first byte of the .text section, if present;
	The address 0. 

For example, you can use these rules to generate an entry point with an assignment statement: if no symbol start is defined within your input files, you can simply define it, assigning it an appropriate value---

start = 0x2020;

The example shows an absolute address, but you can use any expression. For example, if your input object files use some other symbol-name convention for the entry point, you can just assign the value of whatever symbol contains the start address to start:

start = other_symbol ;

We are going to combine all of these above features to run a program without main function. The steps are..

1) Create the program without main() function.

rajkumar.r@17:56:58:~/workspace/raj/workouts/ex_without_main$ cat exe_wo_main.c 
#include 
#include 
const char my_interp[] __attribute__((section(".interp"))) = "/lib/ld-linux.so.2";
int fn_wo_main() {
	printf("This is a function without main\n");
	exit(0);
}
rajkumar.r@17:57:02:~/workspace/raj/workouts/ex_without_main$

Here, you can see we have a strange thing at line number 2. This is to compensate the missing main() and adding the dynamic loader section. 🙂

2) Compile it as a Shared object

gcc -shared -Wl,-e,fn_wo_main exe_wo_main.c -o exe_wo_main.so

3) Run it!!

rajkumar.r@17:57:36:~/workspace/raj/workouts/ex_without_main$ ./exe_wo_main.so 
This is a function without main
rajkumar.r@17:57:36:~/workspace/raj/workouts/ex_without_main$

Done!!!

Building and Booting Linux using Qemu

Previously, we have Built and Booted U-Boot through Qemu.
Now, let us build and boot Linux using Qemu.
Get the latest kernel source from https://www.kernel.org/
I took Stable 3.9.3 as on writing.

mkdir original
mkdir src
cd original
wget -c https://www.kernel.org/pub/linux/kernel/v3.x/linux-3.9.3.tar.xz
cd ../src
tar -xf ../original/linux-3.9.3.tar.xz

Let us define the enviroinment variables that kernel build uses..

export ARCH=arm
export CROSS_COMPILE=arm-none-eabi-

Now, Let us configure this kernel build for Versatile Express.
This config is available at

./arch/arm/configs/vexpress_defconfig

For list of available configs, you can further explore in arch/ directory

make vexpress_defconfig;

Now, We need to make few changes, to make this kernel usable for our
needs in latter times.For this, We can remov module support (for
simplicity) and enabled EABI support as a binary format (allowing also old ABI).
This is necessary to run software compiled with the CodeSourcery toolchain.
Enable

kernel Features ---> Use the ARM EABI to compile the kernel and  
Kernel Features ---> Allow old ABI binaries to run with this kernel 

make menuconfig

We are all set to build the kernel. Now run

make -j4 all

Here, -j4 informs the build to use 4 build Jobs == Number of cores in your machine.
It will take some time to build.

Meanwhile, let us see what is the root file system and why do we need one, and
how to boot the kernel with a simple program.

Here is a definition of Root File System from Linux Information project

	
The root filesystem is the filesystem that is contained on the same 
partition on which the root directory is located, and it is the 
filesystem on which all the other filesystems are mounted (i.e.,
logically attached to the system) as the system is booted up 
(i.e., started up). 

The exact contents of the root filesystem will vary according to the 
computer, but they will include the files that are necessary for 
booting the system and for bringing it up to such a state that
the other filesystems can be mounted as well as tools for fixing a 
broken system and for recovering lost files from backups. The 
contents will include the root directory together with a minimal set 
of subdirectories and files including /boot, /dev, /etc, /bin, /sbin 
and sometimes /tmp (for temporary files).

Hope the kernel would have been built and ready by now.
The build should have completed with a message like,

  OBJCOPY arch/arm/boot/zImage
  Kernel: arch/arm/boot/zImage is ready
rajkumar.r@14:56:56:~/kernel/3_linux_qemu/src/linux-3.9.3$

The kernel will be available at

arch/arm/boot/zImage

Now, we can try to boot the kernel using qemu as below.

qemu-system-arm -M vexpress-a9 -kernel arch/arm/boot/zImage -append\
 "console=tty1"

-M vexpress-a9 : Emulate V Express Board
-kernel arch/arm/boot/zImage : Use this file as kernel
-append "console=tty1" console acts as the tty1
- generall Linux uses tty interface to display console messages

Here, you can read what is tty

But, now, the kernel will end up in panic telling something like,

VFS: Cannot open root device "(null)" or unknown block(0,0): error -6
Please append a correct "root=" boot option;
Kernel panic - not syncing: VFS: Unable to mount root fs on unknown 
block(0,0)

Now, What we discussed about File system comes useful.
As the kernel message is telling, we are missing a root file system. To build
a complete set of root file system is a complex task [ Relatively 😛 ] So, we
are going to generate a simple file system.

Creating a Simple Filesystem :

create file test.c in the src directory with the below content.

#include 

void main() {
	while(1) 
		printf("Hello World!\n");  
}

As the program looks, it will run continouesly as kernel expects the first
program to run forever. Compile this program, using cross compiler for Arm
running Linux
[ This is not same as bare metal toolchain. Bare metal toolchain is for ARM
which has no OS. i.e like arm-none-eabi-, which we have exported while
building kernel.]

arm-none-linux-gnueabi-gcc -static test.c -o test

This will Compile and creates an ELF, staticaly liked to all required code,
in a single binary. We need a Filesystem, but we have a binary file now.
So we need to generate Filesystem using some tool. Before that, we should know,
What is initramfs?

https://wiki.ubuntu.com/Initramfs

initramfs, as the name tells, its the Initial Ram File System. This is
introduced for Linux 2.6 kernel, before which initrd is being used.

From ubuntu Wiki

The key parts of initramfs are:

1) CPIO archive, so no filesystems at all are needed in kernel. 
   The archive is simply unpacked into a ram disk.
2) This unpacking happens before do_basic_setup is called. This means 
   that firmware files are available before in-kernel drivers load.
3) The userspace init is called instead of prepare_namespace. All 
   finding of the root device, and md setup happens in userspace.
4) An initramfs can be built into the kernel directly by adding it to 
   the  ELF archive under the section name .init.ramfs initramfs' can be
   stacked. Providing an initramfs to the kernel using the traditional
   initrd mechanisms causes it to be unpacked along side the initramfs'
   that are built into the kernel.
5) All magic naming of the root device goes away. Integrating udev into 
   the initramfs means that the exact same view of the /dev tree can be 
   used throughout the boot sequence. This should solve the majority of 
   the SATA failures that are seen where an install can succeed, but the
   initrd cannot boot.

This initramfs uses a format called newc. Now, to get the cpio archive,
initramfs from the binary, run the below command.

echo test | cpio -o --format=newc > rootfs

Now, we have the zImage kernel and rootfs – Initramfs. Let us load the kernel

qemu-system-arm -M vexpress-a9 -kernel linux-3.9.3/arch/arm/boot/zImage\
-initrd rootfs -append "root=/dev/ram rdinit=/test"

Here, 

-initrd rootfs : Qemu option which tells, rootfs is the Filesystem 
binary image.

root=/dev/ram and 
rdinit=/test are kernel options passed to the kernel we load.

rdinit=/test tells the kernel to run "test" executable we built as init.

Now, we can see the “hello world” being printed.

Voila!! Done!!

Booting Uboot in QEMU

U-Boot is universal bootloader, which is used very widely. Its a starting point to learn the low level hardware interactions for me. Qemu supports Versatile PB and we can try to emulate uboor on versatile PB using QEMU.

Creating workbench:

$ mkdir uboot
$ cd uboot;
$ mkdir original
$ mkdir src
$ cd original

Download U-Boot source code from U-Boot FTP.

$ wget -c ftp://ftp.denx.de/pub/u-boot/u-boot-latest.tar.bz2
$ cd ../src/
$ tar -xf ../original/u-boot-latest.tar.bz2
$ cd u-boot-2010.03

Compile the u-Boot code.

$ export ARCH=arm;
$ export CROSS_COMPILE=arm-none-eabi-;

$ make versatilepb_config 
$ make all

Now, if build went right, you should have u-boot.bin in the folder, and to boot this, run,

$ qemu-system-arm -M versatilepb -m 128M -nographic -kernel u-boot.bin;

Now you will be in uBoot prompt!!

U-Boot 2010.03-dirty (May 17 2013 - 15:52:24)

DRAM:   0 kB
## Unknown FLASH on Bank 1 - Size = 0x00000000 = 0 MB
Flash:  0 kB
*** Warning - bad CRC, using default environment

In:    serial
Out:   serial
Err:   serial
Net:   SMC91111-0
VersatilePB #

Try command like ‘?’, ‘help’ etc..

Numbering in Linux man page

man page

Today, when I was looking the man page of ‘init’, it was pointing to few more man pages as “See init(5) for more details on configuring Upstart.”

Then I noticed there is several such init with numbers. I have never noticed this numbering in the man pages.

A search on this lead to see man page of the man application, which explained the details further..

$ man man

NAME
       man - an interface to the on-line reference manuals

DESCRIPTION
       man  is  the  system's manual pager. Each page argument given to man is normally the name of a
       program, utility or function.  The manual page associated with each of these arguments is then
       found  and  displayed. A section, if provided, will direct man to look only in that section of
       the manual.  The default action is to search in all of the  available  sections,  following  a
       pre-defined  order  and to show only the first page found, even if page exists in several sec-
       tions.

       The table below shows the section numbers of the manual followed by the types  of  pages  they
       contain.

       1   Executable programs or shell commands
       2   System calls (functions provided by the kernel)
       3   Library calls (functions within program libraries)
       4   Special files (usually found in /dev)
       5   File formats and conventions eg /etc/passwd
       6   Games
       7   Miscellaneous (including macro packages and conventions), e.g. man(7), groff(7)
       8   System administration commands (usually only for root)
       9   Kernel routines [Non standard]

       A manual page consists of several sections.

       Conventional  section  names  include  NAME,  SYNOPSIS,  CONFIGURATION,  DESCRIPTION, OPTIONS,
       EXIT STATUS, RETURN VALUE, ERRORS, ENVIRONMENT, FILES, VERSIONS, CONFORMING TO,  NOTES,  BUGS,
       EXAMPLE, AUTHORS, and SEE ALSO.

And from this stack exchange question, I came to know more usages of man command as following.

  1. man printf – Shows default man page.
  2. man -a printf – Shows all available man pages, one by one as you quit each
  3. man 1 printf – Shows only man page in the corresponding section
  4. man -k printf – Shows all man pages having printf in name

what is this Ingress Protection ??

Recently Sony Xperia ZR is out, claiming to be compliant with IP55 and IP58 [with water proof and dust proof, which is technically measured through, Ingress Protection Code]

So, what is this Ingress Protection??

Here is some info..
http://en.wikipedia.org/wiki/IP_Code
http://answers.yahoo.com/question/index?qid=20061122080321AAAa6wP

IP (Ingress Protection) codes define how well an enclosure protects against the external environment.

As the number gets higher, the protection gets better.

The first digit defines the protection level against “solids”, the second against “liquids”. You MUST use both digits to define an enclosure.

For the first digit:
0 = No protection
1 = Protected against objects greater than 50 mm diameter
2 = Protected against objects greater than 12 mm diameter
3 = Protected against objects greater than 2.5 mm diameter
4 = Protected against objects greater than 1.0 mm diameter
5 = Dust protected
6 = Dust tight

For the second digit:
0 = Not protected
1 = Protected against dripping water (i.e. vertical plane)
2 = Protected against dripping water up to 15 degree angle
3 = Protected against spraying water
4 = Protected against splashing water
5 = Protected against water jets
6 = Protected against heavy seas
7 = Protected against effects of immersion
8 = Protected against submersion

BR,
Rajkumar R

Cross Compiling – Reblog

Original Blog : http://baruch.siach.name/blog/posts/introduction_to_cross_compilation_part_1/

Introduction to Cross Compilation, Part 1

This post is the first in a series on cross compilation. In this series I’ll introduce the concept of cross compilation, and how to used it. Although there are many different uses for cross compilation, I’ll focus in this series in its use for embedded Linux systems development.

What is Cross Compilation?

When you develop a desktop or server application, almost always the development platform (the machine that runs your compiler) and the target platform (the machine that runs your application) are the same. By “platform” I mean the combination of CPU architecture, and Operating System. The process of building executable binaries on one machine, and run them on another machine when the CPU architecture or the Operating System are different is called “cross compilation”. A special compiler is needed for doing cross compilation that is called “cross compiler“, and sometimes just “toolchain”.

For example, desktop PC application developers for Windows or Linux can build and run their binaries on the very same machine. Even developers of server applications generally have the same basic architecture and Operating System on both their development machine and server machine. The compiler used in these cases is called “native compiler”.

On the other hand, developers of an embedded Linux application that runs on a non PC architecture (like ARM, PowerPC, MIPS, etc.) tend to use a cross compiler to generate executable binaries from source code. The cross compiler must be specifically tailored for doing cross compilation from the development machine’s architecture (sometimes called “host”), to the embedded machine’s architecture (called “target”).

Note: cross compilation is only needed when generating binary executables from source code written in a compiled language, like C or C++. Programs written in interpreted language, like Perl, Python, PHP, or JavaScript, do not need a cross compiler. In most cases interpreted programs should be able run unchanged on any target. You do need, however, a suitable interpreter running on the target machine.

What is Cross Compilation Good for?

I have covered above one reason for doing cross compilation, that is, the target machine has a different CPU architecture that the development host. In this case cross compilation is necessary because the binaries that the native compiler generates won’t run on the target embedded machine.

Sometimes cross compilation is not strictly necessary, but native compilation in not practical, or inconvenient. Consider, for example, a slow ARM9 based target machine running Linux. Having the compiler run on this target will make the build process painfully slow. In many cases target machine is just under-powered, in terms of storage and RAM, for the task of running a modern compiler.

Practically speaking, almost all embedded Linux development is being done with cross compilers. Strong PC workstation machines are used as development hosts to run the development environment (text editor, IDE), and the cross compiler.
Obtaining a Cross Compiler

The easiest way to obtain a cross compiler is to download a ready made pre-built one. Besides being easy to obtain a pre-built binary toolchain is the most useful for the general case of building a kernel and a userspace filesystem. Some special cases require a specially tailored toolchain built from source. I’ll show how to build a toolchain from source in the next post.

A short terminology note: in the following text I use the terms “cross compiler” and “toolchain” interchangeably. The have the same meaning in this context. The term “toolchain” seems to be more popular, however.

Sourcery

The most well known source of pre-built cross compilers is the embedded software division of Mentor Graphics, formerly known as CodeSourcery, an independent company that Mentor has acquired in 2010. They release the “Sourcery CodeBench Lite Edition” free of charge. Sourcery CodeBench is a collection of cross compilers for several CPU architectures, including ARM, PowerPC, MIPS, and Intel x86 among the others. For each architecture there are a number of target options. The one you need for embedded Linux work is the “GNU/Linux release”. Always select the latest version, unless you have a very good reason to avoid it. Then, there are a few packaging formats to choose from. I prefer the “IA32 GNU/Linux TAR” format. Installing it is just a matter of extracting the tar file in the /opt directory. For example, to install the latest MIPS toolchain do as root

tar xjf mips-2011.09-75-mips-linux-gnu-i686-pc-linux-gnu.tar.bz2 -C /opt
One big advantage of Sourcery’s toolchains is that those making them, former CodeSourcery employees, are deeply involved in upstream development of the GCC compiler.

Linaro

The Linaro organization also releases pre-built cross compilers for their target platform, newer ARM processor based on Cortex-A. Download the latest version from here. You need the “Linux binary” one.

Using the Cross Compiler

This is just a quick peek at cross compiling for the impatient new embedded Linux coder. I’ll come back to this issue later in some greater depth.

First, put your newly installed toolchain in your path. For example, the Sourcery MIPS toolchain mentioned above needs the following command:

export PATH=$PATH:/opt/mips-2011.09/bin

Create a simple “Hello World” program, and save it in hello.c:

#include 
int main (void) {
    printf ("Hello World!\n");
    return 0;
}

Compile your program using the MIPS toolchain as follows:

mips-linux-gnu-gcc -Wall -o hello hello.c

Copy the resulting hello binary file to you target machine and run it there. If all goes well you should see the expected output.

There are many details to get wrong here, ranging from ABI issues, to C library and kernel version compatibility. I’ll cover some of these issues in future posts.