[WIP][Notes]Install Gentoo on embeded SoC

[PoP]

I pick a Raspberry Pi like board for my DIY personal NAS server project: NanoPi M4V2 with 4x SATA HAT.

After the hot swap case issue solved, hooked up the power switch on the casing to the board, it's time to prepare the boot up SD

First boot

Search result of boot SD image: Armbian, openmediavault

I tried the Armbian image but it doesn't boot, skipped openmediavault but tried the manufacturer's FriendlyCore image and it boot up fast.

1. plugin card reader / TF card to a Linux computer
2. checkout the device assignment from dmesg (mine is /dev/sdd)
3. login with supervisor rights from terminal, type:
   

   代碼: 選擇全部

   dd if=rk3399-sd-friendlycore-bionic-4.4-arm64-YYYYMMDD.img of=/dev/sdd

4. insert SD card into board's card slot
5. plugin keyboard, hdmi
6. press the Power button on the case
7. if nothing goes wrong, you will see a text console on your hdmi monitor
8. install hard disk(s) to the hot swap assembly and test hot plug/unplug
9. if sata Hat and casing hot swap works, then power down board from console:
   

   代碼: 選擇全部

   poweroff

10. it's time to get Gentoo!

Ref: Making a handmade fanless NAS with NanoPi M4 and SATA Hat, NanoPi M4 DiskStation / SATA NAS / Up to 56 TB Network Storage!, support page of manufacturer

Get Gentoo stage 3/4

1. Go to Gentoo official site and find the download links for Arm64 stage 3/4 files
2. power on the coming soon NAS server
3. ssh to the NAS from another Linux computer's terminal (default password for user pi is pi)
4. prepare the installed hard disk(s) (setup RAID0/1, etc., assign partitions, format with supported filesystem of your choice)
   *read Gentoo AMD64 Handbook for details
5. download and install the stage 3/4 files from connected ssh session
6. Chroot and customize your system in Gentoo way.

[PoP]

Hardware Info

• Wiki
• Boot option
  

  

  Boot flow

Boot method

1. Rockchip tools
   typical Rockchip boot flow with Rockchip miniloader
2. U-boot tool
   used for most SoCs with U-Boot TPL for ddr init and SPL for trust(ATF/OP-TEE) load and run into next stage;

Boot modes

• normal mode: full blown U-Boot -> kernel
• Falcon Mode: minimum required initialization (SPL -> U-Boot) -> kernel

Default storage map

Default storage map

Official wiki page: http://opensource.rock-chips.com/wiki_Partitions

Boot flow

代碼: 選擇全部

+--------+----------------+----------+-------------+---------+
| Boot   | Terminology #1 | Actual   | Rockchip    | Image   |
| stage  |                | program  |  Image      | Location|
| number |                | name     |   Name      | (sector)|
+--------+----------------+----------+-------------+---------+
| 1      |  Primary       | ROM code | BootRom     |         |
|        |  Program       |          |             |         |
|        |  Loader        |          |             |         |
|        |                |          |             |         |
| 2      |  Secondary     | U-Boot   |idbloader.img| 0x40    | pre-loader
|        |  Program       | TPL/SPL  |             |         |
|        |  Loader (SPL)  |          |             |         |
|        |                |          |             |         |
| 3      |  -             | U-Boot   | u-boot.itb  | 0x4000  | including u-boot and atf
|        |                |          | uboot.img   |         | only used with miniloader
|        |                |          |             |         |
|        |                | ATF/TEE  | trust.img   | 0x6000  | only used with miniloader
|        |                |          |             |         |
| 4      |  -             | kernel   | boot.img    | 0x8000  |
|        |                |          |             |         |
| 5      |  -             | rootfs   | rootfs.img  | 0x40000 |
+--------+----------------+----------+-------------+---------+

[PoP]

Compile Kernel

1. before chroot or open another ssh session:
   

   代碼: 選擇全部

   modprobe configs

2. Since board default config is not available for current(4.19/5.3) kernel sources, have to get the current working config as base:
   

   代碼: 選擇全部

   cd /usr/src/linux
   zcat /proc/config.gz > .config
   make oldconfig
   make menuconfig

3. copy .config to /usr/src, in case you needed to clean compile kernel later on.

4. 代碼: 選擇全部

   make
   make modules_install

Compile ARM Trusted Firmware

1. Prerequisites:
   
   • ARM Trusted Firmware: Cortex-M0 code needs an arm-none-eabi toolchain
   • Fetch and build arm-none-eabi toolchain: official site
   
   

   代碼: 選擇全部

   cd /usr/src
   git clone https://github.com/ARM-software/arm-trusted-firmware.git
   cd arm-trusted-firmware
   make realclean
   CFLAGS='-gdwarf-2' make DEBUG=0 ERROR_DEPRECATED=1 PLAT=rk3399 bl31

Compile U-Boot

*official site stated that supportted version is 2014.10

1. Prerequisites:
   
   • u-boot-tools
     

     代碼: 選擇全部

     emerge dev-embedded/u-boot-tools
     dev-python/pyelftools

   • dtc (not stable for aarch64, have to compile)
     

     代碼: 選擇全部

     emerge sys-apps/dtc

     or
     compile and install dtc commands to /usr/local:
     

     代碼: 選擇全部

     cd /usr/src
     git clone https://git.kernel.org/pub/scm/utils/dtc/dtc.git
     cd dtc
     make clean
     make
     make install PREFIX=/usr/local

2. 代碼: 選擇全部

   cd /usr/src

   
   official U-Boot sources
   

   代碼: 選擇全部

   git clone https://gitlab.denx.de/u-boot/u-boot.git
   cd u-boot

   or
   Rockchip sources
   

   代碼: 選擇全部

   git clone https://gitlab.denx.de/u-boot/custodians/u-boot-rockchip.git
   cd u-boot-rockchip

3. configure u-boot:
   

   代碼: 選擇全部

   make distclean
   make nanopi-m4-rk3399_defconfig

   
   *my board is version 2 which use DDR4, have to enable CONFIG_RAM_RK3399_LPDDR4:
   

   代碼: 選擇全部

   make menuconfig

   
   Device Drivers -> LPDDR4 support for Rockchip RK3399
   *also disable functions/devices you don't intended to use in kernel, to reduce compile time
4. copy bl31 to u-boot sources and compile:
   

   代碼: 選擇全部

   cp ../arm-trusted-firmware/build/rk3399/release/bl31/bl31.elf .
   make

[PoP]

prepare the images

1. Create tpl/spl idbloader image
   

   代碼: 選擇全部

   cd /usr/src/u-boot
   mkimage -n rk3399 -T rksd -d tpl/u-boot-tpl.bin idbloader.img

   
   

   Image Type: Rockchip RK33 (SD/MMC) boot image
   Data Size: 63488 bytes

   
   

   代碼: 選擇全部

   cat spl/u-boot-spl.bin >> idbloader.img

2. prepare boot.img
   * Uncompress kernel image for kernel 5, confirm overwrite existing Image file
   

   • 代碼: 選擇全部

     gzip -d arch/arm64/boot/Image.gz

   代碼: 選擇全部

   cd /usr/src/linux
   mkdir boot
   cp arch/arm64/boot/Image boot/

   
   for kernel 4:
   

   代碼: 選擇全部

   cp arch/arm64/boot/dts/rockchip/rk3399-evb.dtb boot/rk3399.dtb

   
   for kernel 5:
   

   代碼: 選擇全部

   cp arch/arm64/boot/dts/rockchip/rk3399-nanopi-m4.dtb boot/rk3399.dtb

3. get parameters from current kernel, modify loader.timestamp to a near future value (also remove uboot_logo if you don't use boot logo)
   

   代碼: 選擇全部

   dmesg | grep 'command line'

   
   

   [ 0.000000] Kernel command line: earlycon=uart8250,mmio32,0xff1a0000 swiotlb=1 console=ttyFIQ0 rootwait coherent_pool=1m root=/dev/mmcblk0p1 rw rootfstype=ext4 data=/dev/mmcblk0p2 consoleblank=0 storagemedia=sd uboot_logo=0x02000000@0xf5c00000 loader.timestamp=2019-08-06_19:08:49 SecureBootCheckOk=0

4. create extlinux.conf
   

   代碼: 選擇全部

   mkdir boot/extlinux
   nano boot/extlinux/extlinux.conf

   
   

   label gentoo-kernel-4.19.81
   kernel /Image
   fdt /rk3399.dtb
   append earlycon=uart8250,mmio32,0xff1a0000 swiotlb=1 rootwait coherent_pool=1m root=/dev/sda2 rw rootfstype=ext4 consoleblank=0 loader.timestamp=2019-10-31_23:59:00

5. Generate fatfs boot partition
   

   代碼: 選擇全部

   dd if=/dev/zero of=boot.img bs=1M count=32
   mkfs.fat boot.img
   mkdir tmp
   mount boot.img tmp/
   cp -r boot/* tmp/
   umount tmp

write to SD

1. manage partition (mine is /dev/sdb)
   

   代碼: 選擇全部

   fdisk /dev/sdb

   
   *delete ALL partitions if SD been used
   

   Command (m for help): n
   

   Partition type
   p primary (0 primary, 0 extended, 4 free)
   e extended (container for logical partitions)
   Select (default p): p

   
   

   Partition number (1-4, default 1): 1

   
   

   First sector (2048-15278079, default 2048): 196608

   
   

   Last sector, +/-sectors or +/-size{K,M,G,T,P} (196608-15278079, default 15278079): [enter]
   
   Created a new partition 1 of type 'Linux' and of size 7.2 GiB
   
   

   
   

   Command (m for help): p
   

   Disk /dev/sdb: 7.3 GiB, 7822376960 bytes, 15278080 sectors
   Disk model: Flash Reader
   Units: sectors of 1 * 512 = 512 bytes
   Sector size (logical/physical): 512 bytes / 512 bytes
   I/O size (minimum/optimal): 512 bytes / 512 bytes
   Disklabel type: dos
   Disk identifier: 0xefeea434
   
   Device Boot Start End Sectors Size Id Type
   /dev/sdb1 196608 15278079 15081472 7.2G 83 Linux
   
   

   
   

   Command (m for help): wq
   

   The partition table has been altered.
   Calling ioctl() to re-read partition table.
   Syncing disks.
   

   
   

2. 代碼: 選擇全部

   dd if=/usr/src/u-boot/idbloader.img of=/dev/sdb seek=64
   dd if=/usr/src/u-boot/u-boot.itb of=/dev/sdb seek=16384
   dd if=/usr/src/linux/boot.img of=/dev/sdb seek=32768
   sync

[PoP]

Updates for PineBookPro, use rk-3399 with minor hardware differences from friendlyarm, namely WiFi/BT chip. Firmware can be found on Armbian Rockpi-4b image.

Gentoo on Pinebook Pro - super stable but seems not for friendlyarm w/sata HAT

U-Boot with direct NVMe boot support for eMMC/SPI Flash

Pinebook Pro SPI

[PoP]

Update on u-boot: gentoo repository have 2020.07 that works.