Mission ------- Build a crosstool-ng toolchain for i486. Use makepkg with patched PKGBUILDs for cross-compilation. We cannot run tests this way, as we don't assume the i486 is running on x64_64 (which it does, but for the sake of the general usage of the approach we MUST assume it doesn't!). Packages built for the chroot must be also accessible to the cross-compiler (for header files, tools and libraries). For this we unpack all generated chroot packages in the sysroot of the toolchain. x-tools and sysroot: the cross-compiler (cross-ng) and the installed packages (using bsdtar) in the sysroot of the toolchain. i486-root destination root, being the boot image of the temporary system for i486. The goal is to get to a self-contained system with as few packages as possible which allows a basic i486 to be built. This requires at least some development tools, pacman and some base system to be cross-compiled. We also want basic network connetivity, so we can use the i486 VMs or real i486 as build slaves. Also, having a minimal bootable ISO for the i486 system (or even a set of floppies or PXE boot via a boot floppy would be very useful). We cannot have kernel modules in a ramdisk or systemd for this basic system because they interfere heavily with memory constains on a 486! So, we build up to an openssh server. On top, the full Archlinux i486 universe can then be built (either with a i486 chroot or via a i486 master/ distcc slaves setup or a distributed i486 cluster setup). There is another caveat: we are patching the PKGBUILDs to do proper cross-compilation (--with-host, --with-target). Still we try to stay close to the original PKGBUILDs. References ---------- https://archlinuxarm.org/wiki/Distcc_Cross-Compiling https://wiki.archlinux.org/index.php/Cross-compiling_tools_package_guidelines http://mgalgs.github.io/2011/12/08/creating-arch-linux-packages-by-hand.html http://clfs.org/view/svn/ppc/chroot/before-chroot.html http://www.linuxfromscratch.org/clfs/ https://www.dotslashlinux.com/2017/04/29/booting-the-linux-kernel-without-an-initrd-initramfs/ https://rubenerd.com/sata-on-qemu/ https://archlinuxarm.org/forum/viewtopic.php?f=57&t=6163 https://dustymabe.com/2012/12/15/mounting-a-partition-within-a-disk-image/ http://preshing.com/20141119/how-to-build-a-gcc-cross-compiler/ http://wiki.osdev.org/GCC_Cross-Compiler http://www.linuxfromscratch.org/lfs/view/6.2/chapter05/adjusting.html https://wiki.debian.org/FakeRoot https://archlinuxarm.org/forum/viewtopic.php?f=57&t=6163 https://www.phenix.bnl.gov/~purschke/RescueCD/ https://github.com/ivandavidov/minimal-linux-script https://askubuntu.com/questions/109413/how-do-i-use-overlayfs https://how-to-build-for-arm.wikispaces.com/ https://arsv.github.io/perl-cross/ http://www.ibb.net/~anne/keyboard.html Recipe ------ ############## # PREPARATIONS ############## # Prepare the host. We use Archlinux 64-bit as we don't fully trust the # 32-bit toolchain in Archlinux32 (yet). And a 64-bit based toolchain is # more usable anyway nowadays. # Archlinux with base and base-devel (2018.01.01-x86_64.iso). # OpenSSH and editor (joe), grub. # Install necessary tools ./install_host.sh # Prepare the cross-compiler for the destination platform, in our # case i486. # # configuration of crosstools-ng: # # Target architecture i486 # Bitness 32-bit # Architecture level i486 # Emit assembly for CPU i486 # Toolchain bug URL: https://bugs.archlinux32.org # Type: Cross # Build System: empty (was: x86_64-pc-linux-gnu) # No NLS # Target OS linux # version of linux 4.13.9 # binutils 2.29.1 # select ld, gold as linkers # C-library: glibc # gcc 7.2.0 # no libmpx # Don't forget to enable C++! ct-ng menuconfig cp .config ct-ng.config # build the toolchain, results in a 486 toolchain in /home/cross/x-tools ./build_cross.sh # prepare repo where we build stage 1 with the cross-compiler su cross ./prepare_stage1_repo.sh ######### # STAGE 1 ######### # Build stage1 in $STAGE1_BUILD with the cross-compiler and modified # PKGBUILDs and patches into $STAGE1_CHROOT. # If necessary to fullfill the dependencies of makepkg, we create phantom # packages by hand without makepkg with files from the toolchain's sysroot. # So, we build gcc-libs for compiler libraries (C and C++) and the glibc # we will NOT build them using the cross-compiler as packages but later # at the end of phase 1. This ensures glibc and gcc-libs are around as # dependency for makepkg all the time, if not the original one, so close # ones fitting to the crosstoolchain. # In this stage we are not allowed to have loops when building, the build # tree must be a real tree. # The goal is to get a booting system of the target architecture (for instance # to be installed on real hardware for a build cluster). su cross ./create_gcc-lib_shim.sh su cross ./create_glibc_shim.sh # take pre-computed cert stores from an existing machine as the # dependencies to build the package are just too many su cross ./create_ca-certificates-utils_shim.sh # Build stage 1 and install it into a chroot and into sysroot ./build_stage1.sh # Build stage 1 ISO (mainly for testing, installing from this CD is # difficult and more manually as with pacstrap) su cross ./create_cdrom.sh # build stage 1 hard disk image (to be installed directly as the # stage 2 build system on the target architecture) su cross ./create_hdd.sh ######### # STAGE 2 ######### # Assuming stage1 of the system is installed on the target architecture # (virtual or a real machine), we have a key-based SSH connection to # the machine and can build packages there. # Build stage2 in $STAGE2_BUILD with the tools on the stage1 system # and modified PKGBUILDs and patches into the target system (replacing # stage 1 packages). Stage 1 serves now the same function as the sysroot # during cross-compilation of stage 1. # The goal is to get a self-hosting system on the target architecture, # not necesseraly containing all of base and base-devel, but enough to # bootstrap itself on the target architecture. su cross ./prepare_stage2_repo.sh # Build stage 2 on the target architecture and install it onto the # stage 1 system. Resulting artifacts get stored also back # in $STAGE2_BUILD. # Most things compile just fine with 64 MB RAM, binutils ld gold needs more # (we gave it 512 MB) ./build_stage2.sh ######### # STAGE 3 ######### # Use stage 2 to build base and base-devel. In this phase we don't care # yet much about testing the packages (as this draws in still too many # dependencies). But we try to be as close to base + base-devel as possible # with minimal patching of PKGBUILDs. # Also ommit most documentation building as it has heavy dependencies. # We break cycles by making the simpler dependency vanish in the more # complex packages: # - libsasl without openldap # - libgcrypt without libsecret # - ldns unbound dnssec-anchors are all in a cyrcle # - ca-certificates-mozilla needs gyp which need ninja which has # a download location only verifiable by the very same certificates :-) # - dbus without systemd (because of a cyrcle), same goes for cryptsetup # - glib2 without shared-mime-info # some packages are simply to hard or important to build: # - licenses: needs links, we don't care for stage3 as they will not # go public in the end # - libsecret, calls gtkdocize in autogen.sh # - systemd-sysvcompat: should be optional, I don't see why this should # be in base # - nss: has big issues with hand-crafted assembler optimizations all # over the place drawing in SSE2 as a requirement, used for # building ca-certificates-mozilla only at the moment # systemd: goal is to have a libsystemd we can link against, but stage 3 # will not have a running systemd init system! su cross ./prepare_stage3_repo.sh # Build stage 3 on the target architecture and install it onto the # stage 2 system. Resulting artifacts get stored also back # in $STAGE3_BUILD. # Most things compile just fine with 64 MB RAM, binutils ld gold needs more # (we gave it 512 MB). s-nail needs much more (we gave it 2 GB). ./build_stage3.sh ######### # STAGE 4 ######### # Use stage 3 to build base and base-devel again. In this phase we try to # use vanilla PKGBUILDs as much as possible, also building documentation # and do all the testing. # We don't go to another build system for now till we can build the base # system without patched PKGBUILDs (at least not where not needed). # We also try to get a running systemd in this stage. # Ommitting the following in this stage: # - bootstrapping issues: # - gcc-ada: needs a running ada to build # - java-runtime: nedds another java-runtime to build # - vala: needs another vala to build su cross ./prepare_stage4_repo.sh # Build stage 4 on the target architecture and install it onto the # stage 3 system. Resulting artifacts get stored also back # in $STAGE4_BUILD. ./build_stage4.sh # Some software doesn't build # - nss: but we need it only to produce a mozilla-certificates-ca, so # can also take the one from the host # # Some testing is impossible to do in stage4: # - some python modules are virtually untestable due to excessive dependencies # e.g, python-packaging # - glibc tests run out of memory