merged with upstream

6 files changed, 167 insertions, 338 deletions

diff --git a/docs/README.altbootmethods b/docs/README.altbootmethods
deleted file mode 100644
index beb6f0d..0000000
--- a/docs/README.altbootmethods
+++ /dev/null
@@ -1,113 +0,0 @@
-INDEX
------
-
-* Alternative boot methods (configs/releng)
-  * ISO in loopback mode
-  * ISO in memdisk mode
-  * Network booting (PXE) [first stage]
-    * DHCP + TFTP
-    * DHCP + HTTP
-    * HTTP/NFS/NBD [second stage]
-
-
-
-*** Alternative boot methods (configs/releng)
-
-ISO images names consist of: archlinux-<YYYY>.<MM>.<DD>-i686.iso
-
-Where:
-    <YYYY> Year
-    <MM> Month
-    <DD> Day
-
-
-** ISO in loopback mode.
-
-Note: Described method is for using with GRUB2.
-      GRUB2 is installed on target media and archlinux-<YYYY>.<MM>.<DD>-i686.iso
-      is at path <TARGET-PATH> on disk <D> and partition <P>,
-      where filesystem is labeled as <TARGET-FS-LABEL>.
-
-menuentry "Arch Linux (i686)" {
-    set isofile="/<TARGET-PATH>/archlinux-<YYYY>.<MM>.<DD>-i686.iso"
-    loopback loop (hd<D>,<P>)$isofile
-    linux (loop)/arch/boot/i686/vmlinuz img_label=<TARGET-FS-LABEL> img_loop=$isofile
-    initrd (loop)/arch/boot/i686/archiso.img
-}
-
-
-** ISO in memdisk mode.
-
-Note: Described method is for using with SYSLINUX. Anyway MEMDISK from SYSLINUX can work
-      with other bootloaders.
-      SYSLINUX is installed on target media and archlinux-<YYYY>.<MM>.<DD>-i686.iso
-      is at path <TARGET-PATH>.
-      On 32-bit systems, is needed to pass vmalloc=nnM to the kernel, where nn is the size
-      of the ISO image plus 64 MiB (or 128 MiB).
-
-
-LABEL arch_x32
-   LINUX memdisk
-   INITRD /<TARGET-PATH>/archlinux-<YYYY>.<MM>.<DD>-i686.iso
-   APPEND iso
-
-
-** Network booting (PXE).
-
-All ISOs are ready to act as PXE server, some manual steps are needed
-to setup the desired PXE boot mode.
-Alternatively it is possible to use an existing PXE server following the same logic.
-Note: Setup network first, adjust IP adresses, and respect all slashes "/".
-
-First stage is for loading kernel and initramfs via PXE, two methods described here:
-
-* DHCP + TFTP
-
-Note: All NIC firmwares should support this.
-
-# dnsmasq --port=0 \
-          --enable-tftp \
-          --tftp-root=/run/archiso/bootmnt \
-          --dhcp-range=192.168.0.2,192.168.0.254,86400 \
-          --dhcp-boot=/arch/boot/syslinux/lpxelinux.0 \
-          --dhcp-option-force=209,boot/syslinux/archiso.cfg \
-          --dhcp-option-force=210,/arch/
-
-* DHCP + HTTP
-
-Note: Not all NIC firmware supports HTTP and DNS (if domain name is used).
-      At least this works with iPXE and gPXE.
-
-# dnsmasq --port=0 \
-          --dhcp-range=192.168.0.2,192.168.0.254,86400 \
-          --dhcp-boot=http://192.168.0.7/arch/boot/syslinux/lpxelinux.0 \
-          --dhcp-option-force=209,boot/syslinux/archiso.cfg \
-          --dhcp-option-force=210,http://192.168.0.7/arch/
-
-
-Once the kernel is started from PXE, SquashFS files and other misc files
-inside "arch" directory must be loaded (second stage). One of the following
-methods can be used to serve the rest of live-medium.
-
-* HTTP
-
-# darkhttpd /run/archiso/bootmnt
-
-
-* NFS
-
-# echo "/run/archiso/bootmnt 192.168.0.*(ro,no_subtree_check,no_root_squash)" >> /etc/exports
-# systemctl start nfs-server.service
-
-
-* NBD
-
-Note: Adjust ARCH_201703 as needed.
-
-# cat << EOF > /tmp/nbd-server.conf
-[generic]
-[archiso]
-    readonly = true
-    exportname = /dev/disk/by-label/ARCH_201703
-EOF
-# nbd-server -C /tmp/nbd-server.conf
diff --git a/docs/README.bootparams b/docs/README.bootparams
deleted file mode 100644
index bcafc0e..0000000
--- a/docs/README.bootparams
+++ /dev/null
@@ -1,141 +0,0 @@
-INDEX
------
-
-* Boot parameters (initramfs stage)
-  * hooks/archiso
-  * hooks/archiso_pxe_common
-  * hooks/archiso_pxe_nbd
-  * hooks/archiso_pxe_http
-  * hooks/archiso_pxe_nfs
-  * hooks/archiso_loop_mnt
-
-* Boot parameters (configs/releng)
-  * scripts/choose-mirror
-
-
-*** Boot parameters (initramfs stage)
-
-** hooks/archiso
-
-* archisolabel=     Set the filesystem label where archiso files reside.
-                    Default: (unset)
-* archisodevice=    Set the device node where archiso medium is located.
-                    Default: "/dev/disk/by-label/${archisolabel}"
-* archisobasedir=   Set the base directory where all files reside.
-                    Default: "arch"
-* copytoram=        If set to "y" or just "copytoram" without arguments,
-                    all SquashFS are copied to "RAM".
-                    Default: (unset)
-* checksum=         If set to "y" or just "checksum" without arguments,
-                    performs a self-test of all files inside ${install_dir},
-                    and continue booting if ok.
-                    Default: (unset)
-* cow_label=        Set the filesystem label where COW file (for dm-snapshot)
-                    or upperdir/workdir files (for overlayfs) must be stored.
-                    Default: (unset)
-* cow_device=       Like cow_label= but using device node.
-                    Default: (unset) or "/dev/disk/by-label/${cow_label}"
-* cow_flags=        Set extra mount options, e.g. for btrfs subvolumes.
-                    Default: defaults
-* cow_directory=    Set a directory inside ${cow_device}.
-                    Default: "/persistent_${archisolabel}/${arch}"
-* cow_persistent=   Set if snapshot is persistent "P" or non-persistent "N".
-                    Only used for dm-snapshot mode, ignored for overlayfs.
-                    Default: "N" (if no ${cow_device} is used) otherwise "P".
-* cow_spacesize=    Set the size for COW space (tmpfs). Valid for both
-                    dm-snapshot and overlayfs mode.
-                    The argument is an integer and optional unit.
-                    Units are M,G (powers of 1024).
-                    Default: "256M"
-* cow_chunksize=    Set chunksize used for dm-snapshot. This is number
-                    of 512 byte blocks to write at once.
-                    Default: "8"
-* copytoram_size=   Set the size of tmpfs. This space is used for
-                    airootfs.sfs image if copytoram=y.
-                    Size is in bytes (suffix with "k", "m" and "g") or
-                    in percentage of available RAM.
-                    Default: "75%"
-* dm_snap_prefix=   Set a prefix for dm-snapshot node names.
-                    Only used for dm-snapshot mode, ignored for overlayfs.
-                    Default: "arch"
-* arch=             Force an architecture type (i686 | x86_64).
-                    Do not set it for normal operations.
-                    Default: (architecture of running kernel)
-
-
-** hooks/archiso_pxe_common
-
-* ip=               This parameter is setup automatically by PXELINUX
-                    when option "SYSAPPEND" is set to 1 or 2 in config.
-                    ip=<client-ip>:<boot-server-ip>:<gw-ip>:<netmask>
-                    Default: (set via PXE server)
-* BOOTIF=           This parameter is setup automatically by PXELINUX
-                    when option "SYSAPPEND" is set to 2 or 3 in config.
-                    BOOTIF=<hardware-address-of-boot-interface>
-                    Default: (set via PXELINUX)
-* copy_resolvconf=  Copy /etc/resolv.conf from initramfs to live-enviroment.
-                    Set to "n" to skip them.
-                    Default: "y"
-
-
-** hooks/archiso_pxe_nbd
-
-* archiso_nbd_name= Set NBD export name used by the server.
-                    Default: archiso
-* archiso_nbd_srv=  Set an IP address where NBD reside.
-                    If ${pxeserver} is used, PXE IP will be used.
-                    Default: (unset)
-
-
-** hooks/archiso_pxe_http
-
-* archiso_http_srv= Set an HTTP URL (must end with /) where ${archisobasedir}
-                    is found with all *.sfs files.
-                    In the IP/domain part if ${pxeserver} is used, use PXE IP.
-                    Default: (unset)
-* archiso_http_spc= Set the size of tmpfs where *.sfs files are downloaded.
-                    Default: "75%"
-
-
-** hooks/archiso_pxe_nfs
-
-* archiso_nfs_srv=  Set the NFS-IP:/path of the server
-                    In the IP part if ${pxeserver} is used, PXE IP will be used.
-                    Default: (unset)
-* archiso_nfs_opt=  Set NFS mount options separated by comma.
-                    Default: (unset, see below)
-                    These are the implicit options:
-                      port            = as given by server portmap daemon
-                      rsize           = 1024
-                      wsize           = 1024
-                      timeo           = 7
-                      retrans         = 3
-                      acregmin        = 3
-                      acregmax        = 60
-                      acdirmin        = 30
-                      acdirmax        = 60
-                      flags           = hard, nointr, noposix, cto, ac
-
-
-** hooks/archiso_loop_mnt
-
-* img_label=        Set the filesystem label where archiso-image.iso.
-                    Default: (unset)
-* img_dev=          Device where archiso-image.iso reside.
-                    Default: (unset) or "/dev/disk/by-label/${img_label}"
-* img_flags=        Set extra mount options, e.g. for btrfs subvolumes.
-                    Default: defaults
-* img_loop=         Full path where archiso-image.iso is located on ${img_dev}
-                    Default: (unset)
-
-
-
-*** Boot parameters (configs/releng)
-
-** scripts/choose-mirror
-
-* mirror=           Takes a mirror URL and creates a new mirrorlist.
-                    When setting mirror=auto, the mirror is taken from
-                    archiso_http_srv= in order to keep using the mirror
-                    selected in the netboot menu.
-                    Default: (unset)
diff --git a/docs/README.build b/docs/README.build
deleted file mode 100644
index efa78d0..0000000
--- a/docs/README.build
+++ /dev/null
@@ -1,68 +0,0 @@
-INDEX
------
-
-* Build requirements
-* Building the most basic Arch Linux live media. (configs/baseline)
-* Building official Arch Linux live media. (configs/releng)
-
-
-
-*** Build requirements
-
-** For mkarchiso script needs these packages (build host):
- + arch-install-scripts    for pacstrap/arch-chroot
- + edk2-shell              for UEFI shell
- + squashfs-tools          for mksquashfs
- + libisoburn              for xorriso
- + btrfs-progs             for mkfs.btrfs (optional)
-
-** For configs/releng build.sh needs theses packages (build host):
- + dosfstools              for mkfs.fat
- + lynx                    for fetching the latest installation guide
-
-** For these hooks needs these packages (on target airootfs)
-* archiso
- + (none)
-* archiso_loop_mnt
- + (none)
-* archiso_pxe_common
- + mkinitcpio-nfs-utils    for ipconfig
-* archiso_pxe_nbd
- + nbd                     for nbd-client
-* archiso_pxe_http
- + curl                    for curl
-* archiso_pxe_nfs
- + mkinitcpio-nfs-utils    for nfsmount
-* archiso_shutdown
- + (none)
-
-
-*** Building the most basic Arch Linux live media. (configs/baseline)
-
-* Install needed packages.
-  # pacman -S git make arch-install-scripts squashfs-tools libisoburn --needed
-
-* Install archiso.
-  # git clone git://projects.archlinux.org/archiso.git
-  # make -C archiso install
-
-* Build a basic iso.
-  # /usr/share/archiso/configs/baseline/build.sh
-
-Note: If you want to customize, just see the configs/releng directory which is
-used to build official images with much more things.
-
-
-*** Building official Arch Linux live media. (configs/releng)
-
-* Install needed packages.
-  # pacman -S git make arch-install-scripts squashfs-tools libisoburn dosfstools lynx --needed
-
-* Install archiso.
-  # git clone git://projects.archlinux.org/archiso.git
-  # make -C archiso install
-
-* Build them!
-  # /usr/share/archiso/configs/releng/build.sh
-
-Note: See build.sh -h for more options. This only runs on x86_64.
diff --git a/docs/README.knownissues b/docs/README.knownissues
deleted file mode 100644
index 3a94764..0000000
--- a/docs/README.knownissues
+++ /dev/null
@@ -1,12 +0,0 @@
-*** Know issues
-
-** (1) On shutdown lots of messages from systemd like:
-
-    "Could not unmount /run/archiso/<ABC>: Device or resource busy"
-    "Could not delete loopback /dev/loop<N>: Device or resource busy"
-    This is not a real issue since, all mounted filesystem, loopback devices
-    and device mapper devices made by archiso will be "free" on "shutdown tmpfs"
-    (A.K.A deinitramfs), build at initramfs by [archiso_shutdown] initcpio hook.
-    Proper shutdown is mostly important when persistent is used.
-
-
diff --git a/docs/README.profile.rst b/docs/README.profile.rst
new file mode 100644
index 0000000..c93228d
--- /dev/null
+++ b/docs/README.profile.rst
@@ -0,0 +1,163 @@
+=======
+profile
+=======
+
+An archiso profile consists of several configuration files and a directory for files to be added to the resulting image.
+
+.. code:: plaintext
+
+   profile/
+   ├── airootfs/
+   ├── efiboot/
+   ├── syslinux/
+   ├── bootstrap_packages.arch
+   ├── packages.arch
+   ├── pacman.conf
+   └── profiledef.sh
+
+The required files and directories are explained in the following sections.
+
+profiledef.sh
+=============
+
+This file describes several attributes of the resulting image and is a place for customization to the general behavior
+of the image.
+
+The image file is constructed from some of the variables in ``profiledef.sh``: ``<iso_name>-<iso_version>-<arch>.iso``
+(e.g. ``archlinux-202010-x86_64.iso``).
+
+* ``iso_name``: The first part of the name of the resulting image (defaults to ``mkarchiso``)
+* ``iso_label``: The ISO's volume label (defaults to ``MKARCHISO``)
+* ``iso_publisher``: A free-form string that states the publisher of the resulting image (defaults to ``mkarchiso``)
+* ``iso_application``: A free-form string that states the application (i.e. its use-case) of the resulting image (defaults
+  to ``mkarchiso iso``)
+* ``iso_version``: A string that states the version of the resulting image (defaults to ``""``)
+* ``install_dir``: A string (maximum eight characters long, which **must** consist of ``[a-z0-9]``) that states the
+  directory on the resulting image into which all files will be installed (defaults to ``mkarchiso``)
+* ``buildmodes``: An optional list of strings, that state the build modes that the profile uses. Only the following are
+  understood:
+
+  - ``bootstrap``: Build a compressed file containing a minimal system to bootstrap from
+  - ``iso``: Build a bootable ISO image (implicit default, if no ``buildmodes`` are set)
+  - ``netboot``: Build artifacts required for netboot using iPXE
+* ``bootmodes``: A list of strings, that state the supported boot modes of the resulting image. Only the following are
+  understood:
+
+  - ``bios.syslinux.mbr``: Syslinux for x86 BIOS booting from a disk
+  - ``bios.syslinux.eltorito``: Syslinux for x86 BIOS booting from an optical disc
+  - ``uefi-x64.systemd-boot.esp``: systemd-boot for x86_64 UEFI booting from a disk
+  - ``uefi-x64.systemd-boot.eltorito``: systemd-boot for x86_64 UEFI booting from an optical disc
+    Note that BIOS El Torito boot mode must always be listed before UEFI El Torito boot mode.
+* ``arch``: The architecture (e.g. ``x86_64``) to build the image for. This is also used to resolve the name of the packages
+  file (e.g. ``packages.x86_64``)
+* ``pacman_conf``: The ``pacman.conf`` to use to install packages to the work directory when creating the image (defaults to
+  the host's ``/etc/pacman.conf``)
+* ``airootfs_image_type``: The image type to create. The following options are understood (defaults to ``squashfs``):
+
+  - ``squashfs``: Create a squashfs image directly from the airootfs work directory
+  - ``ext4+squashfs``: Create an ext4 partition, copy the airootfs work directory to it and create a squashfs image from it
+  - ``erofs``: Create an EROFS image for the airootfs work directory
+* ``airootfs_image_tool_options``: An array of options to pass to the tool to create the airootfs image. ``mksquashfs`` and
+  ``mkfs.erofs`` are supported. See ``mksquashfs --help`` or ``mkfs.erofs --help`` for all possible options
+* ``file_permissions``: An associative array that lists files and/or directories who need specific ownership or
+  permissions. The array's keys contain the path and the value is a colon separated list of owner UID, owner GID and
+  access mode. E.g. ``file_permissions=(["/etc/shadow"]="0:0:400")``. When directories are listed with a trailing backslash (``/``) **all** files and directories contained within the listed directory will have the same owner UID, owner GID, and access mode applied recursively.
+
+bootstrap_packages.arch
+=======================
+
+All packages to be installed into the environment of a bootstrap image have to be listed in an architecture specific
+file (e.g. ``bootstrap_packages.x86_64``), which resides top-level in the profile.
+
+Packages have to be listed one per line. Lines starting with a ``#`` and blank lines are ignored.
+
+This file is required when generating bootstrap images using the ``bootstrap`` build mode.
+
+packages.arch
+=============
+
+All packages to be installed into the environment of an ISO image have to be listed in an architecture specific file
+(e.g. ``packages.x86_64``), which resides top-level in the profile.
+
+Packages have to be listed one per line. Lines starting with a ``#`` and blank lines are ignored.
+
+  .. note::
+
+    The **mkinitcpio** and **mkinitcpio-archiso** packages are mandatory (see `#30
+    <https://gitlab.archlinux.org/archlinux/archiso/-/issues/30>`_).
+
+This file is required when generating ISO images using the ``iso`` or ``netboot`` build modes.
+
+pacman.conf
+===========
+
+A configuration for pacman is required per profile.
+
+Some configuration options will not be used or will be modified:
+
+* ``CacheDir``: the profile's option is **only** used if it is not the default (i.e. ``/var/cache/pacman/pkg``) and if it is
+  not the same as the system's option. In all other cases the system's pacman cache is used.
+* ``HookDir``: it is **always** set to the ``/etc/pacman.d/hooks`` directory in the work directory's airootfs to allow
+  modification via the profile and ensure interoparability with hosts using dracut (see `#73
+  <https://gitlab.archlinux.org/archlinux/archiso/-/issues/73>`_)
+* ``RootDir``: it is **always** removed, as setting it explicitely otherwise refers to the host's root filesystem (see
+  ``man 8 pacman`` for further information on the ``-r`` option used by ``pacstrap``)
+* ``LogFile``: it is **always** removed, as setting it explicitely otherwise refers to the host's pacman log file (see
+  ``man 8 pacman`` for further information on the ``-r`` option used by ``pacstrap``)
+* ``DBPath``: it is **always** removed, as setting it explicitely otherwise refers to the host's pacman database (see
+  ``man 8 pacman`` for further information on the ``-r`` option used by ``pacstrap``)
+
+airootfs
+========
+
+This optional directory may contain files and directories that will be copied to the work directory of the resulting
+image's root filesystem.
+The files are copied before packages are being installed to work directory location.
+Ownership and permissions of files and directories from the profile's ``airootfs`` directory are not preserved. The mode
+will be ``644`` for files and ``755`` for directories, all of them will be owned by root. To set custom ownership and/or
+permissions, use ``file_permissions`` in ``profiledef.sh``.
+
+With this overlay structure it is possible to e.g. create users and set passwords for them, by providing
+``airootfs/etc/passwd``, ``airootfs/etc/shadow``, ``airootfs/etc/gshadow`` (see ``man 5 passwd``, ``man 5 shadow`` and ``man 5 gshadow`` respectively).
+If user home directories exist in the profile's ``airootfs``, their ownership and (and top-level) permissions will be
+altered according to the provided information in the password file.
+
+Boot loader configuration
+=========================
+
+A profile may contain configuration for several boot loaders. These reside in specific top-level directories, which are
+explained in the following subsections.
+
+The following *custom template identifiers* are understood and will be replaced according to the assignments of the
+respective variables in ``profiledef.sh``:
+
+* ``%ARCHISO_LABEL%``: Set this using the ``iso_label`` variable in ``profiledef.sh``.
+* ``%INSTALL_DIR%``: Set this using the ``iso_label`` variable in ``profiledef.sh``.
+* ``%ARCH%``: Set this using the ``arch`` variable in ``profiledef.sh``.
+
+
+efiboot
+-------
+
+This directory is mandatory when the ``uefi-x64.systemd-boot.esp`` or ``uefi-x64.systemd-boot.eltorito`` bootmodes are
+selected in ``profiledef.sh``. It contains configuration for `systemd-boot
+<https://www.freedesktop.org/wiki/Software/systemd/systemd-boot/>`_.
+
+  .. note::
+
+    The directory is a top-level representation of the systemd-boot configuration directories and files found in the
+    root of an EFI system partition.
+
+The *custom template identifiers* are **only** understood in the boot loader entry `.conf` files (i.e. **not** in
+``loader.conf``).
+
+syslinux
+--------
+
+This directory is mandatory when the ``bios.syslinux.mbr`` or the ``bios.syslinux.eltorito`` bootmodes are selected in
+``profiledef.sh``.
+It contains configuration files for `syslinux <https://wiki.syslinux.org/wiki/index.php?title=SYSLINUX>`_ or `isolinux
+<https://wiki.syslinux.org/wiki/index.php?title=ISOLINUX>`_ , or `pxelinux
+<https://wiki.syslinux.org/wiki/index.php?title=PXELINUX>`_ used in the resuling image.
+
+The *custom template identifiers* are understood in all `.cfg` files in this directory.
diff --git a/docs/README.transfer b/docs/README.transfer
index 2bb2b3d..aed5f92 100644
--- a/docs/README.transfer
+++ b/docs/README.transfer
@@ -13,7 +13,7 @@ INDEX
 
 *** Transfer ISO image to target medium (configs/releng)
 
-ISO images names consist of: archlinux-<YYYY>.<MM>.<DD>-i686.iso
+ISO images names consist of: archlinux32-<YYYY>.<MM>.<DD>-i686.iso
 
 Where:
     <YYYY> Year
@@ -34,7 +34,7 @@ Nomeclature:
 
 
 1) Write it directly using your favorite recording program.
-# cdrecord dev=<B>,<T>,<L> -dao archlinux-<YYYY>.<MM>.<DD>-i686.iso
+# cdrecord dev=<B>,<T>,<L> -dao archlinux32-<YYYY>.<MM>.<DD>-i686.iso
 
 
 ** To -> USB Flash Drive (USB-key) / Memory card (SD) /
@@ -53,8 +53,8 @@ Nomeclature:
                 (example: /dev/sdx1)
 <MNT-TARGET-N>: Mount point path where <DEV-TARGET-N> is mounted
                 (example: /mnt/sdx/1)
-<ISO-SOURCE>:   Path to the ISO file archlinux-<YYYY>.<MM>.<DD>-i686.iso
-                (example: ~/archlinux-2017.03.01-i686.iso)
+<ISO-SOURCE>:   Path to the ISO file archlinux32-<YYYY>.<MM>.<DD>-i686.iso
+                (example: ~/archlinux32-2017.03.01-i686.iso)
 <FS-LABEL>:     Represents the filesystem label of the <ISO-SOURCE>
                 (example: ARCH_201703)