View source for OnixOS:Building From Scratch

= Building From Scratch =

This document covers the complete build lifecycle of the OnixOS Build System — from environment setup and package repository builds to ISO creation, syncing, Docker-based builds, and live build monitoring.

__TOC__

== Prerequisites ==

The build system runs on '''Arch Linux''' (or an Arch-based environment). The following packages are required for ISO builds and are installed automatically by <code>build.py</code>:

{| class="wikitable"
! Package !! Purpose
|-
| <code>archiso</code> || Core ISO creation tool (<code>mkarchiso</code>)
|-
| <code>base-devel</code> || Compiler toolchain
|-
| <code>devtools</code> || <code>makechrootpkg</code>, <code>mkarchroot</code> for isolated chroot builds
|-
| <code>arch-install-scripts</code> || <code>pacstrap</code>, <code>arch-chroot</code>
|-
| <code>squashfs-tools</code> || SquashFS filesystem creation
|-
| <code>dosfstools</code> || FAT filesystem creation
|-
| <code>syslinux</code> || Bootloader
|-
| <code>git</code> / <code>git-lfs</code> || Source fetching
|-
| <code>rsync</code> || File synchronisation
|-
| <code>go</code> || Go language toolchain
|-
| <code>nasm</code> || Assembler
|-
| <code>ninja</code> || Build system
|-
| <code>gperf</code> || Hash function generator
|-
| <code>imagemagick</code> || Image processing
|-
| <code>python-sphinx</code> / <code>xmlto</code> || Documentation toolchain
|-
| <code>gnome-shell</code> / <code>qtcreator</code> || Desktop environment build deps
|-
| <code>gnupg</code> || GPG package signing
|-
| <code>qemu-user-static</code> || Cross-architecture emulation (ARM builds on x86_64 host)
|-
| <code>binfmt-support</code> || Binary format registration for qemu-user-static
|-
| <code>sudo</code> || Privilege escalation
|}

For '''repository builds''' the system additionally uses <code>makepkg</code> (included in <code>base-devel</code>) and the AUR snapshot endpoint:

 https://aur.archlinux.org/cgit/aur.git/snapshot

== Project Structure ==

<pre>
onix-build-system/
├── build.py                  # Main build entry point
├── clean.py                  # Cleans output/ and mounts
├── watcher.py                # Terminal build status monitor
├── qemu.py                   # Launch built ISO in QEMU
├── Makefile                  # Convenience make targets
├── buildkit-docker/
│   ├── Dockerfile            # Build container image
│   ├── entrypoint.sh         # Docker entrypoint (chroot bootstrap)
│   ├── conf/
│   │   └── makepkg.conf      # Generic x86-64 baseline CFLAGS
│   ├── docker-compose.yml        # Dev shell container
│   ├── docker-compose.iso.yml    # Automated ISO build container
│   └── docker-compose.repos.yml  # Automated repo build container
├── onixos/
│   ├── archconfig.py         # Multiarch configuration (CFLAGS, chroot, qemu)
│   ├── iso.py                # ISO build logic (mkarchiso wrapper)
│   ├── pkgbuild.py           # PKGBUILD / .SRCINFO parser
│   ├── proc.py               # Subprocess wrapper with logging
│   └── repository.py         # Repository build logic (chroot, sanitize, sign)
├── sources/
│   ├── packages/             # Local package sources (PKGBUILD)
│   ├── profiles/             # archiso profiles (core, gnome, xfce, icewm, security, kurumsal, odesk)
│   ├── packages.list         # Local packages to build
│   ├── aur.list              # AUR packages to build
│   ├── profiles.list         # ISO profiles to build
│   ├── signing.conf          # GPG signing configuration (example)
│   ├── sync/                 # Repo upload config
│   │   ├── sourceforge.conf
│   │   └── local.conf
│   └── isosync/              # ISO upload config
│       ├── sourceforge.conf
│       └── local.conf
├── output/                   # All build artefacts (generated, per-arch)
└── logs/                     # Build logs (generated, per-arch)
</pre>

== Build Commands ==

All commands must be run from the '''repository root'''.

=== Full Build ===

Builds all repositories '''and''' all ISO profiles in sequence:

 sudo python build.py

=== Repository Build ===

Builds the <code>onix-base</code> (local packages) and <code>onix-aur</code> (AUR packages) repositories for '''all supported architectures''' by default:

 python build.py repos

To build for a '''specific architecture''':

 python build.py repos --arch=x86_64_v3

The build process per architecture:
# Bootstraps a clean chroot with <code>base-devel</code> (isolated from host system).
# Generates architecture-specific <code>makepkg.conf</code> inside the chroot (generic CFLAGS baseline).
# Sanitizes PKGBUILD files to prevent instruction leakage (<code>-march=native</code> → architecture baseline).
# Builds packages using <code>makechrootpkg -c -r &lt;chroot&gt;</code> (host isolation, dependency resolution inside chroot).
# Signs each <code>.pkg.tar.zst</code> with GPG detach-signature (if signing key is configured).
# Generates repository database with <code>repo-add</code>.

'''Supported architectures:'''

{| class="wikitable"
! Architecture !! CFLAGS Baseline !! Description !! QEMU Required
|-
| <code>x86_64</code> || <code>-march=x86-64</code> || Generic x86-64 (2004+, all CPUs) || No
|-
| <code>x86_64_v2</code> || <code>-march=x86-64-v2</code> || SSE4.2, POPCNT (2008+: Nehalem, Bulldozer) || No
|-
| <code>x86_64_v3</code> || <code>-march=x86-64-v3</code> || AVX, AVX2, BMI1/2, FMA (2015+: Haswell, Zen) || No
|-
| <code>i686</code> || <code>-march=i686</code> || 32-bit x86 (Pentium Pro+) || No
|-
| <code>aarch64</code> || <code>-march=armv8-a</code> || 64-bit ARM (ARMv8-A) || Yes
|-
| <code>armv7h</code> || <code>-march=armv7-a -mfloat-abi=hard</code> || 32-bit ARM hard-float (ARMv7-A) || Yes
|-
| <code>armv6h</code> || <code>-march=armv6 -mfloat-abi=hard</code> || 32-bit ARM hard-float (ARMv6, Raspberry Pi 1) || Yes
|}

{{Note|ARM architectures require <code>qemu-user-static</code> and <code>binfmt-support</code> for cross-architecture emulation. The Docker image includes these packages.}}

=== ISO Build ===

Builds ISO images for '''all ISO-supported architectures''' (x86_64 and aarch64) and every profile listed in <code>sources/profiles.list</code>:

 sudo python build.py iso

To build for a '''specific architecture''':

 sudo python build.py iso --arch=aarch64

To build '''without chroot isolation''' (direct <code>mkarchiso</code> invocation, useful for debugging):

 sudo python build.py iso --no-chroot

The build process per profile per architecture:
# Installs ISO build dependencies via <code>pacman</code>.
# Creates a '''staging copy''' of the profile with architecture-specific overrides:
#* Rewrites <code>profiledef.sh</code> with the target architecture and boot modes.
#* Selects <code>packages.&lt;arch&gt;</code> as the package list.
#* Generates architecture-specific <code>pacman.conf</code> (with <code>$arch</code> repo URLs).
#* Removes <code>syslinux/</code> for aarch64 (UEFI-only).
# When '''chroot isolation''' is enabled (default):
## Bootstraps a clean chroot with <code>base-devel</code>, <code>archiso</code>, <code>squashfs-tools</code>, <code>dosfstools</code>, and <code>arch-install-scripts</code>.
## Bind-mounts <code>/proc</code>, <code>/sys</code>, <code>/dev</code>, <code>/dev/pts</code>, source profile, work directory, and output directory into the chroot.
## Runs <code>mkarchiso</code> inside the chroot with sanitized environment variables.
## Unmounts all bind mounts (deepest first) on exit, error, or signal (SIGINT/SIGTERM).
# When '''chroot isolation''' is disabled (<code>--no-chroot</code>):
## Unmounts any stale mounts from a previous failed build.
## Cleans the work directory.
## Runs <code>mkarchiso</code> directly on the host.
# Writes the finished <code>.iso</code> file to <code>output/iso/&lt;profile&gt;/&lt;arch&gt;/output/</code>.

'''ISO-supported architectures:'''

{| class="wikitable"
! Architecture !! Boot Modes !! Syslinux !! Notes
|-
| <code>x86_64</code> || BIOS (syslinux) + UEFI (systemd-boot) || Yes || Standard x86-64 ISO
|-
| <code>aarch64</code> || UEFI only (systemd-boot) || No || ARM64, excludes x86-only packages (microcode, video drivers)
|}

{{Note|The aarch64 ISO excludes packages that are x86-specific: <code>amd-ucode</code>, <code>intel-ucode</code>, <code>memtest86+</code>, <code>syslinux</code>, <code>broadcom-wl</code>, <code>b43-fwcutter</code>, x86-only video drivers (<code>xf86-video-*</code>), and <code>edk2-shell</code>.}}

=== Upload / Sync ===

Uploads built repositories to the configured remote server(s):

 python build.py upload

Uploads built ISO images:

 python build.py iso-sync

Both commands read from the relevant <code>*.conf</code> files under <code>sources/sync/</code> and <code>sources/isosync/</code> respectively (see [[#Sync Configuration|Sync Configuration]]).

== Make Targets ==

=== Core Build Targets ===

<pre>
make            # clean → repos → iso (full build, all architectures)
make clean      # Remove output artefacts and unmount
make repos      # Repository build for all architectures
make repos ARCH=x86_64_v3  # Repository build for specific architecture
make iso        # ISO build for all architectures (x86_64 + aarch64)
make iso ARCH=x86_64       # ISO build for specific architecture
make iso-all    # ISO build for all architectures (explicit)
make iso-nochroot          # ISO build without chroot isolation
make iso-nochroot ARCH=aarch64  # ISO build without chroot for specific architecture
</pre>

=== Chroot Management ===

<pre>
make chroot_bootstrap              # Bootstrap chroots for all architectures
make chroot_bootstrap ARCH=x86_64  # Bootstrap chroot for specific architecture
make chroot_clean                  # Remove all chroots
make chroot_clean ARCH=x86_64      # Remove chroot for specific architecture
make chroot_shell ARCH=x86_64      # Spawn shell inside chroot (requires ONIX_CHROOT)
</pre>

=== Testing ===

<pre>
make test                    # Run all tests for all architectures
make test ARCH=x86_64_v3     # Run all tests for specific architecture
make test-march              # Verify -march baseline in compiled binaries
make test-sign               # Verify GPG signatures
make test-deps               # Check for missing shared-object dependencies
make test-sanitize           # Verify PKGBUILD sanitizer rewrites flags
make test-smoke              # End-to-end smoke test (build single package)
</pre>

=== Docker Targets ===

<pre>
make docker_build            # Build the Docker image
make docker_publish          # Push image to registry (olproject/buildkit)
make docker_pull             # Pull image from registry
make docker_shell            # Open interactive shell in container

make docker_iso_build        # Run ISO build inside Docker for all architectures
make docker_iso_build ARCH=aarch64  # Run ISO build for specific architecture
make docker_repo_build       # Run repo build inside Docker for all architectures
make docker_repo_build ARCH=aarch64  # Run repo build for specific architecture
make docker_test             # Build repos in Docker, then run tests locally
</pre>

=== Environment Variables ===

{| class="wikitable"
! Variable !! Default !! Description
|-
| <code>ARCH</code> || (all) || Target architecture for make targets
|-
| <code>ONIX_ARCH</code> || <code>x86_64</code> || Architecture (used by entrypoint.sh and docker-compose)
|-
| <code>ONIX_CHROOT</code> || (auto) || Explicit chroot path for repo builds (Jenkins per-build)
|-
| <code>ONIX_ISO_CHROOT</code> || (auto) || Explicit chroot path for ISO builds
|-
| <code>ONIX_CHROOT_PREFIX</code> || <code>/tmp/onix-chroot</code> || Chroot path prefix (auto-suffix with arch+pid+ts)
|-
| <code>ONIX_SIGN_KEY</code> || (default key) || GPG key ID / email for signing
|-
| <code>ONIX_GPG_PASSPHRASE</code> || (none) || GPG passphrase for batch mode
|}

== Docker Builds ==

The Docker image is based on <code>archlinux</code> and ships with all build dependencies pre-installed, including <code>devtools</code> (for <code>makechrootpkg</code>), <code>qemu-user-static</code> (for ARM emulation), and <code>gnupg</code> (for package signing). The image tag is <code>olproject/buildkit</code>. All Docker-related files are located under <code>buildkit-docker/</code>.

=== Building the Image ===

<pre>
docker build -f buildkit-docker/Dockerfile . -t olproject/buildkit
# or via make:
make docker_build
</pre>

The Dockerfile installs:
* <code>base-devel</code>, <code>devtools</code> — compiler toolchain and <code>makechrootpkg</code>
* <code>qemu-user-static</code>, <code>binfmt-support</code> — ARM cross-architecture emulation
* <code>gnupg</code> — GPG package signing
* <code>makepkg.conf</code> with generic x86-64 baseline CFLAGS

=== ISO Build via Docker ===

Builds ISO images for '''all ISO-supported architectures''' by default:

<pre>
make docker_iso_build
</pre>

To build for a '''specific architecture''':

<pre>
make docker_iso_build ARCH=aarch64
</pre>

The make target automatically:
# Runs the Docker container with <code>ONIX_ARCH</code> set.
# Copies <code>output/</code> and <code>logs/</code> from the container.
# Removes the container.
# Iterates over all ISO architectures if no <code>ARCH</code> is specified.

The container entrypoint (<code>entrypoint.sh</code>) automatically:
# Resolves the chroot directory (from <code>ONIX_CHROOT</code> or generates dynamic path).
# Tears down any existing chroot and bootstraps a clean one with <code>mkarchroot</code>.
# Ensures <code>GNUPGHOME</code> directory exists.
# Passes <code>--arch</code> parameter to <code>python build.py iso</code>.

=== Repository Build via Docker ===

Builds repositories for '''all architectures''' by default:

<pre>
docker-compose -f buildkit-docker/docker-compose.repos.yml up --abort-on-container-exit --build
docker cp buildkit:/home/buildkit/builder/output/. ./output/
docker cp buildkit:/home/buildkit/builder/logs/. ./logs/
docker rm buildkit
# or via make:
make docker_repo_build
</pre>

To build for a '''specific architecture''':

<pre>
ONIX_ARCH=x86_64_v3 docker-compose -f buildkit-docker/docker-compose.repos.yml up --abort-on-container-exit --build
# or via make:
make docker_repo_build ARCH=x86_64_v3
</pre>

The container entrypoint (<code>entrypoint.sh</code>) automatically:
# Resolves the chroot directory (from <code>ONIX_CHROOT</code> or generates dynamic path).
# Tears down any existing chroot and bootstraps a clean one with <code>mkarchroot</code>.
# Ensures <code>GNUPGHOME</code> directory exists.
# Passes <code>--arch</code> parameter to <code>python build.py repos</code> or <code>python build.py iso</code>.

=== Interactive Shell ===

Starts a persistent container with output and logs mounted as volumes, useful for manual inspection or debugging:

<pre>
docker-compose -f buildkit-docker/docker-compose.yml up -d
docker exec -it buildkit bash
# or via make:
make docker_shell
</pre>

The container mounts:
* <code>./output/</code> → <code>/home/buildkit/builder/output</code>
* <code>./logs/</code> → <code>/home/buildkit/builder/logs</code>
* <code>gnupg</code> volume → <code>/gnupg</code> (GPG keyring)

{{Note|The container runs in <code>--privileged</code> mode, which is required by <code>mkarchiso</code> for loop device and mount operations, and by <code>makechrootpkg</code> for chroot operations.}}

== Profiles ==

ISO profiles are listed in <code>sources/profiles.list</code>, one per line. Each profile corresponds to a directory under <code>sources/profiles/</code> that contains a complete <code>archiso</code> profile.

Each profile directory contains architecture-specific files:

{| class="wikitable"
! File !! Purpose
|-
| <code>profiledef.sh</code> || ISO metadata (name, label, version, boot modes, arch) — '''rewritten at build time''' for target architecture
|-
| <code>pacman.conf</code> || Pacman config for ISO build (with <code>$arch</code> repo URLs)
|-
| <code>pacman.aarch64.conf</code> || Pacman config for aarch64 (no multilib, <code>Architecture = aarch64</code>)
|-
| <code>packages.x86_64</code> || Package list for x86_64
|-
| <code>packages.aarch64</code> || Package list for aarch64 (excludes x86-only packages)
|-
| <code>airootfs/</code> || Files to overlay onto the ISO root filesystem
|-
| <code>syslinux/</code> || Syslinux bootloader configs (x86_64 only, removed for aarch64)
|-
| <code>efiboot/</code> || EFI boot files (per-architecture entries)
|}

Available profiles:

[Profiles are now maintained in an external repository](https://gitlab.com/onix-os/onixos-profiles). The repository contains the same profile directories as before. You can clone it and point the build system to the directory.


{| class="wikitable"
! Profile !! Description
|-
| <code>core</code> || Minimal base system
|-
| <code>gnome</code> || GNOME desktop environment
|-
| <code>xfce</code> || Xfce desktop environment
|-
| <code>icewm</code> || IceWM lightweight desktop
|-
| <code>security</code> || Security-focused variant
|-
<!-- Corporate profile removed -->
|-
| <code>odesk</code> || Office desktop environment
|}

<!-- Kurumsal sub‑profiles removed -->

To build only a specific profile, edit <code>sources/profiles.list</code> temporarily or call <code>ISO.mkarchiso()</code> directly.

== Packages ==

=== Local Packages (onix-base) ===

Defined in <code>sources/packages.list</code>. Sources live under <code>sources/packages/&lt;name&gt;/</code>:

{| class="wikitable"
! Package !! Purpose
|-
| <code>olang</code> || OnixOS language runtime
|-
| <code>onix-base</code> || Base system metapackage
|-
| <code>onix-graphics</code> || Graphics stack
|-
| <code>onix-meta</code> || Meta package
|-
| <code>calamares-onix</code> || Graphical installer (Qt6, git source, with branding)
|-
| <code>linux-onix-zen</code> || OnixOS ZEN kernel (7.1.7-zen1)
|-
| <code>linux-onix-rt</code> || OnixOS real-time kernel
|-
| <code>linux-onix-generic</code> || OnixOS generic kernel (Arch mainline 7.1.7.arch1)
|-
| <code>plymouth-onix-boot</code> || Boot splash screen
|-
| <code>openxenmanager</code> || Xen hypervisor management tool
|-
| <code>onix-updater</code> || OnixOS system updater
|-
| <code>mkinitcpio-openswap</code> || Encrypted swap support for mkinitcpio
|-
| <code>ckbcomp</code> || Keyboard composition utility
|}

=== External Packages Repository ===

The package definitions have been moved to an external repository:

[OnixOS Packages Repository](https://gitlab.com/onix-os/onixos-packages)

You can clone this repository and refer to it for package sources.


=== AUR Packages (onix-aur) ===

Defined in <code>sources/aur.list</code>. Downloaded from the AUR snapshot endpoint at build time. Output goes to <code>output/aur/</code>.

== Sync Configuration ==

Both repository and ISO sync use SSH (<code>rsync</code> over SSH). Configuration files follow the standard <code>configparser</code> INI format.

=== Repository Sync ===

Config files read by <code>build.py upload</code> (both are used if they exist):
* <code>sources/sync/sourceforge.conf</code> — SourceForge remote
* <code>sources/sync/local.conf</code> — Local/internal server

Example:

<pre>
[sync]
server  = frs.sourceforge.net
user    = olproject
path    = /home/frs/project/onixos/Repo
port    = 22
keyfile = /home/ted/.ssh/id_ed25519
</pre>

Uploads <code>output/base/&lt;arch&gt;/repo</code> → <code>&lt;path&gt;/base/&lt;arch&gt;</code> and <code>output/aur/&lt;arch&gt;/repo</code> → <code>&lt;path&gt;/aur/&lt;arch&gt;</code> for each architecture.

Example remote structure:

<pre>
/home/frs/project/onixos/Repo/
├── base/
│   ├── x86_64/
│   │   ├── onix-base.db.tar.gz
│   │   ├── package1.pkg.tar.zst
│   │   └── package1.pkg.tar.zst.sig
│   ├── x86_64_v2/
│   ├── x86_64_v3/
│   ├── i686/
│   ├── aarch64/
│   ├── armv7h/
│   └── armv6h/
└── aur/
    ├── x86_64/
    ├── x86_64_v2/
    └── ...
</pre>

=== ISO Sync ===

Config files read by <code>build.py iso-sync</code>:
* <code>sources/isosync/sourceforge.conf</code>
* <code>sources/isosync/local.conf</code>

Uploads <code>output/iso/&lt;profile&gt;/&lt;arch&gt;/output</code> → <code>&lt;path&gt;/ISO/&lt;arch&gt;</code> for each profile and architecture.

Example remote structure:

<pre>
/home/frs/project/onixos/Daily/ISO/
├── x86_64/
│   ├── onixos-core-*.iso
│   ├── onixos-gnome-*.iso
│   ├── onixos-xfce-*.iso
│   ├── onixos-icewm-*.iso
│   └── onixos-security-*.iso
└── aarch64/
    ├── onixos-core-*.iso
    ├── onixos-gnome-*.iso
    └── ...
</pre>

== Debug Mode ==

Append <code>-d</code> to <code>iso</code> or <code>repos</code> subcommands to stream <code>stdout</code>/<code>stderr</code> to the terminal in real time:

<pre>
sudo python build.py iso -d
sudo python build.py iso --arch=aarch64 -d
python build.py repos -d
python build.py repos --arch=x86_64_v3 -d
</pre>

Without <code>-d</code>, command output is only written to the log files in <code>logs/&lt;arch&gt;/</code>.

== Build Watcher ==

<code>watcher.py</code> provides a live terminal dashboard that tracks ISO and repository build status, active mounts, and recent log output. It refreshes every 3 seconds.

'''Standalone (curses UI):'''

 python watcher.py

Press <code>q</code> or <code>Esc</code> to exit.

'''Attached to a build (GUI overlay):'''

<pre>
sudo python build.py iso -gui
python build.py repos -gui
</pre>

The <code>-gui</code> flag starts the watcher in a background thread alongside the build process and stops it automatically when the build finishes.

'''What the watcher shows:'''
* '''ISO Builds''' — profile name, ISO filename, file size, build timestamp, architecture, and status. Scans per-arch directories (<code>output/iso/&lt;profile&gt;/&lt;arch&gt;/output/</code>).
* '''Repositories''' — repo name, total package count across all architectures, DB update timestamp, and status. Scans per-arch directories (<code>output/base/&lt;arch&gt;/repo/</code>).
* '''Active Mounts''' — any loop/overlay mounts still active inside <code>output/</code>.
* '''Last Build Log''' — last 6 lines of <code>logs/isobuild.log</code> with colour-coded errors and warnings.

== Testing with QEMU ==

Launch a built ISO in a QEMU virtual machine for quick validation:

 python qemu.py <path-to-iso> [--arch=<arch>]

Examples:

<pre>
python qemu.py output/iso/gnome/x86_64/output/onixos-gnome-x86_64.iso
python qemu.py output/iso/core/aarch64/output/onixos-core-aarch64.iso --arch=aarch64
</pre>

VM parameters:

{| class="wikitable"
! Parameter !! x86_64 !! aarch64
|-
| Binary || <code>qemu-system-x86_64</code> || <code>qemu-system-aarch64</code>
|-
| Machine || (default) || <code>virt</code>
|-
| CPU || <code>x86_64</code> || <code>cortex-a57</code>
|-
| RAM || 4 GB || 4 GB
|-
| Cores || 1 || 1
|-
| VGA || qxl || virtio-gpu-pci
|-
| Network || User-mode (NAT) || User-mode (NAT)
|-
| Boot || CD-ROM || CD-ROM
|}

<code>qemu-system-x86_64</code> or <code>qemu-system-aarch64</code> must be installed and available in <code>PATH</code>.

== Cleaning the Build ==

Removes all build artefacts, clears the pacman package cache, unmounts any stale mounts, and deletes log files:

<pre>
sudo python clean.py
# or via make:
make clean
</pre>

What <code>clean.py</code> does in order:
# Reads <code>/proc/mounts</code> and unmounts (with <code>umount -l</code>) anything mounted inside <code>output/</code>, deepest paths first.
# Clears the pacman package cache: <code>find /var/cache/pacman/pkg/ -name '*.pkg.tar.*' -delete</code>.
# Removes all <code>logs/*.log</code> files.
# Removes all contents of <code>output/</code> with <code>sudo rm -rfv output/*</code>.

{{Warning|This is destructive. All previously built ISOs and repository packages will be deleted.}}

To remove only the chroot directories (without touching output):

<pre>
make chroot_clean
# or for a specific architecture:
make chroot_clean ARCH=x86_64_v3
</pre>

== Isolated Chroot Build ==

Repository builds use <code>makechrootpkg</code> from the <code>devtools</code> package to compile packages in an '''isolated chroot environment'''. ISO builds use a similar chroot isolation approach with <code>mkarchiso</code>. This prevents:

# '''Host pollution''' — packages cannot accidentally link against host libraries not listed in <code>makedepends</code>.
# '''Instruction leakage''' — packages are compiled with generic CFLAGS baseline, not host-specific optimizations.
# '''Non-reproducible builds''' — the build environment is deterministic and isolated from host updates.

=== Repository Chroot Lifecycle ===

# '''Bootstrap''': <code>mkarchroot &lt;chroot&gt;/root base-devel</code> creates a minimal Arch Linux root filesystem.
# '''Build''': <code>makechrootpkg -c -r &lt;chroot&gt;</code> builds the package inside the chroot. The <code>-c</code> flag creates a copy-on-write snapshot, so the base chroot remains clean.
# '''Cleanup''': After each build, the snapshot is discarded. The base chroot persists for reuse.

=== ISO Chroot Lifecycle ===

# '''Bootstrap''': <code>mkarchroot &lt;chroot&gt;/root base-devel archiso squashfs-tools dosfstools arch-install-scripts</code> creates a chroot with all ISO build tools.
# '''Mount''': Bind-mounts virtual filesystems and directories into the chroot:
#* <code>/proc</code> → <code>chroot/root/proc</code>
#* <code>/sys</code> → <code>chroot/root/sys</code>
#* <code>/dev</code> → <code>chroot/root/dev</code>
#* <code>/dev/pts</code> → <code>chroot/root/dev/pts</code>
#* Staging profile directory → <code>chroot/root/src</code>
#* Work directory → <code>chroot/root/work</code>
#* Output directory → <code>chroot/root/out</code>
# '''Build''': <code>chroot &lt;chroot&gt;/root /usr/bin/env -i PATH=... mkarchiso ...</code> runs inside the chroot with sanitized environment.
# '''Cleanup''': All bind mounts are unmounted (deepest first) on exit, error, or signal (SIGINT/SIGTERM). Mount cleanup is guaranteed via <code>try/finally</code>.

{{Note|ISO chroot isolation can be disabled with <code>--no-chroot</code> for debugging or when running in an already-isolated environment (e.g., Docker).}}

=== Dynamic Chroot Path ===

The chroot directory is resolved dynamically per architecture:

'''Repository builds:'''
# '''Explicit''': <code>ONIX_CHROOT</code> environment variable (Jenkins per-build isolation).
# '''Prefix-based''': <code>ONIX_CHROOT_PREFIX</code> + architecture + PID + timestamp (workspace-aware).
# '''Auto-generated''': <code>/tmp/onix-chroot-&lt;arch&gt;-&lt;pid&gt;-&lt;timestamp&gt;</code> (unique per process).

'''ISO builds:'''
# '''Explicit''': <code>ONIX_ISO_CHROOT</code> environment variable.
# '''Prefix-based''': <code>ONIX_CHROOT_PREFIX</code> + architecture + PID + timestamp.
# '''Auto-generated''': <code>/tmp/onix-iso-chroot-&lt;arch&gt;-&lt;pid&gt;-&lt;timestamp&gt;</code> (unique per process).

Example for Jenkins:

<pre>
export ONIX_CHROOT="/var/lib/jenkins/onix-chroot-${BUILD_NUMBER}"
python build.py repos
</pre>

=== Manual Chroot Management ===

Bootstrap chroots for all architectures:

<pre>
make chroot_bootstrap
</pre>

Bootstrap a specific architecture:

<pre>
make chroot_bootstrap ARCH=aarch64
</pre>

Spawn a shell inside a chroot (requires <code>ONIX_CHROOT</code> to be set):

<pre>
export ONIX_CHROOT=/tmp/onix-chroot
make chroot_bootstrap ARCH=x86_64
make chroot_shell ARCH=x86_64
</pre>

== PKGBUILD Sanitization ==

Before building each package, the build system '''sanitizes''' the <code>PKGBUILD</code> file to prevent instruction leakage:

# Searches for hardcoded <code>-march=native</code>, <code>-march=haswell</code>, <code>-mavx2</code>, etc.
# Rewrites them to the architecture-specific baseline (e.g., <code>-march=x86-64</code> for <code>x86_64</code>).
# Strips explicit AVX/SSE4 feature flags (<code>-mavx</code>, <code>-mfma</code>, <code>-msse4.2</code>).
# Backs up the original <code>PKGBUILD</code> to <code>PKGBUILD.bak</code> (idempotent — only first run).

This ensures packages compiled on a modern build server (e.g., with AVX-512) will run on older CPUs without triggering <code>Illegal instruction</code> errors.

=== Sanitization Patterns ===

{| class="wikitable"
! Pattern !! Replacement (x86_64) !! Replacement (x86_64_v3)
|-
| <code>-march=native</code> || <code>-march=x86-64</code> || <code>-march=x86-64-v3</code>
|-
| <code>-march=haswell</code> || <code>-march=x86-64</code> || <code>-march=x86-64-v3</code>
|-
| <code>-march=skylake</code> || <code>-march=x86-64</code> || <code>-march=x86-64-v3</code>
|-
| <code>-mtune=native</code> || <code>-mtune=generic</code> || <code>-mtune=generic</code>
|-
| <code>-mavx2</code> || (removed) || (removed)
|-
| <code>-mfma</code> || (removed) || (removed)
|-
| <code>-msse4.2</code> || (removed) || (removed)
|}

=== Verification ===

Test the sanitizer with a synthetic PKGBUILD:

<pre>
make test-sanitize ARCH=x86_64
</pre>

== GPG Signing ==

Each built package is signed with a GPG detach-signature before being added to the repository database.

=== Signing Configuration ===

Create <code>sources/signing.conf</code> (example):

<pre>
[sign]
key_id = "FINGERPRINT_OR_EMAIL_OF_SIGNING_KEY"
passphrase_env = "ONIX_GPG_PASSPHRASE"
</pre>

=== Key Setup ===

Import the private key into the GPG keyring:

<pre>
export GNUPGHOME=/gnupg  # or ~/.gnupg for local builds
gpg --import <private-key.asc>
gpg --edit-key <key_id>
  > trust
  > 5
  > save
</pre>

=== Environment Variables ===

{| class="wikitable"
! Variable !! Description
|-
| <code>GNUPGHOME</code> || Path to GPG keyring directory (default: <code>/gnupg</code> in Docker)
|-
| <code>ONIX_SIGN_KEY</code> || GPG key ID or email to use with <code>--local-user</code>
|-
| <code>ONIX_GPG_PASSPHRASE</code> || Passphrase for the signing key (batch mode with <code>--pinentry-mode loopback</code>)
|}

=== Verification ===

Verify signatures on built packages:

<pre>
make test-sign ARCH=x86_64
</pre>

== Testing ==

The build system includes a comprehensive test suite to verify package integrity.

=== Test Targets ===

{| class="wikitable"
! Target !! Description
|-
| <code>make test</code> || Run all tests for all architectures
|-
| <code>make test ARCH=x86_64_v3</code> || Run all tests for specific architecture
|-
| <code>make test-march</code> || Verify compiled binaries use generic <code>-march</code> baseline (no instruction leakage)
|-
| <code>make test-sign</code> || Verify GPG signatures exist and validate
|-
| <code>make test-deps</code> || Check for missing shared-object dependencies (host makedepends leak detection)
|-
| <code>make test-sanitize</code> || Verify PKGBUILD sanitizer rewrites flags correctly
|-
| <code>make test-smoke</code> || End-to-end smoke test: build single package, verify integrity
|}

=== test-march ===

Inspects every ELF binary in built packages using <code>readelf -n</code> to verify the <code>march</code> value matches the architecture baseline. Fails if binaries were compiled with host-specific optimizations (e.g., <code>-march=skylake</code> on an <code>x86_64</code> build).

=== test-sign ===

Verifies that every <code>.pkg.tar.zst</code> file has a corresponding <code>.pkg.tar.zst.sig</code> file and that the signature validates against the GPG keyring.

=== test-deps ===

Extracts each package, runs <code>ldd</code> on ELF binaries, and checks for "not found" shared libraries. Detects cases where a package was compiled against host libraries not listed in <code>makedepends</code> (chroot isolation failure).

=== test-sanitize ===

Creates a synthetic <code>PKGBUILD</code> with instruction leakage (<code>-march=native</code>, <code>-mavx2</code>, etc.), runs the sanitizer, and verifies:
# Flags were rewritten to the architecture baseline.
# Original <code>PKGBUILD</code> was backed up to <code>PKGBUILD.bak</code>.
# No leaked flags remain in the sanitized file.

=== test-smoke ===

Builds a single small AUR package (e.g., <code>lolcat</code>) end-to-end in a fresh chroot, then runs <code>test-march</code> on the result. Useful as a fast CI gate before building the full ~1000-package list.

=== Example Test Run ===

<pre>
# Build repos for x86_64_v3
make repos ARCH=x86_64_v3

# Run all tests
make test ARCH=x86_64_v3

# Or run individual tests
make test-march ARCH=x86_64_v3
make test-sign ARCH=x86_64_v3
make test-deps ARCH=x86_64_v3
</pre>

== Output Structure ==

After a successful full build (all architectures):

<pre>
output/
├── base/
│   ├── x86_64/
│   │   └── repo/              # onix-base repository for x86_64
│   ├── x86_64_v2/
│   │   └── repo/              # onix-base repository for x86_64_v2
│   ├── x86_64_v3/
│   │   └── repo/              # onix-base repository for x86_64_v3
│   ├── i686/
│   │   └── repo/              # onix-base repository for i686
│   ├── aarch64/
│   │   └── repo/              # onix-base repository for aarch64
│   ├── armv7h/
│   │   └── repo/              # onix-base repository for armv7h
│   └── armv6h/
│       └── repo/              # onix-base repository for armv6h
├── aur/
│   ├── x86_64/
│   │   └── repo/              # onix-aur repository for x86_64
│   ├── x86_64_v2/
│   │   └── repo/
│   ├── x86_64_v3/
│   │   └── repo/
│   ├── i686/
│   │   └── repo/
│   ├── aarch64/
│   │   └── repo/
│   ├── armv7h/
│   │   └── repo/
│   └── armv6h/
│       └── repo/
└── iso/
    ├── core/
    │   ├── x86_64/
    │   │   ├── work/              # mkarchiso working directory (temporary)
    │   │   └── output/            # core-*.iso (x86_64)
    │   └── aarch64/
    │       ├── work/
    │       └── output/            # core-*.iso (aarch64)
    ├── gnome/
    │   ├── x86_64/
    │   │   ├── work/
    │   │   └── output/            # gnome-*.iso (x86_64)
    │   └── aarch64/
    │       ├── work/
    │       └── output/            # gnome-*.iso (aarch64)
    ├── xfce/
    │   ├── x86_64/
    │   └── aarch64/
    ├── icewm/
    │   ├── x86_64/
    │   └── aarch64/
    ├── security/
    │   ├── x86_64/
    │   └── aarch64/
    ├── kurumsal/
    │   ├── firewall/
    │   │   ├── work/
    │   │   └── output/            # kurumsal-firewall-*.iso
    │   ├── server/
    │   │   ├── work/
    │   │   └── output/            # kurumsal-server-*.iso
    │   └── desktop/
    │       ├── work/
    │       └── output/            # kurumsal-desktop-*.iso
    └── odesk/
        ├── work/
        └── output/                # odesk-*.iso
</pre>

Each repository directory contains:
* <code>*.pkg.tar.zst</code> — built packages
* <code>*.pkg.tar.zst.sig</code> — GPG detach-signatures
* <code>onix-base.db.tar.gz</code> or <code>onix-aur.db.tar.gz</code> — repository database
* <code>onix-base.files.tar.gz</code> or <code>onix-aur.files.tar.gz</code> — file list database

== Logs ==

Build logs are written to <code>logs/&lt;arch&gt;/</code> in the repository root (per-architecture):

<pre>
logs/
├── x86_64/
│   ├── chroot-bootstrap.log   # mkarchroot output (repo builds)
│   ├── iso-chroot-bootstrap.log  # mkarchroot output (ISO chroot)
│   ├── isobuild.log           # mkarchiso output (ISO builds)
│   ├── pacinst.log            # pacman dependency installation
│   ├── &lt;package&gt;.log          # makechrootpkg output per package
│   ├── sign.log               # GPG signing output
│   ├── repoadd.log            # repo-add output
│   └── copy.log               # package copy output
├── x86_64_v2/
│   └── ...
├── x86_64_v3/
│   └── ...
├── i686/
│   └── ...
├── aarch64/
│   ├── chroot-bootstrap.log
│   ├── iso-chroot-bootstrap.log
│   ├── isobuild.log
│   └── ...
├── armv7h/
│   └── ...
├── armv6h/
│   └── ...
└── clean.log                  # clean operation output
</pre>

Each log entry records the command, date/time, exit code, stderr, and stdout.