Main Page: Difference between revisions

From OnixOS
← Older editNewer edit →
VisualWikitext
No edit summary
No edit summary
Line 1: Line 1:
= OnixOS =
= Building From Scratch =


[[File:OnixOS.png|thumb|right|OnixOS Logo]]
This document covers the complete build lifecycle of the OnixOS Build System — from environment setup and package repository builds to ISO creation, syncing, Docker-based builds, and live build monitoring.


'''OnixOS''' is an open-source, Arch Linux-based distribution. It ships with '''O Language''' — a system-level functional programming language developed specifically for OnixOS — and offers a curated set of developer tools.
__TOC__
 
== Prerequisites ==
 
The build system runs on '''Arch Linux''' (or an Arch-based environment). The following packages are required for ISO builds and are installed automatically by <code>build.py</code>:
 
{| class="wikitable"
! Package !! Purpose
|-
| <code>archiso</code> || Core ISO creation tool (<code>mkarchiso</code>)
|-
| <code>base-devel</code> || Compiler toolchain
|-
| <code>devtools</code> || <code>makechrootpkg</code>, <code>mkarchroot</code> for isolated chroot builds
|-
| <code>arch-install-scripts</code> || <code>pacstrap</code>, <code>arch-chroot</code>
|-
| <code>squashfs-tools</code> || SquashFS filesystem creation
|-
| <code>dosfstools</code> || FAT filesystem creation
|-
| <code>syslinux</code> || Bootloader
|-
| <code>git</code> / <code>git-lfs</code> || Source fetching
|-
| <code>rsync</code> || File synchronisation
|-
| <code>go</code> || Go language toolchain
|-
| <code>nasm</code> || Assembler
|-
| <code>ninja</code> || Build system
|-
| <code>gperf</code> || Hash function generator
|-
| <code>imagemagick</code> || Image processing
|-
| <code>python-sphinx</code> / <code>xmlto</code> || Documentation toolchain
|-
| <code>gnome-shell</code> / <code>qtcreator</code> || Desktop environment build deps
|-
| <code>gnupg</code> || GPG package signing
|-
| <code>qemu-user-static</code> || Cross-architecture emulation (ARM builds on x86_64 host)
|-
| <code>binfmt-support</code> || Binary format registration for qemu-user-static
|-
| <code>sudo</code> || Privilege escalation
|}
 
For '''repository builds''' the system additionally uses <code>makepkg</code> (included in <code>base-devel</code>) and the AUR snapshot endpoint:
 
https://aur.archlinux.org/cgit/aur.git/snapshot
 
== Project Structure ==
 
<pre>
onix-build-system/
├── build.py                  # Main build entry point
├── clean.py                  # Cleans output/ and mounts
├── watcher.py                # Terminal build status monitor
├── qemu.py                  # Launch built ISO in QEMU
├── Makefile                  # Convenience make targets
├── buildkit-docker/
│  ├── Dockerfile            # Build container image
│  ├── entrypoint.sh        # Docker entrypoint (chroot bootstrap)
│  ├── conf/
│  │  └── makepkg.conf      # Generic x86-64 baseline CFLAGS
│  ├── docker-compose.yml        # Dev shell container
│  ├── docker-compose.iso.yml    # Automated ISO build container
│  └── docker-compose.repos.yml  # Automated repo build container
├── onixos/
│  ├── archconfig.py        # Multiarch configuration (CFLAGS, chroot, qemu)
│  ├── iso.py                # ISO build logic (mkarchiso wrapper)
│  ├── pkgbuild.py          # PKGBUILD / .SRCINFO parser
│  ├── proc.py              # Subprocess wrapper with logging
│  └── repository.py        # Repository build logic (chroot, sanitize, sign)
├── sources/
│  ├── packages/            # Local package sources (PKGBUILD)
│  ├── profiles/            # archiso profiles (core, gnome, xfce, icewm, security, kurumsal, odesk)
│  ├── packages.list        # Local packages to build
│  ├── aur.list              # AUR packages to build
│  ├── profiles.list        # ISO profiles to build
│  ├── signing.conf          # GPG signing configuration (example)
│  ├── sync/                # Repo upload config
│  │  ├── sourceforge.conf
│  │  └── local.conf
│  └── isosync/              # ISO upload config
│      ├── sourceforge.conf
│      └── local.conf
├── output/                  # All build artefacts (generated, per-arch)
└── logs/                    # Build logs (generated, per-arch)
</pre>
 
== Build Commands ==
 
All commands must be run from the '''repository root'''.
 
=== Full Build ===
 
Builds all repositories '''and''' all ISO profiles in sequence:
 
sudo python build.py
 
=== Repository Build ===
 
Builds the <code>onix-base</code> (local packages) and <code>onix-aur</code> (AUR packages) repositories for '''all supported architectures''' by default:
 
python build.py repos
 
To build for a '''specific architecture''':
 
python build.py repos --arch=x86_64_v3
 
The build process per architecture:
# Bootstraps a clean chroot with <code>base-devel</code> (isolated from host system).
# Generates architecture-specific <code>makepkg.conf</code> inside the chroot (generic CFLAGS baseline).
# Sanitizes PKGBUILD files to prevent instruction leakage (<code>-march=native</code> → architecture baseline).
# Builds packages using <code>makechrootpkg -c -r &lt;chroot&gt;</code> (host isolation, dependency resolution inside chroot).
# Signs each <code>.pkg.tar.zst</code> with GPG detach-signature (if signing key is configured).
# Generates repository database with <code>repo-add</code>.
 
'''Supported architectures:'''
 
{| class="wikitable"
! Architecture !! CFLAGS Baseline !! Description !! QEMU Required
|-
| <code>x86_64</code> || <code>-march=x86-64</code> || Generic x86-64 (2004+, all CPUs) || No
|-
| <code>x86_64_v2</code> || <code>-march=x86-64-v2</code> || SSE4.2, POPCNT (2008+: Nehalem, Bulldozer) || No
|-
| <code>x86_64_v3</code> || <code>-march=x86-64-v3</code> || AVX, AVX2, BMI1/2, FMA (2015+: Haswell, Zen) || No
|-
| <code>i686</code> || <code>-march=i686</code> || 32-bit x86 (Pentium Pro+) || No
|-
| <code>aarch64</code> || <code>-march=armv8-a</code> || 64-bit ARM (ARMv8-A) || Yes
|-
| <code>armv7h</code> || <code>-march=armv7-a -mfloat-abi=hard</code> || 32-bit ARM hard-float (ARMv7-A) || Yes
|-
| <code>armv6h</code> || <code>-march=armv6 -mfloat-abi=hard</code> || 32-bit ARM hard-float (ARMv6, Raspberry Pi 1) || Yes
|}
 
{{Note|ARM architectures require <code>qemu-user-static</code> and <code>binfmt-support</code> for cross-architecture emulation. The Docker image includes these packages.}}
 
=== ISO Build ===
 
Builds ISO images for '''all ISO-supported architectures''' (x86_64 and aarch64) and every profile listed in <code>sources/profiles.list</code>:
 
sudo python build.py iso
 
To build for a '''specific architecture''':
 
sudo python build.py iso --arch=aarch64
 
To build '''without chroot isolation''' (direct <code>mkarchiso</code> invocation, useful for debugging):
 
sudo python build.py iso --no-chroot
 
The build process per profile per architecture:
# Installs ISO build dependencies via <code>pacman</code>.
# Creates a '''staging copy''' of the profile with architecture-specific overrides:
#* Rewrites <code>profiledef.sh</code> with the target architecture and boot modes.
#* Selects <code>packages.&lt;arch&gt;</code> as the package list.
#* Generates architecture-specific <code>pacman.conf</code> (with <code>$arch</code> repo URLs).
#* Removes <code>syslinux/</code> for aarch64 (UEFI-only).
# When '''chroot isolation''' is enabled (default):
## Bootstraps a clean chroot with <code>base-devel</code>, <code>archiso</code>, <code>squashfs-tools</code>, <code>dosfstools</code>, and <code>arch-install-scripts</code>.
## Bind-mounts <code>/proc</code>, <code>/sys</code>, <code>/dev</code>, <code>/dev/pts</code>, source profile, work directory, and output directory into the chroot.
## Runs <code>mkarchiso</code> inside the chroot with sanitized environment variables.
## Unmounts all bind mounts (deepest first) on exit, error, or signal (SIGINT/SIGTERM).
# When '''chroot isolation''' is disabled (<code>--no-chroot</code>):
## Unmounts any stale mounts from a previous failed build.
## Cleans the work directory.
## Runs <code>mkarchiso</code> directly on the host.
# Writes the finished <code>.iso</code> file to <code>output/iso/&lt;profile&gt;/&lt;arch&gt;/output/</code>.
 
'''ISO-supported architectures:'''
 
{| class="wikitable"
! Architecture !! Boot Modes !! Syslinux !! Notes
|-
| <code>x86_64</code> || BIOS (syslinux) + UEFI (systemd-boot) || Yes || Standard x86-64 ISO
|-
| <code>aarch64</code> || UEFI only (systemd-boot) || No || ARM64, excludes x86-only packages (microcode, video drivers)
|}
 
{{Note|The aarch64 ISO excludes packages that are x86-specific: <code>amd-ucode</code>, <code>intel-ucode</code>, <code>memtest86+</code>, <code>syslinux</code>, <code>broadcom-wl</code>, <code>b43-fwcutter</code>, x86-only video drivers (<code>xf86-video-*</code>), and <code>edk2-shell</code>.}}
 
=== Upload / Sync ===
 
Uploads built repositories to the configured remote server(s):
 
python build.py upload
 
Uploads built ISO images:
 
python build.py iso-sync
 
Both commands read from the relevant <code>*.conf</code> files under <code>sources/sync/</code> and <code>sources/isosync/</code> respectively (see [[#Sync Configuration|Sync Configuration]]).
 
== Make Targets ==
 
=== Core Build Targets ===
 
<pre>
make            # clean → repos → iso (full build, all architectures)
make clean      # Remove output artefacts and unmount
make repos      # Repository build for all architectures
make repos ARCH=x86_64_v3  # Repository build for specific architecture
make iso        # ISO build for all architectures (x86_64 + aarch64)
make iso ARCH=x86_64      # ISO build for specific architecture
make iso-all    # ISO build for all architectures (explicit)
make iso-nochroot          # ISO build without chroot isolation
make iso-nochroot ARCH=aarch64  # ISO build without chroot for specific architecture
</pre>
 
=== Chroot Management ===
 
<pre>
make chroot_bootstrap              # Bootstrap chroots for all architectures
make chroot_bootstrap ARCH=x86_64  # Bootstrap chroot for specific architecture
make chroot_clean                  # Remove all chroots
make chroot_clean ARCH=x86_64      # Remove chroot for specific architecture
make chroot_shell ARCH=x86_64      # Spawn shell inside chroot (requires ONIX_CHROOT)
</pre>
 
=== Testing ===
 
<pre>
make test                    # Run all tests for all architectures
make test ARCH=x86_64_v3    # Run all tests for specific architecture
make test-march              # Verify -march baseline in compiled binaries
make test-sign              # Verify GPG signatures
make test-deps              # Check for missing shared-object dependencies
make test-sanitize          # Verify PKGBUILD sanitizer rewrites flags
make test-smoke              # End-to-end smoke test (build single package)
</pre>
 
=== Docker Targets ===
 
<pre>
make docker_build            # Build the Docker image
make docker_publish          # Push image to registry (olproject/buildkit)
make docker_pull            # Pull image from registry
make docker_shell            # Open interactive shell in container
 
make docker_iso_build        # Run ISO build inside Docker for all architectures
make docker_iso_build ARCH=aarch64  # Run ISO build for specific architecture
make docker_repo_build      # Run repo build inside Docker for all architectures
make docker_repo_build ARCH=aarch64  # Run repo build for specific architecture
make docker_test            # Build repos in Docker, then run tests locally
</pre>
 
=== Environment Variables ===
 
{| class="wikitable"
! Variable !! Default !! Description
|-
| <code>ARCH</code> || (all) || Target architecture for make targets
|-
| <code>ONIX_ARCH</code> || <code>x86_64</code> || Architecture (used by entrypoint.sh and docker-compose)
|-
| <code>ONIX_CHROOT</code> || (auto) || Explicit chroot path for repo builds (Jenkins per-build)
|-
| <code>ONIX_ISO_CHROOT</code> || (auto) || Explicit chroot path for ISO builds
|-
| <code>ONIX_CHROOT_PREFIX</code> || <code>/tmp/onix-chroot</code> || Chroot path prefix (auto-suffix with arch+pid+ts)
|-
| <code>ONIX_SIGN_KEY</code> || (default key) || GPG key ID / email for signing
|-
| <code>ONIX_GPG_PASSPHRASE</code> || (none) || GPG passphrase for batch mode
|}
 
== Docker Builds ==
 
The Docker image is based on <code>archlinux</code> and ships with all build dependencies pre-installed, including <code>devtools</code> (for <code>makechrootpkg</code>), <code>qemu-user-static</code> (for ARM emulation), and <code>gnupg</code> (for package signing). The image tag is <code>olproject/buildkit</code>. All Docker-related files are located under <code>buildkit-docker/</code>.
 
=== Building the Image ===
 
<pre>
docker build -f buildkit-docker/Dockerfile . -t olproject/buildkit
# or via make:
make docker_build
</pre>
 
The Dockerfile installs:
* <code>base-devel</code>, <code>devtools</code> — compiler toolchain and <code>makechrootpkg</code>
* <code>qemu-user-static</code>, <code>binfmt-support</code> — ARM cross-architecture emulation
* <code>gnupg</code> — GPG package signing
* <code>makepkg.conf</code> with generic x86-64 baseline CFLAGS
 
=== ISO Build via Docker ===
 
Builds ISO images for '''all ISO-supported architectures''' by default:
 
<pre>
make docker_iso_build
</pre>
 
To build for a '''specific architecture''':
 
<pre>
make docker_iso_build ARCH=aarch64
</pre>
 
The make target automatically:
# Runs the Docker container with <code>ONIX_ARCH</code> set.
# Copies <code>output/</code> and <code>logs/</code> from the container.
# Removes the container.
# Iterates over all ISO architectures if no <code>ARCH</code> is specified.
 
The container entrypoint (<code>entrypoint.sh</code>) automatically:
# Resolves the chroot directory (from <code>ONIX_CHROOT</code> or generates dynamic path).
# Tears down any existing chroot and bootstraps a clean one with <code>mkarchroot</code>.
# Ensures <code>GNUPGHOME</code> directory exists.
# Passes <code>--arch</code> parameter to <code>python build.py iso</code>.
 
=== Repository Build via Docker ===
 
Builds repositories for '''all architectures''' by default:
 
<pre>
docker-compose -f buildkit-docker/docker-compose.repos.yml up --abort-on-container-exit --build
docker cp buildkit:/home/buildkit/builder/output/. ./output/
docker cp buildkit:/home/buildkit/builder/logs/. ./logs/
docker rm buildkit
# or via make:
make docker_repo_build
</pre>
 
To build for a '''specific architecture''':
 
<pre>
ONIX_ARCH=x86_64_v3 docker-compose -f buildkit-docker/docker-compose.repos.yml up --abort-on-container-exit --build
# or via make:
make docker_repo_build ARCH=x86_64_v3
</pre>
 
The container entrypoint (<code>entrypoint.sh</code>) automatically:
# Resolves the chroot directory (from <code>ONIX_CHROOT</code> or generates dynamic path).
# Tears down any existing chroot and bootstraps a clean one with <code>mkarchroot</code>.
# Ensures <code>GNUPGHOME</code> directory exists.
# Passes <code>--arch</code> parameter to <code>python build.py repos</code> or <code>python build.py iso</code>.
 
=== Interactive Shell ===
 
Starts a persistent container with output and logs mounted as volumes, useful for manual inspection or debugging:
 
<pre>
docker-compose -f buildkit-docker/docker-compose.yml up -d
docker exec -it buildkit bash
# or via make:
make docker_shell
</pre>
 
The container mounts:
* <code>./output/</code> → <code>/home/buildkit/builder/output</code>
* <code>./logs/</code> → <code>/home/buildkit/builder/logs</code>
* <code>gnupg</code> volume → <code>/gnupg</code> (GPG keyring)
 
{{Note|The container runs in <code>--privileged</code> mode, which is required by <code>mkarchiso</code> for loop device and mount operations, and by <code>makechrootpkg</code> for chroot operations.}}
 
== Profiles ==
 
ISO profiles are listed in <code>sources/profiles.list</code>, one per line. Each profile corresponds to a directory under <code>sources/profiles/</code> that contains a complete <code>archiso</code> profile.
 
Each profile directory contains architecture-specific files:
 
{| class="wikitable"
! File !! Purpose
|-
| <code>profiledef.sh</code> || ISO metadata (name, label, version, boot modes, arch) — '''rewritten at build time''' for target architecture
|-
| <code>pacman.conf</code> || Pacman config for ISO build (with <code>$arch</code> repo URLs)
|-
| <code>pacman.aarch64.conf</code> || Pacman config for aarch64 (no multilib, <code>Architecture = aarch64</code>)
|-
| <code>packages.x86_64</code> || Package list for x86_64
|-
| <code>packages.aarch64</code> || Package list for aarch64 (excludes x86-only packages)
|-
| <code>airootfs/</code> || Files to overlay onto the ISO root filesystem
|-
| <code>syslinux/</code> || Syslinux bootloader configs (x86_64 only, removed for aarch64)
|-
| <code>efiboot/</code> || EFI boot files (per-architecture entries)
|}
 
Available profiles:
 
[Profiles are now maintained in an external repository](https://gitlab.com/onix-os/onixos-profiles). The repository contains the same profile directories as before. You can clone it and point the build system to the directory.
 
 
{| class="wikitable"
! Profile !! Description
|-
| <code>core</code> || Minimal base system
|-
| <code>gnome</code> || GNOME desktop environment
|-
| <code>xfce</code> || Xfce desktop environment
|-
| <code>icewm</code> || IceWM lightweight desktop
|-
| <code>security</code> || Security-focused variant
|-
<!-- Corporate profile removed -->
|-
| <code>odesk</code> || Office desktop environment
|}
 
<!-- Kurumsal sub‑profiles removed -->
 
To build only a specific profile, edit <code>sources/profiles.list</code> temporarily or call <code>ISO.mkarchiso()</code> directly.
 
== Packages ==
 
=== Local Packages (onix-base) ===
 
Defined in <code>sources/packages.list</code>. Sources live under <code>sources/packages/&lt;name&gt;/</code>:
 
{| class="wikitable"
! Package !! Purpose
|-
| <code>olang</code> || OnixOS language runtime
|-
| <code>onix-base</code> || Base system metapackage
|-
| <code>onix-graphics</code> || Graphics stack
|-
| <code>onix-meta</code> || Meta package
|-
| <code>calamares-onix</code> || Graphical installer (Qt6, git source, with branding)
|-
| <code>linux-onix-zen</code> || OnixOS ZEN kernel (7.1.7-zen1)
|-
| <code>linux-onix-rt</code> || OnixOS real-time kernel
|-
| <code>linux-onix-generic</code> || OnixOS generic kernel (Arch mainline 7.1.7.arch1)
|-
| <code>plymouth-onix-boot</code> || Boot splash screen
|-
| <code>openxenmanager</code> || Xen hypervisor management tool
|-
| <code>onix-updater</code> || OnixOS system updater
|-
| <code>mkinitcpio-openswap</code> || Encrypted swap support for mkinitcpio
|-
| <code>ckbcomp</code> || Keyboard composition utility
|}
 
=== External Packages Repository ===
 
The package definitions have been moved to an external repository:
 
[OnixOS Packages Repository](https://gitlab.com/onix-os/onixos-packages)
 
You can clone this repository and refer to it for package sources.
 
 
=== AUR Packages (onix-aur) ===
 
Defined in <code>sources/aur.list</code>. Downloaded from the AUR snapshot endpoint at build time. Output goes to <code>output/aur/</code>.
 
== Sync Configuration ==
 
Both repository and ISO sync use SSH (<code>rsync</code> over SSH). Configuration files follow the standard <code>configparser</code> INI format.
 
=== Repository Sync ===
 
Config files read by <code>build.py upload</code> (both are used if they exist):
* <code>sources/sync/sourceforge.conf</code> — SourceForge remote
* <code>sources/sync/local.conf</code> — Local/internal server
 
Example:
 
<pre>
[sync]
server  = frs.sourceforge.net
user    = olproject
path    = /home/frs/project/onixos/Repo
port    = 22
keyfile = /home/ted/.ssh/id_ed25519
</pre>
 
Uploads <code>output/base/&lt;arch&gt;/repo</code> → <code>&lt;path&gt;/base/&lt;arch&gt;</code> and <code>output/aur/&lt;arch&gt;/repo</code> → <code>&lt;path&gt;/aur/&lt;arch&gt;</code> for each architecture.
 
Example remote structure:
 
<pre>
/home/frs/project/onixos/Repo/
├── base/
│  ├── x86_64/
│  │  ├── onix-base.db.tar.gz
│  │  ├── package1.pkg.tar.zst
│  │  └── package1.pkg.tar.zst.sig
│  ├── x86_64_v2/
│  ├── x86_64_v3/
│  ├── i686/
│  ├── aarch64/
│  ├── armv7h/
│  └── armv6h/
└── aur/
    ├── x86_64/
    ├── x86_64_v2/
    └── ...
</pre>
 
=== ISO Sync ===
 
Config files read by <code>build.py iso-sync</code>:
* <code>sources/isosync/sourceforge.conf</code>
* <code>sources/isosync/local.conf</code>
 
Uploads <code>output/iso/&lt;profile&gt;/&lt;arch&gt;/output</code> → <code>&lt;path&gt;/ISO/&lt;arch&gt;</code> for each profile and architecture.
 
Example remote structure:
 
<pre>
/home/frs/project/onixos/Daily/ISO/
├── x86_64/
│  ├── onixos-core-*.iso
│  ├── onixos-gnome-*.iso
│  ├── onixos-xfce-*.iso
│  ├── onixos-icewm-*.iso
│  └── onixos-security-*.iso
└── aarch64/
    ├── onixos-core-*.iso
    ├── onixos-gnome-*.iso
    └── ...
</pre>
 
== Debug Mode ==
 
Append <code>-d</code> to <code>iso</code> or <code>repos</code> subcommands to stream <code>stdout</code>/<code>stderr</code> to the terminal in real time:
 
<pre>
sudo python build.py iso -d
sudo python build.py iso --arch=aarch64 -d
python build.py repos -d
python build.py repos --arch=x86_64_v3 -d
</pre>
 
Without <code>-d</code>, command output is only written to the log files in <code>logs/&lt;arch&gt;/</code>.
 
== Build Watcher ==
 
<code>watcher.py</code> provides a live terminal dashboard that tracks ISO and repository build status, active mounts, and recent log output. It refreshes every 3 seconds.
 
'''Standalone (curses UI):'''
 
python watcher.py
 
Press <code>q</code> or <code>Esc</code> to exit.
 
'''Attached to a build (GUI overlay):'''
 
<pre>
sudo python build.py iso -gui
python build.py repos -gui
</pre>
 
The <code>-gui</code> flag starts the watcher in a background thread alongside the build process and stops it automatically when the build finishes.
 
'''What the watcher shows:'''
* '''ISO Builds''' — profile name, ISO filename, file size, build timestamp, architecture, and status. Scans per-arch directories (<code>output/iso/&lt;profile&gt;/&lt;arch&gt;/output/</code>).
* '''Repositories''' — repo name, total package count across all architectures, DB update timestamp, and status. Scans per-arch directories (<code>output/base/&lt;arch&gt;/repo/</code>).
* '''Active Mounts''' — any loop/overlay mounts still active inside <code>output/</code>.
* '''Last Build Log''' — last 6 lines of <code>logs/isobuild.log</code> with colour-coded errors and warnings.
 
== Testing with QEMU ==
 
Launch a built ISO in a QEMU virtual machine for quick validation:
 
python qemu.py <path-to-iso> [--arch=<arch>]
 
Examples:
 
<pre>
python qemu.py output/iso/gnome/x86_64/output/onixos-gnome-x86_64.iso
python qemu.py output/iso/core/aarch64/output/onixos-core-aarch64.iso --arch=aarch64
</pre>
 
VM parameters:


__TOC__
{| class="wikitable"
! Parameter !! x86_64 !! aarch64
|-
| Binary || <code>qemu-system-x86_64</code> || <code>qemu-system-aarch64</code>
|-
| Machine || (default) || <code>virt</code>
|-
| CPU || <code>x86_64</code> || <code>cortex-a57</code>
|-
| RAM || 4 GB || 4 GB
|-
| Cores || 1 || 1
|-
| VGA || qxl || virtio-gpu-pci
|-
| Network || User-mode (NAT) || User-mode (NAT)
|-
| Boot || CD-ROM || CD-ROM
|}
 
<code>qemu-system-x86_64</code> or <code>qemu-system-aarch64</code> must be installed and available in <code>PATH</code>.
 
== Cleaning the Build ==
 
Removes all build artefacts, clears the pacman package cache, unmounts any stale mounts, and deletes log files:
 
<pre>
sudo python clean.py
# or via make:
make clean
</pre>
 
What <code>clean.py</code> does in order:
# Reads <code>/proc/mounts</code> and unmounts (with <code>umount -l</code>) anything mounted inside <code>output/</code>, deepest paths first.
# Clears the pacman package cache: <code>find /var/cache/pacman/pkg/ -name '*.pkg.tar.*' -delete</code>.
# Removes all <code>logs/*.log</code> files.
# Removes all contents of <code>output/</code> with <code>sudo rm -rfv output/*</code>.
 
{{Warning|This is destructive. All previously built ISOs and repository packages will be deleted.}}
 
To remove only the chroot directories (without touching output):
 
<pre>
make chroot_clean
# or for a specific architecture:
make chroot_clean ARCH=x86_64_v3
</pre>
 
== Isolated Chroot Build ==
 
Repository builds use <code>makechrootpkg</code> from the <code>devtools</code> package to compile packages in an '''isolated chroot environment'''. ISO builds use a similar chroot isolation approach with <code>mkarchiso</code>. This prevents:
 
# '''Host pollution''' — packages cannot accidentally link against host libraries not listed in <code>makedepends</code>.
# '''Instruction leakage''' — packages are compiled with generic CFLAGS baseline, not host-specific optimizations.
# '''Non-reproducible builds''' — the build environment is deterministic and isolated from host updates.
 
=== Repository Chroot Lifecycle ===
 
# '''Bootstrap''': <code>mkarchroot &lt;chroot&gt;/root base-devel</code> creates a minimal Arch Linux root filesystem.
# '''Build''': <code>makechrootpkg -c -r &lt;chroot&gt;</code> builds the package inside the chroot. The <code>-c</code> flag creates a copy-on-write snapshot, so the base chroot remains clean.
# '''Cleanup''': After each build, the snapshot is discarded. The base chroot persists for reuse.
 
=== ISO Chroot Lifecycle ===
 
# '''Bootstrap''': <code>mkarchroot &lt;chroot&gt;/root base-devel archiso squashfs-tools dosfstools arch-install-scripts</code> creates a chroot with all ISO build tools.
# '''Mount''': Bind-mounts virtual filesystems and directories into the chroot:
#* <code>/proc</code> → <code>chroot/root/proc</code>
#* <code>/sys</code> → <code>chroot/root/sys</code>
#* <code>/dev</code> → <code>chroot/root/dev</code>
#* <code>/dev/pts</code> → <code>chroot/root/dev/pts</code>
#* Staging profile directory → <code>chroot/root/src</code>
#* Work directory → <code>chroot/root/work</code>
#* Output directory → <code>chroot/root/out</code>
# '''Build''': <code>chroot &lt;chroot&gt;/root /usr/bin/env -i PATH=... mkarchiso ...</code> runs inside the chroot with sanitized environment.
# '''Cleanup''': All bind mounts are unmounted (deepest first) on exit, error, or signal (SIGINT/SIGTERM). Mount cleanup is guaranteed via <code>try/finally</code>.
 
{{Note|ISO chroot isolation can be disabled with <code>--no-chroot</code> for debugging or when running in an already-isolated environment (e.g., Docker).}}
 
=== Dynamic Chroot Path ===
 
The chroot directory is resolved dynamically per architecture:
 
'''Repository builds:'''
# '''Explicit''': <code>ONIX_CHROOT</code> environment variable (Jenkins per-build isolation).
# '''Prefix-based''': <code>ONIX_CHROOT_PREFIX</code> + architecture + PID + timestamp (workspace-aware).
# '''Auto-generated''': <code>/tmp/onix-chroot-&lt;arch&gt;-&lt;pid&gt;-&lt;timestamp&gt;</code> (unique per process).
 
'''ISO builds:'''
# '''Explicit''': <code>ONIX_ISO_CHROOT</code> environment variable.
# '''Prefix-based''': <code>ONIX_CHROOT_PREFIX</code> + architecture + PID + timestamp.
# '''Auto-generated''': <code>/tmp/onix-iso-chroot-&lt;arch&gt;-&lt;pid&gt;-&lt;timestamp&gt;</code> (unique per process).
 
Example for Jenkins:
 
<pre>
export ONIX_CHROOT="/var/lib/jenkins/onix-chroot-${BUILD_NUMBER}"
python build.py repos
</pre>
 
=== Manual Chroot Management ===
 
Bootstrap chroots for all architectures:
 
<pre>
make chroot_bootstrap
</pre>
 
Bootstrap a specific architecture:
 
<pre>
make chroot_bootstrap ARCH=aarch64
</pre>
 
Spawn a shell inside a chroot (requires <code>ONIX_CHROOT</code> to be set):
 
<pre>
export ONIX_CHROOT=/tmp/onix-chroot
make chroot_bootstrap ARCH=x86_64
make chroot_shell ARCH=x86_64
</pre>
 
== PKGBUILD Sanitization ==
 
Before building each package, the build system '''sanitizes''' the <code>PKGBUILD</code> file to prevent instruction leakage:
 
# Searches for hardcoded <code>-march=native</code>, <code>-march=haswell</code>, <code>-mavx2</code>, etc.
# Rewrites them to the architecture-specific baseline (e.g., <code>-march=x86-64</code> for <code>x86_64</code>).
# Strips explicit AVX/SSE4 feature flags (<code>-mavx</code>, <code>-mfma</code>, <code>-msse4.2</code>).
# Backs up the original <code>PKGBUILD</code> to <code>PKGBUILD.bak</code> (idempotent — only first run).


== About ==
This ensures packages compiled on a modern build server (e.g., with AVX-512) will run on older CPUs without triggering <code>Illegal instruction</code> errors.


OnixOS is built on top of Arch Linux and inherits its rolling-release model and package ecosystem.
=== Sanitization Patterns ===


{| class="wikitable"
{| class="wikitable"
! Property !! Value
! Pattern !! Replacement (x86_64) !! Replacement (x86_64_v3)
|-
|-
| Architecture || x86_64
| <code>-march=native</code> || <code>-march=x86-64</code> || <code>-march=x86-64-v3</code>
|-
|-
| Base || Arch Linux (rolling release)
| <code>-march=haswell</code> || <code>-march=x86-64</code> || <code>-march=x86-64-v3</code>
|-
|-
| Language || O Language (functional, system-level)
| <code>-march=skylake</code> || <code>-march=x86-64</code> || <code>-march=x86-64-v3</code>
|-
|-
| Installer || Calamares (graphical)
| <code>-mtune=native</code> || <code>-mtune=generic</code> || <code>-mtune=generic</code>
|-
|-
| Kernel || linux-onix (custom patched)
| <code>-mavx2</code> || (removed) || (removed)
|-
|-
| Desktop variants || Core, GNOME, Xfce, IceWM, Security
| <code>-mfma</code> || (removed) || (removed)
|-
|-
| License || Open Source
| <code>-msse4.2</code> || (removed) || (removed)
|}
|}


== Getting Started ==
=== Verification ===


=== Download ===
Test the sanitizer with a synthetic PKGBUILD:


ISO images are available on SourceForge and are built automatically by the OnixOS Build System.
<pre>
make test-sanitize ARCH=x86_64
</pre>


* [[Download]] — Latest ISO images
== GPG Signing ==
* [https://sourceforge.net/projects/onixos SourceForge Project Page]


=== Installation ===
Each built package is signed with a GPG detach-signature before being added to the repository database.


Boot from the downloaded ISO and launch the '''Calamares''' graphical installer. For advanced setups refer to [[Documentation]].
=== Signing Configuration ===


== Documentation ==
Create <code>sources/signing.conf</code> (example):


* [[OnixOS:Building From Scratch]] — Full build system guide (repos, ISO, Docker, QEMU)
<pre>
* [[Documentation]] — General documentation index
[sign]
* [[O Language]] — O Language reference and tutorials
key_id = "FINGERPRINT_OR_EMAIL_OF_SIGNING_KEY"
passphrase_env = "ONIX_GPG_PASSPHRASE"
</pre>


== Development ==
=== Key Setup ===


OnixOS is developed openly. Contributions are welcome via the project GitLab repositories.
Import the private key into the GPG keyring:


* Build system: [[OnixOS:Building From Scratch]]
<pre>
* Package sources: <code>sources/packages/</code> in the build repository
export GNUPGHOME=/gnupg  # or ~/.gnupg for local builds
* ISO profiles: <code>sources/profiles/</code> in the build repository
gpg --import <private-key.asc>
gpg --edit-key <key_id>
  > trust
  > 5
  > save
</pre>


== Sources ==
=== Environment Variables ===


{| class="wikitable"
{| class="wikitable"
! Resource !! Link
! Variable !! Description
|-
|-
| Project page || [https://sourceforge.net/projects/onixos SourceForge]
| <code>GNUPGHOME</code> || Path to GPG keyring directory (default: <code>/gnupg</code> in Docker)
|-
|-
| OnixOS source || [https://gitlab.com/onix-os GitLab - OnixOS]
| <code>ONIX_SIGN_KEY</code> || GPG key ID or email to use with <code>--local-user</code>
|-
|-
| O Language source || [https://gitlab.com/olanguage/ GitLab - O Language]
| <code>ONIX_GPG_PASSPHRASE</code> || Passphrase for the signing key (batch mode with <code>--pinentry-mode loopback</code>)
|}
 
=== Verification ===
 
Verify signatures on built packages:
 
<pre>
make test-sign ARCH=x86_64
</pre>
 
== Testing ==
 
The build system includes a comprehensive test suite to verify package integrity.
 
=== Test Targets ===
 
{| class="wikitable"
! Target !! Description
|-
| <code>make test</code> || Run all tests for all architectures
|-
| <code>make test ARCH=x86_64_v3</code> || Run all tests for specific architecture
|-
| <code>make test-march</code> || Verify compiled binaries use generic <code>-march</code> baseline (no instruction leakage)
|-
|-
| Contributors || [[OnixOS:Contributors]]
| <code>make test-sign</code> || Verify GPG signatures exist and validate
|-
| <code>make test-deps</code> || Check for missing shared-object dependencies (host makedepends leak detection)
|-
| <code>make test-sanitize</code> || Verify PKGBUILD sanitizer rewrites flags correctly
|-
| <code>make test-smoke</code> || End-to-end smoke test: build single package, verify integrity
|}
|}
=== test-march ===
Inspects every ELF binary in built packages using <code>readelf -n</code> to verify the <code>march</code> value matches the architecture baseline. Fails if binaries were compiled with host-specific optimizations (e.g., <code>-march=skylake</code> on an <code>x86_64</code> build).
=== test-sign ===
Verifies that every <code>.pkg.tar.zst</code> file has a corresponding <code>.pkg.tar.zst.sig</code> file and that the signature validates against the GPG keyring.
=== test-deps ===
Extracts each package, runs <code>ldd</code> on ELF binaries, and checks for "not found" shared libraries. Detects cases where a package was compiled against host libraries not listed in <code>makedepends</code> (chroot isolation failure).
=== test-sanitize ===
Creates a synthetic <code>PKGBUILD</code> with instruction leakage (<code>-march=native</code>, <code>-mavx2</code>, etc.), runs the sanitizer, and verifies:
# Flags were rewritten to the architecture baseline.
# Original <code>PKGBUILD</code> was backed up to <code>PKGBUILD.bak</code>.
# No leaked flags remain in the sanitized file.
=== test-smoke ===
Builds a single small AUR package (e.g., <code>lolcat</code>) end-to-end in a fresh chroot, then runs <code>test-march</code> on the result. Useful as a fast CI gate before building the full ~1000-package list.
=== Example Test Run ===
<pre>
# Build repos for x86_64_v3
make repos ARCH=x86_64_v3
# Run all tests
make test ARCH=x86_64_v3
# Or run individual tests
make test-march ARCH=x86_64_v3
make test-sign ARCH=x86_64_v3
make test-deps ARCH=x86_64_v3
</pre>
== Output Structure ==
After a successful full build (all architectures):
<pre>
output/
├── base/
│  ├── x86_64/
│  │  └── repo/              # onix-base repository for x86_64
│  ├── x86_64_v2/
│  │  └── repo/              # onix-base repository for x86_64_v2
│  ├── x86_64_v3/
│  │  └── repo/              # onix-base repository for x86_64_v3
│  ├── i686/
│  │  └── repo/              # onix-base repository for i686
│  ├── aarch64/
│  │  └── repo/              # onix-base repository for aarch64
│  ├── armv7h/
│  │  └── repo/              # onix-base repository for armv7h
│  └── armv6h/
│      └── repo/              # onix-base repository for armv6h
├── aur/
│  ├── x86_64/
│  │  └── repo/              # onix-aur repository for x86_64
│  ├── x86_64_v2/
│  │  └── repo/
│  ├── x86_64_v3/
│  │  └── repo/
│  ├── i686/
│  │  └── repo/
│  ├── aarch64/
│  │  └── repo/
│  ├── armv7h/
│  │  └── repo/
│  └── armv6h/
│      └── repo/
└── iso/
    ├── core/
    │  ├── x86_64/
    │  │  ├── work/              # mkarchiso working directory (temporary)
    │  │  └── output/            # core-*.iso (x86_64)
    │  └── aarch64/
    │      ├── work/
    │      └── output/            # core-*.iso (aarch64)
    ├── gnome/
    │  ├── x86_64/
    │  │  ├── work/
    │  │  └── output/            # gnome-*.iso (x86_64)
    │  └── aarch64/
    │      ├── work/
    │      └── output/            # gnome-*.iso (aarch64)
    ├── xfce/
    │  ├── x86_64/
    │  └── aarch64/
    ├── icewm/
    │  ├── x86_64/
    │  └── aarch64/
    ├── security/
    │  ├── x86_64/
    │  └── aarch64/
    ├── kurumsal/
    │  ├── firewall/
    │  │  ├── work/
    │  │  └── output/            # kurumsal-firewall-*.iso
    │  ├── server/
    │  │  ├── work/
    │  │  └── output/            # kurumsal-server-*.iso
    │  └── desktop/
    │      ├── work/
    │      └── output/            # kurumsal-desktop-*.iso
    └── odesk/
        ├── work/
        └── output/                # odesk-*.iso
</pre>
Each repository directory contains:
* <code>*.pkg.tar.zst</code> — built packages
* <code>*.pkg.tar.zst.sig</code> — GPG detach-signatures
* <code>onix-base.db.tar.gz</code> or <code>onix-aur.db.tar.gz</code> — repository database
* <code>onix-base.files.tar.gz</code> or <code>onix-aur.files.tar.gz</code> — file list database
== Logs ==
Build logs are written to <code>logs/&lt;arch&gt;/</code> in the repository root (per-architecture):
<pre>
logs/
├── x86_64/
│  ├── chroot-bootstrap.log  # mkarchroot output (repo builds)
│  ├── iso-chroot-bootstrap.log  # mkarchroot output (ISO chroot)
│  ├── isobuild.log          # mkarchiso output (ISO builds)
│  ├── pacinst.log            # pacman dependency installation
│  ├── &lt;package&gt;.log          # makechrootpkg output per package
│  ├── sign.log              # GPG signing output
│  ├── repoadd.log            # repo-add output
│  └── copy.log              # package copy output
├── x86_64_v2/
│  └── ...
├── x86_64_v3/
│  └── ...
├── i686/
│  └── ...
├── aarch64/
│  ├── chroot-bootstrap.log
│  ├── iso-chroot-bootstrap.log
│  ├── isobuild.log
│  └── ...
├── armv7h/
│  └── ...
├── armv6h/
│  └── ...
└── clean.log                  # clean operation output
</pre>
Each log entry records the command, date/time, exit code, stderr, and stdout.

Revision as of 16:32, 9 August 2026

Building From Scratch

This document covers the complete build lifecycle of the OnixOS Build System — from environment setup and package repository builds to ISO creation, syncing, Docker-based builds, and live build monitoring.

Prerequisites

The build system runs on Arch Linux (or an Arch-based environment). The following packages are required for ISO builds and are installed automatically by build.py:

Package Purpose
archiso Core ISO creation tool (mkarchiso)
base-devel Compiler toolchain
devtools makechrootpkg, mkarchroot for isolated chroot builds
arch-install-scripts pacstrap, arch-chroot
squashfs-tools SquashFS filesystem creation
dosfstools FAT filesystem creation
syslinux Bootloader
git / git-lfs Source fetching
rsync File synchronisation
go Go language toolchain
nasm Assembler
ninja Build system
gperf Hash function generator
imagemagick Image processing
python-sphinx / xmlto Documentation toolchain
gnome-shell / qtcreator Desktop environment build deps
gnupg GPG package signing
qemu-user-static Cross-architecture emulation (ARM builds on x86_64 host)
binfmt-support Binary format registration for qemu-user-static
sudo Privilege escalation

For repository builds the system additionally uses makepkg (included in base-devel) and the AUR snapshot endpoint: 

https://aur.archlinux.org/cgit/aur.git/snapshot

Project Structure

onix-build-system/
├── build.py                  # Main build entry point
├── clean.py                  # Cleans output/ and mounts
├── watcher.py                # Terminal build status monitor
├── qemu.py                   # Launch built ISO in QEMU
├── Makefile                  # Convenience make targets
├── buildkit-docker/
│   ├── Dockerfile            # Build container image
│   ├── entrypoint.sh         # Docker entrypoint (chroot bootstrap)
│   ├── conf/
│   │   └── makepkg.conf      # Generic x86-64 baseline CFLAGS
│   ├── docker-compose.yml        # Dev shell container
│   ├── docker-compose.iso.yml    # Automated ISO build container
│   └── docker-compose.repos.yml  # Automated repo build container
├── onixos/
│   ├── archconfig.py         # Multiarch configuration (CFLAGS, chroot, qemu)
│   ├── iso.py                # ISO build logic (mkarchiso wrapper)
│   ├── pkgbuild.py           # PKGBUILD / .SRCINFO parser
│   ├── proc.py               # Subprocess wrapper with logging
│   └── repository.py         # Repository build logic (chroot, sanitize, sign)
├── sources/
│   ├── packages/             # Local package sources (PKGBUILD)
│   ├── profiles/             # archiso profiles (core, gnome, xfce, icewm, security, kurumsal, odesk)
│   ├── packages.list         # Local packages to build
│   ├── aur.list              # AUR packages to build
│   ├── profiles.list         # ISO profiles to build
│   ├── signing.conf          # GPG signing configuration (example)
│   ├── sync/                 # Repo upload config
│   │   ├── sourceforge.conf
│   │   └── local.conf
│   └── isosync/              # ISO upload config
│       ├── sourceforge.conf
│       └── local.conf
├── output/                   # All build artefacts (generated, per-arch)
└── logs/                     # Build logs (generated, per-arch)

Build Commands

All commands must be run from the repository root.

Full Build

Builds all repositories and all ISO profiles in sequence: 

sudo python build.py

Repository Build

Builds the onix-base (local packages) and onix-aur (AUR packages) repositories for all supported architectures by default: 

python build.py repos

To build for a specific architecture: 

python build.py repos --arch=x86_64_v3

The build process per architecture:

  1. Bootstraps a clean chroot with base-devel (isolated from host system).
  2. Generates architecture-specific makepkg.conf inside the chroot (generic CFLAGS baseline).
  3. Sanitizes PKGBUILD files to prevent instruction leakage (-march=native → architecture baseline).
  4. Builds packages using makechrootpkg -c -r <chroot> (host isolation, dependency resolution inside chroot).
  5. Signs each .pkg.tar.zst with GPG detach-signature (if signing key is configured).
  6. Generates repository database with repo-add.

Supported architectures:

Architecture CFLAGS Baseline Description QEMU Required
x86_64 -march=x86-64 Generic x86-64 (2004+, all CPUs) No
x86_64_v2 -march=x86-64-v2 SSE4.2, POPCNT (2008+: Nehalem, Bulldozer) No
x86_64_v3 -march=x86-64-v3 AVX, AVX2, BMI1/2, FMA (2015+: Haswell, Zen) No
i686 -march=i686 32-bit x86 (Pentium Pro+) No
aarch64 -march=armv8-a 64-bit ARM (ARMv8-A) Yes
armv7h -march=armv7-a -mfloat-abi=hard 32-bit ARM hard-float (ARMv7-A) Yes
armv6h -march=armv6 -mfloat-abi=hard 32-bit ARM hard-float (ARMv6, Raspberry Pi 1) Yes

Template:Note

ISO Build

Builds ISO images for all ISO-supported architectures (x86_64 and aarch64) and every profile listed in sources/profiles.list: 

sudo python build.py iso

To build for a specific architecture: 

sudo python build.py iso --arch=aarch64

To build without chroot isolation (direct mkarchiso invocation, useful for debugging): 

sudo python build.py iso --no-chroot

The build process per profile per architecture:

  1. Installs ISO build dependencies via pacman.
  2. Creates a staging copy of the profile with architecture-specific overrides:
    • Rewrites profiledef.sh with the target architecture and boot modes.
    • Selects packages.<arch> as the package list.
    • Generates architecture-specific pacman.conf (with $arch repo URLs).
    • Removes syslinux/ for aarch64 (UEFI-only).
  3. When chroot isolation is enabled (default):
    1. Bootstraps a clean chroot with base-devel, archiso, squashfs-tools, dosfstools, and arch-install-scripts.
    2. Bind-mounts /proc, /sys, /dev, /dev/pts, source profile, work directory, and output directory into the chroot.
    3. Runs mkarchiso inside the chroot with sanitized environment variables.
    4. Unmounts all bind mounts (deepest first) on exit, error, or signal (SIGINT/SIGTERM).
  4. When chroot isolation is disabled (--no-chroot):
    1. Unmounts any stale mounts from a previous failed build.
    2. Cleans the work directory.
    3. Runs mkarchiso directly on the host.
  5. Writes the finished .iso file to output/iso/<profile>/<arch>/output/.

ISO-supported architectures:

Architecture Boot Modes Syslinux Notes
x86_64 BIOS (syslinux) + UEFI (systemd-boot) Yes Standard x86-64 ISO
aarch64 UEFI only (systemd-boot) No ARM64, excludes x86-only packages (microcode, video drivers)

Template:Note

Upload / Sync

Uploads built repositories to the configured remote server(s): 

python build.py upload

Uploads built ISO images: 

python build.py iso-sync

Both commands read from the relevant *.conf files under sources/sync/ and sources/isosync/ respectively (see Sync Configuration).

Make Targets

Core Build Targets

make            # clean → repos → iso (full build, all architectures)
make clean      # Remove output artefacts and unmount
make repos      # Repository build for all architectures
make repos ARCH=x86_64_v3  # Repository build for specific architecture
make iso        # ISO build for all architectures (x86_64 + aarch64)
make iso ARCH=x86_64       # ISO build for specific architecture
make iso-all    # ISO build for all architectures (explicit)
make iso-nochroot          # ISO build without chroot isolation
make iso-nochroot ARCH=aarch64  # ISO build without chroot for specific architecture

Chroot Management

make chroot_bootstrap              # Bootstrap chroots for all architectures
make chroot_bootstrap ARCH=x86_64  # Bootstrap chroot for specific architecture
make chroot_clean                  # Remove all chroots
make chroot_clean ARCH=x86_64      # Remove chroot for specific architecture
make chroot_shell ARCH=x86_64      # Spawn shell inside chroot (requires ONIX_CHROOT)

Testing

make test                    # Run all tests for all architectures
make test ARCH=x86_64_v3     # Run all tests for specific architecture
make test-march              # Verify -march baseline in compiled binaries
make test-sign               # Verify GPG signatures
make test-deps               # Check for missing shared-object dependencies
make test-sanitize           # Verify PKGBUILD sanitizer rewrites flags
make test-smoke              # End-to-end smoke test (build single package)

Docker Targets

make docker_build            # Build the Docker image
make docker_publish          # Push image to registry (olproject/buildkit)
make docker_pull             # Pull image from registry
make docker_shell            # Open interactive shell in container

make docker_iso_build        # Run ISO build inside Docker for all architectures
make docker_iso_build ARCH=aarch64  # Run ISO build for specific architecture
make docker_repo_build       # Run repo build inside Docker for all architectures
make docker_repo_build ARCH=aarch64  # Run repo build for specific architecture
make docker_test             # Build repos in Docker, then run tests locally

Environment Variables

Variable Default Description
ARCH (all) Target architecture for make targets
ONIX_ARCH x86_64 Architecture (used by entrypoint.sh and docker-compose)
ONIX_CHROOT (auto) Explicit chroot path for repo builds (Jenkins per-build)
ONIX_ISO_CHROOT (auto) Explicit chroot path for ISO builds
ONIX_CHROOT_PREFIX /tmp/onix-chroot Chroot path prefix (auto-suffix with arch+pid+ts)
ONIX_SIGN_KEY (default key) GPG key ID / email for signing
ONIX_GPG_PASSPHRASE (none) GPG passphrase for batch mode

Docker Builds

The Docker image is based on archlinux and ships with all build dependencies pre-installed, including devtools (for makechrootpkg), qemu-user-static (for ARM emulation), and gnupg (for package signing). The image tag is olproject/buildkit. All Docker-related files are located under buildkit-docker/.

Building the Image

docker build -f buildkit-docker/Dockerfile . -t olproject/buildkit
# or via make:
make docker_build

The Dockerfile installs:

  • base-devel, devtools — compiler toolchain and makechrootpkg
  • qemu-user-static, binfmt-support — ARM cross-architecture emulation
  • gnupg — GPG package signing
  • makepkg.conf with generic x86-64 baseline CFLAGS

ISO Build via Docker

Builds ISO images for all ISO-supported architectures by default: 

make docker_iso_build

To build for a specific architecture: 

make docker_iso_build ARCH=aarch64

The make target automatically:

  1. Runs the Docker container with ONIX_ARCH set.
  2. Copies output/ and logs/ from the container.
  3. Removes the container.
  4. Iterates over all ISO architectures if no ARCH is specified.

The container entrypoint (entrypoint.sh) automatically:

  1. Resolves the chroot directory (from ONIX_CHROOT or generates dynamic path).
  2. Tears down any existing chroot and bootstraps a clean one with mkarchroot.
  3. Ensures GNUPGHOME directory exists.
  4. Passes --arch parameter to python build.py iso.

Repository Build via Docker

Builds repositories for all architectures by default: 

docker-compose -f buildkit-docker/docker-compose.repos.yml up --abort-on-container-exit --build
docker cp buildkit:/home/buildkit/builder/output/. ./output/
docker cp buildkit:/home/buildkit/builder/logs/. ./logs/
docker rm buildkit
# or via make:
make docker_repo_build

To build for a specific architecture: 

ONIX_ARCH=x86_64_v3 docker-compose -f buildkit-docker/docker-compose.repos.yml up --abort-on-container-exit --build
# or via make:
make docker_repo_build ARCH=x86_64_v3

The container entrypoint (entrypoint.sh) automatically:

  1. Resolves the chroot directory (from ONIX_CHROOT or generates dynamic path).
  2. Tears down any existing chroot and bootstraps a clean one with mkarchroot.
  3. Ensures GNUPGHOME directory exists.
  4. Passes --arch parameter to python build.py repos or python build.py iso.

Interactive Shell

Starts a persistent container with output and logs mounted as volumes, useful for manual inspection or debugging: 

docker-compose -f buildkit-docker/docker-compose.yml up -d
docker exec -it buildkit bash
# or via make:
make docker_shell

The container mounts:

  • ./output/ → /home/buildkit/builder/output
  • ./logs/ → /home/buildkit/builder/logs
  • gnupg volume → /gnupg (GPG keyring)

Template:Note

Profiles

ISO profiles are listed in sources/profiles.list, one per line. Each profile corresponds to a directory under sources/profiles/ that contains a complete archiso profile.

Each profile directory contains architecture-specific files:

File Purpose
profiledef.sh ISO metadata (name, label, version, boot modes, arch) — rewritten at build time for target architecture
pacman.conf Pacman config for ISO build (with $arch repo URLs)
pacman.aarch64.conf Pacman config for aarch64 (no multilib, Architecture = aarch64)
packages.x86_64 Package list for x86_64
packages.aarch64 Package list for aarch64 (excludes x86-only packages)
airootfs/ Files to overlay onto the ISO root filesystem
syslinux/ Syslinux bootloader configs (x86_64 only, removed for aarch64)
efiboot/ EFI boot files (per-architecture entries)

Available profiles:

[Profiles are now maintained in an external repository](https://gitlab.com/onix-os/onixos-profiles). The repository contains the same profile directories as before. You can clone it and point the build system to the directory.


Profile Description
core Minimal base system
gnome GNOME desktop environment
xfce Xfce desktop environment
icewm IceWM lightweight desktop
security Security-focused variant
odesk Office desktop environment


To build only a specific profile, edit sources/profiles.list temporarily or call ISO.mkarchiso() directly.

Packages

Local Packages (onix-base)

Defined in sources/packages.list. Sources live under sources/packages/<name>/:

Package Purpose
olang OnixOS language runtime
onix-base Base system metapackage
onix-graphics Graphics stack
onix-meta Meta package
calamares-onix Graphical installer (Qt6, git source, with branding)
linux-onix-zen OnixOS ZEN kernel (7.1.7-zen1)
linux-onix-rt OnixOS real-time kernel
linux-onix-generic OnixOS generic kernel (Arch mainline 7.1.7.arch1)
plymouth-onix-boot Boot splash screen
openxenmanager Xen hypervisor management tool
onix-updater OnixOS system updater
mkinitcpio-openswap Encrypted swap support for mkinitcpio
ckbcomp Keyboard composition utility

External Packages Repository

The package definitions have been moved to an external repository:

[OnixOS Packages Repository](https://gitlab.com/onix-os/onixos-packages)

You can clone this repository and refer to it for package sources.


AUR Packages (onix-aur)

Defined in sources/aur.list. Downloaded from the AUR snapshot endpoint at build time. Output goes to output/aur/.

Sync Configuration

Both repository and ISO sync use SSH (rsync over SSH). Configuration files follow the standard configparser INI format.

Repository Sync

Config files read by build.py upload (both are used if they exist):

  • sources/sync/sourceforge.conf — SourceForge remote
  • sources/sync/local.conf — Local/internal server

Example: 

[sync]
server  = frs.sourceforge.net
user    = olproject
path    = /home/frs/project/onixos/Repo
port    = 22
keyfile = /home/ted/.ssh/id_ed25519

Uploads output/base/<arch>/repo → <path>/base/<arch> and output/aur/<arch>/repo → <path>/aur/<arch> for each architecture.

Example remote structure: 

/home/frs/project/onixos/Repo/
├── base/
│   ├── x86_64/
│   │   ├── onix-base.db.tar.gz
│   │   ├── package1.pkg.tar.zst
│   │   └── package1.pkg.tar.zst.sig
│   ├── x86_64_v2/
│   ├── x86_64_v3/
│   ├── i686/
│   ├── aarch64/
│   ├── armv7h/
│   └── armv6h/
└── aur/
    ├── x86_64/
    ├── x86_64_v2/
    └── ...

ISO Sync

Config files read by build.py iso-sync:

  • sources/isosync/sourceforge.conf
  • sources/isosync/local.conf

Uploads output/iso/<profile>/<arch>/output → <path>/ISO/<arch> for each profile and architecture.

Example remote structure: 

/home/frs/project/onixos/Daily/ISO/
├── x86_64/
│   ├── onixos-core-*.iso
│   ├── onixos-gnome-*.iso
│   ├── onixos-xfce-*.iso
│   ├── onixos-icewm-*.iso
│   └── onixos-security-*.iso
└── aarch64/
    ├── onixos-core-*.iso
    ├── onixos-gnome-*.iso
    └── ...

Debug Mode

Append -d to iso or repos subcommands to stream stdout/stderr to the terminal in real time: 

sudo python build.py iso -d
sudo python build.py iso --arch=aarch64 -d
python build.py repos -d
python build.py repos --arch=x86_64_v3 -d

Without -d, command output is only written to the log files in logs/<arch>/.

Build Watcher

watcher.py provides a live terminal dashboard that tracks ISO and repository build status, active mounts, and recent log output. It refreshes every 3 seconds.

Standalone (curses UI): 

python watcher.py

Press q or Esc to exit.

Attached to a build (GUI overlay): 

sudo python build.py iso -gui
python build.py repos -gui

The -gui flag starts the watcher in a background thread alongside the build process and stops it automatically when the build finishes.

What the watcher shows:

  • ISO Builds — profile name, ISO filename, file size, build timestamp, architecture, and status. Scans per-arch directories (output/iso/<profile>/<arch>/output/).
  • Repositories — repo name, total package count across all architectures, DB update timestamp, and status. Scans per-arch directories (output/base/<arch>/repo/).
  • Active Mounts — any loop/overlay mounts still active inside output/.
  • Last Build Log — last 6 lines of logs/isobuild.log with colour-coded errors and warnings.

Testing with QEMU

Launch a built ISO in a QEMU virtual machine for quick validation: 

python qemu.py <path-to-iso> [--arch=<arch>]

Examples: 

python qemu.py output/iso/gnome/x86_64/output/onixos-gnome-x86_64.iso
python qemu.py output/iso/core/aarch64/output/onixos-core-aarch64.iso --arch=aarch64

VM parameters:

Parameter x86_64 aarch64
Binary qemu-system-x86_64 qemu-system-aarch64
Machine (default) virt
CPU x86_64 cortex-a57
RAM 4 GB 4 GB
Cores 1 1
VGA qxl virtio-gpu-pci
Network User-mode (NAT) User-mode (NAT)
Boot CD-ROM CD-ROM

qemu-system-x86_64 or qemu-system-aarch64 must be installed and available in PATH.

Cleaning the Build

Removes all build artefacts, clears the pacman package cache, unmounts any stale mounts, and deletes log files: 

sudo python clean.py
# or via make:
make clean

What clean.py does in order:

  1. Reads /proc/mounts and unmounts (with umount -l) anything mounted inside output/, deepest paths first.
  2. Clears the pacman package cache: find /var/cache/pacman/pkg/ -name '*.pkg.tar.*' -delete.
  3. Removes all logs/*.log files.
  4. Removes all contents of output/ with sudo rm -rfv output/*.

Template:Warning

To remove only the chroot directories (without touching output): 

make chroot_clean
# or for a specific architecture:
make chroot_clean ARCH=x86_64_v3

Isolated Chroot Build

Repository builds use makechrootpkg from the devtools package to compile packages in an isolated chroot environment. ISO builds use a similar chroot isolation approach with mkarchiso. This prevents:

  1. Host pollution — packages cannot accidentally link against host libraries not listed in makedepends.
  2. Instruction leakage — packages are compiled with generic CFLAGS baseline, not host-specific optimizations.
  3. Non-reproducible builds — the build environment is deterministic and isolated from host updates.

Repository Chroot Lifecycle

  1. Bootstrap: mkarchroot <chroot>/root base-devel creates a minimal Arch Linux root filesystem.
  2. Build: makechrootpkg -c -r <chroot> builds the package inside the chroot. The -c flag creates a copy-on-write snapshot, so the base chroot remains clean.
  3. Cleanup: After each build, the snapshot is discarded. The base chroot persists for reuse.

ISO Chroot Lifecycle

  1. Bootstrap: mkarchroot <chroot>/root base-devel archiso squashfs-tools dosfstools arch-install-scripts creates a chroot with all ISO build tools.
  2. Mount: Bind-mounts virtual filesystems and directories into the chroot:
    • /proc → chroot/root/proc
    • /sys → chroot/root/sys
    • /dev → chroot/root/dev
    • /dev/pts → chroot/root/dev/pts
    • Staging profile directory → chroot/root/src
    • Work directory → chroot/root/work
    • Output directory → chroot/root/out
  3. Build: chroot <chroot>/root /usr/bin/env -i PATH=... mkarchiso ... runs inside the chroot with sanitized environment.
  4. Cleanup: All bind mounts are unmounted (deepest first) on exit, error, or signal (SIGINT/SIGTERM). Mount cleanup is guaranteed via try/finally.

Template:Note

Dynamic Chroot Path

The chroot directory is resolved dynamically per architecture:

Repository builds:

  1. Explicit: ONIX_CHROOT environment variable (Jenkins per-build isolation).
  2. Prefix-based: ONIX_CHROOT_PREFIX + architecture + PID + timestamp (workspace-aware).
  3. Auto-generated: /tmp/onix-chroot-<arch>-<pid>-<timestamp> (unique per process).

ISO builds:

  1. Explicit: ONIX_ISO_CHROOT environment variable.
  2. Prefix-based: ONIX_CHROOT_PREFIX + architecture + PID + timestamp.
  3. Auto-generated: /tmp/onix-iso-chroot-<arch>-<pid>-<timestamp> (unique per process).

Example for Jenkins: 

export ONIX_CHROOT="/var/lib/jenkins/onix-chroot-${BUILD_NUMBER}"
python build.py repos

Manual Chroot Management

Bootstrap chroots for all architectures: 

make chroot_bootstrap

Bootstrap a specific architecture: 

make chroot_bootstrap ARCH=aarch64

Spawn a shell inside a chroot (requires ONIX_CHROOT to be set): 

export ONIX_CHROOT=/tmp/onix-chroot
make chroot_bootstrap ARCH=x86_64
make chroot_shell ARCH=x86_64

PKGBUILD Sanitization

Before building each package, the build system sanitizes the PKGBUILD file to prevent instruction leakage:

  1. Searches for hardcoded -march=native, -march=haswell, -mavx2, etc.
  2. Rewrites them to the architecture-specific baseline (e.g., -march=x86-64 for x86_64).
  3. Strips explicit AVX/SSE4 feature flags (-mavx, -mfma, -msse4.2).
  4. Backs up the original PKGBUILD to PKGBUILD.bak (idempotent — only first run).

This ensures packages compiled on a modern build server (e.g., with AVX-512) will run on older CPUs without triggering Illegal instruction errors.

Sanitization Patterns

Pattern Replacement (x86_64) Replacement (x86_64_v3)
-march=native -march=x86-64 -march=x86-64-v3
-march=haswell -march=x86-64 -march=x86-64-v3
-march=skylake -march=x86-64 -march=x86-64-v3
-mtune=native -mtune=generic -mtune=generic
-mavx2 (removed) (removed)
-mfma (removed) (removed)
-msse4.2 (removed) (removed)

Verification

Test the sanitizer with a synthetic PKGBUILD: 

make test-sanitize ARCH=x86_64

GPG Signing

Each built package is signed with a GPG detach-signature before being added to the repository database.

Signing Configuration

Create sources/signing.conf (example): 

[sign]
key_id = "FINGERPRINT_OR_EMAIL_OF_SIGNING_KEY"
passphrase_env = "ONIX_GPG_PASSPHRASE"

Key Setup

Import the private key into the GPG keyring: 

export GNUPGHOME=/gnupg  # or ~/.gnupg for local builds
gpg --import <private-key.asc>
gpg --edit-key <key_id>
  > trust
  > 5
  > save

Environment Variables

Variable Description
GNUPGHOME Path to GPG keyring directory (default: /gnupg in Docker)
ONIX_SIGN_KEY GPG key ID or email to use with --local-user
ONIX_GPG_PASSPHRASE Passphrase for the signing key (batch mode with --pinentry-mode loopback)

Verification

Verify signatures on built packages: 

make test-sign ARCH=x86_64

Testing

The build system includes a comprehensive test suite to verify package integrity.

Test Targets

Target Description
make test Run all tests for all architectures
make test ARCH=x86_64_v3 Run all tests for specific architecture
make test-march Verify compiled binaries use generic -march baseline (no instruction leakage)
make test-sign Verify GPG signatures exist and validate
make test-deps Check for missing shared-object dependencies (host makedepends leak detection)
make test-sanitize Verify PKGBUILD sanitizer rewrites flags correctly
make test-smoke End-to-end smoke test: build single package, verify integrity

test-march

Inspects every ELF binary in built packages using readelf -n to verify the march value matches the architecture baseline. Fails if binaries were compiled with host-specific optimizations (e.g., -march=skylake on an x86_64 build).

test-sign

Verifies that every .pkg.tar.zst file has a corresponding .pkg.tar.zst.sig file and that the signature validates against the GPG keyring.

test-deps

Extracts each package, runs ldd on ELF binaries, and checks for "not found" shared libraries. Detects cases where a package was compiled against host libraries not listed in makedepends (chroot isolation failure).

test-sanitize

Creates a synthetic PKGBUILD with instruction leakage (-march=native, -mavx2, etc.), runs the sanitizer, and verifies:

  1. Flags were rewritten to the architecture baseline.
  2. Original PKGBUILD was backed up to PKGBUILD.bak.
  3. No leaked flags remain in the sanitized file.

test-smoke

Builds a single small AUR package (e.g., lolcat) end-to-end in a fresh chroot, then runs test-march on the result. Useful as a fast CI gate before building the full ~1000-package list.

Example Test Run

# Build repos for x86_64_v3
make repos ARCH=x86_64_v3

# Run all tests
make test ARCH=x86_64_v3

# Or run individual tests
make test-march ARCH=x86_64_v3
make test-sign ARCH=x86_64_v3
make test-deps ARCH=x86_64_v3

Output Structure

After a successful full build (all architectures): 

output/
├── base/
│   ├── x86_64/
│   │   └── repo/              # onix-base repository for x86_64
│   ├── x86_64_v2/
│   │   └── repo/              # onix-base repository for x86_64_v2
│   ├── x86_64_v3/
│   │   └── repo/              # onix-base repository for x86_64_v3
│   ├── i686/
│   │   └── repo/              # onix-base repository for i686
│   ├── aarch64/
│   │   └── repo/              # onix-base repository for aarch64
│   ├── armv7h/
│   │   └── repo/              # onix-base repository for armv7h
│   └── armv6h/
│       └── repo/              # onix-base repository for armv6h
├── aur/
│   ├── x86_64/
│   │   └── repo/              # onix-aur repository for x86_64
│   ├── x86_64_v2/
│   │   └── repo/
│   ├── x86_64_v3/
│   │   └── repo/
│   ├── i686/
│   │   └── repo/
│   ├── aarch64/
│   │   └── repo/
│   ├── armv7h/
│   │   └── repo/
│   └── armv6h/
│       └── repo/
└── iso/
    ├── core/
    │   ├── x86_64/
    │   │   ├── work/              # mkarchiso working directory (temporary)
    │   │   └── output/            # core-*.iso (x86_64)
    │   └── aarch64/
    │       ├── work/
    │       └── output/            # core-*.iso (aarch64)
    ├── gnome/
    │   ├── x86_64/
    │   │   ├── work/
    │   │   └── output/            # gnome-*.iso (x86_64)
    │   └── aarch64/
    │       ├── work/
    │       └── output/            # gnome-*.iso (aarch64)
    ├── xfce/
    │   ├── x86_64/
    │   └── aarch64/
    ├── icewm/
    │   ├── x86_64/
    │   └── aarch64/
    ├── security/
    │   ├── x86_64/
    │   └── aarch64/
    ├── kurumsal/
    │   ├── firewall/
    │   │   ├── work/
    │   │   └── output/            # kurumsal-firewall-*.iso
    │   ├── server/
    │   │   ├── work/
    │   │   └── output/            # kurumsal-server-*.iso
    │   └── desktop/
    │       ├── work/
    │       └── output/            # kurumsal-desktop-*.iso
    └── odesk/
        ├── work/
        └── output/                # odesk-*.iso

Each repository directory contains:

  • *.pkg.tar.zst — built packages
  • *.pkg.tar.zst.sig — GPG detach-signatures
  • onix-base.db.tar.gz or onix-aur.db.tar.gz — repository database
  • onix-base.files.tar.gz or onix-aur.files.tar.gz — file list database

Logs

Build logs are written to logs/<arch>/ in the repository root (per-architecture): 

logs/
├── x86_64/
│   ├── chroot-bootstrap.log   # mkarchroot output (repo builds)
│   ├── iso-chroot-bootstrap.log  # mkarchroot output (ISO chroot)
│   ├── isobuild.log           # mkarchiso output (ISO builds)
│   ├── pacinst.log            # pacman dependency installation
│   ├── <package>.log          # makechrootpkg output per package
│   ├── sign.log               # GPG signing output
│   ├── repoadd.log            # repo-add output
│   └── copy.log               # package copy output
├── x86_64_v2/
│   └── ...
├── x86_64_v3/
│   └── ...
├── i686/
│   └── ...
├── aarch64/
│   ├── chroot-bootstrap.log
│   ├── iso-chroot-bootstrap.log
│   ├── isobuild.log
│   └── ...
├── armv7h/
│   └── ...
├── armv6h/
│   └── ...
└── clean.log                  # clean operation output

Each log entry records the command, date/time, exit code, stderr, and stdout.

Retrieved from "https://onix-project.com/index.php?title=Main_Page&oldid=134"