Compare commits

..
Author SHA1 Message Date
Oliver Smith c5fa233f37 CI: aports version check: modernize with new rules (MR 3615)
Update the script once more with the goal that we no longer need to
comment on pkgrel/pkgver bumps. Parse not only the version of the old
and new package, but also the checksums and remote sources.

Replace the previous check of simply having $pkgver-r$pkgrel higher
than the upstream APKBUILD with more detailed rules:

  APKBUILD has remote sources?      ------>       Did sha512sums change?
   |                                  No            |         |
   | Yes                                            | No      | Yes
   v                                                |         v
  Did the file name of a remote                     |     * Increase pkgver
  source change?                    ----.           |     * Set pkgrel=0
   |                                    |           |
   | Yes                                | No        |
   v                                    v           v
  Is pkgver=9999?         ----->   * Don't change pkgver
  (Forked from Alpine)     Yes     * Bump pkgrel by 1
   |
   | No
   v
  * Set the pkgver to the version
    of the source tarball
  * Set pkgrel=0

I'll also put this in the wiki at:
https://postmarketos.org/howto-bump-pkgrel-pkgver

Besides the new rules, print a detailed error message below each package
where the test failed that explains exactly what was expected and what
the reasoning behind that is. Also add a hint that users can run this
locally with 'pmbootstrap ci' now.

Use logging instead of print, so we get colored ERROR: and NOTE:
statements from pmbootstrap code, this should make it easier to read.
2022-11-11 08:35:11 +01:00
Oliver Smith c56d305101 CI: move parse_apkbuild_checksums to common (MR 3615)
Move parse_source_from_checksums() from .ci/testcases/test_source.py to
.ci/lib/common.py and give it the more fitting name
parse_apkbuild_checksums(). Create a symlink for common.py to make it
also available in .ci/lib/common.py.

This is in preparation for using it in
.ci/lib/check_changed_aports_versions.py in a future patch.
2022-11-11 08:35:11 +01:00
Oliver Smith 3e5d1e5767 CI: implement distfile-check as python test (MR 3615)
Reimplement the distfile-check as python test inside this git
repository, so we don't need to download a tarball of ci-common.git to
run it. This would not have been nice for running the test with
'pmbootstrap ci', as we don't want it to get downloaded every time the
test runs.

This new implementation is done in less than 40 lines of code, very fast
and doesn't need to create a lot of files in /tmp to build a
"distfiletree".
2022-11-11 08:35:11 +01:00
Oliver Smith 6629d977aa CI: make test_unreferenced_files more generic (MR 3615)
Prepare to add a new test making use of the
parse_source_from_checksums() function in test_unreferenced_files.py by
extending the function to not only return the checksum file names, but
also the checksums. Also rename the file from test_unreferenced_files.py
to test_source.py.
2022-11-11 08:35:10 +01:00
Oliver Smith 5aadba867d CI: move_logs.sh: put in .ci/lib (MR 3615) 2022-11-11 08:35:10 +01:00
Oliver Smith fa122fa7ef CI: kconfig adjust for 'pmbootstrap ci' (MR 3615) 2022-11-11 08:35:10 +01:00
Oliver Smith be67922f2b CI: adjust .ci/common.py users to pmbootstrap ci (MR 3615)
Adjust all CI scripts that make use of .ci/common.py to
'pmbootstrap ci'. Move all scripts that are not direct entry points to
running CI scripts to .ci/lib.

Comment out the dtb check, as it is failing. Apparently it didn't run
properly before. Let's fix this after this CI change is done.
2022-11-11 08:35:10 +01:00
Oliver Smith 2e8212ed7e CI: editor-config: adjust for 'pmbootstrap ci' (MR 3615) 2022-11-11 08:35:10 +01:00
Oliver Smith 42906590de CI: shellcheck: adjust for 'pmbootstrap ci' (MR 3615)
Split out the grep for CHANGEME into a separate .ci/grep.sh script, as
it's not related to shellcheck and also there's another grep check that
I'll move there in a follow-up patch.
2022-11-11 08:35:10 +01:00
Oliver Smith 7c9fabf58a CI: flake8: adjust for 'pmbootstrap ci' (MR 3615) 2022-11-11 08:35:10 +01:00
Oliver Smith 4604406755 CI: wiki: adjust for 'pmbootstrap ci' (MR 3615)
Add a wrapper .ci/wiki.sh, that pmbootstrap ci can run and use it in
gitlab-ci.yml. Add a .ci/lib/gitlab_prepare_ci.sh script that makes a
user available with sudo, as expected by the pmbootstrap ci compatible
scripts. Set up binfmt_misc too while at it, it will be needed for
running pmbootstrap.

The next patches will adjust the rest of the CI scripts and move the old
scripts in the .ci/lib dir, so only the scripts that should be called
directly are in the .ci dir.

Related: https://postmarketos.org/pmb-ci
2022-11-11 08:35:10 +01:00
Oliver Smith a6ba51552f .ci/check_devices_in_wiki: remove <> around url (MR 3615)
I added this around all urls in the beginning of postmarketOS, but now I
think it makes it less readable and doesn't have a benefit.
2022-11-11 08:35:09 +01:00
8681 changed files with 241700 additions and 941239 deletions

No files matched your search

+14 -4
View File
@@ -1,12 +1,22 @@
#!/bin/sh -e
# Description: run apkbuild-lint on modified APKBUILDs
# Options: with-dot-git
# https://postmarketos.org/pmb-ci
# Options: native
# Use 'native' because it requires git commit history.
# https://postmarktos.org/pmb-ci
if [ "$(id -u)" = 0 ]; then
set -x
apk add atools git python3
exec su "${TESTUSER:-build}" -c "sh -e $0"
wget "https://gitlab.com/postmarketOS/ci-common/-/raw/master/install_pmbootstrap.sh"
sh ./install_pmbootstrap.sh
exec su "${TESTUSER:-pmos}" -c "sh -e $0"
fi
# Wrap pmbootstrap to use this repository for --aports
pmaports="$(cd $(dirname $0)/..; pwd -P)"
_pmbootstrap="$(command -v pmbootstrap)"
pmbootstrap() {
"$_pmbootstrap" --aports="$pmaports" "$@"
}
.ci/lib/apkbuild_linting.py
-106
View File
@@ -1,106 +0,0 @@
#!/bin/sh -e
if [ "$(id -u)" = 0 ]; then
set -x
apk add git openssh-keygen grep curl
wget "https://gitlab.postmarketos.org/postmarketOS/ci-common/-/raw/master/install_pmbootstrap.sh"
sh ./install_pmbootstrap.sh
exec su "${TESTUSER:-pmos}" -c "sh -e $0 $*"
fi
git config --global --add safe.directory "$CI_PROJECT_DIR"
git config --global user.name "postmarketOS CI"
# The username for the bot associated with the access token
# e.g. project_226_bot_bfa19b84bf7f559c8ff7438a01fd29b9
git config --global user.email "$PMAPORTS_TOKEN_BOT@noreply.gitlab.postmarketos.org"
# Configure SSH key for signing
mkdir -p ~/.ssh
cp "$PMAPORTS_SSH_KEY" ~/.ssh/pmaports
chmod 600 ~/.ssh/pmaports
# Generate public key from private
ssh-keygen -t ecdsa -f ~/.ssh/pmaports -y > ~/.ssh/pmaports.pub
git config --global gpg.format ssh
git config --global user.signingkey ~/.ssh/pmaports.pub
export PYTHONUNBUFFERED=1
PKGS="$1"
if [ -z "$PKGS" ]; then
echo "Usage: $0 <packages>"
exit 1
fi
echo "Checking for new versions of packages: $PKGS"
update_linux_next() {
# Check for the latest -next tag
local latest
local pkgver
local new_pkgver
# Get the current version
# shellcheck source=/dev/null
pkgver=$(. device/testing/linux-next/APKBUILD; echo "$pkgver")
curl -s https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/refs/tags/ | grep -oE "<a href='.*'>(.*)</a>.*\.tar\.gz</a>" | awk -F'[<>]' '{print $3}' > /tmp/tags
latest=$(grep -v "v" < /tmp/tags | head -n1)
if [ -z "$latest" ]; then
echo "Failed to get the latest -next tag"
exit 1
fi
echo "Latest -next tag: $latest"
new_pkgver=$(grep "v" < /tmp/tags | head -n1)
if [ -z "$new_pkgver" ]; then
echo "Failed to get the latest mainline tag from atom feed, reusing current pkgver"
new_pkgver="${pkgver%_git*}"
else
echo "Latest mainline tag: $new_pkgver"
# new_pkgver is like 'v6.9-rc1' but we want '6.9'
new_pkgver=$(echo "$new_pkgver" | sed -e 's/^v//' -e 's/-.*//')
fi
new_pkgver="${new_pkgver}_git${latest#*-}"
if [ "$pkgver" = "$new_pkgver" ]; then
echo "No new version of linux-next"
exit 0
fi
echo "Updating linux-next from $pkgver to $new_pkgver..."
sed -i -e "s/pkgver=$pkgver/pkgver=$new_pkgver/" device/testing/linux-next/APKBUILD
sed -i -e "s/^pkgrel=.*/pkgrel=0/" device/testing/linux-next/APKBUILD
# Update the checksums
pmbootstrap checksum linux-next
# Ensure it builds
pmbootstrap --details-to-stdout build linux-next
# Commit
git add device/testing/linux-next/APKBUILD
git commit -S -m "linux-next: upgrade to $latest"
}
# For testing
#git remote add gitlab https://pmos-ci:"$PMAPORTS_PUSH_TOKEN"@gitlab.postmarketos.org/postmarketOS/pmaports-autoupdate-test.git
git remote add gitlab https://pmos-ci:"$PMAPORTS_PUSH_TOKEN"@gitlab.postmarketos.org/postmarketOS/pmaports.git
git fetch gitlab
git checkout --track gitlab/master
for pkg in $PKGS; do
case "$pkg" in
linux-next)
update_linux_next
;;
*)
echo "Unknown package: $pkg"
exit 1
;;
esac
done
git push
-20
View File
@@ -1,20 +0,0 @@
#!/bin/sh -e
# Copyright 2025 Pablo Correa Gomez
# SPDX-License-Identifier: GPL-3.0-or-later
# Description: create documentation with antora
# Options: with-dot-git
# Artifacts: build/site
# https://postmarketos.org/pmb-ci
# Install dependencies
if [ "$(id -u)" = 0 ]; then
set -x
apk -q add py3-pip make git
apk add dint --allow-untrusted --repository=https://mirror.postmarketos.org/postmarketos/master
exec su "${TESTUSER:-build}" -c "sh -e $0"
fi
cd docs/
make venv
make
-142
View File
@@ -1,142 +0,0 @@
image: alpine:latest
stages:
- build
- hardware tests
variables:
CI_TRON_TEMPLATE_PROJECT: &ci-tron-template-project postmarketOS/ci-common
CI_TRON_JOB_TEMPLATE_PROJECT_URL: $CI_SERVER_URL/$CI_TRON_TEMPLATE_PROJECT
CI_TRON_JOB_TEMPLATE_COMMIT: &ci-tron-template-commit 7c95b5f2d53533e8722abf57c73e558168e811f3
include:
- project: *ci-tron-template-project
ref: *ci-tron-template-commit
file: '/ci-tron/common.yml'
workflow:
rules:
- if: $CI_PIPELINE_SOURCE == "parent_pipeline"
.build:
# Create jobs, but do not run them unless this is
# overriden in jobs that extend it
stage: build
interruptible: true
before_script:
# upgrade is mostly needed for "qemu" runners that run edge based on a VM
# The other build jobs run natively in docker on latest alpine stable and
# those should always have whatever we need
- apk upgrade -U
- .ci/lib/gitlab_prepare_ci.sh
after_script:
- cp -r /home/pmos/.local/var/pmbootstrap/packages/ packages/ || true
- mkdir -p keys
- cp /home/pmos/.local/var/pmbootstrap/config_apk_keys/pmos@local*.pub keys/ || true
- .ci/lib/move_logs.sh $CI_PROJECT_DIR
artifacts:
expire_in: 1 week
paths:
- packages/
- keys/
timeout: 10 h
build:
extends: .build
parallel:
matrix:
{% for arch in archs %}
- ARCH: {{ arch }}
TAG: {{ archtag[arch] }}
{% endfor %}
{% if archs|length == 0 %}
- ARCH: placeholder
TAG: shared
{% endif %}
tags:
- $TAG
script:
# Needed because gitlab fails if no jobs are ran:
# - https://gitlab.com/gitlab-org/gitlab/-/issues/368248
{% if archs|length == 0 %}
- 'true'
{% else %}
- .ci/build-$ARCH.sh
{% endif %}
# Hardware testing
# Define here all the gitlab-ci snippets that could be common between the
# different packages
# Extends from this job if the the build job generates a kernel and initramfs
# artifact for your target device. This will not work if your device generates
# boot images.
.device-boot-flow-separate-artifacts:
variables:
CI_TRON_KERNEL__URL: "glartifact://${CI_TRON__PMB_BUILD_JOB}/${CI_TRON__PMB_EXPORT_PATH}/${KERNEL_NAME}"
CI_TRON_INITRAMFS__INITRAMFS__URL: "glartifact://${CI_TRON__PMB_BUILD_JOB}/${CI_TRON__PMB_EXPORT_PATH}/initramfs"
# NOTE: For fastboot devices, we will need to use something like this
# .ci-tron-device-boot-flow-fastboot-image:
# variables:
# CI_TRON_FASTBOOT__BOOT_IMAGE: "glartifact://${CI_TRON__PMB_BUILD_JOB}/${CI_TRON__PMB_EXPORT_PATH}/bootimg"
# Include all the devices that need and can to be tested
{% for device in devices_under_test if device.is_present_in_ci %}
prepare-{{ device.name }}:
stage: hardware tests
extends:
.pmos-ci-tron-build-boot-artifacts
{%- if device.has_kernel_variants %}
# Let's build all the boot artifacts for all the supported kernels in
# different jobs, but hide them behind a single job name so as not to pollute
# the pipeline.
parallel:
matrix:
- KERNEL_VARIANT:
{%- for kernel in device.kernels %}
- {{ kernel }}
{%- endfor %}
{%- endif %}
needs:
- job: "build"
parallel:
matrix:
- ARCH: "{{ device.arch }}"
TAG: "{{ archtag[device.arch] }}"
variables:
DEVICE_NAME: {{ device.name }}
# NOTE: All the packages that may influence testing should have an
# `install-if postmarketos-mkinitfs-hook-ci`, so no need to add them here
INSTALL_PACKAGES: {{ device.pkgname }} {{ device.pkgname }}-kernel-${KERNEL_VARIANT} postmarketos-mkinitfs-hook-ci
.test-{{ device.name }}:
stage: hardware tests
extends:
- .pmos-ci-tron-initramfs-test
{%- if device.has_kernel_variants %}
# Let's test all the kernels in different jobs, but hide them behind a single
# job name so as not to pollute the pipeline.
parallel:
matrix:
- KERNEL_VARIANT:
{%- for kernel in device.kernels %}
- {{ kernel }}
{%- endfor %}
{%- endif %}
dependencies: []
needs:
- job: 'prepare-{{ device.name }}'
artifacts: false
# NOTE: We can't wait for a particular job in the matrix until GitLab implements support for it:
# https://gitlab.com/gitlab-org/gitlab/-/issues/423553
variables:
CI_TRON__PMB_BUILD_JOB: "prepare-{{ device.name }}{% if device.has_kernel_variants %}: [$KERNEL_VARIANT]{% endif %}"
CI_TRON_KERNEL_CMDLINE__DEVICEINFO: '{{ device.deviceinfo.kernel_cmdline }}'
# Include the gitlab ci yml fragment, as found in
# `device/$category/device-$devicename/gitlab-ci.yml.j2` which defines the jobs
{{ device.gitlab_ci_fragment }}
{% endfor %}
-1
View File
@@ -1 +0,0 @@
lib/build_changed_aports.sh
+3 -3
View File
@@ -1,17 +1,17 @@
#!/bin/sh -e
# Copyright 2023 Oliver Smith
# Copyright 2022 Oliver Smith
# SPDX-License-Identifier: GPL-3.0-or-later
# Description: check pkgver/pkgrel bumps, amount of changed pkgs etc
# Options: native
# Use 'native' because it requires git commit history.
# https://postmarketos.org/pmb-ci
# https://postmarktos.org/pmb-ci
if [ "$(id -u)" = 0 ]; then
set -x
# In .gitlab-ci.yml currently .ci/pytest.sh runs before this and
# already downloads and runs install_pmbootstrap.sh.
if ! [ -e install_pmbootstrap.sh ]; then
wget "https://gitlab.postmarketos.org/postmarketOS/ci-common/-/raw/master/install_pmbootstrap.sh"
wget "https://gitlab.com/postmarketOS/ci-common/-/raw/master/install_pmbootstrap.sh"
sh ./install_pmbootstrap.sh
fi
exec su "${TESTUSER:-pmos}" -c "sh -e $0"
-13
View File
@@ -1,13 +0,0 @@
#!/bin/sh -e
# Description: Validate and lint deviceinfo files
# Options: with-dot-git
# https://postmarketos.org/pmb-ci
if [ "$(id -u)" = 0 ]; then
set -x
apk add git python3
apk add dint --allow-untrusted --repository=https://mirror.postmarketos.org/postmarketos/master
exec su "${TESTUSER:-build}" -c "sh -e $0"
fi
.ci/lib/deviceinfo_linting.py
+1 -1
View File
@@ -1,6 +1,6 @@
#!/bin/sh -e
# Description: editorconfig-checker: lint for trailing whitespaces etc.
# https://postmarketos.org/pmb-ci
# https://postmarktos.org/pmb-ci
if [ "$(id -u)" = 0 ]; then
set -x
Executable
+14
View File
@@ -0,0 +1,14 @@
#!/bin/sh -e
# Description: lint CI related python scripts/tests
# https://postmarktos.org/pmb-ci
if [ "$(id -u)" = 0 ]; then
set -x
apk -q add py3-flake8
exec su "${TESTUSER:-build}" -c "sh -e $0"
fi
set -x
# shellcheck disable=SC2046
flake8 --ignore E501,F401,E722,W504,W605 $(find .ci -name '*.py')
+10 -99
View File
@@ -1,8 +1,6 @@
#!/bin/sh -e
# Description: check various bad patterns with grep
# https://postmarketos.org/pmb-ci
exit_code=0
# https://postmarktos.org/pmb-ci
if [ "$(id -u)" = 0 ]; then
set -x
@@ -11,103 +9,16 @@ if [ "$(id -u)" = 0 ]; then
fi
# Find CHANGEMEs in APKBUILDs
if grep -qr '(CHANGEME!)' -- *; then
if grep -qr '(CHANGEME!)' *; then
echo "ERROR: Please replace '(CHANGEME!)' in the following files:"
grep --color=always -r '(CHANGEME!)' -- *
exit_code=1
grep --color=always -r '(CHANGEME!)' *
exit 1
fi
# DTBs installed to /usr/share/db
# shellcheck disable=SC2016
if grep -qr 'INSTALL_DTBS_PATH="$pkgdir"/usr/share/dtb' device/; then
echo 'ERROR: Please do not install dtbs to /usr/share/dtb!'
echo 'ERROR: Unless you have a good reason not to, please put them in /boot/dtbs'
echo 'ERROR: Files that need fixing:'
# shellcheck disable=SC2016
grep --color=always -r 'INSTALL_DTBS_PATH="$pkgdir"/usr/share/dtb' device/
exit_code=1
fi
# Find old mkinitfs paths (pre mkinitfs 2.0)
if grep -qr '/etc/postmarketos-mkinitfs' -- *; then
echo "ERROR: Please replace '/etc/postmarketos-mkinitfs' with '/usr/share/mkinitfs' in the following files:"
grep --color=always -r '/etc/postmarketos-mkinitfs' -- *
exit_code=1
fi
if grep -qr '/usr/share/postmarketos-mkinitfs' -- *; then
echo "ERROR: Please replace '/usr/share/postmarketos-mkinitfs' with '/usr/share/mkinitfs' in the following files:"
grep --color=always -r '/usr/share/postmarketos-mkinitfs' -- *
exit_code=1
fi
# Direct sourcing of deviceinfo
if grep --exclude='source_deviceinfo' -qEr 'source /etc/deviceinfo|\. /etc/deviceinfo' -- *; then
echo 'ERROR: Please source the source_deviceinfo script instead of sourcing deviceinfo directly!'
grep --color=always --exclude='rootfs-usr-share-misc-source_deviceinfo' -Er 'source /etc/deviceinfo|\. /etc/deviceinfo' -- *
exit_code=1
fi
# Removed deviceinfo variable
if grep -qr 'deviceinfo_modules_initfs' -- *; then
echo 'ERROR: deviceinfo_modules_initfs variable has been removed. Use "modules-initfs" file instead.'
grep --color=always -r 'deviceinfo_modules_initfs' -- *
exit_code=1
fi
POSTMARKETOS_WALLPAPER_PATH='/usr/share/wallpapers/postmarketos.jpg'
# The excluded devices are "grandfathered in". New devices should not be added here.
# See https://gitlab.postmarketos.org/postmarketOS/pmaports/-/issues/2529
if grep -qr $POSTMARKETOS_WALLPAPER_PATH \
--exclude-dir='device-pine64-pinetab' \
--exclude-dir='device-oneplus-kebab' \
--exclude-dir='device-xiaomi-willow' \
-- device; then
echo "ERROR: Please don't include configuration files that set the default wallpaper in device-specific packages!"
grep --color=always -r $POSTMARKETOS_WALLPAPER_PATH \
--exclude-dir='device-pine64-pinetab' \
--exclude-dir='device-oneplus-kebab' \
--exclude-dir='device-xiaomi-willow' \
-- device
exit_code=1
fi
OPENRC_SERVICE_FILES=$(find . -name '*.initd')
# shellcheck disable=SC2086
if grep -q 'before wpa_supplicant' $OPENRC_SERVICE_FILES; then
echo "ERROR: Please use 'before wlan' in OpenRC service files! This ensures compatibility with both wpa_supplicant and iwd."
grep --color=always 'before wpa_supplicant' $OPENRC_SERVICE_FILES
exit_code=1
fi
if grep -qEr 'PMOS_NO_OUTPUT_REDIRECT' -- *; then
echo "ERROR: PMOS_NO_OUTPUT_REDIRECT is deprecated and doesn't do anything."
echo "Please remove it from the following files:"
grep --color=always -Er 'PMOS_NO_OUTPUT_REDIRECT' -- *
exit_code=1
fi
if grep -qEr '^deviceinfo_kernel_cmdline.*[\"\s]console=null' -- device/; then
echo "ERROR: Do not use console=null in the kernel command line."
echo "Use these params to quiet console on boot: quiet loglevel=2"
echo "For more information, see: https://gitlab.postmarketos.org/postmarketOS/pmaports/-/issues/2989"
exit_code=1
fi
# Disallow installation of files to /etc/modules-load.d and /etc/modprobe.d
if grep -qr '/etc/modules-load.d' --exclude-dir='device-samsung-golden-downstream' -- *; then
echo "ERROR: Please replace '/etc/modules-load.d' with '/usr/lib/modules-load.d' in the following files:"
grep --color=always -r '/etc/modules-load.d' --exclude-dir='device-samsung-golden-downstream' -- *
exit_code=1
fi
# n900 is excluded because the string is in a comment linking to a repo.
# No other non-archived devices should be excluded.
if grep -qr '/etc/modprobe.d' --exclude-dir='archived' \
--exclude-dir='device-nokia-n900' -- *; then
echo "ERROR: Please replace '/etc/modprobe.d' with '/usr/lib/modprobe.d' in the following files:"
grep --color=always -r '/etc/modprobe.d' --exclude-dir='device-nokia-n900' \
--exclude-dir='archived' -- *
exit_code=1
fi
exit "$exit_code"
# FIXME: doesn't pass currently, fix it and enable it afterwards
# if grep -r 'INSTALL_DTBS_PATH="$pkgdir"/usr/share/dtb'; then
# echo 'Please do not install dtbs to /usr/share/dtb!'
# echo 'Unless you have a good reason not to, please put them in /boot/dtbs'
# exit 1
# fi
+11 -5
View File
@@ -1,16 +1,22 @@
#!/bin/sh -e
# Description: check modified kernel configs
# Description: check all kernel configs with 'pmbootstrap kconfig check'
# Options: native
# Use 'native' because it requires running pmbootstrap.
# https://postmarketos.org/pmb-ci
# https://postmarktos.org/pmb-ci
if [ "$(id -u)" = 0 ]; then
set -x
wget "https://gitlab.postmarketos.org/postmarketOS/ci-common/-/raw/master/install_pmbootstrap.sh"
wget "https://gitlab.com/postmarketOS/ci-common/-/raw/master/install_pmbootstrap.sh"
sh ./install_pmbootstrap.sh
exec su "${TESTUSER:-pmos}" -c "sh -e $0"
fi
export PYTHONUNBUFFERED=1
# Wrap pmbootstrap to use this repository for --aports
pmaports="$(cd $(dirname $0)/..; pwd -P)"
_pmbootstrap="$(command -v pmbootstrap)"
pmbootstrap() {
"$_pmbootstrap" --aports="$pmaports" "$@"
}
.ci/lib/check_changed_kernels.py
set -x
pmbootstrap kconfig check
+12 -44
View File
@@ -2,62 +2,30 @@
# SPDX-License-Identifier: GPL-3.0-or-later
import common
import os
import os.path
import sys
import subprocess
import tomllib
custom_valid_options = [
"!pmb:crossdirect",
"!pmb:kconfigcheck",
"pmb:cross-native",
"pmb:cross-native2",
"pmb:drm",
"pmb:gpu-accel", # deprecated
"pmb:strict",
"pmb:systemd",
"pmb:systemd-never",
]
def get_kconfigcheck_categories() -> list [str]:
with open("kconfigcheck.toml", "rb") as kconfigcheck_config:
kconfigcheck_data = tomllib.load(kconfigcheck_config)
kconfigcheck_categories = []
for category_name in kconfigcheck_data:
if category_name == "aliases":
for alias_name in kconfigcheck_data["aliases"]:
kconfigcheck_categories.append(f"pmb:kconfigcheck-{alias_name}")
else:
kconfigcheck_categories.append(category_name)
return [item.replace("category:", "pmb:kconfigcheck-") for item in kconfigcheck_categories]
if __name__ == "__main__":
kconfigcheck_categories = get_kconfigcheck_categories()
custom_valid_options += kconfigcheck_categories
os.environ["CUSTOM_VALID_OPTIONS"] = " ".join(custom_valid_options)
common.add_upstream_git_remote()
apkbuilds = {file for file in common.get_changed_files(removed=False)
if os.path.basename(file) == "APKBUILD"}
if len(apkbuilds) < 1:
print("No APKBUILDs to lint")
sys.exit(0)
apkbuilds_filtered = []
packages = []
for apkbuild in apkbuilds:
if apkbuild.startswith("temp/") or apkbuild.startswith("cross/"):
print(f"NOTE: Skipping linting of {apkbuild}")
continue
apkbuilds_filtered.append(apkbuild)
if len(apkbuilds_filtered) < 1:
packages.append(os.path.basename(os.path.dirname(apkbuild)))
if len(packages) < 1:
print("No APKBUILDs to lint")
sys.exit(0)
try:
subprocess.run(["apkbuild-lint", *apkbuilds_filtered], text=True, check=True)
except subprocess.CalledProcessError as exception:
sys.exit(exception.returncode)
result = common.run_pmbootstrap(["-q", "lint"] + packages, output_return=True)
if len(result) > 0:
print("Linting issues found:")
print(result)
sys.exit(1)
+38 -61
View File
@@ -2,32 +2,41 @@
# Copyright 2021 Oliver Smith
# SPDX-License-Identifier: GPL-3.0-or-later
import sys
import os
import pathlib
# Same dir
import common
# pmbootstrap
import add_pmbootstrap_to_import_path
import pmb.core
import pmb.parse
import pmb.parse._apkbuild
import pmb.helpers.pmaports
from pmb.core.context import get_context
def build_strict(packages, arch):
common.run_pmbootstrap(["build_init"])
# We set the timeout to 1 hour because linking the Linux kernel with
# ThinLTO on our aarch64 runners takes slightly over 30 minutes
common.run_pmbootstrap(["--details-to-stdout", "--no-ccache",
"--timeout", "3600",
"build",
common.run_pmbootstrap(["--details-to-stdout", "--no-ccache", "build",
"--strict", "--force",
"--arch", arch, ] + list(packages))
def verify_checksums(packages, arch):
# Only do this with one build-{arch} job
arch_verify = "x86_64"
if arch != arch_verify:
print(f"NOTE: doing checksum verification in build-{arch_verify} job,"
" not here.")
return
if len(packages) == 0:
print("no packages changed, not doing any checksums verification")
return
common.run_pmbootstrap(["build_init"])
common.run_pmbootstrap(["--details-to-stdout", "checksum", "--verify"] +
list(packages))
if __name__ == "__main__":
# Architecture to build for (as in build-{arch})
if len(sys.argv) != 2:
@@ -36,38 +45,29 @@ if __name__ == "__main__":
arch = sys.argv[1]
# Get and print modified packages
packages = common.get_changed_packages(skip_archived=True)
common.add_upstream_git_remote()
packages = common.get_changed_packages()
# Load context
sys.argv = ["pmbootstrap.py", "chroot"]
# Package count sanity check
common.get_changed_packages_sanity_check(len(packages))
# [ci:skip-build]: verify checksums and stop
verify_only = common.commit_message_has_string("[ci:skip-build]")
if verify_only:
print("WARNING: not building changed packages ([ci:skip-build])!")
print("verifying checksums: " + ", ".join(packages))
verify_checksums(packages, arch)
sys.exit(0)
# Prepare "args" to use pmbootstrap code
sys.argv = ["pmbootstrap", "chroot"]
args = pmb.parse.arguments()
context = get_context()
# Get set of all buildable packages for the enabled repos, for skipping unbuildable
# aports later. We might be given a changed aport from e.g. extra-repos/systemd when
# that repo is not enabled
buildable_pkgs = set()
for path in pmb.core.pkgrepo.pkgrepo_iter_package_dirs():
buildable_pkgs.add(os.path.basename(path))
# To store a list of packages from extra-repos/systemd for special handling later:
systemd_pkgs = list()
# Filter out packages that either:
# 1. can't be built for given arch
# 2. are not found in enabled repos
# Filter out packages that can't be built for given arch
# (Iterate over copy of packages, because we modify it in this loop)
for package in packages.copy():
if package not in buildable_pkgs:
print(f"{package}: not in any available repos, skipping")
packages.remove(package)
# FIXME: this should probably be more generic, if other repos are added later?
# This just tosses the package into the list of packages to try building later w/ systemd enabled, and assumes it'll be found there.
systemd_pkgs.append(package)
continue
apkbuild_path = pmb.helpers.pmaports.find(package)
apkbuild = pmb.parse._apkbuild.apkbuild(pathlib.Path(apkbuild_path, "APKBUILD"))
apkbuild_path = pmb.helpers.pmaports.find(args, package)
apkbuild = pmb.parse._apkbuild.apkbuild(f"{apkbuild_path}/APKBUILD")
if not pmb.helpers.pmaports.check_arches(apkbuild["arch"], arch):
print(f"{package}: not enabled for {arch}, skipping")
@@ -78,29 +78,6 @@ if __name__ == "__main__":
print(f"no packages changed, which can be built for {arch}")
sys.exit(0)
# Build packages
print(f"building in strict mode for {arch}: {', '.join(packages)}")
build_strict(packages, arch)
# Build packages in extra-repos/systemd
# FIXME: this should probably be more generic, if other repos are added later?
if systemd_pkgs:
common.run_pmbootstrap(["config", "systemd", "always"])
# To fix the ERROR: Chroot 'native' is for the 'edge' channel, but you are on the
# 'systemd-edge' channel. Run 'pmbootstrap zap' to delete your chroots and try again.
# To do this automatically, run 'pmbootstrap config auto_zap_misconfigured_chroots yes'.
common.run_pmbootstrap(["config", "auto_zap_misconfigured_chroots", "yes"])
# filter out packages that can't be built for arch
# (Iterate over copy of `systemd_pkgs`, because we modify it in this loop)
for package in systemd_pkgs.copy():
apkbuild_path = pmb.helpers.pmaports.find(package, True, True, with_extra_repos="enabled")
apkbuild = pmb.parse._apkbuild.apkbuild(pathlib.Path(apkbuild_path, "APKBUILD"))
if not pmb.helpers.pmaports.check_arches(apkbuild["arch"], arch):
print(f"(extra-repos/systemd) {package}: not enabled for {arch}, skipping")
systemd_pkgs.remove(package)
# No packages: skip build
if len(systemd_pkgs) == 0:
print(f"no packages changed, which can be built for {arch}")
sys.exit(0)
build_strict(systemd_pkgs, arch)
+2 -2
View File
@@ -1,11 +1,11 @@
#!/bin/sh -e
# Description: build modified packages for this architecture
# Options: native slow
# https://postmarketos.org/pmb-ci
# https://postmarktos.org/pmb-ci
if [ "$(id -u)" = 0 ]; then
set -x
wget "https://gitlab.postmarketos.org/postmarketOS/ci-common/-/raw/master/install_pmbootstrap.sh"
wget "https://gitlab.com/postmarketOS/ci-common/-/raw/master/install_pmbootstrap.sh"
sh ./install_pmbootstrap.sh
exec su "${TESTUSER:-pmos}" -c "sh -e $0"
fi
+123 -147
View File
@@ -1,12 +1,13 @@
#!/usr/bin/env python3
# Copyright 2021 Oliver Smith
# Copyright 2022 Oliver Smith
# SPDX-License-Identifier: GPL-3.0-or-later
import glob
import pathlib
import tempfile
import sys
import logging
import os
import subprocess
import sys
import tempfile
# Same dir
import common
@@ -16,9 +17,21 @@ import add_pmbootstrap_to_import_path # noqa
import pmb.parse
import pmb.parse.version
import pmb.helpers.logging
from pmb.core.context import get_context
def get_package_contents(package, revision, check=True):
def get_package_version_sources(args, package, revision, check=True):
"""
Get the version ($pkgver-r$pkgrel), checksums and a list of remote files
(not bundled with the APKBUILD) from a specific git revision.
:param package: name of the package
:param revision: git revision to look at
:param check: complain if the file does not exist
:returns: {"checksums": see lib.common.parse_apkbuild_checksums(),
"pkgrel": 3,
"pkgver": "1.0.0",
"sources_remote": ["x-1.0.0-r2.tar.bz2", ...],
"version": "1.0.0-r2"}
"""
# Redirect stderr to /dev/null, so git doesn't complain about files not
# existing in upstream branch for new packages
stderr = None
@@ -37,21 +50,27 @@ def get_package_contents(package, revision, check=True):
# Save APKBUILD to a temporary path and parse it from there. (Not the best
# way to do things, but good enough for this CI script.)
with tempfile.TemporaryDirectory() as tempdir:
with open(tempdir + "/APKBUILD", "w", encoding="utf-8") as handle:
apkbuild_path = f"{tempdir}/APKBUILD"
with open(apkbuild_path, "w", encoding="utf-8") as handle:
handle.write(apkbuild_content)
parsed = pmb.parse.apkbuild(pathlib.Path(tempdir, "APKBUILD"),
False, False)
parsed = pmb.parse.apkbuild(apkbuild_path, False, False)
checksums = common.parse_apkbuild_checksums(args, apkbuild_path)
return parsed
# Build sources_remote from files in checksums that are not bundled.
# To make the logic not overly complicated, only glob in the currently
# checked out directory. This is inaccurate for the previous commit, but
# good enough for how it's used in check_versions() below.
sources_remote = []
for filename, checksum in checksums.items():
bundled_path_pattern = f"{os.path.dirname(path)}/**/{filename}"
if not glob.glob(bundled_path_pattern, recursive=True):
sources_remote += filename
def get_package_version(package, revision, check=True):
""" returns version in the format "{pkgver}-r{pkgrel}", or None if no
matching package is found """
parsed = get_package_contents(package, revision, check)
if parsed is None:
return None
return parsed["pkgver"] + "-r" + parsed["pkgrel"]
return {"checksums": checksums,
"pkgrel": int(parsed['pkgrel']),
"pkgver": parsed['pkgver'],
"sources_remote": sources_remote,
"version": f"{parsed['pkgver']}-r{parsed['pkgrel']}"}
def version_compare_operator(result):
@@ -68,33 +87,23 @@ def version_compare_operator(result):
def exit_with_error_message():
branch = common.get_upstream_branch()
print()
print("ERROR: Modified package(s) don't have an increased version or a")
print("new package has a nonzero pkgrel!")
print()
print("This can happen if you added a new package with a nonzero")
print("pkgrel, or if you did not change the pkgver/pkgrel")
print("variables in the APKBUILDs. Or you did change them, but the")
print(f"packages have been updated in the official '{branch}' branch, and")
print("now your versions are not higher anymore.")
print()
print("Your options:")
print("a) If you made changes to the packages, and did not increase the")
print(" pkgrel/pkgver: increase them now, and force push your branch.")
print(" => https://postmarketos.org/howto-bump-pkgrel-pkgver")
print("b) If you had already increased the package versions, rebase on")
print(f" '{branch}', increase the versions again and then force push:")
print(" => https://postmarketos.org/rebase")
print("c) If you made a change, that does not require rebuilding the")
print(" packages, such as only changing the arch=... line: you can")
print(" disable this check by adding '[ci:skip-vercheck]' to the")
print(" latest commit message, then force push.")
print()
print("Thank you and sorry for the inconvenience.")
logging.error("")
logging.error("NOTE: use 'pmbootstrap ci commits' to run this check"
" on your PC.")
logging.error("")
logging.error("Some more hints:")
logging.error(f"* You may need to rebase on {branch}.")
logging.error("* Detailed rules for bumping pkgrel/pkgver:")
logging.error(" https://postmarketos.org/howto-bump-pkgrel-pkgver")
logging.error("* If the error doesn't make sense (e.g. you want to")
logging.error(" intentionally downgrade a package), add [ci:skip-build]")
logging.error(" with reasoning to the last commit message.")
logging.error("")
logging.error("Thank you and sorry for the inconvenience.")
exit(1)
def check_versions(packages):
def check_versions(args, packages):
error = False
# Get relevant commits: compare HEAD against upstream branch or HEAD~1
@@ -104,144 +113,111 @@ def check_versions(packages):
# desired here, since we already know what packages changed, and really
# want to check if the version was increased towards *current* upstream
# branch HEAD.
common.run_git(["remote", "add", "upstream",
"https://gitlab.postmarketos.org/postmarketOS/pmaports.git"], False)
common.run_git(["fetch", "-q", "upstream"])
commit = f"upstream/{common.get_upstream_branch()}"
if common.run_git(["rev-parse", "HEAD"]) == common.run_git(["rev-parse",
commit]):
print(f"NOTE: {commit} is on same commit as HEAD, comparing"
" HEAD against HEAD~1.")
logging.info(f"NOTE: {commit} is on same commit as HEAD, comparing"
" HEAD against HEAD~1.")
commit = "HEAD~1"
# Exclude linux-next from pkgrel checks, as it is updated automatically.
#
# Without this, .ci/commits.sh complains about the following:
# ERROR: Modified package(s) don't have an increased version or a new
# package has a nonzero pkgrel!
if "linux-next" in packages:
packages.remove("linux-next")
for package in packages:
# Get versions, skip new packages
head = get_package_version(package, "HEAD")
upstream = get_package_version(package, commit, False)
logging.info("")
# Get versions and sources, skip new packages
head = get_package_version_sources(args, package, "HEAD")
head_v = head["version"]
upstream = get_package_version_sources(args, package, commit, False)
if not upstream:
if head.rpartition('r')[2] != "0":
print(f"- {package}: {head} (HEAD) (new package) [ERROR]")
if head["pkgrel"] != 0:
logging.error(f"- {package}: {head_v} (HEAD) (new package)")
logging.error(" ERROR: new package should have pkgrel=0")
error = True
else:
print(f"- {package}: {head} (HEAD) (new package)")
logging.info(f"- {package}: {head_v} (HEAD) (new package)")
continue
upstream_v = upstream["version"]
# Check pkgver follows expected format for device packages
pkgver = head.rpartition('-r')[0]
if package.startswith('device-') and not pkgver.isdigit():
print(f" - {package}: invalid pkgver \"{pkgver}\""
"See: https://wiki.postmarketos.org/wiki/Packaging#device_packages_and_other_packages_without_sources")
error = True
# Compare head and upstream versions based on rules from:
# https://postmarketos.org/howto-bump-pkgrel-pkgver
error_str = ""
if head["sources_remote"]:
if head["sources_remote"] == upstream["sources_remote"]:
pkgver = upstream["pkgver"]
pkgrel = upstream["pkgrel"] + 1
if head["pkgver"] != pkgver or head["pkgrel"] != pkgrel:
error_str = f"expected pkgver={pkgver}, pkgrel={pkgrel}" \
" (pkgver unchanged, pkgrel + 1), because" \
" this package has unchanged remote sources"
else: # remote sources changed
if upstream["pkgver"] == "9999":
pkgver = "9999"
pkgrel = upstream["pkgrel"] + 1
if head["pkgver"] != pkgver or head["pkgrel"] != pkgrel:
error_str = f"expected pkgver={pkgver}," \
f" pkgrel={pkgrel} (pkgver unchanged," \
" pkgrel + 1), because this package has" \
" pkgver=9999"
elif head["pkgver"] == upstream["pkgver"] or head["pkgrel"] != 0:
error_str = "expected pkgver to change and pkgrel=0," \
" because this package has changed remote" \
" sources"
else: # no remote sources
if head["checksums"] == upstream["checksums"]:
pkgver = upstream["pkgver"]
pkgrel = upstream["pkgrel"] + 1
if head["pkgver"] != pkgver or head["pkgrel"] != pkgrel:
error_str = f"expected pkgver={pkgver}, pkgrel={pkgrel} " \
" (pkgver unchanged, pkgrel + 1), because" \
" this package has no remote sources and the" \
" bundled sources didn't change"
else: # checksums changed
if head["pkgver"] == upstream["pkgver"] or head["pkgrel"] != 0:
error_str = "expected pkgver to change and pkgrel=0," \
" because this package has no remote sources" \
" but the bundled files changed"
# Additional checks for device packages
if package.startswith('device-'):
head_parsed = get_package_contents(package, "HEAD", False)
upstream_parsed = get_package_contents(package, commit, False)
# checksums did not change
if head_parsed["sha512sums"] == upstream_parsed["sha512sums"]:
# Check that pkgver did not change
if head_parsed["pkgver"] != upstream_parsed["pkgver"]:
print(f" - {package}: pkgver should not change when package source checksums did not change."
"See: https://wiki.postmarketos.org/wiki/Packaging#device_packages_and_other_packages_without_sources")
error = True
# We do not check for a bumped pkgrel, because not everything
# needs it, e.g: pmb_recommends
# checksums changed
else:
# Check that pkgrel was reset to 0
if head_parsed["pkgrel"] != "0":
print(f" - {package}: pkgrel should be 0 when package source checksums change."
"See: https://wiki.postmarketos.org/wiki/Packaging#device_packages_and_other_packages_without_sources")
error = True
# Check that pkgver was changed
if head_parsed["pkgver"] == upstream_parsed["pkgver"]:
print(f" - {package}: pkgver should change when package source checksums change."
"See: https://wiki.postmarketos.org/wiki/Packaging#device_packages_and_other_packages_without_sources")
error = True
# Compare head and upstream versions
result = pmb.parse.version.compare(head, upstream)
result = pmb.parse.version.compare(head_v, upstream_v)
if result != 1:
error = True
error_str = "new version is not higher than the previous version"
# Print result line ("- hello-world: 1-r2 (HEAD) > 1-r1 (HEAD~1)")
formatstr = "- {}: {} (HEAD) {} {} ({})"
if result != 1:
formatstr += " [ERROR]"
operator = version_compare_operator(result)
print(formatstr.format(package, head, operator, upstream, commit))
logging.error(formatstr.format(package, head_v, operator, upstream_v,
commit))
# Detailed error message should help the user to resolve it and explain
# why it failed.
if error_str:
logging.error(f" ERROR: {error_str}")
error = True
if error:
exit_with_error_message()
def check_package_count(count):
for mark in ["[ci:ignore-count]", "[ci:skip-build]"]:
if common.commit_message_has_string(mark):
print("NOTE: package count sanity check skipped (" + mark + ").")
return
if count <= 10:
return
branch = common.get_upstream_branch()
print(f"""
ERROR: Too many packages have changed!
This is a sanity check, so we don't end up building packages that
have not been modified. CI won't run for more than three hours
anyway.
Your options:
a) If you *did not* modify everything listed above, then rebase
your branch on the official postmarketOS/pmaports.git {branch}
branch. Feel free to ask in the chat for help if you need any.
b) If you *did* modify all these packages, and you assume that
they will build within one hour: skip this sanity check by
adding '[ci:ignore-count]' to the commit message of the last
commit in the merge request (then force push).
c) If you *did* modify all these packages, and you are sure that
they won't build in time, please add '[ci:skip-build]' to the
commit message (then force push). Make sure that all packages
build with 'pmbootstrap build --strict'!
Thank you and sorry for the inconvenience.
""")
exit(1)
if __name__ == "__main__":
# Get and print modified packages
common.add_upstream_git_remote()
packages = common.get_changed_packages()
if len(packages) == 0:
print("no aports changed in this branch")
exit(0)
logging.info(f"Changed packages: {packages}")
print(f"Changed packages: {packages}")
check_package_count(len(packages))
# Verify modified package count
common.get_changed_packages_sanity_check(len(packages))
if len(packages) == 0:
logging.info("no aports changed in this branch")
exit(0)
# Potentially skip this check
if common.commit_message_has_string("[ci:skip-vercheck]"):
print("WARNING: not checking for changed package versions"
" ([ci:skip-vercheck])!")
logging.warning("WARNING: not checking for changed package versions"
" ([ci:skip-vercheck])!")
exit(0)
# Initialize args (so we can use pmbootstrap's APKBUILD parsing)
sys.argv = ["pmbootstrap.py", "chroot"]
args = pmb.parse.arguments()
context = get_context()
# pmb.helpers.logging.init(args)
pmb.helpers.logging.init(context.log, args.verbose, context.details_to_stdout)
pmb.helpers.logging.init(args)
# Verify package versions
print("checking changed package versions...")
check_versions(packages)
logging.info("Checking changed package versions...")
check_versions(args, packages)
-99
View File
@@ -1,99 +0,0 @@
#!/usr/bin/env python3
# Copyright 2024 Oliver Smith
# SPDX-License-Identifier: GPL-3.0-or-later
import glob
import tempfile
import sys
import subprocess
# Same dir
import common
def check_kconfig(pkgnames):
last_failed = None
for i in range(len(pkgnames)):
pkgname = pkgnames[i]
print(f" ({i+1}/{len(pkgnames)}) {pkgname}")
p = subprocess.run(["pmbootstrap", "kconfig", "check", "--keep-going", pkgname],
check=False)
if p.returncode:
last_failed = pkgname
return last_failed
def check_kconfig_all():
p = subprocess.run(["pmbootstrap", "kconfig", "check", "--keep-going"], check=False)
return p.returncode == 0
def show_error(last_failed):
print("")
print("---")
print("")
print("Please adjust your kernel config. This is required for getting your"
" patch merged.")
print("")
print("Edit your kernel config:")
print(f" pmbootstrap kconfig edit {last_failed}")
print("")
print("Test this kernel config again:")
print(f" pmbootstrap kconfig check {last_failed}")
print("")
print("Run this check again (on all kernels you modified):")
print(" pmbootstrap ci kconfig")
print("")
print("---")
print("")
def show_error_all():
print("")
print("---")
print("")
print("Please adjust the failed kernel configs. This is required for")
print("getting your patch merged.")
print("")
print("If a kernel in the testing category failed, and fixing is too much")
print("effort, remove 'pmb:kconfigcheck-community' from the APKBUILD.")
print("")
print("Edit a kernel config:")
print(" pmbootstrap kconfig edit <PACKAGE NAME>")
print("")
print("Test a specific kernel config again:")
print(" pmbootstrap kconfig check <PACKAGE NAME>")
print("")
print("Run this check again (on all kernels you modified):")
print(" pmbootstrap ci kconfig")
print("")
print("---")
print("")
if __name__ == "__main__":
if "kconfigcheck.toml" in common.get_changed_files():
print("kconfigcheck.toml changed -> checking all kernels")
if not check_kconfig_all():
show_error_all()
exit(1)
exit(0)
pkgnames = common.get_changed_kernels(skip_archived=True)
print(f"Changed kernels: {pkgnames}")
if len(pkgnames) == 0:
print("No kernels changed in this branch")
exit(0)
elif common.commit_message_has_string("[ci:skip-kconfigcheck]"):
print("WARNING: not checking kernel configs ([ci:skip-kconfigcheck])")
exit(0)
last_failed = check_kconfig(pkgnames)
if last_failed:
show_error(last_failed)
exit(1)
+22 -35
View File
@@ -1,21 +1,21 @@
#!/usr/bin/env python3
# Copyright 2023 Oliver Smith
# Copyright 2022 Oliver Smith
# SPDX-License-Identifier: GPL-3.0-or-later
from pathlib import Path
import argparse
import glob
import os
import sys
import urllib.request
# Same dir
import common
def get_devices() -> list[str]:
def get_devices():
""":returns: list of all devices"""
ret = []
pmaports = Path(__file__).resolve().parents[2]
for path in pmaports.glob("device/*/device-*/"):
device = path.name.split("device-", 1)[1]
pmaports = (os.path.realpath(os.path.join(os.path.dirname(__file__) +
"/../..")))
for path in glob.glob(pmaports + "/device/*/device-*/"):
device = os.path.dirname(path).split("device-", 1)[1]
# -downstream suffix is used when packaging the downstream kernel for
# devices that have a working mainline kernel. Those are usually
@@ -28,7 +28,7 @@ def get_devices() -> list[str]:
return sorted(ret)
def get_wiki_devices_html(path: Path | None) -> dict[str, str]:
def get_wiki_devices_html(path):
""":param path: to a local file with the saved content of the devices wiki
page or None to download a fresh copy
:returns: HTML of the page, split into booting and not booting:
@@ -41,9 +41,8 @@ def get_wiki_devices_html(path: Path | None) -> dict[str, str]:
content = handle.read()
else:
# Download wiki page
url = "https://wiki.postmarketos.org/wiki/Devices"
with urllib.request.urlopen(url) as f:
content = f.read().decode("utf-8")
url = "http://wiki.postmarketos.org/wiki/Devices"
content = urllib.request.urlopen(url).read().decode("utf-8")
# Split into booting and not booting
split = content.split("<span class=\"mw-headline\" id=\"Non-booting_devices\">")
@@ -54,16 +53,14 @@ def get_wiki_devices_html(path: Path | None) -> dict[str, str]:
return {"booting": split[0], "not_booting": split[1]}
def get_wiki_renamed_devices_html() -> str:
def get_wiki_renamed_devices_html():
""":returns: HTML of the page"""
# Download wiki page
url = "https://wiki.postmarketos.org/wiki/Renamed_Devices"
with urllib.request.urlopen(url) as f:
res: bytes = f.read()
return res.decode("utf-8")
url = "http://wiki.postmarketos.org/wiki/Renamed_Devices"
return urllib.request.urlopen(url).read().decode("utf-8")
def check_device(device: str, html: dict[str, str], is_booting: bool) -> bool:
def check_device(device, html, is_booting):
""":param is_booting: require the device to be in the booting section, not
just anywhere in the page (i.e. in the not booting
table).
@@ -85,7 +82,7 @@ def check_device(device: str, html: dict[str, str], is_booting: bool) -> bool:
return False
def main() -> int:
def main():
# Parse arguments
parser = argparse.ArgumentParser()
parser.add_argument("--booting", help="devices must be in the upper table,"
@@ -95,31 +92,21 @@ def main() -> int:
action="store_true")
parser.add_argument("--path", help="instead of downloading the devices"
" page from the wiki, use a local HTML file",
type=Path, default=None)
parser.add_argument("--all", help="check all devices (default: check only"
" devices changed from branch point)",
action="store_true")
default=None)
args = parser.parse_args()
# Check all devices
html = get_wiki_devices_html(args.path)
html["renamed"] = get_wiki_renamed_devices_html()
error = False
devices = []
if args.all:
devices = get_devices()
else:
devices = common.get_changed_devices()
# remove 'device-' prefix
devices = [d[7:] for d in devices]
devices = [d.removesuffix("-downstream") for d in devices]
for device in devices:
for device in get_devices():
if not check_device(device, html, args.booting):
error = True
# Ask to adjust the wiki
if error:
print("*** Wiki check failed!")
print("Thank you for porting postmarketOS to a new device! \\o/")
print("Thank you for porting postmarketOS to a new device! \o/")
print("")
print("Now it's time to add some documentation:")
print("1) Create a device specific wiki page as described here:")
@@ -139,8 +126,8 @@ def main() -> int:
print("wiki as well and make sure to add it here:")
print("https://postmarketos.org/renamed")
return 1
print("*** Wiki check successful!")
else:
print("*** Wiki check successful!")
return 0
+104 -56
View File
@@ -2,11 +2,13 @@
# Copyright 2021 Oliver Smith
# SPDX-License-Identifier: GPL-3.0-or-later
# Various functions used in CI scripts
# Various functions used in CI scripts and tests. This file is in
# .ci/lib/common.py and symlinked to .ci/testcases/common.py.
import configparser
import os
import subprocess
import sys
cache = {}
@@ -32,23 +34,25 @@ def run_git(parameters, check=True, stderr=None):
return None
def add_upstream_git_remote():
""" Add a remote pointing to postmarketOS/pmaports. """
run_git(["remote", "add", "upstream",
"https://gitlab.com/postmarketOS/pmaports.git"], False)
run_git(["fetch", "-q", "upstream"])
def commit_message_has_string(needle):
base_commit = get_base_commit()
return needle in run_git(["log", "--pretty=format:%B", f"{base_commit}..HEAD"])
return needle in run_git(["show", "-s", "--format=full", "HEAD"])
def all_committed_by_merge_bot():
base_commit = get_base_commit()
commit_emails = run_git(["log", "--pretty=format:%ce", f"{base_commit}..HEAD"]).splitlines()
bot_email = os.getenv("PMOS_MERGE_BOT_EMAIL", "merge-bot@postmarketos.org")
return all(bot_email == ce for ce in commit_emails)
def run_pmbootstrap(parameters):
def run_pmbootstrap(parameters, output_return=False):
""" Run pmbootstrap with the pmaports dir as --aports """
cmd = ["pmbootstrap", "--aports", get_pmaports_dir()] + parameters
subprocess.run(cmd, universal_newlines=True, check=True)
stdout = subprocess.PIPE if output_return else None
result = subprocess.run(cmd, stdout=stdout, universal_newlines=True)
result.check_returncode()
if output_return:
return result.stdout
def get_upstream_branch():
@@ -56,11 +60,6 @@ def get_upstream_branch():
channels.cfg from master to retrieve the upstream branch.
:returns: branch name, e.g. "v20.05" """
# Prefer gitlab CI target branch name if it's set (i.e. running in gitlab CI)
if target_branch := os.environ.get("CI_MERGE_REQUEST_TARGET_BRANCH_NAME"):
return target_branch
global cache
if "upstream_branch" in cache:
return cache["upstream_branch"]
@@ -87,24 +86,17 @@ def get_upstream_branch():
return ret
def get_base_commit() -> str:
"""Get the base commit that can be compared with HEAD to get all commits in the
given branch/merge request."""
commit = os.getenv("CI_MERGE_REQUEST_DIFF_BASE_SHA")
if commit is not None:
return commit
# Add a remote pointing to postmarketOS/pmaports
run_git(["remote", "add", "upstream",
"https://gitlab.postmarketos.org/postmarketOS/pmaports.git"], False)
run_git(["fetch", "-q", "upstream"])
def get_changed_files(removed=True):
""" Get all changed files and print them, as well as the branch and the
commit that was used for the diff.
:param removed: also return removed files (default: True)
:returns: set of changed files
"""
branch_upstream = f"upstream/{get_upstream_branch()}"
commit_head = run_git(["rev-parse", "HEAD"])[:-1]
commit_upstream = run_git(["rev-parse", branch_upstream])[:-1]
print("commit HEAD: " + commit_head)
print(f"commit {branch_upstream}: {commit_upstream}")
print(f"commit {branch_upstream}: f{commit_upstream}")
# Check if we are HEAD on the upstream branch
if commit_head == commit_upstream:
@@ -115,17 +107,6 @@ def get_base_commit() -> str:
commit = run_git(["merge-base", branch_upstream, "HEAD"])[:-1]
print("comparing HEAD with: " + commit)
return commit
def get_changed_files(removed=True):
""" Get all changed files and print them, as well as the branch and the
commit that was used for the diff.
:param removed: also return removed files (default: True)
:returns: set of changed files
"""
commit = get_base_commit()
# Changed files
ret = set()
print("changed file(s):")
@@ -141,7 +122,41 @@ def get_changed_files(removed=True):
return ret
def get_changed_packages(skip_archived: bool = False):
def get_changed_packages_sanity_check(count):
for mark in ["[ci:ignore-count]", "[ci:skip-build]"]:
if commit_message_has_string(mark):
print("NOTE: package count sanity check skipped (" + mark + ").")
return
if count <= 10:
return
branch = get_upstream_branch()
print()
print("ERROR: Too many packages have changed!")
print()
print("This is a sanity check, so we don't end up building packages that")
print("have not been modified. CI won't run for more than one hour")
print("anyway.")
print()
print("Your options:")
print("a) If you *did not* modify everything listed above, then rebase")
print(f" your branch on the official postmarketOS/pmaports.git {branch}")
print(" branch. Feel free to ask in the chat for help if you need any.")
print("b) If you *did* modify all these packages, and you assume that")
print(" they will build within one hour: skip this sanity check by")
print(" adding '[ci:ignore-count]' to the commit message (then force")
print(" push).")
print("c) If you *did* modify all these packages, and you are sure that")
print(" they won't build in time, please add '[ci:skip-build]' to the")
print(" commit message (then force push). Make sure that all packages")
print(" build with 'pmbootstrap build --strict'!")
print()
print("Thank you and sorry for the inconvenience.")
sys.exit(1)
def get_changed_packages():
ret = set()
for file in get_changed_files():
dirname, filename = os.path.split(file)
@@ -149,7 +164,7 @@ def get_changed_packages(skip_archived: bool = False):
# Skip files:
# * in the root dir of pmaports (e.g. README.md)
# * path with a dot (e.g. .ci/, device/.shared-patches/)
if not dirname or file.startswith(".") or "/." in file or skip_archived and dirname.startswith("device/archived"):
if not dirname or file.startswith(".") or "/." in file:
continue
if filename != "APKBUILD":
@@ -178,16 +193,49 @@ def get_changed_packages(skip_archived: bool = False):
return ret
def get_changed_kernels(skip_archived: bool = False):
ret = []
for pkgname in get_changed_packages(skip_archived):
if pkgname.startswith("linux-"):
ret += [pkgname]
return ret
def parse_apkbuild_checksums(args, apkbuild_path):
"""
Read the APKBUILD file and parse source files from the checksums at the
bottom. This has always the same format, even if $source is built with hard
to parse shell code (like in postmarketos-base).
def get_changed_devices():
ret = []
for pkgname in get_changed_packages():
if pkgname.startswith("device-"):
ret += [pkgname]
:param apkbuild_path: full path to the APKBUILD
:returns: dict of parsed "source" filenames and checksums, e.g.:
{"first.patch": "b4dc4f3…",
"second.patch": "b4dc4f3…"}
"""
start = 'sha512sums="'
in_block = False
ret = {}
with open(apkbuild_path, encoding="utf-8") as handle:
for line in handle.readlines():
# Find start
if not in_block:
if line.startswith(start):
in_block = True
else:
continue
# sha512sums may have lines without checksums:
# https://gitlab.alpinelinux.org/alpine/abuild/-/merge_requests/73
if " " not in line:
continue
try:
checksum, filename = line.rstrip().split(" ", 2)
except ValueError:
raise ValueError("Failed to parse checksums. Try to delete the"
" checksums and generate them again with"
f" 'pmbootstrap checksum': {apkbuild_path}")
# Cut off 'sha512sums="' if the first checksum is in that line
if checksum.startswith(start):
checksum = checksum[len(start):]
# Find end
if filename.endswith('"'):
filename = filename[:-1]
ret[filename] = checksum
return ret
-20
View File
@@ -1,20 +0,0 @@
#!/usr/bin/env python3
# Copyright 2025 Stefan Hansson
# SPDX-License-Identifier: GPL-3.0-or-later
import common
import os.path
import subprocess
import sys
if __name__ == "__main__":
if common.commit_message_has_string("[ci:skip-dint]"):
print("WARNING: not linting deviceinfo files ([ci:skip-dint])")
exit(0)
deviceinfo_files = {file for file in common.get_changed_files(removed=False)
if os.path.basename(file) == "deviceinfo"}
try:
subprocess.run(["dint", "check", *deviceinfo_files], text=True, check=True)
except subprocess.CalledProcessError as exception:
sys.exit(exception.returncode)
-239
View File
@@ -1,239 +0,0 @@
#!/usr/bin/env python3
# Copyright 2025 Pablo Correa Gomez
# SPDX-License-Identifier: GPL-3.0-or-later
from functools import cached_property
from pathlib import Path
from typing import Self
import sys
import traceback
from jinja2 import Template
import pmb.parse
from pmb.parse.deviceinfo import Deviceinfo
from pmb.types import Apkbuild
import pmb.helpers.devices
import pmb.helpers.logging
import pmb.helpers.package
from pmb.core.arch import Arch
# Same dir
import common
class Device:
def __init__(self, codename: str):
self.codename = codename
self.full_path = pmb.helpers.devices.find_path(codename)
def __repr__(self):
return f"Device({self.codename})"
@property
def name(self):
return self.full_path.name.removeprefix("device-")
@cached_property
def gitlab_ci_fragment(self) -> str | None:
fragment_path = self.full_path / "gitlab-ci.yml.j2"
try:
fragment_tmpl = fragment_path.read_text()
except Exception:
return None
return Template(fragment_tmpl).render(device=self)
@property
def is_present_in_ci(self):
return bool(self.gitlab_ci_fragment)
@cached_property
def pmaports_path(self) -> Path:
# Find the root of pmaports
pmaports_root = self.full_path.parent
while (not (pmaports_root / "pmaports.cfg").exists() and
pmaports_root != pmaports_root.root):
pmaports_root = pmaports_root.parent
assert pmaports_root != pmaports_root.root
return self.full_path.relative_to(pmaports_root)
@cached_property
def apkbuild(self) -> Apkbuild:
return pmb.parse.apkbuild(self.full_path)
@property
def pkgname(self) -> str:
return self.apkbuild['pkgname']
@property
def arch(self) -> Arch:
return Arch(self.apkbuild['arch'][0])
@property
def testing_dependencies(self) -> set[str]:
# TODO: Source this from the APKGBUILD
return {"postmarketos-mkinitfs-hook-ci"}
@cached_property
def package_dependencies(self) -> list[str]:
# HACK: Work around a bug in depends_recurse which prevents listing all the dependencies
# Bug: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/issues/2623
return {"postmarketos-initramfs"}
# return pmb.helpers.package.depends_recurse(pkgname=self.pkgname,
# arch=self.arch)
@cached_property
def dependencies(self) -> set[str]:
return self.package_dependencies | self.testing_dependencies
@cached_property
def kernels(self) -> list[str] | None:
kernels = []
subpackage_prefix = f"device-{self.codename}-kernel-"
for subpkgname in self.apkbuild.get('subpackages', []):
if not subpkgname.startswith(subpackage_prefix):
continue
kernel_name = subpkgname.removeprefix(subpackage_prefix)
# Ignore `none` kernels as they are not bootable
if kernel_name != "none":
kernels.append(kernel_name)
if kernels:
return kernels
else:
return None
@cached_property
def deviceinfo(self) -> dict[str, Deviceinfo] | Deviceinfo:
# NOTE: deviceinfo files that have different values per kernel are
# currently unsupported due to a gitlab-ci issue which prevents us
# from having a per-kernel list of variables that would be selected
# using `.extends`:
# https://gitlab.com/gitlab-org/gitlab/-/issues/299423
return Deviceinfo(self.full_path / "deviceinfo", None)
@property
def has_kernel_variants(self):
return len(self.kernels) > 0
@classmethod
def supported_devices(cls) -> dict[Path, Self]:
# NOTE: Not sure I like relying on pmboostrap just to avoid listing
# devices ourselves, but since we need it to parse the APKBUILD, then
# let's avoid duplicating the logic?
supported_devices = {}
for vendor in pmb.helpers.devices.list_vendors():
for device in pmb.helpers.devices.list_codenames(vendor):
try:
dev = cls(device.codename)
# Ignore devices that do not have a gitlab ci fragment,
# since we won't be able to make use of them
if not dev.gitlab_ci_fragment:
continue
supported_devices[dev.pmaports_path] = dev
except Exception:
traceback.print_exc()
return supported_devices
class ArchTagSet(set):
supported_arches = [
Arch.x86_64,
Arch.x86,
Arch.aarch64,
Arch.armv7,
Arch.armhf,
Arch.riscv64,
Arch.ppc64le,
]
def update(self, iterable):
# This ignores things like !armv7, that could be a follow-up optimization
if 'noarch' in iterable or 'all' in iterable:
iterable = [arch for arch in self.supported_arches]
super().update([Arch(arch) for arch in iterable if Arch(arch) in self.supported_arches])
if __name__ == "__main__":
# Needs input to output if we should create the jobs
if len(sys.argv) != 3:
print("usage: generate_build_jobs.py TEMPLATE CHILD_PIPELINE")
print(sys.argv)
sys.exit(1)
template = sys.argv[1]
child_pipeline = sys.argv[2]
# pmb logging has to be initialized for later pmb commands to work, setting
# to /dev/null since we don't care about the output. Later this could
# be changed to a file and added as a CI artifact if we need to debug
# something.
pmb.logging.init(Path("/dev/null"), False)
# Load context
sys.argv = ["pmbootstrap.py", "chroot"]
args = pmb.parse.arguments()
# Get the list of supported devices
supported_devices = Device.supported_devices()
archs = ArchTagSet()
devices_under_test = set()
packages_modified = set()
# Get and print modified packages
for file in common.get_changed_files():
path = Path(file)
# Check if the modified's file parent folder is one of a supported dev
if device := supported_devices.get(path.parent):
devices_under_test.add(device)
if path.name != "APKBUILD":
continue
elif not path.exists():
continue # APKBUILD was deleted
apkbuild = pmb.parse.apkbuild(path)
packages_modified.add(apkbuild['pkgname'])
archs.update(apkbuild["arch"])
# Add all the devices found in CI that depend on the package that got
# modified
for device in supported_devices.values():
if apkbuild['pkgname'] in device.dependencies:
devices_under_test.add(device)
skip = False
if common.commit_message_has_string("[ci:skip-build]"):
print("User requested skipping build, skipping build")
skip = True
if common.all_committed_by_merge_bot():
print("All commits in branch committed by merge bot, skipping build")
skip = True
if skip:
archs = ArchTagSet()
devices_under_test = set()
print(f"Architectures to build: {archs}")
print(f"Devices under test: {devices_under_test}")
with open(template) as f:
rendered = Template(f.read()).render(
archs=archs,
devices_under_test=devices_under_test,
packages_modified=packages_modified,
archtag={
Arch.x86_64: "shared",
Arch.x86: "shared",
Arch.aarch64: "arm64",
Arch.armv7: "qemu",
Arch.armhf: "qemu",
Arch.riscv64: "qemu",
Arch.ppc64le: "ppc64le",
},
)
with open(child_pipeline, "w") as fw:
fw.write(rendered)
+1 -1
View File
@@ -1,5 +1,5 @@
#!/bin/sh -e
# Copyright 2023 Oliver Smith
# Copyright 2022 Oliver Smith
# SPDX-License-Identifier: GPL-3.0-or-later
if ! mount | grep -q /proc/sys/fs/binfmt_misc; then
+1 -1
View File
@@ -3,7 +3,7 @@
# SPDX-License-Identifier: GPL-3.0-or-later
if [ -z "$1" ]; then
echo "usage: $(basename "$0") \$CI_PROJECT_DIR"
echo "usage: $(basename $0) \$CI_PROJECT_DIR"
exit 1
fi
-283
View File
@@ -1,283 +0,0 @@
#!/bin/sh
# Unit testing framework for shell scripts
# Invoke with the path to a tests/ subdir of a package and the
# package name as arguments. Or source it with the following
# variables set:
# * results_dir - second argument to the test script
set -e
# Colors :D
RED="\e[31m"
GREEN="\e[32m"
YELLOW="\e[33m"
BLUE="\e[34m"
BOLDBLUE="\e[1;34m"
RESET="\e[0m"
testlib_path="$(realpath "$0")"
_test_assert_failed=""
_test_testn=1
_test_passedn=0
_test_failedn=0
_test_current_test=""
_test_statefile="$(mktemp)"
_test_original_pwd=""
_test_temp_root=""
# Set by each test
results_dir="${results_dir:-}"
test_log() {
# ash supports echo -e
# shellcheck disable=SC3037
echo -e "$*" >&2
}
test_info() {
# shellcheck disable=SC3037
echo -e "${BLUE}$*${RESET}" >&2
}
test_debug() {
# shellcheck disable=SC3037
echo -e "${YELLOW}$*${RESET}" >&2
}
create_test_root() {
_test_original_pwd="$(pwd)"
_test_temp_root="$(mktemp -d)"
cd "$_test_temp_root"
}
cleanup_test_root() {
if [ -n "$_test_original_pwd" ]; then
cd "$_test_original_pwd"
fi
if [ -n "$_test_temp_root" ] && [ -d "$_test_temp_root" ]; then
rm -rf "$_test_temp_root"
fi
_test_original_pwd=""
_test_temp_root=""
}
assert_strequal() {
# $1: first string
# $2: second string
if [ -z "$1" ] || [ -z "$2" ]; then
test_log "ERROR: assert_equal: not enough arguments given"
exit 1
fi
if [ "$1" != "$2" ]; then
_test_assert_failed="$1 != $2"
test_log " ❌ $_test_assert_failed"
return
fi
test_log " ✅ $1 == $2"
}
assert_equal() {
# $1: name of variable
# $2: value to compare
if [ -z "$1" ] || [ -z "$2" ]; then
test_log "ERROR: assert_equal: not enough arguments given"
exit 1
fi
val="$(eval echo "\$$1")"
if [ "$val" != "$2" ]; then
_test_assert_failed="\$$1 != $2 (got $val)"
test_log " ❌ $_test_assert_failed"
return
fi
test_log " ✅ \$$1 == $2"
}
assert_unset() {
if [ -z "$1" ]; then
test_log "ERROR: assert_unset: not enough arguments given"
exit 1
fi
val="$(eval echo \$"$1")"
if [ -n "$val" ]; then
_test_assert_failed="\$$1 should be unset (has value '$val')"
test_log " ❌ $_test_assert_failed"
return
fi
test_log " ✅ \$$1 is unset"
}
assert_exists() {
# $1: filename to test
if [ -z "$1" ]; then
test_log "ERROR: assert_exists: not enough arguments given"
exit 1
fi
if ! [ -e "$1" ]; then
_test_assert_failed="\$$1 should exist but does not"
test_log " ❌ $_test_assert_failed"
return
fi
test_log " ✅ \$$1 exists"
}
assert_not_exists() {
if [ -z "$1" ]; then
test_log "ERROR: assert_not_exists: not enough arguments given"
exit 1
fi
if [ -e "$1" ]; then
_test_assert_failed="\$$1 should NOT exist but does"
test_log " ❌ $_test_assert_failed"
return
fi
test_log " ✅ \$$1 does not exist"
}
assert_contains() {
# $1: string to search in
# $2: substring to find
if [ -z "$1" ] || [ -z "$2" ]; then
test_log "ERROR: assert_contains: not enough arguments given"
exit 1
fi
case "$1" in
*"$2"*)
test_log " ✅ '$2' found in '$1'"
;;
*)
_test_assert_failed="'$2' not found in '$1'"
test_log " ❌ $_test_assert_failed"
return
;;
esac
}
assert_not_contains() {
# $1: string to search in
# $2: substring that should not be found
if [ -z "$1" ] || [ -z "$2" ]; then
test_log "ERROR: assert_not_contains: not enough arguments given"
exit 1
fi
case "$1" in
*"$2"*)
_test_assert_failed="'$2' found in '$1' but should not be"
test_log " ❌ $_test_assert_failed"
return
;;
*)
test_log " ✅ '$2' not found in '$1'"
;;
esac
}
# Log some expression and evaluate it
logeval() {
eval "$*"
test_log "${YELLOW}+ $*${RESET}"
}
# Start a test. Shell scripts involved in the test should be sourced AFTER calling this function!
start_test() {
if [ -n "$_test_current_test" ]; then
test_log "ERROR: multiple calls to start_test without calling end_test first!"
test_log "Current test: $_test_current_test"
exit 1
fi
test_log "==> Test $_test_testn: $1"
_test_current_test="$1"
set > "$_test_statefile"
}
# End a test and reset the environment.
end_test() {
local msg
local save_assert_failed
# Restore environment
save_assert_failed="$_test_assert_failed"
# Splitting is intentional
# shellcheck disable=SC2046
unset $(set | grep "='" | cut -d= -f1 | grep -vE "(PATH|IFS|TERM|LANG|HOME|PWD|SHELL|USER|save_assert_failed|_test_statefile)" )
# shellcheck disable=SC1090
. "$_test_statefile"
_test_assert_failed="$save_assert_failed"
if [ -n "$_test_assert_failed" ]; then
msg="${RED}Failed! ❌ ${_test_assert_failed}${RESET}"
echo "Test $_test_testn: $_test_assert_failed" >> "$results_dir/failed"
_test_failedn=$((_test_failedn+1))
echo "$_test_failedn" > "$results_dir/failedn"
unset _test_assert_failed
else
msg="${GREEN}Passed!${RESET}"
_test_passedn=$((_test_passedn+1))
echo "$_test_passedn" > "$results_dir/passedn"
fi
test_log "<== Test $_test_testn: $msg $_test_current_test\n"
_test_testn=$((_test_testn+1))
_test_current_test=""
}
# Call at the end of each testsuite shell script
end_testsuite() {
if [ -n "$_test_current_test" ]; then
test_log "${RED}ERROR: end_testsuite called without calling end_test first!${RESET}"
test_log "${RED}Current test: $_test_current_test${RESET}"
exit 1
fi
if [ -f "$results_dir/failed" ]; then
exit 1
fi
exit 0
}
run_tests() {
local failed test_results_base
local test name total passedn failedn t_passed t_failed
test_results_base="$(mktemp -d)"
test_log "Running tests for $2"
# FIXME: yeah this isn't quite the best way to iterate the files
# shellcheck disable=SC2010
for name in $(ls "$1" | grep "\.sh$"); do
test="$1/$name"
mkdir "$test_results_base/$name"
echo 0 > "$test_results_base/$name/passedn"
echo 0 > "$test_results_base/$name/failedn"
test_log
test_log "${BOLDBLUE}# Test suite ${name}${RESET}"
$test "$testlib_path" "$test_results_base/$name" || failed="$failed $name"
t_passed=$(cat "$test_results_base/$name/passedn")
t_failed=$(cat "$test_results_base/$name/failedn")
passedn=$((passedn+t_passed))
failedn=$((failedn+t_failed))
total=$((total+t_passed+t_failed))
done
if [ -n "$failed" ]; then
test_log
test_log "Some tests failed:"
for test in $failed; do
test_log "❌ $test"
test_log " ${RED}$(cat "$test_results_base/$test/failed")${RESET}"
done
fi
if [ $passedn -lt $total ]; then
test_log "[${RED}$passedn/$total tests passed${RESET}]"
else
test_log "[${GREEN}$passedn/$total tests passed!${RESET}]"
fi
test_log
if [ -n "$failed" ]; then exit 1; else exit 0; fi
}
# FIXME: probably not the best way to determine if this was sourced
if [ $# -gt 0 ]; then
run_tests "$1" "$2"
fi
-81
View File
@@ -1,81 +0,0 @@
#!/usr/bin/env python3
# Copyright 2025 Pablo Correa Gomez
# SPDX-License-Identifier: GPL-3.0-or-later
import os
import sys
# Same dir
import common
# pmbootstrap
import add_pmbootstrap_to_import_path
import pmb.core
from pmb.core.context import get_context
def verify_checksums(packages):
if len(packages) == 0:
print("no packages changed, not doing any checksums verification")
return
print("verifying checksums: " + ", ".join(packages))
common.run_pmbootstrap(["build_init"])
common.run_pmbootstrap(["--details-to-stdout", "checksum", "--verify"] +
list(packages))
if __name__ == "__main__":
# Get and print modified packages
packages = common.get_changed_packages()
# Load context
sys.argv = ["pmbootstrap.py", "chroot"]
args = pmb.parse.arguments()
context = get_context()
# Get set of all buildable packages for the enabled repos, for skipping unbuildable
# aports later. We might be given a changed aport from e.g. extra-repos/systemd when
# that repo is not enabled
buildable_pkgs = set()
for path in pmb.core.pkgrepo.pkgrepo_iter_package_dirs():
buildable_pkgs.add(os.path.basename(path))
# To store a list of packages from extra-repos/systemd for special handling
# later:
systemd_pkgs = list()
# Filter out packages are not found in enabled repos
# (Iterate over copy of packages, because we modify it in this loop)
for package in packages.copy():
if package not in buildable_pkgs:
print(f"{package}: not in current repo, skipping")
packages.remove(package)
# FIXME: this should probably be more generic, if other repos are added
# later?
# This just tosses the package into the list of packages to try
# building later w/ systemd enabled, and assumes it'll be found
# there.
print(f"{package}: adding to list of packages to check in systemd repo")
systemd_pkgs.append(package)
continue
# No packages: skip build
if len(packages) == 0:
print("no packages to check for the current repo")
else:
verify_checksums(packages)
# FIXME: this should probably be more generic, if other repos are added later?
if systemd_pkgs:
common.run_pmbootstrap(["config", "systemd", "always"])
# To fix the ERROR: Chroot 'native' is for the 'edge' channel, but you are on the
# 'systemd-edge' channel. Run 'pmbootstrap zap' to delete your chroots and try again.
# To do this automatically, run 'pmbootstrap config auto_zap_misconfigured_chroots yes'.
common.run_pmbootstrap(["config", "auto_zap_misconfigured_chroots", "yes"])
# No packages: skip build
if len(systemd_pkgs) == 0:
print("no packages to check for the systemd repo")
else:
verify_checksums(systemd_pkgs)
+22 -8
View File
@@ -1,14 +1,14 @@
#!/bin/sh -e
# Copyright 2023 Oliver Smith
# Copyright 2022 Oliver Smith
# SPDX-License-Identifier: GPL-3.0-or-later
# Description: lint with various python tests
# Options: native
# Use 'native' because it requires pmbootstrap.
# https://postmarketos.org/pmb-ci
# https://postmarktos.org/pmb-ci
if [ "$(id -u)" = 0 ]; then
set -x
wget "https://gitlab.postmarketos.org/postmarketOS/ci-common/-/raw/master/install_pmbootstrap.sh"
wget "https://gitlab.com/postmarketOS/ci-common/-/raw/master/install_pmbootstrap.sh"
sh ./install_pmbootstrap.sh pytest
exec su "${TESTUSER:-pmos}" -c "sh -e $0"
fi
@@ -19,14 +19,28 @@ if [ -z "$(command -v pytest)" ]; then
exit 1
fi
pmaports="$(cd "$(dirname "$0")"/..; pwd -P)"
# Wrap pmbootstrap to use this repository for --aports
pmaports="$(cd $(dirname $0)/..; pwd -P)"
_pmbootstrap="$(command -v pmbootstrap)"
pmbootstrap() {
"$_pmbootstrap" --aports="$pmaports" "$@"
}
# Make sure that the work folder format is up to date, and that there are no
# mounts from aborted test cases (pmbootstrap#1595)
pmbootstrap work_migrate
pmbootstrap config aports "$pmaports"
pmbootstrap -q shutdown
# Needed to import "common"
export PYTHONPATH=".ci/lib"
# Make sure we have a valid device (pmbootstrap#1128)
device="$(pmbootstrap config device)"
deviceinfo="$pmaports/device/*/device-$device/deviceinfo"
if ! [ -e $deviceinfo ]; then
echo "ERROR: Could not find deviceinfo file for selected device '$device'."
echo "Expected path: $deviceinfo"
echo "Maybe you have switched to a branch where your device does not exist?"
echo "Use 'pmbootstrap config device qemu-amd64' to switch to a valid device."
exit 1
fi
# Run testcases
pytest -vv --tb=native "$pmaports/.ci/testcases" "$@"
pytest -vv -x --tb=native "$pmaports/.ci/testcases" "$@"
-14
View File
@@ -1,14 +0,0 @@
#!/bin/sh -e
# Description: lint CI related python scripts/tests
# https://postmarketos.org/pmb-ci
if [ "$(id -u)" = 0 ]; then
set -x
apk -q add ruff
exec su "${TESTUSER:-build}" -c "sh -e $0"
fi
set -x
# shellcheck disable=SC2046
ruff check --ignore E501,F401,E722,W605 $(find .ci -name '*.py')
+13 -13
View File
@@ -1,8 +1,8 @@
#!/bin/sh -e
# Copyright 2023 Oliver Smith
# Copyright 2022 Oliver Smith
# SPDX-License-Identifier: GPL-3.0-or-later
# Description: lint all shell scripts
# https://postmarketos.org/pmb-ci
# https://postmarktos.org/pmb-ci
DIR="$(cd "$(dirname "$0")/.." && pwd -P)"
@@ -15,18 +15,19 @@ fi
# Shell: shellcheck
sh_files="
./main/mdss-fb-init-hack/mdss-fb-init-hack.sh
./main/postmarketos-base-ui/rootfs-usr-lib-NetworkManager-dispatcher.d-50-dns-filter.sh
./main/postmarketos-base-ui/rootfs-usr-lib-NetworkManager-dispatcher.d-50-tethering.sh
./main/osk-sdl/unlock.sh
./main/postmarketos-base/rootfs-usr-lib-firmwareload.sh
./main/postmarketos-installkernel/installkernel-pmos
./main/postmarketos-initramfs/init.sh
./main/postmarketos-initramfs/init_functions.sh
./main/postmarketos-mkinitfs/init.sh
./main/postmarketos-mkinitfs/init_functions.sh
./main/postmarketos-mkinitfs-hook-debug-shell/20-debug-shell.sh
./main/postmarketos-mkinitfs-hook-netboot/netboot.sh
./main/postmarketos-ui-os-installer/rootfs-usr-bin-pmos_setup.sh
./main/postmarketos-update-kernel/update-kernel.sh
./main/swclock-offset/swclock-offset-boot.sh
./main/swclock-offset/swclock-offset-shutdown.sh
./main/ttyescape/*.post-install
./main/unl0kr/unlock.sh
./device/community/soc-qcom/call_audio_idle_suspend_workaround.sh
./device/testing/device-apple-mac-aarch64/*.sh
./extra-repos/systemd/postmarketos-base-systemd/rootfs-usr-libexec-systemd-apk-trigger
./main/msm-firmware-loader/*.post-install
$(find . -path './main/postmarketos-ui-*/*.sh')
$(find . -path './main/postmarketos-ui-*/*.pre-install')
@@ -40,12 +41,11 @@ sh_files="
$(find . -path './main/devicepkg-dev/*.sh')
$(find . -path './main/postmarketos-mvcfg/*.sh')
$(find . -path './.ci/**.sh')
$(find . -path '**/tests/*.sh')
$(find . -path '.ci/*.sh')
"
for file in $sh_files; do
echo "Test with shellcheck: $file"
cd "$DIR/$(dirname "$file")"
shellcheck -S warning -e SC1008 -e SC3043 -x "$(basename "$file")"
shellcheck -e SC1008 -x "$(basename "$file")"
done
@@ -7,7 +7,7 @@ import sys
import os
import importlib
import importlib.util
from site import getsitepackages
from distutils.sysconfig import get_python_lib
def path_pmbootstrap():
@@ -15,26 +15,23 @@ def path_pmbootstrap():
code from there.
returns: pmbootstrap installation folder
"""
# This variable is set by pmbootstrap 1.52 and later
# If it's undefined, try to find 'pmbootstrap' in path
bin = os.environ.get("PMBOOTSTRAP_CMD") or shutil.which("pmbootstrap")
# Find 'pmbootstrap' executable
bin = shutil.which("pmbootstrap")
if not bin:
print("ERROR: 'pmbootstrap' not found in $PATH")
sys.exit(1)
# Resolve the symlink and verify the folder
directory = os.path.dirname(os.path.realpath(bin))
if os.path.exists(directory + "/pmb/__init__.py"):
return directory
dir = os.path.dirname(os.path.realpath(bin))
if os.path.exists(dir + "/pmb/__init__.py"):
return dir
directories = getsitepackages()
for directory in directories:
if os.path.exists(directory + "/pmb/__init__.py"):
return directory
dir = get_python_lib()
if os.path.exists(dir + "/pmb/__init__.py"):
return dir
# Symlink not set up properly
print("ERROR: unable to find path to pmbootstrap.py. Try setting 'PMBOOTSTRAP_CMD'.")
print("ERROR: 'pmbootstrap' is not a symlink to pmbootstrap.py")
sys.exit(1)
+1
View File
@@ -0,0 +1 @@
../lib/common.py
+10 -19
View File
@@ -2,33 +2,24 @@
# Copyright 2021 Oliver Smith
# SPDX-License-Identifier: GPL-3.0-or-later
from pathlib import Path
import add_pmbootstrap_to_import_path
import pmb.parse
from pmb.helpers.args import init as init_args
from pmb.types import PmbArgs
import pmb.config
from pmb.core.pkgrepo import pkgrepo_default_path
import pytest
import sys
import os
@pytest.fixture(scope="session", autouse=True)
@pytest.fixture
def args(request):
# Initialize args
pmaports = Path(os.path.realpath(f"{os.path.dirname(__file__)}/../.."))
args = PmbArgs()
args.aports = [pmaports]
args.config = pmb.config.defaults["config"]
args.timeout = 900
args.details_to_stdout = False
args.quiet = False
args.verbose = True
args.offline = False
args.action = "init"
args.cross = False
args.log = None
pmaports = os.path.realpath(f"{os.path.dirname(__file__)}/../..")
sys.argv = ["pmbootstrap",
"--aports", pmaports,
"--log", "$WORK/log_testsuite_pmaports.txt"
"chroot"]
args = pmb.parse.arguments()
init_args(args)
# Initialize logging
pmb.helpers.logging.init(args)
request.addfinalizer(pmb.helpers.logging.logfd.close)
return args
+50 -48
View File
@@ -1,24 +1,15 @@
#!/usr/bin/env python3
# Copyright 2024 Oliver Smith
# Copyright 2021 Oliver Smith
# SPDX-License-Identifier: GPL-3.0-or-later
import fnmatch
import glob
import os
import pytest
import sys
import os
import add_pmbootstrap_to_import_path
import pmb.parse
import pmb.parse._apkbuild
from pmb.core.pkgrepo import pkgrepo_default_path, pkgrepo_iglob, pkgrepo_relative_path
from pmb.core.arch import Arch
# Don't complain if these nicknames are the only maintainers of an APKBUILD,
# because they are actually a group of people
gitlab_groups = [
"@sdm845-mainline",
]
def device_dependency_check(apkbuild, path):
@@ -26,53 +17,66 @@ def device_dependency_check(apkbuild, path):
(e.g. because it should be in a subpackage instead). """
for depend in apkbuild["depends"]:
if depend == "linux-firmware-none":
continue
if (depend.startswith("firmware-") or
depend.startswith("linux-firmware")):
raise RuntimeError("Firmware package '" + depend + "' found in"
" depends of " + path + ". These go into"
" subpackages now, see"
" <https://postmarketos.org/devicepkg>.")
if depend == "mesa-dri-gallium":
raise RuntimeError(f"{path}: mesa-dri-gallium shouldn't be in"
" depends anymore (see pmaports!3478)")
def test_aports_device():
def test_aports_device(args):
"""
Various tests performed on the /device/*/device-* aports.
"""
for path in pkgrepo_iglob("device/*/device-*/APKBUILD"):
for path in glob.iglob(args.aports + "/device/*/device-*/APKBUILD"):
apkbuild = pmb.parse.apkbuild(path)
# Depends: Require "postmarketos-base"
depend_flag = False
for dependency in apkbuild["depends"]:
if "postmarketos-base" == dependency or "postmarketos-base>" in dependency:
depend_flag = True
if not depend_flag:
raise RuntimeError(f"Missing 'postmarketos-base' in depends of {path}")
if "postmarketos-base" not in apkbuild["depends"]:
raise RuntimeError("Missing 'postmarketos-base' in depends of " +
path)
# Depends: Must not have specific packages
# Depends: Must not have firmware packages
for depend in apkbuild["depends"]:
device_dependency_check(apkbuild, path)
# Architecture
device = apkbuild["pkgname"][len("device-"):]
deviceinfo = pmb.parse.deviceinfo(device)
if Arch.from_str("".join(apkbuild["arch"])) != deviceinfo.arch:
raise RuntimeError("wrong architecture, please change to arch=\""
f"{deviceinfo.arch}\": {path}")
deviceinfo = pmb.parse.deviceinfo(args, device)
if "".join(apkbuild["arch"]) != deviceinfo["arch"]:
raise RuntimeError("wrong architecture, please change to arch=\"" +
deviceinfo["arch"] + "\": " + path)
if "!archcheck" not in apkbuild["options"]:
raise RuntimeError(f"!archcheck missing in options= line: {path}")
raise RuntimeError("!archcheck missing in options= line: " + path)
if deviceinfo["flash_kernel_on_update"] == "true" \
and "postmarketos-update-kernel" not in apkbuild["depends"]:
raise RuntimeError(
"Deviceinfo indicates this device supports automatic kernel"
" flashing on update, but doesn't depend on"
" 'postmarketos-update-kernel'. Please add the missing"
" dependency.")
def test_aports_device_kernel():
def test_aports_device_kernel(args):
"""
Verify the kernels specified in the device packages:
* Kernel must not be in depends when kernels are in subpackages
* Check if only one kernel is defined in depends
"""
aports = pkgrepo_default_path()
# Iterate over device aports
for path in aports.glob("device/*/device-*/APKBUILD"):
for path in glob.glob(args.aports + "/device/*/device-*/APKBUILD"):
# Parse apkbuild and kernels from subpackages
apkbuild = pmb.parse.apkbuild(path)
device = apkbuild["pkgname"][len("device-"):]
kernels_subpackages = pmb.parse._apkbuild.kernels(device)
kernels_subpackages = pmb.parse._apkbuild.kernels(args, device)
# Parse kernels from depends
kernels_depends = []
@@ -83,50 +87,48 @@ def test_aports_device_kernel():
# Kernel in subpackages *and* depends
if kernels_subpackages:
raise RuntimeError(f"Kernel package '{depend}' needs to"
raise RuntimeError("Kernel package '" + depend + "' needs to"
" be removed when using kernel" +
f" subpackages: {path}")
" subpackages: " + path)
# No kernel
if not kernels_depends and not kernels_subpackages:
raise RuntimeError("Device doesn't have a kernel in depends or"
f" subpackages: {path}")
" subpackages: " + path)
# Multiple kernels in depends
if len(kernels_depends) > 1:
raise RuntimeError("Please use kernel subpackages instead of"
" multiple kernels in depends (see"
f" <https://postmarketos.org/devicepkg>): {path}")
" <https://postmarketos.org/devicepkg>): " +
path)
# @pytest.mark.xfail # Not all aports have been updated yet
def test_aports_maintained():
def test_aports_maintained(args):
"""
Ensure that aports in /device/{main,community} have "Maintainer:" and
"Co-Maintainer:" (only required for main) listed in their APKBUILDs.
"""
for path in pkgrepo_iglob("device/main/*/APKBUILD"):
if 'firmware-' in path.parent.name:
for path in glob.iglob(f"{args.aports}/device/main/*/APKBUILD"):
if '/firmware-' in path:
continue
maintainers = pmb.parse._apkbuild.maintainers(path)
assert maintainers and len(maintainers) >= 2, \
f"{path} in main needs at least 1 Maintainer and 1 Co-Maintainer"
for path in pkgrepo_iglob("device/community/*/APKBUILD"):
if 'firmware-' in path.parent.name:
for path in glob.iglob(f"{args.aports}/device/community/*/APKBUILD"):
if '/firmware-' in path:
continue
maintainers = pmb.parse._apkbuild.maintainers(path)
assert maintainers, f"{path} in community needs at least 1 Maintainer"
def test_aports_archived():
def test_aports_unmaintained(args):
"""
Ensure that aports in /device/archived have an "Archived:" comment
that describes why the aport is archived.
Ensure that aports in /device/unmaintained have an "Unmaintained:" comment
that describes why the aport is unmaintained.
"""
for path in pkgrepo_iglob("device/archived/*/APKBUILD"):
archived = pmb.parse._apkbuild.archived(path)
assert archived, f"{path} should have an Archived: " +\
"comment that describes why the package is archived"
for path in glob.iglob(f"{args.aports}/device/unmaintained/*/APKBUILD"):
unmaintained = pmb.parse._apkbuild.unmaintained(path)
assert unmaintained, f"{path} should have an Unmaintained: " +\
"comment that describes why the package is unmaintained"
+34 -13
View File
@@ -10,46 +10,67 @@ import sys
import add_pmbootstrap_to_import_path
import pmb.parse
from pmb.core.pkgrepo import pkgrepo_iglob
def test_deviceinfo():
def deviceinfo_obsolete(info):
"""
Test for obsolete options used in the deviceinfo file. They must still be
defined in pmbootstrap's config/__init__.py.
"""
obsolete_options = ["weston_pixman_type"]
for option in obsolete_options:
if info[option]:
raise RuntimeError("option '" + option + "' is obsolete, please"
" remove it (reasons for removal are at"
" <https://postmarketos.org/deviceinfo>)")
def test_deviceinfo(args):
"""
Parse all deviceinfo files successfully and run checks on the parsed data.
"""
# Iterate over all devices
last_exception = None
count = 0
pattern = re.compile("^deviceinfo_[a-zA-Z0-9_]*=\".*\"(\\s*# .*)?$")
pattern = re.compile("^deviceinfo_[a-zA-Z0-9_]*=\".*\"$")
for folder in pkgrepo_iglob("device/*/device-*"):
device = folder.name.split("-", 1)[1]
for folder in glob.glob(args.aports + "/device/*/device-*"):
device = folder[len(args.aports):].split("-", 1)[1]
f = open(folder / "deviceinfo")
f = open(folder[len(args.aports):][1:] + "/deviceinfo")
lines = f.read().split("\n")
f.close()
try:
for line in lines:
# Require space after # for comments
if line.startswith("#") and not line.startswith("# "):
raise RuntimeError("Comment style: please change '#' to"
f" '# ': {line}")
# Skip empty lines and comments
if not line or line.startswith("# "):
continue
# Variable can not be empty
if '=""' in line:
raise RuntimeError("Please remove the empty variable: " + line)
# Check line against regex (can't use multiple lines etc.)
if not pattern.match(line) or line.endswith("\\\""):
raise RuntimeError("Line looks invalid, maybe missing"
" quotes/multi-line string/malformed"
f" inline comment? {line}")
" quotes/multi-line string/comment next"
f" to line instead of above? {line}")
# Successful deviceinfo parsing / obsolete options
info = pmb.parse.deviceinfo(device)
info = pmb.parse.deviceinfo(args, device)
deviceinfo_obsolete(info)
# deviceinfo_name must start with manufacturer
name = info.name
manufacturer = info.manufacturer
name = info["name"]
manufacturer = info["manufacturer"]
if not name.startswith(manufacturer) and \
not name.startswith("Google") and \
not name.startswith("Jolla"):
not name.startswith("Google"):
raise RuntimeError("Please add the manufacturer in front of"
" the deviceinfo_name, e.g.: '" +
manufacturer + " " + name + "'")
+9 -15
View File
@@ -6,12 +6,10 @@ import stat
expected_directories = [
"cross",
"device/archived",
"device/community",
"device/downstream",
"device/main",
"device/testing",
"extra-repos/systemd",
"device/unmaintained",
"main",
"modem",
"non-free",
@@ -30,9 +28,9 @@ def test_directories():
# Ensure no file in pmaports are executable.
# see https://gitlab.postmarketos.org/postmarketOS/pmaports/-/issues/593.
# see https://gitlab.com/postmarketOS/pmaports/-/issues/593.
def test_executable_files():
for file in glob.iglob("[!temp]**/**/*", recursive=True):
for file in glob.iglob("**/*", recursive=True):
if os.path.isdir(file) or os.path.islink(file):
continue
# still check other less common inode types
@@ -46,20 +44,20 @@ def test_executable_files():
"m0755` or a variation thereof.")
# Make sure files belong to a package (below a directory with APKBUILD)
# Make sure files are either:
# - in root directory (README.md)
# - hidden (.ci/, device/.shared-patches/)
# - or belong to a package (below a directory with APKBUILD)
def test_files_belong_to_package():
# Walk directories and set package_dir when we find an APKBUILD
# This allows matching files in subdirectories to the package directory.
package_dir = None
for dirpath, dirs, files in os.walk("."):
# Skip "hidden" directories (.ci/, device/.shared-patches/)
# Skip "hidden" directories
dirs[:] = [d for d in dirs if not d.startswith(".")]
# Ignore files in root directory (README.md)
# Ignore files in root directory
if dirpath == '.':
continue
# Ignore files under docs/
if dirpath.startswith('./docs'):
continue
# Switched to another directory?
if package_dir and not dirpath.startswith(package_dir + os.sep):
@@ -70,9 +68,5 @@ def test_files_belong_to_package():
"both contain an APKBUILD"
package_dir = dirpath
# Skip specific ".gitlab-ci.yml" files outside of usual structure
if dirpath in ["./cross"]:
files[:] = [f for f in files if f != ".gitlab-ci.yml"]
assert not files or package_dir, "Found files that do not belong to any package: " \
f"{dirpath}/{files}"
+3 -6
View File
@@ -7,16 +7,14 @@ import os
import add_pmbootstrap_to_import_path
import pmb.parse
from pmb.core.pkgrepo import pkgrepo_default_path, pkgrepo_iglob
def test_aports_firmware():
def test_aports_firmware(args):
"""
Various tests performed on the /**/firmware-* aports.
"""
excluded = [
"firmware-motorola-ali", # Depends on firmware-qcom-adreno-a530
"firmware-motorola-potter", # Depends on soc-qcom-msm8916-ucm
"firmware-oneplus-msm8998", # Depends on soc-qcom-sdm845-nonfree-firmware
"firmware-xiaomi-sagit", # Depends on soc-qcom-sdm845-nonfree-firmware
@@ -25,12 +23,11 @@ def test_aports_firmware():
"firmware-samsung-maguro", # Depends on firmware-aosp-broadcom-wlan
"firmware-xiaomi-ferrari", # Depends on soc-qcom-msm8916
"firmware-xiaomi-willow", # Doesn't build, source link is dead (pma#1212)
"firmware-samsung-i9105p", # Depends on firmware-aosp-broadcom-wlan
]
for path in pkgrepo_iglob("**/firmware-*/APKBUILD", recursive=True):
for path in glob.iglob(f"{args.aports}/**/firmware-*/APKBUILD", recursive=True):
apkbuild = pmb.parse.apkbuild(path)
aport_name = os.path.basename(path.parent)
aport_name = os.path.basename(os.path.dirname(path))
if aport_name not in excluded:
if "pmb:cross-native" not in apkbuild["options"]:
+4 -5
View File
@@ -15,10 +15,9 @@ import sys
import add_pmbootstrap_to_import_path
import pmb.config
import pmb.parse
from pmb.core.pkgrepo import pkgrepo_iter_package_dirs
def get_categorized_packages():
def get_categorized_packages(args):
"""
Parse all aports and categorize them.
@@ -29,7 +28,7 @@ def get_categorized_packages():
"""
ret = {}
for path in pkgrepo_iter_package_dirs():
for path in glob.glob(args.aports + "/*/*/APKBUILD"):
# Parse APKBUILD
apkbuild = pmb.parse.apkbuild(path)
url = apkbuild["url"]
@@ -110,10 +109,10 @@ def check_categories(categories):
return ret
def test_framework_versions():
def test_framework_versions(args):
"""
Make sure that packages of the same framework have the same version.
"""
categories = get_categorized_packages()
categories = get_categorized_packages(args)
if not check_categories(categories):
raise RuntimeError("Framework version check failed!")
+4 -17
View File
@@ -6,26 +6,20 @@ import glob
import pytest
import sys
import os
import re
from pathlib import Path
import add_pmbootstrap_to_import_path
import pmb.parse
from pmb.core.pkgrepo import pkgrepo_iglob, pkgrepo_relative_path
def test_aports_kernel():
def test_aports_kernel(args):
"""
Various tests performed on the /**/linux-* aports.
"""
for path in pkgrepo_iglob("**/linux-*/APKBUILD", recursive=True):
for path in glob.iglob(f"{args.aports}/**/linux-*/APKBUILD", recursive=True):
apkbuild = pmb.parse.apkbuild(path)
aport_name = os.path.basename(os.path.dirname(path))
if aport_name == "linux-pam-pmos":
continue # This package isn't a linux kernel!
if "pmb:cross-native" not in apkbuild["options"]:
raise RuntimeError(f"{aport_name}: \"pmb:cross-native\" missing in"
" options= line")
@@ -41,15 +35,8 @@ def test_aports_kernel():
" automatically.")
# check some options only for main and community devices
_, relpath = pkgrepo_relative_path(path)
for part in ["main", "community"]:
if part in relpath.parts:
for dir in ["main", "device/main", "device/community"]:
if path.startswith(f"{args.aports}/{dir}"):
if "pmb:kconfigcheck-community" not in apkbuild["options"]:
raise RuntimeError(f"{aport_name}: \"pmb:kconfigcheck-community\" missing in"
" options= line, required for all community/main devices.")
# check for postmarketos-installkernel in makedepends when installing kernel with make
if bool(re.search("make z?install", path.read_text(encoding="utf-8"))):
if "postmarketos-installkernel" not in apkbuild["makedepends"]:
raise RuntimeError(f"{aport_name}: \"postmarketos-installkernel\" missing in"
" makedepends, required when using make install/zinstall.")
+23 -30
View File
@@ -1,17 +1,15 @@
#!/usr/bin/env python3
# Copyright 2023 Oliver Smith
# Copyright 2022 Oliver Smith
# SPDX-License-Identifier: GPL-3.0-or-later
import glob
import logging
import os
import pytest
import sys
import re
import add_pmbootstrap_to_import_path
import pmb.parse
import pmb.parse._apkbuild
from pmb.core.pkgrepo import pkgrepo_iter_package_dirs, pkgrepo_relative_path
def apkbuild_check_provides(path, apkbuild, version, pkgname, subpkgname=None):
@@ -45,40 +43,35 @@ def apkbuild_check_provides(path, apkbuild, version, pkgname, subpkgname=None):
# already replaces the variables, so we check against the inserted
# values here.
if not provide.endswith(f"={version}"):
# Valid version strings, per the APKBUILD reference, are fine. This regex
# attempts to detect those.
pattern = r'^\d+(\.\d+)*[a-z]?(_(?:alpha|beta|pre|rc|cvs|svn|git|hg|p)\d*)*-r\d+$'
[provide_no_ver, provide_ver] = provide.split("=", 1)
if not re.match(pattern, provide_ver):
error = f"error in , provide_ver{path}:\n"
if subpkgname:
error += f"in subpackage:\n {subpkgname}\n"
else:
error += f"in package:\n {pkgname}\n"
error += f"broken provides entry:\n {provide_no_ver}\n"
error += "\n"
error += "This provides entry needs to be changed to"
error += f" '{provide_no_ver}=$pkgver-r$pkgrel'"
error += " (do not replace the variables, without '').\n"
error += "\n"
error += "If you know what you are doing and didn't add the"
error += " version on purpose, you also need to set a"
error += " provider_priority (pma#1766).\n"
error += "Reference:"
error += " https://wiki.alpinelinux.org/wiki/APKBUILD_Reference#provides"
error += f"\nVERSION: {provide_no_ver}"
ret += [error]
provide_no_ver = provide.split("=", 1)[0]
error = f"error in {path}:\n"
if subpkgname:
error += f"in subpackage:\n {subpkgname}\n"
else:
error += f"in package:\n {pkgname}\n"
error += f"broken provides entry:\n {provide_no_ver}\n"
error += "\n"
error += "This provides entry needs to be changed to"
error += f" '{provide_no_ver}=$pkgver-r$pkgrel'"
error += " (do not replace the variables, without '').\n"
error += "\n"
error += "If you know what you are doing and didn't add the"
error += " version on purpose, you also need to set a"
error += " provider_priority (pma#1766).\n"
error += "Reference:"
error += " https://wiki.alpinelinux.org/wiki/APKBUILD_Reference#provides"
ret += [error]
return ret
def test_provides():
def test_provides(args):
errors = []
for path in pkgrepo_iter_package_dirs():
path = path / "APKBUILD"
for path in glob.iglob(f"{args.aports}/**/APKBUILD", recursive=True):
apkbuild = pmb.parse.apkbuild(path)
pkgname = apkbuild["pkgname"]
version = f"{apkbuild['pkgver']}-r{apkbuild['pkgrel']}"
_, path_rel = pkgrepo_relative_path(path)
path_rel = os.path.relpath(path, args.aports)
errors += apkbuild_check_provides(path_rel, apkbuild, version, pkgname)
for subpkg, subpkg_data in apkbuild["subpackages"].items():
+15 -65
View File
@@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright 2023 Oliver Smith
# Copyright 2022 Oliver Smith
# SPDX-License-Identifier: GPL-3.0-or-later
# Various checks on source= in the APKBUILDs
@@ -9,76 +9,28 @@ import os
import pytest
import sys
# Same dir
import common
# pmbootstrap
import add_pmbootstrap_to_import_path
import pmb.parse
import pmb.parse._apkbuild
import pmb.parse.apkindex
import pmb.helpers.repo
from pmb.core.pkgrepo import pkgrepo_iter_package_dirs, pkgrepo_relative_path
def parse_source_from_checksums(apkbuild_path):
"""
Read the APKBUILD file and parse source files from the checksums at the
bottom. This has always the same format, even if $source is built with hard
to parse shell code (like in postmarketos-base).
:param apkbuild_path: full path to the APKBUILD
:returns: dict of parsed "source" filenames and checksums, e.g.:
{"first.patch": "b4dc4f3…",
"second.patch": "b4dc4f3…"}
"""
start = 'sha512sums="'
in_block = False
ret = {}
with open(apkbuild_path, encoding="utf-8") as handle:
for line in handle.readlines():
# Find start
if not in_block:
if line.startswith(start):
in_block = True
else:
continue
# sha512sums may have lines without checksums:
# https://gitlab.alpinelinux.org/alpine/abuild/-/merge_requests/73
if " " not in line:
continue
try:
checksum, filename = line.rstrip().split(" ", 2)
except ValueError:
raise ValueError("Failed to parse checksums. Try to delete the"
" checksums and generate them again with"
f" 'pmbootstrap checksum': {apkbuild_path}")
# Cut off 'sha512sums="' if the first checksum is in that line
if checksum.startswith(start):
checksum = checksum[len(start):]
# Find end
if filename.endswith('"'):
filename = filename[:-1]
ret[filename] = checksum
return ret
def test_aports_unreferenced_files():
def test_aports_unreferenced_files(args):
"""
Raise an error if an unreferenced file is found
"""
for apkbuild_path in pkgrepo_iter_package_dirs():
for apkbuild_path in glob.iglob(args.aports + "/**/APKBUILD", recursive=True):
# pmbootstrap parser has some issues with complicated APKBUILDs, skip those.
_, relpath = pkgrepo_relative_path(apkbuild_path)
if relpath.parts[0] == "cross":
if apkbuild_path.startswith(args.aports + "/cross/"):
continue
apkbuild_path = apkbuild_path / "APKBUILD"
apkbuild = pmb.parse.apkbuild(apkbuild_path)
sources_chk = parse_source_from_checksums(apkbuild_path)
sources_chk = common.parse_apkbuild_checksums(args, apkbuild_path)
# Collect files from subpackages
subpackage_installs = []
@@ -99,18 +51,17 @@ def test_aports_unreferenced_files():
for file in glob.iglob(dirname + "/**", recursive=True):
rel_file_path = os.path.relpath(file, dirname)
# Skip APKBUILDs and directories
if rel_file_path in ["APKBUILD", "gitlab-ci.yml.j2"] or os.path.isdir(file):
if rel_file_path == "APKBUILD" or os.path.isdir(file):
continue
if os.path.basename(rel_file_path) not in sources_chk \
and rel_file_path not in apkbuild["install"] \
and rel_file_path not in subpackage_installs \
and rel_file_path not in trigger_sources \
and not rel_file_path.startswith("tests/"):
and rel_file_path not in trigger_sources:
raise RuntimeError(f"{apkbuild_path}: found unreferenced file: {rel_file_path}")
def test_distfiles_conflict():
def test_distfiles_conflict(args):
"""
Make sure that each filename mentioned in any source= of any APKBUILD
always has the same checksum. This is important because apk caches
@@ -120,11 +71,10 @@ def test_distfiles_conflict():
with a checksum error.
"""
source_all = {}
for apkbuild_path in pkgrepo_iter_package_dirs():
apkbuild_path = apkbuild_path / "APKBUILD"
source = parse_source_from_checksums(apkbuild_path)
for apkbuild_path in glob.iglob(f"{args.aports}/**/APKBUILD", recursive=True):
source = common.parse_apkbuild_checksums(args, apkbuild_path)
dir_path = os.path.dirname(apkbuild_path)
_, apkbuild_rel = pkgrepo_relative_path(apkbuild_path)
apkbuild_rel = os.path.relpath(apkbuild_path, args.aports)
for filename, checksum in source.items():
# Files bundled with the APKBUILD don't get copied to the distfiles
# cache, so not relevant for this check. Use glob.glob here and not
-25
View File
@@ -1,25 +0,0 @@
#!/usr/bin/env python3
# Copyright 2025 Oliver Smith
# SPDX-License-Identifier: GPL-3.0-or-later
import logging
import os
import pmb.parse
from pmb.core.context import get_context
from pathlib import Path
def test_systemd_stage0_version():
"""Ensure systemd-stage0 and systemd have the same pkgver."""
# Don't clutter output with verbose APKBUILD parsing messages
logging.getLogger().setLevel(logging.DEBUG)
config = get_context().config
pma_systemd = os.path.join(config.aports[0], "extra-repos/systemd")
sd0 = pmb.parse.apkbuild(Path(os.path.join(pma_systemd, "systemd-stage0/APKBUILD")))
sd1 = pmb.parse.apkbuild(Path(os.path.join(pma_systemd, "systemd/APKBUILD")))
assert sd0["pkgver"] == sd1["pkgver"], (
f"systemd-stage0 pkgver ({sd0['pkgver']}) doesn't match systemd pkgver ({sd1['pkgver']})!"
)
+16 -21
View File
@@ -2,36 +2,32 @@
# Copyright 2021 Oliver Smith
# SPDX-License-Identifier: GPL-3.0-or-later
from pathlib import Path
import glob
import pytest
import sys
import os
import common
import add_pmbootstrap_to_import_path
import pmb.parse
def test_aports_ui():
def test_aports_ui(args):
"""
Raise an error if package in _pmb_recommends is not found
"""
pmaports_cfg = pmb.config.pmaports.read_config()
pmaports_cfg = pmb.config.pmaports.read_config(args)
for arch in pmaports_cfg["supported_arches"].split(","):
for path in common.get_changed_files():
path = Path(path)
if path.name != "APKBUILD":
continue
if not path.parent.name.startswith("postmarketos-ui"):
continue
for path in glob.iglob(args.aports + "/main/postmarketos-ui-*/APKBUILD"):
apkbuild = pmb.parse.apkbuild(path)
# Skip if arch isn't enabled
if not pmb.helpers.package.check_arch(apkbuild["pkgname"], arch, False):
if not pmb.helpers.package.check_arch(args, apkbuild["pkgname"], arch, False):
continue
for package in apkbuild["_pmb_recommends"]:
depend = pmb.helpers.package.get(
package, arch, must_exist=False, try_other_arches=False
)
if depend is None:
depend = pmb.helpers.package.get(args, package,
arch, must_exist=False)
if depend is None or not pmb.helpers.package.check_arch(args, package, arch):
raise RuntimeError(f"{path}: package '{package}' from"
f" _pmb_recommends not found for arch '{arch}'")
@@ -39,9 +35,8 @@ def test_aports_ui():
if f"{apkbuild['pkgname']}-extras" in apkbuild["subpackages"]:
apkbuild = apkbuild["subpackages"][f"{apkbuild['pkgname']}-extras"]
for package in apkbuild["_pmb_recommends"]:
depend = pmb.helpers.package.get(
package, arch, must_exist=False, try_other_arches=False
)
if depend is None:
depend = pmb.helpers.package.get(args, package,
arch, must_exist=False)
if depend is None or not pmb.helpers.package.check_arch(args, package, arch):
raise RuntimeError(f"{path}: package '{package}' from _pmb_recommends "
f"of -extras subpackage is not found for arch '{arch}'")
-19
View File
@@ -1,19 +0,0 @@
#!/bin/sh -e
# Description: run unit-tests for packages
# https://postmarketos.org/pmb-ci
#
# This script only exists to allow running unit-tests locally.
# In actual CI the package tests/ subdir has it's own YAML
# fine defining a job which calls testlib for that package.
# This way we can run all the package unit-tests locally but
# have them be individual jobs in GitLab.
if [ "$(id -u)" = 0 ]; then
set -x
exec su "${TESTUSER:-build}" -c "sh -e $0"
fi
find . -type d -name "tests" | while read -r testdir; do
pkg="$(basename "$(dirname "$testdir")")"
.ci/lib/testlib.sh "$testdir" "$pkg"
done
-16
View File
@@ -1,16 +0,0 @@
#!/bin/sh -e
# Description: verify checksums of modified packages
# Options: native
# Use 'native' because it requires running pmbootstrap.
# https://postmarketos.org/pmb-ci
if [ "$(id -u)" = 0 ]; then
set -x
wget "https://gitlab.postmarketos.org/postmarketOS/ci-common/-/raw/master/install_pmbootstrap.sh"
sh ./install_pmbootstrap.sh
exec su "${TESTUSER:-pmos}" -c "sh -e $0"
fi
export PYTHONUNBUFFERED=1
.ci/lib/verify_checksums.py
+2 -4
View File
@@ -2,16 +2,14 @@
# Description: verify devices are documented in the wiki
# Options: native
# Run natively so we don't need to set up a chroot, python3 is the only dep.
# https://postmarketos.org/pmb-ci
# https://postmarktos.org/pmb-ci
if [ "$(id -u)" = 0 ]; then
set -x
apk -q add \
git \
python3
exec su "${TESTUSER:-build}" -c "sh -e $0"
fi
set -x
# shellcheck disable=SC2068
.ci/lib/check_devices_in_wiki.py --booting $@
.ci/lib/check_devices_in_wiki.py --booting
+2 -2
View File
@@ -1,4 +1,4 @@
{
"SpacesAftertabs": true,
"Exclude": ["temp"]
"Version": "2.6.0",
"SpacesAftertabs": true
}
+1 -13
View File
@@ -7,20 +7,8 @@
# Top-most EditorConfig file.
root = true
[{**.{ini,sh,rules,nft},**/APKBUILD,CODEOWNERS}]
[{**.{md,ini,sh,rules,nft},**/APKBUILD}]
indent_style = tab
end_of_line = lf
trim_trailing_whitespace = true
insert_final_newline = true
[**.md]
# No indent_size since space size has meaning in markdown
indent_style = space
end_of_line = lf
trim_trailing_whitespace = true
insert_final_newline = true
# Forked from Alpine
[extra-repos/systemd/networkmanager/networkmanager.rules]
indent_style = space
indent_size = 4
-129
View File
@@ -1,129 +0,0 @@
#!/bin/sh
#
# This hook checks that all local sources specified in the staged APKBUILDs
# are staged too or already committed and that checksums are correct.
#
set -eu
# Maximal allowed size (in bytes) of a file.
FILE_SIZE_LIMIT=262144 # 256 kiB
if ! command -v sha512sum >/dev/null; then
# macOS / BSDs (?) don't have sha512sum, but shasum.
alias sha512sum='shasum -a 512'
fi
error() {
printf '\033[0;31mpre-commit:\033[0m %s\n' "$1" >&2 # red
}
# Prints paths of created or modified files being committed.
changed_files() {
git diff-index \
--name-only \
--cached \
--diff-filter=ACMR HEAD \
-- "$@"
}
# Prints file names and checksums (in format <SHA-512>:<filename>) of local
# sources specified in the APKBUILD ($1).
apkbuild_local_sources() {
apkbuild="$1"
set +eu
. "$apkbuild" || {
error "$apkbuild is invalid"
return 1
}
set -eu
status=0
: ${source:=""}
for src in $source; do
# Skip remote sources.
case "$src" in */*) continue;; esac
echo "$sha512sums" | awk -v src="$src" '
{ if ($2 == src) { ok=1; print($2 ":" $1) } }
END { if (ok != 1) exit 1 }' || {
status=1
error "${apkbuild%/*}: file \"$src\" is missing in \$sha512sums (hint: run pmbootstrap checksum)"
}
done
return $status
}
# check if given object is a symlink
is_symlink() {
test "$(git ls-files --stage "$1" | awk '{print $1}')" = "120000"
}
# Checks that all local sources specified in the APKBUILD file ($1) are
# available in git tree and checksums are correct.
check_local_sources() {
local apkbuild="$1"
local startdir="$2"
local status=0
local checksum_act checksum_exp content filename line sources
sources=$(apkbuild_local_sources "$apkbuild")
for line in $sources; do
filename=${line%%:*}
checksum_exp=${line#*:}
if ! git cat-file -e ":$startdir/$filename" 2>/dev/null; then
error "$startdir: missing file \"$filename\""
status=1
continue
fi
if is_symlink ":$startdir/$filename"; then
continue
fi
checksum_act=$(git show ":$startdir/$filename" | sha512sum)
if [ "$checksum_act" != "$checksum_exp -" ]; then
local cmd="pmbootstrap checksum $(basename "$startdir")"
error "$startdir: bad checksum for file \"$filename\" (hint: run $cmd)"
status=1
fi
done
return $status
}
# Checks if the file ($1) being committed is not bigger than FILE_SIZE_LIMIT.
check_file_size() {
local path="$1"
local size
size=$(git cat-file -s ":$path")
if [ $size -gt $FILE_SIZE_LIMIT ]; then
local size_kb=$(( size / 1024 ))
error "file \"$path\" is quite big ($(( size / 1024 )) kiB), better put a remote url in source="
return 1
fi
}
cleanup() {
[ -f "$_staged" ] && rm "$_staged"
}
trap cleanup EXIT
for apkbuild in $(changed_files '**/APKBUILD'); do
_staged="$(mktemp)"
git show ":$apkbuild" >"$_staged"
check_local_sources "$_staged" "${apkbuild%/*}"
cleanup
done
for path in $(changed_files); do
check_file_size "$path"
done
trap - EXIT
-72
View File
@@ -1,72 +0,0 @@
#!/bin/sh
#
# This hook adds prefix "<repo>/<pkgname>: " to the commit message when
# committing changes of a single package.
#
MSG_FILE="$1"
SOURCE="$2"
longest_common_prefix() {
awk -F/ '
(NR == 1) { split($0, prefix); prefix_len = NF }
(NR > 1) {
for (i = 1; i <= prefix_len; i++) {
if (prefix[i] != $i) {
prefix_len = i - 1; break
}
}
}
(prefix_len == 0) { exit }
END {
res = prefix[1]
for (i = 2; i <= prefix_len; i++) {
res = res FS prefix[i]
}
print(res)
}'
}
prepend_msg() {
local prefix="$1"
printf '%s\n%s\n' "$prefix" "$(cat "$MSG_FILE")" > "$MSG_FILE"
}
# Do nothing if message has been given using -m, template, merge etc.
[ -z "$SOURCE" ] || exit 0
lcp=$(git diff-index --name-only --cached HEAD | longest_common_prefix)
prefix=""
case "$lcp" in
extra-repos/systemd/systemd/*)
prefix="systemd"
;;
extra-repos/systemd/*)
prefix="systemd/$(echo "$lcp" | cut -d/ -f3)"
;;
device/*/*)
prefix="$(echo "$lcp" | cut -d/ -f3)"
;;
docs/*.adoc)
prefix="docs/$(basename $lcp .adoc)"
;;
docs/*)
prefix="docs"
;;
device/*)
# Multiple packages touched for same device category. More
# logic could be added to detect if they all belong to same
# device.
;;
[^.]*/*)
prefix="$(echo "$lcp" | cut -d/ -f1-2)"
;;
esac
if [ -n "$prefix" ]; then
prepend_msg "$prefix: "
fi
exit 0
+2 -6
View File
@@ -65,9 +65,8 @@ instance/
# Scrapy stuff:
.scrapy
# Documentation
public
docs/deviceinfo-reference.md
# Sphinx documentation
docs/_build/
# PyBuilder
target/
@@ -119,6 +118,3 @@ ENV/
# Allow having untracked packages in aports using a custom-something directory
/custom-*/
# child pipeline
.ci/build-jobs.yaml
+84 -168
View File
@@ -6,64 +6,56 @@ after_script:
- .ci/lib/move_logs.sh $CI_PROJECT_DIR
stages:
- lint
- prepare
- build
- autoupdate
- test
- deploy
variables:
# Default of 20 is pretty small, sometimes MRs have >20 commits in them. 100
# seems like a reasonable balance. If this is too low, git merge-base (from ci/
# common.py) can fail
GIT_DEPTH: 100
# This defines the rules for when a pipeline should run.
workflow:
rules:
# Run for merge requests
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
# Some things have to run in the default branch
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
# Run scheduled pipeline for autoupdate or manually triggered pipeline
- if: $CI_PIPELINE_SOURCE == "schedule" || $CI_PIPELINE_SOURCE == "web"
# Some packages have their own yaml files defining tests. All of these should contain
# only jobs which extend the .package-test job!
include: "**/.gitlab-ci.yml"
# defaults for "only"
# We need to run the CI jobs in a "merge request specific context", if CI is
# running in a merge request. Otherwise the environment variable that holds the
# merge request ID is not available. This means, we must set the "only"
# variable accordingly - and if we only do it for one job, all other jobs will
# not get executed. So have the defaults here, and use them in all jobs that
# should run on both the master branch, and in merge requests.
# https://docs.gitlab.com/ee/ci/merge_request_pipelines/index.html#excluding-certain-jobs
.only-default: &only-default
only:
- master
- merge_requests
- tags
# device documentation
wiki:
stage: lint
rules:
- if: &mr_rule $CI_PIPELINE_SOURCE == "merge_request_event"
changes:
- .ci/**/*
- .ci/build-jobs.yaml.j2
- .gitlab-ci.yml
- device/*/device-*/*
<<: *only-default
only:
refs:
- merge_requests
changes:
- device/*/device-*/*
script:
- .ci/lib/gitlab_prepare_ci.sh
- .ci/wiki.sh
# testcases linting
ruff:
flake8:
stage: lint
rules:
- if: *mr_rule
changes:
- .ci/*
- .ci/*/*
- .ci/build-jobs.yaml.j2
- .gitlab-ci.yml
<<: *only-default
only:
refs:
- merge_requests
changes:
- .ci/*
- .ci/*/*
before_script:
- apk -q add py-flake8
script:
- .ci/lib/gitlab_prepare_ci.sh
- .ci/ruff.sh
- .ci/flake8.sh
# shellcheck and various grep checks
shellcheck-grep:
stage: lint
rules:
- if: *mr_rule
<<: *only-default
image: alpine:edge
script:
- .ci/lib/gitlab_prepare_ci.sh
- .ci/shellcheck.sh
@@ -71,8 +63,8 @@ shellcheck-grep:
editor-config:
stage: lint
rules:
- if: *mr_rule
<<: *only-default
image: alpine:edge
script:
- .ci/lib/gitlab_prepare_ci.sh
- .ci/ec.sh
@@ -80,8 +72,7 @@ editor-config:
# aports checks (generic)
pytest-commits:
stage: lint
rules:
- if: *mr_rule
<<: *only-default
script:
- .ci/lib/gitlab_prepare_ci.sh
- .ci/pytest.sh
@@ -97,154 +88,79 @@ pytest-commits:
# APKBUILD linting
aport-lint:
stage: lint
rules:
- if: *mr_rule
script:
- .ci/lib/gitlab_prepare_ci.sh
- .ci/apkbuild-lint.sh
only:
- merge_requests
allow_failure: true
# deviceinfo linting
dint:
stage: lint
rules:
- if: *mr_rule
changes:
- .ci/**/*
- .gitlab-ci.yml
- device/*/device-*/deviceinfo
script:
- .ci/lib/gitlab_prepare_ci.sh
- .ci/dint.sh
allow_failure:
exit_codes:
- 2 # returned by dint on non-fatal warning
# kernel kconfig check
kernel-kconfig:
stage: lint
rules:
- if: *mr_rule
changes:
- kconfigcheck.toml
allow_failure: true
- if: *mr_rule
changes:
- device/*/linux-*/config-*
only:
refs:
- master
- merge_requests
changes:
- device/*/linux-*/config-*
- main/linux-*/config-*
script:
- .ci/lib/gitlab_prepare_ci.sh
- .ci/kconfig.sh
# Verify checksums
verify-checksums:
stage: lint
rules:
- if: *mr_rule
script:
- .ci/lib/gitlab_prepare_ci.sh
- .ci/verify-checksums.sh
# MR settings
# (Checks for "Allow commits from members who can merge to the target branch")
mr-settings:
stage: lint
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
only:
- merge_requests
before_script:
- apk -q add python3
script:
- wget -q "https://gitlab.postmarketos.org/postmarketOS/ci-common/-/raw/master/check_mr_settings.py"
- wget -q "https://gitlab.com/postmarketOS/ci-common/-/raw/master/check_mr_settings.py"
- python3 ./check_mr_settings.py
# Calculate build of changed aports using a dynamic downstream pipelines:
# https://docs.gitlab.com/ci/pipelines/downstream_pipelines/#dynamic-child-pipelines
# For those to work, we need two jobs:
# * First one (this one) that generates a valid yaml file with the job yaml as
# an artifact
# * Second one (build-jobs) that has a "trigger" key with the yaml file from
# which to trigger pipelines
# To avoid reworking the whole CI file, for now we are re-using this same file
# to trigger the downstream pipeline. In that case, $CI_PIPELINE_SOURCE will
# be set to "parent_pipeline", and only the jobs that extend .build will run
generate-build-jobs:
stage: prepare
# build changed aports
.build:
stage: build
<<: *only-default
before_script:
- .ci/lib/gitlab_prepare_ci.sh
needs: []
script:
- wget "https://gitlab.postmarketos.org/postmarketOS/ci-common/-/raw/master/install_pmbootstrap.sh"
- sh ./install_pmbootstrap.sh py3-jinja2
# HACK: Make the pmb library happy to use the currently-cloned pmaports repo
- mkdir -p /root/.local/var/pmbootstrap/cache_git/ && ln -s $PWD /root/.local/var/pmbootstrap/cache_git/pmaports
- PYTHONPATH=/tmp/pmbootstrap/ .ci/lib/generate_build_jobs.py .ci/build-jobs.yaml.j2 .ci/build-jobs.yaml
artifacts:
paths:
- .ci/build-jobs.yaml
rules:
- if: *mr_rule
build-jobs:
stage: build
trigger:
include:
- artifact: .ci/build-jobs.yaml
job: generate-build-jobs
strategy: depend
forward:
# So that we don't have to duplicate GIT_DEPTH
pipeline_variables: true
rules:
- if: *mr_rule
build-docs:
stage: build
before_script:
- adduser -D build
script:
- .ci/build-docs.sh
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
changes:
- docs/**
deploy-docs:
stage: deploy
trigger:
project: postmarketOS/docs.postmarketos.org
strategy: depend
rules:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH && $CI_PROJECT_NAMESPACE == "postmarketOS"
changes:
- docs/**
- if: $CI_PIPELINE_SOURCE == "schedule"
when: never
auto-update:
stage: autoupdate
rules:
# This variable is set in the scheduled pipeline configuration. It should be a space separated list of
# package names to update.
- if: $AUTOUPDATE_PACKAGES != null && $CI_PIPELINE_SOURCE == "schedule" || $CI_PIPELINE_SOURCE == "web"
before_script:
- .ci/lib/gitlab_prepare_ci.sh
script:
- .ci/autoupdate.sh "$AUTOUPDATE_PACKAGES"
after_script:
- .ci/lib/move_logs.sh $CI_PROJECT_DIR
- cp -r /home/pmos/.local/var/pmbootstrap/packages/ packages/ || true
artifacts:
when: on_failure
expire_in: 1 week
paths:
- log.txt
- log_testsuite_pmaports.txt
- pmbootstrap.cfg
- packages/
timeout: 10 h
# Template for package test jobs defined per-package
.package-test:
stage: test
rules:
- if: *mr_rule
before_script:
- .ci/lib/gitlab_prepare_ci.sh
build-x86_64:
extends: .build
script:
- .ci/build-x86_64.sh
build-x86:
extends: .build
script:
- .ci/build-x86.sh
build-aarch64:
extends: .build
script:
- .ci/build-aarch64.sh
build-armv7:
extends: .build
script:
- .ci/build-armv7.sh
build-armhf:
extends: .build
script:
- .ci/build-armhf.sh
build-riscv64:
extends: .build
script:
- .ci/build-riscv64.sh
@@ -1,31 +0,0 @@
<!-- Use this template to kick off the discussion for changing one or more
related kernel config options.
-->
### Read the fine manual, please :)
- [ ] I have read https://postmarketos.org/kconfig-adjust-multiple
### What benefit does the change have?
### Which option(s) would you like to change?
<!-- Make sure to specify if you would like to enable/disable/set a value. -->
### Did you already test that this works as expected?
<!-- Not required, but recommended. If you did test it, please list which
device you have tested it with. -->
### For which kernel configs would you do the change?
<!-- If it makes sense, we encourage to adjust all kernels used by devices in
the main and community categories, so devices behave consistently. -->
- [ ] all in main and community
- [ ] certain SoC(s):
- [ ] other (please specify):
<!-- leave this, so it applies the correct label in gitlab -->
/label ~"change-kernel-config"
-66
View File
@@ -1,66 +0,0 @@
<!--
Thank you for using postmarketOS! In order to speed up process of resolving your issue,
we would like you to fill in the following form.
*** NOTE FOR CONFIDENTIAL ISSUES: ***
A lot of people can read confidential pmaports issues! If you want to
submit something that only the postmarketOS Core Team can see, write to
team@postmarketos.org
instead of opening an issue here!
-->
### Describe your issue
#### What's the expected behaviour?
#### What's the current behaviour?
#### How to reproduce your issue?
### What device are you using?
<!--
Leave name of your device in vendor-codename format.
If unsure, check name in the device package or leave full name of device
PinePhone: which revision exactly? See https://wiki.pine64.org/index.php/PinePhone#Hardware_revisions
-->
### On what postmarketOS version did you encounter the issue?
<!--
Replace space inside `[ ]` with `x` to fill the checkbox
-->
* [ ] edge (`master` branch)
* [ ] `v25.06`
* [ ] I confirm that the issue still is present after running `sudo apk upgrade -a`
### On what environment did you encounter the issue?
#### Environments
* [ ] COSMIC
* [ ] GNOME Shell on Mobile
* [ ] Phosh
* [ ] Plasma Mobile
* [ ] Sxmo (Wayland/Sway) **Please post the output of `sxmo_version.sh`**
* [ ] Other: **Please fill out**
### Which init system are you using?
* [ ] OpenRC
* [ ] systemd
### How did you get postmarketOS image?
* [ ] from https://images.postmarketos.org
* [ ] I built it using pmbootstrap
* [ ] It was preinstalled on my device
### What's the build date of the image? (in yyyy-mm-dd format)
<!--
Notes:
can be left empty if unsure
If you downloaded image from images.postmarketos.org, date will be in yyyymmdd format
-->
### Additional information
<!--
logs, etc. -- make use <details> for long listings
If your device displays an error message on boot or drops to the debug
shell, attaching it to your PC with a USB cable may show a removable USB
drive. You can drag and drop the .tar.gz file or otherwise upload it
here.
Refer to the README file on the device for more information.
-->
-3
View File
@@ -1,3 +0,0 @@
Casey Connolly <casey.connolly@linaro.org> <caleb.connolly@linaro.org>
Casey Connolly <kcxt@postmarketos.org> <caleb@postmarketos.org>
Achill Gilgenast <achill@achill.org> <fossdd@pwned.life>
+19
View File
@@ -0,0 +1,19 @@
{
"subject_format": {
"pass": [
"^[a-z]+/[a-z-0-9*{}]+: new aport(s|)( \\(MR \\d+\\)|)$",
"^[a-z]+/[a-z-0-9*{}]+: pkgrel bump( \\(MR \\d+\\)|)$",
"^[a-z-0-9*{}]+: new device \\([^\\)]+\\)( \\(MR \\d+\\)|)",
"^[a-z]+/[a-z-0-9*{}]+: upgrade to [0-9\\.a-z\\-_]+( \\(MR \\d+\\)|)$",
"^[a-z]+/[a-z-0-9*{}]+: remove( \\(MR \\d+\\)|)$",
"^[a-z]+/[a-z-0-9*{}]+: remove, upstreamed( \\(MR \\d+\\)|)$",
"^CI: [a-z-0-9*{}\\(\\)\\.,_ ][A-Za-z\\/\\-0-9*{}\\(\\)\\.,_ ]+( \\(MR \\d+\\)|)$",
"^mrhlpr.json: [a-z-0-9*{}\\(\\)\\.,_ ][A-Za-z\\/\\-0-9*{}\\(\\)\\.,_ ]+( \\(MR \\d+\\)|)$",
"^pmaports.cfg: [a-z-0-9*{}\\(\\)\\.,_ ][A-Za-z\\/\\-0-9*{}\\(\\)\\.,_ ]+( \\(MR \\d+\\)|)$"
],
"unknown": [
"^[a-z-0-9_.*{}\\/]+: [a-z-0-9*{}\\(\\)\\.,_ ][A-Za-z\\/\\-0-9*{}\\(\\)\\.,_ ]+( \\(MR \\d+\\)|)$"
]
}
}
-1
View File
@@ -1 +0,0 @@
https://postmarketos.org/funding.json
+68
View File
@@ -0,0 +1,68 @@
# https://docs.gitlab.com/ee/user/project/code_owners.html
device/*/*-htc-golfu/ @Newbyte
device/*/*-pine64-pinephone/ @craftyguy @milesalan @bshah @proycon @anjandev @StacyHarper
device/*/*-htc-m8qlul/ @Newbyte
device/*/*-purism-librem5/ @craftyguy @bshah
device/*/device-lg-judyln/ @Newbyte
device/*/device-samsung-m0/ @Newbyte
device/*/linux-postmarketos-exynos4/ @Newbyte
device/*/linux-postmarketos-msm8939/ @Newbyte
device/*/linux-postmarketos-stericsson/ @Newbyte
device/*/*-qcom-sdm845/ @sdm845-mainline
device/*/*-xiaomi-berrylium/ @joelselvaraj
device/*/device-oneplus-enchilada/ @calebccff
device/*/device-oneplus-fajita/ @calebccff
device/*/firmware-oneplus-sdm845/ @calebccff
device/*/device-shift-axolotl/ @amartinz @calebccff @dylanvanassche
device/*/*-lenovo-a6010/ @jenneron
device/*/*-lenovo-a6000/ @jenneron
device/*/device-microsoft-surface-rt/ @jenneron
device/*/device-google-fennel/ @jenneron
device/*/device-google-veyron-minnie/ @jenneron
device/*/device-google-burnet/ @jenneron
device/*/device-google-elm/ @jenneron
device/*/device-google-nyan-big/ @jenneron
device/*/device-google-peach-pi/ @jenneron
device/*/device-google-veyron-speedy/ @jenneron
device/*/linux-google-veyron/ @jenneron
device/*/u-boot-peach-pi/ @jenneron
device/*/device-google-cozmo/ @jenneron
device/*/u-boot-veyron-speedy/ @jenneron
device/*/device-google-juniper/ @jenneron
device/*/u-boot-peach-pit/ @jenneron
device/*/device-google-damu/ @jenneron
device/*/u-boot-nyan-4gb/ @jenneron
device/*/device-google-kenzo/ @jenneron
device/*/device-google-kodama/ @jenneron
device/*/u-boot-snow/ @jenneron
device/*/device-google-kevin/ @jenneron
device/*/linux-postmarketos-exynos5/ @jenneron
device/*/device-google-krane/ @jenneron
device/*/soc-mediatek-mt8183/ @jenneron
device/*/u-boot-veyron-minnie/ @jenneron
device/*/device-google-fennel14/ @jenneron
device/*/linux-google-nyan/ @jenneron
device/*/device-google-willow/ @jenneron
device/*/linux-postmarketos-mediatek-mt8173/ @jenneron
device/*/device-google-nyan-blaze/ @jenneron
device/*/firmware-powervr-gx6250/ @jenneron
device/*/device-google-hana/ @jenneron
device/*/device-google-snow/ @jenneron
device/*/device-google-kappa/ @jenneron
device/*/u-boot-nyan-2gb/ @jenneron
device/*/device-google-veyron-jerry/ @jenneron
device/*/device-google-kakadu/ @jenneron
device/*/firmware-google-veyron-brcm/ @jenneron
device/*/device-google-peach-pit/ @jenneron
device/*/device-lg-judyp/ @jenneron
main/postmarketos-update-depthcharge-kernel/ @jenneron
main/osk-sdl/ @craftyguy
main/postmarketos-ui-plasma-mobile/ @bshah
main/ttyescape @calebccff
temp/u-boot-librem5/ @craftyguy @bshah
temp/alsa-ucm-conf-google-kukui/ @jenneron
temp/alsa-ucm-conf-google-gru/ @jenneron
# vim: ts=8 sw=4 et
-102
View File
@@ -1,102 +0,0 @@
# Policy for commits in pmaports
This document defines policy for organising and titling commits for
inclusion in pmaports.
## Definitions
### Device-specific package
A device-specific package is used to apply configuration and
functionality specifically for device support in postmarketOS.
This is in contrast to other packages
(e.g. main/postmarketos-ui-phosh) in postmarketOS that are used
for configuring the distribution without consideration of the
target device.
## General policy
We follow Alpine's aports commit style with some minor exceptions
and additions. Alpine's commit style can be found here:
https://gitlab.alpinelinux.org/alpine/aports/-/blob/master/COMMITSTYLE.md
## Exceptions
### Changes to packages inside of the device directory
While Alpine has the policy of always prefixing package names with
the name of the repository they are in, in the case of pmaports
this can lead to excessively long commit titles due to the nested
subfolders of the device directory. As such, commits changing
packages inside of the device directory should omit the directory
prefix, for example:
device-pine64-pinephone: description of change
Or:
soc-qcom-msm8996: description of change
If the commit only makes device-specific changes, the
commit title can opt to use the following format:
manufacturer-codename: description of change
### Changes to packages inside of the systemd directory
We also shorten the prefixes for packages in `extra-repos/systemd`.
When the `systemd` package itself gets changed, use `systemd: ` as
prefix. For example:
systemd: add triggers for reloading/restarting services
When another package inside the systemd directory gets changed, use
`systemd/$pkgname: ` as prefix. For example:
systemd/cups: restart on upgrade
### Moving devices between categories
As we do not include directory prefixes in device-related commits,
Alpine's commit style for moving packages does not apply. Instead,
we do the following:
manufacturer-codename: move from category to category
For example:
samsung-m0: move from testing to community
## Additions
### Forking packages from Alpine
Run `pmbootstrap aportgen --fork-alpine package` and modify it.
#### Template
- temp/package-name: fork from Alpine
Example: temp/pulseaudio: fork from Alpine
#### Rules
One commit per forked package.
### Adding new devices
Adding a new device package and optionally also a device-specific
Linux and firmware package.
#### Template
- manufacturer-codename: new device
Example: htc-m8qlul: new device
#### Rules
If the device has the option of making use of a device-specific
kernel and/or firmware package, those packages should be included
in the same commit that adds the device package. One device per
commit.
+1 -16
View File
@@ -1,23 +1,8 @@
# postmarketOS aports repository
This repository contains the APKBUILD files for postmarketOS-specific packages, along with the required patches and scripts, if any.
There are many more packages defined in the [Alpine Linux aports](https://gitlab.alpinelinux.org/alpine/aports/) on which these packages depend.
# postmarketOS aports
Helpful resources:
* [How to create a package](https://wiki.postmarketos.org/wiki/Create_a_package)
* [APKBUILD reference](https://wiki.alpinelinux.org/wiki/APKBUILD_Reference)
* [pmaports commit style](./COMMITSTYLE.md)
* [Review and merging guidelines](./docs/modules/ROOT/pages/merging-rules.adoc)
* [Alpine Linux aports](https://gitlab.alpinelinux.org/alpine/aports/)
* [Alpine Linux package search](https://pkgs.alpinelinux.org/packages)
* [postmarketOS package search](https://pkgs.postmarketos.org/packages)
## Git Hooks
You can find some useful git hooks in the `.githooks` directory.
To use them, run the following command after cloning this repository:
```sh
git config --local core.hooksPath .githooks
```
+1 -43
View File
@@ -8,50 +8,8 @@ branch_pmaports=master
branch_aports=master
mirrordir_alpine=edge
[v25.12]
description=Upcoming stable release (DO NOT USE)
branch_pmaports=v25.12
branch_aports=master
mirrordir_alpine=v3.23
[v25.06]
description=Latest release / Recommended for best stability
branch_pmaports=v25.06
branch_aports=3.22-stable
mirrordir_alpine=v3.22
[v24.12]
description=Old release (unsupported)
branch_pmaports=v24.12
branch_aports=3.21-stable
mirrordir_alpine=v3.21
[v24.06]
description=Old release (unsupported)
branch_pmaports=v24.06
branch_aports=3.20-stable
mirrordir_alpine=v3.20
[v23.12]
description=Old release (unsupported)
branch_pmaports=v23.12
branch_aports=3.19-stable
mirrordir_alpine=v3.19
[v23.06]
description=Old release (unsupported)
branch_pmaports=v23.06
branch_aports=3.18-stable
mirrordir_alpine=v3.18
[v22.12]
description=Old release (unsupported)
branch_pmaports=v22.12
branch_aports=3.17-stable
mirrordir_alpine=v3.17
[v22.06]
description=Old release (unsupported)
description=Latest release / Recommended for best stability
branch_pmaports=v22.06
branch_aports=3.16-stable
mirrordir_alpine=v3.16
-21
View File
@@ -1,21 +0,0 @@
cross-gcc-tests:
extends: .package-test
tags: [qemu]
before_script:
- wget "https://gitlab.postmarketos.org/postmarketOS/ci-common/-/raw/master/install_pmbootstrap.sh"
- sh ./install_pmbootstrap.sh
script:
- su pmos -c "pmbootstrap build gcc-aarch64"
- su pmos -c "pmbootstrap build --arch=aarch64 hello-world --force"
- su pmos -c "pmbootstrap build --arch=aarch64 iskey --force"
- su pmos -c "pmbootstrap build --arch=aarch64 pbsplash --force"
- su pmos -c "pmbootstrap build --arch=aarch64 gcc-x86_64"
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
changes:
- cross/gcc-x86_64/APKBUILD
artifacts:
when: always
paths:
- log.txt
expire_in: 1 week
@@ -0,0 +1,55 @@
From bbf05b742fde518fb97c789b043fe0d3aaf549f0 Mon Sep 17 00:00:00 2001
From: Ariadne Conill <ariadne@dereferenced.org>
Date: Tue, 21 Sep 2021 14:53:13 +0000
Subject: [PATCH] Revert "PR25882, .gnu.attributes are not checked for shared
libraries"
This revert is needed to avoid wrongly tagging objects with the incompatible
IBM long double ABI, which is not supported by musl and will result in
linking errors if used.
This reverts commit a8acd6eeb6dc2cc5460ece90f90ebe36b56b20ba.
---
bfd/elf32-tic6x.c | 3 ---
ld/ldlang.c | 10 ++++------
2 files changed, 4 insertions(+), 9 deletions(-)
diff --git a/bfd/elf32-tic6x.c b/bfd/elf32-tic6x.c
index 5754f3cb860..3ad1d612749 100644
--- a/bfd/elf32-tic6x.c
+++ b/bfd/elf32-tic6x.c
@@ -3735,9 +3735,6 @@ elf32_tic6x_merge_attributes (bfd *ibfd, struct bfd_link_info *info)
case Tag_ABI_PIC:
case Tag_ABI_PID:
- /* Don't transfer these tags from dynamic objects. */
- if ((ibfd->flags & DYNAMIC) != 0)
- continue;
if (out_attr[i].i > in_attr[i].i)
out_attr[i].i = in_attr[i].i;
break;
diff --git a/ld/ldlang.c b/ld/ldlang.c
index 37b64c89ee1..f13beaef9d9 100644
--- a/ld/ldlang.c
+++ b/ld/ldlang.c
@@ -7071,13 +7071,11 @@ lang_check (void)
bfd_printable_name (input_bfd), input_bfd,
bfd_printable_name (link_info.output_bfd));
}
-
- /* If the input bfd has no contents, it shouldn't set the
- private data of the output bfd. */
- else if (!file->flags.just_syms
- && ((input_bfd->flags & DYNAMIC) != 0
- || bfd_count_sections (input_bfd) != 0))
+ else if (bfd_count_sections (input_bfd))
{
+ /* If the input bfd has no contents, it shouldn't set the
+ private data of the output bfd. */
+
bfd_error_handler_type pfn = NULL;
/* If we aren't supposed to warn about mismatched input
--
2.33.0
+155
View File
@@ -0,0 +1,155 @@
# Automatically generated aport, do not edit!
# Generator: pmbootstrap aportgen binutils-aarch64
# Based on: main/binutils
CTARGET_ARCH=aarch64
CTARGET="$(arch_to_hostspec $CTARGET_ARCH)"
pkgname=binutils-aarch64
pkgver=2.39
pkgrel=2
pkgdesc="Tools necessary to build programs for aarch64 targets"
url="https://www.gnu.org/software/binutils/"
makedepends_build="bison flex texinfo"
makedepends_host="zlib-dev"
makedepends="$makedepends_build $makedepends_host"
arch="x86_64"
license="GPL-2.0 GPL-3.0-or-later LGPL-2.0 BSD"
subpackages="binutils-dev binutils-doc"
source="https://ftp.gnu.org/gnu/binutils/binutils-$pkgver.tar.xz
binutils-ld-fix-static-linking.patch
gold-mips.patch
ld-bfd-mips.patch
0001-Revert-PR25882-.gnu.attributes-are-not-checked-for-s.patch
binutils-mips-disable-assert.patch
CVE-2022-38533.patch
"
builddir="$srcdir/binutils-$pkgver"
if [ "$CHOST" = "$CBUILD" ] && [ "$CBUILD" = "$CTARGET" ] && [ "$CTARGET_ARCH" != "riscv64" ]; then
subpackages="$subpackages binutils-gold"
fi
if [ "$CHOST" != "$CTARGET" ]; then
pkgname="binutils-$CTARGET_ARCH"
subpackages=""
sonameprefix="binutils:"
fi
# secfixes:
# 2.39-r2:
# - CVE-2022-38533
# 2.39-r0:
# - CVE-2022-38126
# 2.35.2-r1:
# - CVE-2021-3487
# 2.32-r0:
# - CVE-2018-19931
# - CVE-2018-19932
# - CVE-2018-20002
# - CVE-2018-20712
# 2.28-r1:
# - CVE-2017-7614
build() {
local _sysroot=/
local _cross_configure="--enable-install-libiberty --enable-shared"
local _arch_configure=""
local _gold_configure="--disable-gold"
local _plugin_configure="--enable-plugins"
if [ "$CHOST" != "$CTARGET" ]; then
_sysroot="$CBUILDROOT"
_cross_configure="--disable-install-libiberty"
_plugin_configure="--disable-plugins"
fi
if [ "$CHOST" = "$CBUILD" ] && [ "$CBUILD" = "$CTARGET" ] && [ "$CTARGET_ARCH" != "riscv64" ]; then
_gold_configure="--enable-gold"
fi
if [ "$CTARGET_ARCH" = "x86_64" ]; then
_arch_configure="--enable-targets=x86_64-pep"
fi
if [ "$CTARGET_ARCH" = "riscv64" ]; then
_gold_configure="--disable-gold"
fi
case "$CTARGET_ARCH" in
mips*) _hash_style_configure="--enable-default-hash-style=sysv" ;;
*) _hash_style_configure="--enable-default-hash-style=gnu" ;;
esac
./configure \
--build=$CBUILD \
--host=$CHOST \
--target=$CTARGET \
--with-build-sysroot="$CBUILDROOT" \
--with-sysroot=$_sysroot \
--prefix=/usr \
--mandir=/usr/share/man \
--infodir=/usr/share/info \
--disable-multilib \
--disable-gprofng \
--enable-ld=default \
$_gold_configure \
--enable-64-bit-bfd \
$_plugin_configure \
--enable-relro \
--enable-deterministic-archives \
--enable-default-execstack=no \
$_cross_configure \
$_arch_configure \
$_hash_style_configure \
--with-pic \
--disable-werror \
--disable-nls \
--with-mmap \
--with-system-zlib
make
}
package() {
make install DESTDIR="$pkgdir"
if [ -d "$pkgdir"/usr/lib64 ]; then
mv "$pkgdir"/usr/lib64/* "$pkgdir"/usr/lib/
rmdir "$pkgdir"/usr/lib64
fi
if [ "$CHOST" != "$CTARGET" ]; then
# creating cross tools: remove any files that would conflict
# with the native tools, or other cross tools
rm -r "${pkgdir:?}"/usr/share
rm -f "$pkgdir"/usr/lib/libiberty.a
rm -r "${pkgdir:?}"/usr/lib/bfd-plugins
fi
}
libs() {
pkgdesc="Runtime libraries from binutils - libbfd and libopcodes"
mkdir -p "$subpkgdir"/usr/lib
mv "$pkgdir"/usr/lib/lib*.so "$subpkgdir"/usr/lib/
}
gold() {
pkgdesc="GNU binutils - gold linker"
if [ -e "$pkgdir"/usr/bin/ld.gold ]; then
mkdir -p "$subpkgdir"/usr/bin
mv "$pkgdir"/usr/bin/ld.gold "$subpkgdir"/usr/bin
fi
mkdir -p "$subpkgdir"/usr/$CTARGET/bin
mv "$pkgdir"/usr/$CTARGET/bin/ld.gold "$subpkgdir"/usr/$CTARGET/bin/ld.gold
}
sha512sums="
68e038f339a8c21faa19a57bbc447a51c817f47c2e06d740847c6e9cc3396c025d35d5369fa8c3f8b70414757c89f0e577939ddc0d70f283182504920f53b0a3 binutils-2.39.tar.xz
ecee33b0e435aa704af1c334e560f201638ff79e199aa11ed78a72f7c9b46f85fbb227af5748e735fd681d1965fcc42ac81b0c8824e540430ce0c706c81e8b49 binutils-ld-fix-static-linking.patch
f55cf2e0bf82f97583a1abe10710e4013ecf7d64f1da2ef8659a44a06d0dd8beaf58dab98a183488ea137f03e32d62efc878d95f018f836f8cec870bc448556f gold-mips.patch
314d2ef9071c89940aa6c8118e8a1e2f191a5d0a4bf596da1ad9cc84f884d8bc7dea8bd7b9fc3f8f1bddd3fd41c6eb017e1e804044b3bf084df1ed9e6e095e2d ld-bfd-mips.patch
70ec22bd72ef6dddecfd970613387dd4a8cdc8730dd3cbf03d5a0c3a7c4d839383167bb06dad21bf7c235329fd44b5dc4aefe762f68544f17155cf002bf1be4a 0001-Revert-PR25882-.gnu.attributes-are-not-checked-for-s.patch
609cd90d8b334eb309f586b17b9d335a08d3dbb6def7c3eb5c010028fcb681674031e5b9d853aa7a39a50304356a86afc184b85562b3f228f8197f4d29395c8f binutils-mips-disable-assert.patch
631726e557524633ace39c597acbc5ac39bfd25f7b292eee1f0b2a80a503f4ff799f027757d12d5a24b2fa3c07c7e14602d8a86d96e19876d6d0d5d1a02817e7 CVE-2022-38533.patch
"
@@ -0,0 +1,22 @@
X-Git-Url: https://sourceware.org/git/?p=binutils-gdb.git;a=blobdiff_plain;f=bfd%2Fcoffcode.h;h=52027981c3f052d98e043cd0c877f0bb5da97c7f;hp=67aaf158ca1829fe63f0065f31a0b1009b227b36;hb=ef186fe54aa6d281a3ff8a9528417e5cc614c797;hpb=80075150727d653c39e9d037e15c6b87dcef08cd
diff --git a/bfd/coffcode.h b/bfd/coffcode.h
index 67aaf158ca1..52027981c3f 100644
--- a/bfd/coffcode.h
+++ b/bfd/coffcode.h
@@ -4302,10 +4302,13 @@ coff_set_section_contents (bfd * abfd,
rec = (bfd_byte *) location;
recend = rec + count;
- while (rec < recend)
+ while (recend - rec >= 4)
{
+ size_t len = bfd_get_32 (abfd, rec);
+ if (len == 0 || len > (size_t) (recend - rec) / 4)
+ break;
+ rec += len * 4;
++section->lma;
- rec += bfd_get_32 (abfd, rec) * 4;
}
BFD_ASSERT (rec == recend);
@@ -0,0 +1,46 @@
This fixes static linking for our hardened toolchain
diff --git a/ld/scripttempl/elf.sc b/ld/scripttempl/elf.sc
index e8126cb..9532bfb 100644
--- a/ld/scripttempl/elf.sc
+++ b/ld/scripttempl/elf.sc
@@ -235,8 +235,8 @@ test "${LARGE_SECTIONS}" = "yes" && LARGE_SECTIONS="
if test "${ENABLE_INITFINI_ARRAY}" = "yes"; then
SORT_INIT_ARRAY="KEEP (*(SORT_BY_INIT_PRIORITY(.init_array.*) SORT_BY_INIT_PRIORITY(.ctors.*)))"
SORT_FINI_ARRAY="KEEP (*(SORT_BY_INIT_PRIORITY(.fini_array.*) SORT_BY_INIT_PRIORITY(.dtors.*)))"
- CTORS_IN_INIT_ARRAY="EXCLUDE_FILE (*crtbegin.o *crtbegin?.o *crtend.o *crtend?.o $OTHER_EXCLUDE_FILES) .ctors"
- DTORS_IN_FINI_ARRAY="EXCLUDE_FILE (*crtbegin.o *crtbegin?.o *crtend.o *crtend?.o $OTHER_EXCLUDE_FILES) .dtors"
+ CTORS_IN_INIT_ARRAY="EXCLUDE_FILE (*crtbegin*.o *crtend*.o $OTHER_EXCLUDE_FILES) .ctors"
+ DTORS_IN_FINI_ARRAY="EXCLUDE_FILE (*crtbegin*.o *crtend*.o $OTHER_EXCLUDE_FILES) .dtors"
else
SORT_INIT_ARRAY="KEEP (*(SORT(.init_array.*)))"
SORT_FINI_ARRAY="KEEP (*(SORT(.fini_array.*)))"
@@ -270,15 +270,14 @@ CTOR=".ctors ${CONSTRUCTING-0} :
doesn't matter which directory crtbegin.o
is in. */
- KEEP (*crtbegin.o(.ctors))
- KEEP (*crtbegin?.o(.ctors))
+ KEEP (*crtbegin*.o(.ctors))
/* We don't want to include the .ctor section from
the crtend.o file until after the sorted ctors.
The .ctor section from the crtend file contains the
end of ctors marker and it must be last */
- KEEP (*(EXCLUDE_FILE (*crtend.o *crtend?.o $OTHER_EXCLUDE_FILES) .ctors))
+ KEEP (*(EXCLUDE_FILE (*crtend*.o $OTHER_EXCLUDE_FILES) .ctors))
KEEP (*(SORT(.ctors.*)))
KEEP (*(.ctors))
${CONSTRUCTING+${CTOR_END}}
@@ -286,9 +285,8 @@ CTOR=".ctors ${CONSTRUCTING-0} :
DTOR=".dtors ${CONSTRUCTING-0} :
{
${CONSTRUCTING+${DTOR_START}}
- KEEP (*crtbegin.o(.dtors))
- KEEP (*crtbegin?.o(.dtors))
- KEEP (*(EXCLUDE_FILE (*crtend.o *crtend?.o $OTHER_EXCLUDE_FILES) .dtors))
+ KEEP (*crtbegin*.o(.dtors))
+ KEEP (*(EXCLUDE_FILE (*crtend*.o $OTHER_EXCLUDE_FILES) .dtors))
KEEP (*(SORT(.dtors.*)))
KEEP (*(.dtors))
${CONSTRUCTING+${DTOR_END}}
@@ -0,0 +1,14 @@
diff -urN binutils-2.35.2.orig/bfd/elfxx-mips.c binutils-2.35.2/bfd/elfxx-mips.c
--- binutils-2.35.2.orig/bfd/elfxx-mips.c 2021-01-30 01:38:03.000000000 -0700
+++ binutils-2.35.2/bfd/elfxx-mips.c 2021-05-13 17:35:34.706913601 -0600
@@ -6744,8 +6744,10 @@
in the relocation. */
if (h != NULL && ! SYMBOL_REFERENCES_LOCAL (info, &h->root))
{
+#if 0
BFD_ASSERT (htab->root.target_os == is_vxworks
|| h->global_got_area != GGA_NONE);
+#endif
indx = h->root.dynindx;
if (SGI_COMPAT (output_bfd))
defined_p = h->root.def_regular;
+39
View File
@@ -0,0 +1,39 @@
# DP: Fix gold on mips64 targets.
gold/
2016-08-09 Aurelien Jarno <aurelien@aurel32.net>
* configure.tgt: Add mips64el*-*-*|mips64le*-*-* and mips64*-*-*.
--- a/gold/configure.tgt
+++ b/gold/configure.tgt
@@ -153,6 +153,13 @@ aarch64*-*)
targ_big_endian=false
targ_extra_big_endian=true
;;
+mips*64*el*-*-*|mips*64*le*-*-*)
+ targ_obj=mips
+ targ_machine=EM_MIPS_RS3_LE
+ targ_size=64
+ targ_big_endian=false
+ targ_extra_big_endian=true
+ ;;
mips*el*-*-*|mips*le*-*-*)
targ_obj=mips
targ_machine=EM_MIPS_RS3_LE
@@ -160,6 +167,13 @@ mips*el*-*-*|mips*le*-*-*)
targ_big_endian=false
targ_extra_big_endian=true
;;
+mips*64*-*-*)
+ targ_obj=mips
+ targ_machine=EM_MIPS
+ targ_size=64
+ targ_big_endian=true
+ targ_extra_big_endian=false
+ ;;
mips*-*-*)
targ_obj=mips
targ_machine=EM_MIPS
+19
View File
@@ -0,0 +1,19 @@
--- ./ld/configure.tgt.orig
+++ ./ld/configure.tgt
@@ -541,12 +541,12 @@
;;
mips*-*-windiss) targ_emul=elf32mipswindiss
;;
-mips64*el-*-linux-*) targ_emul=elf32ltsmipn32
- targ_extra_emuls="elf32btsmipn32 elf32ltsmip elf32btsmip elf64ltsmip elf64btsmip"
+mips64*el-*-linux-*) targ_emul=elf64ltsmip
+ targ_extra_emuls="elf32ltsmipn32 elf32btsmipn32 elf32ltsmip elf32btsmip elf64btsmip"
targ_extra_libpath=$targ_extra_emuls
;;
-mips64*-*-linux-*) targ_emul=elf32btsmipn32
- targ_extra_emuls="elf32ltsmipn32 elf32btsmip elf32ltsmip elf64btsmip elf64ltsmip"
+mips64*-*-linux-*) targ_emul=elf64btsmip
+ targ_extra_emuls="elf32btsmipn32 elf32ltsmipn32 elf32btsmip elf32ltsmip elf64ltsmip"
targ_extra_libpath=$targ_extra_emuls
;;
mips*el-*-linux-*) targ_emul=elf32ltsmip
@@ -0,0 +1,55 @@
From bbf05b742fde518fb97c789b043fe0d3aaf549f0 Mon Sep 17 00:00:00 2001
From: Ariadne Conill <ariadne@dereferenced.org>
Date: Tue, 21 Sep 2021 14:53:13 +0000
Subject: [PATCH] Revert "PR25882, .gnu.attributes are not checked for shared
libraries"
This revert is needed to avoid wrongly tagging objects with the incompatible
IBM long double ABI, which is not supported by musl and will result in
linking errors if used.
This reverts commit a8acd6eeb6dc2cc5460ece90f90ebe36b56b20ba.
---
bfd/elf32-tic6x.c | 3 ---
ld/ldlang.c | 10 ++++------
2 files changed, 4 insertions(+), 9 deletions(-)
diff --git a/bfd/elf32-tic6x.c b/bfd/elf32-tic6x.c
index 5754f3cb860..3ad1d612749 100644
--- a/bfd/elf32-tic6x.c
+++ b/bfd/elf32-tic6x.c
@@ -3735,9 +3735,6 @@ elf32_tic6x_merge_attributes (bfd *ibfd, struct bfd_link_info *info)
case Tag_ABI_PIC:
case Tag_ABI_PID:
- /* Don't transfer these tags from dynamic objects. */
- if ((ibfd->flags & DYNAMIC) != 0)
- continue;
if (out_attr[i].i > in_attr[i].i)
out_attr[i].i = in_attr[i].i;
break;
diff --git a/ld/ldlang.c b/ld/ldlang.c
index 37b64c89ee1..f13beaef9d9 100644
--- a/ld/ldlang.c
+++ b/ld/ldlang.c
@@ -7071,13 +7071,11 @@ lang_check (void)
bfd_printable_name (input_bfd), input_bfd,
bfd_printable_name (link_info.output_bfd));
}
-
- /* If the input bfd has no contents, it shouldn't set the
- private data of the output bfd. */
- else if (!file->flags.just_syms
- && ((input_bfd->flags & DYNAMIC) != 0
- || bfd_count_sections (input_bfd) != 0))
+ else if (bfd_count_sections (input_bfd))
{
+ /* If the input bfd has no contents, it shouldn't set the
+ private data of the output bfd. */
+
bfd_error_handler_type pfn = NULL;
/* If we aren't supposed to warn about mismatched input
--
2.33.0
+155
View File
@@ -0,0 +1,155 @@
# Automatically generated aport, do not edit!
# Generator: pmbootstrap aportgen binutils-armhf
# Based on: main/binutils
CTARGET_ARCH=armhf
CTARGET="$(arch_to_hostspec $CTARGET_ARCH)"
pkgname=binutils-armhf
pkgver=2.39
pkgrel=2
pkgdesc="Tools necessary to build programs for armhf targets"
url="https://www.gnu.org/software/binutils/"
makedepends_build="bison flex texinfo"
makedepends_host="zlib-dev"
makedepends="$makedepends_build $makedepends_host"
arch="x86_64"
license="GPL-2.0 GPL-3.0-or-later LGPL-2.0 BSD"
subpackages="binutils-dev binutils-doc"
source="https://ftp.gnu.org/gnu/binutils/binutils-$pkgver.tar.xz
binutils-ld-fix-static-linking.patch
gold-mips.patch
ld-bfd-mips.patch
0001-Revert-PR25882-.gnu.attributes-are-not-checked-for-s.patch
binutils-mips-disable-assert.patch
CVE-2022-38533.patch
"
builddir="$srcdir/binutils-$pkgver"
if [ "$CHOST" = "$CBUILD" ] && [ "$CBUILD" = "$CTARGET" ] && [ "$CTARGET_ARCH" != "riscv64" ]; then
subpackages="$subpackages binutils-gold"
fi
if [ "$CHOST" != "$CTARGET" ]; then
pkgname="binutils-$CTARGET_ARCH"
subpackages=""
sonameprefix="binutils:"
fi
# secfixes:
# 2.39-r2:
# - CVE-2022-38533
# 2.39-r0:
# - CVE-2022-38126
# 2.35.2-r1:
# - CVE-2021-3487
# 2.32-r0:
# - CVE-2018-19931
# - CVE-2018-19932
# - CVE-2018-20002
# - CVE-2018-20712
# 2.28-r1:
# - CVE-2017-7614
build() {
local _sysroot=/
local _cross_configure="--enable-install-libiberty --enable-shared"
local _arch_configure=""
local _gold_configure="--disable-gold"
local _plugin_configure="--enable-plugins"
if [ "$CHOST" != "$CTARGET" ]; then
_sysroot="$CBUILDROOT"
_cross_configure="--disable-install-libiberty"
_plugin_configure="--disable-plugins"
fi
if [ "$CHOST" = "$CBUILD" ] && [ "$CBUILD" = "$CTARGET" ] && [ "$CTARGET_ARCH" != "riscv64" ]; then
_gold_configure="--enable-gold"
fi
if [ "$CTARGET_ARCH" = "x86_64" ]; then
_arch_configure="--enable-targets=x86_64-pep"
fi
if [ "$CTARGET_ARCH" = "riscv64" ]; then
_gold_configure="--disable-gold"
fi
case "$CTARGET_ARCH" in
mips*) _hash_style_configure="--enable-default-hash-style=sysv" ;;
*) _hash_style_configure="--enable-default-hash-style=gnu" ;;
esac
./configure \
--build=$CBUILD \
--host=$CHOST \
--target=$CTARGET \
--with-build-sysroot="$CBUILDROOT" \
--with-sysroot=$_sysroot \
--prefix=/usr \
--mandir=/usr/share/man \
--infodir=/usr/share/info \
--disable-multilib \
--disable-gprofng \
--enable-ld=default \
$_gold_configure \
--enable-64-bit-bfd \
$_plugin_configure \
--enable-relro \
--enable-deterministic-archives \
--enable-default-execstack=no \
$_cross_configure \
$_arch_configure \
$_hash_style_configure \
--with-pic \
--disable-werror \
--disable-nls \
--with-mmap \
--with-system-zlib
make
}
package() {
make install DESTDIR="$pkgdir"
if [ -d "$pkgdir"/usr/lib64 ]; then
mv "$pkgdir"/usr/lib64/* "$pkgdir"/usr/lib/
rmdir "$pkgdir"/usr/lib64
fi
if [ "$CHOST" != "$CTARGET" ]; then
# creating cross tools: remove any files that would conflict
# with the native tools, or other cross tools
rm -r "${pkgdir:?}"/usr/share
rm -f "$pkgdir"/usr/lib/libiberty.a
rm -r "${pkgdir:?}"/usr/lib/bfd-plugins
fi
}
libs() {
pkgdesc="Runtime libraries from binutils - libbfd and libopcodes"
mkdir -p "$subpkgdir"/usr/lib
mv "$pkgdir"/usr/lib/lib*.so "$subpkgdir"/usr/lib/
}
gold() {
pkgdesc="GNU binutils - gold linker"
if [ -e "$pkgdir"/usr/bin/ld.gold ]; then
mkdir -p "$subpkgdir"/usr/bin
mv "$pkgdir"/usr/bin/ld.gold "$subpkgdir"/usr/bin
fi
mkdir -p "$subpkgdir"/usr/$CTARGET/bin
mv "$pkgdir"/usr/$CTARGET/bin/ld.gold "$subpkgdir"/usr/$CTARGET/bin/ld.gold
}
sha512sums="
68e038f339a8c21faa19a57bbc447a51c817f47c2e06d740847c6e9cc3396c025d35d5369fa8c3f8b70414757c89f0e577939ddc0d70f283182504920f53b0a3 binutils-2.39.tar.xz
ecee33b0e435aa704af1c334e560f201638ff79e199aa11ed78a72f7c9b46f85fbb227af5748e735fd681d1965fcc42ac81b0c8824e540430ce0c706c81e8b49 binutils-ld-fix-static-linking.patch
f55cf2e0bf82f97583a1abe10710e4013ecf7d64f1da2ef8659a44a06d0dd8beaf58dab98a183488ea137f03e32d62efc878d95f018f836f8cec870bc448556f gold-mips.patch
314d2ef9071c89940aa6c8118e8a1e2f191a5d0a4bf596da1ad9cc84f884d8bc7dea8bd7b9fc3f8f1bddd3fd41c6eb017e1e804044b3bf084df1ed9e6e095e2d ld-bfd-mips.patch
70ec22bd72ef6dddecfd970613387dd4a8cdc8730dd3cbf03d5a0c3a7c4d839383167bb06dad21bf7c235329fd44b5dc4aefe762f68544f17155cf002bf1be4a 0001-Revert-PR25882-.gnu.attributes-are-not-checked-for-s.patch
609cd90d8b334eb309f586b17b9d335a08d3dbb6def7c3eb5c010028fcb681674031e5b9d853aa7a39a50304356a86afc184b85562b3f228f8197f4d29395c8f binutils-mips-disable-assert.patch
631726e557524633ace39c597acbc5ac39bfd25f7b292eee1f0b2a80a503f4ff799f027757d12d5a24b2fa3c07c7e14602d8a86d96e19876d6d0d5d1a02817e7 CVE-2022-38533.patch
"
+22
View File
@@ -0,0 +1,22 @@
X-Git-Url: https://sourceware.org/git/?p=binutils-gdb.git;a=blobdiff_plain;f=bfd%2Fcoffcode.h;h=52027981c3f052d98e043cd0c877f0bb5da97c7f;hp=67aaf158ca1829fe63f0065f31a0b1009b227b36;hb=ef186fe54aa6d281a3ff8a9528417e5cc614c797;hpb=80075150727d653c39e9d037e15c6b87dcef08cd
diff --git a/bfd/coffcode.h b/bfd/coffcode.h
index 67aaf158ca1..52027981c3f 100644
--- a/bfd/coffcode.h
+++ b/bfd/coffcode.h
@@ -4302,10 +4302,13 @@ coff_set_section_contents (bfd * abfd,
rec = (bfd_byte *) location;
recend = rec + count;
- while (rec < recend)
+ while (recend - rec >= 4)
{
+ size_t len = bfd_get_32 (abfd, rec);
+ if (len == 0 || len > (size_t) (recend - rec) / 4)
+ break;
+ rec += len * 4;
++section->lma;
- rec += bfd_get_32 (abfd, rec) * 4;
}
BFD_ASSERT (rec == recend);
@@ -0,0 +1,46 @@
This fixes static linking for our hardened toolchain
diff --git a/ld/scripttempl/elf.sc b/ld/scripttempl/elf.sc
index e8126cb..9532bfb 100644
--- a/ld/scripttempl/elf.sc
+++ b/ld/scripttempl/elf.sc
@@ -235,8 +235,8 @@ test "${LARGE_SECTIONS}" = "yes" && LARGE_SECTIONS="
if test "${ENABLE_INITFINI_ARRAY}" = "yes"; then
SORT_INIT_ARRAY="KEEP (*(SORT_BY_INIT_PRIORITY(.init_array.*) SORT_BY_INIT_PRIORITY(.ctors.*)))"
SORT_FINI_ARRAY="KEEP (*(SORT_BY_INIT_PRIORITY(.fini_array.*) SORT_BY_INIT_PRIORITY(.dtors.*)))"
- CTORS_IN_INIT_ARRAY="EXCLUDE_FILE (*crtbegin.o *crtbegin?.o *crtend.o *crtend?.o $OTHER_EXCLUDE_FILES) .ctors"
- DTORS_IN_FINI_ARRAY="EXCLUDE_FILE (*crtbegin.o *crtbegin?.o *crtend.o *crtend?.o $OTHER_EXCLUDE_FILES) .dtors"
+ CTORS_IN_INIT_ARRAY="EXCLUDE_FILE (*crtbegin*.o *crtend*.o $OTHER_EXCLUDE_FILES) .ctors"
+ DTORS_IN_FINI_ARRAY="EXCLUDE_FILE (*crtbegin*.o *crtend*.o $OTHER_EXCLUDE_FILES) .dtors"
else
SORT_INIT_ARRAY="KEEP (*(SORT(.init_array.*)))"
SORT_FINI_ARRAY="KEEP (*(SORT(.fini_array.*)))"
@@ -270,15 +270,14 @@ CTOR=".ctors ${CONSTRUCTING-0} :
doesn't matter which directory crtbegin.o
is in. */
- KEEP (*crtbegin.o(.ctors))
- KEEP (*crtbegin?.o(.ctors))
+ KEEP (*crtbegin*.o(.ctors))
/* We don't want to include the .ctor section from
the crtend.o file until after the sorted ctors.
The .ctor section from the crtend file contains the
end of ctors marker and it must be last */
- KEEP (*(EXCLUDE_FILE (*crtend.o *crtend?.o $OTHER_EXCLUDE_FILES) .ctors))
+ KEEP (*(EXCLUDE_FILE (*crtend*.o $OTHER_EXCLUDE_FILES) .ctors))
KEEP (*(SORT(.ctors.*)))
KEEP (*(.ctors))
${CONSTRUCTING+${CTOR_END}}
@@ -286,9 +285,8 @@ CTOR=".ctors ${CONSTRUCTING-0} :
DTOR=".dtors ${CONSTRUCTING-0} :
{
${CONSTRUCTING+${DTOR_START}}
- KEEP (*crtbegin.o(.dtors))
- KEEP (*crtbegin?.o(.dtors))
- KEEP (*(EXCLUDE_FILE (*crtend.o *crtend?.o $OTHER_EXCLUDE_FILES) .dtors))
+ KEEP (*crtbegin*.o(.dtors))
+ KEEP (*(EXCLUDE_FILE (*crtend*.o $OTHER_EXCLUDE_FILES) .dtors))
KEEP (*(SORT(.dtors.*)))
KEEP (*(.dtors))
${CONSTRUCTING+${DTOR_END}}
@@ -0,0 +1,14 @@
diff -urN binutils-2.35.2.orig/bfd/elfxx-mips.c binutils-2.35.2/bfd/elfxx-mips.c
--- binutils-2.35.2.orig/bfd/elfxx-mips.c 2021-01-30 01:38:03.000000000 -0700
+++ binutils-2.35.2/bfd/elfxx-mips.c 2021-05-13 17:35:34.706913601 -0600
@@ -6744,8 +6744,10 @@
in the relocation. */
if (h != NULL && ! SYMBOL_REFERENCES_LOCAL (info, &h->root))
{
+#if 0
BFD_ASSERT (htab->root.target_os == is_vxworks
|| h->global_got_area != GGA_NONE);
+#endif
indx = h->root.dynindx;
if (SGI_COMPAT (output_bfd))
defined_p = h->root.def_regular;
+39
View File
@@ -0,0 +1,39 @@
# DP: Fix gold on mips64 targets.
gold/
2016-08-09 Aurelien Jarno <aurelien@aurel32.net>
* configure.tgt: Add mips64el*-*-*|mips64le*-*-* and mips64*-*-*.
--- a/gold/configure.tgt
+++ b/gold/configure.tgt
@@ -153,6 +153,13 @@ aarch64*-*)
targ_big_endian=false
targ_extra_big_endian=true
;;
+mips*64*el*-*-*|mips*64*le*-*-*)
+ targ_obj=mips
+ targ_machine=EM_MIPS_RS3_LE
+ targ_size=64
+ targ_big_endian=false
+ targ_extra_big_endian=true
+ ;;
mips*el*-*-*|mips*le*-*-*)
targ_obj=mips
targ_machine=EM_MIPS_RS3_LE
@@ -160,6 +167,13 @@ mips*el*-*-*|mips*le*-*-*)
targ_big_endian=false
targ_extra_big_endian=true
;;
+mips*64*-*-*)
+ targ_obj=mips
+ targ_machine=EM_MIPS
+ targ_size=64
+ targ_big_endian=true
+ targ_extra_big_endian=false
+ ;;
mips*-*-*)
targ_obj=mips
targ_machine=EM_MIPS
+19
View File
@@ -0,0 +1,19 @@
--- ./ld/configure.tgt.orig
+++ ./ld/configure.tgt
@@ -541,12 +541,12 @@
;;
mips*-*-windiss) targ_emul=elf32mipswindiss
;;
-mips64*el-*-linux-*) targ_emul=elf32ltsmipn32
- targ_extra_emuls="elf32btsmipn32 elf32ltsmip elf32btsmip elf64ltsmip elf64btsmip"
+mips64*el-*-linux-*) targ_emul=elf64ltsmip
+ targ_extra_emuls="elf32ltsmipn32 elf32btsmipn32 elf32ltsmip elf32btsmip elf64btsmip"
targ_extra_libpath=$targ_extra_emuls
;;
-mips64*-*-linux-*) targ_emul=elf32btsmipn32
- targ_extra_emuls="elf32ltsmipn32 elf32btsmip elf32ltsmip elf64btsmip elf64ltsmip"
+mips64*-*-linux-*) targ_emul=elf64btsmip
+ targ_extra_emuls="elf32btsmipn32 elf32ltsmipn32 elf32btsmip elf32ltsmip elf64ltsmip"
targ_extra_libpath=$targ_extra_emuls
;;
mips*el-*-linux-*) targ_emul=elf32ltsmip
@@ -0,0 +1,55 @@
From bbf05b742fde518fb97c789b043fe0d3aaf549f0 Mon Sep 17 00:00:00 2001
From: Ariadne Conill <ariadne@dereferenced.org>
Date: Tue, 21 Sep 2021 14:53:13 +0000
Subject: [PATCH] Revert "PR25882, .gnu.attributes are not checked for shared
libraries"
This revert is needed to avoid wrongly tagging objects with the incompatible
IBM long double ABI, which is not supported by musl and will result in
linking errors if used.
This reverts commit a8acd6eeb6dc2cc5460ece90f90ebe36b56b20ba.
---
bfd/elf32-tic6x.c | 3 ---
ld/ldlang.c | 10 ++++------
2 files changed, 4 insertions(+), 9 deletions(-)
diff --git a/bfd/elf32-tic6x.c b/bfd/elf32-tic6x.c
index 5754f3cb860..3ad1d612749 100644
--- a/bfd/elf32-tic6x.c
+++ b/bfd/elf32-tic6x.c
@@ -3735,9 +3735,6 @@ elf32_tic6x_merge_attributes (bfd *ibfd, struct bfd_link_info *info)
case Tag_ABI_PIC:
case Tag_ABI_PID:
- /* Don't transfer these tags from dynamic objects. */
- if ((ibfd->flags & DYNAMIC) != 0)
- continue;
if (out_attr[i].i > in_attr[i].i)
out_attr[i].i = in_attr[i].i;
break;
diff --git a/ld/ldlang.c b/ld/ldlang.c
index 37b64c89ee1..f13beaef9d9 100644
--- a/ld/ldlang.c
+++ b/ld/ldlang.c
@@ -7071,13 +7071,11 @@ lang_check (void)
bfd_printable_name (input_bfd), input_bfd,
bfd_printable_name (link_info.output_bfd));
}
-
- /* If the input bfd has no contents, it shouldn't set the
- private data of the output bfd. */
- else if (!file->flags.just_syms
- && ((input_bfd->flags & DYNAMIC) != 0
- || bfd_count_sections (input_bfd) != 0))
+ else if (bfd_count_sections (input_bfd))
{
+ /* If the input bfd has no contents, it shouldn't set the
+ private data of the output bfd. */
+
bfd_error_handler_type pfn = NULL;
/* If we aren't supposed to warn about mismatched input
--
2.33.0
+155
View File
@@ -0,0 +1,155 @@
# Automatically generated aport, do not edit!
# Generator: pmbootstrap aportgen binutils-armv7
# Based on: main/binutils
CTARGET_ARCH=armv7
CTARGET="$(arch_to_hostspec $CTARGET_ARCH)"
pkgname=binutils-armv7
pkgver=2.39
pkgrel=2
pkgdesc="Tools necessary to build programs for armv7 targets"
url="https://www.gnu.org/software/binutils/"
makedepends_build="bison flex texinfo"
makedepends_host="zlib-dev"
makedepends="$makedepends_build $makedepends_host"
arch="x86_64"
license="GPL-2.0 GPL-3.0-or-later LGPL-2.0 BSD"
subpackages="binutils-dev binutils-doc"
source="https://ftp.gnu.org/gnu/binutils/binutils-$pkgver.tar.xz
binutils-ld-fix-static-linking.patch
gold-mips.patch
ld-bfd-mips.patch
0001-Revert-PR25882-.gnu.attributes-are-not-checked-for-s.patch
binutils-mips-disable-assert.patch
CVE-2022-38533.patch
"
builddir="$srcdir/binutils-$pkgver"
if [ "$CHOST" = "$CBUILD" ] && [ "$CBUILD" = "$CTARGET" ] && [ "$CTARGET_ARCH" != "riscv64" ]; then
subpackages="$subpackages binutils-gold"
fi
if [ "$CHOST" != "$CTARGET" ]; then
pkgname="binutils-$CTARGET_ARCH"
subpackages=""
sonameprefix="binutils:"
fi
# secfixes:
# 2.39-r2:
# - CVE-2022-38533
# 2.39-r0:
# - CVE-2022-38126
# 2.35.2-r1:
# - CVE-2021-3487
# 2.32-r0:
# - CVE-2018-19931
# - CVE-2018-19932
# - CVE-2018-20002
# - CVE-2018-20712
# 2.28-r1:
# - CVE-2017-7614
build() {
local _sysroot=/
local _cross_configure="--enable-install-libiberty --enable-shared"
local _arch_configure=""
local _gold_configure="--disable-gold"
local _plugin_configure="--enable-plugins"
if [ "$CHOST" != "$CTARGET" ]; then
_sysroot="$CBUILDROOT"
_cross_configure="--disable-install-libiberty"
_plugin_configure="--disable-plugins"
fi
if [ "$CHOST" = "$CBUILD" ] && [ "$CBUILD" = "$CTARGET" ] && [ "$CTARGET_ARCH" != "riscv64" ]; then
_gold_configure="--enable-gold"
fi
if [ "$CTARGET_ARCH" = "x86_64" ]; then
_arch_configure="--enable-targets=x86_64-pep"
fi
if [ "$CTARGET_ARCH" = "riscv64" ]; then
_gold_configure="--disable-gold"
fi
case "$CTARGET_ARCH" in
mips*) _hash_style_configure="--enable-default-hash-style=sysv" ;;
*) _hash_style_configure="--enable-default-hash-style=gnu" ;;
esac
./configure \
--build=$CBUILD \
--host=$CHOST \
--target=$CTARGET \
--with-build-sysroot="$CBUILDROOT" \
--with-sysroot=$_sysroot \
--prefix=/usr \
--mandir=/usr/share/man \
--infodir=/usr/share/info \
--disable-multilib \
--disable-gprofng \
--enable-ld=default \
$_gold_configure \
--enable-64-bit-bfd \
$_plugin_configure \
--enable-relro \
--enable-deterministic-archives \
--enable-default-execstack=no \
$_cross_configure \
$_arch_configure \
$_hash_style_configure \
--with-pic \
--disable-werror \
--disable-nls \
--with-mmap \
--with-system-zlib
make
}
package() {
make install DESTDIR="$pkgdir"
if [ -d "$pkgdir"/usr/lib64 ]; then
mv "$pkgdir"/usr/lib64/* "$pkgdir"/usr/lib/
rmdir "$pkgdir"/usr/lib64
fi
if [ "$CHOST" != "$CTARGET" ]; then
# creating cross tools: remove any files that would conflict
# with the native tools, or other cross tools
rm -r "${pkgdir:?}"/usr/share
rm -f "$pkgdir"/usr/lib/libiberty.a
rm -r "${pkgdir:?}"/usr/lib/bfd-plugins
fi
}
libs() {
pkgdesc="Runtime libraries from binutils - libbfd and libopcodes"
mkdir -p "$subpkgdir"/usr/lib
mv "$pkgdir"/usr/lib/lib*.so "$subpkgdir"/usr/lib/
}
gold() {
pkgdesc="GNU binutils - gold linker"
if [ -e "$pkgdir"/usr/bin/ld.gold ]; then
mkdir -p "$subpkgdir"/usr/bin
mv "$pkgdir"/usr/bin/ld.gold "$subpkgdir"/usr/bin
fi
mkdir -p "$subpkgdir"/usr/$CTARGET/bin
mv "$pkgdir"/usr/$CTARGET/bin/ld.gold "$subpkgdir"/usr/$CTARGET/bin/ld.gold
}
sha512sums="
68e038f339a8c21faa19a57bbc447a51c817f47c2e06d740847c6e9cc3396c025d35d5369fa8c3f8b70414757c89f0e577939ddc0d70f283182504920f53b0a3 binutils-2.39.tar.xz
ecee33b0e435aa704af1c334e560f201638ff79e199aa11ed78a72f7c9b46f85fbb227af5748e735fd681d1965fcc42ac81b0c8824e540430ce0c706c81e8b49 binutils-ld-fix-static-linking.patch
f55cf2e0bf82f97583a1abe10710e4013ecf7d64f1da2ef8659a44a06d0dd8beaf58dab98a183488ea137f03e32d62efc878d95f018f836f8cec870bc448556f gold-mips.patch
314d2ef9071c89940aa6c8118e8a1e2f191a5d0a4bf596da1ad9cc84f884d8bc7dea8bd7b9fc3f8f1bddd3fd41c6eb017e1e804044b3bf084df1ed9e6e095e2d ld-bfd-mips.patch
70ec22bd72ef6dddecfd970613387dd4a8cdc8730dd3cbf03d5a0c3a7c4d839383167bb06dad21bf7c235329fd44b5dc4aefe762f68544f17155cf002bf1be4a 0001-Revert-PR25882-.gnu.attributes-are-not-checked-for-s.patch
609cd90d8b334eb309f586b17b9d335a08d3dbb6def7c3eb5c010028fcb681674031e5b9d853aa7a39a50304356a86afc184b85562b3f228f8197f4d29395c8f binutils-mips-disable-assert.patch
631726e557524633ace39c597acbc5ac39bfd25f7b292eee1f0b2a80a503f4ff799f027757d12d5a24b2fa3c07c7e14602d8a86d96e19876d6d0d5d1a02817e7 CVE-2022-38533.patch
"
+22
View File
@@ -0,0 +1,22 @@
X-Git-Url: https://sourceware.org/git/?p=binutils-gdb.git;a=blobdiff_plain;f=bfd%2Fcoffcode.h;h=52027981c3f052d98e043cd0c877f0bb5da97c7f;hp=67aaf158ca1829fe63f0065f31a0b1009b227b36;hb=ef186fe54aa6d281a3ff8a9528417e5cc614c797;hpb=80075150727d653c39e9d037e15c6b87dcef08cd
diff --git a/bfd/coffcode.h b/bfd/coffcode.h
index 67aaf158ca1..52027981c3f 100644
--- a/bfd/coffcode.h
+++ b/bfd/coffcode.h
@@ -4302,10 +4302,13 @@ coff_set_section_contents (bfd * abfd,
rec = (bfd_byte *) location;
recend = rec + count;
- while (rec < recend)
+ while (recend - rec >= 4)
{
+ size_t len = bfd_get_32 (abfd, rec);
+ if (len == 0 || len > (size_t) (recend - rec) / 4)
+ break;
+ rec += len * 4;
++section->lma;
- rec += bfd_get_32 (abfd, rec) * 4;
}
BFD_ASSERT (rec == recend);
@@ -0,0 +1,46 @@
This fixes static linking for our hardened toolchain
diff --git a/ld/scripttempl/elf.sc b/ld/scripttempl/elf.sc
index e8126cb..9532bfb 100644
--- a/ld/scripttempl/elf.sc
+++ b/ld/scripttempl/elf.sc
@@ -235,8 +235,8 @@ test "${LARGE_SECTIONS}" = "yes" && LARGE_SECTIONS="
if test "${ENABLE_INITFINI_ARRAY}" = "yes"; then
SORT_INIT_ARRAY="KEEP (*(SORT_BY_INIT_PRIORITY(.init_array.*) SORT_BY_INIT_PRIORITY(.ctors.*)))"
SORT_FINI_ARRAY="KEEP (*(SORT_BY_INIT_PRIORITY(.fini_array.*) SORT_BY_INIT_PRIORITY(.dtors.*)))"
- CTORS_IN_INIT_ARRAY="EXCLUDE_FILE (*crtbegin.o *crtbegin?.o *crtend.o *crtend?.o $OTHER_EXCLUDE_FILES) .ctors"
- DTORS_IN_FINI_ARRAY="EXCLUDE_FILE (*crtbegin.o *crtbegin?.o *crtend.o *crtend?.o $OTHER_EXCLUDE_FILES) .dtors"
+ CTORS_IN_INIT_ARRAY="EXCLUDE_FILE (*crtbegin*.o *crtend*.o $OTHER_EXCLUDE_FILES) .ctors"
+ DTORS_IN_FINI_ARRAY="EXCLUDE_FILE (*crtbegin*.o *crtend*.o $OTHER_EXCLUDE_FILES) .dtors"
else
SORT_INIT_ARRAY="KEEP (*(SORT(.init_array.*)))"
SORT_FINI_ARRAY="KEEP (*(SORT(.fini_array.*)))"
@@ -270,15 +270,14 @@ CTOR=".ctors ${CONSTRUCTING-0} :
doesn't matter which directory crtbegin.o
is in. */
- KEEP (*crtbegin.o(.ctors))
- KEEP (*crtbegin?.o(.ctors))
+ KEEP (*crtbegin*.o(.ctors))
/* We don't want to include the .ctor section from
the crtend.o file until after the sorted ctors.
The .ctor section from the crtend file contains the
end of ctors marker and it must be last */
- KEEP (*(EXCLUDE_FILE (*crtend.o *crtend?.o $OTHER_EXCLUDE_FILES) .ctors))
+ KEEP (*(EXCLUDE_FILE (*crtend*.o $OTHER_EXCLUDE_FILES) .ctors))
KEEP (*(SORT(.ctors.*)))
KEEP (*(.ctors))
${CONSTRUCTING+${CTOR_END}}
@@ -286,9 +285,8 @@ CTOR=".ctors ${CONSTRUCTING-0} :
DTOR=".dtors ${CONSTRUCTING-0} :
{
${CONSTRUCTING+${DTOR_START}}
- KEEP (*crtbegin.o(.dtors))
- KEEP (*crtbegin?.o(.dtors))
- KEEP (*(EXCLUDE_FILE (*crtend.o *crtend?.o $OTHER_EXCLUDE_FILES) .dtors))
+ KEEP (*crtbegin*.o(.dtors))
+ KEEP (*(EXCLUDE_FILE (*crtend*.o $OTHER_EXCLUDE_FILES) .dtors))
KEEP (*(SORT(.dtors.*)))
KEEP (*(.dtors))
${CONSTRUCTING+${DTOR_END}}
@@ -0,0 +1,14 @@
diff -urN binutils-2.35.2.orig/bfd/elfxx-mips.c binutils-2.35.2/bfd/elfxx-mips.c
--- binutils-2.35.2.orig/bfd/elfxx-mips.c 2021-01-30 01:38:03.000000000 -0700
+++ binutils-2.35.2/bfd/elfxx-mips.c 2021-05-13 17:35:34.706913601 -0600
@@ -6744,8 +6744,10 @@
in the relocation. */
if (h != NULL && ! SYMBOL_REFERENCES_LOCAL (info, &h->root))
{
+#if 0
BFD_ASSERT (htab->root.target_os == is_vxworks
|| h->global_got_area != GGA_NONE);
+#endif
indx = h->root.dynindx;
if (SGI_COMPAT (output_bfd))
defined_p = h->root.def_regular;
+39
View File
@@ -0,0 +1,39 @@
# DP: Fix gold on mips64 targets.
gold/
2016-08-09 Aurelien Jarno <aurelien@aurel32.net>
* configure.tgt: Add mips64el*-*-*|mips64le*-*-* and mips64*-*-*.
--- a/gold/configure.tgt
+++ b/gold/configure.tgt
@@ -153,6 +153,13 @@ aarch64*-*)
targ_big_endian=false
targ_extra_big_endian=true
;;
+mips*64*el*-*-*|mips*64*le*-*-*)
+ targ_obj=mips
+ targ_machine=EM_MIPS_RS3_LE
+ targ_size=64
+ targ_big_endian=false
+ targ_extra_big_endian=true
+ ;;
mips*el*-*-*|mips*le*-*-*)
targ_obj=mips
targ_machine=EM_MIPS_RS3_LE
@@ -160,6 +167,13 @@ mips*el*-*-*|mips*le*-*-*)
targ_big_endian=false
targ_extra_big_endian=true
;;
+mips*64*-*-*)
+ targ_obj=mips
+ targ_machine=EM_MIPS
+ targ_size=64
+ targ_big_endian=true
+ targ_extra_big_endian=false
+ ;;
mips*-*-*)
targ_obj=mips
targ_machine=EM_MIPS
+19
View File
@@ -0,0 +1,19 @@
--- ./ld/configure.tgt.orig
+++ ./ld/configure.tgt
@@ -541,12 +541,12 @@
;;
mips*-*-windiss) targ_emul=elf32mipswindiss
;;
-mips64*el-*-linux-*) targ_emul=elf32ltsmipn32
- targ_extra_emuls="elf32btsmipn32 elf32ltsmip elf32btsmip elf64ltsmip elf64btsmip"
+mips64*el-*-linux-*) targ_emul=elf64ltsmip
+ targ_extra_emuls="elf32ltsmipn32 elf32btsmipn32 elf32ltsmip elf32btsmip elf64btsmip"
targ_extra_libpath=$targ_extra_emuls
;;
-mips64*-*-linux-*) targ_emul=elf32btsmipn32
- targ_extra_emuls="elf32ltsmipn32 elf32btsmip elf32ltsmip elf64btsmip elf64ltsmip"
+mips64*-*-linux-*) targ_emul=elf64btsmip
+ targ_extra_emuls="elf32btsmipn32 elf32ltsmipn32 elf32btsmip elf32ltsmip elf64ltsmip"
targ_extra_libpath=$targ_extra_emuls
;;
mips*el-*-linux-*) targ_emul=elf32ltsmip
@@ -0,0 +1,55 @@
From bbf05b742fde518fb97c789b043fe0d3aaf549f0 Mon Sep 17 00:00:00 2001
From: Ariadne Conill <ariadne@dereferenced.org>
Date: Tue, 21 Sep 2021 14:53:13 +0000
Subject: [PATCH] Revert "PR25882, .gnu.attributes are not checked for shared
libraries"
This revert is needed to avoid wrongly tagging objects with the incompatible
IBM long double ABI, which is not supported by musl and will result in
linking errors if used.
This reverts commit a8acd6eeb6dc2cc5460ece90f90ebe36b56b20ba.
---
bfd/elf32-tic6x.c | 3 ---
ld/ldlang.c | 10 ++++------
2 files changed, 4 insertions(+), 9 deletions(-)
diff --git a/bfd/elf32-tic6x.c b/bfd/elf32-tic6x.c
index 5754f3cb860..3ad1d612749 100644
--- a/bfd/elf32-tic6x.c
+++ b/bfd/elf32-tic6x.c
@@ -3735,9 +3735,6 @@ elf32_tic6x_merge_attributes (bfd *ibfd, struct bfd_link_info *info)
case Tag_ABI_PIC:
case Tag_ABI_PID:
- /* Don't transfer these tags from dynamic objects. */
- if ((ibfd->flags & DYNAMIC) != 0)
- continue;
if (out_attr[i].i > in_attr[i].i)
out_attr[i].i = in_attr[i].i;
break;
diff --git a/ld/ldlang.c b/ld/ldlang.c
index 37b64c89ee1..f13beaef9d9 100644
--- a/ld/ldlang.c
+++ b/ld/ldlang.c
@@ -7071,13 +7071,11 @@ lang_check (void)
bfd_printable_name (input_bfd), input_bfd,
bfd_printable_name (link_info.output_bfd));
}
-
- /* If the input bfd has no contents, it shouldn't set the
- private data of the output bfd. */
- else if (!file->flags.just_syms
- && ((input_bfd->flags & DYNAMIC) != 0
- || bfd_count_sections (input_bfd) != 0))
+ else if (bfd_count_sections (input_bfd))
{
+ /* If the input bfd has no contents, it shouldn't set the
+ private data of the output bfd. */
+
bfd_error_handler_type pfn = NULL;
/* If we aren't supposed to warn about mismatched input
--
2.33.0
+155
View File
@@ -0,0 +1,155 @@
# Automatically generated aport, do not edit!
# Generator: pmbootstrap aportgen binutils-riscv64
# Based on: main/binutils
CTARGET_ARCH=riscv64
CTARGET="$(arch_to_hostspec $CTARGET_ARCH)"
pkgname=binutils-riscv64
pkgver=2.39
pkgrel=2
pkgdesc="Tools necessary to build programs for riscv64 targets"
url="https://www.gnu.org/software/binutils/"
makedepends_build="bison flex texinfo"
makedepends_host="zlib-dev"
makedepends="$makedepends_build $makedepends_host"
arch="x86_64"
license="GPL-2.0 GPL-3.0-or-later LGPL-2.0 BSD"
subpackages="binutils-dev binutils-doc"
source="https://ftp.gnu.org/gnu/binutils/binutils-$pkgver.tar.xz
binutils-ld-fix-static-linking.patch
gold-mips.patch
ld-bfd-mips.patch
0001-Revert-PR25882-.gnu.attributes-are-not-checked-for-s.patch
binutils-mips-disable-assert.patch
CVE-2022-38533.patch
"
builddir="$srcdir/binutils-$pkgver"
if [ "$CHOST" = "$CBUILD" ] && [ "$CBUILD" = "$CTARGET" ] && [ "$CTARGET_ARCH" != "riscv64" ]; then
subpackages="$subpackages binutils-gold"
fi
if [ "$CHOST" != "$CTARGET" ]; then
pkgname="binutils-$CTARGET_ARCH"
subpackages=""
sonameprefix="binutils:"
fi
# secfixes:
# 2.39-r2:
# - CVE-2022-38533
# 2.39-r0:
# - CVE-2022-38126
# 2.35.2-r1:
# - CVE-2021-3487
# 2.32-r0:
# - CVE-2018-19931
# - CVE-2018-19932
# - CVE-2018-20002
# - CVE-2018-20712
# 2.28-r1:
# - CVE-2017-7614
build() {
local _sysroot=/
local _cross_configure="--enable-install-libiberty --enable-shared"
local _arch_configure=""
local _gold_configure="--disable-gold"
local _plugin_configure="--enable-plugins"
if [ "$CHOST" != "$CTARGET" ]; then
_sysroot="$CBUILDROOT"
_cross_configure="--disable-install-libiberty"
_plugin_configure="--disable-plugins"
fi
if [ "$CHOST" = "$CBUILD" ] && [ "$CBUILD" = "$CTARGET" ] && [ "$CTARGET_ARCH" != "riscv64" ]; then
_gold_configure="--enable-gold"
fi
if [ "$CTARGET_ARCH" = "x86_64" ]; then
_arch_configure="--enable-targets=x86_64-pep"
fi
if [ "$CTARGET_ARCH" = "riscv64" ]; then
_gold_configure="--disable-gold"
fi
case "$CTARGET_ARCH" in
mips*) _hash_style_configure="--enable-default-hash-style=sysv" ;;
*) _hash_style_configure="--enable-default-hash-style=gnu" ;;
esac
./configure \
--build=$CBUILD \
--host=$CHOST \
--target=$CTARGET \
--with-build-sysroot="$CBUILDROOT" \
--with-sysroot=$_sysroot \
--prefix=/usr \
--mandir=/usr/share/man \
--infodir=/usr/share/info \
--disable-multilib \
--disable-gprofng \
--enable-ld=default \
$_gold_configure \
--enable-64-bit-bfd \
$_plugin_configure \
--enable-relro \
--enable-deterministic-archives \
--enable-default-execstack=no \
$_cross_configure \
$_arch_configure \
$_hash_style_configure \
--with-pic \
--disable-werror \
--disable-nls \
--with-mmap \
--with-system-zlib
make
}
package() {
make install DESTDIR="$pkgdir"
if [ -d "$pkgdir"/usr/lib64 ]; then
mv "$pkgdir"/usr/lib64/* "$pkgdir"/usr/lib/
rmdir "$pkgdir"/usr/lib64
fi
if [ "$CHOST" != "$CTARGET" ]; then
# creating cross tools: remove any files that would conflict
# with the native tools, or other cross tools
rm -r "${pkgdir:?}"/usr/share
rm -f "$pkgdir"/usr/lib/libiberty.a
rm -r "${pkgdir:?}"/usr/lib/bfd-plugins
fi
}
libs() {
pkgdesc="Runtime libraries from binutils - libbfd and libopcodes"
mkdir -p "$subpkgdir"/usr/lib
mv "$pkgdir"/usr/lib/lib*.so "$subpkgdir"/usr/lib/
}
gold() {
pkgdesc="GNU binutils - gold linker"
if [ -e "$pkgdir"/usr/bin/ld.gold ]; then
mkdir -p "$subpkgdir"/usr/bin
mv "$pkgdir"/usr/bin/ld.gold "$subpkgdir"/usr/bin
fi
mkdir -p "$subpkgdir"/usr/$CTARGET/bin
mv "$pkgdir"/usr/$CTARGET/bin/ld.gold "$subpkgdir"/usr/$CTARGET/bin/ld.gold
}
sha512sums="
68e038f339a8c21faa19a57bbc447a51c817f47c2e06d740847c6e9cc3396c025d35d5369fa8c3f8b70414757c89f0e577939ddc0d70f283182504920f53b0a3 binutils-2.39.tar.xz
ecee33b0e435aa704af1c334e560f201638ff79e199aa11ed78a72f7c9b46f85fbb227af5748e735fd681d1965fcc42ac81b0c8824e540430ce0c706c81e8b49 binutils-ld-fix-static-linking.patch
f55cf2e0bf82f97583a1abe10710e4013ecf7d64f1da2ef8659a44a06d0dd8beaf58dab98a183488ea137f03e32d62efc878d95f018f836f8cec870bc448556f gold-mips.patch
314d2ef9071c89940aa6c8118e8a1e2f191a5d0a4bf596da1ad9cc84f884d8bc7dea8bd7b9fc3f8f1bddd3fd41c6eb017e1e804044b3bf084df1ed9e6e095e2d ld-bfd-mips.patch
70ec22bd72ef6dddecfd970613387dd4a8cdc8730dd3cbf03d5a0c3a7c4d839383167bb06dad21bf7c235329fd44b5dc4aefe762f68544f17155cf002bf1be4a 0001-Revert-PR25882-.gnu.attributes-are-not-checked-for-s.patch
609cd90d8b334eb309f586b17b9d335a08d3dbb6def7c3eb5c010028fcb681674031e5b9d853aa7a39a50304356a86afc184b85562b3f228f8197f4d29395c8f binutils-mips-disable-assert.patch
631726e557524633ace39c597acbc5ac39bfd25f7b292eee1f0b2a80a503f4ff799f027757d12d5a24b2fa3c07c7e14602d8a86d96e19876d6d0d5d1a02817e7 CVE-2022-38533.patch
"
@@ -0,0 +1,22 @@
X-Git-Url: https://sourceware.org/git/?p=binutils-gdb.git;a=blobdiff_plain;f=bfd%2Fcoffcode.h;h=52027981c3f052d98e043cd0c877f0bb5da97c7f;hp=67aaf158ca1829fe63f0065f31a0b1009b227b36;hb=ef186fe54aa6d281a3ff8a9528417e5cc614c797;hpb=80075150727d653c39e9d037e15c6b87dcef08cd
diff --git a/bfd/coffcode.h b/bfd/coffcode.h
index 67aaf158ca1..52027981c3f 100644
--- a/bfd/coffcode.h
+++ b/bfd/coffcode.h
@@ -4302,10 +4302,13 @@ coff_set_section_contents (bfd * abfd,
rec = (bfd_byte *) location;
recend = rec + count;
- while (rec < recend)
+ while (recend - rec >= 4)
{
+ size_t len = bfd_get_32 (abfd, rec);
+ if (len == 0 || len > (size_t) (recend - rec) / 4)
+ break;
+ rec += len * 4;
++section->lma;
- rec += bfd_get_32 (abfd, rec) * 4;
}
BFD_ASSERT (rec == recend);
@@ -0,0 +1,46 @@
This fixes static linking for our hardened toolchain
diff --git a/ld/scripttempl/elf.sc b/ld/scripttempl/elf.sc
index e8126cb..9532bfb 100644
--- a/ld/scripttempl/elf.sc
+++ b/ld/scripttempl/elf.sc
@@ -235,8 +235,8 @@ test "${LARGE_SECTIONS}" = "yes" && LARGE_SECTIONS="
if test "${ENABLE_INITFINI_ARRAY}" = "yes"; then
SORT_INIT_ARRAY="KEEP (*(SORT_BY_INIT_PRIORITY(.init_array.*) SORT_BY_INIT_PRIORITY(.ctors.*)))"
SORT_FINI_ARRAY="KEEP (*(SORT_BY_INIT_PRIORITY(.fini_array.*) SORT_BY_INIT_PRIORITY(.dtors.*)))"
- CTORS_IN_INIT_ARRAY="EXCLUDE_FILE (*crtbegin.o *crtbegin?.o *crtend.o *crtend?.o $OTHER_EXCLUDE_FILES) .ctors"
- DTORS_IN_FINI_ARRAY="EXCLUDE_FILE (*crtbegin.o *crtbegin?.o *crtend.o *crtend?.o $OTHER_EXCLUDE_FILES) .dtors"
+ CTORS_IN_INIT_ARRAY="EXCLUDE_FILE (*crtbegin*.o *crtend*.o $OTHER_EXCLUDE_FILES) .ctors"
+ DTORS_IN_FINI_ARRAY="EXCLUDE_FILE (*crtbegin*.o *crtend*.o $OTHER_EXCLUDE_FILES) .dtors"
else
SORT_INIT_ARRAY="KEEP (*(SORT(.init_array.*)))"
SORT_FINI_ARRAY="KEEP (*(SORT(.fini_array.*)))"
@@ -270,15 +270,14 @@ CTOR=".ctors ${CONSTRUCTING-0} :
doesn't matter which directory crtbegin.o
is in. */
- KEEP (*crtbegin.o(.ctors))
- KEEP (*crtbegin?.o(.ctors))
+ KEEP (*crtbegin*.o(.ctors))
/* We don't want to include the .ctor section from
the crtend.o file until after the sorted ctors.
The .ctor section from the crtend file contains the
end of ctors marker and it must be last */
- KEEP (*(EXCLUDE_FILE (*crtend.o *crtend?.o $OTHER_EXCLUDE_FILES) .ctors))
+ KEEP (*(EXCLUDE_FILE (*crtend*.o $OTHER_EXCLUDE_FILES) .ctors))
KEEP (*(SORT(.ctors.*)))
KEEP (*(.ctors))
${CONSTRUCTING+${CTOR_END}}
@@ -286,9 +285,8 @@ CTOR=".ctors ${CONSTRUCTING-0} :
DTOR=".dtors ${CONSTRUCTING-0} :
{
${CONSTRUCTING+${DTOR_START}}
- KEEP (*crtbegin.o(.dtors))
- KEEP (*crtbegin?.o(.dtors))
- KEEP (*(EXCLUDE_FILE (*crtend.o *crtend?.o $OTHER_EXCLUDE_FILES) .dtors))
+ KEEP (*crtbegin*.o(.dtors))
+ KEEP (*(EXCLUDE_FILE (*crtend*.o $OTHER_EXCLUDE_FILES) .dtors))
KEEP (*(SORT(.dtors.*)))
KEEP (*(.dtors))
${CONSTRUCTING+${DTOR_END}}
@@ -0,0 +1,14 @@
diff -urN binutils-2.35.2.orig/bfd/elfxx-mips.c binutils-2.35.2/bfd/elfxx-mips.c
--- binutils-2.35.2.orig/bfd/elfxx-mips.c 2021-01-30 01:38:03.000000000 -0700
+++ binutils-2.35.2/bfd/elfxx-mips.c 2021-05-13 17:35:34.706913601 -0600
@@ -6744,8 +6744,10 @@
in the relocation. */
if (h != NULL && ! SYMBOL_REFERENCES_LOCAL (info, &h->root))
{
+#if 0
BFD_ASSERT (htab->root.target_os == is_vxworks
|| h->global_got_area != GGA_NONE);
+#endif
indx = h->root.dynindx;
if (SGI_COMPAT (output_bfd))
defined_p = h->root.def_regular;
+39
View File
@@ -0,0 +1,39 @@
# DP: Fix gold on mips64 targets.
gold/
2016-08-09 Aurelien Jarno <aurelien@aurel32.net>
* configure.tgt: Add mips64el*-*-*|mips64le*-*-* and mips64*-*-*.
--- a/gold/configure.tgt
+++ b/gold/configure.tgt
@@ -153,6 +153,13 @@ aarch64*-*)
targ_big_endian=false
targ_extra_big_endian=true
;;
+mips*64*el*-*-*|mips*64*le*-*-*)
+ targ_obj=mips
+ targ_machine=EM_MIPS_RS3_LE
+ targ_size=64
+ targ_big_endian=false
+ targ_extra_big_endian=true
+ ;;
mips*el*-*-*|mips*le*-*-*)
targ_obj=mips
targ_machine=EM_MIPS_RS3_LE
@@ -160,6 +167,13 @@ mips*el*-*-*|mips*le*-*-*)
targ_big_endian=false
targ_extra_big_endian=true
;;
+mips*64*-*-*)
+ targ_obj=mips
+ targ_machine=EM_MIPS
+ targ_size=64
+ targ_big_endian=true
+ targ_extra_big_endian=false
+ ;;
mips*-*-*)
targ_obj=mips
targ_machine=EM_MIPS
+19
View File
@@ -0,0 +1,19 @@
--- ./ld/configure.tgt.orig
+++ ./ld/configure.tgt
@@ -541,12 +541,12 @@
;;
mips*-*-windiss) targ_emul=elf32mipswindiss
;;
-mips64*el-*-linux-*) targ_emul=elf32ltsmipn32
- targ_extra_emuls="elf32btsmipn32 elf32ltsmip elf32btsmip elf64ltsmip elf64btsmip"
+mips64*el-*-linux-*) targ_emul=elf64ltsmip
+ targ_extra_emuls="elf32ltsmipn32 elf32btsmipn32 elf32ltsmip elf32btsmip elf64btsmip"
targ_extra_libpath=$targ_extra_emuls
;;
-mips64*-*-linux-*) targ_emul=elf32btsmipn32
- targ_extra_emuls="elf32ltsmipn32 elf32btsmip elf32ltsmip elf64btsmip elf64ltsmip"
+mips64*-*-linux-*) targ_emul=elf64btsmip
+ targ_extra_emuls="elf32btsmipn32 elf32ltsmipn32 elf32btsmip elf32ltsmip elf64ltsmip"
targ_extra_libpath=$targ_extra_emuls
;;
mips*el-*-linux-*) targ_emul=elf32ltsmip
+36
View File
@@ -0,0 +1,36 @@
# Automatically generated aport, do not edit!
# Generator: pmbootstrap aportgen busybox-static-aarch64
# Stub for apkbuild-lint
if [ -z "$(type -t arch_to_hostspec)" ]; then
arch_to_hostspec() { :; }
fi
pkgname=busybox-static-aarch64
pkgver=1.35.0
pkgrel=27
_arch="aarch64"
_mirror="http://dl-4.alpinelinux.org/alpine/"
url="http://busybox.net"
license="GPL2"
arch="x86_64"
options="!check !strip"
pkgdesc="Statically linked Busybox for $_arch"
_target="$(arch_to_hostspec $_arch)"
source="
busybox-static-$pkgver-r$pkgrel-$_arch-edge.apk::$_mirror/edge/main/$_arch/busybox-static-$pkgver-r$pkgrel.apk
"
package() {
mkdir -p "$pkgdir/usr/$_target"
cd "$pkgdir/usr/$_target"
tar -xf $srcdir/busybox-static-$pkgver-r$pkgrel-$_arch-edge.apk
rm .PKGINFO .SIGN.*
}
sha512sums="
1df3f1400461a3b9c951be9bf4ade500af9b83b4a11025bb244224cee197753d95471a8373a2f5f45d343599f632629c005dc7eba2c3d558dfbcfa56063ed110 busybox-static-1.35.0-r27-aarch64-edge.apk
"
+36
View File
@@ -0,0 +1,36 @@
# Automatically generated aport, do not edit!
# Generator: pmbootstrap aportgen busybox-static-armhf
# Stub for apkbuild-lint
if [ -z "$(type -t arch_to_hostspec)" ]; then
arch_to_hostspec() { :; }
fi
pkgname=busybox-static-armhf
pkgver=1.35.0
pkgrel=27
_arch="armhf"
_mirror="http://dl-4.alpinelinux.org/alpine/"
url="http://busybox.net"
license="GPL2"
arch="x86_64"
options="!check !strip"
pkgdesc="Statically linked Busybox for $_arch"
_target="$(arch_to_hostspec $_arch)"
source="
busybox-static-$pkgver-r$pkgrel-$_arch-edge.apk::$_mirror/edge/main/$_arch/busybox-static-$pkgver-r$pkgrel.apk
"
package() {
mkdir -p "$pkgdir/usr/$_target"
cd "$pkgdir/usr/$_target"
tar -xf $srcdir/busybox-static-$pkgver-r$pkgrel-$_arch-edge.apk
rm .PKGINFO .SIGN.*
}
sha512sums="
cdc8cfb4e013ad0761d2417fba91f75221d97074705bcd4f8ae7a64d0badadff2340595b5d03ff0e5ca608f543ce85f4ba82820a4fd501ecb758d77432afcf90 busybox-static-1.35.0-r27-armhf-edge.apk
"
+36
View File
@@ -0,0 +1,36 @@
# Automatically generated aport, do not edit!
# Generator: pmbootstrap aportgen busybox-static-armv7
# Stub for apkbuild-lint
if [ -z "$(type -t arch_to_hostspec)" ]; then
arch_to_hostspec() { :; }
fi
pkgname=busybox-static-armv7
pkgver=1.35.0
pkgrel=27
_arch="armv7"
_mirror="http://dl-4.alpinelinux.org/alpine/"
url="http://busybox.net"
license="GPL2"
arch="x86_64"
options="!check !strip"
pkgdesc="Statically linked Busybox for $_arch"
_target="$(arch_to_hostspec $_arch)"
source="
busybox-static-$pkgver-r$pkgrel-$_arch-edge.apk::$_mirror/edge/main/$_arch/busybox-static-$pkgver-r$pkgrel.apk
"
package() {
mkdir -p "$pkgdir/usr/$_target"
cd "$pkgdir/usr/$_target"
tar -xf $srcdir/busybox-static-$pkgver-r$pkgrel-$_arch-edge.apk
rm .PKGINFO .SIGN.*
}
sha512sums="
65678aa673f8fb11d0a68b7d22203ab9ce2795fb6b3d47c117f8cbecf61305316bb0b0de12812df5d7fe2febe0630fd6ca3961434ffe8051da6a0e194c133cb4 busybox-static-1.35.0-r27-armv7-edge.apk
"
+2 -2
View File
@@ -5,7 +5,7 @@
pkgname=ccache-cross-symlinks
pkgver=3
pkgrel=3
pkgrel=0
pkgdesc="Enable ccache for cross-compilers with symlinks"
url="https://ccache.samba.org/"
arch="noarch"
@@ -15,7 +15,7 @@ options="!check"
package() {
# Keep in sync with pmb.config.build_device_architectures
local _archs="armhf armv7 aarch64 x86_64 x86 riscv64 ppc64le loongarch64"
local _archs="armhf armv7 aarch64 x86_64 x86"
local _bins="c++ cc cpp g++ gcc clang clang++"
mkdir -p "$pkgdir/usr/lib/ccache/bin"
+22 -75
View File
@@ -1,6 +1,3 @@
# Maintainer: Oliver Smith <ollieparanoid@postmarketos.org>
# https://docs.postmarketos.org/pmbootstrap/cross_compiling.html
#
# Wrapper for the "crossdirect" compilation method.
# pmbootstrap mounts the native chroot in the foreign arch (e.g. armhf) chroot
# as /native. This package gets installed into the native chroot, and creates
@@ -12,33 +9,26 @@
# -> /native/usr/bin/rustc --target=armv6-alpine-linux-musleabihf ...
#
# When building packages in the armhf chroot, PATH will get prepended with
# "/native/usr/lib/crossdirect/armhf". The end game is of course invoking
# programs such as cross compilers and archivers from the native chroot,
# running at native speed, whenever they get called from the foreign arch
# chroot. See crossdirect.c for implementation details of the C version (llvm,
# fakeroot, rpath). The rust version and redirect for archivers are implemented
# as simple shell wrappers.
# "/native/usr/lib/crossdirect/armhf". The end game is of course invoking the
# cross compiler from the native chroot, running at native speed, whenever
# calling the compiler from the foreign arch chroot. See crossdirect.c for
# implementation details of the C version (llvm, fakeroot, rpath). The rust
# version is implemented as simple shell wrappers.
pkgname=crossdirect
pkgver=5.3.1
pkgrel=1
pkgdesc="Wrappers to launch native programs (e.g. cross compilers) in foreign chroots"
pkgver=4
pkgrel=3
pkgdesc="Wrappers to launch native cross compilers in foreign chroots"
url="https://postmarketOS.org"
arch="all"
license="MIT"
options="!check"
source="
cargo.sh
crossdirect.c
exec_native.sh
rust-qemu-linker.sh
rustc.sh
"
source="crossdirect.c rustc.sh rust-qemu-linker.sh"
build() {
cd "$srcdir"
# Architectures and binaries
_archs="x86_64 armhf armv7 aarch64 loongarch64 ppc64le riscv64"
_archs="x86_64 armhf armv7 aarch64"
for _arch in $_archs; do
[ "$_arch" == "$CARCH" ] && continue
_hostspec="$(arch_to_hostspec $_arch)"
@@ -50,52 +40,20 @@ build() {
-Werror \
-DHOSTSPEC="\"$_hostspec\"" \
crossdirect.c
# rustc
sed "s/@ARCH@/$_arch/g" rustc.sh >rustc-"$_arch"
done
}
package() {
# Architectures and binaries
_archs="
aarch64
armhf
armv7
loongarch64
ppc64le
riscv64
x86_64
"
_bin_compilers="
c++
cc
clang
clang++
cpp
g++
gcc
"
_bin_archivers="
bzip2
gunzip
gzip
lzip
lzma
lzop
pigz
unlzma
unpigz
unxz
unzip
xz
"
_archs="x86_64 armhf armv7 aarch64"
_bins="c++ cc cpp g++ gcc clang clang++"
# Rust wrappers
# Rust: qemu-linker
install -Dm755 "$srcdir/rust-qemu-linker.sh" \
"$pkgdir/usr/lib/crossdirect/rust-qemu-linker"
install -Dm755 "$srcdir/cargo.sh" "$pkgdir/usr/lib/crossdirect/cargo.sh"
install -Dm755 "$srcdir/rustc.sh" "$pkgdir/usr/lib/crossdirect/rustc.sh"
# For archivers
install -Dm755 "$srcdir/exec_native.sh" -t "$pkgdir/usr/lib/crossdirect/"
# Iterate over architectures
for _arch in $_archs; do
@@ -110,26 +68,15 @@ package() {
cp "$srcdir/crossdirect-$_arch" "./"
# GCC: create compiler symlinks
for _bin in $_bin_compilers; do
for _bin in $_bins; do
ln -s "crossdirect-$_arch" "$_bin"
ln -s "crossdirect-$_arch" "$_hostspec-$_bin"
done
# Archivers
for _bin in $_bin_archivers; do
ln -s ../exec_native.sh "$_bin"
done
# Rust
ln -s ../cargo.sh cargo
ln -s ../rustc.sh rustc
# Rust: arch-specific rustc wrapper
install -Dm755 "$srcdir/rustc-$_arch" "$_bindir/rustc"
done
}
sha512sums="
0f4b48f250425dc57f63955fc8b38477d23db793bee367c3fbe03c3d2a559ba576434d07518310db4cae9d90d501af4051b80038b10fae94b980e537fc9374ad cargo.sh
b2e602b7fc95ed403d29f0d754879d5f773c39ee33e726bb10cff17b3c088406dac84f2ca6d8ad563c64370007d98af805d6ee9cd4a06fe3d4c61c0cb90c5c0c crossdirect.c
73e2b5d2d8ef7c6002b3a787d14945586991493b8d81f59e5c0beda0d3b290543e1fdc6f288da726c5dd121c0861d2a894410b7b749a537df0df5b409e102330 exec_native.sh
ea9bf8db3810d03d0a7395057f3d6e57f7127d87c55deaedc171c255288f5f3cc6fbcc680a5a1b92786cf573875d5dc22521173799fe2639acc97d0715ff905b rust-qemu-linker.sh
de2aa6929bbff5db9132673e667f4525b333d062cb1f7a0597dd924de9e5c7215a9f8e2e6263b6fb5ace06f2c4b64f4ebfd0ede9b78add07c2cd08ec8613e98f rustc.sh
"
sha512sums="500107e5aff8b34e74b54982ebcd447fc41bc8283956d506561b031a055cb30ec12698047c2604918b4b6d8560fe03db427a63dff2b83c907e8494b7f5233f29 crossdirect.c
6be16ba88720e628a3ecc8fa53f8e7a21d2af268b0509745d989139874d6b94b640bfcff09575eaa19073810be6ef91169c1f83e94f5cf8e6819f2670d9408de rustc.sh
ea9bf8db3810d03d0a7395057f3d6e57f7127d87c55deaedc171c255288f5f3cc6fbcc680a5a1b92786cf573875d5dc22521173799fe2639acc97d0715ff905b rust-qemu-linker.sh"
-66
View File
@@ -1,66 +0,0 @@
#!/bin/sh -e
if [ -n "$CROSSDIRECT_DEBUG" ]; then
set -x
fi
native_version=$(LD_LIBRARY_PATH=/native/lib:/native/usr/lib /native/usr/bin/rustc -V)
target_version=$(/usr/bin/rustc -V)
if [ "$target_version" != "$native_version" ]; then
echo "ERROR: crossdirect: rustc version mismatch between native and build chroot. Please update your chroots and try again." >&2
exit 1
fi
triplet=$(/usr/bin/rustc -vV | sed -n 's/host: //p')
cmd=$1
shift
case "$cmd" in
build|test|run)
;;
*)
echo "WARNING: crossdirect: 'cargo $cmd $*' command not supported, running in QEMU (slow!)" >&2
export PATH="$(echo "$PATH" | sed 's,/native/usr/lib/crossdirect/[^:]*:,,')"
exec /usr/bin/cargo "$cmd" "$@"
;;
esac
target_dir=${CARGO_TARGET_DIR:-target}
target_already_specified=${CARGO_BUILD_TARGET:+1}
if [ -z "$target_specified" ]; then
for arg in "$@"; do
if [ "$arg_target_dir" = 1 ]; then
target_dir=$arg
arg_target_dir=0
fi
case "$arg" in
--target|--target=*)
target_already_specified=1
break
;;
--target-dir)
arg_target_dir=1
;;
--target-dir=*)
target_dir=${arg##--target-dir=}
;;
esac
done
fi
if [ -n "$target_already_specified" ]; then
exec /usr/bin/cargo "$cmd" "$@"
fi
/usr/bin/cargo "$cmd" --target=$triplet "$@"
# copy target/$triplet/{release,debug,...} to target/{release,debug,...}
if [ -d "$target_dir/$triplet" ]; then
for dir in "$target_dir/$triplet"/*/; do
build=$(basename "$dir")
rm -rf "$target_dir/$build"
cp -r "$target_dir/$triplet/$build" "$target_dir/$build"
done
fi
+8 -13
View File
@@ -43,8 +43,9 @@
void exit_userfriendly()
{
fprintf(stderr, "Please report this at: https://gitlab.postmarketos.org/postmarketOS/pmaports/issues\n");
fprintf(stderr, "As a workaround, you can compile without crossdirect with options=\"!pmb:crossdirect\".\n");
fprintf(stderr, "Please report this at: https://gitlab.com/postmarketOS/pmaports/issues\n");
fprintf(stderr, "As a workaround, you can compile without crossdirect.\n");
fprintf(stderr, "See 'pmbootstrap -h' for related options.\n");
exit(1);
}
@@ -117,17 +118,11 @@ int main(int argc, char **argv)
NULL };
char *ldPreload = getenv("LD_PRELOAD");
if (ldPreload) {
if (strstr(ldPreload, "libfakeroot.so")) {
fprintf(stderr, "============================================================================================\n");
fprintf(stderr, "ERROR: crossdirect was called with: LD_PRELOAD=%s\n", ldPreload);
fprintf(stderr, "This means your package tried to run a compiler during package().\n");
fprintf(stderr, "This is not supported by crossdirect, and usually not a good idea.\n");
fprintf(stderr, "* Try to fix your APKBUILD so it does not run the compiler during package(), only in build()\n");
fprintf(stderr, " * If you're using 'meson install', try to add '--no-rebuild'\n");
fprintf(stderr, "* If this is not possible, you can work around it by setting options=\"!pmb:crossdirect\"\n");
fprintf(stderr, " (compilation will be slower!)\n");
fprintf(stderr, "============================================================================================\n");
exit(1);
if (strcmp(ldPreload, "/usr/lib/libfakeroot.so") == 0) {
env[0] = "LD_PRELOAD=/native/usr/lib/libfakeroot.so";
} else {
fprintf(stderr, "ERROR: crossdirect: can't handle LD_PRELOAD: %s\n", ldPreload);
exit_userfriendly();
}
}
-27
View File
@@ -1,27 +0,0 @@
#!/bin/sh -e
# Execute the binary from the native chroot if it is installed there. This is
# done for archiver tools such as unxz, so they don't need to go through QEMU.
BASENAME="$(basename "$0")"
if [ -n "$LD_PRELOAD" ]; then
# The archiver tool gets called during package(), which abuild runs
# with libfakeroot.so. Some packages do this, like bootchart2, which
# compresses man pages in its "make install". We can't run the native
# binary here due to the LD_PRELOAD, so run the foreign arch one.
export PATH="/usr/bin:/usr/sbin:/sbin:/bin"
exec "$BASENAME" "$@"
fi
# Try /native first, then the foreign arch paths. Never try
# /native/usr/lib/crossdirect as this would result in a loop.
UNWRAPPED_PATH="/native/usr/bin:/native/usr/sbin:/native/sbin:/native/bin:/usr/bin:/usr/sbin:/sbin:/bin"
UNWRAPPED_BIN="$(PATH="$UNWRAPPED_PATH" command -v "$BASENAME")"
case "$UNWRAPPED_BIN" in
/native/*)
export LD_LIBRARY_PATH="/native/usr/lib:/native/lib"
export PATH="$UNWRAPPED_PATH"
;;
esac
exec "$UNWRAPPED_BIN" "$@"
+41 -31
View File
@@ -1,38 +1,48 @@
#!/bin/sh -e
if [ -n "$CROSSDIRECT_DEBUG" ]; then
set -x
fi
rust_triplet() {
# Find the triplets in Alpine's rust APKBUILD or with:
# pmbootstrap chroot -barmhf --add=rust -- ls /usr/lib/rustlib
case "$1" in
x86_64)
echo "x86_64-alpine-linux-musl"
;;
armhf)
echo "armv6-alpine-linux-musleabihf"
;;
armv7)
echo "armv7-alpine-linux-musleabihf"
;;
aarch64)
echo "aarch64-alpine-linux-musl"
;;
*)
echo "ERROR: don't know the rust triple for $1!" >&2
exit 1
;;
esac
}
# return the correct host architecture when cargo requests it
if [ "$*" = "-vV" ] || [ "$*" = "-V" ]; then
exec /usr/bin/rustc "$*"
fi
arch="@ARCH@" # filled in by APKBUILD
# pmbootstrap installs sccache when building a rust program, unless --no-ccache
# is set. In that case it also sets CCACHE_DISABLE.
SCCACHE=/native/usr/bin/sccache
if ! [ -e "$SCCACHE" ] || [ -n "$CCACHE_DISABLE" ]; then
SCCACHE=""
fi
# We expect the right target to be set in the arguments if compiling for the
# target architecture. Our cargo wrapper passes the right "--target" argument
# automatically. If no target is provided, this is probably a macro or a
# build script, so it should be compiled for the native architecture.
if echo "$*" | grep -qFe "--target"; then
# Not using sccache when building for foreign architecture, as it
# doesn't cache the output with "Non-cacheable reasons: --sysroot"
LD_LIBRARY_PATH=/native/lib:/native/usr/lib \
/native/usr/bin/rustc \
if ! LD_LIBRARY_PATH=/native/lib:/native/usr/lib \
/native/usr/bin/rustc \
-Clinker=/native/usr/lib/crossdirect/rust-qemu-linker \
--target=$(rust_triplet "$arch") \
--sysroot=/usr \
"$@"
else
PATH=/native/usr/bin:/native/bin \
LD_LIBRARY_PATH=/native/lib:/native/usr/lib \
$SCCACHE \
/native/usr/bin/rustc \
-Clink-arg=-Wl,-rpath,/native/lib:/native/usr/lib \
"$@"
"$@"; then
echo "---" >&2
echo "WARNING: crossdirect: cross compiling with rustc failed, trying"\
"again with rustc + qemu" >&2
echo "---" >&2
# Usually the crossdirect approach works; however, when passing
# --extern to rustc with a dynamic library (.so), it fails with an
# error like 'can't find crate for `serde_derive`' (although the crate
# does exist). I think it fails to parse the metadata of the so file
# for some reason. We probably need to adjust rustc's
# librustc_metadata/locator.rs or something (and upstream that
# change!), but I've spent enough time on this already. Let's simply
# fall back to compiling in qemu in the very few cases where this is
# necessary.
/usr/bin/rustc "$@"
fi
@@ -1,130 +0,0 @@
From 824eae920f0f64dd7687c969a2436b129161fdc9 Mon Sep 17 00:00:00 2001
From: Jinyang He <hejinyang@loongson.cn>
Date: Wed, 29 Oct 2025 16:07:35 +0800
Subject: [PATCH] LoongArch: Only allow valid binary op when optimize
conditional move
It is wrong that optimize from `if (cond) dest op= 1 << shift` to
`dest op= (cond ? 1 : 0) << shift` when `dest op 0 != dest`.
Like `and`, `mul` or `div`.
And in this optimization `mul` and `div` is optimized to shift.
gcc/ChangeLog:
* config/loongarch/loongarch.cc
(loongarch_expand_conditional_move): Only allow valid binary
op when optimize conditional move.
gcc/testsuite/ChangeLog:
* gcc.target/loongarch/conditional-move-opt-1.c: Remove mul.
* gcc.target/loongarch/conditional-move-opt-2.c: Remove and.
* gcc.target/loongarch/conditional-move-opt-3.c: New test.
Co-Authored-By: Peng Fan <fanpeng@loongson.cn>
---
gcc/config/loongarch/loongarch.cc | 26 ++++++++++++++++---
.../loongarch/conditional-move-opt-1.c | 4 +--
.../loongarch/conditional-move-opt-2.c | 2 +-
.../loongarch/conditional-move-opt-3.c | 14 ++++++++++
4 files changed, 40 insertions(+), 6 deletions(-)
create mode 100644 gcc/testsuite/gcc.target/loongarch/conditional-move-opt-3.c
diff --git a/gcc/config/loongarch/loongarch.cc b/gcc/config/loongarch/loongarch.cc
index bdf290654..740c8611a 100644
--- a/gcc/config/loongarch/loongarch.cc
+++ b/gcc/config/loongarch/loongarch.cc
@@ -5533,12 +5533,32 @@ loongarch_expand_conditional_move (rtx *operands)
}
}
+ auto is_binary_op_0_keep_orig = [](enum rtx_code code)
+ {
+ switch (code)
+ {
+ case PLUS:
+ case MINUS:
+ case IOR:
+ case XOR:
+ case ROTATE:
+ case ROTATERT:
+ case ASHIFT:
+ case ASHIFTRT:
+ case LSHIFTRT:
+ return true;
+ default:
+ return false;
+ }
+ };
+
/* Check if the optimization conditions are met. */
if (value_if_true_insn
&& value_if_false_insn
- /* Make sure that value_if_false and var are the same. */
- && BINARY_P (value_if_true_insn_src
- = SET_SRC (single_set (value_if_true_insn)))
+ /* Make sure that the orig value OP 0 keep orig. */
+ && (value_if_true_insn_src
+ = SET_SRC (single_set (value_if_true_insn)))
+ && is_binary_op_0_keep_orig ( GET_CODE (value_if_true_insn_src))
/* Make sure that both value_if_true and value_if_false
has the same var. */
&& rtx_equal_p (XEXP (value_if_true_insn_src, 0),
diff --git a/gcc/testsuite/gcc.target/loongarch/conditional-move-opt-1.c b/gcc/testsuite/gcc.target/loongarch/conditional-move-opt-1.c
index ed13471aa..47802aa96 100644
--- a/gcc/testsuite/gcc.target/loongarch/conditional-move-opt-1.c
+++ b/gcc/testsuite/gcc.target/loongarch/conditional-move-opt-1.c
@@ -27,7 +27,7 @@ void
test_lt ()
{
if (lm < ln)
- lr *= (1 << 16);
+ lr += (1 << 16);
lr += lm;
}
@@ -35,7 +35,7 @@ void
test_le ()
{
if (lm <= ln)
- lr = lm * ((long)1 << 32);
+ lr = lm + ((long)1 << 32);
else
lr = lm;
lr += lm;
diff --git a/gcc/testsuite/gcc.target/loongarch/conditional-move-opt-2.c b/gcc/testsuite/gcc.target/loongarch/conditional-move-opt-2.c
index ac72d4d93..743fd5e67 100644
--- a/gcc/testsuite/gcc.target/loongarch/conditional-move-opt-2.c
+++ b/gcc/testsuite/gcc.target/loongarch/conditional-move-opt-2.c
@@ -29,7 +29,7 @@ void
test_lez ()
{
if (lm <= 0)
- lr &= (1 << 16);
+ lr |= (1 << 16);
lr += lm;
}
diff --git a/gcc/testsuite/gcc.target/loongarch/conditional-move-opt-3.c b/gcc/testsuite/gcc.target/loongarch/conditional-move-opt-3.c
new file mode 100644
index 000000000..95887980c
--- /dev/null
+++ b/gcc/testsuite/gcc.target/loongarch/conditional-move-opt-3.c
@@ -0,0 +1,14 @@
+/* { dg-do compile } */
+/* { dg-options "-O2" } */
+/* { dg-final { scan-assembler "maskeqz" } } */
+/* { dg-final { scan-assembler "masknez" } } */
+
+extern long lm, ln, lr;
+
+void
+test_and ()
+{
+ if (lm < 0)
+ lr &= (1 << 16);
+ lr += lm;
+}
--
2.51.1
+2 -5
View File
@@ -1,4 +1,4 @@
From 94cf7d2b3514ed3a4401861a519ead450e022afc Mon Sep 17 00:00:00 2001
From 74c7b0df9971b6bc75d7d172c2819ffd21d51050 Mon Sep 17 00:00:00 2001
From: Szabolcs Nagy <nsz@port70.net>
Date: Fri, 26 Jan 2018 20:32:50 +0000
Subject: [PATCH] posix_memalign
@@ -8,7 +8,7 @@ Subject: [PATCH] posix_memalign
1 file changed, 5 insertions(+), 4 deletions(-)
diff --git a/gcc/config/i386/pmm_malloc.h b/gcc/config/i386/pmm_malloc.h
index d1d79a32fc9..069c3bb8324 100644
index 3b2e4f47915..20f790904a2 100644
--- a/gcc/config/i386/pmm_malloc.h
+++ b/gcc/config/i386/pmm_malloc.h
@@ -27,12 +27,13 @@
@@ -37,6 +37,3 @@ index d1d79a32fc9..069c3bb8324 100644
return __ptr;
else
return NULL;
--
2.50.1
@@ -1,77 +0,0 @@
From 21b7d08714c4964403848db4144003835ff5245a Mon Sep 17 00:00:00 2001
From: Linsen Zhou <i@lin.moe>
Date: Fri, 17 Oct 2025 11:05:04 +0800
Subject: [PATCH] tree-object-size.cc: Fix assert constant offset in
check_for_plus_in_loops [PR122012]
After commit 51b85dfeb19652bf3e0aaec08828ba7cee1e641c, when the
pointer offset is a variable in the loop, the object size of the
pointer may also need to be reexamined.
Which make gcc_assert in the check_for_plus_in_loops failed.
gcc/ChangeLog:
PR tree-optimization/122012
* tree-object-size.cc (check_for_plus_in_loops): Skip check
for the variable offset
gcc/testsuite/ChangeLog:
PR tree-optimization/122012
* gcc.dg/torture/pr122012.c: New test.
Signed-off-by: Linsen Zhou <i@lin.moe>
(cherry picked from commit 82cefc4898d4ccabe76e28d6626b91ca9e998923)
---
gcc/testsuite/gcc.dg/torture/pr122012.c | 18 ++++++++++++++++++
gcc/tree-object-size.cc | 7 +++----
2 files changed, 21 insertions(+), 4 deletions(-)
create mode 100644 gcc/testsuite/gcc.dg/torture/pr122012.c
diff --git a/gcc/testsuite/gcc.dg/torture/pr122012.c b/gcc/testsuite/gcc.dg/torture/pr122012.c
new file mode 100644
index 00000000000..055915a1e98
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/torture/pr122012.c
@@ -0,0 +1,18 @@
+/* { dg-do compile } */
+#include <stdlib.h>
+
+void foo();
+
+void test(size_t step) {
+ char *buf = malloc(64);
+ char *p = buf;
+ size_t i;
+
+ for(i = 0; i < 64; ++i) {
+ p += 4;
+ if (__builtin_object_size (p, 2) != 0)
+ foo();
+ p += step;
+ }
+ free(buf);
+}
diff --git a/gcc/tree-object-size.cc b/gcc/tree-object-size.cc
index 2d13ab79a84..76dc2446059 100644
--- a/gcc/tree-object-size.cc
+++ b/gcc/tree-object-size.cc
@@ -2153,12 +2153,11 @@ check_for_plus_in_loops (struct object_size_info *osi, tree var)
&& gimple_assign_rhs_code (stmt) == POINTER_PLUS_EXPR)
{
tree basevar = gimple_assign_rhs1 (stmt);
- tree cst = gimple_assign_rhs2 (stmt);
-
- gcc_assert (TREE_CODE (cst) == INTEGER_CST);
+ tree offset = gimple_assign_rhs2 (stmt);
/* Skip non-positive offsets. */
- if (integer_zerop (cst) || compare_tree_int (cst, offset_limit) > 0)
+ if (TREE_CODE (offset) != INTEGER_CST
+ || integer_zerop (offset) || compare_tree_int (offset, offset_limit) > 0)
return;
osi->depths[SSA_NAME_VERSION (basevar)] = 1;
--
2.51.0
@@ -1,6 +1,6 @@
From 0d5e04099a3d2609105719f7396a4ff941b873b6 Mon Sep 17 00:00:00 2001
From dd1e75ff4ecda96957465b34681f245e818928bd Mon Sep 17 00:00:00 2001
From: Khem Raj <raj.khem@gmail.com>
Date: Thu, 23 Jan 2025 17:12:51 -0800
Date: Fri, 29 Mar 2013 08:59:00 +0400
Subject: [PATCH] gcc: poison-system-directories
Add /sw/include and /opt/include based on the original
@@ -11,25 +11,25 @@ aborted.
Instead, we add the two missing items to the current scan. If the user
wants this to be a failure, they can add "-Werror=poison-system-directories".
Upstream-Status: Inappropriate [OE configuration]
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: sunil dora <sunilkumar.dora@windriver.com>
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Upstream-Status: Pending
---
gcc/common.opt | 4 ++++
gcc/config.in | 10 ++++++++++
gcc/configure | 19 +++++++++++++++++++
gcc/configure.ac | 16 ++++++++++++++++
gcc/config.in | 6 ++++++
gcc/configure | 16 ++++++++++++++++
gcc/configure.ac | 10 ++++++++++
gcc/doc/invoke.texi | 9 +++++++++
gcc/gcc.cc | 12 ++++++++++--
gcc/incpath.cc | 25 +++++++++++++++++++++++++
7 files changed, 93 insertions(+), 2 deletions(-)
gcc/gcc.cc | 2 ++
gcc/incpath.cc | 21 +++++++++++++++++++++
7 files changed, 68 insertions(+)
diff --git a/gcc/common.opt b/gcc/common.opt
index e3fa0dacec4..bea1adc0940 100644
index 8a0dafc522d..0357868e22c 100644
--- a/gcc/common.opt
+++ b/gcc/common.opt
@@ -729,6 +729,10 @@ Wreturn-local-addr
@@ -710,6 +710,10 @@ Wreturn-local-addr
Common Var(warn_return_local_addr) Init(1) Warning
Warn about returning a pointer/reference to a local or temporary variable.
@@ -41,10 +41,10 @@ index e3fa0dacec4..bea1adc0940 100644
Common Var(warn_shadow) Warning
Warn when one variable shadows another. Same as -Wshadow=global.
diff --git a/gcc/config.in b/gcc/config.in
index a79c51adb2b..8a531ed591c 100644
index 64c27c9cfac..f0fbf123aa8 100644
--- a/gcc/config.in
+++ b/gcc/config.in
@@ -249,6 +249,16 @@
@@ -230,6 +230,12 @@
#endif
@@ -52,28 +52,24 @@ index a79c51adb2b..8a531ed591c 100644
+#ifndef USED_FOR_TARGET
+#undef ENABLE_POISON_SYSTEM_DIRECTORIES
+#endif
+/* Define to warn for use of native system header directories */
+#ifndef USED_FOR_TARGET
+#undef POISON_BY_DEFAULT
+#endif
+
+
/* Define if you want all operations on RTL (the basic data structure of the
optimizer and back end) to be checked for dynamic type safety at runtime.
This is quite expensive. */
diff --git a/gcc/configure b/gcc/configure
index 16965953f05..0f4a5d52c30 100755
index 5ce0557719a..e9060112384 100755
--- a/gcc/configure
+++ b/gcc/configure
@@ -1051,6 +1051,7 @@ enable_maintainer_mode
@@ -1023,6 +1023,7 @@ enable_maintainer_mode
enable_link_mutex
enable_link_serialization
enable_version_specific_runtime_libs
+enable_poison_system_directories
enable_plugin
enable_host_shared
enable_host_pie
@@ -1828,6 +1829,8 @@ Optional Features:
enable_libquadmath_support
@@ -1785,6 +1786,8 @@ Optional Features:
--enable-version-specific-runtime-libs
specify that runtime libraries should be installed
in a compiler-specific directory
@@ -81,8 +77,8 @@ index 16965953f05..0f4a5d52c30 100755
+ warn for use of native system header directories
--enable-plugin enable plugin support
--enable-host-shared build host code as shared libraries
--enable-host-pie build host code as PIE
@@ -34027,6 +34030,22 @@ if test "${enable_version_specific_runtime_libs+set}" = set; then :
--disable-libquadmath-support
@@ -31982,6 +31985,19 @@ if test "${enable_version_specific_runtime_libs+set}" = set; then :
fi
@@ -93,12 +89,9 @@ index 16965953f05..0f4a5d52c30 100755
+ enable_poison_system_directories=no
+fi
+
+if test "x${enable_poison_system_directories}" != "xno"; then
+if test "x${enable_poison_system_directories}" = "xyes"; then
+
+$as_echo "#define ENABLE_POISON_SYSTEM_DIRECTORIES 1" >>confdefs.h
+if test "$enable_poison_system_directories" = "error"; then
+$as_echo "#define POISON_BY_DEFAULT 1" >>confdefs.h
+fi
+
+fi
+
@@ -106,105 +99,68 @@ index 16965953f05..0f4a5d52c30 100755
diff --git a/gcc/configure.ac b/gcc/configure.ac
index 9f67e62950a..b0e3615e5aa 100644
index 23bee7010a3..75a7048045c 100644
--- a/gcc/configure.ac
+++ b/gcc/configure.ac
@@ -7502,6 +7502,22 @@ AC_ARG_ENABLE(version-specific-runtime-libs,
@@ -7421,6 +7421,16 @@ AC_ARG_ENABLE(version-specific-runtime-libs,
[specify that runtime libraries should be
installed in a compiler-specific directory])])
+AC_ARG_ENABLE([poison-system-directories],
+ AS_HELP_STRING([--enable-poison-system-directories],
+ [warn for use of native system header directories (no/yes/error)]),,
+ [warn for use of native system header directories]),,
+ [enable_poison_system_directories=no])
+AC_MSG_NOTICE([poisoned directories $enable_poison_system_directories])
+if test "x${enable_poison_system_directories}" != "xno"; then
+ AC_MSG_NOTICE([poisoned directories enabled])
+if test "x${enable_poison_system_directories}" = "xyes"; then
+ AC_DEFINE([ENABLE_POISON_SYSTEM_DIRECTORIES],
+ [1],
+ [Define to warn for use of native system header directories])
+ if test $enable_poison_system_directories = "error"; then
+ AC_MSG_NOTICE([poisoned directories are fatal])
+ AC_DEFINE([POISON_BY_DEFAULT], [1], [Define to make poison warnings errors])
+ fi
+fi
+
# Substitute configuration variables
AC_SUBST(subdirs)
AC_SUBST(srcdir)
diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi
index 72009c33cd0..29aea724731 100644
index 07b440190c3..b2de464798a 100644
--- a/gcc/doc/invoke.texi
+++ b/gcc/doc/invoke.texi
@@ -405,6 +405,7 @@ Objective-C and Objective-C++ Dialects}.
-Wpacked -Wno-packed-bitfield-compat -Wpacked-not-aligned -Wpadded
-Wparentheses -Wno-pedantic-ms-format
-Wpointer-arith -Wno-pointer-compare -Wno-pointer-to-int-cast
+-Wno-poison-system-directories
-Wno-pragmas -Wno-pragma-once-outside-header -Wno-prio-ctor-dtor
-Wno-psabi
-Wredundant-decls -Wrestrict
@@ -10727,6 +10728,14 @@ an error. @option{Wint-to-pointer-cast} is enabled by default.
Suppress warnings from casts from a pointer to an integer type of a
different size.
@@ -379,6 +379,7 @@ Objective-C and Objective-C++ Dialects}.
-Wpacked -Wno-packed-bitfield-compat -Wpacked-not-aligned -Wpadded @gol
-Wparentheses -Wno-pedantic-ms-format @gol
-Wpointer-arith -Wno-pointer-compare -Wno-pointer-to-int-cast @gol
+-Wno-poison-system-directories @gol
-Wno-pragmas -Wno-prio-ctor-dtor -Wredundant-decls @gol
-Wrestrict -Wno-return-local-addr -Wreturn-type @gol
-Wno-scalar-storage-order -Wsequence-point @gol
@@ -8029,6 +8030,14 @@ made up of data only and thus requires no special treatment. But, for
most targets, it is made up of code and thus requires the stack to be
made executable in order for the program to work properly.
+@opindex Wno-poison-system-directories
+@item -Wno-poison-system-directories
+@opindex Wno-poison-system-directories
+Do not warn for @option{-I} or @option{-L} options using system
+directories such as @file{/usr/include} when cross compiling. This
+option is intended for use in chroot environments when such
+directories contain the correct headers and libraries for the target
+system rather than the host.
+
@opindex Winvalid-pch
@opindex Wno-invalid-pch
@item -Winvalid-pch
@item -Wfloat-equal
@opindex Wfloat-equal
@opindex Wno-float-equal
diff --git a/gcc/gcc.cc b/gcc/gcc.cc
index 4fd87f2c4a1..79d4920a047 100644
index bb07cc244e3..47fb5802194 100644
--- a/gcc/gcc.cc
+++ b/gcc/gcc.cc
@@ -909,6 +909,12 @@ proper position among the other output files. */
#define ASM_MAP ""
#endif
+#ifdef POISON_BY_DEFAULT
+#define POISON_IS_ERROR " -Werror=poison-system-directories"
+#else
+#define POISON_IS_ERROR
+#endif
+
/* Assembler options for compressed debug sections. */
#if HAVE_LD_COMPRESS_DEBUG == 0
/* Reject if the linker cannot write compressed debug sections. */
@@ -1166,6 +1172,8 @@ proper position among the other output files. */
@@ -1159,6 +1159,8 @@ proper position among the other output files. */
"%{fuse-ld=*:-fuse-ld=%*} " LINK_COMPRESS_DEBUG_SPEC \
"%X %{o*} %{e*} %{N} %{n} %{r}\
%{s} %{t} %{u*} %{z} %{Z} %{!nostdlib:%{!r:%{!nostartfiles:%S}}} \
+ %{Wno-poison-system-directories:--no-poison-system-directories} \
+ %{Werror=poison-system-directories:--error-poison-system-directories} \
%{static|no-pie|static-pie:} %@{L*} %(link_libgcc) " \
%{static|no-pie|static-pie:} %@{L*} %(mfwrap) %(link_libgcc) " \
VTABLE_VERIFICATION_SPEC " " SANITIZER_EARLY_SPEC " %o "" \
%{fopenacc|fopenmp|%:gt(%{ftree-parallelize-loops=*:%*} 1):\
@@ -1268,7 +1276,7 @@ static const char *cpp_options =
"%(cpp_unique_options) %1 %{m*} %{std*&ansi&trigraphs} %{W*&pedantic*} %{w}\
%{f*} %{g*:%{%:debug-level-gt(0):%{g*}\
%{!fno-working-directory:-fworking-directory}}} %{O*}\
- %{undef} %{save-temps*:-fpch-preprocess}";
+ %{undef} %{save-temps*:-fpch-preprocess}" POISON_IS_ERROR;
/* Pass -d* flags, possibly modifying -dumpdir, -dumpbase et al.
@@ -1297,7 +1305,7 @@ static const char *cc1_options =
%{coverage:-fprofile-arcs -ftest-coverage}\
%{fprofile-arcs|fcondition-coverage|fpath-coverage|fprofile-generate*|coverage:\
%{!fprofile-update=single:\
- %{pthread:-fprofile-update=prefer-atomic}}}";
+ %{pthread:-fprofile-update=prefer-atomic}}}" POISON_IS_ERROR;
static const char *asm_options =
"%{-target-help:%:print-asm-header()} "
diff --git a/gcc/incpath.cc b/gcc/incpath.cc
index c1bbfd3872a..2cdbebc4c50 100644
index bd2a97938eb..c80f100f476 100644
--- a/gcc/incpath.cc
+++ b/gcc/incpath.cc
@@ -26,6 +26,7 @@
@@ -215,20 +171,9 @@ index c1bbfd3872a..2cdbebc4c50 100644
/* Microsoft Windows does not natively support inodes.
VMS has non-numeric inodes. */
@@ -273,6 +274,10 @@ remove_duplicates (cpp_reader *pfile, struct cpp_dir *head,
cur->name, xstrerror (errno));
reason = REASON_NOENT;
}
+#ifdef ENABLE_POISON_SYSTEM_DIRECTORIES
+ pcur = &cur->next;
+ continue;
+#endif
}
else if (!S_ISDIR (st.st_mode))
cpp_error_with_line (pfile, CPP_DL_WARNING, 0, 0,
@@ -411,6 +416,26 @@ merge_include_chains (const char *sysroot, cpp_reader *pfile, int verbose)
fprintf (stderr, _("End of #embed search list.\n"));
@@ -399,6 +400,26 @@ merge_include_chains (const char *sysroot, cpp_reader *pfile, int verbose)
}
fprintf (stderr, _("End of search list.\n"));
}
+
+#ifdef ENABLE_POISON_SYSTEM_DIRECTORIES
@@ -237,22 +182,19 @@ index c1bbfd3872a..2cdbebc4c50 100644
+ struct cpp_dir *p;
+
+ for (p = heads[INC_QUOTE]; p; p = p->next)
+ {
+ if ((!strncmp (p->name, "/usr/include", 12))
+ || (!strncmp (p->name, "/usr/local/include", 18))
+ || (!strncmp (p->name, "/usr/X11R6/include", 18))
+ || (!strncmp (p->name, "/sw/include", 11))
+ || (!strncmp (p->name, "/opt/include", 12)))
+ warning (OPT_Wpoison_system_directories,
+ "include location \"%s\" is unsafe for "
+ "cross-compilation",
+ p->name);
+ }
+ {
+ if ((!strncmp (p->name, "/usr/include", 12))
+ || (!strncmp (p->name, "/usr/local/include", 18))
+ || (!strncmp (p->name, "/usr/X11R6/include", 18))
+ || (!strncmp (p->name, "/sw/include", 11))
+ || (!strncmp (p->name, "/opt/include", 12)))
+ warning (OPT_Wpoison_system_directories,
+ "include location \"%s\" is unsafe for "
+ "cross-compilation",
+ p->name);
+ }
+ }
+#endif
}
/* Use given -I paths for #include "..." but not #include <...>, and
--
2.50.1
Loaded 100 of 8681 files, more files were not shown because too many files have changed in this diff. Show more