tools/coreutils: update to 9.1 by graysky2 · Pull Request #9347 · openwrt/openwrt

graysky2 · 2022-02-26T13:39:53Z

In addition to the version update, this commit also applies a fixup to allow building on MacOS involving renaming: [gt_TYPE_WINT_T] --> [gt_TYPE_WINT_T_GNUTLS] suggested by zhanhb.

Build system: x86_64
Build-tested: bcm2711/RPi4B

Signed-off-by: John Audia [email protected]

paper42 · 2022-02-26T19:14:18Z

This bugfix sounds important and I think should be patched: https://git.savannah.gnu.org/cgit/coreutils.git/commit/?id=e8b56ebd536e82b15542a00c888109471936bfda. Arch, Alpine and Void import this patch.

neheb · 2022-02-26T19:22:10Z

Prefix title with tools/

graysky2 · 2022-02-26T21:51:03Z

Does anyone have some insight into why the macos build failed?

aparcar · 2022-02-27T08:54:46Z

I'm surprised no logs are stored. I'll look into that later

graysky2 · 2022-03-01T12:40:11Z

@aparcar - At least there is some output now:

 make[2] -C tools/libtool compile
    ERROR: tools/coreutils failed to build.
make -r tools/install: build failed. Please re-run make with -j1 V=s or V=sc for a higher verbosity level to see what's going on
make: *** [/Volumes/OpenWrt/openwrt/include/toplevel.mk:230: tools/install] Error 1
Error: Process completed with exit code 2.

Is there a way to build with the verbose setting? I do not have a MacOS device. Perhaps it can be virtualized. Never looked into that.

ynezz · 2022-03-01T20:20:13Z

Does anyone have some insight into why the macos build failed?

configure.ac:38: error: version mismatch.  This is Automake 1.16.5,
configure.ac:38: but the definition used by this AM_INIT_AUTOMAKE
configure.ac:38: comes from Automake 1.16.4.  You should recreate
configure.ac:38: aclocal.m4 with aclocal and run automake again.
WARNING: 'automake-1.16' is probably too old.
         You should only need it if you modified 'Makefile.am' or
         'configure.ac' or m4 files included by 'configure.ac'.
make[1]: *** [tools/Makefile:169: tools/coreutils/compile] Error 1
         The 'automake' program is part of the GNU Automake package:

Is there a way to build with the verbose setting?

diff --git a/.github/workflows/tools.yml b/.github/workflows/tools.yml
index 5b400b616d69..fcf7c04ceedc 100644
--- a/.github/workflows/tools.yml
+++ b/.github/workflows/tools.yml
@@ -114,12 +114,13 @@ jobs:
         run: |
           cd "$WORKPATH"
           pwd
+          printf 'CONFIG_DEVEL=y\nCONFIG_BUILD_LOG=y\n' > .config
           make defconfig
 
       - name: Build tools
         run: |
           cd "$WORKPATH"
-          make tools/install -j$(nproc) BUILD_LOG=1
+          make tools/install -j$(nproc) || make tools/install V=s -j1
 
       - name: Upload logs
         if: failure()

I do not have a MacOS device.

You've access to GH Actions so you've access to such machine.

Perhaps it can be virtualized.

Via GH Actions you've already access to the Linux/Windows/macOS virtual machines. The question is, for how long we're going to enjoy this free lunch :-)

This bugfix sounds important and I think should be patched

I'm not sure whether it's clear, but tools/coreutils is only being used on not Linux OSes, such as macOS. On Linux we take whatever is available on the host
This bugfix is for chmod, but actually only date readlink touch ln chown is being built/used from tools/coreutils

neheb · 2022-03-01T20:43:33Z

Does anyone have some insight into why the macos build failed?

configure.ac:38: error: version mismatch.  This is Automake 1.16.5,
configure.ac:38: but the definition used by this AM_INIT_AUTOMAKE
configure.ac:38: comes from Automake 1.16.4.  You should recreate
configure.ac:38: aclocal.m4 with aclocal and run automake again.
WARNING: 'automake-1.16' is probably too old.
         You should only need it if you modified 'Makefile.am' or
         'configure.ac' or m4 files included by 'configure.ac'.
make[1]: *** [tools/Makefile:169: tools/coreutils/compile] Error 1
         The 'automake' program is part of the GNU Automake package:

Is there a way to build with the verbose setting?

diff --git a/.github/workflows/tools.yml b/.github/workflows/tools.yml
index 5b400b616d69..fcf7c04ceedc 100644
--- a/.github/workflows/tools.yml
+++ b/.github/workflows/tools.yml
@@ -114,12 +114,13 @@ jobs:
         run: |
           cd "$WORKPATH"
           pwd
+          printf 'CONFIG_DEVEL=y\nCONFIG_BUILD_LOG=y\n' > .config
           make defconfig
 
       - name: Build tools
         run: |
           cd "$WORKPATH"
-          make tools/install -j$(nproc) BUILD_LOG=1
+          make tools/install -j$(nproc) || make tools/install V=s -j1
 
       - name: Upload logs
         if: failure()

I do not have a MacOS device.

You've access to GH Actions so you've access to such machine.

Perhaps it can be virtualized.

Via GH Actions you've already access to the Linux/Windows/macOS virtual machines. The question is, for how long we're going to enjoy this free lunch :-)

Long time. Microsoft has enough money to subsidize CI.

This bugfix sounds important and I think should be patched

1. I'm not sure whether it's clear, but `tools/coreutils` is only being used on not Linux OSes, such as macOS. On Linux we take whatever is available on the host

2. This bugfix is for `chmod`, but actually only `date readlink touch ln chown` is being built/used from `tools/coreutils`

aparcar · 2022-03-09T13:27:17Z

Please rebase this onto a5eeac8 since it fixes the log issue for macOS.

make tools/install -j$(nproc) || make tools/install V=s -j1

@ynezz I thought of this to however it may result in a failure of the parallel build which is fixed in a second run and thereby hidden. I prefer the build log artifact variant since a build finishes and can't silently fail.

aparcar · 2022-03-09T13:27:52Z

I added the blocked tag, this shouldn't be added prior the branch.

graysky2 · 2022-03-09T18:33:56Z

Please rebase this onto a5eeac8 since it fixes the log issue for macOS.

@aparcar - Not sure if I did that correctly... Guessing not.

aparcar · 2022-03-09T19:04:39Z

@graysky2 you did, I find the CI artifacts right here:

neheb · 2022-03-09T20:06:36Z

host/coreutils-9.0/build-aux/missing automake-1.16 --gnu Makefile
aclocal.m4:17: warning: this file was generated for autoconf 2.71.
You have another version of autoconf.  It may work, but is not guaranteed to.
If you have problems, you may need to regenerate the build system entirely.
To do so, use the procedure documented by the package, typically 'autoreconf'.
configure.ac:38: error: version mismatch.  This is Automake 1.16.5,
configure.ac:38: but the definition used by this AM_INIT_AUTOMAKE
configure.ac:38: comes from Automake 1.16.4.  You should recreate
configure.ac:38: aclocal.m4 with aclocal and run automake again.
WARNING: 'automake-1.16' is probably too old.
         You should only need it if you modified 'Makefile.am' or
         'configure.ac' or m4 files included by 'configure.ac'.
         The 'automake' program is part of the GNU Automake package:
         <https://www.gnu.org/software/automake>
         It also requires GNU Autoconf, GNU m4 and Perl in order to run:
         <https://www.gnu.org/software/autoconf>
         <https://www.gnu.org/software/m4/>
         <https://www.perl.org/>
make[3]: *** [Makefile:7393: Makefile.in] Error 63
make[3]: Leaving directory '/Volumes/OpenWrt/openwrt/build_dir/host/coreutils-9.0'
make[2]: *** [Makefile:38: /Volumes/OpenWrt/openwrt/build_dir/host/coreutils-9.0/.built] Error 2
time: tools/coreutils/compile#77.61#61.28#959.31

zhanhb · 2022-03-19T03:50:53Z

There are two issues to be resolved.

autoreconf should be run to regenerate aclocal.m4
function gt_TYPE_WINT_T conflict in coreutils-9.0/m4/wint_t.m4 and missing-macros/src/m4/wint_t.m4

Suggest solution:

add PKG_FIXUP := autoreconf to tools/coreutils/Makefile and $(curdir)/coreutils/compile := $(curdir)/automake/compile to tools/Makefile
Create a patch to rename gt_TYPE_WINT_T like https://github.com/openwrt/openwrt/blob/master/package/libs/gettext-full/patches/010-m4.patch or update the content of missing-macros.

neheb · 2022-03-19T04:33:35Z

I would have linked to https://github.com/openwrt/packages/blob/master/libs/gnutls/patches/010-m4.patch but sure.

graysky2 · 2022-03-19T07:18:25Z

Since there are two coreutils, I am confused how to prep the source dir in order to make a patch with quilt. For example, this command will prep coreutils from the packages repo but not the intended one locating in the openwrt repo/tools: make package/coreutils/{clean,prepare} V=s QUILT=1

I am following this wiki page.

graysky2 · 2022-03-19T19:29:15Z

@zhanhb - Made the patch and implemented your edits but still choked on macos.

zhanhb · 2022-03-20T01:02:04Z

File coreutils-9.0/m4/wint_t.m4 is not patched.
IMO, it's better to rename to gt_TYPE_WINT_T_COREUTILS rather than gt_TYPE_WINT_T_GNUTLS.

graysky2 · 2022-03-20T11:15:48Z

File coreutils-9.0/m4/wint_t.m4 is not patched.

Missed that, thanks.

@aparcar - Is the PR good to merge after the branch as-is ie with the 2nd commit pulling in a5eeac8

aparcar · 2022-03-20T13:43:45Z

Can you rebase this? The macOS patch should already be in master branch.

graysky2 · 2022-08-09T12:01:00Z

tools/Makefile lists coreutils twice. The second entry should be removed.

I removed that line but masos-latest still fails

Ansuel · 2022-10-06T14:22:09Z

News on this?

PolynomialDivision · 2022-10-23T13:17:23Z

Can you push -j1 V=s in another commit on the CI just to see the macos error?

graysky2 · 2022-12-19T18:08:23Z

Where are we at with this PR?

neheb · 2023-01-12T04:02:23Z

Needs a rebase.

graysky2 · 2023-01-13T17:07:32Z

macos test is still failing it seems :/

misaka36 · 2023-02-03T05:34:20Z

@graysky2 Please remove upstreamed patch: 001-chmod-fix_exit_status_when_ignoring_sylinks.patch.

graysky2 · 2023-02-03T08:32:20Z

@lovehackintosh - why do you request this? please see conversation string above for rationale to include.

misaka36 · 2023-02-03T08:47:25Z

@graysky2 I downloaded https://ftp.gnu.org/gnu/coreutils/coreutils-9.1.tar.xz and unpacked,
Then found that 001-chmod-fix_exit_status_when_ignoring_sylinks.patch has been applied.

graysky2 · 2023-02-03T09:14:12Z

@lovehackintosh - you are right, fixed

misaka36 · 2023-02-03T09:17:43Z

001-m4.patch requires refresh, please run make tools/coreutils/refresh.

graysky2 · 2023-02-03T11:23:44Z

@lovehackintosh

% make tools/coreutils/refresh V=s        
make[2]: Entering directory '/scratch/union/scripts/config'
make[2]: 'conf' is up to date.
make[2]: Leaving directory '/scratch/union/scripts/config'
make[1]: Entering directory '/scratch/union'
make[1]: *** No rule to make target 'tools/coreutils/refresh'.  Stop.
make[1]: Leaving directory '/scratch/union'
make: *** [/scratch/union/include/toplevel.mk:231: tools/coreutils/refresh] Error 2

misaka36 · 2023-02-03T12:08:39Z

Try to run echo "CONFIG_DEVEL=y\nCONFIG_BUILD_ALL_HOST_TOOLS=y" >> .config && make defconfig

graysky2 · 2023-02-03T12:18:01Z

Same error

misaka36 · 2023-02-03T12:24:00Z

I refreshed it on macOS, you just need to apply this patch
1.patch

graysky2 · 2023-02-03T12:26:53Z

The "dirty" patch must still cleanly apply or else the two checks that ran would have errored out, no?

https://github.com/openwrt/openwrt/actions/runs/4082720887/jobs/7037402090
https://github.com/openwrt/openwrt/actions/runs/4082720887/jobs/7037402262

misaka36 · 2023-02-03T12:33:11Z

What is the meaning of the two checks?

graysky2 · 2023-02-03T12:44:57Z

I am just pointing out that neither of these checks detected a dirty patch.

misaka36 · 2023-02-03T13:05:46Z

Applying patch 001-m4.patch
patching file m4/gnulib-comp.m4
Hunk #1 succeeded at 2671 (offset -12 lines).
patching file m4/stdint.m4
patching file m4/vasnprintf.m4
patching file m4/wchar_h.m4
patching file m4/wctype_h.m4
patching file m4/wcwidth.m4
patching file m4/wint_t.m4

I think we need to make the patch apply cleanly, so a refresh is required.

graysky2 · 2023-02-03T13:45:21Z

I applied it, but not sure why the build checks did not flag it. I am also not sure why I cannot refresh it myself.

misaka36 · 2023-02-04T12:14:29Z

If you want to refresh it on Linux, you need to run sed -i 66a\tools-y+=coreutils tools/Makefile to set coreutils as default tools.

misaka36 · 2023-02-05T12:29:10Z

@hauke Can you merge this?

In addition to version update, this commit applies a fixup to allow building on MacOS involving renaming: [gt_TYPE_WINT_T] --> [gt_TYPE_WINT_T_GNUTLS] suggested by zhanhb. Build system: x86_64 Build-tested: bcm2711/RPi4B Signed-off-by: John Audia <[email protected]>

graysky2 force-pushed the coreutils branch 2 times, most recently from 708e850 to b017602 Compare February 26, 2022 20:53

mans0n added the build/scripts/tools pull request/issues for build, scripts and tools related changes label Feb 27, 2022

graysky2 force-pushed the coreutils branch from b017602 to 46654c2 Compare March 1, 2022 10:24

graysky2 changed the title ~~coreutils: update to 9.0~~ tools/coreutils: update to 9.0 Mar 6, 2022

aparcar added the blocked pull request blocked by/waiting for another change label Mar 9, 2022

graysky2 force-pushed the coreutils branch from 46654c2 to ca83959 Compare March 9, 2022 18:09

graysky2 force-pushed the coreutils branch from ca83959 to 97ebdd6 Compare March 19, 2022 18:45

graysky2 force-pushed the coreutils branch 2 times, most recently from 669ade4 to adc3530 Compare March 20, 2022 11:14

aparcar removed the blocked pull request blocked by/waiting for another change label Mar 20, 2022

graysky2 force-pushed the coreutils branch from bf2ee50 to 784c0e4 Compare January 12, 2023 07:25

graysky2 force-pushed the coreutils branch from 784c0e4 to 807631d Compare February 3, 2023 09:13

graysky2 force-pushed the coreutils branch from 807631d to 2ea562c Compare February 3, 2023 13:44

hauke force-pushed the coreutils branch from 2ea562c to c560822 Compare February 5, 2023 20:57

openwrt-bot merged commit c560822 into openwrt:master Feb 5, 2023

graysky2 deleted the coreutils branch February 6, 2023 11:28

mcprat mentioned this pull request Mar 21, 2023

tools/coreutils: update to 9.2 #12233

Merged

Uh oh!

Conversation

graysky2 commented Feb 26, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

paper42 commented Feb 26, 2022

Uh oh!

neheb commented Feb 26, 2022

Uh oh!

graysky2 commented Feb 26, 2022

Uh oh!

aparcar commented Feb 27, 2022

Uh oh!

graysky2 commented Mar 1, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ynezz commented Mar 1, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

neheb commented Mar 1, 2022

Uh oh!

aparcar commented Mar 9, 2022

Uh oh!

aparcar commented Mar 9, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

graysky2 commented Mar 9, 2022

Uh oh!

aparcar commented Mar 9, 2022

Uh oh!

neheb commented Mar 9, 2022

Uh oh!

zhanhb commented Mar 19, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

neheb commented Mar 19, 2022

Uh oh!

graysky2 commented Mar 19, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

graysky2 commented Mar 19, 2022

Uh oh!

zhanhb commented Mar 20, 2022

Uh oh!

graysky2 commented Mar 20, 2022

Uh oh!

aparcar commented Mar 20, 2022

Uh oh!

graysky2 commented Aug 9, 2022

Uh oh!

Ansuel commented Oct 6, 2022

Uh oh!

PolynomialDivision commented Oct 23, 2022

Uh oh!

graysky2 commented Dec 19, 2022

Uh oh!

neheb commented Jan 12, 2023

Uh oh!

graysky2 commented Jan 13, 2023

Uh oh!

misaka36 commented Feb 3, 2023

Uh oh!

graysky2 commented Feb 3, 2023

Uh oh!

misaka36 commented Feb 3, 2023

Uh oh!

graysky2 commented Feb 3, 2023

Uh oh!

misaka36 commented Feb 3, 2023

Uh oh!

graysky2 commented Feb 3, 2023

Uh oh!

misaka36 commented Feb 3, 2023

Uh oh!

graysky2 commented Feb 3, 2023

Uh oh!

misaka36 commented Feb 3, 2023

Uh oh!

graysky2 commented Feb 3, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

graysky2 commented Feb 26, 2022 •

edited

Loading

graysky2 commented Mar 1, 2022 •

edited

Loading

ynezz commented Mar 1, 2022 •

edited

Loading

aparcar commented Mar 9, 2022 •

edited

Loading

zhanhb commented Mar 19, 2022 •

edited

Loading

graysky2 commented Mar 19, 2022 •

edited

Loading

graysky2 commented Feb 3, 2023 •

edited

Loading