make: centralize wget/curl & unzip/7z feature test by Kijewski · Pull Request #1431 · RIOT-OS/RIOT

Kijewski · 2014-07-15T22:46:46Z

With many open PRs that could benefit from loading SDKs when needed,
instead adding vast amounts of code to RIOTs master, this PR provides
the "functions" $(DOWNLOAD_TO_STDOUT), $(DOWNLOAD_TO_FILE), and
$(UNZIP_HERE).

The first "function" takes one argument, the URL from where to download
the content. It is then piped to stdout. To be used e.g. with tar xz.

The second "function" taken two arguments, the destination file name,
and the source URL. If the previous invocation was interrupted, then the
download gets continued, if possible.

The last "function" takes one argument, the source ZIP file. The file
gets extracted into the cwd, so best use this "function" with
cd $(SOME_WHERE) &&.

The clumsy name $(UNZIP_HERE) is taken because the program "unzip"
takes the environment variable UNZIP as the source file, even if
another file name was given on the command line. The rationale for that
is that the hackers of "unzip" hate their users. Also they sacrifice
hamsters to Satan.

LudwigKnuepfer · 2014-07-16T00:17:45Z

What kind of hamsters?

Kijewski · 2014-07-16T00:23:42Z

The cute kind of hamsters, these devils!

LudwigKnuepfer · 2014-07-16T00:31:36Z

Whaaat!?! That's too cute to bear.
Regarding the broken treatment of environment variables.. I assume they are unwilling to change their evil ways due to backwards compatibility, but did you try opening a ticket with them nonetheless? (I know this wont change the PR at hand.)

Kijewski · 2014-07-16T09:49:10Z

The behavior is kinda specified in the man page: http://www.unix.com/man-page/opensolaris/0/UNZIP/ (section "ENVIRONMENT OPTIONS")

Environment options are, in effect, considered to be just like any other command-line options, except that they are effectively the first options on the command line.

I guess change this behavior now would cause the maintainers too much trouble.

At least FreeBSD does not mention the environment variable in it's man page, but Linux, Solaris and DragonflyBSD do.

LudwigKnuepfer · 2014-07-16T09:52:53Z

OT: They could change the "API" to make it saner, that's what major releases are for ;-)

Kijewski · 2014-07-16T13:38:05Z

So … ACK?

Kijewski · 2014-07-18T16:49:33Z

Ping.

LudwigKnuepfer · 2014-07-21T09:32:49Z

Makefile.include

why isn't axel treated in the same manner as wget and curl (or the other way around)?

axel can't output to stdout.

I was thinking more about the if/else construct, but you could still give /dev/stdout as a file.. as long as axel does not spam around that is.

Nope, you can't. While downloading axel always creates a file with the same name + some extension, so you can continue an interrupted download.

Or do you mean

DOWNLOAD_TO_FILE := $(if $(AXEL),$(AXEL) -n 4 -q -a -o,$(if $(WGET),$(WGET) -nv -c -O,$(CURL) -s -o))

I think this is unreadable. But maybe

$(strip $(if $(AXEL), $(AXEL) -n 4 -q -a -o, $(if $(WGET), $(WGET) -nv -c -O, $(CURL) -s -o)))

?

The last one reflects the logic better I'd say.

While downloading axel always creates a file with the same name + some extension, so you can continue an interrupted download.

OK then.

Kijewski · 2014-07-22T12:06:50Z

@LudwigOrtmann please see my last commit

Kijewski · 2014-07-22T12:42:05Z

Nope, nesting $(if)s does not reliably work. See the history towards 887ed57 if you are interested in what I've tried …

LudwigKnuepfer · 2014-07-22T13:04:33Z

pkg/openwsn/Makefile

FETCH is not defined?

LudwigKnuepfer · 2014-07-23T05:16:46Z

I suggest you change the documentation around this along with the PR (i.e. pkg/Makefile.http).

Kijewski · 2014-07-24T11:21:15Z

Hm, I think I'll rewrite the pkg'ing description in the wiki soonish. I stumbled upon some gotchas that are not addressed in the examples.

LudwigKnuepfer · 2014-07-24T11:51:11Z

Is there some known test application for OpenWSN?

Kijewski · 2014-07-24T19:18:07Z

Nope, it's an outstanding issue: #1231.

Kijewski · 2014-07-29T16:32:58Z

I don't think we should wait for #1231 to be resolved. :)
@thomaseichinger, could you test this PR, please?

Kijewski · 2014-08-15T13:10:21Z

Ping. This PR was meant to make the work easier for the SWP people. I don't know if this is needed anymore after the month delay.

LudwigKnuepfer · 2014-08-15T15:16:41Z

@thomaseichinger ping

thomaseichinger · 2014-08-15T15:22:57Z

I know, I'm on it, but I'll have to "undo" all my additional setups to test this.

thomaseichinger · 2014-08-15T15:44:05Z

pkg/openwsn/Makefile

This line doesn't work, please change to

$(AD)$(UNZIP_HERE) $< -d $(PKG_NAME)-$(PKG_VERSION)

It creates $@

$(AD)cd $(CURDIR) && $(UNZIP_HERE) $<? I intended this command not to have any further parameters. E.g. -d is not understood by 7z.

The archive extracts to a not "pretty"/long name and if I use -d option in the package Makefile it's just an extension to your command and I don't need to change the accompanying scripts :-)

thomaseichinger · 2014-08-15T15:52:12Z

Tested for OS X and FreeBSD with wget/curl & unzip. I don't have any 7z installed.
ACK when above comment is addressed.

With many open PRs that could benefit from loading SDKs when needed, instead adding vast amounts of code to RIOTs master, this PR provides the "functions" `$(DOWNLOAD_TO_STDOUT)`, `$(DOWNLOAD_TO_FILE)`, and `$(UNZIP_HERE)`. The first "function" takes one argument, the URL from where to download the content. It is then piped to stdout. To be used e.g. with `tar xz`. The second "function" taken two arguments, the destination file name, and the source URL. If the previous invocation was interrupted, then the download gets continued, if possible. The last "function" takes one argument, the source ZIP file. The file gets extracted into the cwd, so best use this "function" with `cd $(SOME_WHERE) &&`. The clumsy name `$(UNZIP_HERE)` is taken because the program "unzip" takes the environment variable `UNZIP` as the source file, even if another file name was given on the command line. The rationale for that is that the hackers of "unzip" hate their users. Also they sacrifice hamsters to Satan.

Kijewski · 2014-08-21T17:08:38Z

@thomaseichinger ping.

thomaseichinger · 2014-08-21T17:15:25Z

ACK

make: centralize wget/curl & unzip/7z feature test

Kijewski added make labels Jul 15, 2014

Kijewski added this to the Release NEXT MAJOR milestone Jul 15, 2014

Kijewski assigned haukepetersen Jul 15, 2014

LudwigKnuepfer reviewed Jul 21, 2014
View reviewed changes

LudwigKnuepfer reviewed Jul 22, 2014
View reviewed changes

pkg/openwsn/Makefile Outdated

Copy link
Copy Markdown

Member

LudwigKnuepfer Jul 22, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FETCH is not defined?

Kijewski added the NEEDS SQUASHING label Jul 29, 2014

Kijewski removed the NEEDS SQUASHING label Aug 11, 2014

thomaseichinger reviewed Aug 15, 2014
View reviewed changes

Kijewski added a commit that referenced this pull request Aug 21, 2014

Merge pull request #1431 from Kijewski/wget-for-everyone-woooohooooo

b22c2f4

make: centralize wget/curl & unzip/7z feature test

Kijewski merged commit b22c2f4 into RIOT-OS:master Aug 21, 2014

Kijewski deleted the wget-for-everyone-woooohooooo branch August 21, 2014 19:23

crasbe mentioned this pull request Oct 29, 2025

makefiles: remove DOWNLOAD_TO_STDOUT #21823

Merged

Conversation

Kijewski commented Jul 15, 2014

Uh oh!

LudwigKnuepfer commented Jul 16, 2014

Uh oh!

Kijewski commented Jul 16, 2014

Uh oh!

LudwigKnuepfer commented Jul 16, 2014

Uh oh!

Kijewski commented Jul 16, 2014

Uh oh!

LudwigKnuepfer commented Jul 16, 2014

Uh oh!

Kijewski commented Jul 16, 2014

Uh oh!

Kijewski commented Jul 18, 2014

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Kijewski commented Jul 22, 2014

Uh oh!

Kijewski commented Jul 22, 2014

Uh oh!

Choose a reason for hiding this comment

Uh oh!

LudwigKnuepfer commented Jul 23, 2014

Uh oh!

Kijewski commented Jul 24, 2014

Uh oh!

LudwigKnuepfer commented Jul 24, 2014

Uh oh!

Kijewski commented Jul 24, 2014

Uh oh!

Kijewski commented Jul 29, 2014

Uh oh!

Kijewski commented Aug 15, 2014

Uh oh!

LudwigKnuepfer commented Aug 15, 2014

Uh oh!

thomaseichinger commented Aug 15, 2014

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

thomaseichinger commented Aug 15, 2014

Uh oh!

Kijewski commented Aug 21, 2014

Uh oh!

thomaseichinger commented Aug 21, 2014

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants