Skip to content

makefiles: add code generator targets#8573

Merged
fjmolinas merged 1 commit intoRIOT-OS:masterfrom
aabadie:pr/generator
May 20, 2020
Merged

makefiles: add code generator targets#8573
fjmolinas merged 1 commit intoRIOT-OS:masterfrom
aabadie:pr/generator

Conversation

@aabadie
Copy link
Copy Markdown
Contributor

@aabadie aabadie commented Feb 16, 2018

Contribution description

This PR adds 3 new code bootstrap targets (bootstrap-{example,board,test}) in the build system. The feature relies on riotgen, a small Python tool I wrote to generate empty RIOT code projects.

The idea is to use these targets when one wants to start a new example application, test application or board support: all copyright headers are correctly setup, with main.c files, README, etc. This should simplify initial boring work and improve contribution quality of newcomers.

Just call them using make and answer a few questions:

$ make bootstrap-example
Example application name (no space): new-example
Example application brief description []: This is a new example application
Target board [native]: 
Required modules (comma separated) []: xtimer,fmt
Required packages (comma separated) []: 
Required board features (comma separated) []: uart
Author name [Alexandre Abadie]: 
Author email [[email protected]]: 
Organization [Alexandre Abadie]: Inria
Application 'new-example' generated with success!

Since I'm not a Makefile expert, comments are very welcome.

Hope people will find this useful.

Follow-up potential work:

  • add notes about that in contributing.md
  • add other bootstrap targets (package, sys module, driver, etc)

Issues/PRs references

None

@aabadie aabadie added Type: enhancement The issue suggests enhanceable parts / The PR enhances parts of the codebase / documentation Area: build system Area: Build system Type: new feature The issue requests / The PR implemements a new feature for RIOT CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR Discussion: RFC The issue/PR is used as a discussion starting point about the item of the issue/PR labels Feb 16, 2018
@aabadie
Copy link
Copy Markdown
Contributor Author

aabadie commented Feb 16, 2018

Might be something of interest for you @jia200x and @basilfx

@smlng
Copy link
Copy Markdown
Member

smlng commented Feb 16, 2018

Question (maybe I miss something): what's the benefit of these make targets? IIRC instead of calling make bootstrap-example I could just run riotgen example, right - which is even shorter.

@aabadie
Copy link
Copy Markdown
Contributor Author

aabadie commented Feb 16, 2018

what's the benefit of these make targets?

RIOTBASE is set automatically, so the the question about this parameter is skipped in the cli wizard.

@aabadie
Copy link
Copy Markdown
Contributor Author

aabadie commented Feb 16, 2018

Using riotgen directly, you would need: RIOTBASE=<riot base> riotgen example

@basilfx
Copy link
Copy Markdown
Member

basilfx commented Feb 17, 2018

I gave it a try, but it doesn't properly escape output for some reason:

basilfx:RIOT_aabadie/ (pr/generator) $ make bootstrap-example
-e -n \033[1;31m"riotgen" command is not available, please
-e -n \033[1;31mconsider installing it from
-e \033[1;31mhttps://pypi.python.org/pypi/riotgen\033[0m
make: *** [riotgen-installed] Error 1

(using zsh on macOS 10.13, normal RIOT-OS errors are colored.)

@jia200x
Copy link
Copy Markdown
Member

jia200x commented Feb 19, 2018

@aabadie awesome!

Question (maybe I miss something): what's the benefit of these make targets? IIRC instead of calling make bootstrap-example I could just run riotgen example, right - which is even shorter.

In addition, the riotgen script might live in dist/tools. So, a make target could invoke the tool with all required env variables.

@aabadie
Copy link
Copy Markdown
Contributor Author

aabadie commented Feb 19, 2018

the riotgen script might live in dist/tools

I'm not sure it's a good idea. riotgen is not only a script, but several, and it also contains template files. That's why I tried to create a self contained python package: it's easy to install and won't bloat the RIOT code base.

a make target could invoke the tool with all required env variables.

In any case, riotgen will have to be modified to support extra env variables (BOARD?)

@aabadie
Copy link
Copy Markdown
Contributor Author

aabadie commented Feb 19, 2018

I gave it a try, but it doesn't properly escape output for some reason

apparently the color.inc.mk is not working as I expected, I'll look into it. And it's not used somewhere else btw. (yes it is in Makefile.include)

@@ -0,0 +1,17 @@
.PHONY: riotgen-installed bootstrap-%
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO "bootstrap" is misleading

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What would you propose instead ? init-%, generate-% ?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe generate-%? or start-%?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 generate propositions, let's go for this one

$(GENERATORS): %: bootstrap-%

bootstrap-%: riotgen-installed
@RIOTBASE=. riotgen $*
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why "RIOTBASE=."?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why "RIOTBASE=."?

because otherwise there will be a question for setting the riotbase directory. Using this environment variable skips this.

@jia200x
Copy link
Copy Markdown
Member

jia200x commented Feb 22, 2018

I'm not sure it's a good idea. riotgen is not only a script, but several, and it also contains template files. That's why I tried to create a self contained python package: it's easy to install and won't bloat the RIOT code base.

And maybe cloning from RIOT repo? Probably not a big improvement to have it in dist/tools, but could allow to "bootstrap-%" without worrying about installing external tools :)

@jia200x
Copy link
Copy Markdown
Member

jia200x commented Feb 22, 2018

I confirm:

  • bootstrap-{test, example, board} work, each case as expected (proper files, descriptions are OK, etc)
  • riotgen undetected works as expected with colored output

@jia200x
Copy link
Copy Markdown
Member

jia200x commented Feb 22, 2018

BTW, code looks good to me

@aabadie aabadie added the Community: Hack'n'ACK candidate This PR is a candidate for review and discussion during one of RIOT's monthly Hack'n'ACK parties label Feb 27, 2018
@aabadie aabadie changed the title makefiles: add bootstrap targets makefiles: add code generator targets Feb 27, 2018
@aabadie aabadie force-pushed the pr/generator branch 4 times, most recently from bc7fed4 to 614ea94 Compare March 4, 2018 18:14
@aabadie
Copy link
Copy Markdown
Contributor Author

aabadie commented Mar 5, 2018

@jia200x I updated the makefile to include recents changes in the upstream riot-generator (pkg generator added) and improved the targets : make auto-completion is now supported for the new targets.

@aabadie
Copy link
Copy Markdown
Contributor Author

aabadie commented May 22, 2018

any interest for this ?

@aabadie
Copy link
Copy Markdown
Contributor Author

aabadie commented Nov 13, 2018

up

@stale
Copy link
Copy Markdown

stale bot commented Apr 18, 2020

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you want me to ignore this issue, please mark it with the "State: don't stale" label. Thank you for your contributions.

@stale stale bot added the State: stale State: The issue / PR has no activity for >185 days label Apr 18, 2020
@aabadie
Copy link
Copy Markdown
Contributor Author

aabadie commented Apr 18, 2020

Quite an old one indeed.

@stale stale bot removed the State: stale State: The issue / PR has no activity for >185 days label Apr 18, 2020
@fjmolinas
Copy link
Copy Markdown
Contributor

Testing it now @aabadie looks quite nice, it seems like it could use an entry in the docs!

@fjmolinas
Copy link
Copy Markdown
Contributor

Tested all commands inside RIOTBASE and outside RIOTBASE, works fine on linux, I'll test on MAC now.

@fjmolinas
Copy link
Copy Markdown
Contributor

Tested on MAC as well:

→ make generate-example
Example application name: test
Example application brief description: asdfasd
Target board [native]: 
Required modules (comma separated) []: 
Required packages (comma separated) []: 
Required features (comma separated) []: 
Author name []: Francisco
Author email []: [email protected]
Organization []: Inria
Example 'test' generated in examples/test with success!

@fjmolinas
Copy link
Copy Markdown
Contributor

@aabadie please correct me if I'm wrong, but as I understand the main benefit to have a make target
is having RIOTBASE set right?

Other I like this tool but I would like it documented somewhere in out doc, maybe in getting started?

@kaspar030 you have some change requests from a while back that @aabadie seems to have answered, are you fine with the changes?

Note: I haven't looked at the python code, but this doesn't seem relevant here, the functionality is what we want here.

@fjmolinas fjmolinas self-assigned this May 12, 2020
@fjmolinas fjmolinas self-requested a review May 12, 2020 08:42
@kaspar030 kaspar030 dismissed their stale review May 12, 2020 08:50

comments addressed

@kaspar030
Copy link
Copy Markdown
Contributor

Note: I haven't looked at the python code, but this doesn't seem relevant here, the functionality is what we want here.

Can I quote you on that? ;)

@fjmolinas
Copy link
Copy Markdown
Contributor

Can I quote you on that? ;)

I guess, we are not reviewing external code here. If the project is merged into the organization then we can review it IMO.

@aabadie
Copy link
Copy Markdown
Contributor Author

aabadie commented May 12, 2020

I understand the main benefit to have a make target is having RIOTBASE set right?

Yes and the fact that is can bootstrap a skeleton application/pkg/board/test from RIOT itself.

@aabadie
Copy link
Copy Markdown
Contributor Author

aabadie commented May 12, 2020

I like this tool but I would like it documented somewhere in out doc, maybe in getting started?

Good idea, I'll push a commit asap.

I haven't looked at the python code, but this doesn't seem relevant here, the functionality is what we want here.

I improved it slightly during the last days: now it uses Jinja2 templating engine to render the files (I should have done that from the start...). There's still room for improvement/refactoring and I'm now in the process of adding tests.

we are not reviewing external code here. If the project is merged into the organization then we can review it IMO.

If the community is OK to maintain such a tool in a separate repository, that would be great indeed.

@aabadie
Copy link
Copy Markdown
Contributor Author

aabadie commented May 15, 2020

I would like it documented somewhere in out doc, maybe in getting started?

I can propose something. Is it ok if it's in a follow-up PR ?

These targets will bootstrap code for an example application, a test application or a new board support. The 'riotgen' tool is used behind
@aabadie
Copy link
Copy Markdown
Contributor Author

aabadie commented May 17, 2020

The latest version of riotgen is now also able to generate the source code of a driver and of a module. I adapted the PR accordingly.

@fjmolinas
Copy link
Copy Markdown
Contributor

I can propose something. Is it ok if it's in a follow-up PR ?

If you open it right away :)

@fjmolinas
Copy link
Copy Markdown
Contributor

If you open it right away :)

(so we can nitpick there, while getting this in)

@aabadie
Copy link
Copy Markdown
Contributor Author

aabadie commented May 19, 2020

here is the documentation @fjmolinas => #14103

@fjmolinas
Copy link
Copy Markdown
Contributor

ACK!

@fjmolinas fjmolinas merged commit 99e7b81 into RIOT-OS:master May 20, 2020
@aabadie aabadie deleted the pr/generator branch May 20, 2020 14:00
@aabadie
Copy link
Copy Markdown
Contributor Author

aabadie commented May 20, 2020

Thank you for the review @fjmolinas !

@miri64 miri64 added this to the Release 2020.07 milestone Jun 24, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area: build system Area: Build system CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR Discussion: RFC The issue/PR is used as a discussion starting point about the item of the issue/PR Type: enhancement The issue suggests enhanceable parts / The PR enhances parts of the codebase / documentation Type: new feature The issue requests / The PR implemements a new feature for RIOT

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants