Skip to content

style-check/vera++: add style validator and scripts#9778

Merged
smlng merged 4 commits intoRIOT-OS:masterfrom
jia200x:pr/vera++
Feb 18, 2020
Merged

style-check/vera++: add style validator and scripts#9778
smlng merged 4 commits intoRIOT-OS:masterfrom
jia200x:pr/vera++

Conversation

@jia200x
Copy link
Copy Markdown
Member

@jia200x jia200x commented Aug 14, 2018

Contribution description

This PR adds configuration files and a check script for using Vera++ in RIOT. This tool does C/C++ style checks with a given set of rules. The difference to other tools (like Uncrustify) is the fact this is not a code beautifier but a code validator.

New rules can be added in TCL, Python or LUA. The tool defines a "profile" which consists in a set of rules to be applied and some rule-specific parameters. A default riot profile was added.

This PR also adds a exclude file for excluding source files based on regular expressions (see --exclusions command here)

The riot profile enables all rules except:

  • F002 File names should be well-formed
  • T002 Reserved names should not be used for preprocessor macros
  • T011 Curly brackets from the same pair should be either in the same line or in the same column
  • T012 Negation operator should not be used in its short form
  • T014 Source files should refer the Boost Software License
  • T016 Calls to min/max should be protected against accidental macro substitution

And also defines some parameters for certain rules:

  • L004: parameter=max-line-length=80
  • L005: parameter=max-consecutive-empty-lines=1

I also enabled "T008 Keywords catch, for, if, switch and while should be followed by a single space" but not sure if it's intended.

All feedback is more than welcome!

Check README.md file for more details.

Test procedure

Run dist/tools/vera++/check.sh

Issues/PRs references

#8519 (which could be dropped if this one gets merged)

@jia200x jia200x requested review from aabadie and smlng August 14, 2018 14:17
@jia200x jia200x added Type: enhancement The issue suggests enhanceable parts / The PR enhances parts of the codebase / documentation Discussion: RFC The issue/PR is used as a discussion starting point about the item of the issue/PR Area: CI Area: Continuous Integration of RIOT components labels Aug 14, 2018
@jia200x
Copy link
Copy Markdown
Member Author

jia200x commented Aug 14, 2018

This is how it looks

@jia200x jia200x added the CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR label Aug 15, 2018
@miri64
Copy link
Copy Markdown
Member

miri64 commented Aug 15, 2018

From how I interpret the coding conventions, line length >80 should be a warning, >100 should be an error. Is that possible to implement with Vera++?

@jia200x
Copy link
Copy Markdown
Member Author

jia200x commented Aug 15, 2018

From how I interpret the coding conventions, line length >80 should be a warning, >100 should be an error. Is that possible to implement with Vera++?

You are right. This is not possible directly, but can be solved adding a riot-force profile (that contains critical stuff like line length > 80) and keep the riot profile with warnings.

Then, it's required to execute vera++ 2 times (one with warnings and the other one with errors). What do you think?

@kaspar030
Copy link
Copy Markdown
Contributor

So vera complains but cannot not reformat automatically?

@jia200x
Copy link
Copy Markdown
Member Author

jia200x commented Aug 15, 2018

So vera complains but cannot not reformat automatically?

Yes it can. There are some default transformations we might use (trailing whitespaces, etc)

@kaspar030
Copy link
Copy Markdown
Contributor

IMO uncrustify was a good idea. See #8519 (comment).

@jcarrano
Copy link
Copy Markdown
Contributor

@jia200x @miri64 If we are willing to modify the TCL files, the warning on line length >80 and throwing an error on length > 100 should be perfectly possible.

Vera has only one "report" function, and it's up to the user to define a convention for report lines (something like Warning:/Error: etc). Then we can have two config values and have the script emit either a Warning or and Error.

Also, if I understand correctly, each script is run independently, meaning each file is traversed multiple times?

@jia200x I suggest we include a few example configurations for popular editors, so that people can run Vera/Uncrustify while they are editing the file. It should encourage people to not wait till the last moment to fix the source.

@jia200x
Copy link
Copy Markdown
Member Author

jia200x commented Aug 20, 2018

@jia200x @miri64 If we are willing to modify the TCL files, the warning on line length >80 and throwing an error on length > 100 should be perfectly possible.

Vera has only one "report" function, and it's up to the user to define a convention for report lines (something like Warning:/Error: etc). Then we can have two config values and have the script emit either a Warning or and Error.

True

Also, if I understand correctly, each script is run independently, meaning each file is traversed multiple times?

Not sure, but tend to think yes. I will check.

@jia200x I suggest we include a few example configurations for popular editors, so that people can run Vera/Uncrustify while they are editing the file. It should encourage people to not wait till the last moment to fix the source.

That's a nice idea. It's like what some editors do with PEP8 in Python. Pointing out errors is exactly a Vera++ job, because Uncrustify can only show the diff and not the reason (e.g line 294: extra empty line). However, it's possible to configure these editors to run Uncrustify with a user defined command.

@jia200x
Copy link
Copy Markdown
Member Author

jia200x commented Aug 20, 2018

Also, if I understand correctly, each script is run independently, meaning each file is traversed multiple times?

On second thoughts, I think it parses/tokenizes first and then run all scripts at token level.

@jcarrano
Copy link
Copy Markdown
Contributor

However, it's possible to configure these editors to run Uncrustify with a user defined command.

My point is, if vim is so cool as it's followers claim it should be 100% possible to run Vera in "lint mode" (i.e. visually mark lines that are problematic). I don't use vim so I don't know.

@jia200x
Copy link
Copy Markdown
Member Author

jia200x commented Aug 20, 2018

My point is, if vim is so cool as it's followers claim it should be 100% possible to run Vera in "lint mode" (i.e. visually mark lines that are problematic). I don't use vim so I don't know.

Yes, it's possible

@miri64
Copy link
Copy Markdown
Member

miri64 commented Aug 20, 2018

My point is, if vim is so cool as it's followers claim it should be 100% possible to run Vera in "lint mode" (i.e. visually mark lines that are problematic). I don't use vim so I don't know.

Should be possible, and I imagine, that popular linter-plugins like syntastic already provide a config for vera.

@jcarrano
Copy link
Copy Markdown
Contributor

From the vera README:

Vera++ is mainly an engine that parses C++ source files ....

Looking at the API, though, this statement may be a bit exaggerated. AFAICT vera only tokenizes the files. Any parsing is done by the scripts with ad-hoc state machines (like

)

@jia200x
Copy link
Copy Markdown
Member Author

jia200x commented Aug 20, 2018

Looking at the API, though, this statement may be a bit exaggerated. AFAICT vera only tokenizes the files. Any parsing is done by the scripts with ad-hoc state machines (like

Yes, I think they refer to "interpreting" the C++ code into tokens. It's similar to what Uncrustify does

@cladmi
Copy link
Copy Markdown
Contributor

cladmi commented Oct 11, 2018

@jia200x
Copy link
Copy Markdown
Member Author

jia200x commented Oct 11, 2018

@cladmi I forgot to add it. Just updated it in the description.

For testing, run dist/tools/vera++/check.sh

@jia200x
Copy link
Copy Markdown
Member Author

jia200x commented Jul 5, 2019

I pushed the v1.2.1 files within the last commit, so it's compatible with the binary package shipped in Ubuntu

@jia200x
Copy link
Copy Markdown
Member Author

jia200x commented Dec 9, 2019

@smlng @aabadie @kaspar030 I just fixed an error with the check script and now static tests pass.
Can someone review again?

In theory this would be ready to be merged and would finally add some exhaustive lint checks to our C code.

@jia200x
Copy link
Copy Markdown
Member Author

jia200x commented Dec 16, 2019

ping

#!/usr/bin/tclsh

set rules {
L004
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is also in the force profile, to me it seems like either this or the other profile but not both, right?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah sorry, just saw that this goes with the parameter below, so please ignore

Copy link
Copy Markdown
Member

@smlng smlng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to me this looks good, and any additional auto check can only help.

one minor note: in the profile file one cannot infer what a rule does just by its number, e.g. L004, one always needs to lookup the rule file and see code and comments there. Maybe add short comment in the profile which rules are active, e.g.

    L004 # max line length

But that's not a blocker, just something I found

@jia200x
Copy link
Copy Markdown
Member Author

jia200x commented Dec 17, 2019

one minor note: in the profile file one cannot infer what a rule does just by its number, e.g. L004, one always needs to lookup the rule file and see code and comments there. Maybe add short comment in the profile which rules are active, e.g.

It makes sense. I will

@jia200x
Copy link
Copy Markdown
Member Author

jia200x commented Dec 20, 2019

@smlng done!
I also fixed added some param files (required by the old vera++)

@jia200x
Copy link
Copy Markdown
Member Author

jia200x commented Dec 20, 2019

(btw I couldn't add inline comments. They seems to be invalid in the TCL parser :( )

@jia200x
Copy link
Copy Markdown
Member Author

jia200x commented Jan 16, 2020

@smlng may I rebase?

@smlng
Copy link
Copy Markdown
Member

smlng commented Jan 17, 2020

yes please rebase (and squash if needed)

@jia200x
Copy link
Copy Markdown
Member Author

jia200x commented Jan 27, 2020

rebased and squashed!

@jia200x jia200x added the CI: skip compile test If set, CI server will run only non-compile jobs, but no compile jobs or their dependent jobs label Feb 14, 2020
@jia200x
Copy link
Copy Markdown
Member Author

jia200x commented Feb 14, 2020

@smlng all test passed :)

@smlng smlng added CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR and removed CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR labels Feb 17, 2020
@jia200x
Copy link
Copy Markdown
Member Author

jia200x commented Feb 17, 2020

thanks! Can someone merge then? :)

@smlng smlng merged commit 2aaf811 into RIOT-OS:master Feb 18, 2020
@smlng
Copy link
Copy Markdown
Member

smlng commented Feb 18, 2020

done

@jia200x
Copy link
Copy Markdown
Member Author

jia200x commented Feb 18, 2020

thanks!

@benpicco
Copy link
Copy Markdown
Contributor

Hm did you check how many errors vera++ reports for the existing code?
It seems like new PRs are now required to clean up existing code if they touch files with style errors.

@jia200x
Copy link
Copy Markdown
Member Author

jia200x commented Feb 18, 2020

Hm did you check how many errors vera++ reports for the existing code?
It seems like new PRs are now required to clean up existing code if they touch files with style errors.

Hmmm if it's being a problem, maybe we can try to set them to warnings and fix those errors in parallel? Then we can re-enable Vera++ errors

@leandrolanzieri leandrolanzieri added this to the Release 2020.04 milestone Feb 20, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area: CI Area: Continuous Integration of RIOT components CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR CI: skip compile test If set, CI server will run only non-compile jobs, but no compile jobs or their dependent jobs Discussion: RFC The issue/PR is used as a discussion starting point about the item of the issue/PR Type: enhancement The issue suggests enhanceable parts / The PR enhances parts of the codebase / documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants