[cling] Rewrite implementation of `InputValidator::validate()`; fixes ROOT-9202 by jalopezg-git · Pull Request #8771 · root-project/root

jalopezg-git · 2021-07-29T14:36:31Z

This pull request replaces the implementation of InputValidator::validate() by simpler, more maintainable code that also fixes JIRA issue ROOT-9202.

The previous implementation was unable to properly handle line continuation after ',' or '\'. Specifically, it skipped over non-punctuation tokens, entering continuation mode even if ',' or '' were not the last tokens in the input, e.g.

int a, b

or

int a \      b

caused cling to request more input, where it shouldn't.

Changes or fixes:

MetaLexer:
- Return /* and */ as independent tokens.
- Added ReadToEndOfLine() function (consume input until '\r', '\n', or EOF).
- Added MetaLexer::RAII that saves the current lexing position and restores it on destruction.
- Remove unused LexPunctuatorAndAdvance().
Rewrite of InputValidator::validate().

Checklist:

tested changes locally

Fixes ROOT-9202.

phsft-bot · 2021-07-29T14:36:39Z

Starting build on ROOT-debian10-i386/cxx14, ROOT-performance-centos8-multicore/default, ROOT-ubuntu16/nortcxxmod, mac1014/python3, mac11.0/cxx17, windows10/cxx14
How to customize builds

jalopezg-git · 2021-07-29T14:53:10Z

@Axel-Naumann, @vgvassilev, as I will be off in August, again feel free to merge this if, after the review, it can be merged right away.

phsft-bot · 2021-07-29T15:16:52Z

Build failed on ROOT-debian10-i386/cxx14.
Running on pcepsft10.dyndns.cern.ch:/build/workspace/root-pullrequests-build
See console output.

Errors:

[2021-07-29T14:38:29.932Z] stderr: error: could not read '.git/rebase-apply/head-name': No such file or directory

Failing tests:

projectroot.roottest.python.cpp.roottest_python_cpp_cpp

interpreter/cling/include/cling/MetaProcessor/MetaLexer.h

interpreter/cling/lib/MetaProcessor/InputValidator.cpp

phsft-bot · 2021-07-30T09:44:33Z

Starting build on ROOT-debian10-i386/cxx14, ROOT-performance-centos8-multicore/default, ROOT-ubuntu16/nortcxxmod, mac1014/python3, mac11.0/cxx17, windows10/cxx14
How to customize builds

phsft-bot · 2021-07-30T10:24:31Z

Build failed on ROOT-debian10-i386/cxx14.
Running on pcepsft10.dyndns.cern.ch:/build/workspace/root-pullrequests-build
See console output.

Errors:

[2021-07-30T09:46:24.300Z] stderr: error: could not read '.git/rebase-apply/head-name': No such file or directory

Failing tests:

projectroot.roottest.python.cpp.roottest_python_cpp_cpp

vgvassilev

Thanks for working on this!

interpreter/cling/include/cling/MetaProcessor/MetaLexer.h

interpreter/cling/lib/MetaProcessor/InputValidator.cpp

phsft-bot · 2021-07-31T09:21:36Z

Starting build on ROOT-debian10-i386/cxx14, ROOT-performance-centos8-multicore/default, ROOT-ubuntu16/nortcxxmod, mac1014/python3, mac11.0/cxx17, windows10/cxx14
How to customize builds

…Line()` This commit includes the following changes to MetaLexer: - Update `MetaLexer::Lex()` to return `/*` (tok::l_comment), `*/` (tok::r_comment), and `<` (tok::less) as tokens. - Added `MetaLexer::ReadToEndOfLine()` function: consume input until `\r` or `\n`. - Added `MetaLexer::RAII`, a RAII object that saves/restores the current position.

Replace implementation of `InputValidator::validate()` by simpler, more maintainable code that also fixes ROOT-9202. The previous implementation was unable to properly handle line continuation after ',' or '\'. Specifically, it skipped over non-punctuation tokens, entering continuation mode even if ',' or '\' were not the last tokens in the input, e.g. `int a, b` or 'int a \ b'. Fixes ROOT-9202.

phsft-bot · 2021-07-31T09:34:45Z

Starting build on ROOT-debian10-i386/cxx14, ROOT-performance-centos8-multicore/default, ROOT-ubuntu16/nortcxxmod, mac1014/python3, mac11.0/cxx17, windows10/cxx14
How to customize builds

phsft-bot · 2021-07-31T10:13:24Z

Build failed on ROOT-debian10-i386/cxx14.
Running on pcepsft10.dyndns.cern.ch:/build/workspace/root-pullrequests-build
See console output.

Errors:

[2021-07-31T09:35:16.144Z] stderr: error: could not read '.git/rebase-apply/head-name': No such file or directory

Failing tests:

projectroot.roottest.python.cpp.roottest_python_cpp_cpp

Axel-Naumann

Thanks!

jalopezg-git · 2021-09-08T10:27:47Z

@phsft-bot build

phsft-bot · 2021-09-08T10:27:53Z

Starting build on ROOT-debian10-i386/cxx14, ROOT-performance-centos8-multicore/default, ROOT-ubuntu16/nortcxxmod, mac1014/python3, mac11.0/cxx17, windows10/cxx14
How to customize builds

phsft-bot · 2021-09-08T11:51:31Z

Build failed on mac11.0/cxx17.
Running on macphsft23.dyndns.cern.ch:/Users/sftnight/build/workspace/root-pullrequests-build
See console output.

Failing tests:

jalopezg-git · 2021-09-08T14:22:35Z

@phsft-bot build just on mac11.0/cxx17

phsft-bot · 2021-09-08T14:22:47Z

Starting build on mac11.0/cxx17
How to customize builds

jalopezg-git added bug affects:master in:Cling labels Jul 29, 2021

jalopezg-git requested review from Axel-Naumann, hahnjo and vgvassilev July 29, 2021 14:36

jalopezg-git self-assigned this Jul 29, 2021

hahnjo reviewed Jul 30, 2021

View reviewed changes

interpreter/cling/include/cling/MetaProcessor/MetaLexer.h Outdated Show resolved Hide resolved

interpreter/cling/lib/MetaProcessor/InputValidator.cpp Outdated Show resolved Hide resolved

interpreter/cling/lib/MetaProcessor/InputValidator.cpp Outdated Show resolved Hide resolved

jalopezg-git force-pushed the cling-rewrite-inputvalidator branch from da769b9 to 30b1783 Compare July 30, 2021 09:44

jalopezg-git mentioned this pull request Jul 30, 2021

[Rint] Correctly handle line continuation after '\' #8772

Merged

1 task

vgvassilev reviewed Jul 30, 2021

View reviewed changes

interpreter/cling/include/cling/MetaProcessor/MetaLexer.h Outdated Show resolved Hide resolved

interpreter/cling/lib/MetaProcessor/InputValidator.cpp Show resolved Hide resolved

jalopezg-git force-pushed the cling-rewrite-inputvalidator branch from 30b1783 to ccbf00d Compare July 31, 2021 09:21

jalopezg-git added 4 commits July 31, 2021 11:34

[cling] Cleanup MetaLexer; remove unused LexPunctuatorAndAdvance()

b1e7e4c

[cling] InputValidator::validate(): add test against ROOT-9202

79653e4

jalopezg-git force-pushed the cling-rewrite-inputvalidator branch from ccbf00d to 79653e4 Compare July 31, 2021 09:34

Axel-Naumann approved these changes Sep 6, 2021

View reviewed changes

jalopezg-git closed this Sep 8, 2021

jalopezg-git reopened this Sep 8, 2021

jalopezg-git merged commit a6984fd into root-project:master Sep 8, 2021

jalopezg-git deleted the cling-rewrite-inputvalidator branch September 8, 2021 17:15

Conversation

jalopezg-git commented Jul 29, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes or fixes:

Checklist:

Uh oh!

phsft-bot commented Jul 29, 2021

Uh oh!

jalopezg-git commented Jul 29, 2021

Uh oh!

phsft-bot commented Jul 29, 2021

Errors:

Failing tests:

Uh oh!

Uh oh!

Uh oh!

Uh oh!

phsft-bot commented Jul 30, 2021

Uh oh!

phsft-bot commented Jul 30, 2021

Errors:

Failing tests:

Uh oh!

vgvassilev left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

phsft-bot commented Jul 31, 2021

Uh oh!

phsft-bot commented Jul 31, 2021

Uh oh!

phsft-bot commented Jul 31, 2021

Errors:

Failing tests:

Uh oh!

Axel-Naumann left a comment

Choose a reason for hiding this comment

Uh oh!

jalopezg-git commented Sep 8, 2021

Uh oh!

phsft-bot commented Sep 8, 2021

Uh oh!

phsft-bot commented Sep 8, 2021

Failing tests:

Uh oh!

jalopezg-git commented Sep 8, 2021

Uh oh!

phsft-bot commented Sep 8, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

jalopezg-git commented Jul 29, 2021 •

edited

Loading