Skip to content

Adding experimental online parser#2770

Merged
IvanGoncharov merged 5 commits intographql:masterfrom
0xnmn:advanced-parser
Sep 17, 2020
Merged

Adding experimental online parser#2770
IvanGoncharov merged 5 commits intographql:masterfrom
0xnmn:advanced-parser

Conversation

@0xnmn
Copy link
Copy Markdown
Contributor

@0xnmn 0xnmn commented Aug 29, 2020

This PR includes implementation of an experimental online parser based on JSON rules set. This parser can be used with IDEs to implement syntax highlighting and auto suggestions. It is a state-full parser which parser the source string incrementally i.e. emits the next token each time.

@0xnmn 0xnmn force-pushed the advanced-parser branch 2 times, most recently from a51166b to 608c3a7 Compare August 29, 2020 22:19
@IvanGoncharov IvanGoncharov added the PR: feature 🚀 requires increase of "minor" version number label Aug 29, 2020
Comment thread src/language/online-parser/index.d.ts Outdated
@@ -0,0 +1,2 @@
export { Parser } from './parser';
export * from './types';
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please rename the folder to experimentalOnlineParser?

Comment thread src/language/online-parser/types.d.ts Outdated
export interface LanguageType {
rules: Rules;
}
export interface Rules {
Copy link
Copy Markdown
Member

@IvanGoncharov IvanGoncharov Aug 30, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add GraphQLGrammar prefix to types for JSON ruleset?

Comment thread src/language/online-parser/types.d.ts Outdated
export declare type Styles = {
[name: string]: string;
};
export declare type ParserConfig = {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For types related to parser can you please use OnlineParser prefix?

Comment thread src/language/online-parser/types.d.ts Outdated
export declare type ParserConfig = {
tabSize: number;
};
export declare type ParserConfigOption = {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please move Parser specific types into onlineParser.js?
and rename it to grammarTypes?

Comment thread src/language/online-parser/index.js Outdated
@@ -0,0 +1,2 @@
export { Parser } from './parser';
export * from './types';
Copy link
Copy Markdown
Member

@IvanGoncharov IvanGoncharov Aug 30, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use explicit exports and expose only parser related types.

Comment thread src/language/online-parser/language.js Outdated
@@ -0,0 +1 @@
export { default } from './rules';
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so just remove this file

Comment thread src/language/online-parser/parser.js Outdated
export class Parser {
state: ParserState;
lexer: Lexer;
styles: Styles;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please move it to GraphiQL code?

Comment thread src/language/online-parser/parser.d.ts Outdated
state: ParserState;
lexer: Lexer;
styles: Styles;
config: ParserConfig;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
config: ParserConfig;
_config: ParserConfig;

Comment thread src/language/online-parser/parser.d.ts Outdated

export declare class Parser {
state: ParserState;
lexer: Lexer;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
lexer: Lexer;
_lexer: Lexer;

Comment thread src/language/online-parser/parser.d.ts Outdated
} from './types';

export declare class Parser {
state: ParserState;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
state: ParserState;
_state: ParserState;

Comment thread src/language/online-parser/parser.d.ts Outdated
sol(): boolean;
parseToken(): Token;
indentation(): number;
private readonly parseTokenConstraint;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please rename them and add just add underscore?

Comment thread src/language/online-parser/parser.d.ts Outdated
private readonly getNextRule;
private readonly popMatchedRule;
private readonly rollbackRule;
pushRule(
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

public methods should be grouped together.

state: ?ParserState,
styles: ?Styles,
config: ?ParserConfigOption,
source: string,
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nono-ptional parameters should go first.

Comment thread src/language/online-parser/parser.js Outdated
styles,
config,
source,
}: {|
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please convert to positional arguments?
especially since we need to remove styles.

[name: string]: GraphQLGrammarRule;
}

export declare type GraphQLGrammarRule =
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove declare.

@@ -0,0 +1,76 @@
// @flow
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove these lines in all files

@@ -0,0 +1,938 @@
{
"rules": {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove rules wrapper.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We might want to integrate this with ast later on or many include lex rules. So that is why I nested it.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hereisnaman In this case rules and lexerRules would be confusing so we would need to rename it anyway.
It's a not a stable API so we can always introduce wrapper latter if needed.

@@ -0,0 +1,936 @@
{
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like Deno doesn't support importing JSON:
denoland/deno#5633

Can you please convert it to rules.js and also merge it with grammarTypes.js?
On the plus side, we would be able to type-check rules content with TS.
Also please name the resulting file grammar.js.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it triggers a coverage check please add necessary files to list of excluded files inside .nycrc.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll update.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

PR: feature 🚀 requires increase of "minor" version number

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants