[K6 Generator] various enhancements (request body example data extraction, support for generating scenario tests and load tests out of the box, and much more) by mostafa · Pull Request #11106 · OpenAPITools/openapi-generator

mostafa · 2021-12-13T14:45:21Z

This PR builds on top of the PR #10614 by @ideas-into-software and fixes an issue I found while reviewing the code, and that is the empty lines in the output script.

@agilob @wing328 @ideas-into-software I'd be happy to have your feedback.

PR checklist

Read the contribution guidelines.
Pull Request title clearly describes the work in the pull request and Pull Request description provides details about how to validate the work. Missing information here may result in delayed response from the community.
Run the following to build the project and update samples:
```
./mvnw clean package 
./bin/generate-samples.sh
./bin/utils/export_docs_generators.sh
```
Commit all changed files.
This is important, as CI jobs will verify all generator outputs of your HEAD commit as it would merge with master.
These must match the expectations made by your contribution.
You may regenerate an individual generator by passing the relevant config(s) as an argument to the script, for example ./bin/generate-samples.sh bin/configs/java*.
For Windows users, please run the script in Git BASH.
File the PR against the correct branch: master (5.3.0), 6.0.x
If your PR is targeting a particular programming language, @mention the technical committee members, so they are more likely to review the pull request.

* request body example data extraction * request grouping and ordering * response visibility * request data extraction for chaining requests Signed-off-by: Michael H. Siemaszko <[email protected]>

- regenerated samples Signed-off-by: Michael H. Siemaszko <[email protected]>

* resolved merge conflicts * replaced deprecated dependency `org.apache.commons.lang3.StringEscapeUtils` Signed-off-by: Michael H. Siemaszko <[email protected]>

agilob · 2021-12-13T18:07:23Z

                        }
                    }
-                } catch (NullPointerException e) {
+                } catch (NullPointerException ignored) {


let's log this exception and learn what's broken

agilob · 2021-12-13T18:08:43Z

        return params != null && params.size() > 0;
    }

+    private static boolean nonempty(Map<?, ?> params) {


agilob · 2021-12-13T18:10:02Z

+
+                if (cgOperation.getHasVendorExtensions()
+                        && cgOperation.vendorExtensions.containsKey(X_OPERATION_DATAEXTRACT)
+                        && cgOperation.vendorExtensions.get(X_OPERATION_DATAEXTRACT) instanceof java.util.Map) {


instanceof java.util.Map, don't we have Map already imported to do intanceof Map?

That's a boolean check.

agilob · 2021-12-13T18:13:19Z

+            Map<String, DataExtractSubstituteParameter> dataExtractSubstituteParams, HTTPRequestGroup requestGroup) {
+
+        if (!dataExtractSubstituteParams.isEmpty()) {
+            List<String> existingVariablesNames = requestGroup.variables.stream().map(v -> v.key)


List<String> existingVariablesNames = requestGroup.variables.stream().map(v -> v.key) .collect(Collectors.toList()); Set<DataExtractSubstituteParameter> initializeVariables = dataExtractSubstituteParams.values().stream() .filter(p -> !existingVariablesNames.contains(toVarName(p.paramName))).collect(Collectors.toSet());

this filter condition will be O(1) fast if existingVariablesNames is a Set, when existingVariablesNames is a List filter will be O(n)

agilob · 2021-12-13T18:13:51Z

+            Set<DataExtractSubstituteParameter> initializeVariables = dataExtractSubstituteParams.values().stream()
+                    .filter(p -> !existingVariablesNames.contains(toVarName(p.paramName))).collect(Collectors.toSet());
+
+            if (!initializeVariables.isEmpty()) {


check if (!initializeVariables.isEmpty()) { not required when using for-each loop

agilob · 2021-12-13T18:14:14Z

-        check(request, {
+        {
+          let url = BASE_URL + `/pet/findByStatus?status=${status}`;
+


still triple empty lines :P

I think I haven't regenerated the file.

Interesting, the pipeline passed on 5082527 when the sample project was out of date.

That was strange to me, too.

agilob · 2021-12-13T18:14:29Z

+
+
+          let request = http.del(url);
+


4 empty lines!

agilob · 2021-12-13T18:16:23Z

+
+                String operationId = operation.getOperationId();
+
+                boolean hasRequestBodyExample = false;


initialisation of this can go to line 558

An outer block needs this variable if I move it inside the if block.

agilob · 2021-12-13T18:17:05Z

+
+                final CodegenOperation cgOperation = super.fromOperation(path, method.name(), operation, null);
+
+                String operationId = operation.getOperationId();


this line can go a lot lower than here

agilob · 2021-12-13T18:17:36Z


                final Operation operation = methodOperation.getValue();
                final PathItem.HttpMethod method = methodOperation.getKey();
+                OptionalInt operationGroupingOrder = OptionalInt.empty();


had no idea OptionalInt is a thing 👍

agilob · 2021-12-13T18:23:13Z

+            List<Parameter> bodyOrFormParams) {
+
+        Optional<Map.Entry<String, Example>> requestBodyExampleEntry = requestBody.getContent().get(contentTypeValue)
+                .getExamples().entrySet().stream().findFirst();


this chain map->set->stream->optional is quite slow, how often is this method call? might be worth to new ArrayList(getExamples().entrySet()).get(0)?

Only once per request. It doesn't matter much.

mostafa · 2021-12-14T10:38:19Z

     * @return true if parameter exists, false otherwise
     */
-    private static boolean nonempty(List<?> params) {
+    private static boolean nonEmpty(List<?> params) {


@agilob This is unrelated to our code, but I changed it nevertheless.

…nhancements

agilob · 2021-12-14T12:36:30Z

+            for (Map.Entry<?, ?> respExtensionEntry : respExtensions.entrySet()) {
+
+                if (X_OPERATION_RESPONSE_HIDE.equals(String.valueOf(respExtensionEntry.getKey()))) {
+                    hideOperationResponse = Boolean.parseBoolean(respExtensionEntry.getValue().toString());


should this have a break statement after setting this to true? effectively, now hideOperationResponse will have the last value of respExtensionEntry.getValue(), and if you want the last value, then we don't need a loop here, and since we're iterating on respExtensions.entrySet() a Set, the last value is not deterministic, so method might produce different value on every run.

agilob

nice, thanks 👍

mostafa · 2021-12-14T13:46:40Z

@agilob Thanks to you for the meticulous review! Finally mergeable?

agilob · 2021-12-14T18:07:27Z

There is no technical commette for k6, so it now depends on when @wing328 or @jimschubert can merge it

wing328 · 2021-12-17T02:17:21Z

+inputSpec: modules/openapi-generator/src/test/resources/3_0/petstore-with-fake-endpoints-models-for-testing.yaml
+templateDir: modules/openapi-generator/src/main/resources/k6
+additionalProperties:
+  appName: PetstoreClient


There's already one in ./bin/configs/other/k6.yaml.

I'll remove it after merging this PR.

DevMrRober · 2024-07-03T11:48:35Z

Hello, good afternoon,

I have detected that in the version change from 5.3.0 to 5.3.1 there is a bug that is dragging to the latest current version v7.7.0.

The bug itself is to generate more than 2 methods per path, ie if I have a path "/users" with a "GET, POST, PUT, PATCH" I will only generate 2 of the methods leaving the others.

I pass code

V5.3.0

group("/users/{userId}", () => {
let locale = 'TODO_EDIT_THE_LOCALE'; // specify value as there is no example value for this parameter in OpenAPI spec
let userId = 'TODO_EDIT_THE_USERID'; // specify value as there is no example value for this parameter in OpenAPI spec
let url = BASE_URL + /users/${userId}?locale=${locale};
// Request No. 1
let request = http.get(url);
check(request, {
"User found. Resource obtained successfully.": (r) => r.status === 200
});
sleep(SLEEP_DURATION);

    // Request No. 2
    // TODO: edit the parameters of the request body.
    body = {"user": {"ids": "useridmodel", "firstName": "string", "lastName": "string", "email": "string", "dateOfBirth": "date", "isEmailVerified": "boolean", "creationDate": "date"}};
    params = {headers: {"Content-Type": "application/json"}};
    request = http.put(url, body, params);
    check(request, {
        "User full information updated.": (r) => r.status === 200
    });
    sleep(SLEEP_DURATION);

    // Request No. 3
    request = http.del(url);
    check(request, {
        "Resource deleted successfully.": (r) => r.status === 204
    });
    sleep(SLEEP_DURATION);

    // Request No. 4
    // TODO: edit the parameters of the request body.
    body = {"user": {"ids": "useridmodel", "firstName": "string", "lastName": "string", "email": "string", "dateOfBirth": "date", "isEmailVerified": "boolean", "creationDate": "date"}};
    params = {headers: {"Content-Type": "application/json"}};
    request = http.patch(url, body, params);
    check(request, {
        "User partial information updated successfully.": (r) => r.status === 200
    });
    sleep(SLEEP_DURATION);
});


V5.3.1

group("/users/{userId}", () => {
let locale = 'TODO_EDIT_THE_LOCALE'; // specify value as there is no example value for this parameter in OpenAPI spec
let userId = 'TODO_EDIT_THE_USERID'; // specify value as there is no example value for this parameter in OpenAPI spec

    // Request No. 1
    {
        let url = BASE_URL + `/users/${userId}?locale=${locale}`;
        let request = http.get(url);

        check(request, {
            "User found. Resource obtained successfully.": (r) => r.status === 200
        });

        sleep(SLEEP_DURATION);
    }

    // Request No. 2
    {
        let url = BASE_URL + `/users/${userId}?locale=${locale}`;
        // TODO: edit the parameters of the request body.
        let body = {"user": {"ids": "useridmodel", "firstName": "string", "lastName": "string", "email": "string", "dateOfBirth": "date", "isEmailVerified": "boolean", "creationDate": "date"}};
        let params = {headers: {"Content-Type": "application/json", "Accept": "application/json"}};
        let request = http.patch(url, JSON.stringify(body), params);

        check(request, {
            "User partial information updated successfully.": (r) => r.status === 200
        });
    }
});


im using this commands:

docker run --rm -v "${PWD}:/local" openapitools/openapi-generator-cli:v5.3.0 generate -i /local/users.swagger.v2.0.0.json -g k6 -o /local/k6-test/ --skip-validate-spec

docker run --rm -v "${PWD}:/local" openapitools/openapi-generator-cli:v5.3.1 generate -i /local/users.swagger.v2.0.0.json -g k6 -o /local/k6-test/ --skip-validate-spec

ideas-into-software and others added 6 commits October 17, 2021 20:14

Further K6 OpenAPI generator enhancements

67eac90

* request body example data extraction * request grouping and ordering * response visibility * request data extraction for chaining requests Signed-off-by: Michael H. Siemaszko <[email protected]>

Further K6 OpenAPI generator enhancements

809dadd

- regenerated samples Signed-off-by: Michael H. Siemaszko <[email protected]>

Further K6 OpenAPI generator enhancements

ed3f00c

* resolved merge conflicts * replaced deprecated dependency `org.apache.commons.lang3.StringEscapeUtils` Signed-off-by: Michael H. Siemaszko <[email protected]>

Fix suggested changes by linter

66dc3ed

Fix extra spaces in the template

a79d0c7

Merge branch 'master' into k6-openapi-extract-examples-enhancements

5082527

agilob reviewed Dec 13, 2021

View reviewed changes

mostafa added 6 commits December 14, 2021 10:19

Log exception

5d28180

Rename function signature to camelCase

d9e10b8

Address comments on Big-O

6e106df

Move declaration of variable near the usage

4cb1819

Add config file for generating k6 script

fd19b33

Regenerate k6 script

a42d924

mostafa commented Dec 14, 2021

View reviewed changes

mostafa added 2 commits December 14, 2021 12:10

Merge branch 'OpenAPITools:master' into k6-openapi-extract-examples-e…

86ffce3

…nhancements

Regenerate samples

070f730

agilob reviewed Dec 14, 2021

View reviewed changes

mostafa added 2 commits December 14, 2021 14:36

Fix predicate

8788910

Fix missing import

e79063b

agilob approved these changes Dec 14, 2021

View reviewed changes

wing328 reviewed Dec 17, 2021

View reviewed changes

wing328 added the Client: k6 label Dec 17, 2021

wing328 added this to the 5.3.1 milestone Dec 17, 2021

wing328 added the Enhancement: General label Dec 17, 2021

wing328 merged commit fedc54a into OpenAPITools:master Dec 17, 2021

wing328 mentioned this pull request Dec 17, 2021

[ K6 Generator ] Further K6 OpenAPI generator enhancements (request body example data extraction, support for generating scenario tests and load tests out of the box, and much more) #10614

Closed

5 tasks

mostafa deleted the k6-openapi-extract-examples-enhancements branch December 17, 2021 09:43


		String operationId = operation.getOperationId();

		boolean hasRequestBodyExample = false;


		final CodegenOperation cgOperation = super.fromOperation(path, method.name(), operation, null);

		String operationId = operation.getOperationId();

Uh oh!

Conversation

mostafa commented Dec 13, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR checklist

Uh oh!

agilob Dec 13, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

agilob Dec 13, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

agilob left a comment

Choose a reason for hiding this comment

Uh oh!

mostafa commented Dec 14, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

agilob commented Dec 14, 2021

Uh oh!

Choose a reason for hiding this comment

Uh oh!

DevMrRober commented Jul 3, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

mostafa commented Dec 13, 2021 •

edited

Loading

agilob Dec 13, 2021 •

edited

Loading

agilob Dec 13, 2021 •

edited

Loading

mostafa commented Dec 14, 2021 •

edited

Loading

DevMrRober commented Jul 3, 2024 •

edited

Loading