Adds Back-end for analysis submission#325
Conversation
| } | ||
|
|
||
| @ApiOperation(value = "list submissions") | ||
| @RequestMapping(value = "/api/v1/admin/submissions", method = RequestMethod.GET) |
There was a problem hiding this comment.
Let's unify terminology: why submissions here and analyses in all other places?
There was a problem hiding this comment.
It's legacy, of course I can change it to analysis since this is analysis.
But this requires to make changes on the UI side.
| return result; | ||
| } | ||
|
|
||
| @ApiOperation(value = "Invalidate all unfinished analyses") |
There was a problem hiding this comment.
Shouldn't this method be in AnalysisController?
There was a problem hiding this comment.
May be, leaved as is just for the same reasons as above.
| return analysisService.invalidateAllUnfinishedAnalyses(getUser(principal)); | ||
| } | ||
|
|
||
| @ApiOperation(value = "list submissions") |
There was a problem hiding this comment.
Shouldn't this method be in AnalysisController?
| } | ||
| } | ||
|
|
||
| private void saveAnalysisFiles(Analysis analysis, List<MultipartFile> files) throws IOException, ZipException { |
There was a problem hiding this comment.
Shouldn't this go into an appropriate service?
| import org.springframework.context.event.EventListener; | ||
| import org.springframework.core.convert.support.GenericConversionService; | ||
|
|
||
| public class AnalysisEventController { |
There was a problem hiding this comment.
Why do we really need it? I mean, these are synchronous events received from the same module
|
|
||
| AnalysisStateEntry stateEntry = new AnalysisStateEntry(new Date(), | ||
| AnalysisState.CREATED, | ||
| "Get request to execute analysis", |
There was a problem hiding this comment.
Not sure whether I get meaning of the message
| import com.fasterxml.jackson.annotation.JsonProperty; | ||
| import javax.persistence.Embeddable; | ||
|
|
||
| /** |
There was a problem hiding this comment.
Please use standard headers
|
If you submit ZIP with an imported cohort, the results received back doesn't differ from the uploaded ZIP - cohort is not calculated |
* New version: 1.14.0 * ARACHNE-2359. Back-merge of fix of OHDSI SQL translation for Oracle * ARACHNE-2548. Fixed IR fails in Oracle and schroot (#253) * ARACHNE-2554. Problem with cohort import from ATLAS (#264) * fixes failed test (#269) * ARACHNE-2540 - Fixed error message when external system is not available (cherry picked from commit a46468d) * ARACHNE-2569 - Fixed working with atlasClientPool (cherry picked from commit 92de367) * ARACHNE-2557. Cohort / IR execution imported from ATLAS fails (#262) (cherry picked from commit 58ac04c) * ARACHNE-2573. Unable to register at Node side [back_merge] (#277) * ARACHNE-2566. Excluded deleted ds from health check and achilles scheduler [back_merge] (#279) * Fixed ojdbc6 version * ARACHNE-2564. Added new reg form fields [back_merge] (#281) * ARACHNE-2586. Added network parameter for achilles container (#284) * ARACHNE-2600. Fixed Achilles import by specifying concrete version of used scripts (#286) * Setting to specify custom mapping of Achilles temp folder (#289) * ARACHNE-2627. Added configurable Feign timeout (#291) * Arachne 2598. Updated schema parameters for PLE and PLP (#293) * ARACHNE-2610. Caught 'logout' and 'bad credentials' auth exception (#294) * Adds Support of BQ keyfile json for authentication of EE jobs (#298) * Adds import of new Estimation and Prediction analyses (#299) * Fixes GraalVM inside Spring Boot (#304) * Issue-621. Support BIGING and NUMERIC types for Achilles report exporting (#305) * Updates used Achilles version (#306) * Implements new CC import (#302) * Arachne-ui-issue-651. Fixes issues with reports import (#310) * Back merge from 1.14 & version increment (#312) * conforms arachne-commons structure changes * Conforms arachne-commons structure changes (#317) * conforms arachne-commons structure changes * Fix Feign encoders (#314) * Improve swagger documentation (#320) * improve swagger documentation * improve swagger documentation - update and configure swagger * improve swagger documentation - document remaining API methods * Fixes failed signature validation (#326) * issue-321 - sending node token to central cause signature validation failure * create Arachne community edition build and run manual - apply review comments * Analysis submission via Datanode (#325) * Add default value for the submission.result.files.exclusions property in test config (#329) * Fixes execution engine connection properties (#330) * Utilizes Authenticator & introduces Standalone mode (#328) * Fixes Datanode shows info in Sources list inconsistent with Central (#337) * Prohibits update of published Data source name in Standalone mode (#334) * Fixes authentication filter logic (#336) * Fixes tests after Authenticator addition (#338) * Upgrade to zip4j 2.1.1 (#339) * Fixes analysis import (#340) * Back merge release 1.15.0 (#363) * ARACHNE-2554. Problem with cohort import from ATLAS (#263) * Fixed test (#267) * ARACHNE-2540 - Fixed error message when external system is not available (#265) * ARACHNE-2569 - Fixed working with atlasClientPool (#270) * ARACHNE-2557. Cohort / IR execution imported from ATLAS fails (#262) (cherry picked from commit 58ac04c) * ARACHNE-2573. Fixed registration at Node side (#276) * ARACHNE-2566. Excluded deleted DSs from health check and Achilles scheduler (#278) * ARACHNE-2564. Added new reg form fields (#280) * Fixed ojdbc6 version * ARACHNE-2586. Added network parameter for achilles container (#283) * ARACHNE-2600. Fixed Achilles import by specifying concrete version of used scripts (#285) ARACHNE-2600. Fixed Achilles import by specifying concrete version of used scripts * Setting to specify custom mapping of Achilles temp folder (#288) * ARACHNE-2627. Added configurable Feign timeout (#290) * ARACHNE-2598. Updated schema parameters for PLP and PLE (#292) * ARACHNE-2610. Replaced throwing errors to log messages (#287) * update version to 1.13.1-QA (#300) * 1.14.0-QA (#307) * Arachne-ui-issue-651. Fixes issues with reports import (#310) (cherry picked from commit 1f438df) * QA 1.15.0 (#341) * Update Authenticator dependency (#342) * Removes packrat from IR (#343) * adds sanitation for cohort files (#345) * hides Heracles and Pathways analyses from submissions (#347) * Fixes BigQuery datasource create/edit (#349) * Fix IR drivers path (#351) * Fixes validations for Impala DBMS (#353) * fixes datasource validator tests * fixes datasource validator tests * Fixes empty admin FQDN for Kerberos auth (#357) * ARACHNE-2723 IR fails on BigQuery (#358) * ARACHNE-2723 IR fails on BigQuery - pass result schema as temp schema to the SqlRender - replace usage of deprecated translateSql method with translate * ARACHNE-2723 IR fails on BigQuery (fix PR remarks) - use render instead of deprecated renderSql - add tempDatabaseSchema as default argument in method * Arachne-2723. Fixes failure on BQ. Fixes issues with Strata processing (#360) * ARACHNE-2722 Cohort (simple counts) fails on BigQuery (#361) * back-merge 1.15 * Enable Cohort counts (checkbox) by default (#364) * ARACHNE-2744 Cohort Characterization fails on BigQuery (#372) * ARACHNE-2744 Cohort Characterization fails on BigQuery - generate sql with tempSchema(add tempSchema as argument to the SqlRender::translate method * ARACHNE-2744 Cohort Characterization fails on BigQuery. fix review remarks - set tempSchema * Validate datasource name duplication before the save (#359) * Validate datasource name duplication before the save and show user friendly message if validation fails * code review comments * fixes generalized type * Code review issues * Adds New pathway (#315) * Add support for new prediction/estimation * Adds import of new Estimation and Prediction analyses * Implement new CC import * import of cohort pathway * Implement new Pathway import * Adds import of new Estimation and Prediction analyses * removes redundant methods * Implement new CC import * fixes mapper inheritance * fixes mapper inheritance * Implement new CC import * merged from latest develop * merged from latest develop * fix Feign encoders * fix Feign encoders * new pathway analysis * new pathway analysis * fixes resource path * provides running in a standalone mode * updates Pathways analysis to 1.16 * adds sanitation for cohort names * fixes after review * fix broken build (#377) * ARACHNE-2759 Errors during prod Upgrade (#378) - fix: violates foreign key constraint "fk_analysis_id" * Implements connection check while adding datasource (#366) * ARACHNE-2715 - move check functionality from enterprise * ARACHNE-2715 - refactoring of health check - add unit test * Issue 2715 no connection check for new data source - fix PR remarks -- remove heartbeat constant -- remove wrong ee host from application-base * Issue 2715 no connection check for new data source - call autodetectFields for any data-node-mode.(it used to be only for Network) * Issue 2715 no connection check for new data source - fix after merge - fix unit test * ARACHNE-2759 Errors during prod Upgrade (#381) - fix script for empty DB * Support heracles analyses (#382) * ARACHNE-2708 HERACLES * Heracles Analysis Support * Heracles Analysis Support * Heracles Analysis Support - delete unpacked files * Heracles Analysis Support - self-review code tweaks * replace broken IncidenceRate_v0.0.1.zip with the fresh version from the IR Repo (#384) * Update vulnerable and some other dependencies (#386) * ARACHNE - update some dependencies versions * ARACHNE-2719 Update AD/LDAP user import routines to work with Authen… (#385) * ARACHNE-2719 Update AD/LDAP user import routines to work with Authenticator - upgrade version of authenticator * ARACHNE-2719 Update AD/LDAP user import routines to work with Authenticator. fix PR remarks - add check to avoid NPE * ARACHNE-2719 Update AD/LDAP user import routines to work with Authenticator. fix PR remarks - UserInfo signature was changed * ARACHNE-2719 Update AD/LDAP user import routines to work with Authenticator. fix PR remarks - UserInfo signature was changed * ARACHNE-2719 Update AD/LDAP user import routines to work with Authenticator. fix PR remarks - UserInfo signature was changed * Analysis import (#387) generate analysis description field during import * ARACHNE-2719 Update AD/LDAP user import routines to work with Authen… (#389) * ARACHNE-2719 Update AD/LDAP user import routines to work with Authenticator. fix PR remarks - fix for creating data source in stand-alone mode * RACHNE-2696 files naming convention, fix PR remarks - set notNull token field * ARACHNE-2719 Update AD/LDAP user import routines to work with Authenticator. fix PR remarks - fix for creating data source in stand-alone mode * RACHNE-2696 files naming convention, fix PR remarks - remove unused constant * ARACHNE-2719 Update AD/LDAP user import routines to work with Authenticator. fix PR remarks - make datanode.token column nullable * Update pom.xml (#390) remove http://repo.maven.apache.org/maven2 http is not longer supported by maven central. Default repo point to the correct https version https://stackoverflow.com/a/59764670/1167673 * Arachne 2696 files naming convention (#388) * import description - refactoring * import description - meta files * generate analysis description field during import * generate analysis description field during import * generate analysis description field during import * generate analysis description field during import * generate analysis description field during import * RACHNE-2696 files naming convention - use AnalysesArchiveUtils.getArchiveFileName method to generate zip file name * RACHNE-2696 files naming convention - fix merge Co-authored-by: Anton Gackovka <[email protected]> Co-authored-by: Vitaly Koulakov <[email protected]> Co-authored-by: anastasiia klochkova <[email protected]> Co-authored-by: OdysseusBot <[email protected]> Co-authored-by: mpozhidaeva <[email protected]> Co-authored-by: Pavel Grafkin <[email protected]> Co-authored-by: Alex Cumarav <[email protected]>
* update version to 1.13.1-QA (#300) * 1.14.0-QA (#307) * Arachne-ui-issue-651. Fixes issues with reports import (#310) (cherry picked from commit 1f438df) * QA 1.15.0 (#341) * Update Authenticator dependency (#342) * Removes packrat from IR (#343) * adds sanitation for cohort files (#345) * hides Heracles and Pathways analyses from submissions (#347) * Fixes BigQuery datasource create/edit (#349) * Fix IR drivers path (#351) * Fixes validations for Impala DBMS (#353) * fixes datasource validator tests * fixes datasource validator tests * Fixes empty admin FQDN for Kerberos auth (#357) * ARACHNE-2723 IR fails on BigQuery (#358) * ARACHNE-2723 IR fails on BigQuery - pass result schema as temp schema to the SqlRender - replace usage of deprecated translateSql method with translate * ARACHNE-2723 IR fails on BigQuery (fix PR remarks) - use render instead of deprecated renderSql - add tempDatabaseSchema as default argument in method * Arachne-2723. Fixes failure on BQ. Fixes issues with Strata processing (#360) * ARACHNE-2722 Cohort (simple counts) fails on BigQuery (#361) * Backmerge of fix for DataNode create in Standalone mode (#367) * fixes datanode creation in standalone mode (cherry picked from commit b1f335d) * tests added (cherry picked from commit f0b3932) * improves standalone test (cherry picked from commit 44a73c3) * removes extra settings requirements (cherry picked from commit 1a7af48) * removes unused drivers for tests (cherry picked from commit 1810062) * fixes after review (cherry picked from commit 47b1df0) * Hotfix UUID migration qa backmerge (#369) * migration should not run in standalone mode * fixes after review * ARACHNE-2744 Cohort Characterization fails on BigQuery (#372) (#373) * ARACHNE-2744 Cohort Characterization fails on BigQuery - generate sql with tempSchema(add tempSchema as argument to the SqlRender::translate method * ARACHNE-2744 Cohort Characterization fails on BigQuery. fix review remarks - set tempSchema * ARACHNE-2729 Add IAP auth support into Standalone Node (#374) * ARACHNE-2729 Add IAP auth support into Standalone Node * ARACHNE-2729 Add IAP auth support into Standalone Node - create user by IAP token * ARACHNE-2729 Add IAP auth support into Standalone Node - add AuthorizationMode * ARACHNE-2729 Add IAP auth support into Standalone Node - clean up code * ARACHNE-2729 Add IAP auth support into Standalone Node - Authenticator class was moved in other pkg * New version: 1.15.1-QA * ARACHNE-2729 Add IAP auth support into Standalone Node - fix pr remarks. - code convention * ARACHNE-2729 Add IAP auth support into Standalone Node - fix unit tests * set 0.0.2-QA version of Authenticator (#376) * Qa 1.16.0 (#391) * New version: 1.14.0 * ARACHNE-2359. Back-merge of fix of OHDSI SQL translation for Oracle * ARACHNE-2548. Fixed IR fails in Oracle and schroot (#253) * ARACHNE-2554. Problem with cohort import from ATLAS (#264) * fixes failed test (#269) * ARACHNE-2540 - Fixed error message when external system is not available (cherry picked from commit a46468d) * ARACHNE-2569 - Fixed working with atlasClientPool (cherry picked from commit 92de367) * ARACHNE-2557. Cohort / IR execution imported from ATLAS fails (#262) (cherry picked from commit 58ac04c) * ARACHNE-2573. Unable to register at Node side [back_merge] (#277) * ARACHNE-2566. Excluded deleted ds from health check and achilles scheduler [back_merge] (#279) * Fixed ojdbc6 version * ARACHNE-2564. Added new reg form fields [back_merge] (#281) * ARACHNE-2586. Added network parameter for achilles container (#284) * ARACHNE-2600. Fixed Achilles import by specifying concrete version of used scripts (#286) * Setting to specify custom mapping of Achilles temp folder (#289) * ARACHNE-2627. Added configurable Feign timeout (#291) * Arachne 2598. Updated schema parameters for PLE and PLP (#293) * ARACHNE-2610. Caught 'logout' and 'bad credentials' auth exception (#294) * Adds Support of BQ keyfile json for authentication of EE jobs (#298) * Adds import of new Estimation and Prediction analyses (#299) * Fixes GraalVM inside Spring Boot (#304) * Issue-621. Support BIGING and NUMERIC types for Achilles report exporting (#305) * Updates used Achilles version (#306) * Implements new CC import (#302) * Arachne-ui-issue-651. Fixes issues with reports import (#310) * Back merge from 1.14 & version increment (#312) * conforms arachne-commons structure changes * Conforms arachne-commons structure changes (#317) * conforms arachne-commons structure changes * Fix Feign encoders (#314) * Improve swagger documentation (#320) * improve swagger documentation * improve swagger documentation - update and configure swagger * improve swagger documentation - document remaining API methods * Fixes failed signature validation (#326) * issue-321 - sending node token to central cause signature validation failure * create Arachne community edition build and run manual - apply review comments * Analysis submission via Datanode (#325) * Add default value for the submission.result.files.exclusions property in test config (#329) * Fixes execution engine connection properties (#330) * Utilizes Authenticator & introduces Standalone mode (#328) * Fixes Datanode shows info in Sources list inconsistent with Central (#337) * Prohibits update of published Data source name in Standalone mode (#334) * Fixes authentication filter logic (#336) * Fixes tests after Authenticator addition (#338) * Upgrade to zip4j 2.1.1 (#339) * Fixes analysis import (#340) * Back merge release 1.15.0 (#363) * ARACHNE-2554. Problem with cohort import from ATLAS (#263) * Fixed test (#267) * ARACHNE-2540 - Fixed error message when external system is not available (#265) * ARACHNE-2569 - Fixed working with atlasClientPool (#270) * ARACHNE-2557. Cohort / IR execution imported from ATLAS fails (#262) (cherry picked from commit 58ac04c) * ARACHNE-2573. Fixed registration at Node side (#276) * ARACHNE-2566. Excluded deleted DSs from health check and Achilles scheduler (#278) * ARACHNE-2564. Added new reg form fields (#280) * Fixed ojdbc6 version * ARACHNE-2586. Added network parameter for achilles container (#283) * ARACHNE-2600. Fixed Achilles import by specifying concrete version of used scripts (#285) ARACHNE-2600. Fixed Achilles import by specifying concrete version of used scripts * Setting to specify custom mapping of Achilles temp folder (#288) * ARACHNE-2627. Added configurable Feign timeout (#290) * ARACHNE-2598. Updated schema parameters for PLP and PLE (#292) * ARACHNE-2610. Replaced throwing errors to log messages (#287) * update version to 1.13.1-QA (#300) * 1.14.0-QA (#307) * Arachne-ui-issue-651. Fixes issues with reports import (#310) (cherry picked from commit 1f438df) * QA 1.15.0 (#341) * Update Authenticator dependency (#342) * Removes packrat from IR (#343) * adds sanitation for cohort files (#345) * hides Heracles and Pathways analyses from submissions (#347) * Fixes BigQuery datasource create/edit (#349) * Fix IR drivers path (#351) * Fixes validations for Impala DBMS (#353) * fixes datasource validator tests * fixes datasource validator tests * Fixes empty admin FQDN for Kerberos auth (#357) * ARACHNE-2723 IR fails on BigQuery (#358) * ARACHNE-2723 IR fails on BigQuery - pass result schema as temp schema to the SqlRender - replace usage of deprecated translateSql method with translate * ARACHNE-2723 IR fails on BigQuery (fix PR remarks) - use render instead of deprecated renderSql - add tempDatabaseSchema as default argument in method * Arachne-2723. Fixes failure on BQ. Fixes issues with Strata processing (#360) * ARACHNE-2722 Cohort (simple counts) fails on BigQuery (#361) * back-merge 1.15 * Enable Cohort counts (checkbox) by default (#364) * ARACHNE-2744 Cohort Characterization fails on BigQuery (#372) * ARACHNE-2744 Cohort Characterization fails on BigQuery - generate sql with tempSchema(add tempSchema as argument to the SqlRender::translate method * ARACHNE-2744 Cohort Characterization fails on BigQuery. fix review remarks - set tempSchema * Validate datasource name duplication before the save (#359) * Validate datasource name duplication before the save and show user friendly message if validation fails * code review comments * fixes generalized type * Code review issues * Adds New pathway (#315) * Add support for new prediction/estimation * Adds import of new Estimation and Prediction analyses * Implement new CC import * import of cohort pathway * Implement new Pathway import * Adds import of new Estimation and Prediction analyses * removes redundant methods * Implement new CC import * fixes mapper inheritance * fixes mapper inheritance * Implement new CC import * merged from latest develop * merged from latest develop * fix Feign encoders * fix Feign encoders * new pathway analysis * new pathway analysis * fixes resource path * provides running in a standalone mode * updates Pathways analysis to 1.16 * adds sanitation for cohort names * fixes after review * fix broken build (#377) * ARACHNE-2759 Errors during prod Upgrade (#378) - fix: violates foreign key constraint "fk_analysis_id" * Implements connection check while adding datasource (#366) * ARACHNE-2715 - move check functionality from enterprise * ARACHNE-2715 - refactoring of health check - add unit test * Issue 2715 no connection check for new data source - fix PR remarks -- remove heartbeat constant -- remove wrong ee host from application-base * Issue 2715 no connection check for new data source - call autodetectFields for any data-node-mode.(it used to be only for Network) * Issue 2715 no connection check for new data source - fix after merge - fix unit test * ARACHNE-2759 Errors during prod Upgrade (#381) - fix script for empty DB * Support heracles analyses (#382) * ARACHNE-2708 HERACLES * Heracles Analysis Support * Heracles Analysis Support * Heracles Analysis Support - delete unpacked files * Heracles Analysis Support - self-review code tweaks * replace broken IncidenceRate_v0.0.1.zip with the fresh version from the IR Repo (#384) * Update vulnerable and some other dependencies (#386) * ARACHNE - update some dependencies versions * ARACHNE-2719 Update AD/LDAP user import routines to work with Authen… (#385) * ARACHNE-2719 Update AD/LDAP user import routines to work with Authenticator - upgrade version of authenticator * ARACHNE-2719 Update AD/LDAP user import routines to work with Authenticator. fix PR remarks - add check to avoid NPE * ARACHNE-2719 Update AD/LDAP user import routines to work with Authenticator. fix PR remarks - UserInfo signature was changed * ARACHNE-2719 Update AD/LDAP user import routines to work with Authenticator. fix PR remarks - UserInfo signature was changed * ARACHNE-2719 Update AD/LDAP user import routines to work with Authenticator. fix PR remarks - UserInfo signature was changed * Analysis import (#387) generate analysis description field during import * ARACHNE-2719 Update AD/LDAP user import routines to work with Authen… (#389) * ARACHNE-2719 Update AD/LDAP user import routines to work with Authenticator. fix PR remarks - fix for creating data source in stand-alone mode * RACHNE-2696 files naming convention, fix PR remarks - set notNull token field * ARACHNE-2719 Update AD/LDAP user import routines to work with Authenticator. fix PR remarks - fix for creating data source in stand-alone mode * RACHNE-2696 files naming convention, fix PR remarks - remove unused constant * ARACHNE-2719 Update AD/LDAP user import routines to work with Authenticator. fix PR remarks - make datanode.token column nullable * Update pom.xml (#390) remove http://repo.maven.apache.org/maven2 http is not longer supported by maven central. Default repo point to the correct https version https://stackoverflow.com/a/59764670/1167673 * Arachne 2696 files naming convention (#388) * import description - refactoring * import description - meta files * generate analysis description field during import * generate analysis description field during import * generate analysis description field during import * generate analysis description field during import * generate analysis description field during import * RACHNE-2696 files naming convention - use AnalysesArchiveUtils.getArchiveFileName method to generate zip file name * RACHNE-2696 files naming convention - fix merge Co-authored-by: Anton Gackovka <[email protected]> Co-authored-by: Vitaly Koulakov <[email protected]> Co-authored-by: anastasiia klochkova <[email protected]> Co-authored-by: OdysseusBot <[email protected]> Co-authored-by: mpozhidaeva <[email protected]> Co-authored-by: Pavel Grafkin <[email protected]> Co-authored-by: Alex Cumarav <[email protected]> * Qa 1.16.0 fix (#395) * ARACHNE-2729 Add IAP auth support into Standalone Node (#392) * set standard authentication mode by default(ARACHNE-2729) (#394) Co-authored-by: Alex Cumarav <[email protected]> Co-authored-by: Alex Cumarav <[email protected]> Co-authored-by: Konstantin Iaroshovets <[email protected]> Co-authored-by: molodkov yaroslav <[email protected]> Co-authored-by: Anton Gackovka <[email protected]> Co-authored-by: anastasiia klochkova <[email protected]> Co-authored-by: OdysseusBot <[email protected]> Co-authored-by: mpozhidaeva <[email protected]> Co-authored-by: Pavel Grafkin <[email protected]>
Resolves #323