Skip to content

Fix Config File Path Issue#1681

Merged
abhishekkumams merged 22 commits intomainfrom
docker-debug
Sep 7, 2023
Merged

Fix Config File Path Issue#1681
abhishekkumams merged 22 commits intomainfrom
docker-debug

Conversation

@abhishekkumams
Copy link
Copy Markdown
Contributor

@abhishekkumams abhishekkumams commented Sep 4, 2023

Why make this change?

What is this change?

  • Because of the way we used to fetch the cofigFileName, the path was lost, due to which using config file present in a different directory was giving file not found error.
  • So we updated the method to fetch the configFileNamewihoutExtension to keep the original path.
  • To run Azure Container Apps --ConfigFileName is provided with the full path name to the mounted config file.

NOTE:

  • This issue might be caught in Az Container apps but this is not specifically related to Containers, because the current file name generation process was itself incorrect in 0.8.* release.
  • When we try to get the fileNameWithoutExtension in the method GetFileNameForEnvironment, we used to lose directory information, for example: if baseConfigFile isdab/configs/dab-config.json, then configFileNameWithoutExtension was coming as dab-config, which is incorrect.
  • This issue started happening from 0.8.* because in the earlier release (0.7.6), if config file name is provided by user, we directly used it without searching for overrides and environments, unlike 0.8.*. And the precedence check used to happen for cases where config file is not provided by user.
  • Now, since 0.8.*, we started using the method GetFileNameWithoutExtension, which is incorrect causing File not found issue whenever the config file is not in current directory.
  • The method DoesFileExistInCurrentDirectory() checks if the file exists by combining the current directory with the fileName. Now if file is existing in some other directory inside current directory, then for proper searching the file should contain complete info about the directory. So, if current dicrectory is C:/dab, and file is in C:/dab/configs/dab-config.json, then the file name should either be full path or /configs/dab-configs.json for it to be searched.

Additional Change

  1. Fixing name of variables and function name to correctly reflect their behavior.
  2. Fixing issue with not honoring the user provided config file, (caused after 0.8 release). (Putting in this PR, because this PR aims at fixing how we are using the config files and not just a bug fix.)

How was this tested?

  • Running In Azure Container Apps
    BEFORE:
    image

AFTER:
image

  • Local Testing
    BEFORE:
    image

AFTER:
image

  • Unit Tests

@abhishekkumams abhishekkumams self-assigned this Sep 4, 2023
@abhishekkumams abhishekkumams added the bug Something isn't working label Sep 4, 2023
Comment thread src/Config/FileSystemRuntimeConfigLoader.cs Outdated
Comment thread src/Config/FileSystemRuntimeConfigLoader.cs Outdated
Comment thread src/Service.Tests/Unittests/ConfigValidationUnitTests.cs Outdated
Comment thread src/Service.Tests/Unittests/ConfigValidationUnitTests.cs Outdated
Comment thread src/Service.Tests/Unittests/ConfigValidationUnitTests.cs Outdated
@seantleonard
Copy link
Copy Markdown
Contributor

seantleonard commented Sep 6, 2023

In the past, we've had issues with local Docker containers finding the desired config file. The root cause of those issues was an incorrect file/folder mapping to the Docker container. could that be the case with this issue too?
In the PR description, can you describe what behavior you observed in Az Container apps?

  • Was the desired config file being overwritten in the file share, when looking for a config file in the root of the file share?
  • Is this issue only seen when the config file is placed in a folder? (not root of file share) e.g. FileShare::\\config.jsonvs. FileShare::\\subFolder\config.json

Copy link
Copy Markdown
Contributor

@aaronburtle aaronburtle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just waiting on some clarification for what we mean by full path, and the comment that sean raised about returning empty.

Comment thread src/Config/FileSystemRuntimeConfigLoader.cs
Comment thread src/Config/FileSystemRuntimeConfigLoader.cs
Comment thread src/Service.Tests/Unittests/ConfigValidationUnitTests.cs Outdated
Comment thread src/Service.Tests/Unittests/ConfigValidationUnitTests.cs Outdated
Comment thread src/Service.Tests/Unittests/ConfigValidationUnitTests.cs Outdated
Comment thread src/Service.Tests/Unittests/ConfigValidationUnitTests.cs
Comment thread src/Service.Tests/Unittests/ConfigValidationUnitTests.cs
Comment thread src/Service.Tests/Unittests/ConfigValidationUnitTests.cs Outdated
Comment thread src/Config/FileSystemRuntimeConfigLoader.cs Outdated
@abhishekkumams abhishekkumams enabled auto-merge (squash) September 7, 2023 19:34
Comment thread src/Config/FileSystemRuntimeConfigLoader.cs Outdated
@abhishekkumams abhishekkumams merged commit 653a8f6 into main Sep 7, 2023
@abhishekkumams abhishekkumams deleted the docker-debug branch September 7, 2023 20:25
Aniruddh25 added a commit that referenced this pull request Sep 7, 2023
## Why make this change?

- Closes #1674 ConfigFile was not found when running in the Azure
Container Apps.

## What is this change?

- Because of the way we used to fetch the cofigFileName, the path was
lost, due to which using config file present in a different directory
was giving `file not found` error.
- So we updated the method to fetch the configFileNamewihoutExtension to
keep the original path.
- To run Azure Container Apps `--ConfigFileName` is provided with the
full path name to the mounted config file.

NOTE:
* This issue might be caught in Az Container apps but this is not
specifically related to Containers, because the current file name
generation process was itself incorrect in 0.8.* release.
* When we try to get the `fileNameWithoutExtension` in the method
`GetFileNameForEnvironment`, we used to lose directory information, for
example: if baseConfigFile is`dab/configs/dab-config.json`, then
`configFileNameWithoutExtension` was coming as `dab-config`, which is
incorrect.
* This issue started happening from 0.8.* because in the earlier release
(0.7.6), if config file name is provided by user, we directly used it
without searching for overrides and environments, unlike 0.8.*. And the
precedence check used to happen for cases where config file is not
provided by user.
* Now, since 0.8.*, we started using the method
`GetFileNameWithoutExtension`, which is incorrect causing File not found
issue whenever the config file is not in current directory.
* The method `DoesFileExistInCurrentDirectory()` checks if the file
exists by combining the current directory with the fileName. Now if file
is existing in some other directory inside current directory, then for
proper searching the file should contain complete info about the
directory. So, if current dicrectory is `C:/dab`, and file is in
`C:/dab/configs/dab-config.json`, then the file name should either be
full path or `/configs/dab-configs.json` for it to be searched.

## Additional Change
1. Fixing name of variables and function name to correctly reflect their
behavior.
2. Fixing issue with not honoring the user provided config file, (caused
after 0.8 release). (Putting in this PR, because this PR aims at fixing
how we are using the config files and not just a bug fix.)

## How was this tested?

- [X] Running In Azure Container Apps
BEFORE:

![image](https://github.com/Azure/data-api-builder/assets/102276754/183d9399-9822-4435-8322-e7bfc5016e1e)

AFTER:

![image](https://github.com/Azure/data-api-builder/assets/102276754/fb7010c9-4410-48cf-9146-9cc757f209a0)

- [X] Local Testing
BEFORE:

![image](https://github.com/Azure/data-api-builder/assets/102276754/742cfd03-ee54-4f0c-8ea4-9e1d2303bbf8)

AFTER:

![image](https://github.com/Azure/data-api-builder/assets/102276754/bef4d153-9f5a-470b-8fbb-27ed7219fac6)

- [X] Unit Tests

---------

Co-authored-by: Aniruddh Munde <[email protected]>
Aniruddh25 added a commit that referenced this pull request Sep 7, 2023
- Cherry Picks #1681

---------

Co-authored-by: abhishekkumams <[email protected]>
rohkhann pushed a commit that referenced this pull request Sep 11, 2023
## Why make this change?

- Closes #1674 ConfigFile was not found when running in the Azure
Container Apps.

## What is this change?

- Because of the way we used to fetch the cofigFileName, the path was
lost, due to which using config file present in a different directory
was giving `file not found` error.
- So we updated the method to fetch the configFileNamewihoutExtension to
keep the original path.
- To run Azure Container Apps `--ConfigFileName` is provided with the
full path name to the mounted config file.

NOTE:
* This issue might be caught in Az Container apps but this is not
specifically related to Containers, because the current file name
generation process was itself incorrect in 0.8.* release.
* When we try to get the `fileNameWithoutExtension` in the method
`GetFileNameForEnvironment`, we used to lose directory information, for
example: if baseConfigFile is`dab/configs/dab-config.json`, then
`configFileNameWithoutExtension` was coming as `dab-config`, which is
incorrect.
* This issue started happening from 0.8.* because in the earlier release
(0.7.6), if config file name is provided by user, we directly used it
without searching for overrides and environments, unlike 0.8.*. And the
precedence check used to happen for cases where config file is not
provided by user.
* Now, since 0.8.*, we started using the method
`GetFileNameWithoutExtension`, which is incorrect causing File not found
issue whenever the config file is not in current directory.
* The method `DoesFileExistInCurrentDirectory()` checks if the file
exists by combining the current directory with the fileName. Now if file
is existing in some other directory inside current directory, then for
proper searching the file should contain complete info about the
directory. So, if current dicrectory is `C:/dab`, and file is in
`C:/dab/configs/dab-config.json`, then the file name should either be
full path or `/configs/dab-configs.json` for it to be searched.

## Additional Change
1. Fixing name of variables and function name to correctly reflect their
behavior.
2. Fixing issue with not honoring the user provided config file, (caused
after 0.8 release). (Putting in this PR, because this PR aims at fixing
how we are using the config files and not just a bug fix.)

## How was this tested?

- [X] Running In Azure Container Apps
BEFORE:

![image](https://github.com/Azure/data-api-builder/assets/102276754/183d9399-9822-4435-8322-e7bfc5016e1e)

AFTER:

![image](https://github.com/Azure/data-api-builder/assets/102276754/fb7010c9-4410-48cf-9146-9cc757f209a0)

- [X] Local Testing
BEFORE:

![image](https://github.com/Azure/data-api-builder/assets/102276754/742cfd03-ee54-4f0c-8ea4-9e1d2303bbf8)

AFTER:

![image](https://github.com/Azure/data-api-builder/assets/102276754/bef4d153-9f5a-470b-8fbb-27ed7219fac6)

- [X] Unit Tests

---------

Co-authored-by: Aniruddh Munde <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

[Bug]: ACA deployment with latest sample script and latest Docker image cannot find DAB config

4 participants