[Perfstress][Storage] Added Datalake perf tests#15861
Merged
annatisch merged 7 commits intoAzure:masterfrom Mar 3, 2021
Merged
[Perfstress][Storage] Added Datalake perf tests#15861annatisch merged 7 commits intoAzure:masterfrom
annatisch merged 7 commits intoAzure:masterfrom
Conversation
annatisch
requested review from
amishra-dev,
kasobol-msft,
rakshith91,
tasherif-msft,
xiafu-msft and
zezha-msft
as code owners
ghost
added
the
Contributor
Member
|
I see no concerns. |
mikeharder
requested changes
Dec 21, 2020
sdk/storage/azure-storage-file-datalake/tests/perfstress_tests/append.py
Show resolved
Hide resolved
sdk/storage/azure-storage-file-datalake/tests/perfstress_tests/append.py
Outdated
Show resolved
Hide resolved
|
|
||
| def run_sync(self): | ||
| data = RandomStream(self.args.size) | ||
| self.file_client.append_data( |
Member
There was a problem hiding this comment.
Can appending to the same file over and over cause issues with service perf or correctness (e.g. is there a max file size)?
Member
Author
There was a problem hiding this comment.
I will check in with the Storage team to see if they can shed light on that
Contributor
There was a problem hiding this comment.
There should be no problems. When you keep re-appending, you would be rewriting the appended data (i.e the last append wins - assuming same length and offset). As for the maximum size, its 4.75 TiB (100 MiB X 50,000 blocks) for version 2016-05-31 and later, and 195 GiB (4 MiB X 50,000 blocks) for all older versions (since a datalake file is a block blob).
sdk/storage/azure-storage-file-datalake/tests/perfstress_tests/read.py
Outdated
Show resolved
Hide resolved
sdk/storage/azure-storage-file-datalake/tests/perfstress_tests/upload.py
Show resolved
Hide resolved
sdk/storage/azure-storage-file-datalake/tests/perfstress_tests/upload_from_file.py
Show resolved
Hide resolved
mikeharder
approved these changes
Jan 12, 2021
annatisch
requested review from
danieljurek,
mitchdenny,
scbedd and
weshaggard
as code owners
tasherif-msft
approved these changes
Feb 22, 2021
iscai-msft
added a commit
that referenced
this pull request
Mar 3, 2021
…into add_sample_check * 'master' of https://github.com/Azure/azure-sdk-for-python: (388 commits) [text analytics] add normalized_text (#17074) Renaming with_token identity function (#17066) Adapt to azure core's cloud event (#17063) align perf tests with js (#17069) [Perfstress][Storage] Added FileShare perf tests (#15834) [formrecognizer] Adding custom forms perf test (#16969) Fix LanguageShort typo (#17068) sas creds updates (#17065) [eventgrid] Fix Sample eh (#17064) [Perfstress][Storage] Added Datalake perf tests (#15861) [text analytics] Healthcare n-ary relations (#16997) ServiceBus dict-representation acceptance and kwarg-update functionality (#14807) [text analytics] add perf tests (#17060) Add cloud event to core (#16800) [Perf] Small fixes to storage-blob (#17055) [EG] Regenerate Code (#17053) Scrub batch shared keys (#17030) [Tables] Add SAS to tables (#16717) T2 containerservice 2021 03 03 (#17050) Addressing issues with CredScan (#16944) ...
iscai-msft
added a commit
to iscai-msft/azure-sdk-for-python
that referenced
this pull request
Mar 4, 2021
…into add_query_kwarg * 'master' of https://github.com/Azure/azure-sdk-for-python: (69 commits) march release (Azure#16966) Release of Device Update for IoT Hub SDK for Python. (Azure#17005) Add Get-AllPackageInfoFromRepo (Azure#16947) Track1 package is incorrectly set as track2 (Azure#17075) [text analytics] add normalized_text (Azure#17074) Renaming with_token identity function (Azure#17066) Adapt to azure core's cloud event (Azure#17063) align perf tests with js (Azure#17069) [Perfstress][Storage] Added FileShare perf tests (Azure#15834) [formrecognizer] Adding custom forms perf test (Azure#16969) Fix LanguageShort typo (Azure#17068) sas creds updates (Azure#17065) [eventgrid] Fix Sample eh (Azure#17064) [Perfstress][Storage] Added Datalake perf tests (Azure#15861) [text analytics] Healthcare n-ary relations (Azure#16997) ServiceBus dict-representation acceptance and kwarg-update functionality (Azure#14807) [text analytics] add perf tests (Azure#17060) Add cloud event to core (Azure#16800) [Perf] Small fixes to storage-blob (Azure#17055) [EG] Regenerate Code (Azure#17053) ...
iscai-msft
added a commit
to iscai-msft/azure-sdk-for-python
that referenced
this pull request
Mar 4, 2021
…into add_reprs * 'master' of https://github.com/Azure/azure-sdk-for-python: (71 commits) EG - more docs imrpovement (Azure#17079) [EventHubs] add logging.info to warn the usage of partition key of non-string type (Azure#17057) march release (Azure#16966) Release of Device Update for IoT Hub SDK for Python. (Azure#17005) Add Get-AllPackageInfoFromRepo (Azure#16947) Track1 package is incorrectly set as track2 (Azure#17075) [text analytics] add normalized_text (Azure#17074) Renaming with_token identity function (Azure#17066) Adapt to azure core's cloud event (Azure#17063) align perf tests with js (Azure#17069) [Perfstress][Storage] Added FileShare perf tests (Azure#15834) [formrecognizer] Adding custom forms perf test (Azure#16969) Fix LanguageShort typo (Azure#17068) sas creds updates (Azure#17065) [eventgrid] Fix Sample eh (Azure#17064) [Perfstress][Storage] Added Datalake perf tests (Azure#15861) [text analytics] Healthcare n-ary relations (Azure#16997) ServiceBus dict-representation acceptance and kwarg-update functionality (Azure#14807) [text analytics] add perf tests (Azure#17060) Add cloud event to core (Azure#16800) ...
iscai-msft
added a commit
to iscai-msft/azure-sdk-for-python
that referenced
this pull request
Mar 4, 2021
…into http_response_json * 'master' of https://github.com/Azure/azure-sdk-for-python: (165 commits) EG - more docs imrpovement (Azure#17079) [EventHubs] add logging.info to warn the usage of partition key of non-string type (Azure#17057) march release (Azure#16966) Release of Device Update for IoT Hub SDK for Python. (Azure#17005) Add Get-AllPackageInfoFromRepo (Azure#16947) Track1 package is incorrectly set as track2 (Azure#17075) [text analytics] add normalized_text (Azure#17074) Renaming with_token identity function (Azure#17066) Adapt to azure core's cloud event (Azure#17063) align perf tests with js (Azure#17069) [Perfstress][Storage] Added FileShare perf tests (Azure#15834) [formrecognizer] Adding custom forms perf test (Azure#16969) Fix LanguageShort typo (Azure#17068) sas creds updates (Azure#17065) [eventgrid] Fix Sample eh (Azure#17064) [Perfstress][Storage] Added Datalake perf tests (Azure#15861) [text analytics] Healthcare n-ary relations (Azure#16997) ServiceBus dict-representation acceptance and kwarg-update functionality (Azure#14807) [text analytics] add perf tests (Azure#17060) Add cloud event to core (Azure#16800) ...
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.