[Storage-Blob] Quick Query API#11991
[Storage-Blob] Quick Query API#11991xiafu-msft merged 29 commits intoAzure:feature/storage-stg73from
Conversation
sdk/storage/azure-storage-blob/azure/storage/blob/_quick_query_helper.py
Outdated
Show resolved
Hide resolved
| This keyword arg could be set for delimited (CSV) serialization or JSON serialization. | ||
| When the input_serialization is set for JSON records, only a record separator in str format is needed. | ||
| :paramtype output_serialization: ~azure.storage.blob.DelimitedTextConfiguration or str. | ||
| :keyword Union[str, Callable[Exception]] errors: |
There was a problem hiding this comment.
does operation_on_error sound better?
There was a problem hiding this comment.
I've modelled the parameter naming based on this:
https://docs.python.org/3.8/library/codecs.html#error-handlers
There was a problem hiding this comment.
@kasobol-msft thoughts? how about error handler
There was a problem hiding this comment.
Yes, I posted comment next to implementation below. I believe this change doesn't follow our feature specification and I'd recommend to change in back.
sdk/storage/azure-storage-blob/azure/storage/blob/_quick_query_helper.py
Outdated
Show resolved
Hide resolved
sdk/storage/azure-storage-blob/azure/storage/blob/_quick_query_helper.py
Outdated
Show resolved
Hide resolved
sdk/storage/azure-storage-blob/azure/storage/blob/_quick_query_helper.py
Outdated
Show resolved
Hide resolved
|
/azp run python - storage - ci |
|
Pull request contains merge conflicts. |
…orage-stg73-query
sdk/storage/azure-storage-blob/azure/storage/blob/_blob_client.py
Outdated
Show resolved
Hide resolved
sdk/storage/azure-storage-blob/azure/storage/blob/_blob_client.py
Outdated
Show resolved
Hide resolved
sdk/storage/azure-storage-blob/azure/storage/blob/_blob_client.py
Outdated
Show resolved
Hide resolved
sdk/storage/azure-storage-blob/azure/storage/blob/_quick_query_helper.py
Outdated
Show resolved
Hide resolved
sdk/storage/azure-storage-blob/azure/storage/blob/_quick_query_helper.py
Outdated
Show resolved
Hide resolved
| a custom DelimitedTextDialect, or alternatively a DelimitedJSON. | ||
| :paramtype blob_format: ~azure.storage.blob.DelimitedTextDialect or ~azure.storage.blob.DelimitedJSON | ||
| :keyword output_format: | ||
| Optional. Defines the output serialization for the data stream. By default the data will be returned |
There was a problem hiding this comment.
Sorry that I told you wrong about this"By default the data will be returned as it is represented in the blob", I tried again and the it shows the default output is also Delimited Text data. While I think it makes more sense to return the format that the blob presently have. I will add a commit to make output format default to the existing blob format.
if output_format:
# existing code
else:
output = input_format
| expression=query_expression, | ||
| input_serialization=serialize_query_format(input_format), | ||
| output_serialization=serialize_query_format(output_format) | ||
| ) |
There was a problem hiding this comment.
| ) | |
| ) | |
| else: | |
| output_format = input_format |
* [Storage]STG73 * [Blob][Swagger]Update Swagger (#10943) * [Blob][Swagger]Regenerate Swagger Code * fix container test failure caused by list_containers include type change * [Storage] Internal avro parser. (#10764) * initial avro parser * try fixing test... * falling in love with python compatibility... * make linter happy. * raise StopIteration when there is no more bytes instead of tracking file length. * async avro parser * fix syntax for Python 3.5 * get rid of 'readers_schema' as we only honor schema that has been written to file ('writer_schema'). * pr feedback * trim unused code. * pr feedback. * simplify skip sync in next. * move avro tests from _shared. * Jumbo blob support (#11176) * wip * initial test coverage. * wip. * wip * single upload. * add async tests. * disable 50k block tests. * datalake append. * async datalake * disable tests that send large payload over network. * pr feedback. * Undelete share (#11394) * Undelete container (#11339) * [Storage][Blob] Added support for Object Replication (#11525) * Blob versioning (#11154) * [Blob][QuickQuery]Add Quick Query Support (#10946) * [Blob][STG73]Blob Tags (#11418) * regenerate code (#11964) * fix the bug which caused only showing fatal error (#11997) * [Storage][STG73]Address API Review Comments (#12111) * [Storage][STG73]Address API Review Comments * [Storage][STG73]dict<policy, rules> -> list(ObjectReplicationPolicy) * fix blob tag_value test * expose ObjectReplicationPolicy and ObjectReplicationRule, fix test * fix test * Changefeed (#10755) * [ChangeFeed]Add ChangeFeed Package * test_avro failure * update dev_requirement.txt * change namespace to azure.storage.blob.changefeed * address comments * optimize memory when reading changefeed events * namespace change * set up package change * fix failed tests * readme and kwargs * Update sdk/storage/azure-storage-blob-changefeed/azure/storage/blob/changefeed/_change_feed_client.py Co-authored-by: Rakshith Bhyravabhotla <[email protected]> * address comments * 'azure-storage-blob>=12.3.0' which does not match the frozen requirement 'azure-storage-blob~=1.3' Co-authored-by: Rakshith Bhyravabhotla <[email protected]> * [Storage-Blob] Quick Query API (#11991) * Renamed query error * Renamed query reader * Updated config models * Updated format request params * Updated iterator * fix the bug which caused only showing fatal error * Updated Error message * Fixed query helper * Started test conversion * small fix * Fixed tests * Updated error handling + json model * Updated recordings * Removed old recording * Added iter tests * Iter test recordings * Fix test * Remove extra recording * Fix pylint * Some docs cleanup * Renamed iter_records -> iter_stream * Review feedback * Updated tests * Missing commas * Fix syntax * Fix pylint Co-authored-by: xiafu <[email protected]> * tag sas (#12258) * tag sas * disable undelete_container * pylint * skip undelete_container tests * [Blob][Versioning]Disable Versioning Live Test (#12281) * [Blob][QQ]Default output_format to input_format (#12283) * [Storage][Jumbo]Remove super (#12314) * [Storage][JumboBlob]remove empty super() * pypy3 * change sas version to latest * set tags account location to central canada * re-recording queue * changefeed paths generator * mark tests for vid as playback only * fix changefeed * fix pylint make the test account location for tags to central canada * add a delay before calling find_blobs_by_tags * remove tags header * mark a large file test playback only * revert "mark a large file test playback only" skip upload large file test address comment * move tag permission and filter_by_tags permission to kwargs Co-authored-by: Kamil Sobol <[email protected]> Co-authored-by: Ze Qian Zhang <[email protected]> Co-authored-by: Rakshith Bhyravabhotla <[email protected]> Co-authored-by: annatisch <[email protected]>
No description provided.