Repository navigation
[Feature] Add support for arrow flight sql #19812
Description
Search before asking
- I had searched in the issues and found no similar issues.
Description
Currently, there are many systems want to pull millions of data from doris and do analysis such as Data Science, AI Platform. MySQL protocol is very slow, Most of the time more than 80% of the total time spent in accessing data is elapsed in serialization/de-serialization step. We need a more high throughput protocol.
Arrow flight is a very popular wired protocol and arrow flight sql is a sql wrapper for SQL based system. I think we could do some investigation and implement it.
Use case
No response
Related issues
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct
Reacted by Yuriy Gavrilov
Activity
yiguolei commentedon May 18, 2023
https://arrow.apache.org/blog/2019/10/13/introducing-arrow-flight/
https://arrow.apache.org/docs/format/Flight.html
yiguolei commentedon May 18, 2023
There is some benchmark for arrow flight. https://dl.acm.org/doi/pdf/10.1145/3527199.3527264
yiguolei commentedon May 18, 2023
apache/arrow#35664
liangjiawei1110 commentedon May 18, 2023
I'd like to try it.
yiguolei commentedon May 19, 2023
@liangjiawei1110 great
YuriyGavrilov commentedon Nov 14, 2023
+1