Python: Bump PyArrow to 12.0.0 by Fokko · Pull Request #7588 · apache/iceberg

Fokko · 2023-05-11T20:31:27Z

This includes the fix apache/arrow#34015

The patch reduces the number of calls from 38 to 33, less is more!

Now:

[root@fa7688db87e9 /]# mc admin trace --call s3 minio
2023-05-11T20:06:32.268 [206 Partial Content] s3.GetObject warehouse.minio:9000/nyc/taxis/metadata/00004-5eeae12f-6f3b-4c36-9413-2fbef26a94b7.metadata.json 172.22.0.2        2.791ms      ↑ 148 B ↓ 9.6 KiB
2023-05-11T20:06:32.558 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/snap-3831732930279655857-1-9f7fb22f-eb4a-42d2-9657-87cc8039004d.avro 172.22.0.1        2.379ms      ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.566 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/snap-3831732930279655857-1-9f7fb22f-eb4a-42d2-9657-87cc8039004d.avro 172.22.0.1        1.925ms      ↑ 159 B ↓ 3.9 KiB
2023-05-11T20:06:32.583 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/8e396fa9-dc45-4998-8d37-38665bf45724-m0.avro 172.22.0.1        528µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.584 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/9f7fb22f-eb4a-42d2-9657-87cc8039004d-m0.avro 172.22.0.1        347µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.584 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/273e2d75-5d3a-4c2c-bebd-a6ef37cadbe2-m0.avro 172.22.0.1        684µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.584 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/5f6429bd-1ae4-4a54-820e-f53387e04e2b-m0.avro 172.22.0.1        502µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.584 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/8f666929-42ff-4198-a203-af5c85cfb233-m0.avro 172.22.0.1        962µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.589 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/5f6429bd-1ae4-4a54-820e-f53387e04e2b-m0.avro 172.22.0.1        1.805ms      ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:06:32.589 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/8e396fa9-dc45-4998-8d37-38665bf45724-m0.avro 172.22.0.1        1.54ms       ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:06:32.588 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/9f7fb22f-eb4a-42d2-9657-87cc8039004d-m0.avro 172.22.0.1        2.27ms       ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:06:32.589 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/273e2d75-5d3a-4c2c-bebd-a6ef37cadbe2-m0.avro 172.22.0.1        1.239ms      ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:06:32.589 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/8f666929-42ff-4198-a203-af5c85cfb233-m0.avro 172.22.0.1        2.783ms      ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:06:32.618 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1        445µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.618 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1        319µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.618 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1        263µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.618 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1        929µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.618 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1        275µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.622 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1        2.981ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:06:32.622 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1        6.991ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:06:32.624 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1        4.944ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:06:32.624 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1        6.287ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:06:32.622 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1        9.967ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:06:32.641 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1        95.637ms     ↑ 159 B ↓ 6.6 MiB
2023-05-11T20:06:32.638 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1        711.24ms     ↑ 159 B ↓ 15 MiB
2023-05-11T20:06:32.638 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1        767.182ms    ↑ 159 B ↓ 16 MiB
2023-05-11T20:06:32.640 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1        1.694288s    ↑ 159 B ↓ 27 MiB
2023-05-11T20:06:32.640 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1        1.748438s    ↑ 159 B ↓ 27 MiB
2023-05-11T20:06:32.640 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1        1.895823s    ↑ 159 B ↓ 29 MiB
2023-05-11T20:06:32.638 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1        1.925349s    ↑ 159 B ↓ 32 MiB
2023-05-11T20:06:32.644 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1        1.944678s    ↑ 159 B ↓ 30 MiB
2023-05-11T20:06:33.200 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1        1.510093s    ↑ 159 B ↓ 27 MiB
2023-05-11T20:06:33.972 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1        1.326988s    ↑ 159 B ↓ 26 MiB

Before:

^C[root@fa7688db87e9 /]# mc admin trace --call s3 minio
2023-05-11T20:19:09.894 [206 Partial Content] s3.GetObject warehouse.minio:9000/nyc/taxis/metadata/00004-5eeae12f-6f3b-4c36-9413-2fbef26a94b7.metadata.json 172.22.0.2        897µs       ↑ 148 B ↓ 9.6 KiB
2023-05-11T20:19:11.092 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/snap-3831732930279655857-1-9f7fb22f-eb4a-42d2-9657-87cc8039004d.avro 172.22.0.1        2.395ms      ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.113 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/snap-3831732930279655857-1-9f7fb22f-eb4a-42d2-9657-87cc8039004d.avro 172.22.0.1        1.439ms      ↑ 159 B ↓ 3.9 KiB
2023-05-11T20:19:11.135 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/5f6429bd-1ae4-4a54-820e-f53387e04e2b-m0.avro 172.22.0.1        1.135ms      ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.136 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/273e2d75-5d3a-4c2c-bebd-a6ef37cadbe2-m0.avro 172.22.0.1        376µs       ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.136 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/8f666929-42ff-4198-a203-af5c85cfb233-m0.avro 172.22.0.1        465µs       ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.136 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/8e396fa9-dc45-4998-8d37-38665bf45724-m0.avro 172.22.0.1        841µs       ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.135 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/9f7fb22f-eb4a-42d2-9657-87cc8039004d-m0.avro 172.22.0.1        2.065ms      ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.143 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/9f7fb22f-eb4a-42d2-9657-87cc8039004d-m0.avro 172.22.0.1        1.153ms      ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:19:11.144 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/8e396fa9-dc45-4998-8d37-38665bf45724-m0.avro 172.22.0.1        1.105ms      ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:19:11.143 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/8f666929-42ff-4198-a203-af5c85cfb233-m0.avro 172.22.0.1        2.29ms       ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:19:11.143 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/5f6429bd-1ae4-4a54-820e-f53387e04e2b-m0.avro 172.22.0.1        2.646ms      ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:19:11.144 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/273e2d75-5d3a-4c2c-bebd-a6ef37cadbe2-m0.avro 172.22.0.1        1.914ms      ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:19:11.176 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1        368µs       ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.176 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1        380µs       ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.176 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1        280µs       ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.177 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1        255µs       ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.177 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1        433µs       ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.180 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1        4.36ms       ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.182 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1        3.465ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.180 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1        6.57ms       ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.180 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1        7.434ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.182 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1        6.438ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.196 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1        3.827ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.196 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1        3.971ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.194 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1        5.977ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.198 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1        7.024ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.196 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1        9.302ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.213 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1        69.011ms     ↑ 159 B ↓ 6.6 MiB
2023-05-11T20:19:11.211 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1        685.555ms    ↑ 159 B ↓ 15 MiB
2023-05-11T20:19:11.213 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1        886.157ms    ↑ 159 B ↓ 16 MiB
2023-05-11T20:19:11.211 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1        1.711973s    ↑ 159 B ↓ 27 MiB
2023-05-11T20:19:11.213 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1        1.740533s    ↑ 159 B ↓ 27 MiB
2023-05-11T20:19:11.211 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1        1.754817s    ↑ 159 B ↓ 29 MiB
2023-05-11T20:19:11.209 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1        1.960224s    ↑ 159 B ↓ 30 MiB
2023-05-11T20:19:11.211 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1        2.066043s    ↑ 159 B ↓ 32 MiB
2023-05-11T20:19:11.784 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1        1.618623s    ↑ 159 B ↓ 27 MiB
2023-05-11T20:19:12.526 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1        1.407968s    ↑ 159 B ↓ 26 MiB

This includes the fix apache/arrow#34015 Now: ``` [root@fa7688db87e9 /]# mc admin trace --call s3 minio 2023-05-11T20:06:32.268 [206 Partial Content] s3.GetObject warehouse.minio:9000/nyc/taxis/metadata/00004-5eeae12f-6f3b-4c36-9413-2fbef26a94b7.metadata.json 172.22.0.2 2.791ms ↑ 148 B ↓ 9.6 KiB 2023-05-11T20:06:32.558 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/snap-3831732930279655857-1-9f7fb22f-eb4a-42d2-9657-87cc8039004d.avro 172.22.0.1 2.379ms ↑ 153 B ↓ 0 B 2023-05-11T20:06:32.566 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/snap-3831732930279655857-1-9f7fb22f-eb4a-42d2-9657-87cc8039004d.avro 172.22.0.1 1.925ms ↑ 159 B ↓ 3.9 KiB 2023-05-11T20:06:32.583 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/8e396fa9-dc45-4998-8d37-38665bf45724-m0.avro 172.22.0.1 528µs ↑ 153 B ↓ 0 B 2023-05-11T20:06:32.584 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/9f7fb22f-eb4a-42d2-9657-87cc8039004d-m0.avro 172.22.0.1 347µs ↑ 153 B ↓ 0 B 2023-05-11T20:06:32.584 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/273e2d75-5d3a-4c2c-bebd-a6ef37cadbe2-m0.avro 172.22.0.1 684µs ↑ 153 B ↓ 0 B 2023-05-11T20:06:32.584 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/5f6429bd-1ae4-4a54-820e-f53387e04e2b-m0.avro 172.22.0.1 502µs ↑ 153 B ↓ 0 B 2023-05-11T20:06:32.584 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/8f666929-42ff-4198-a203-af5c85cfb233-m0.avro 172.22.0.1 962µs ↑ 153 B ↓ 0 B 2023-05-11T20:06:32.589 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/5f6429bd-1ae4-4a54-820e-f53387e04e2b-m0.avro 172.22.0.1 1.805ms ↑ 159 B ↓ 7.1 KiB 2023-05-11T20:06:32.589 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/8e396fa9-dc45-4998-8d37-38665bf45724-m0.avro 172.22.0.1 1.54ms ↑ 159 B ↓ 7.1 KiB 2023-05-11T20:06:32.588 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/9f7fb22f-eb4a-42d2-9657-87cc8039004d-m0.avro 172.22.0.1 2.27ms ↑ 159 B ↓ 7.1 KiB 2023-05-11T20:06:32.589 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/273e2d75-5d3a-4c2c-bebd-a6ef37cadbe2-m0.avro 172.22.0.1 1.239ms ↑ 159 B ↓ 7.1 KiB 2023-05-11T20:06:32.589 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/8f666929-42ff-4198-a203-af5c85cfb233-m0.avro 172.22.0.1 2.783ms ↑ 159 B ↓ 7.1 KiB 2023-05-11T20:06:32.618 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1 445µs ↑ 153 B ↓ 0 B 2023-05-11T20:06:32.618 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1 319µs ↑ 153 B ↓ 0 B 2023-05-11T20:06:32.618 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1 263µs ↑ 153 B ↓ 0 B 2023-05-11T20:06:32.618 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1 929µs ↑ 153 B ↓ 0 B 2023-05-11T20:06:32.618 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1 275µs ↑ 153 B ↓ 0 B 2023-05-11T20:06:32.622 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1 2.981ms ↑ 159 B ↓ 64 KiB 2023-05-11T20:06:32.622 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1 6.991ms ↑ 159 B ↓ 64 KiB 2023-05-11T20:06:32.624 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1 4.944ms ↑ 159 B ↓ 64 KiB 2023-05-11T20:06:32.624 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1 6.287ms ↑ 159 B ↓ 64 KiB 2023-05-11T20:06:32.622 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1 9.967ms ↑ 159 B ↓ 64 KiB 2023-05-11T20:06:32.641 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1 95.637ms ↑ 159 B ↓ 6.6 MiB 2023-05-11T20:06:32.638 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1 711.24ms ↑ 159 B ↓ 15 MiB 2023-05-11T20:06:32.638 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1 767.182ms ↑ 159 B ↓ 16 MiB 2023-05-11T20:06:32.640 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1 1.694288s ↑ 159 B ↓ 27 MiB 2023-05-11T20:06:32.640 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1 1.748438s ↑ 159 B ↓ 27 MiB 2023-05-11T20:06:32.640 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1 1.895823s ↑ 159 B ↓ 29 MiB 2023-05-11T20:06:32.638 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1 1.925349s ↑ 159 B ↓ 32 MiB 2023-05-11T20:06:32.644 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1 1.944678s ↑ 159 B ↓ 30 MiB 2023-05-11T20:06:33.200 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1 1.510093s ↑ 159 B ↓ 27 MiB 2023-05-11T20:06:33.972 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1 1.326988s ↑ 159 B ↓ 26 MiB ``` Before: ``` ^C[root@fa7688db87e9 /]# mc admin trace --call s3 minio 2023-05-11T20:19:09.894 [206 Partial Content] s3.GetObject warehouse.minio:9000/nyc/taxis/metadata/00004-5eeae12f-6f3b-4c36-9413-2fbef26a94b7.metadata.json 172.22.0.2 897µs ↑ 148 B ↓ 9.6 KiB 2023-05-11T20:19:11.092 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/snap-3831732930279655857-1-9f7fb22f-eb4a-42d2-9657-87cc8039004d.avro 172.22.0.1 2.395ms ↑ 153 B ↓ 0 B 2023-05-11T20:19:11.113 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/snap-3831732930279655857-1-9f7fb22f-eb4a-42d2-9657-87cc8039004d.avro 172.22.0.1 1.439ms ↑ 159 B ↓ 3.9 KiB 2023-05-11T20:19:11.135 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/5f6429bd-1ae4-4a54-820e-f53387e04e2b-m0.avro 172.22.0.1 1.135ms ↑ 153 B ↓ 0 B 2023-05-11T20:19:11.136 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/273e2d75-5d3a-4c2c-bebd-a6ef37cadbe2-m0.avro 172.22.0.1 376µs ↑ 153 B ↓ 0 B 2023-05-11T20:19:11.136 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/8f666929-42ff-4198-a203-af5c85cfb233-m0.avro 172.22.0.1 465µs ↑ 153 B ↓ 0 B 2023-05-11T20:19:11.136 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/8e396fa9-dc45-4998-8d37-38665bf45724-m0.avro 172.22.0.1 841µs ↑ 153 B ↓ 0 B 2023-05-11T20:19:11.135 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/9f7fb22f-eb4a-42d2-9657-87cc8039004d-m0.avro 172.22.0.1 2.065ms ↑ 153 B ↓ 0 B 2023-05-11T20:19:11.143 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/9f7fb22f-eb4a-42d2-9657-87cc8039004d-m0.avro 172.22.0.1 1.153ms ↑ 159 B ↓ 7.1 KiB 2023-05-11T20:19:11.144 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/8e396fa9-dc45-4998-8d37-38665bf45724-m0.avro 172.22.0.1 1.105ms ↑ 159 B ↓ 7.1 KiB 2023-05-11T20:19:11.143 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/8f666929-42ff-4198-a203-af5c85cfb233-m0.avro 172.22.0.1 2.29ms ↑ 159 B ↓ 7.1 KiB 2023-05-11T20:19:11.143 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/5f6429bd-1ae4-4a54-820e-f53387e04e2b-m0.avro 172.22.0.1 2.646ms ↑ 159 B ↓ 7.1 KiB 2023-05-11T20:19:11.144 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/273e2d75-5d3a-4c2c-bebd-a6ef37cadbe2-m0.avro 172.22.0.1 1.914ms ↑ 159 B ↓ 7.1 KiB 2023-05-11T20:19:11.176 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1 368µs ↑ 153 B ↓ 0 B 2023-05-11T20:19:11.176 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1 380µs ↑ 153 B ↓ 0 B 2023-05-11T20:19:11.176 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1 280µs ↑ 153 B ↓ 0 B 2023-05-11T20:19:11.177 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1 255µs ↑ 153 B ↓ 0 B 2023-05-11T20:19:11.177 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1 433µs ↑ 153 B ↓ 0 B 2023-05-11T20:19:11.180 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1 4.36ms ↑ 159 B ↓ 64 KiB 2023-05-11T20:19:11.182 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1 3.465ms ↑ 159 B ↓ 64 KiB 2023-05-11T20:19:11.180 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1 6.57ms ↑ 159 B ↓ 64 KiB 2023-05-11T20:19:11.180 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1 7.434ms ↑ 159 B ↓ 64 KiB 2023-05-11T20:19:11.182 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1 6.438ms ↑ 159 B ↓ 64 KiB 2023-05-11T20:19:11.196 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1 3.827ms ↑ 159 B ↓ 64 KiB 2023-05-11T20:19:11.196 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1 3.971ms ↑ 159 B ↓ 64 KiB 2023-05-11T20:19:11.194 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1 5.977ms ↑ 159 B ↓ 64 KiB 2023-05-11T20:19:11.198 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1 7.024ms ↑ 159 B ↓ 64 KiB 2023-05-11T20:19:11.196 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1 9.302ms ↑ 159 B ↓ 64 KiB 2023-05-11T20:19:11.213 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1 69.011ms ↑ 159 B ↓ 6.6 MiB 2023-05-11T20:19:11.211 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1 685.555ms ↑ 159 B ↓ 15 MiB 2023-05-11T20:19:11.213 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1 886.157ms ↑ 159 B ↓ 16 MiB 2023-05-11T20:19:11.211 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1 1.711973s ↑ 159 B ↓ 27 MiB 2023-05-11T20:19:11.213 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1 1.740533s ↑ 159 B ↓ 27 MiB 2023-05-11T20:19:11.211 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1 1.754817s ↑ 159 B ↓ 29 MiB 2023-05-11T20:19:11.209 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1 1.960224s ↑ 159 B ↓ 30 MiB 2023-05-11T20:19:11.211 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1 2.066043s ↑ 159 B ↓ 32 MiB 2023-05-11T20:19:11.784 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1 1.618623s ↑ 159 B ↓ 27 MiB 2023-05-11T20:19:12.526 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1 1.407968s ↑ 159 B ↓ 26 MiB ```

singhpk234

This is Great !!

JonasJ-ap

Thanks!

Fokko · 2023-05-12T10:25:48Z

@jorisvandenbossche You might like this one as well :)

Fokko · 2023-05-12T12:26:54Z

Thanks @nastra, @JonasJ-ap and @singhpk234 for the review!

Fokko added this to the PyIceberg 0.4.0 release milestone May 11, 2023

github-actions bot added the python label May 11, 2023

singhpk234 approved these changes May 11, 2023

View reviewed changes

JonasJ-ap approved these changes May 12, 2023

View reviewed changes

nastra approved these changes May 12, 2023

View reviewed changes

Fokko merged commit 852c1bf into apache:master May 12, 2023

Fokko deleted the fd-pyarrrrrow branch May 12, 2023 12:26

ajantha-bhat mentioned this pull request May 12, 2023

Build: Bump Arrow from 11.0.0 to 12.0.0 #7595

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Python: Bump PyArrow to 12.0.0#7588

Python: Bump PyArrow to 12.0.0#7588
Fokko merged 1 commit intoapache:masterfrom
Fokko:fd-pyarrrrrow

Fokko commented May 11, 2023

Uh oh!

singhpk234 left a comment

Uh oh!

JonasJ-ap left a comment

Uh oh!

Fokko commented May 12, 2023

Uh oh!

Fokko commented May 12, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

Fokko commented May 11, 2023

Uh oh!

singhpk234 left a comment

Choose a reason for hiding this comment

Uh oh!

JonasJ-ap left a comment

Choose a reason for hiding this comment

Uh oh!

Fokko commented May 12, 2023

Uh oh!

Fokko commented May 12, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants