apache-arrow-glib apache-arrow 12.0.0 by chenrui333 · Pull Request #129859 · Homebrew/homebrew-core

chenrui333 · 2023-05-01T14:56:58Z

Have you followed the guidelines for contributing?
Have you ensured that your commits follow the commit style guide?
Have you checked that there aren't other open pull requests for the same formula update/change?
Have you built your formula locally with brew install --build-from-source <formula>, where <formula> is the name of the formula you're submitting?
Is your test running fine brew test <formula>, where <formula> is the name of the formula you're submitting?
Does your build pass brew audit --strict <formula> (after doing brew install --build-from-source <formula>)? If this is a new formula, does it pass brew audit --new <formula>?

carlocab · 2023-05-01T16:25:30Z

gdal and vineyard need revision bumps.

SMillerDev · 2023-05-02T12:20:41Z

/tmp/vineyard-20230501-64018-1c9lxjq/v6d-0.14.3/modules/basic/ds/arrow.cc:335:30: error: variable type 'memory::VineyardMemoryPool' is an abstract class
    memory::VineyardMemoryPool pool(client);
                               ^
  /opt/homebrew/include/arrow/memory_pool.h:141:19: note: unimplemented pure virtual method 'total_bytes_allocated' in 'VineyardMemoryPool'
    virtual int64_t total_bytes_allocated() const = 0;
                    ^
  /opt/homebrew/include/arrow/memory_pool.h:144:19: note: unimplemented pure virtual method 'num_allocations' in 'VineyardMemoryPool'
    virtual int64_t num_allocations() const = 0;
                    ^
  /tmp/vineyard-20230[501](https://github.com/Homebrew/homebrew-core/actions/runs/4857452242/jobs/8658026410?pr=129859#step:8:502)-64018-1c9lxjq/v6d-0.14.3/modules/basic/ds/arrow.cc:456:30: error: variable type 'memory::VineyardMemoryPool' is an abstract class
    memory::VineyardMemoryPool pool(client);
                               ^
  /tmp/vineyard-20230501-64018-1c9lxjq/v6d-0.14.3/modules/basic/ds/arrow.cc:526:30: error: variable type 'memory::VineyardMemoryPool' is an abstract class
    memory::VineyardMemoryPool pool(client);
                               ^
  /tmp/vineyard-20230501-64018-1c9lxjq/v6d-0.14.3/modules/basic/ds/arrow.cc:595:30: error: variable type 'memory::VineyardMemoryPool' is an abstract class
    memory::VineyardMemoryPool pool(client);
                               ^
  /tmp/vineyard-20230501-64018-1c9lxjq/v6d-0.14.3/modules/basic/ds/arrow.cc:647:30: error: variable type 'memory::VineyardMemoryPool' is an abstract class
    memory::VineyardMemoryPool pool(client);
                               ^

raulcd

Thanks for the PR.
We also have to remove PLASMA and enable ACERO, as discussed on the Mailing list from: https://lists.apache.org/thread/2stklo8dgpgn4dkq2lmvy4483jqcxzgx

$ git diff
diff --git a/Formula/apache-arrow.rb b/Formula/apache-arrow.rb
index f0d3868..520448b 100644
--- a/Formula/apache-arrow.rb
+++ b/Formula/apache-arrow.rb
@@ -46,6 +46,7 @@ class ApacheArrow < Formula
 
     args = %W[
       -DCMAKE_INSTALL_RPATH=#{rpath}
+      -DARROW_ACERO=ON
       -DARROW_COMPUTE=ON
       -DARROW_CSV=ON
       -DARROW_DATASET=ON
@@ -57,7 +58,6 @@ class ApacheArrow < Formula
       -DARROW_JSON=ON
       -DARROW_ORC=ON
       -DARROW_PARQUET=ON
-      -DARROW_PLASMA=ON
       -DARROW_PROTOBUF_USE_SHARED=ON
       -DARROW_S3=ON
       -DARROW_WITH_BZ2=ON

raulcd · 2023-05-04T09:26:08Z

It does seem vineyard hasn't implemented the changes on their VineyardMemoryPool to support arrow 12.0.0:
https://github.com/v6d-io/v6d/blob/a4af21dd23df131415a89d22e4f66987cb1a8f2e/modules/basic/ds/arrow_shim/memory_pool.h#L33
Should we pin arrow 11.0.0 on vineyard?

depends_on "[email protected]"

This is the PR that included those on arrow: apache/arrow#33731
And the current issue on vineyard as pointed above:

   /tmp/vineyard-20230503-55118-14bq3js/v6d-0.14.3/modules/basic/ds/arrow.cc:335:30: error: variable type 'memory::VineyardMemoryPool' is an abstract class
    memory::VineyardMemoryPool pool(client);
                               ^
  /opt/homebrew/include/arrow/memory_pool.h:141:19: note: unimplemented pure virtual method 'total_bytes_allocated' in 'VineyardMemoryPool'
    virtual int64_t total_bytes_allocated() const = 0;
                    ^
  /opt/homebrew/include/arrow/memory_pool.h:144:19: note: unimplemented pure virtual method 'num_allocations' in 'VineyardMemoryPool'
    virtual int64_t num_allocations() const = 0;
                    ^
  /tmp/vineyard-20230503-55118-14bq3js/v6d-0.14.3/modules/basic/ds/arrow.cc:456:30: error: variable type 'memory::VineyardMemoryPool' is an abstract class
    memory::VineyardMemoryPool pool(client);
                               ^
  /tmp/vineyard-20230503-55118-14bq3js/v6d-0.14.3/modules/basic/ds/arrow.cc:526:30: error: variable type 'memory::VineyardMemoryPool' is an abstract class
    memory::VineyardMemoryPool pool(client);
                               ^
  /tmp/vineyard-20230503-55118-14bq3js/v6d-0.14.3/modules/basic/ds/arrow.cc:595:30: error: variable type 'memory::VineyardMemoryPool' is an abstract class
    memory::VineyardMemoryPool pool(client);
                               ^
  /tmp/vineyard-20230503-55118-14bq3js/v6d-0.14.3/modules/basic/ds/arrow.cc:647:30: error: variable type 'memory::VineyardMemoryPool' is an abstract class
    memory::VineyardMemoryPool pool(client);
                               ^

cc @kou

SMillerDev · 2023-05-07T08:30:03Z

Should we pin arrow 11.0.0 on vineyard?

Is that version gonna be supported long term?

kou · 2023-05-08T00:01:12Z

Is that version gonna be supported long term?

Unfortunately, no...

kou · 2023-05-08T02:11:08Z

How about backporting v6d-io/v6d#1359 ?

kou · 2023-05-08T02:12:47Z

0.14.4 may be released soon: v6d-io/v6d@9bc031a

sighingnow · 2023-05-08T02:17:17Z

0.14.4 may be released soon: v6d-io/v6d@9bc031a

I have submitted a pull request to bump up the version: #130424

Signed-off-by: Rui Chen <[email protected]>

sighingnow · 2023-05-08T03:30:54Z

I'm working on the vineyard failure. Sorry for the break.

carlocab · 2023-05-08T03:41:55Z

  In file included from /tmp/vineyard-20230507-94520-18we7kx/v6d-0.14.4/modules/basic/ds/arrow_shim/concatenate.cc:59:
  /opt/homebrew/include/arrow/visit_type_inline.h:54:34: error: no matching member function for call to 'Visit'
      ARROW_GENERATE_FOR_ALL_TYPES(TYPE_VISIT_INLINE);
      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~
  /opt/homebrew/include/arrow/visitor_generate.h:66:3: note: expanded from macro 'ARROW_GENERATE_FOR_ALL_TYPES'
    ACTION(RunEndEncoded);                        \
    ^~~~~~~~~~~~~~~~~~~~~
  /opt/homebrew/include/arrow/visit_type_inline.h:29:21: note: expanded from macro 'TYPE_VISIT_INLINE'
      return visitor->Visit(internal::checked_cast<const TYPE_CLASS##Type&>(type), \
             ~~~~~~~~~^~~~~
  /tmp/vineyard-20230507-94520-18we7kx/v6d-0.14.4/modules/basic/ds/arrow_shim/concatenate.cc:247:19: note: in instantiation of function template specialization 'arrow::VisitTypeInline<vineyard::arrow_shim::(anonymous namespace)::ConcatenateImpl>' requested here
      RETURN_NOT_OK(VisitTypeInline(*out_->type, this));
                    ^
  /tmp/vineyard-20230507-94520-18we7kx/v6d-0.14.4/modules/basic/ds/arrow_shim/concatenate.cc:252:10: note: candidate function not viable: no known conversion from 'const arrow::RunEndEncodedType' to 'const arrow::NullType' for 1st argument
    Status Visit(const NullType&) { return Status::OK(); }
           ^
  /tmp/vineyard-20230507-94520-18we7kx/v6d-0.14.4/modules/basic/ds/arrow_shim/concatenate.cc:254:10: note: candidate function not viable: no known conversion from 'const arrow::RunEndEncodedType' to 'const arrow::BooleanType' for 1st argument
    Status Visit(const BooleanType&) {
           ^
  /tmp/vineyard-20230507-94520-18we7kx/v6d-0.14.4/modules/basic/ds/arrow_shim/concatenate.cc:258:10: note: candidate function not viable: no known conversion from 'const arrow::RunEndEncodedType' to 'const arrow::FixedWidthType' for 1st argument
    Status Visit(const FixedWidthType& fixed) {
           ^
  /tmp/vineyard-20230507-94520-18we7kx/v6d-0.14.4/modules/basic/ds/arrow_shim/concatenate.cc:265:10: note: candidate function not viable: no known conversion from 'const arrow::RunEndEncodedType' to 'const arrow::BinaryType' for 1st argument
    Status Visit(const BinaryType&) {
           ^
  /tmp/vineyard-20230507-94520-18we7kx/v6d-0.14.4/modules/basic/ds/arrow_shim/concatenate.cc:275:10: note: candidate function not viable: no known conversion from 'const arrow::RunEndEncodedType' to 'const arrow::LargeBinaryType' for 1st argument
    Status Visit(const LargeBinaryType&) {
           ^
  /tmp/vineyard-20230507-94520-18we7kx/v6d-0.14.4/modules/basic/ds/arrow_shim/concatenate.cc:285:10: note: candidate function not viable: no known conversion from 'const arrow::RunEndEncodedType' to 'const arrow::ListType' for 1st argument
    Status Visit(const ListType&) {
           ^
  /tmp/vineyard-20230507-94520-18we7kx/v6d-0.14.4/modules/basic/ds/arrow_shim/concatenate.cc:295:10: note: candidate function not viable: no known conversion from 'const arrow::RunEndEncodedType' to 'const arrow::LargeListType' for 1st argument
    Status Visit(const LargeListType&) {
           ^
  /tmp/vineyard-20230507-94520-18we7kx/v6d-0.14.4/modules/basic/ds/arrow_shim/concatenate.cc:305:10: note: candidate function not viable: no known conversion from 'const arrow::RunEndEncodedType' to 'const arrow::FixedSizeListType' for 1st argument
    Status Visit(const FixedSizeListType& fixed_size_list) {
           ^
  /tmp/vineyard-20230507-94520-18we7kx/v6d-0.14.4/modules/basic/ds/arrow_shim/concatenate.cc:312:10: note: candidate function not viable: no known conversion from 'const arrow::RunEndEncodedType' to 'const arrow::StructType' for 1st argument
    Status Visit(const StructType& s) {
           ^
  /tmp/vineyard-20230507-94520-18we7kx/v6d-0.14.4/modules/basic/ds/arrow_shim/concatenate.cc:391:10: note: candidate function not viable: no known conversion from 'const arrow::RunEndEncodedType' to 'const arrow::DictionaryType' for 1st argument
    Status Visit(const DictionaryType& d) {
           ^
  /tmp/vineyard-20230507-94520-18we7kx/v6d-0.14.4/modules/basic/ds/arrow_shim/concatenate.cc:419:10: note: candidate function not viable: no known conversion from 'const arrow::RunEndEncodedType' to 'const arrow::UnionType' for 1st argument
    Status Visit(const UnionType& u) {
           ^
  /tmp/vineyard-20230507-94520-18we7kx/v6d-0.14.4/modules/basic/ds/arrow_shim/concatenate.cc:509:10: note: candidate function not viable: no known conversion from 'const arrow::RunEndEncodedType' to 'const arrow::ExtensionType' for 1st argument
    Status Visit(const ExtensionType& e) {
           ^
  1 error generated.
  make[2]: *** [modules/basic/CMakeFiles/vineyard_basic.dir/ds/arrow_shim/concatenate.cc.o] Error 1
  make[1]: *** [modules/basic/CMakeFiles/vineyard_basic.dir/all] Error 2

https://github.com/Homebrew/homebrew-core/actions/runs/4910846169/jobs/8768426022?pr=129859#step:3:502

Related issue number -------------------- See also: - Homebrew/homebrew-core#129859 - apache/arrow#33608 - apache/arrow#33641 Signed-off-by: Tao He <[email protected]>

sighingnow · 2023-05-08T05:34:12Z

I have published a quick bugfix release. Could someone help me to cherry-pick #130429 to this pull request?

raulcd · 2023-05-08T06:50:51Z

I am not sure why the jobs were cancelled. Could it be timeout after 45 minutes? I recall we usually add the long-ci-build labels when adding new versions.

github-actions · 2023-05-08T10:31:58Z

🤖 An automated task has requested bottles to be published to this PR.

carlocab added revision bumps needed Reverse dependencies need to have their revision incremented in the same PR and removed revision bumps needed Reverse dependencies need to have their revision incremented in the same PR labels May 1, 2023

raulcd reviewed May 3, 2023

View reviewed changes

carlocab force-pushed the apache-arrow-glib-apache-arrow-12.0.0 branch from 1b78fa9 to ca4fc69 Compare May 3, 2023 10:09

raulcd mentioned this pull request May 4, 2023

Vineyard fails to compile with Arrow 12.0.0 v6d-io/v6d#1357

Closed

carlocab force-pushed the apache-arrow-glib-apache-arrow-12.0.0 branch from ca4fc69 to f52165e Compare May 8, 2023 02:40

chenrui333 added 2 commits May 8, 2023 10:41

apache-arrow 12.0.0

67e9650

Signed-off-by: Rui Chen <[email protected]>

apache-arrow-glib 12.0.0

3c1f586

Signed-off-by: Rui Chen <[email protected]>

carlocab force-pushed the apache-arrow-glib-apache-arrow-12.0.0 branch from f52165e to 2a40362 Compare May 8, 2023 02:41

carlocab mentioned this pull request May 8, 2023

vineyard 0.14.4 #130424

Closed

sighingnow mentioned this pull request May 8, 2023

Adapt to arrow-11/12 (VisitTypeInline & Run-End encoded data) v6d-io/v6d#1362

Merged

sighingnow and others added 2 commits May 8, 2023 13:47

vineyard 0.14.5

a201c4b

gdal: revision bump (apache-arrow 12.0.0)

aff2718

carlocab force-pushed the apache-arrow-glib-apache-arrow-12.0.0 branch from 2a40362 to aff2718 Compare May 8, 2023 05:47

carlocab added CI-long-timeout Use longer GitHub Actions CI timeout. long build Set a long timeout for formula testing labels May 8, 2023

fxcoudert approved these changes May 8, 2023

View reviewed changes

github-actions Bot removed the CI-long-timeout Use longer GitHub Actions CI timeout. label May 8, 2023

BrewTestBot added 4 commits May 8, 2023 10:33

apache-arrow: update 12.0.0 bottle.

305522b

gdal: update 3.6.4_4 bottle.

e5b9572

apache-arrow-glib: update 12.0.0 bottle.

377abbc

vineyard: update 0.14.5 bottle.

027ff04

github-actions Bot added the CI-published-bottle-commits The commits for the built bottles have been pushed to the PR branch. label May 8, 2023

github-actions Bot approved these changes May 8, 2023

View reviewed changes

BrewTestBot enabled auto-merge May 8, 2023 10:36

BrewTestBot added this pull request to the merge queue May 8, 2023

Merged via the queue into Homebrew:master with commit bf717e9 May 8, 2023

chenrui333 mentioned this pull request May 8, 2023

vineyard 0.14.5 #130429

Closed

chenrui333 deleted the apache-arrow-glib-apache-arrow-12.0.0 branch May 8, 2023 12:04

Conversation

chenrui333 commented May 1, 2023

Uh oh!

carlocab commented May 1, 2023

Uh oh!

SMillerDev commented May 2, 2023

Uh oh!

raulcd left a comment • edited by carlocab Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

raulcd commented May 4, 2023

Uh oh!

SMillerDev commented May 7, 2023

Uh oh!

kou commented May 8, 2023

Uh oh!

kou commented May 8, 2023

Uh oh!

kou commented May 8, 2023

Uh oh!

sighingnow commented May 8, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sighingnow commented May 8, 2023

Uh oh!

carlocab commented May 8, 2023

Uh oh!

sighingnow commented May 8, 2023

Uh oh!

raulcd commented May 8, 2023

Uh oh!

github-actions Bot commented May 8, 2023

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

raulcd left a comment •

edited by carlocab

Loading

sighingnow commented May 8, 2023 •

edited

Loading