roapi icon indicating copy to clipboard operation
roapi copied to clipboard

database support

Open geoHeil opened this issue 3 years ago • 7 comments

For 0.7.1 and a configuration of:

tables:
  - name: "table_foo"
    uri: "mysql://username:password@localhost:3306/database"
    #uri: "table_foo=mysql://username:password@localhost:3306/database"

In any case I get:

 Invalid table URI: cannot detect table extension from uri: mysql://username:password@server:port/databas

back as a result - how can I tell roapi to actually use either postgres or MySQL? In both cases the table is not identified

geoHeil avatar Jul 18 '22 20:07 geoHeil

@houqp I was trying to work with the provided examples - but could not get roapi to load from Postgres or from MySQL.

geoHeil avatar Jul 21 '22 14:07 geoHeil

were you able to get it working with cli arguments? I can try to reproduce with your config file later this weekend.

houqp avatar Jul 22 '22 05:07 houqp

This:

roapi --addr-http 0.0.0.0:8080 --table "table_name=mysql://user:pass@localhost/db"

fails for me with:

Error: Database error: Enable 'database' feature flag to support this

I am using roapi from homebrew with version: 0.7.1.

geoHeil avatar Jul 22 '22 09:07 geoHeil

I do observe the same error from the roapi docker container. Do you think you could publish the binary with DB support enabled?

geoHeil avatar Jul 26 '22 15:07 geoHeil

related/duplicate of https://github.com/roapi/roapi/issues/173

geoHeil avatar Jul 26 '22 15:07 geoHeil

Even after compiling my own version of roapi with database support:

 cannot detect table extension from uri

persists.

When using the --table it works then.

geoHeil avatar Jul 27 '22 12:07 geoHeil

Thanks @geoHeil for the report, it should be fixed by https://github.com/roapi/roapi/pull/183

houqp avatar Aug 08 '22 07:08 houqp

Same error as 0.8.0 installed by homebrew in macOS

Error: Database error: Enable 'database' feature flag to support this

My env

OS          -  macOS 11.6.7 Big Sur
Terminal    -  wezterm
Shell       -  zsh
HOMEBREW_VERSION: 3.6.3

HW

Intel® Core™ i7-9750H CPU @ 2.60GHz (12)
CPU Load    -  16%
Memory      -  7.8 GB/16.8 GB

leopku avatar Sep 28 '22 08:09 leopku

@leopku I think the homebrew formulae needs to be added to add the database flag since it's not enabled by default.

houqp avatar Sep 29 '22 06:09 houqp

https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/roapi.rb#L22 will require setting https://github.com/roapi/roapi/pull/182/files#diff-8aa42ebe3a4357d4bac56925d8eda99a0a36556b008e4b198a819c1a44ee2642R143 i.e. the features of:

features: "simd,rustls,database"

geoHeil avatar Sep 29 '22 16:09 geoHeil

@leopku https://github.com/Homebrew/homebrew-core/pull/111999 I have created this PR - can you help testing it?

geoHeil avatar Sep 29 '22 16:09 geoHeil

@geoHeil Sure。

Reinstall failed because of packed_simd_2. (Intel CPU)

brew reinstall --build-from-source --formula ./ruby/homebrew-core/Formula/roapi.rb
==> Downloading https://github.com/roapi/roapi/archive/refs/tags/roapi-http-v0.8.0.tar.gz
Already downloaded: /Users/leo/Library/Caches/Homebrew/downloads/cd20ae0bdcc0c4293f612141adea9dabe6583ea88e76a6b338357b710b0abedd--roapi-roapi-http-v0.8.0.tar.gz
==> Reinstalling roapi
==> cargo install --no-default-features --features simd,rustls,database roapi
Last 15 lines from /Users/leo/Library/Logs/Homebrew/roapi/01.cargo:
   Compiling crypto-common v0.1.6
   Compiling block-buffer v0.10.3
   Compiling digest v0.9.0
   Compiling block-buffer v0.9.0
   Compiling crypto-mac v0.11.1
   Compiling lexical-core v0.8.5
   Compiling url v2.3.1
   Compiling crossbeam-deque v0.8.2
   Compiling pem v0.8.3
   Compiling env_logger v0.9.1
   Compiling env_logger v0.8.4
For more information about this error, try `rustc --explain E0554`.
error: could not compile `packed_simd_2` due to 3 previous errors
warning: build failed, waiting for other jobs to finish...
error: failed to compile `roapi v0.8.0 (/private/tmp/roapi-20220930-7642-cx9wzb/roapi-roapi-http-v0.8.0/roapi)`, intermediate artifacts can be found at `/private/tmp/roapi-20220930-7642-cx9wzb/roapi-roapi-http-v0.8.0/target`

Do not report this issue to Homebrew/brew or Homebrew/core!

Sorry for not family with rust-lang.

leopku avatar Sep 30 '22 02:09 leopku

https://github.com/Homebrew/homebrew-core/blob/master/CONTRIBUTING.md

brew uninstall --force roapi
brew install --build-from-source roapi
brew test foo
brew audit --strict roapi
brew style roapi

would have been the instructions.

No you are using the wrong version of rust I guess. You must use Rust nightly of some specific date.

geoHeil avatar Sep 30 '22 11:09 geoHeil

nightly-2022-01-03 I believe.

https://github.com/roapi/roapi/blob/main/Dockerfile#L1 is using a newer version by now:

2022-09-24

geoHeil avatar Sep 30 '22 12:09 geoHeil

@houqp it looks like apple silicon is failing.

@leopku was trying rust stable and failed.

https://github.com/Homebrew/homebrew-core/blob/2ec9befee47baa451fdad5f66c94134ad5ab69fd/Formula/roapi.rb#L17 looks like the brew formula is also using stable - I guess the nightly version must be chosen there as well.

  error[E0554]: `#![feature]` may not be used on the stable release channel

https://github.com/Homebrew/homebrew-core/actions/runs/3152983900/jobs/5128991056#step:6:874

@leopku do you think you can update the PR to use rust nightly instead?

geoHeil avatar Sep 30 '22 12:09 geoHeil

@geoHeil Still failed.

rustc --version
rustc 1.66.0-nightly (9c56d9d6f 2022-09-29)

cargo --version
cargo 1.66.0-nightly (f5fed93ba 2022-09-27)
brew reinstall --build-from-source --formula ./Formula/roapi.rb
==> Downloading https://github.com/roapi/roapi/archive/refs/tags/roapi-http-v0.8.0.tar.gz
Already downloaded: /Users/leo/Library/Caches/Homebrew/downloads/cd20ae0bdcc0c4293f612141adea9dabe6583ea88e76a6b338357b710b0abedd--roapi-roapi-http-v0.8.0.tar.gz
==> Reinstalling roapi
==> cargo install --no-default-features --features simd,rustls,database roapi
Last 15 lines from /Users/leo/Library/Logs/Homebrew/roapi/01.cargo:
   Compiling rusoto_core v0.47.0
   Compiling lexical-core v0.8.5
   Compiling block-buffer v0.10.3
   Compiling crypto-common v0.1.6
   Compiling digest v0.9.0
   Compiling block-buffer v0.9.0
   Compiling crypto-mac v0.11.1
   Compiling url v2.3.1
   Compiling crossbeam-deque v0.8.2
   Compiling pem v0.8.3
   Compiling env_logger v0.9.1
For more information about this error, try `rustc --explain E0554`.
error: could not compile `packed_simd_2` due to 3 previous errors
warning: build failed, waiting for other jobs to finish...
error: failed to compile `roapi v0.8.0 (/private/tmp/roapi-20220930-6995-gnxv0x/roapi-roapi-http-v0.8.0/roapi)`, intermediate artifacts can be found at `/private/tmp/roapi-20220930-6995-gnxv0x/roapi-roapi-http-v0.8.0/target`

Do not report this issue to Homebrew/brew or Homebrew/core!

leopku avatar Sep 30 '22 15:09 leopku

@leopku 1) does your manual approach without brew work now 2) indeed brew is not yet updated to use nightly - unless you have contributed it to the MR as outlined in https://github.com/roapi/roapi/issues/178#issuecomment-1263486228 do you think you could contribute this to the brew formula?

geoHeil avatar Sep 30 '22 15:09 geoHeil

Just back from holiday. Sorry for reply so late.

I could not build from source.

Cloning this repo and running cargo build --release --features "simd,database", got result as below:

error: failed to run custom build command for `snmalloc-sys v0.2.28`

Caused by:
  process didn't exit successfully: `/Users/leo/Project/rust/roapi/target/release/build/snmalloc-sys-69dadb740668e3db/build-script-build` (exit status: 101)
  --- stdout
  CMAKE_TOOLCHAIN_FILE_x86_64-apple-darwin = None
  CMAKE_TOOLCHAIN_FILE_x86_64_apple_darwin = None
  HOST_CMAKE_TOOLCHAIN_FILE = None
  CMAKE_TOOLCHAIN_FILE = None
  CMAKE_GENERATOR_x86_64-apple-darwin = None
  CMAKE_GENERATOR_x86_64_apple_darwin = None
  HOST_CMAKE_GENERATOR = None
  CMAKE_GENERATOR = None
  CMAKE_PREFIX_PATH_x86_64-apple-darwin = None
  CMAKE_PREFIX_PATH_x86_64_apple_darwin = None
  HOST_CMAKE_PREFIX_PATH = None
  CMAKE_PREFIX_PATH = None
  CMAKE_x86_64-apple-darwin = None
  CMAKE_x86_64_apple_darwin = None
  HOST_CMAKE = None
  CMAKE = None
  running: "cmake" "-Wdev" "--debug-output" "/Users/leo/.cargo/registry/src/github.com-1ecc6299db9ec823/snmalloc-sys-0.2.28/snmalloc" "-DSNMALLOC_RUST_SUPPORT=ON" "-DCMAKE_INSTALL_PREFIX=/Users/leo/Project/rust/roapi/target/release/build/snmalloc-sys-90064ef68b6d9bcf/out" "-DCMAKE_C_FLAGS= -ffunction-sections -fdata-sections -fPIC -m64 -arch x86_64" "-DCMAKE_C_COMPILER=/usr/bin/cc" "-DCMAKE_CXX_FLAGS= -ffunction-sections -fdata-sections -fPIC -m64 -arch x86_64" "-DCMAKE_CXX_COMPILER=/usr/bin/c++" "-DCMAKE_ASM_FLAGS= -ffunction-sections -fdata-sections -fPIC -m64 -arch x86_64" "-DCMAKE_ASM_COMPILER=/usr/bin/cc" "-DCMAKE_BUILD_TYPE=Release" "-DCMAKE_VERBOSE_MAKEFILE:BOOL=ON"

  --- stderr
  thread 'main' panicked at '
  failed to execute command: No such file or directory (os error 2)
  is `cmake` not installed?

  build script failed, must exit now', /Users/leo/.cargo/registry/src/github.com-1ecc6299db9ec823/cmake-0.1.48/src/lib.rs:975:5
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...

leopku avatar Oct 04 '22 04:10 leopku

@leopku you will need to install cmake for the snmalloc feature, which is enabled by default. Alternatively, we can turn it off by turning off all the default cargo features.

houqp avatar Oct 04 '22 06:10 houqp