Repository navigation
Clean up Feast configuration #525
Description
Creating this card to clean up the Feast Core and Feast Serving configuration files. The configuration files have the following problems
- No clear separation of static Spring or JVM configuration from user level configuration
- Duplicate configuration points for the same thing (BigQuery, or Dataflow configuration)
- Lack of documentation of the options that are configurable.
- Lack of documentation on where new configuration should be placed.
- [EDIT] Adding more scope to this refactor. Configuration should also be easy to define and pass to stores. Validation should happen at the storage layer, not in Feast Core or Feast Serving. This change should make it easier to support stores as interfaces, possibly with factories building the stores/retrievers based on configuration.
This issue tracks the refactor, cleanup, and documentation to solve the above.
Reacted by Ches Martin and David Heryanto
Activity
mrzzy commentedon Mar 26, 2020
Not sure if this should be a separate issue, but this is definitely a pain point. There is almost no documentation on feast.dev the configuring and using Dataflow runner except that it is faster and more reliable than DirectRunner.
Getting Dataflow to work:
core/src/main/java/feast/core/config/JobConfig.java. Also make sure to a set region from dataflow regional endpointsGOOGLE_APPLICATION_CREDENTIALSwoop commentedon Mar 28, 2020
Yea, that is definitely a major pain point. Thanks for pointing it out and being so detailed.
woop commentedon Mar 28, 2020
Throwing this into the ring as well. The following BQ configuration parameters must be set for BQ retrieval to work.
woop commentedon Mar 30, 2020
Two more notes.
feastkeys inapplication.yml. We could easily convert that into markdown and publish it to docs.feast.dev. Iches commentedon Apr 4, 2020
The account which someone apparently created to catch accidental mentions like that one is my favorite thing on the Internet today.
Returning to my seat in the peanut gallery now.
woop commentedon Apr 4, 2020
Just noticed it, hahahahahahha
woop commentedon Apr 16, 2020
Keeping this open for the time being until #533 is done.
Fix for bigquery timeouts + Update image version from 0.4.4 to 0.4.7
woop commentedon May 13, 2020
I believe this issue is done enough. Let's open a new one if needed.