Skip to content

feat(firestore): Add firestore source #9653

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
kodiakhq merged 14 commits into from
Apr 13, 2023
Merged

feat(firestore): Add firestore source #9653

kodiakhq merged 14 commits into from
Apr 13, 2023

Conversation

@bubbajoe
Copy link
Contributor

@bubbajoe bubbajoe commented Apr 4, 2023 •
edited by erezrokah
Loading

Summary

Adds firebase source #9414

  • Read the contribution guidelines 🧑‍🎓
  • Run make lint to ensure the proposed changes follow the coding style 🚨 (install golangci-lint here)
  • Run make test to ensure the proposed changes pass the tests 🧪
  • If changing a source plugin run make gen to ensure docs are up to date 📝
  • Ensure the status checks below are successful ✅

@bubbajoe bubbajoe requested review from a team and disq and removed request for a team April 4, 2023 01:42
@bubbajoe bubbajoe changed the title Add firebase source fix(firestore)! Add firestore source Apr 4, 2023
@bubbajoe bubbajoe changed the title fix(firestore)! Add firestore source fix(firestore)!: Add firestore source Apr 4, 2023
@bubbajoe bubbajoe changed the title fix(firestore)!: Add firestore source feat(firestore)!: Add firestore source Apr 4, 2023
disq
disq reviewed Apr 4, 2023
View reviewed changes
Copy link
Member

@disq disq left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution! Few initial comments.

@erezrokah erezrokah changed the title feat(firestore)!: Add firestore source feat(firestore): Add firestore source Apr 4, 2023
@disq
Copy link
Member

disq commented Apr 4, 2023

Hi @bubbajoe, would you consider maintaining this plugin, by hosting it in its own repository and handling the releases/updates/issues? One such plugin is the SharePoint source plugin (which is also a community plugin) and another is Simple Analytics plugin, a partner-maintained plugin. We can then add it to the Community Plugins section in our docs and you can basically control your own release cycle.

All you would have to do is to open a new repo called cq-source-firestore, and add the changes under plugin/source/firestore to the root of that repository, with possibly some changes in .github/ and .goreleaser.yml (which can be copied from the SharePoint plugin, with minor modifications) and you'd be set. Then, after tagging a release, the config to use the plugin would be something like:

kind: source
spec:
  name: "firestore"
  registry: "github"
  path: "your-repo-org/firestore"
  version: "v1.0.0" # provide the latest stable version
  destinations: ["postgresql"] # provide the list of used destinations
  spec:
# ...

Let me know what you think.

@bubbajoe
Copy link
Contributor Author

bubbajoe commented Apr 4, 2023

@disq Tbh, i don't know if that's a good idea. Perhaps if this was a database which was not so popular, I would consider it, but I think it was be in Cloudquery's best interest to keep it here.

@bubbajoe
Copy link
Contributor Author

bubbajoe commented Apr 5, 2023

@disq For testing, do i need to include some docker compose for setting up a local instance (emulator in firestore's case)?

@bubbajoe bubbajoe requested a review from disq April 5, 2023 08:52
@disq
Copy link
Member

disq commented Apr 5, 2023

@disq For testing, do i need to include some docker compose for setting up a local instance (emulator in firestore's case)?

@bubbajoe Yes that would be the way to test, here are a few examples from the PostgreSQL and MySQL source plugins respectively.

@disq disq linked an issue Apr 11, 2023 that may be closed by this pull request
feat: Add a firestore (firebase) source plugin #9414
Closed
1 task
BubbaJoe added 2 commits April 12, 2023 17:07
@cq-bot cq-bot added the ci label Apr 12, 2023
@erezrokah erezrokah added the automerge Automatically merge once required checks pass label Apr 13, 2023
@kodiakhq kodiakhq bot merged commit 0fe7a6c into cloudquery:main Apr 13, 2023
This was referenced Apr 13, 2023
Merged
Merged
Merged
Merged
Merged
@erezrokah
Copy link
Member

erezrokah commented Apr 13, 2023

Hi @bubbajoe thanks for the PR and @disq for the review. I followed up with a docs PR in #10007, can you please help me ensure it makes sense?

kodiakhq bot pushed a commit that referenced this pull request Apr 13, 2023
@erezrokah
feat(firestore): Add Sentry (#10003)
aad54bc 

#### Summary

Follow up to #9653

<!--
kodiakhq bot pushed a commit that referenced this pull request Apr 13, 2023
@erezrokah
feat(ci): Add Firestore to PR labeler (#10004)
31c9114 

#### Summary

Follow up to #9653

<!--
kodiakhq bot pushed a commit that referenced this pull request Apr 13, 2023
@erezrokah
fix(firestore): Change default version to 'development' (#10005)
8527b66 

#### Summary

Follow up to #9653

We use this value to avoid sending telemetry in dev versions

<!--
kodiakhq bot pushed a commit that referenced this pull request Apr 13, 2023
@erezrokah
feat(ci): Add firestore to release please (#10006)
b160928 

#### Summary

Follow up to #9653

<!--
@cq-bot cq-bot mentioned this pull request Apr 13, 2023
Merged
kodiakhq bot pushed a commit that referenced this pull request Apr 16, 2023
@erezrokah
feat(website): Add Firestore (#10007)
9586efc 

#### Summary

Follow up to #9653. I'll need some help to confirm the docs, cc @bubbajoe 

Docs in https://cloudquery-oo1n5e8ec-cloudquery.vercel.app/docs/plugins/sources/firestore/overview

<!--
kodiakhq bot pushed a commit that referenced this pull request Apr 17, 2023
@cq-bot
chore(main): Release plugins-source-firestore v1.0.0 (#10008)
cbf37af 
🤖 I have created a release *beep* *boop*
---


## 1.0.0 (2023-04-17)


### Features

* **firestore:** Add firestore source ([#9653](#9653)) ([0fe7a6c](0fe7a6c))
* **firestore:** Add Sentry ([#10003](#10003)) ([aad54bc](aad54bc))
* **firestore:** Upgrade to `github.com/cloudquery/plugin-sdk/v2` ([#9956](#9956)) ([44ae1ba](44ae1ba)), closes [#9954](#9954)
* **website:** Add Firestore ([#10007](#10007)) ([9586efc](9586efc))


### Bug Fixes

* **deps:** Update module github.com/cloudquery/plugin-sdk/v2 to v2.2.0 ([#10135](#10135)) ([cf33b89](cf33b89))
* **firestore:** Change default version to 'development' ([#10005](#10005)) ([8527b66](8527b66))

---
This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
@erezrokah
Copy link
Member

erezrokah commented Apr 17, 2023

Officially released in https://github.com/cloudquery/cloudquery/releases/tag/plugins-source-firestore-v1.0.0, great work @bubbajoe and @disq 🎸

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@disq disq disq approved these changes

@hermanschaaf hermanschaaf Awaiting requested review from hermanschaaf

@erezrokah erezrokah Awaiting requested review from erezrokah

@yevgenypats yevgenypats Awaiting requested review from yevgenypats

Assignees

No one assigned

Labels

automerge Automatically merge once required checks pass

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

feat: Add a firestore (firebase) source plugin

4 participants

@bubbajoe @disq @erezrokah @cq-bot