🐛 Convex source connector error messages by emmaling27 · Pull Request #23797 · airbytehq/airbyte

Emma Forman Ling (emmaling27) · 2023-03-06T23:31:54Z

What

This PR adds well-formatted error messages to the Convex source connector. A Convex user encountered unhelpful error messages when their sync failed. This diff adds error messages that should help with debugging.

How

Convex returns HTTP errors as json with the format {'code': ErrorCode, 'message': ErrorMessage}. This diff raises exceptions and formats errors if requests to Convex fail so logs on failed syncs are helpful to folks using Airbyte to connect to Convex.

🚨 User Impact 🚨

Users should get more helpful error messages that are related to Convex.

Pre-merge Checklist

Expand the relevant checklist and delete the others.

Updating a connector

Community member or Airbyter

Grant edit access to maintainers (instructions)
Secrets in the connector's spec are annotated with airbyte_secret
Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
Code reviews completed
Documentation updated
- Connector's README.md
- Connector's bootstrap.md. See description and examples
- Changelog updated in docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
PR name follows PR naming conventions

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

Create a non-forked branch based on this PR and test the below items on it
Build is successful
If new credentials are required for use in CI, add them to GSM. Instructions.
/test connector=connectors/<name> command is passing
New Connector version released on Dockerhub and connector version bumped by running the /publish command described here

Passing tests:

Mal Hancock (archangelic) · 2023-04-26T17:43:51Z

/test connector=connectors/source-convex

🕑 connectors/source-convex https://github.com/airbytehq/airbyte/actions/runs/4811739867
✅ connectors/source-convex https://github.com/airbytehq/airbyte/actions/runs/4811739867
Python tests coverage:

Name                        Stmts   Miss  Cover
-----------------------------------------------
source_convex/__init__.py       2      0   100%
source_convex/source.py       110     23    79%
-----------------------------------------------
TOTAL                         112     23    79%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestBasicRead.test_read: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestFullRefresh.test_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: not found in the config.
======================== 34 passed, 3 skipped in 24.61s ========================

Mal Hancock (archangelic) · 2023-04-26T17:52:45Z

/publish connector=connectors/source-convex

🕑 Publishing the following connectors:
connectors/source-convex
https://github.com/airbytehq/airbyte/actions/runs/4811805556

Connector	Version	Did it publish?	Were definitions generated?
connectors/source-convex	0.1.1	✅	✅

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

Emma Forman Ling (emmaling27) · 2023-05-03T16:09:50Z

Mal Hancock (@archangelic) I see that tests are failing and now there is a merge conflict. Should I make changes or are you handling it from here?

Mal Hancock (archangelic) · 2023-05-30T22:37:36Z

/test connector=connectors/source-convex

🕑 connectors/source-convex https://github.com/airbytehq/airbyte/actions/runs/5126899368
✅ connectors/source-convex https://github.com/airbytehq/airbyte/actions/runs/5126899368
Python tests coverage:

Name                        Stmts   Miss  Cover
-----------------------------------------------
source_convex/__init__.py       2      0   100%
source_convex/source.py       110     23    79%
-----------------------------------------------
TOTAL                         112     23    79%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestBasicRead.test_read: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestFullRefresh.test_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:100: The previous and actual specifications are identical.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:594: The previous and actual discovered catalogs are identical.
======================== 33 passed, 5 skipped in 24.73s ========================

* format errors and add test * update changelog * lint * fix for integration tests * bump versions * auto-bump connector version * bump version --------- Co-authored-by: Mal Hancock <[email protected]> Co-authored-by: Octavia Squidington III <[email protected]>

format errors and add test

df75f6b

Octavia Squidington III (octavia-squidington-iii) added area/connectors Connector related issues community connectors/source/convex labels Mar 6, 2023

update changelog

1728118

Emma Forman Ling (emmaling27) marked this pull request as ready for review March 6, 2023 23:41

Octavia Squidington III (octavia-squidington-iii) added bounty area/documentation Improvements or additions to documentation labels Mar 6, 2023

lint

82dafc3

Marcos Marx (marcosmarxm) added the pr-alpha-beta label Apr 14, 2023

Marcos Marx (marcosmarxm) assigned Mal Hancock (archangelic) Apr 14, 2023

Mal Hancock (archangelic) added 3 commits April 24, 2023 11:15

Merge remote-tracking branch 'origin/master' into source-logging

da3ae1f

fix for integration tests

a311481

bump versions

34453a3

Mal Hancock (archangelic) approved these changes Apr 26, 2023

View reviewed changes

auto-bump connector version

a9decc9

Mal Hancock (archangelic) mentioned this pull request Apr 26, 2023

Source convex error messages test pr 23797 #25576

Closed

42 tasks

Marcos Marx (marcosmarxm) unassigned Mal Hancock (archangelic) May 12, 2023

Marcos Marx (marcosmarxm) added community-legacy and removed community labels May 15, 2023

Marcos Marx (marcosmarxm) added community and removed community-legacy labels May 30, 2023

Mal Hancock (archangelic) added 2 commits May 30, 2023 15:35

Merge remote-tracking branch 'origin' into source-logging

f970cdf

bump version

8e5460a

Mal Hancock (archangelic) merged commit 99a6719 into airbytehq:master May 30, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🐛 Convex source connector error messages#23797

🐛 Convex source connector error messages#23797
Mal Hancock (archangelic) merged 9 commits intoairbytehq:masterfrom
emmaling27:source-logging

Emma Forman Ling (emmaling27) commented Mar 6, 2023 •

edited

Loading

Uh oh!

Mal Hancock (archangelic) commented Apr 26, 2023 •

edited by github-actions bot

Loading

Uh oh!

Mal Hancock (archangelic) commented Apr 26, 2023 •

edited by github-actions bot

Loading

Uh oh!

Emma Forman Ling (emmaling27) commented May 3, 2023

Uh oh!

Mal Hancock (archangelic) commented May 30, 2023 •

edited by github-actions bot

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

Emma Forman Ling (emmaling27) commented Mar 6, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

How

Recommended reading order

🚨 User Impact 🚨

Pre-merge Checklist

Community member or Airbyter

Airbyter

Uh oh!

Mal Hancock (archangelic) commented Apr 26, 2023 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Build Passed

Uh oh!

Mal Hancock (archangelic) commented Apr 26, 2023 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Emma Forman Ling (emmaling27) commented May 3, 2023

Uh oh!

Mal Hancock (archangelic) commented May 30, 2023 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Build Passed

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Emma Forman Ling (emmaling27) commented Mar 6, 2023 •

edited

Loading

Mal Hancock (archangelic) commented Apr 26, 2023 •

edited by github-actions bot

Loading

Mal Hancock (archangelic) commented Apr 26, 2023 •

edited by github-actions bot

Loading

Mal Hancock (archangelic) commented May 30, 2023 •

edited by github-actions bot

Loading