Skip to content

Conversation

@fabian4
Copy link
Member

@fabian4 fabian4 commented Jul 28, 2023

Fixes #4032.

Motivation

Explain the content here.
Explain why you want to make the changes and what problem you're trying to solve.

Modifications

Add support for cloudevent data encoding and decoding with the way in redis-storage.

Documentation

  • Does this pull request introduce a new feature? (yes / no)
  • If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)
  • If a feature is not applicable for documentation, explain why?
  • If a feature is not documented yet in this PR, please create a followup issue for adding the documentation

@codecov
Copy link

codecov bot commented Jul 29, 2023

Codecov Report

Merging #4301 (d55714f) into master (5b8616b) will decrease coverage by 0.52%.
Report is 10 commits behind head on master.
The diff coverage is 46.79%.

❗ Current head d55714f differs from pull request most recent head 8d18898. Consider uploading reports for the commit 8d18898 to get more accurate results

@@             Coverage Diff              @@
##             master    #4301      +/-   ##
============================================
- Coverage     16.87%   16.35%   -0.52%     
+ Complexity     1428     1359      -69     
============================================
  Files           593      594       +1     
  Lines         26035    25419     -616     
  Branches       2366     2394      +28     
============================================
- Hits           4394     4158     -236     
+ Misses        21215    20838     -377     
+ Partials        426      423       -3     
Files Changed Coverage Δ
...entmesh/common/protocol/http/HttpEventWrapper.java 0.00% <0.00%> (ø)
.../header/message/SendMessageBatchRequestHeader.java 0.00% <0.00%> (ø)
...header/message/SendMessageBatchResponseHeader.java 0.00% <0.00%> (ø)
...eader/message/SendMessageBatchV2RequestHeader.java 0.00% <0.00%> (ø)
...ader/message/SendMessageBatchV2ResponseHeader.java 0.00% <0.00%> (ø)
.../http/header/message/SendMessageRequestHeader.java 0.00% <0.00%> (ø)
...http/header/message/SendMessageResponseHeader.java 0.00% <0.00%> (ø)
...otocol/meshmessage/MeshMessageProtocolAdaptor.java 4.34% <0.00%> (ø)
...olver/http/SendMessageBatchV2ProtocolResolver.java 0.00% <0.00%> (ø)
...olver/http/SendMessageRequestProtocolResolver.java 0.00% <0.00%> (ø)
... and 37 more

... and 8 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@fabian4 fabian4 requested a review from pandaapo July 31, 2023 02:47
xwm1992
xwm1992 previously approved these changes Jul 31, 2023
Copy link
Member

@pandaapo pandaapo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. I tried testing your eventmesh-connector-redis module including CloudEventCodec locally, but unfortunately, I didn't see any results. The only configuration change I made was to modify the topic in source-config.yml to TopicTest2. Could you please provide me with the correct verification steps so that I can re-validate? Thank you.

  2. I'm a bit confused. In your code, you write the type for Redis as ConnectRecord, but CloudEventCodec is used for encoding and decoding the CloudEvent type. Could you clarify my confusion?

  3. If CloudEventCodec has no problem, I suggest that you write the comment Duplicate with org.apache.eventmesh.....CloudEventCodec in the code of your module instead of eventmesh-storage-redis module, considering that you copied the code from that module to yours.

@fabian4
Copy link
Member Author

fabian4 commented Aug 1, 2023

  1. I tried testing your eventmesh-connector-redis module including CloudEventCodec locally, but unfortunately, I didn't see any results. The only configuration change I made was to modify the topic in source-config.yml to TopicTest2. Could you please provide me with the correct verification steps so that I can re-validate? Thank you.

It is all fixed now. (only CloudEvent message support yet)

  • Source flow: msg(CloudEvent) -> SourceTopic(Redis Topic) -> SourceWorker -> TopicTest(EventMesh Topic)
  • Sink flow: TopicTest(EventMesh Topic) -> SinkWorker -> SinkTopic(Redis Topic) -> msg(CloudEvent)

To Verify SinkWorker

  1. Use the TCP cliet demo: https://github.com/apache/eventmesh/blob/master/eventmesh-examples/src/main/java/org/apache/eventmesh/tcp/demo/pub/cloudevents/AsyncPublish.java.
  2. make sinkEnable: true in server-config.yml and subject: TEST-TOPIC-TCP-ASYNC in sink-config.yml.
    (TEST-TOPIC-TCP-ASYNC is the top you send to with the TCP cliet demo).
  3. start up your eventmesh-runtime and eventmesh-connector-redis
  4. You can subscribe to the topic SinkTopic in redis first with your favourite cliet.
  5. Sent a cloudevent message with client mentioned in step 1 and you will see it in your redis client.

To Verify SourceWorker

  1. make sourceEnable: true in server-config.yml.
  2. start up your eventmesh-runtime and eventmesh-connector-redis.
  3. Send a cloudevent messang to the topic SourceTopic in redis. Then it will be send to eventmesh, you can check it by logs or in eventmesh-dashboard
  1. I'm a bit confused. In your code, you write the type for Redis as ConnectRecord, but CloudEventCodec is used for encoding and decoding the CloudEvent type. Could you clarify my confusion?
  2. If CloudEventCodec has no problem, I suggest that you write the comment Duplicate with org.apache.eventmesh.....CloudEventCodec in the code of your module instead of eventmesh-storage-redis module, considering that you copied the code from that module to yours.
  • Now the message is write to redis in the CloudEvent type with CloudEventCodec.
  • The comment is moved into eventmesh-connector-redis in java doc

@fabian4 fabian4 requested a review from pandaapo August 1, 2023 09:19
@xwm1992 xwm1992 merged commit 19dd2b1 into apache:master Aug 3, 2023
@fabian4 fabian4 deleted the fix_redis_codec branch August 3, 2023 02:03
xuhongjia pushed a commit to Deckers-Ohana/eventmesh that referenced this pull request Mar 13, 2025
* fix: add cloudEventCodec for redis connector.

* fix: add cloudEventCodec for redis connector.

* fix import order.

* fix: add doc.

* fix: adjust cloudevent encode and decode.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Add Redis Connector modules

4 participants