Describe the bug

To Reproduce

Steps to reproduce the behavior:

1. ...
2. ...
3. ...

Expected behavior

System [please complete the following information]:

• OS: e.g. [Ubuntu 18.04]
• Python Version (if applied): [e.g. Python 3.8]
• Spark / PySpark version: [e.g. PySpark 3.5.1]
• GraphFrames version: [e.g. graphframes-0.9.0]

Component

• Scala Core Internal
• Scala API
• Spark Connect Plugin
• PySpark Classic
• PySpark Connect

Additional context

Are you planning on creating a PR?

• I'm willing to make a pull-request