Skip to content

Commit 556da04

Browse files
authored
feat(amber): Enable R UDF Runtime via Optional texera-rudf Plugin (apache#4164)
<!-- Thanks for sending a pull request (PR)! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: [Contributing to Texera](https://github.com/apache/texera/blob/main/CONTRIBUTING.md) 2. Ensure you have added or run the appropriate tests for your PR 3. If the PR is work in progress, mark it a draft on GitHub. 4. Please write your PR title to summarize what this PR proposes, we are following Conventional Commits style for PR titles as well. 5. Be sure to keep the PR description updated to reflect all changes. --> ### What changes were proposed in this PR? <!-- Please clarify what changes you are proposing. The purpose of this section is to outline the changes. Here are some tips for you: 1. If you propose a new API, clarify the use case for a new API. 2. If you fix a bug, you can clarify why it is a bug. 3. If it is a refactoring, clarify what has been changed. 3. It would be helpful to include a before-and-after comparison using screenshots or GIFs. 4. Please consider writing useful notes for better and faster reviews. --> This PR integrates the `texera-rudf` plugin and enables R UDF operators when the plugin is installed in the runtime environment. Previously, apache#4090 removed runtime support for R UDF operators due to licensing constraints. Following the design proposed in apache#4155, we now manage all R UDF–related code in a separate repository and have the Texera main repository consume it as an optional plugin. For prerequisites and installation instructions, please refer to the `README.md` in the plugin repository: https://github.com/kunwp1/texera-rudf. Note: This repository is currently a prototype under my personal account. Once this PR is merged, I plan to transfer it to the Texera organization. ### Any related issues, documentation, discussions? <!-- Please use this section to link other resources if not mentioned already. 1. If this PR fixes an issue, please include `Fixes apache#1234`, `Resolves apache#1234` or `Closes apache#1234`. If it is only related, simply mention the issue number. 2. If there is design documentation, please add the link. 3. If there is a discussion in the mailing list, please add the link. --> Discussion: apache#4155 PR: apache#4090, apache#4124 ### How was this PR tested? <!-- If tests were added, say they were added here. Or simply mention that if the PR is tested with existing test cases. Make sure to include/update test cases that check the changes thoroughly including negative and positive cases if possible. If it was tested in a way different from regular unit tests, please clarify how you tested step by step, ideally copy and paste-able, so that other reviewers can test and check, and descendants can verify in the future. If tests were not added, please describe why they were not added and/or why it was difficult to add. --> Tested with this workflow [Test.json](https://github.com/user-attachments/files/24662090/Test.json) #### Plugin Uninstalled <img width="1728" height="1117" alt="Screenshot 2026-01-15 at 9 27 17 PM" src="https://github.com/user-attachments/assets/838ddfbc-507d-4961-8edf-8f6f2941effd" /> #### Plugin Installed <img width="1728" height="1117" alt="Screenshot 2026-01-15 at 9 29 51 PM" src="https://github.com/user-attachments/assets/57074c3a-fac4-4b3c-9578-9038e066d18d" /> ### Was this PR authored or co-authored using generative AI tooling? <!-- If generative AI tooling has been used in the process of authoring this PR, please include the phrase: 'Generated-by: ' followed by the name of the tool and its version. If no, write 'No'. Please refer to the [ASF Generative Tooling Guidance](https://www.apache.org/legal/generative-tooling.html) for details. --> No
1 parent 096e925 commit 556da04

2 files changed

Lines changed: 102 additions & 16 deletions

File tree

‎amber/src/main/python/core/architecture/managers/executor_manager.py‎

Lines changed: 19 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -132,13 +132,25 @@ class declaration.
132132
:param language: The language of the operator code.
133133
:return:
134134
"""
135-
assert language not in [
136-
"r-tuple",
137-
"r-table",
138-
], "R language is not supported by default. Please consult third party plugin."
139-
executor: type(Operator) = self.load_executor_definition(code)
140-
self.executor = executor()
141-
self.executor.is_source = is_source
135+
if language in ("r-tuple", "r-table"):
136+
# R support is provided by an optional plugin (texera-rudf)
137+
executor_type = "Tuple" if language == "r-tuple" else "Table"
138+
try:
139+
import texera_r
140+
141+
class_suffix = "SourceExecutor" if is_source else "Executor"
142+
executor_class = getattr(texera_r, f"R{executor_type}{class_suffix}")
143+
except ImportError as e:
144+
raise ImportError(
145+
"R operators require the texera-rudf package.\n"
146+
"Install with: pip install git+https://github.com/Texera/texera-rudf.git\n"
147+
f"Import error: {e}"
148+
)
149+
self.executor = executor_class(code)
150+
else:
151+
executor: type(Operator) = self.load_executor_definition(code)
152+
self.executor = executor()
153+
self.executor.is_source = is_source
142154
assert isinstance(self.executor, SourceOperator) == self.executor.is_source, (
143155
"Please use SourceOperator API for source operators."
144156
)

‎amber/src/main/python/core/architecture/managers/test_executor_manager.py‎

Lines changed: 83 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,9 @@
1515
# specific language governing permissions and limitations
1616
# under the License.
1717

18+
import sys
1819
import pytest
20+
from unittest.mock import MagicMock
1921

2022
from core.architecture.managers.executor_manager import ExecutorManager
2123

@@ -39,7 +41,7 @@ def produce(self) -> Iterator[Union[TupleLike, TableLike, None]]:
3941

4042

4143
class TestExecutorManager:
42-
"""Test suite for ExecutorManager, focusing on R UDF support removal."""
44+
"""Test suite for ExecutorManager, focusing on R UDF plugin support."""
4345

4446
@pytest.fixture
4547
def executor_manager(self):
@@ -50,36 +52,108 @@ def executor_manager(self):
5052
if hasattr(manager, "_fs"):
5153
manager.close()
5254

55+
def _mock_r_plugin(self, executor_class_name, is_source):
56+
"""
57+
Helper to mock the texera_r plugin module.
58+
59+
:param executor_class_name: Name of the executor class (e.g., 'RTupleExecutor')
60+
:param is_source: Whether the executor is a source operator
61+
:return: Tuple of (mock_texera_r, mock_executor_instance)
62+
"""
63+
from core.models import SourceOperator, Operator
64+
65+
mock_texera_r = MagicMock()
66+
mock_executor_class = MagicMock()
67+
setattr(mock_texera_r, executor_class_name, mock_executor_class)
68+
69+
# Use appropriate spec based on operator type
70+
spec_class = SourceOperator if is_source else Operator
71+
mock_executor_instance = MagicMock(spec=spec_class)
72+
mock_executor_instance.is_source = is_source
73+
mock_executor_class.return_value = mock_executor_instance
74+
75+
sys.modules["texera_r"] = mock_texera_r
76+
return mock_texera_r, mock_executor_instance
77+
78+
def _cleanup_r_plugin(self):
79+
"""Remove the mocked texera_r module from sys.modules."""
80+
if "texera_r" in sys.modules:
81+
del sys.modules["texera_r"]
82+
5383
def test_initialization(self, executor_manager):
5484
"""Test that ExecutorManager initializes correctly."""
5585
assert executor_manager.executor is None
5686
assert executor_manager.operator_module_name is None
5787
assert executor_manager.executor_version == 0
5888

5989
def test_reject_r_tuple_language(self, executor_manager):
60-
"""Test that 'r-tuple' language is rejected with AssertionError."""
61-
with pytest.raises(AssertionError) as exc_info:
90+
"""Test that 'r-tuple' language is rejected with ImportError when plugin is not available."""
91+
with pytest.raises(ImportError) as exc_info:
6292
executor_manager.initialize_executor(
6393
code=SAMPLE_OPERATOR_CODE, is_source=False, language="r-tuple"
6494
)
6595

66-
# Verify the error message mentions R UDF support has been dropped
67-
assert "not supported" in str(exc_info.value) or "dropped" in str(
96+
# Verify the error message mentions R operators require the texera-rudf package
97+
assert "texera-rudf" in str(exc_info.value) or "R operators require" in str(
6898
exc_info.value
6999
)
70100

71101
def test_reject_r_table_language(self, executor_manager):
72-
"""Test that 'r-table' language is rejected with AssertionError."""
73-
with pytest.raises(AssertionError) as exc_info:
102+
"""Test that 'r-table' language is rejected with ImportError when plugin is not available."""
103+
with pytest.raises(ImportError) as exc_info:
74104
executor_manager.initialize_executor(
75105
code=SAMPLE_OPERATOR_CODE, is_source=False, language="r-table"
76106
)
77107

78-
# Verify the error message mentions R UDF support has been dropped
79-
assert "not supported" in str(exc_info.value) or "dropped" in str(
108+
# Verify the error message mentions R operators require the texera-rudf package
109+
assert "texera-rudf" in str(exc_info.value) or "R operators require" in str(
80110
exc_info.value
81111
)
82112

113+
def test_accept_r_tuple_language_with_plugin(self, executor_manager):
114+
"""Test that 'r-tuple' language is accepted when plugin is available."""
115+
_, mock_executor = self._mock_r_plugin("RTupleExecutor", is_source=False)
116+
try:
117+
executor_manager.initialize_executor(
118+
code="# R code", is_source=False, language="r-tuple"
119+
)
120+
assert executor_manager.executor == mock_executor
121+
finally:
122+
self._cleanup_r_plugin()
123+
124+
def test_accept_r_table_language_with_plugin(self, executor_manager):
125+
"""Test that 'r-table' language is accepted when plugin is available."""
126+
_, mock_executor = self._mock_r_plugin("RTableExecutor", is_source=False)
127+
try:
128+
executor_manager.initialize_executor(
129+
code="# R code", is_source=False, language="r-table"
130+
)
131+
assert executor_manager.executor == mock_executor
132+
finally:
133+
self._cleanup_r_plugin()
134+
135+
def test_accept_r_tuple_source_with_plugin(self, executor_manager):
136+
"""Test that 'r-tuple' source operators work when plugin is available."""
137+
_, mock_executor = self._mock_r_plugin("RTupleSourceExecutor", is_source=True)
138+
try:
139+
executor_manager.initialize_executor(
140+
code="# R code", is_source=True, language="r-tuple"
141+
)
142+
assert executor_manager.executor == mock_executor
143+
finally:
144+
self._cleanup_r_plugin()
145+
146+
def test_accept_r_table_source_with_plugin(self, executor_manager):
147+
"""Test that 'r-table' source operators work when plugin is available."""
148+
_, mock_executor = self._mock_r_plugin("RTableSourceExecutor", is_source=True)
149+
try:
150+
executor_manager.initialize_executor(
151+
code="# R code", is_source=True, language="r-table"
152+
)
153+
assert executor_manager.executor == mock_executor
154+
finally:
155+
self._cleanup_r_plugin()
156+
83157
def test_accept_python_language_regular_operator(self, executor_manager):
84158
"""Test that 'python' language is accepted for regular operators."""
85159
# This should not raise any assertion error

0 commit comments

Comments
 (0)