Hi @jackye1995 @wjones127 @u70b3
I had prepared a PR with refactor WatsonxEmbeddings component. All changes are backward-compatible.

Could you please take a look into this changes?

Thanks for tagging me. I’m not a core maintainer of this project, but since you asked for a review, I took a careful look at the compatibility and configuration behavior.

The overall direction makes sense, especially adding space_id support and improving credential handling. I did notice a few issues that seem worth addressing before merging:

1. Existing tables may silently switch embedding models

LanceDB persists only the arguments explicitly passed to create():

def create(cls, **kwargs):
    instance = cls(**resolved_kwargs)
    instance._original_args = kwargs

def safe_model_dump(self):
    return self._original_args

The function is later reconstructed from this metadata:

function=self.get(obj["name"]).create(**obj["model"])

Therefore, WatsonxEmbeddings.create() stores {} as its model configuration.

After this PR changes the default:

name: str = "ibm/granite-embedding-278m-multilingual"

an existing table whose vectors were generated with the previous Slate default will use Granite for new query embeddings.

Both models produce 768-dimensional vectors, so this will not raise a dimension error. It may instead silently degrade retrieval correctness.

Suggestion: Preserve the legacy default when reading old metadata, or introduce explicit versioning/migration for the stored embedding configuration.

2. The new supported-model list needs verification

IBM’s current SaaS documentation states:

• all-minilm-l6-v2: deprecated; withdrawal date June 8, 2026
• slate-125m-english-rtrvr-v2: deprecated; withdrawal date August 8, 2026
• slate-30m-english-rtrvr-v2: deprecated; withdrawal date August 8, 2026

IBM supported encoder models

This seems inconsistent with presenting these models as the new current supported set. Could you clarify the targeted watsonx deployment/version and verify the list accordingly?

3. Explicit scope arguments can conflict with environment variables

The current resolution logic is:

project_id = self.project_id or os.environ.get("WATSONX_PROJECT_ID")
space_id = self.space_id or os.environ.get("WATSONX_SPACE_ID")

For example, explicitly passing project_id still fails when WATSONX_SPACE_ID happens to be present in the environment.

It may be more predictable for an explicitly selected scope to take precedence over the opposite environment variable. Environment fallback could be applied only when neither scope was explicitly provided.

4. Missing tests

This PR changes:

• Default-model behavior
• Supported model IDs
• Project/space resolution
• Mutual-exclusion validation
• Metadata compatibility

However, it does not add tests for these behaviors.

It would be helpful to add mocked tests covering:

• Explicit and environment-based project_id
• Explicit and environment-based space_id
• Missing and conflicting scope configurations
• Explicit scope combined with the opposite environment variable
• Model dimensions
• Legacy metadata deserialization

Conclusion

The space_id support is useful, but the default-model change introduces a potentially silent compatibility issue for existing tables.

Because of that risk, I’m not sure this change is fully backward-compatible or best classified as a chore. I suggest addressing the legacy metadata behavior and adding the relevant tests before merging.

@u70b3 I responded to your review., Could you please take a look again?

Thanks for the update. The explicit project_id / space_id precedence issue is fixed, and the added scope-resolution and dimension tests provide good coverage.

Two items remain:

1. Legacy metadata compatibility is not fixed. Existing tables store model: {}, so reloading still selects the new Granite default. Adding legacy IDs to MODELS_DIMS does not affect this path. The new compatibility test explicitly constructs the legacy model instead of deserializing empty metadata, so it does not reproduce the issue.

2. The supported-model list still labels deprecated models as “current (non-deprecated)”. Please either update the list according to the targeted watsonx deployment or clarify which deployment/version this list applies to.

Could you address these two points before merging?

@u70b3 Thanks for your comments. I believe all of the issues you pointed out have now been addressed.

Thanks for the update. The scope precedence issue is fixed, and restoring the
legacy default prevents the immediate silent model switch. Two concerns remain:

1. The compatibility test is not a valid cross-version regression test:

original = create()
reloaded = create(**original.safe_model_dump())
assert reloaded.name == original.name

Both instances use the current class default, so the test still passes if both
silently switch to Granite. Please deserialize representative legacy
"model": {} metadata and assert explicitly:

assert reloaded.name == "ibm/slate-125m-english-rtrvr"

2. model_names() returns every MODELS_DIMS entry:

return list(MODELS_DIMS.keys())

This mixes currently available, deprecated, and compatibility-only model IDs,
while the validation error calls all of them “Available models”.

IBM documents regional availability and withdrawal dates of June 8, 2026 for
all-minilm-l6-v2 and August 8, 2026 for the Slate v2 models:
https://www.ibm.com/docs/en/watsonx/saas?topic=models-supported-encoder

Please separate models advertised to new users from legacy IDs retained only
for metadata compatibility. A test where both scope environment variables are
set would also complete the requested conflict coverage.

Non-blocking: since this adds space_id and changes public configuration
behavior, chore(python) may not be the most accurate PR type.

@u70b3 Done, thanks for comment
@wjones127 Could you please take a look and merge if possible?

@wjones127 Done, thanks!

Hi @wjones127 @jackye1995
Can you merge this PR if possible?

Hi @wjones127 @Xuanwo @u70b3 @AyushExel @jackye1995
Any chance to merge this PR?

LGTM. I don't have write permission to formally approve this PR.
Hope maintainers can review and merge it.