What does this PR do?

Fixes #47920

CLIPPreTrainedModel._init_weights initializes the parameters of child modules by name —
module.q_proj.weight, module.fc1.weight, module.text_projection.weight, and so on. A quantized
checkpoint replaces weight on those children with packed tensors (weight_packed for
compressed-tensors, qweight for GPTQ/AWQ), so loading one raises

AttributeError: 'Linear' object has no attribute 'weight'

from the initialization pass that runs at the end of loading: _finalize_model_loading →
_initialize_missing_keys → initialize_weights → smart_apply → _init_weights
(modeling_utils.py:4517, :2476-2491). For a packed format the packed layout is installed before
the weights are loaded, by transformers itself —
CompressedTensorsHfQuantizer._process_model_before_weight_loading calls
self.compressor.compress_model(model) (quantizers/quantizer_compressed_tensors.py:117-121).

A container such as CLIPAttention owns no parameter of its own, so _initialize_weights' skip
conditions never apply to it for a class living in transformers.*, and it always reaches
_init_weights.

This PR routes every child-weight initialization in CLIPPreTrainedModel._init_weights through a
small helper that skips a child with no weight, which is the guard the generic
PreTrainedModel._init_weights already applies to nn.Linear/nn.Conv* (modeling_utils.py:2392)
and to the norms (:2418-2421, "Norms can exist without weights"). It is the same shape as the
merged #38013, which fixed this class of failure in modeling_qwen2_5_omni.py for AutoAWQ
checkpoints.

MetaClip2PreTrainedModel inherits this method through modular and the converter inlines the body,
so models/metaclip_2/modeling_metaclip_2.py is regenerated in the same commit
(python utils/modular_model_converter.py --files_to_parse src/transformers/models/metaclip_2/modular_metaclip_2.py);
MetaCLIP-2 gets the fix with it.

Initialization of a non-quantized model is unchanged: the seeded state_dict of a tiny CLIPModel
hashes to the same SHA-256 before and after the patch (a6808d27…). The change also removes a
pre-existing crash for CLIPForImageClassification with num_labels == 0, where classifier is an
nn.Identity.

Verification

# the new regression test
pytest tests/models/clip/test_modeling_clip.py -k init_weights_with_quantized
  -> 1 passed          (with the patch)
  -> 1 failed, AttributeError: 'Linear' object has no attribute 'weight'   (with the patch reverted)

# the test files utils/tests_fetcher.py selects for this diff
pytest tests/models/clip/test_modeling_clip.py tests/models/metaclip_2/test_modeling_metaclip_2.py \
       tests/models/sam3/test_modeling_sam3.py tests/models/sam3_video/test_modeling_sam3_video.py \
       tests/models/vision_text_dual_encoder/test_modeling_vision_text_dual_encoder.py
  -> 869 passed, 960 skipped, 6 xfailed, 8 failed in 6m18s

# repo checks
python utils/checkers.py ruff_check, ruff_format, init_isort, sort_auto_mappings, modeling_structure
  -> all OK (ruff over the whole tree)
python utils/checkers.py <all REPO_CONSISTENCY_CHECKERS>
  -> All 17 checks passed

The 8 failures are all test_multi_gpu_data_parallel_forward (4 in the CLIP file, 4 in the MetaCLIP-2
file). They are pre-existing on this machine: the same 8 fail on the parent commit, and nothing else in
those five files fails.

The types checker fails identically on the parent commit with the ty version I have installed, so
it is not affected by this diff.

Scope

The same pattern — _init_weights dereferencing a child module's .weight/.bias — is present in 83
of 488 modeling_*.py files on main, and I measured the identical AttributeError for T5,
Mamba2, LLaVA (CLIP vision tower) and Gemma3 (SigLIP vision tower). The issue linked above has
the full analysis, including why marking the quantized modules _is_hf_initialized (as the
bitsandbytes/torchao/quanto/mxfp4/sinq integrations do) does not fix it, and why the 4.x
"skip a fully loaded subtree" rule does not either. I kept this PR to CLIP; happy to extend the guard
to the other families in whatever batching you prefer.

Code Agent Policy

• (First-time contributors only): I confirm that this PR description and code is not written by an LLM or code agent

Not applicable — I am not a first-time contributor (#46277, #47940). Per the Agentic contributions
section of CONTRIBUTING.md: this change was prepared with AI assistance (Claude Code). I have
reviewed every changed line, the analysis and every number above come from runs against the installed
packages, the test commands and results are quoted above, the coordination link is #47920, and I am
not aware of another open PR touching CLIP's _init_weights. I opened this alongside the issue so the
diff is concrete rather than hypothetical -- if you would rather settle the scope there first (one
model vs. the 83 files that share the pattern), say so and I will hold this PR until then.

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline and the
  Pull Request checks?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case. — see the linked issue
• Did you make sure to update the documentation with your changes according to the guidelines? — no documented behavior changes
• Did you write any new necessary tests?
  tests/models/clip/test_modeling_clip.py::CLIPModelTest::test_init_weights_with_quantized_children

Who can review?

@Cyrilvallez (the failure is in the from_pretrained initialization pass), @molbap (CLIP)