Skip to content

[Auto Parallel] Add co_shard spmd_rule for softmax#74829

Merged
From00 merged 13 commits into
PaddlePaddle:developfrom
ooooo-create:co_shard_for_softmax_
Sep 1, 2025
Merged

[Auto Parallel] Add co_shard spmd_rule for softmax#74829
From00 merged 13 commits into
PaddlePaddle:developfrom
ooooo-create:co_shard_for_softmax_

Conversation

@ooooo-create
Copy link
Copy Markdown
Contributor

@ooooo-create ooooo-create commented Aug 22, 2025 •
edited
Loading

PR Category

Auto Parallel

PR Types

New features

Description

适配 ShardingMergeForTensors

  1. 创建一个结构A std::unordered_map<std::string, std::vector<std::vector<int64_t>>> ，用来表示张量的一个维度表示，有几个不同的 dim_map 表示（被哪些 mesh dim 切，组成的 vector）
  2. 遍历 张量维度，先对维度对应的 A 按照并行度降序排序，并行度高的就在前，之后进行并集合并
  3. 验证 mesh 并集是够切，现在设置的依据是移除最后一个 mesh_dim，因为这样可以较好的保持之前较大并行度的 mesh dim 顺序，同时最坏的情况下也拥有之前最大的并行度（相当于用最大的并行度去 reshard 别的 tensor）
  4. 处理多个张量维度被同一个 mesh 切的情况，仅保留张量维度最大的切分
std::unordered_map<std::string, std::vector<int64_t>> ShardingMergeForTensors(
    const std::vector<
        std::pair<std::string, std::vector<std::vector<int64_t>>>>&
        tensor_axes_to_dim_pairs,
    const std::unordered_map<std::string, int64_t>& axis_sizes, # 新增
    const std::vector<int64_t>& mesh_shape, # 新增
    const bool merge_conflicts)

因此新增函数 GetAxesSizes， 利用计算 axes -> shape 的映射

@paddle-bot
Copy link
Copy Markdown

paddle-bot Bot commented Aug 22, 2025

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

@paddle-bot paddle-bot Bot added the contributor External developers label Aug 22, 2025
@ooooo-create
Copy link
Copy Markdown
Contributor Author

ooooo-create commented Aug 29, 2025

/re-run all-failed

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Aug 29, 2025 •
edited
Loading

Codecov Report

❌ Patch coverage is 95.17241% with 7 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (develop@2e7b020). Learn more about missing BASE report.

Files with missing lines Patch % Lines
paddle/phi/infermeta/spmd_rules/utils.cc 92.55% 7 Missing ⚠️
Additional details and impacted files 
@@            Coverage Diff             @@
##             develop   #74829   +/-   ##
==========================================
  Coverage           ?   95.17%           
==========================================
  Files              ?        2           
  Lines              ?      145           
  Branches           ?        0           
==========================================
  Hits               ?      138           
  Misses             ?        7           
  Partials           ?        0

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ooooo-create
Copy link
Copy Markdown
Contributor Author

ooooo-create commented Aug 30, 2025

/re-run all-failed

1 similar comment
@ooooo-create
Copy link
Copy Markdown
Contributor Author

ooooo-create commented Sep 1, 2025

/re-run all-failed

@ooooo-create
Copy link
Copy Markdown
Contributor Author

ooooo-create commented Sep 1, 2025

/re-run all-failed

@liufengwei0103
Copy link
Copy Markdown
Contributor

liufengwei0103 commented Sep 1, 2025

/re-run all-failed

1 similar comment
@ooooo-create
Copy link
Copy Markdown
Contributor Author

ooooo-create commented Sep 1, 2025

/re-run all-failed

@From00 From00 merged commit fd37915 into PaddlePaddle:develop Sep 1, 2025
78 of 82 checks passed
@ooooo-create ooooo-create mentioned this pull request Sep 4, 2025
Closed
@ooooo-create ooooo-create deleted the co_shard_for_softmax_ branch September 29, 2025 09:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@XieYunshen XieYunshen XieYunshen approved these changes

@LiYuRio LiYuRio Awaiting requested review from LiYuRio LiYuRio is a code owner

@ForFishes ForFishes Awaiting requested review from ForFishes ForFishes is a code owner

@zhiqiu zhiqiu Awaiting requested review from zhiqiu

+1 more reviewer

@liufengwei0103 liufengwei0103 liufengwei0103 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

contributor External developers

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@ooooo-create @codecov-commenter @liufengwei0103 @XieYunshen @From00