Skip to content

[ci] remove s3_stat_parser and scribe upload #81166

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
suo wants to merge 2 commits into from
Closed

[ci] remove s3_stat_parser and scribe upload #81166

suo wants to merge 2 commits into from

Conversation

@suo
Copy link
Member

@suo suo commented Jul 9, 2022 •
edited
Loading

@suo
[ci] remove s3_stat_parser
7fb6696 
All the users of this are removed in the stack below, so this is the
last to go rip

[ghstack-poisoned]
This was referenced Jul 9, 2022
Closed
Closed
Closed
Closed
@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Jul 9, 2022 •
edited
Loading

🔗 Helpful links

❌ 1 New Failures

As of commit adf60d9 (more details on the Dr. CI page):

Expand to see more
  • 1/1 failures introduced in this PR

🕵️ 1 new failure recognized by patterns

The following CI failures do not appear to be due to upstream breakages

See GitHub Actions build pull / linux-focal-py3.7-gcc7 / test (backwards_compat, 1, 1, linux.2xlarge) (1/1)

Step: "Test" (full log | diagnosis details | 🔁 rerun)

2022-07-09T07:31:59.8325518Z The PR is introduc...m to confirm whether this change is wanted or not. 
2022-07-09T07:31:59.8309061Z processing existing schema:  duration_ns(__torch__.torch.classes.profiling.InstructionStats _0) -> int _0
2022-07-09T07:31:59.8310730Z processing existing schema:  source(__torch__.torch.classes.profiling.SourceStats _0) -> __torch__.torch.classes.profiling.SourceRef _0
2022-07-09T07:31:59.8313014Z processing existing schema:  line_map(__torch__.torch.classes.profiling.SourceStats _0) -> Dict(int, __torch__.torch.classes.profiling.InstructionStats) _0
2022-07-09T07:31:59.8314525Z processing existing schema:  __init__(__torch__.torch.classes.profiling._ScriptProfile _0) -> NoneType _0
2022-07-09T07:31:59.8315973Z processing existing schema:  enable(__torch__.torch.classes.profiling._ScriptProfile _0) -> NoneType _0
2022-07-09T07:31:59.8317552Z processing existing schema:  disable(__torch__.torch.classes.profiling._ScriptProfile _0) -> NoneType _0
2022-07-09T07:31:59.8320029Z processing existing schema:  _dump_stats(__torch__.torch.classes.profiling._ScriptProfile _0) -> __torch__.torch.classes.profiling.SourceStats[] _0
2022-07-09T07:31:59.8321651Z processing existing schema:  __init__(__torch__.torch.classes.c10d.ProcessGroup _0, int _1, int _2) -> NoneType _0
2022-07-09T07:31:59.8323210Z processing existing schema:  __init__(__torch__.torch.classes.c10d.Work _0) -> NoneType _0
2022-07-09T07:31:59.8325194Z processing existing schema:  __init__(__torch__.torch.classes.dist_rpc.WorkerInfo _0, str _1, int _2) -> NoneType _0
2022-07-09T07:31:59.8325518Z The PR is introducing backward incompatible changes to the operator library. Please contact PyTorch team to confirm whether this change is wanted or not. 
2022-07-09T07:31:59.8325540Z 
2022-07-09T07:31:59.8325613Z Broken ops: [
2022-07-09T07:31:59.8325978Z 	__getstate__(__torch__.torch.classes.sparse.LinearPackedParamsBase _0) -> ((Tensor, Tensor?, int[]) _0)
2022-07-09T07:31:59.8326277Z 	__setstate__(__torch__.torch.classes.sparse.LinearPackedParamsBase _0, (Tensor, Tensor?, int[]) _1) -> NoneType _0
2022-07-09T07:31:59.8326339Z ]
2022-07-09T07:31:59.9541419Z ##[error]Process completed with exit code 1.
2022-07-09T07:31:59.9572390Z Prepare all required actions
2022-07-09T07:31:59.9572625Z Getting action download info
2022-07-09T07:32:00.0789604Z ##[group]Run ./.github/actions/get-workflow-job-id
2022-07-09T07:32:00.0789691Z with:
This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

@suo
Update on "[ci] remove s3_stat_parser and scribe upload"
adf60d9 
All the users of this are removed in the stack below, so these are the
last to go rip

[ghstack-poisoned]
@suo suo changed the title [ci] remove s3_stat_parser [ci] remove s3_stat_parser and scribe upload Jul 9, 2022
suo added a commit that referenced this pull request Jul 9, 2022
@suo
[ci] remove s3_stat_parser and scribe upload
27dc801 
All the users of this are removed in the stack below, so these are the
last to go rip

ghstack-source-id: 3fe7126
Pull Request resolved: #81166
janeyx99
janeyx99 reviewed Jul 11, 2022
View reviewed changes
tools/stats/scribe.py
return str(invoke_lambda("gh-ci-scribe-proxy", event))


def _send_to_scribe_via_http(access_token: str, logs: str) -> str:
Copy link
Contributor

@janeyx99 janeyx99 Jul 11, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait...we probably still want to host an internal copy of our test stuff, right? Not sure the best principles here, but the internal scuba stats could serve as good backup if we can port this to the upload test reports workflow perhaps?

Copy link
Member Author

@suo suo Jul 11, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the motivation for keeping it around? I thought we were trying to move the stats to OSS.

If it's for backup reasons—all test stats are backed up to S3 anyway, so it should be easy to backfill old data.

@github-actions
Copy link
Contributor

github-actions bot commented Sep 9, 2022

Looks like this PR hasn't been updated in a while so we're going to go ahead and mark this as Stale.
Feel free to remove the Stale label if you feel this was a mistake.
If you are unable to remove the Stale label please contact a maintainer in order to do so.
If you want the bot to never mark this PR stale again, add the no-stale label.
Stale pull requests will automatically be closed after 30 days of inactivity.

@github-actions github-actions bot added the Stale label Sep 9, 2022
@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Oct 4, 2022

/easycla

As part of the transition to the PyTorch Foundation, this project now requires contributions be covered under the new CLA. See #85559 for additional details.

This comment will trigger a new check of this PR. If you are already covered, you will simply see a new "EasyCLA" check that passes. If you are not covered, a bot will leave a new comment with a link to sign.

@linux-foundation-easycla
Copy link

linux-foundation-easycla bot commented Oct 4, 2022

CLA Not Signed

@github-actions github-actions bot closed this Nov 3, 2022
@facebook-github-bot facebook-github-bot deleted the gh/suo/591/head branch June 8, 2023 18:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@janeyx99 janeyx99 janeyx99 left review comments

Assignees

No one assigned

Labels

cla signed Stale

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@suo @facebook-github-bot @janeyx99