Skip to content
This repository was archived by the owner on Feb 25, 2025. It is now read-only.

Conversation

@XilaiZhang
Copy link
Contributor

@XilaiZhang XilaiZhang commented Jun 22, 2022

context: #34112
#34180

This pr is based on discussion with zra, that we should remove changes to zip.py, zip_bundle.gni and other templates. Instead, use generated_files.

For the leaf nodes, the closest i can get was to use get_target_outputs to take out names from source files. However, get_target_outputs only works for actions, template and such, and would not work for sources such as //flutter/impeller/compiler:impellerc. So in this PR we still use hard coded file names. I can update with more information tomorrow morning.

@XilaiZhang XilaiZhang changed the title generated_files [gn] use generated_files to encode entitlement information Jun 22, 2022
@zanderso
Copy link
Member

What happens with //flutter/impeller/compiler:impellerc?

@XilaiZhang
Copy link
Contributor Author

Yes more context: Ideally I could obtain the leaf node filename with something like get_target_outputs("//flutter/impeller/compiler:impellerc"), but this doesn't quite work because impellerc is in a impeller_component, and get_target_outputs only supports copy, generated_file, and action targets. So in the end I hard coded the file names. Maybe there is a smarter way to avoid hard coding filenames.

@XilaiZhang XilaiZhang requested a review from CaseyHillers June 23, 2022 15:50
destination = "libtessellator$dll"
},
]
if (host_os == "mac" && full_target_platform_name == "darwin-x64") {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to sign arm64 artifacts?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My reasoning is #34112 (comment). Maybe @christopherfujino could help decide if it is a better choice to have a tight constraint?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we don't currently build arm artifacts; however, I don't think you should exclude arm here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good, I have relieved the constraints

@CaseyHillers
Copy link
Contributor

Should the previous PRs remain open?

#34112
#34180

@zanderso
Copy link
Member

Yes more context: Ideally I could obtain the leaf node filename with something like get_target_outputs("//flutter/impeller/compiler:impellerc"), but this doesn't quite work because impellerc is in a impeller_component, and get_target_outputs only supports copy, generated_file, and action targets. So in the end I hard coded the file names. Maybe there is a smarter way to avoid hard coding filenames.

Ah, I see. I'm still of the opinion that this will be easier to maintain/extend if the metadata is attached to the "leaf" targets. When adding the metadata to the leaf targets, it should be possible to reconstruct the outputs without the need for get_target_outputs().

@XilaiZhang
Copy link
Contributor Author

I am currently looking for a way to commit and push files out of the scope of //flutter. For example, if I can make changes to //build/compiled_action.gni, and add a line forward_variables_from(invoker, ["metadata"]) , I would be able to forward the metadata variable and collect them from generated_files.

The recently pushed up commits (which includes everything under src/flutter directory), along with changes made outside of src/flutter directory (src/build etc.) would work together to produce the desired output. Currently only files under src/flutter are pushed, and they themselves would fail the checks.

@zanderso
Copy link
Member

@XilaiZhang
Copy link
Contributor Author

Exactly, Thank you! I pushed up flutter/buildroot#585 on the buildroot side.

},
{
source = "$root_out_dir/libtessellator$dll"
destination = "libtessellator$dll"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently I found some file differences when comparing BUILD.gn and GCS artifact. For example, for darwin-x64/artifacts.zip,

  1. the BUILD.gn in cs.opensource suggests the zip file should not contain gen_snapshot, and contain 3 exectuables in total. (https://cs.opensource.google/flutter/engine/+/master:build/archives/BUILD.gn;l=58-69)
  2. the BUILD.gn in this file suggests that the zip file should not contain gen_snapshot, but contain flutter_tester_fractional_translation, with 4 executables in total (83fef4b)
  3. However, on the latest artifact downloaded from GCS, the zip file extracted contains gen_snapshot, and does not contain flutter_tester_fractional_translation. and has 4 executables in total (https://pantheon.corp.google.com/storage/browser/_details/flutter_infra_release/flutter/ea756c55f29a1f1143a17379c70422d5438ad891/darwin-x64/artifacts.zip;tab=live_object?authuser=0)
    Which one should be the source of truth? Thank you!

@XilaiZhang
Copy link
Contributor Author

looks like the failure are caused by, the change in flutter/buildroot#585 did not get propagated into engine.

I tested locally with:

  1. fetch upstream
  2. remove everything under src/build directory
  3. run gclient sync to fetch everything from buildroot, and re-generate the src/build directory
    However, the newly synced files do not include changes we made in [gn] add forward variables to compiled_action buildroot#585. And without the change on buildroot side, the engine build would fail with the same failures as we have seen in the presubmit builds.

What would be the correct way to propagate changes on buildroot side into engine? Thank you!

@godofredoc
Copy link
Contributor

looks like the failure are caused by, the change in flutter/buildroot#585 did not get propagated into engine.

I tested locally with:

  1. fetch upstream
  2. remove everything under src/build directory
  3. run gclient sync to fetch everything from buildroot, and re-generate the src/build directory
    However, the newly synced files do not include changes we made in [gn] add forward variables to compiled_action buildroot#585. And without the change on buildroot side, the engine build would fail with the same failures as we have seen in the presubmit builds.

What would be the correct way to propagate changes on buildroot side into engine? Thank you!

You need to manually roll the hash in this line https://github.com/flutter/engine/blob/main/DEPS#L114. I'd recommend that you do that in a separate PR.

@XilaiZhang
Copy link
Contributor Author

Thanks for the instructions! Pushed up #34297

Copy link
Member

@zanderso zanderso left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@XilaiZhang XilaiZhang merged commit ce4156d into flutter:main Jun 27, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants