[iOS][CI] add testing code to PR jobs #27594
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This was referenced Oct 9, 2019
xta0
added
the
module: ios
## Summary Since the nightly jobs are lack of testing phases, we don't really have a way to test the binary before uploading it to AWS. To make the work more solid, we need to figure out a way to verify the binary. Fortunately, the XCode tool chain offers a way to build your app without XCode app, which is the [xcodebuild](https://developer.apple.com/library/archive/technotes/tn2339/_index.html) command. Now we can link our binary to a testing app and run `xcodebuild` to to see if there is any linking error. The PRs below have already done some of the preparation jobs - [#26261](#26261) - [#26632](#26632) The challenge comes when testing the arm64 build as we don't have a way to code-sign our TestApp. Circle CI has a [tutorial](https://circleci.com/docs/2.0/ios-codesigning/) but is too complicated to implement. Anyway, I figured out an easier way to do it 1. Disable automatically code sign in XCode 2. Export the encoded developer certificate and provisioning profile to org-context in Circle CI (done) 3. Install the developer certificate to the key chain store on CI machines via Fastlane. 4. Add the testing code to PR jobs and verify the result. 5. Add the testing code to nightly jobs and verify the result. ## Test Plan - Both PR jobs and nightly jobs can finish successfully. - `xcodebuild` can finish successfully
## Summary Since the nightly jobs are lack of testing phases, we don't really have a way to test the binary before uploading it to AWS. To make the work more solid, we need to figure out a way to verify the binary. Fortunately, the XCode tool chain offers a way to build your app without XCode app, which is the [xcodebuild](https://developer.apple.com/library/archive/technotes/tn2339/_index.html) command. Now we can link our binary to a testing app and run `xcodebuild` to to see if there is any linking error. The PRs below have already done some of the preparation jobs - [#26261](#26261) - [#26632](#26632) The challenge comes when testing the arm64 build as we don't have a way to code-sign our TestApp. Circle CI has a [tutorial](https://circleci.com/docs/2.0/ios-codesigning/) but is too complicated to implement. Anyway, I figured out an easier way to do it 1. Disable automatically code sign in XCode 2. Export the encoded developer certificate and provisioning profile to org-context in Circle CI (done) 3. Install the developer certificate to the key chain store on CI machines via Fastlane. 4. Add the testing code to PR jobs and verify the result. 5. Add the testing code to nightly jobs and verify the result. ## Test Plan - Both PR jobs and nightly jobs can finish successfully. - `xcodebuild` can finish successfully
ljk53
approved these changes
Oct 10, 2019
## Summary Since the nightly jobs are lack of testing phases, we don't really have a way to test the binary before uploading it to AWS. To make the work more solid, we need to figure out a way to verify the binary. Fortunately, the XCode tool chain offers a way to build your app without XCode app, which is the [xcodebuild](https://developer.apple.com/library/archive/technotes/tn2339/_index.html) command. Now we can link our binary to a testing app and run `xcodebuild` to to see if there is any linking error. The PRs below have already done some of the preparation jobs - [#26261](#26261) - [#26632](#26632) The challenge comes when testing the arm64 build as we don't have a way to code-sign our TestApp. Circle CI has a [tutorial](https://circleci.com/docs/2.0/ios-codesigning/) but is too complicated to implement. Anyway, I figured out an easier way to do it 1. Disable automatically code sign in XCode 2. Export the encoded developer certificate and provisioning profile to org-context in Circle CI (done) 3. Install the developer certificate to the key chain store on CI machines via Fastlane. 4. Add the testing code to PR jobs and verify the result. 5. Add the testing code to nightly jobs and verify the result. ## Test Plan - Both PR jobs and nightly jobs can finish successfully. - `xcodebuild` can finish successfully
## Summary Since the nightly jobs are lack of testing phases, we don't really have a way to test the binary before uploading it to AWS. To make the work more solid, we need to figure out a way to verify the binary. Fortunately, the XCode tool chain offers a way to build your app without XCode app, which is the [xcodebuild](https://developer.apple.com/library/archive/technotes/tn2339/_index.html) command. Now we can link our binary to a testing app and run `xcodebuild` to to see if there is any linking error. The PRs below have already done some of the preparation jobs - [#26261](#26261) - [#26632](#26632) The challenge comes when testing the arm64 build as we don't have a way to code-sign our TestApp. Circle CI has a [tutorial](https://circleci.com/docs/2.0/ios-codesigning/) but is too complicated to implement. Anyway, I figured out an easier way to do it 1. Disable automatically code sign in XCode 2. Export the encoded developer certificate and provisioning profile to org-context in Circle CI (done) 3. Install the developer certificate to the key chain store on CI machines via Fastlane. 4. Add the testing code to PR jobs and verify the result. 5. Add the testing code to nightly jobs and verify the result. ## Test Plan - Both PR jobs and nightly jobs can finish successfully. - `xcodebuild` can finish successfully
## Summary Since the nightly jobs are lack of testing phases, we don't really have a way to test the binary before uploading it to AWS. To make the work more solid, we need to figure out a way to verify the binary. Fortunately, the XCode tool chain offers a way to build your app without XCode app, which is the [xcodebuild](https://developer.apple.com/library/archive/technotes/tn2339/_index.html) command. Now we can link our binary to a testing app and run `xcodebuild` to to see if there is any linking error. The PRs below have already done some of the preparation jobs - [#26261](#26261) - [#26632](#26632) The challenge comes when testing the arm64 build as we don't have a way to code-sign our TestApp. Circle CI has a [tutorial](https://circleci.com/docs/2.0/ios-codesigning/) but is too complicated to implement. Anyway, I figured out an easier way to do it 1. Disable automatically code sign in XCode 2. Export the encoded developer certificate and provisioning profile to org-context in Circle CI (done) 3. Install the developer certificate to the key chain store on CI machines via Fastlane. 4. Add the testing code to PR jobs and verify the result. 5. Add the testing code to nightly jobs and verify the result. ## Test Plan - Both PR jobs and nightly jobs can finish successfully. - `xcodebuild` can finish successfully
## Summary Since the nightly jobs are lack of testing phases, we don't really have a way to test the binary before uploading it to AWS. To make the work more solid, we need to figure out a way to verify the binary. Fortunately, the XCode tool chain offers a way to build your app without XCode app, which is the [xcodebuild](https://developer.apple.com/library/archive/technotes/tn2339/_index.html) command. Now we can link our binary to a testing app and run `xcodebuild` to to see if there is any linking error. The PRs below have already done some of the preparation jobs - [#26261](#26261) - [#26632](#26632) The challenge comes when testing the arm64 build as we don't have a way to code-sign our TestApp. Circle CI has a [tutorial](https://circleci.com/docs/2.0/ios-codesigning/) but is too complicated to implement. Anyway, I figured out an easier way to do it 1. Disable automatically code sign in XCode 2. Export the encoded developer certificate and provisioning profile to org-context in Circle CI (done) 3. Install the developer certificate to the key chain store on CI machines via Fastlane. 4. Add the testing code to PR jobs and verify the result. 5. Add the testing code to nightly jobs and verify the result. ## Test Plan - Both PR jobs and nightly jobs can finish successfully. - `xcodebuild` can finish successfully
Contributor
xta0
added a commit
that referenced
this pull request
Oct 12, 2019
## Summary Since the nightly jobs are running without any testing code, we don't really have a way to verify the binary before uploading it to AWS. To make the work more solid, I came up with an approach to test our builds. ## How it works The XCode tool chain offers a way to build your app without XCode app, which is the [xcodebuild](https://developer.apple.com/library/archive/technotes/tn2339/_index.html) command. So the approach is link our binaries to a testing app and run `xcodebuild` to see if there is any linking error. The PRs below have already done some of the preparation jobs - [#26261](#26261) adds a dummy testing app - [#26632](#26632) adds a ruby script that does all the XCode configuration. The challenge comes when testing the arm64 build as we don't have a way to code-sign our TestApp. Circle CI has a [tutorial](https://circleci.com/docs/2.0/ios-codesigning/) but is too complicated to implement. Anyway, I figured out an easier way to do it 1. Disable automatically code sign in XCode (done #27591 ) 2. Export the encoded developer certificate and provisioning profile to org-context in Circle CI (done) 3. Install the developer certificate to the key chain store on CI machines via Fastlane. (done #27593 ) 4. Add the testing code to PR jobs and verify the result. (done #27594 ) 5. Add the testing code to nightly jobs and verify the result. ## Test Plan - Both PR jobs and nightly jobs can finish successfully. - `xcodebuild` can finish successfully Differential Revision: [D17893271](https://our.internmc.facebook.com/intern/diff/D17893271)
xta0
added a commit
that referenced
this pull request
Oct 12, 2019
## Summary Since the nightly jobs are running without any testing code, we don't really have a way to verify the binary before uploading it to AWS. To make the work more solid, I came up with an approach to test our builds. ## How it works The XCode tool chain offers a way to build your app without XCode app, which is the [xcodebuild](https://developer.apple.com/library/archive/technotes/tn2339/_index.html) command. So the approach is link our binaries to a testing app and run `xcodebuild` to see if there is any linking error. The PRs below have already done some of the preparation jobs - [#26261](#26261) adds a dummy testing app - [#26632](#26632) adds a ruby script that does all the XCode configuration. The challenge comes when testing the arm64 build as we don't have a way to code-sign our TestApp. Circle CI has a [tutorial](https://circleci.com/docs/2.0/ios-codesigning/) but is too complicated to implement. Anyway, I figured out an easier way to do it 1. Disable automatically code sign in XCode (done #27591 ) 2. Export the encoded developer certificate and provisioning profile to org-context in Circle CI (done) 3. Install the developer certificate to the key chain store on CI machines via Fastlane. (done #27593 ) 4. Add the testing code to PR jobs and verify the result. (done #27594 ) 5. Add the testing code to nightly jobs and verify the result. ## Test Plan - Both PR jobs and nightly jobs can finish successfully. - `xcodebuild` can finish successfully Differential Revision: [D17893271](https://our.internmc.facebook.com/intern/diff/D17893271)
facebook-github-bot
pushed a commit
that referenced
this pull request
Oct 12, 2019
Summary: Pull Request resolved: #27784 ## Summary Since the nightly jobs are running without any testing code, we don't really have a way to verify the binary before uploading it to AWS. To make the work more solid, I came up with an approach to test our builds. ## How it works The XCode tool chain offers a way to build your app without XCode app, which is the [xcodebuild](https://developer.apple.com/library/archive/technotes/tn2339/_index.html) command. So the approach is link our binaries to a testing app and run `xcodebuild` to see if there is any linking error. The PRs below have already done some of the preparation jobs - [#26261](#26261) adds a dummy testing app - [#26632](#26632) adds a ruby script that does all the XCode configuration. The challenge comes when testing the arm64 build as we don't have a way to code-sign our TestApp. Circle CI has a [tutorial](https://circleci.com/docs/2.0/ios-codesigning/) but is too complicated to implement. Anyway, I figured out an easier way to do it 1. Disable automatically code sign in XCode (done #27591 ) 2. Export the encoded developer certificate and provisioning profile to org-context in Circle CI (done) 3. Install the developer certificate to the key chain store on CI machines via Fastlane. (done #27593 ) 4. Add the testing code to PR jobs and verify the result. (done #27594 ) 5. Add the testing code to nightly jobs and verify the result. ## Test Plan - Both PR jobs and nightly jobs can finish successfully. - `xcodebuild` can finish successfully Test Plan: Imported from OSS Differential Revision: D17893271 Pulled By: xta0 fbshipit-source-id: cb7679224e062a4884615f625a2933cad8bd4c11
thiagocrepaldi
pushed a commit
to thiagocrepaldi/pytorch
that referenced
this pull request
Feb 4, 2020
Summary: Pull Request resolved: pytorch#27594 ## Summary Since the nightly jobs are lack of testing phases, we don't really have a way to test the binary before uploading it to AWS. To make the work more solid, we need to figure out a way to verify the binary. Fortunately, the XCode tool chain offers a way to build your app without XCode app, which is the [xcodebuild](https://developer.apple.com/library/archive/technotes/tn2339/_index.html) command. Now we can link our binary to a testing app and run `xcodebuild` to to see if there is any linking error. The PRs below have already done some of the preparation jobs - [pytorch#26261](pytorch#26261) - [pytorch#26632](pytorch#26632) The challenge comes when testing the arm64 build as we don't have a way to code-sign our TestApp. Circle CI has a [tutorial](https://circleci.com/docs/2.0/ios-codesigning/) but is too complicated to implement. Anyway, I figured out an easier way to do it 1. Disable automatically code sign in XCode 2. Export the encoded developer certificate and provisioning profile to org-context in Circle CI (done) 3. Install the developer certificate to the key chain store on CI machines via Fastlane. 4. Add the testing code to PR jobs and verify the result. 5. Add the testing code to nightly jobs and verify the result. ## Test Plan - Both PR jobs and nightly jobs can finish successfully. - `xcodebuild` can finish successfully Test Plan: Imported from OSS Differential Revision: D17850703 Pulled By: xta0 fbshipit-source-id: ab220061c6e2ec75cae23684ad999c4f9c276820
thiagocrepaldi
pushed a commit
to thiagocrepaldi/pytorch
that referenced
this pull request
Feb 4, 2020
Summary: Pull Request resolved: pytorch#27784 ## Summary Since the nightly jobs are running without any testing code, we don't really have a way to verify the binary before uploading it to AWS. To make the work more solid, I came up with an approach to test our builds. ## How it works The XCode tool chain offers a way to build your app without XCode app, which is the [xcodebuild](https://developer.apple.com/library/archive/technotes/tn2339/_index.html) command. So the approach is link our binaries to a testing app and run `xcodebuild` to see if there is any linking error. The PRs below have already done some of the preparation jobs - [pytorch#26261](pytorch#26261) adds a dummy testing app - [pytorch#26632](pytorch#26632) adds a ruby script that does all the XCode configuration. The challenge comes when testing the arm64 build as we don't have a way to code-sign our TestApp. Circle CI has a [tutorial](https://circleci.com/docs/2.0/ios-codesigning/) but is too complicated to implement. Anyway, I figured out an easier way to do it 1. Disable automatically code sign in XCode (done pytorch#27591 ) 2. Export the encoded developer certificate and provisioning profile to org-context in Circle CI (done) 3. Install the developer certificate to the key chain store on CI machines via Fastlane. (done pytorch#27593 ) 4. Add the testing code to PR jobs and verify the result. (done pytorch#27594 ) 5. Add the testing code to nightly jobs and verify the result. ## Test Plan - Both PR jobs and nightly jobs can finish successfully. - `xcodebuild` can finish successfully Test Plan: Imported from OSS Differential Revision: D17893271 Pulled By: xta0 fbshipit-source-id: cb7679224e062a4884615f625a2933cad8bd4c11
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Stack from ghstack:
Summary
Since the nightly jobs are lack of testing phases, we don't really have a way to test the binary before uploading it to AWS. To make the work more solid, we need to figure out a way to verify the binary.
Fortunately, the XCode tool chain offers a way to build your app without XCode app, which is the xcodebuild command. Now we can link our binary to a testing app and run
xcodebuildto to see if there is any linking error. The PRs below have already done some of the preparation jobsThe challenge comes when testing the arm64 build as we don't have a way to code-sign our TestApp. Circle CI has a tutorial but is too complicated to implement. Anyway, I figured out an easier way to do it
Test Plan
xcodebuildcan finish successfullyDifferential Revision: D17850703