Skip to content

Enable unpack/repack of RPM containers#15405

Merged
NikolaMilosavljevic merged 8 commits intodotnet:mainfrom
NikolaMilosavljevic:rpm.repack
Jan 15, 2025
Merged

Enable unpack/repack of RPM containers#15405
NikolaMilosavljevic merged 8 commits intodotnet:mainfrom
NikolaMilosavljevic:rpm.repack

Conversation

@NikolaMilosavljevic
Copy link
Member

@NikolaMilosavljevic NikolaMilosavljevic commented Jan 13, 2025

Fixes #14437

This work enables unpack and repack of .deb containers. I've tested this with all current .NET RPM packages. As the code requires the use of cpio tool, it is conditioned to only work on Linux.

@NikolaMilosavljevic
Copy link
Member Author

NikolaMilosavljevic commented Jan 13, 2025

Looks like some bad auto-merge causing build issues. Will fix this soon.

mmitche
mmitche reviewed Jan 13, 2025
View reviewed changes
mmitche
mmitche reviewed Jan 13, 2025
View reviewed changes
mmitche
mmitche reviewed Jan 13, 2025
View reviewed changes
src/Microsoft.DotNet.SignTool/src/BatchSignUtil.cs
else if (!_signTool.VerifySignedRpm(log, file.FullPath))
{
_log.LogError($"Deb package {file.FullPath} is not signed properly.");
}
Copy link
Member

@mmitche mmitche Jan 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add an else here with a "signed properly" message?

Copy link
Member Author

@NikolaMilosavljevic NikolaMilosavljevic Jan 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll add it for both DEB and RPM.

mmitche
mmitche reviewed Jan 13, 2025
View reviewed changes
mmitche
mmitche reviewed Jan 13, 2025
View reviewed changes
@build-analysis build-analysis bot mentioned this pull request Jan 14, 2025
Open
3 tasks
ellahathaway
ellahathaway reviewed Jan 14, 2025
View reviewed changes
src/Microsoft.DotNet.SignTool.Tests/SignToolTests.cs
{".psc1", new List<SignInfo>{ new SignInfo("PSCCertificate") } },
{".dylib", new List<SignInfo>{ new SignInfo("DylibCertificate") } },
{".deb", new List<SignInfo>{ new SignInfo("LinuxSign") } },
{".rpm", new List<SignInfo>{ new SignInfo("LinuxSign") } },
Copy link
Member

@ellahathaway ellahathaway Jan 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the RPM extension also need to be added to s_fileExtensionSignInfoWithCollisionId, s_fileExtensionSignInfoPostBuild, and SignableExtensions? It's possible that this may be addressed in #14433

@build-analysis build-analysis bot mentioned this pull request Jan 14, 2025
Open
2 tasks
@NikolaMilosavljevic
Copy link
Member Author

NikolaMilosavljevic commented Jan 14, 2025

Hmm, Windows tests are failing as they're trying to run the RPM-specific test. Helix log indicates that this is executed on Linux and both commands are failing: https://helixr1107v0xdeko0k025g8.blob.core.windows.net/dotnet-arcade-refs-pull-15405-merge-c0a2b5fcd9fa4337af/Microsoft.DotNet.SignTool.Tests.dll/2/console.5f9debda.log?helixlogtype=result

Console log: 'Microsoft.DotNet.SignTool.Tests.dll' from job c0a2b5fc-d9fa-4337-af25-e28ec6a8a902 (ubuntu.2204.amd64.open) using docker image mcr.microsoft.com/dotnet-buildtools/prereqs:almalinux-8-helix-amd64 on a008SX5

bash: cpio: command not found
find: 'standard output': Broken pipe
find: write error
find: 'file': No such file or directory

Linux tests are failing with the same issue - https://helixr1107v0xdeko0k025g8.blob.core.windows.net/dotnet-arcade-refs-pull-15405-merge-52fab730ffb84b0d9b/Microsoft.DotNet.SignTool.Tests.dll/2/console.cb9f2381.log?helixlogtype=result

@NikolaMilosavljevic
Copy link
Member Author

NikolaMilosavljevic commented Jan 14, 2025 •
edited
Loading

Before I add cpio and file packages to helix almalinux image, I wonder if there is something wrong with test jobs in arcade. Each test leg (Windows and Linux) sends for testing to three targets, Windows, Ubuntu and almalinux. This seems wrong.

From: https://dev.azure.com/dnceng-public/public/_build/results?buildId=917110&view=logs&j=21143b0a-ace1-5058-215e-b11404388d17&t=b6047ca6-0230-5e90-2195-54379d550513

  Job c0a2b5fc-d9fa-4337-af25-e28ec6a8a902 on (AlmaLinux.8.Amd64.Open)[email protected]/dotnet-buildtools/prereqs:almalinux-8-helix-amd64 is completed with 24 finished work items.
  Job f653c336-8b5d-4237-bebb-5c589e318064 on (Debian.12.Amd64.Open)[email protected]/dotnet-buildtools/prereqs:debian-12-helix-amd64 is completed with 24 finished work items.
  Job d39cdf1a-e378-4f8f-a6e0-af041e3208e5 on Windows.10.Amd64.Open is completed with 24 finished work items.

This PR is failing the RPM test on almalinux and debian 12 and skipped on Windows.

@mmitche

@NikolaMilosavljevic
Copy link
Member Author

NikolaMilosavljevic commented Jan 14, 2025

I definitely need to add cpio and file packages to debian-12-helix-amd64 image.

@NikolaMilosavljevic
Copy link
Member Author

NikolaMilosavljevic commented Jan 14, 2025

Before I add cpio and file packages to helix almalinux image, I wonder if there is something wrong with test jobs in arcade. Each test leg (Windows and Linux) sends for testing to three targets, Windows, Ubuntu and almalinux. This seems wrong.

From: https://dev.azure.com/dnceng-public/public/_build/results?buildId=917110&view=logs&j=21143b0a-ace1-5058-215e-b11404388d17&t=b6047ca6-0230-5e90-2195-54379d550513

  Job c0a2b5fc-d9fa-4337-af25-e28ec6a8a902 on (AlmaLinux.8.Amd64.Open)[email protected]/dotnet-buildtools/prereqs:almalinux-8-helix-amd64 is completed with 24 finished work items.
  Job f653c336-8b5d-4237-bebb-5c589e318064 on (Debian.12.Amd64.Open)[email protected]/dotnet-buildtools/prereqs:debian-12-helix-amd64 is completed with 24 finished work items.
  Job d39cdf1a-e378-4f8f-a6e0-af041e3208e5 on Windows.10.Amd64.Open is completed with 24 finished work items.

This PR is failing the RPM test on almalinux and debian 12 and skipped on Windows.

@mmitche

All 3 Helix queues are run regardless of the job target (Windows vs Linux) due to:

arcade/tests/UnitTests.proj

Lines 61 to 65 in 2696b08

<ItemGroup Condition=" '$(HelixAccessToken)' != '' ">
<HelixTargetQueue Include="(AlmaLinux.8.Amd64)[email protected]/dotnet-buildtools/prereqs:almalinux-8-helix-amd64"/>
<HelixTargetQueue Include="Windows.10.Amd64"/>
<HelixTargetQueue Include="(Debian.12.Amd64)[email protected]/dotnet-buildtools/prereqs:debian-12-helix-amd64"/>
</ItemGroup>

@NikolaMilosavljevic NikolaMilosavljevic mentioned this pull request Jan 14, 2025
Merged
@NikolaMilosavljevic
Copy link
Member Author

NikolaMilosavljevic commented Jan 15, 2025

/azp run

@azure-pipelines
Copy link

azure-pipelines bot commented Jan 15, 2025

Azure Pipelines successfully started running 1 pipeline(s).

@NikolaMilosavljevic
Copy link
Member Author

NikolaMilosavljevic commented Jan 15, 2025

All checks are now passing - I've updated the Helix images to add the missing packages with dotnet/dotnet-buildtools-prereqs-docker#1322

@NikolaMilosavljevic NikolaMilosavljevic merged commit fcbe4fd into dotnet:main Jan 15, 2025
11 checks passed
This was referenced Jan 17, 2025
Merged
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mmitche mmitche mmitche approved these changes

@ellahathaway ellahathaway ellahathaway approved these changes

@jkoritzinsky jkoritzinsky Awaiting requested review from jkoritzinsky

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Enable unpack/repack of .rpm containers

3 participants

@NikolaMilosavljevic @mmitche @ellahathaway