TASK 38260338: Updated the release pipeline to produce Linux builds capable of complying with the FIPS 140-3 standard.

[al-msft]

Microsoft's version of Go provides the FIPS capability. The FIPS-capable builds are produced alongside the existing builds, which don't use Microsoft's version of Go.

Description

The CSI driver image is built by another team at Microsoft. That image needs to comply with the FIPS 140-3 standard by June 30th. A FIPS-compliance checker scans all binaries in the image; AzCopy is one of those binaries. To pass the FIPS-compliance check, each binary must be built with the Microsoft build of Go in the particular way the check expects and pass a run-time check that considers the host OS. The CSI driver image downloads azcopy as an RPM package on Azure Linux 3.0. To reduce the risk of migrating AzCopy from the official build of Go to the Microsoft build of Go, it was decided to release FIPS-capable builds of AzCopy - only for Linux - alongside the existing builds, and not to release Docker container images containing FIPS-capable builds. All existing Linux builds will now have a FIPS-capable analog, other than the Mariner 2 builds, because Mariner 2 has reached end of life.

This PR includes only the changes to the release pipeline. The changes to the test pipeline will be included in a separate PR, once the FIPS-enabled tests are working in the test pipeline (they have to be run on a new, FIPS-enabled agent, and the only available agent image, which runs Azure Linux 3.0, doesn't work correctly with workload identity).

Cut for Time/Risk:

• FIPS-capable builds for any OS other than Linux
• FIPS-capable builds in Docker container images
• Updating GitHub Actions
• Logging a message when AzCopy is invoked that states whether FIPS is enabled
• Updating the performance-test pipeline to test the FIPS-capable builds
• Addressing technical debt in the pipelines
• Optimizing the run-time, parallelism, amount of duplication, and the use of parameters and variables in the pipelines
• Scanning the FIPS-capable binaries for FIPS compliance in the pipeline(s)
• Validating the details of the DEB and RPM packages in the release pipeline (probably in verify_linux.xml) - especially the descriptions and conflicts this PR changed, but also the pre-existing details, like version, maintainer, and URL

Related Links:

• TASK 38260338
• BlobFuse FIPS Teams chat
• AZCopy FIPS Compliance Teams chat
• FIPS Support in the Microsoft build of Go (Warning: This is misleading in some places)

Type of Change

• Bug fix
• New feature
• Documentation update required
• Code quality improvement
• Other (describe):

How Has This Been Tested?

• The test pipeline (which runs on PRs, including this one) is being updated in TASK 38260338: Updated the pipelines to produce Linux builds capable of complying with the FIPS 140-3 standard. #3482 to additionally test the FIPS-capable AMD64 Linux build in two distinct scenarios: with FIPS enabled and with FIPS disabled. That pipeline has been manually run several times, and although there are still issues specific to the agent being used to test with FIPS enabled, some tests pass, and the same tests ultimately fail each time. For example, see the test results for https://dev.azure.com/azstorage/AzCopy-NextGen/_build/results?buildId=30978&view=results.
• The release pipeline has been run on this branch without actually publishing the artifacts to packages.microsoft.com or GitHub.
• The artifacts the updated release pipeline creates have been scanned by manually running a FIPS-compliance checker we were instructed to use and that is part of the same tool suite as the FIPS-compliance checker used to scan the CSI driver image and the binaries within it.
• The package info for the FIPS and non-FIPS .deb and .rpm packages the updated release pipeline creates have been manually reviewed. The conflict indicators in the AMD64 FIPS and non-FIPS .deb and .rpm packages (other than the SE Linux packages and the untouched .rpm packages for Mariner 2) have been tested on FIPS-disabled Ubuntu (with the .deb packages) and FIPS-enabled Azure Linux 3.0 (with the .rpm packages) in various combinations of the FIPS and non-FIPS packages the updated release pipeline creates and previous azcopy packages that lack a conflict indicator.
• The AMD64 FIPS and non-FIPS .deb and .rpm packages the updated release pipeline creates (other than the SE Linux packages and the untouched .rpm packages for Mariner 2) have been installed on FIPS-disabled Ubuntu (with the .deb packages) and FIPS-enabled Azure Linux 3.0 (with the .rpm packages). After installing each package, azcopy was invoked with and without GOFIPS-1, and (as intended) it only panicked when the FIPS .deb package was installed on Ubuntu and azcopy was invoked with GOFIPS-1.