windows: enable bootstrapping#89988
Closed
charles-zablit wants to merge 33 commits into
Closed
Conversation
This is preparatory work to allow us to build multiple "experimental"-style SDKs for bootstrapping. (cherry picked from commit 995cfa2)
We already have a checkout for the dependencies of early-swift-driver. Reuse the existing sources rather than checking them out from git again. (cherry picked from commit f230c75)
(cherry picked from commit f6e4c72)
(cherry picked from commit 06c0608)
Use Remove-Item with -ErrorAction Stop for the output directories removed by -Clean. This keeps missing paths non-fatal, but aborts the build when a locked file or another removal error would leave stale output behind. Also shorten the -Clean help text. (cherry picked from commit 506b7e0)
Compute driver-specific compile and debug flags from compiler descriptors instead of keeping that logic inside Build-CMakeProject. This keeps Build-CMakeProject focused on applying a selected compiler, and makes later bootstrap-stage compiler definitions a straightforward extension of the same model. (cherry picked from commit 02d4020)
Thread explicit compiler selections through the SDK leaf builders for dispatch, zlib, brotli, libxml2, and curl. This lets each SDK phase build its dependencies with the compiler for that stage instead of reaching back to the pinned or host compiler globals. (cherry picked from commit e3005b0)
Add the first Windows bootstrap compiler image under the stage0 project cache. Install the compiler, Clang resource pieces, Swift Clang builtin headers, lld, and the SwiftSyntax DLLs needed by the staged compiler. Limit LLVM targets to the build architecture. The Bootstrap SDK runtime build invokes the stage0 Swift driver, which expects lld-link beside the staged tools. Copy the pinned runtime and cmark DLLs into the image so downstream bootstrap steps can run tools by absolute path. (cherry picked from commit 8cbe68a)
Introduce Build-SDK for staged SDK construction, and use it to build the Bootstrap SDK with the stage0 compiler. The bootstrap SDK keeps the surface needed by the next compiler stage without building Foundation, XCTest, Testing, or the full distribution supplemental set. (cherry picked from commit 289f920)
Add the second Windows bootstrap compiler cache and build stage1 with the stage0 compiler against the Bootstrap SDK. Stage1 expands the target set and keeps the tools needed by downstream SDK builds, including lld, the autolink driver, llvm-ar, llvm-ranlib, and the SwiftSyntax DLLs needed by the staged compiler. Install it under its own project cache so stage0 and stage1 cannot pollute each other. (cherry picked from commit d81f0bd)
Build the host SDK image with the stage1 compiler before configuring the stage2 compiler and toolchain consumers. This gives the self-hosting compiler a staged SDK produced by the previous compiler stage instead of relying on the pinned toolchain SDK for the rest of the build. (cherry picked from commit 7fad743)
Use the stage1 compiler and staged host SDK to build the stage2 compiler. Then build the host toolchain components against that same SDK. The host dynamic SDK slice can be reused by the later distribution SDK pass, keeping the self-hosted build from creating a second independent host runtime image. (cherry picked from commit 034a158)
Replace the experimental-only SDK helper with the generic Build-SDK path for staged SDK builds. This removes duplicated runtime, overlay, and SDK construction logic so the bootstrap, host, and distribution SDK phases share the same builder. (cherry picked from commit 3456fcc)
Remove the legacy SDK selection and version plumbing now that the staged SDK builder produces the platform SDK directly. Keep package-manager runtime, runtime packaging, and SDK installation consumers on the canonical platform SDK layout instead of a separate Toolchain or Experimental image. (cherry picked from commit f7d8d9e)
When CMake configures Android builds, it may auto-detect the NDK clang as the target compiler. Sending that compiler through clang-cache.exe from the staged Windows toolchain can mix LLVM versions. That version skew makes dependency scanning look for builtin headers in the wrong resource directory. Skip the launcher for Android SDK builds while keeping caching for paths that use the staged compiler directly. (cherry picked from commit 0fb741a)
The staged compiler table carries MSVC-style clang-cl entries in .C/.CXX and GNU-style clang entries in .GNUC/.GNUCXX. Select the GNU drivers for non-Windows SDK dependencies, compiler-rt, and dispatch so Android builds do not receive MSVC-only flags such as /GS- and /Gw. (cherry picked from commit 209d3c8)
Build Android swift-inspect with a target-aware compiler view that exposes the stage1 GNU clang drivers under .C/.CXX. This keeps the NDK clang from seeing MSVC-style flags while still using the stage1 Swift compiler and Android SDK. (cherry picked from commit fc7c61c)
Add Set-WindowsManifestDependencies to merge dependency entries into an embedded manifest and re-embed it with mt. This gives later SxS binding steps a shared path for binding tools without replacing unrelated manifest content. (cherry picked from commit 5450656)
Stage each Swift runtime DLL needed by the laid-out toolchain as a private per-DLL SxS assembly under usr/bin. Compute each EXE's static import graph and write the full Swift runtime closure into the EXE's application manifest. Disable linker-generated DLL manifests for Swift's CMake-built DLLs and for the mimalloc MSBuild project, then validate existing DLL manifest resources before binding: reject UAC execution-level metadata and unexpected reserved RT_MANIFEST resource IDs so implicit linker manifests cannot poison SxS activation. Keep the checked-in WiX authoring canonical and warn when the live import graph drifts from bld/cli/dbg/ide package ownership. (cherry picked from commit a4c76ff)
Build the Stage2 test-time tools needed by check-swift before the final test build. Bind the runtime-loading EXEs to the platform SDK runtime they are tested against. Use the build-host architecture so host-run tools remain runnable when the final toolchain target differs. Keep this separate from the distribution SxS pass because these tools execute from the build tree. (cherry picked from commit 7249c26)
Run the dispatch, XCTest, and llbuild test builds with the Stage1 compiler artifacts instead of the legacy Compilers cache. The swift-format and SourceKit-LSP SwiftPM tests also need SwiftSyntax from the Stage2 compiler build, matching the CMake build rules for those projects. Use the Stage2 host library path there and fix the stale swift-markdown CAtomic include path in the SourceKit-LSP test arguments. (cherry picked from commit c68327f)
Disable dispatch in the Windows bootstrap compiler caches. Stage0 and Stage1 do not build SourceKit or the runtime SDK, so they do not need Swift's Libdispatch.cmake path to configure nested libdispatch ExternalProjects. Keep passing the staged DynamicCDispatch package to the Stage2 compiler build, where SourceKit and tool components still need host dispatch. Drop the Stage0CDispatch build and stop feeding dispatch packages to Stage0 and Stage1. (cherry picked from commit 3c71b48)
When cross-compiling the NoAsserts toolchain, we would no copy the mimalloc into the staged NoAsserts toolchain. This would break the packaging as that expects to find the dependency for distribution. (cherry picked from commit 659990e)
This simplifies the use of the script: `-Toolchain`: build the toolchain (default) `-Windows`: build the Windows SDK (opt-in) `-Android`: build the Android SDK (opt-in) `-Package`: package the toolchain (opt-in) (cherry picked from commit 4c0fa34)
We now have the early swift driver + bootstrapping. At this point, everything that is needed to enable WMO is available. Restore the optimization when building the toolchain to avoid unnecessary pessimisation. (cherry picked from commit 2115df5)
Runtime clients of swiftCore need the SwiftRT image-registration object on their link lines. On COFF this object supplies pSwiftImageConstructor and the .sw5* section range symbols used to register Swift metadata and conformances for each image. CMake propagates usage requirements when an OBJECT library appears in an INTERFACE_LINK_LIBRARIES chain, but it does not add that object's payload to the final consumer link line. Keep the swiftrt targets in the interface for their usage requirements, and explicitly propagate their object files with TARGET_OBJECTS. (cherry picked from commit e86c870)
Allow non-Darwin compiler builds to consume an already-built libdispatch package instead of always creating a nested ExternalProject. When dispatch_DIR is set, import the dispatch package and use it for the host SDK dispatch target. Continue building libdispatch for other non-Darwin target SDKs so cross-compilation still gets target-specific dispatch variants. (cherry picked from commit 604bc52)
When %host-build-swift builds plugin DLLs on Windows, those DLLs are loaded by the compiler under test. Pass the configured host SDK and resource directory so the plugins link against the runtime that compiler will load instead of the Stage2 stdlib resources. (cherry picked from commit fc6d841)
Embed the Windows RT_MANIFEST entry through the target resource script instead of post-processing the binary with mt.exe. Keep VERSIONINFO and the assembly manifest in the same generated .rc file so the resource state is deterministic. Disable linker-generated manifests for runtime DLLs. Runtime DLLs should only carry the assembly manifest we embed, because reserved RT_MANIFEST resource IDs have loader semantics. (cherry picked from commit 2bafcc3)
Preserve an explicitly configured SWIFT_HOST_SDKROOT instead of always deriving it from SWIFT_SDKS. This lets downstream test drivers point lit at the SDK that matches the host compiler under test. (cherry picked from commit d926de4)
charles-zablit
changed the title
Build-SDK to Build-LegacySDK
Contributor
Author
|
@swift-ci please test |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is a strict copy of #89719 with 064c699 cherry picked on top.
The original PR should be merged instead.