Fix ApiValidator failures, harden Dvl.exe check, parallelize test runner

- Add src/drivers/test/Directory.Build.targets that overrides the WDK's
  ApiValidator MSBuild target with an empty target.  The WDK NuGet packages
  reference ApiValidator.exe in WindowsDriver.common.targets but do not ship
  the binary itself, so the post-build step always failed with MSB3721 for
  builds using the Universal driver target platform.  Replacing the target
  with a no-op cleanly suppresses the step; API validation is irrelevant
  for the CodeQL static analysis these tests perform.  This is the cause of
  the 6 still-failing tests (UnsafeCallInGlobalInit, MultithreadedAVCondition,
  StaticInitializer, DeviceInitApi, FloatSafeExit, FloatUnsafeExit).

- Remove the previous <ApiValidatorEnabled>false</ApiValidatorEnabled>
  workaround from Directory.Build.props -- the WDK targets do not honor that
  property name, so it had no effect.

- dvl_tests.ps1: when Dvl.exe cannot be located in either the NuGet packages
  directory or the system WDK install path, exit with a clear failure instead
  of silently skipping the dvl command-type tests.  Skipping let regressions
  go undetected.

- build_create_analyze_test.py: parallelize the test runner.  Each ql_test
  uses isolated working/<name>, TestDB/<name>, and AnalysisFiles/<name>.sarif
  paths, so multiple tests are safe to execute concurrently.  Use
  multiprocessing.pool.ThreadPool (already imported) with a worker count
  controlled by a new -j/--jobs flag, defaulting to os.cpu_count().  Pass
  --jobs 1 to fall back to the legacy sequential behaviour.  Print output
  is already protected by print_mutex; added results_mutex around the shared
  health_df / detailed_health_df DataFrame writes.  Refactored the body of
  the per-test loop into _run_single_test for use by the pool's
  imap_unordered.

Agent-Logs-Url: https://github.com/microsoft/Windows-Driver-Developer-Supplemental-Tools/sessions/71e6036e-2c98-4bb7-85ab-2270cbce9c68

Co-authored-by: NateD-MSFT <34494373+NateD-MSFT@users.noreply.github.com>

Author: Copilot

src/drivers/test/Directory.Build.props

@@ -14,14 +14,6 @@
   -->
   <PropertyGroup>
     <WdkNuGetPackagesDir>$(MSBuildThisFileDirectory)..\..\..\packages\</WdkNuGetPackagesDir>
-    <!--
-      ApiValidator.exe is not shipped in the WDK NuGet packages used by this
-      repo's CI (it requires a full WDK install).  Disable the post-build API
-      validation step so that builds using the WindowsApplicationForDrivers10.0
-      toolset (e.g. ApplicationForDriversTestTemplate) can succeed without it.
-      The validation is irrelevant for the CodeQL analysis we are performing.
-    -->
-    <ApiValidatorEnabled>false</ApiValidatorEnabled>
   </PropertyGroup>
   <Import Project="$(WdkNuGetPackagesDir)Microsoft.Windows.WDK.x64.10.0.26100.6584\build\native\Microsoft.Windows.WDK.x64.props" Condition="Exists('$(WdkNuGetPackagesDir)Microsoft.Windows.WDK.x64.10.0.26100.6584\build\native\Microsoft.Windows.WDK.x64.props') and '$(Platform)' == 'x64'" />
   <Import Project="$(WdkNuGetPackagesDir)Microsoft.Windows.WDK.arm64.10.0.26100.6584\build\native\Microsoft.Windows.WDK.arm64.props" Condition="Exists('$(WdkNuGetPackagesDir)Microsoft.Windows.WDK.arm64.10.0.26100.6584\build\native\Microsoft.Windows.WDK.arm64.props') and '$(Platform)' == 'ARM64'" />

src/drivers/test/Directory.Build.targets

@@ -0,0 +1,28 @@
+<Project>
+  <!--
+    Auto-imported by MSBuild AFTER each project's main targets (including the
+    WDK targets pulled in by Directory.Build.props), for every project under
+    src\drivers\test\.
+
+    Override the WDK's `ApiValidator` post-build target with an empty target.
+    The WDK NuGet packages restored by the CI workflow do not ship the
+    `ApiValidator.exe` binary even though they reference its path in
+    `WindowsDriver.common.targets`.  Without this override the target fails
+    with:
+
+        error MSB3721: The command "...\ApiValidator.exe -DriverPackagePath:..."
+        exited with code 1
+
+    breaking the `codeql database create` invocation for tests built with the
+    Universal driver target platform (UnsafeCallInGlobalInit,
+    MultithreadedAVCondition, StaticInitializer, DeviceInitApi, FloatSafeExit,
+    FloatUnsafeExit, ...).
+
+    API validation is irrelevant to the CodeQL static analysis these tests
+    perform, so replacing the target with a no-op is safe here.  Setting
+    `ApiValidatorExePath` to an empty value would still leave the target
+    running an empty exec; replacing the target itself is the most reliable
+    suppression.
+  -->
+  <Target Name="ApiValidator" />
+</Project>

src/drivers/test/build_create_analyze_test.py

@@ -22,6 +22,7 @@
 
 
 print_mutex = threading.Lock()
+results_mutex = threading.Lock()
 health_df = pd.DataFrame()
 detailed_health_df = pd.DataFrame()
 
@@ -830,6 +831,36 @@ def compare_health_results(curr_results_path):
         os.remove(prev_results)
         exit(0)
 
+def _run_single_test(ql_test):
+    """
+    Run a single ql_test and update the shared results DataFrames.
+    Returns the test name on failure, or None on success.  Designed to be
+    safe to invoke concurrently from a thread pool: each ql_test uses its
+    own working/, TestDB/, and AnalysisFiles/<name>.sarif paths, and shared
+    DataFrame mutations are guarded by ``results_mutex``.
+    """
+    result_sarif = run_test(ql_test)
+    if args.build_database_only:
+        return None
+    if not result_sarif:
+        with print_mutex:
+            print("Error running test: " + ql_test.get_ql_name(), "Skipping...")
+        return ql_test.get_ql_name()
+    try:
+        analysis_results, detailed_analysis_results = sarif_results(ql_test, result_sarif)
+    except Exception as e:
+        with print_mutex:
+            print("Error reading sarif results for " + ql_test.get_ql_name() + ": " + str(e))
+        return ql_test.get_ql_name()
+    with results_mutex:
+        health_df.at[ql_test.get_ql_name(), "Result"] = str(
+            int(analysis_results['error']) +
+            int(analysis_results['warning']) +
+            int(analysis_results['note']))
+        detailed_health_df.at[ql_test.get_ql_name(), "Result"] = str(detailed_analysis_results)
+    return None
+
+
 def run_tests(ql_tests_dict):
     """
     Run the given CodeQL tests.
@@ -842,20 +873,32 @@ def run_tests(ql_tests_dict):
     """
     ql_tests_with_attributes = parse_attributes(ql_tests_dict)
 
-    total_tests = 0
+    total_tests = len(ql_tests_with_attributes)
     failed_tests = []
 
-    for ql_test in ql_tests_with_attributes:
-        total_tests += 1
-        result_sarif = run_test(ql_test)
-        if not args.build_database_only:
-            if not result_sarif:
-                print("Error running test: " + ql_test.get_ql_name(),"Skipping...")
-                failed_tests.append(ql_test.get_ql_name())
-                continue
-            analysis_results, detailed_analysis_results = sarif_results(ql_test, result_sarif)
-            health_df.at[ql_test.get_ql_name(), "Result"] = str(int(analysis_results['error'])+int(analysis_results['warning'])+int(analysis_results['note']))
-            detailed_health_df.at[ql_test.get_ql_name(), "Result"] = str(detailed_analysis_results) 
+    # Each ql_test runs against its own isolated working/, TestDB/, and
+    # AnalysisFiles/<name>.sarif paths, so it is safe to execute multiple
+    # tests concurrently.  ``--jobs`` controls the worker count; the default
+    # of ``os.cpu_count()`` mirrors what `msbuild` and `codeql` do internally
+    # for their own parallelism.  Use jobs=1 to fall back to the legacy
+    # sequential behaviour (useful when debugging a single test).
+    jobs = max(1, args.jobs) if args.jobs is not None else (os.cpu_count() or 1)
+    if jobs == 1:
+        for ql_test in ql_tests_with_attributes:
+            failed = _run_single_test(ql_test)
+            if failed is not None:
+                failed_tests.append(failed)
+    else:
+        with print_mutex:
+            print("Running " + str(total_tests) + " tests with " + str(jobs) + " parallel workers")
+        pool = ThreadPool(jobs)
+        try:
+            for failed in pool.imap_unordered(_run_single_test, ql_tests_with_attributes):
+                if failed is not None:
+                    failed_tests.append(failed)
+        finally:
+            pool.close()
+            pool.join()
 
     # save results
     result_file = "functiontestresults.xlsx"
@@ -950,6 +993,7 @@ def find_sln_file(path):
     parser.add_argument('--compare_results_no_build',help='Compare results to previous run',type=str,required=False,)
     parser.add_argument('--codeql_path', help='Path to the codeql executable',type=str,required=False,)
     parser.add_argument('--build_database_only', help='Build database only',action='store_true',required=False,)
+    parser.add_argument('-j', '--jobs', help='Number of tests to run in parallel (default: number of CPU cores). Use 1 to disable parallelism.', type=int, required=False)
     args = parser.parse_args()
 
     if args.codeql_path: