.azurepipelines: Add Platform CI template

REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2570

Add template for Platform CI steps for a Pytools based build.
Add README to describe the template and how to use it.
Add helpful information for working with azurepipelines, templates, and
lessons learned.

Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>

Reviewed-by: Shenglei Zhang <shenglei.zhang@intel.com>
Reviewed-by: Bret Barkelew <bret.barkelew@microsoft.com>

2 files changed, 193 insertions, 0 deletions

diff --git a/.azurepipelines/templates/ReadMe.md b/.azurepipelines/templates/ReadMe.md
new file mode 100644
index 0000000000..fa433e3ef5
--- /dev/null
+++ b/.azurepipelines/templates/ReadMe.md
@@ -0,0 +1,59 @@
+# CI Templates

+

+This folder contains azure pipeline yml templates for "Core" and "Platform" Continuous Integration and PR validation.

+

+## Common CI templates

+

+### basetools-build-steps.yml

+

+This template compiles the Edk2 basetools from source.  The steps in this template are

+conditional and will only run if variable `pkg_count` is greater than 0.

+

+It also has two conditional steps only used when the toolchain contains GCC. These two steps

+use `apt` to update the system packages and add those necessary for Edk2 builds.

+

+## Core CI templates

+

+### pr-gate-build-job.yml

+

+This templates contains the jobs and most importantly the matrix of which packages and

+targets to run for Core CI.

+

+### pr-gate-steps.yml

+

+This template is the main Core CI template.  It controls all the steps run and is responsible for most functionality of the Core CI process.  This template sets

+the `pkg_count` variable using the `stuart_pr_eval` tool when the

+build type is "pull request"

+

+### spell-check-prereq-steps.yml

+

+This template installs the node based tools used by the spell checker plugin. The steps

+in this template are conditional and will only run if variable `pkg_count` is greater than 0.

+

+## Platform CI templates

+

+### platform-build-run-steps.yml

+

+This template makes heavy use of pytools to build and run a platform in the Edk2 repo

+

+Also uses basetools-build-steps.yml to compile basetools

+

+#### Special Notes

+

+* For a build type of pull request it will conditionally build if the patches change files that impact the platform.

+  * uses `stuart_pr_eval` to determine impact

+* For manual builds or CI builds it will always build the platform

+* It compiles basetools from source

+* Will use `stuart_build --FlashOnly` to attempt to run the built image if the `Run` parameter is set.

+* See the parameters block for expected configuration options

+* Parameter `extra_install_step` allows the caller to insert extra steps.  This is useful if additional dependencies, tools, or other things need to be installed.  Here is an example of installing qemu on Windows.

+

+    ``` yaml

+    steps:

+    - template: ../../.azurepipelines/templates/build-run-steps.yml

+      parameters:

+        extra_install_step:

+        - powershell: choco install qemu; Write-Host "##vso[task.prependpath]c:\Program Files\qemu"

+          displayName: Install QEMU and Set QEMU on path # friendly name displayed in the UI

+          condition: and(gt(variables.pkg_count, 0), succeeded())

+    ```

diff --git a/.azurepipelines/templates/platform-build-run-steps.yml b/.azurepipelines/templates/platform-build-run-steps.yml
new file mode 100644
index 0000000000..ebf674bfc7
--- /dev/null
+++ b/.azurepipelines/templates/platform-build-run-steps.yml
@@ -0,0 +1,134 @@
+

+## @file

+# File steps.yml

+#

+# template file containing the steps to build

+#

+# Copyright (c) Microsoft Corporation.

+# SPDX-License-Identifier: BSD-2-Clause-Patent

+##

+

+parameters:

+- name: tool_chain_tag

+  type: string

+  default: ''

+- name: build_pkg

+  type: string

+  default: ''

+- name: build_target

+  type: string

+  default: ''

+- name: build_arch

+  type: string

+  default: ''

+- name: build_file

+  type: string

+  default: ''

+- name: build_flags

+  type: string

+  default: ''

+- name: run_flags

+  type: string

+  default: ''

+

+- name: extra_install_step

+  type: stepList

+  default: []

+

+steps:

+- checkout: self

+  clean: true

+  fetchDepth: 1

+

+- task: UsePythonVersion@0

+  inputs:

+    versionSpec: "3.8.x"

+    architecture: "x64"

+

+- script: pip install -r pip-requirements.txt --upgrade

+  displayName: 'Install/Upgrade pip modules'

+

+# Set default

+- bash: echo "##vso[task.setvariable variable=pkg_count]${{ 1 }}"

+

+# trim the package list if this is a PR

+- task: CmdLine@1

+  displayName: Check if ${{ parameters.build_pkg }} need testing

+  inputs:

+    filename: stuart_pr_eval

+    arguments: -c ${{ parameters.build_file }} -t ${{ parameters.build_target}} -a ${{ parameters.build_arch}} --pr-target origin/$(System.PullRequest.targetBranch) --output-count-format-string "##vso[task.setvariable variable=pkg_count;isOutpout=true]{pkgcount}"

+  condition: eq(variables['Build.Reason'], 'PullRequest')

+

+ # Setup repo

+- task: CmdLine@1

+  displayName: Setup

+  inputs:

+    filename: stuart_setup

+    arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}} -t ${{ parameters.build_target}} -a ${{ parameters.build_arch}} ${{ parameters.build_flags}}

+  condition: and(gt(variables.pkg_count, 0), succeeded())

+

+# Stuart Update

+- task: CmdLine@1

+  displayName: Update

+  inputs:

+    filename: stuart_update

+    arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}} -t ${{ parameters.build_target}} -a ${{ parameters.build_arch}} ${{ parameters.build_flags}}

+  condition: and(gt(variables.pkg_count, 0), succeeded())

+

+# build basetools

+#   do this after setup and update so that code base dependencies

+#   are all resolved.

+- template: basetools-build-steps.yml

+  parameters:

+    tool_chain_tag: ${{ parameters.tool_chain_tag }}

+

+# Potential Extra steps

+- ${{ parameters.extra_install_step }}

+

+# Build

+- task: CmdLine@1

+  displayName: Build

+  inputs:

+    filename: stuart_build

+    arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}} TARGET=${{ parameters.build_target}} -a ${{ parameters.build_arch}} ${{ parameters.build_flags}}

+  condition: and(gt(variables.pkg_count, 0), succeeded())

+

+# Run

+- task: CmdLine@1

+  displayName: Run to shell

+  inputs:

+    filename: stuart_build

+    arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}} TARGET=${{ parameters.build_target}} -a ${{ parameters.build_arch}} ${{ parameters.build_flags}} ${{ parameters.run_flags }} --FlashOnly

+  condition: and(and(gt(variables.pkg_count, 0), succeeded()), eq(variables['Run'], true))

+  timeoutInMinutes: 1

+

+# Copy the build logs to the artifact staging directory

+- task: CopyFiles@2

+  displayName: "Copy build logs"

+  inputs:

+    targetFolder: "$(Build.ArtifactStagingDirectory)"

+    SourceFolder: "Build"

+    contents: |

+      BUILDLOG_*.txt

+      BUILDLOG_*.md

+      CI_*.txt

+      CI_*.md

+      CISETUP.txt

+      SETUPLOG.txt

+      UPDATE_LOG.txt

+      PREVALLOG.txt

+      TestSuites.xml

+      **/BUILD_TOOLS_REPORT.html

+      **/OVERRIDELOG.TXT

+      BASETOOLS_BUILD*.*

+    flattenFolders: true

+  condition: succeededOrFailed()

+

+# Publish build artifacts to Azure Artifacts/TFS or a file share

+- task: PublishBuildArtifacts@1

+  continueOnError: true

+  displayName: "Publish build logs"

+  inputs:

+    pathtoPublish: "$(Build.ArtifactStagingDirectory)"

+    artifactName: "Build Logs $(System.JobName)"

+  condition: succeededOrFailed()