Skip to content

Commit 217526a

Browse files
authored
Create simple tag generator for ADO pipelines (#222)
Creates a script and pipeline that allows for automated tagging based on release version and commit messages. This template should be called for an Azure Devops hosted repository. The consumer of this template is expected to ensure the proper permissions for the build agent to be able to create the tag and commit tag notes. The TagGenerator Script provides the following functionality: - Scans git history for the most recent matching tag, e.g. 202302.5.10 - Generates the new tag version, incrementing the "major" version based on the presence of breaking changes. - Generates release notes including commits by type, links to Azure Devops PRs, and contributors.
1 parent bc8d89d commit 217526a

3 files changed

Lines changed: 371 additions & 0 deletions

File tree

Jobs/GenerateTag.yml

Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
## @file
2+
# Template file used to generate tags on ADO. This template requires that the
3+
# consumer specifies this repository as a resource named mu_devops.
4+
#
5+
# Copyright (c) Microsoft Corporation.
6+
# SPDX-License-Identifier: BSD-2-Clause-Patent
7+
##
8+
9+
parameters:
10+
- name: major_version
11+
displayName: The major version.
12+
type: string
13+
default: ""
14+
- name: git_name
15+
displayName: Name to use for creating tag.
16+
type: string
17+
default: ""
18+
- name: git_email
19+
displayName: Email to use for creating tag.
20+
type: string
21+
default: ""
22+
- name: notes_file
23+
displayName: Path to the notes file to generate.
24+
type: string
25+
default: "ReleaseNotes.md"
26+
- name: extra_prepare_steps
27+
displayName: Extra Prepare Steps
28+
type: stepList
29+
default:
30+
- script: echo No extra prepare steps provided
31+
32+
jobs:
33+
- job: Create_Release_Tag
34+
steps:
35+
- checkout: self
36+
clean: true
37+
fetchTags: true
38+
persistCredentials: true
39+
path: "target"
40+
fetchDepth: 0
41+
42+
- checkout: mu_devops
43+
path: "mu_devops"
44+
fetchDepth: 1
45+
46+
- template: ../Steps/SetupPythonPreReqs.yml
47+
parameters:
48+
install_pip_modules: false
49+
50+
- script: |
51+
python -m pip install --upgrade pip
52+
pip install GitPython
53+
displayName: "Install Dependencies"
54+
55+
- ${{ parameters.extra_prepare_steps }}
56+
57+
# Checking the parameters should occur after extra_prepare_steps in case
58+
# the caller is using those steps to initialize a consumed variable.
59+
- script: |
60+
if [ -z "${{ parameters.major_version }}"] || \
61+
[ -z "${{ parameters.git_name }}"] || \
62+
[ -z "${{ parameters.git_email }}"]
63+
then
64+
echo "##vso[task.complete result=Failed;]"
65+
fi
66+
displayName: "Check Parameters"
67+
68+
- script: |
69+
git config --global user.name "${{ parameters.git_name }}"
70+
git config --global user.email "${{ parameters.git_email }}"
71+
displayName: "Setup Git"
72+
73+
- script: |
74+
python mu_devops/Scripts/TagGenerator/TagGenerator.py -r target/ --major ${{ parameters.major_version }} -v --printadovar tag_name --notes target/${{ parameters.notes_file }} --url $(Build.Repository.Uri)
75+
displayName: "Run Tag Generator"
76+
workingDirectory: $(Agent.BuildDirectory)
77+
78+
- script: |
79+
set -e
80+
git branch
81+
git add ${{ parameters.notes_file }}
82+
git commit -m "Release notes for $(tag_name)"
83+
git tag $(tag_name)
84+
git push origin HEAD:$(Build.SourceBranchName)
85+
git push origin $(tag_name)
86+
continueOnError: false
87+
displayName: "Create Tag"
88+
workingDirectory: $(Agent.BuildDirectory)/target

Scripts/TagGenerator/Readme.md

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
# Tag Generator Script
2+
3+
[TagGenerator.py](./TagGenerator.py) will automatically generate the next version tag
4+
and add notes to a release notes file for the current git HEAD. The Tag Generator
5+
script is primarily intended for use by the [Generate Tag Pipeline](../../Jobs/GenerateTag.yml)
6+
but can be used locally as well. This script is intended to be used for ADO repositories,
7+
but may be used for GitHub, though certain features may not work in their current
8+
form such as PR links in tag notes.
9+
10+
## Versioning Scheme
11+
12+
This script uses the `major.minor.patch` versioning scheme, but diverges from semantic
13+
versioning in some significant ways.
14+
15+
- `major version` - Indicates the EDKII release tag that the repo is compiled against, e.g. `202302`.
16+
- `minor version` - Indicates the breaking change number since the last major version change.
17+
- `patch version` - Indicates the number of non-breaking changes since the last minor version.
18+
19+
## Repro Requirements
20+
21+
For this script to work properly it makes assumptions about the repository and
22+
project structure for tag history and generating notes.
23+
24+
### Pull Request Template
25+
26+
To determine what kind of change each commit is, this script expects certain strings
27+
exists in the commit message. It is recommended consumers include these in the PR
28+
templates for the repository. The script expects `[x] Breaking Change` for breaking
29+
changes, `[x] Security Fix` for security changes, and `[x] New Feature` for new
30+
features. The template forms of these are provided below.
31+
32+
```md
33+
- [ ] Breaking Change
34+
- [ ] Security Fix
35+
- [ ] New Feature
36+
```
Lines changed: 247 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,247 @@
1+
#
2+
# Module for automatically tagging a commit with a release version.
3+
#
4+
# Copyright (c) Microsoft Corporation
5+
# SPDX-License-Identifier: BSD-2-Clause-Patent
6+
#
7+
8+
import argparse
9+
import re
10+
import time
11+
import logging
12+
from git import Repo
13+
14+
15+
def main():
16+
"""Main entry point for the TagGenerator script"""
17+
18+
args = get_cli_options()
19+
repo = Repo(args.repo)
20+
log_level = logging.DEBUG if args.verbose else logging.INFO
21+
logging.basicConfig(format="%(levelname)s - %(message)s", level=log_level)
22+
23+
logging.debug(f"Generating tag name for: {repo.head.commit}")
24+
25+
# Get the previous tag and increment values as needed.
26+
prev_tag, breaking, commits = get_last_tag(repo, args.first)
27+
28+
# Generate the new tag name
29+
minor = 0
30+
patch = 0
31+
if prev_tag is not None:
32+
if prev_tag.commit == repo.head.commit:
33+
logging.info("No changes since last tag")
34+
return
35+
36+
version_split = prev_tag.name.split('.')
37+
if version_split[0] == args.major:
38+
minor = int(version_split[1])
39+
patch = int(version_split[2])
40+
if breaking:
41+
minor += 1
42+
patch = 0
43+
else:
44+
patch += 1
45+
else:
46+
logging.critical(
47+
f"Different major version. {version_split[0]} -> {args.major}")
48+
if int(version_split[0]) > int(args.major):
49+
raise Exception("Major version has decreased!")
50+
51+
version = f"{args.major}.{minor}.{patch}"
52+
logging.info(f"New tag: {version}")
53+
54+
# Before going further, ensure this is not a duplicate. This can happen if
55+
# there are tags detached from their intended branch.
56+
for tag in repo.tags:
57+
if tag.name == version:
58+
raise Exception(
59+
"The new tag name already exists! Check tags already present in the repo.")
60+
61+
if args.create:
62+
repo.create_tag(version, message=f"Release Tag {version}")
63+
64+
if args.notes is not None:
65+
generate_notes(version, commits, args.notes, args.url)
66+
67+
if args.printadovar is not None:
68+
print(f"##vso[task.setvariable variable={args.printadovar};]{version}")
69+
70+
71+
def get_cli_options():
72+
parser = argparse.ArgumentParser()
73+
74+
parser.add_argument("-r", "--repo", default=".",
75+
help="Path to the repo directory.")
76+
parser.add_argument("-m", "--major", type=str, required=True,
77+
help="The major release version. This must be provided")
78+
parser.add_argument("-n", "--notes", type=str,
79+
help="Provides path to the release notes markdown file.")
80+
parser.add_argument("--printadovar", type=str,
81+
help="An ADO variable to set to the tag name")
82+
parser.add_argument("--url", type=str, default="",
83+
help="The URL to the repo, used for tag notes.")
84+
parser.add_argument("--create", action="store_true",
85+
help="Create the new tag")
86+
parser.add_argument("--first", action="store_true",
87+
help="Indicates this is expected to be the first tag.")
88+
parser.add_argument("-v", "--verbose", action="store_true",
89+
help="Enabled verbose script prints.")
90+
91+
args = parser.parse_args()
92+
return args
93+
94+
95+
def get_last_tag(repo, first):
96+
"""Retrieves the last tag name in the given HEAD history. This will
97+
exclude any tag that does not match the #.#.# format.
98+
99+
repo - Provides the Git Repo object which will be searched.
100+
first - Indicates this may be the first tag generation run.
101+
"""
102+
103+
breaking = False
104+
included_commits = []
105+
commits = repo.iter_commits(repo.head.commit)
106+
107+
# Find all the eligible tags first.
108+
tags = []
109+
pattern = re.compile("^[0-9]+\.[0-9]+\.[0-9]+$")
110+
for tag in repo.tags:
111+
if pattern.match(tag.name) is None:
112+
logging.debug(f"Skipping unrecognized tag format. Tag: {tag}")
113+
continue
114+
115+
tags.append(tag)
116+
117+
# Find the most recent commit with a tag.
118+
for commit in commits:
119+
if is_breaking_change(commit.message):
120+
breaking = True
121+
122+
logging.debug(f"Checking commit {commit.hexsha}")
123+
for tag in tags:
124+
if tag.commit == commit:
125+
logging.info(f"Previous tag: {tag} Breaking: {breaking}")
126+
return tag, breaking, included_commits
127+
128+
included_commits.append(commit)
129+
130+
if not first:
131+
raise Exception("No previous tag found!")
132+
133+
# No tag found, return all commits and non-breaking.
134+
logging.info("No previous tag found.")
135+
return None, False, commits
136+
137+
138+
def generate_notes(version, commits, filepath, url):
139+
"""Generates notes for the provided tag version including the provided commit
140+
list. These notes will include the list of Breaking, Security, and other
141+
commits. These notes will be prepended to the file specify by filepath
142+
143+
version - The tag version string.
144+
commits - The list of commits since the last tag.
145+
filepath - The path to the file to prepend the notes to.
146+
url - The URL of the repository.
147+
"""
148+
149+
notes_file = open(filepath, 'r+')
150+
old_lines = notes_file.readlines()
151+
152+
# Collect all the notable changes
153+
breaking_changes = []
154+
security_changes = []
155+
features = []
156+
other_changes = []
157+
contributors = []
158+
159+
for commit in commits:
160+
if commit.author not in contributors:
161+
contributors.append(commit.author)
162+
163+
if is_breaking_change(commit.message):
164+
breaking_changes.append(commit)
165+
elif is_security_change(commit.message):
166+
security_changes.append(commit)
167+
elif is_new_feature(commit.message):
168+
features.append(commit)
169+
else:
170+
other_changes.append(commit)
171+
172+
timestamp = time.strftime("%a, %D %T", time.gmtime())
173+
notes = f"\n# Release {version}\n\n"
174+
notes += f"Created {timestamp} GMT\n\n"
175+
notes += f"{len(commits)} commits. {len(contributors)} contributors.\n"
176+
177+
if len(breaking_changes) > 0:
178+
notes += f"\n## Breaking Changes\n\n"
179+
notes += get_change_list(breaking_changes, url)
180+
181+
if len(security_changes) > 0:
182+
notes += f"\n## Security Changes\n\n"
183+
notes += get_change_list(security_changes, url)
184+
185+
if len(features) > 0:
186+
notes += f"\n## New Features\n\n"
187+
notes += get_change_list(features, url)
188+
189+
if len(other_changes) > 0:
190+
notes += f"\n## Changes\n\n"
191+
notes += get_change_list(other_changes, url)
192+
193+
notes += "\n## Contributors\n\n"
194+
for contributor in contributors:
195+
notes += f"- {contributor.name} <<{contributor.email}>>"
196+
197+
notes += "\n"
198+
199+
# Add new notes at the top and write out existing content.
200+
notes_file.seek(0)
201+
notes_file.write(notes)
202+
for line in old_lines:
203+
notes_file.write(line)
204+
notes_file.close()
205+
206+
207+
def get_change_list(commits, url):
208+
"""Generates a list of changes for the given commits. The routine will attempt
209+
to create links to the appropriate ADO pages from the URL script argument where
210+
applicable.
211+
212+
commits - The list of commits which to create the list for
213+
url - The URL of the repository.
214+
"""
215+
216+
changes = ""
217+
218+
for commit in commits:
219+
pr = None
220+
msg = commit.message.split('\n', 1)[0]
221+
match = re.match('Merged PR [0-9]+:', msg, flags=re.IGNORECASE)
222+
if match:
223+
pr = msg[len("Merged PR "):match.end() - 1]
224+
msg = f"[{msg[0:match.end()]}]({url}/pullrequest/{pr}){msg[match.end():]}"
225+
226+
changes += f"- {msg} ~ _{commit.author}_\n"
227+
228+
return changes
229+
230+
231+
def is_breaking_change(message):
232+
"""Checks if the given commit message contains the breaking change tag"""
233+
return re.search('\[x\] breaking change', message, flags=re.IGNORECASE) is not None
234+
235+
236+
def is_security_change(message):
237+
"""Checks if the given commit message contains the security change tag"""
238+
return re.search('\[x\] security fix', message, flags=re.IGNORECASE) is not None
239+
240+
241+
def is_new_feature(message):
242+
"""Checks if the given commit message contains the new feature tag"""
243+
return re.search('\[x\] new feature', message, flags=re.IGNORECASE) is not None
244+
245+
246+
if __name__ == '__main__':
247+
main()

0 commit comments

Comments
 (0)