Usecase
We have Azure VMSS (build agents) with U24 image and installing Azure Monitor Linux Agent extension in it.
We have noticed that the agents remain unhealthy affecting the pipeline to wait for couple of hours to start.
Investigation
While checking logs after loging into a VM, it seems pre-install script is failing
azureuser@agents000004:~$ cat /var/log/azure/Microsoft.Azure.Monitor.AzureMonitorLinuxAgent/extension.log
2026/05/07 14:32:51 [Microsoft.Azure.Monitor.AzureMonitorLinuxAgent-1.41.0] sequence number is 0
2026/05/07 14:32:51 [Microsoft.Azure.Monitor.AzureMonitorLinuxAgent-1.41.0] setting file path is/var/lib/waagent/Microsoft.Azure.Monitor.AzureMonitorLinuxAge
nt-1.41.0/config/0.settings
2026/05/07 14:32:51 [Microsoft.Azure.Monitor.AzureMonitorLinuxAgent-1.41.0] JSON config:
2026/05/07 14:32:51 ERROR:[Microsoft.Azure.Monitor.AzureMonitorLinuxAgent-1.41.0] JSON exception decoding
2026/05/07 14:32:51 ERROR:[Microsoft.Azure.Monitor.AzureMonitorLinuxAgent-1.41.0] JSON error processing settings file:
2026/05/07 14:32:51 [Microsoft.Azure.Monitor.AzureMonitorLinuxAgent-1.41.0] OS detected from /etc/os-release: ubuntu 24.04
2026/05/07 14:32:51 [Microsoft.Azure.Monitor.AzureMonitorLinuxAgent-1.41.0] Final OS detection result: ubuntu 24.04
2026/05/07 14:32:51 [Microsoft.Azure.Monitor.AzureMonitorLinuxAgent-1.41.0] OS detected from /etc/os-release: ubuntu 24.04
2026/05/07 14:32:51 [Microsoft.Azure.Monitor.AzureMonitorLinuxAgent-1.41.0] Final OS detection result: ubuntu 24.04
2026/05/07 14:32:51 [Microsoft.Azure.Monitor.AzureMonitorLinuxAgent-1.41.0] OS detected from /etc/os-release: ubuntu 24.04
2026/05/07 14:32:51 [Microsoft.Azure.Monitor.AzureMonitorLinuxAgent-1.41.0] Final OS detection result: ubuntu 24.04
2026/05/07 14:32:52 [Microsoft.Azure.Monitor.AzureMonitorLinuxAgent-1.41.0] Output of command "dpkg-query -W -f='${Package}_${Version}
2026/05/07 14:32:52 ' 'azuremonitoragent*' 2>/dev/null":
2026/05/07 14:32:52 azuremonitoragent_1.40.3-1349
2026/05/07 14:32:52 [Microsoft.Azure.Monitor.AzureMonitorLinuxAgent-1.41.0] Found installed azuremonitoragent version(s): ['azuremonitoragent_1.40.3-1349']
2026/05/07 14:32:52 [Microsoft.Azure.Monitor.AzureMonitorLinuxAgent-1.41.0] Current bundle file: azuremonitoragent_1.41.0-1382_x86_64
2026/05/07 14:32:52 [Microsoft.Azure.Monitor.AzureMonitorLinuxAgent-1.41.0] Installing Azure Monitor Agent package.
2026/05/07 14:32:52 [Microsoft.Azure.Monitor.AzureMonitorLinuxAgent-1.41.0] Running command "dpkg --force-overwrite --force-confnew -i /var/lib/waagent/Micro
soft.Azure.Monitor.AzureMonitorLinuxAgent-1.41.0/packages/azuremonitoragent_1.41.0-1382_x86_64.deb"
2026/05/07 14:33:27 ERROR:CalledProcessError. Error Code is 1
2026/05/07 14:33:27 ERROR:CalledProcessError. Command string was dpkg --force-overwrite --force-confnew -i /var/lib/waagent/Microsoft.Azure.Monitor.AzureMon
itorLinuxAgent-1.41.0/packages/azuremonitoragent_1.41.0-1382_x86_64.deb
2026/05/07 14:33:27 ERROR:CalledProcessError. Command result was (Reading database ... 133536 files and directories currently installed.)
2026/05/07 14:33:27 ERROR:Preparing to unpack .../azuremonitoragent_1.41.0-1382_x86_64.deb ...
2026/05/07 14:33:27 ERROR:PRE-RM: [upgrade 1.41.0-1382] [ubuntu 24.04]
2026/05/07 14:33:27 ERROR:PRE-INSTALL: [upgrade 1.40.3-1349] [ubuntu 24.04]
2026/05/07 14:33:27 ERROR:dpkg: error processing archive /var/lib/waagent/Microsoft.Azure.Monitor.AzureMonitorLinuxAgent-1.41.0/packages/azuremonitoragent_1.
41.0-1382_x86_64.deb (--install):
2026/05/07 14:33:27 ERROR: new azuremonitoragent package pre-installation script subprocess returned error exit status 1
2026/05/07 14:33:27 ERROR:POST-RM: [abort-upgrade 1.40.3-1349] [ubuntu 24.04 deb]
2026/05/07 14:33:27 ERROR:POST-INSTALL: [abort-upgrade 1.41.0-1382] [ubuntu 24.04 deb]
2026/05/07 14:33:27 ERROR:Errors were encountered while processing:
2026/05/07 14:33:27 ERROR: /var/lib/waagent/Microsoft.Azure.Monitor.AzureMonitorLinuxAgent-1.41.0/packages/azuremonitoragent_1.41.0-1382_x86_64.deb
2026/05/07 14:33:27 [Microsoft.Azure.Monitor.AzureMonitorLinuxAgent-1.41.0] Output of command "dpkg --force-overwrite --force-confnew -i /var/lib/waagent/Mic
rosoft.Azure.Monitor.AzureMonitorLinuxAgent-1.41.0/packages/azuremonitoragent_1.41.0-1382_x86_64.deb":
2026/05/07 14:33:27 (Reading database ... 133536 files and directories currently installed.)
2026/05/07 14:33:27 Preparing to unpack .../azuremonitoragent_1.41.0-1382_x86_64.deb ...
2026/05/07 14:33:27 PRE-RM: [upgrade 1.41.0-1382] [ubuntu 24.04]
2026/05/07 14:33:27 PRE-INSTALL: [upgrade 1.40.3-1349] [ubuntu 24.04]
2026/05/07 14:33:27 dpkg: error processing archive /var/lib/waagent/Microsoft.Azure.Monitor.AzureMonitorLinuxAgent-1.41.0/packages/azuremonitoragent_1.41.0-1
382_x86_64.deb (--install):
2026/05/07 14:33:27 new azuremonitoragent package pre-installation script subprocess returned error exit status 1
2026/05/07 14:33:27 POST-RM: [abort-upgrade 1.40.3-1349] [ubuntu 24.04 deb]
2026/05/07 14:33:27 POST-INSTALL: [abort-upgrade 1.41.0-1382] [ubuntu 24.04 deb]
2026/05/07 14:33:27 Errors were encountered while processing:
2026/05/07 14:33:27 /var/lib/waagent/Microsoft.Azure.Monitor.AzureMonitorLinuxAgent-1.41.0/packages/azuremonitoragent_1.41.0-1382_x86_64.deb
2026/05/07 14:33:27 ERROR:Install failed with exit code 1. For error details, check logs in /var/log/azure/Microsoft.Azure.Monitor.AzureMonitorLinuxAgent
2026/05/07 14:33:27 ERROR:[Microsoft.Azure.Monitor.AzureMonitorLinuxAgent-1.41.0] Install failed with exit code 1. For error details, check logs in /var/log/
azure/Microsoft.Azure.Monitor.AzureMonitorLinuxAgent
2026/05/07 14:33:27 [Microsoft.Azure.Monitor.AzureMonitorLinuxAgent-1.41.0] Install,failed,1,Install failed with exit code 1. For error details, check logs i
n /var/log/azure/Microsoft.Azure.Monitor.AzureMonitorLinuxAgent
Trying to install the latest extension version 1.41 manually as depicted in the above error which clearly indicates issue with pre-installation script execution
azureuser@agents000004:~$ sudo dpkg -i /var/lib/waagent/Microsoft.Azure.Monitor.AzureMonitorLinuxAgent-1.41.0/packages/azuremonitoragent_1.41.0-1382_x86_64.d
eb
(Reading database ... 133566 files and directories currently installed.)
Preparing to unpack .../azuremonitoragent_1.41.0-1382_x86_64.deb ...
PRE-RM: [upgrade 1.41.0-1382] [ubuntu 24.04]
PRE-INSTALL: [upgrade 1.40.3-1349] [ubuntu 24.04]
dpkg: error processing archive /var/lib/waagent/Microsoft.Azure.Monitor.AzureMonitorLinuxAgent-1.41.0/packages/azuremonitoragent_1.41.0-1382_x86_64.deb (--in
stall):
new azuremonitoragent package pre-installation script subprocess returned error exit status 1
POST-RM: [abort-upgrade 1.40.3-1349] [ubuntu 24.04 deb]
POST-INSTALL: [abort-upgrade 1.41.0-1382] [ubuntu 24.04 deb]
Errors were encountered while processing:
/var/lib/waagent/Microsoft.Azure.Monitor.AzureMonitorLinuxAgent-1.41.0/packages/azuremonitoragent_1.41.0-1382_x86_64.deb
Unpacking the deb package to inspect the contents of pre-inst script in /tmp folder
azureuser@agents000004:~$ cd /tmp
azureuser@agents000004:/tmp$ sudo dpkg-deb -R /var/lib/waagent/Microsoft.Azure.Monitor.AzureMonitorLinuxAgent-1.41.0/packages/azuremonitoragent_1.41.0-1382_x
86_64.deb ama-debug
azureuser@agents000004:/tmp$ cat ama-debug/DEBIAN/preinst
#!/bin/bash -e
# The original file is in EngSys-AzSec-LinuxExtensions
DISTRO=
VER=
SSL_CERT_DIR=
SSL_CERT_FILE=
function GetLnxBase {
case $DISTRO in
ubuntu | debian | raspbian | cbld | elementary)
lnxBase="deb"
;;
centos | redhat | fedoraproject | sles | opensuse | oracle | mariner | azurelinux | amazon | rocky | almalinux)
lnxBase="lsb"
;;
*)
lnxBase="unsupported"
;;
esac
echo $lnxBase
}
function SetSSL {
case $DISTRO in
ubuntu | debian | raspbian | cbld | elementary)
SSL_CERT_DIR="/etc/ssl/certs"
;;
centos | redhat | fedoraproject | oracle | mariner | azurelinux | amazon | rocky | almalinux)
SSL_CERT_FILE="/etc/pki/tls/certs/ca-bundle.crt"
;;
sles | opensuse)
if [[ $VER == 11* ]] ; then
SSL_CERT_DIR="/etc/ssl/certs"
SSL_CERT_FILE="/etc/ssl/certs/mdsd-ca-certs.pem"
elif [[ $VER == 12* ]] ; then
SSL_CERT_DIR="/var/lib/ca-certificates/openssl"
elif [[ $VER == 15* ]] ; then
SSL_CERT_DIR="/etc/ssl/certs"
fi
;;
esac
}
detect_installer()
{
INSTALLER=
# If DPKG lives here, assume we use that. Otherwise we use RPM.
check_if_program_in_path dpkg
if [ $? -eq 0 ]; then
INSTALLER=DPKG
else
check_if_program_in_path rpm
if [ $? -eq 0 ]; then
INSTALLER=RPM
else
#Exit with code 51 if system is not deb or rpm
echo "Error: This system does not have supported package manager"
echo "Supported Sytems: 'DPKG' & 'RPM'"
exit 1
fi
fi
}
# $1 - The name of the package to check as to whether it's installed
check_if_pkg_is_installed()
{
detect_installer
if [ "$INSTALLER" = "DPKG" ]; then
dpkg -s $1 2> /dev/null | grep Status | grep " installed" 1> /dev/null
else
rpm -q $1 > /dev/null 2>&1
fi
return $?
}
check_if_program_in_path()
{
# Parameter: name of program to check
# Returns: 0 if program is in path, non-zero if not
if [ $# -ne 1 ]; then
echo "INTERNAL ERROR: Incorrect number of parameters passed to check_if_program_in_path" >&2
return 1
fi
which $1 > /dev/null 2>&1
return $?
}
check_if_program_exists_on_system()
{
# Parameters: $1 - name of program to check
# Returns: 0 if program is in system or installed as a package, 1 if not
if [ $# -ne 1 ]; then
echo "INTERNAL ERROR: Incorrect number of parameters passed to check_if_program_exists_on_system" >&2
return 1
fi
local exists=1
check_if_pkg_is_installed $1
[ $? -eq 0 ] && exists=0
check_if_program_in_path $1
[ $? -eq 0 ] && exists=0
return $exists
}
# start/stop/restart service
# OpService name [start|stop|restart]
# returns the exit code for the given service command; to check result or ignore without
# the calling script exiting (if -e is set), append "|| true" or "|| ret=$?" to the call
function OpService {
servicename=$1
cmd=$2
if [ -z $servicename ]; then
echo "error in OpService: service name is not set"
return 1
fi
if [ -z $cmd ]; then
echo "error in OpService: command is not set"
return 1
fi
# Only run the following commands if this system is booted with systemd.
# For example, an Ubuntu-based container is not booted with systemd,
# so even though we detect systemd, running systemctl commands is unnecessary.
if [ -d /run/systemd/system ]; then
systemctl $cmd $servicename.service
exit_code=$?
if [ "$exit_code" -ne 0 ]; then
echo "Failed to $cmd $servicename"
return $exit_code
fi
fi
}
function OpServiceIfExist {
servicename=$1
cmd=$2
if [ -z $servicename ]; then
echo "error in OpServiceIfExist: service name is not set"
return 1
fi
if [ -f /opt/microsoft/azuremonitoragent/init/$servicename ]; then
OpService $servicename $cmd || true
return 0
fi
# when the service not found in init check it using package manager
set +e # this is important to prevent exiting when $? changed
check_if_program_exists_on_system $servicename
if [ $? -eq 0 ]; then
OpService $servicename $cmd
fi
set -e # reset it back
}
# determine distro and version
if [ -e /etc/system-release-cpe ]; then
cpe_version=`awk -F":" '{print $2}' /etc/system-release-cpe`
if [ $cpe_version = "/o" ]; then
DISTRO=`awk -F":" '{print $3}' /etc/system-release-cpe`
VER=`awk -F":" '{print $5}' /etc/system-release-cpe`
else
DISTRO=`awk -F":" '{print $4}' /etc/system-release-cpe`
VER=`awk -F":" '{print $6}' /etc/system-release-cpe`
fi
elif [ -e /etc/os-release ]; then
DISTRO=`grep "^ID=" /etc/os-release | awk -F"=" '{print $2}' | sed 's/"//g' | awk -F"-" '{print $1}'`
VER=`grep "^VERSION_ID=" /etc/os-release | awk -F"=" '{print $2}' | sed 's/"//g'`
fi
LNXBASE=$(GetLnxBase)
SetSSL
echo "PRE-INSTALL: [$1 $2] [$DISTRO $VER]"
MdsdBin=/opt/microsoft/azuremonitoragent/bin/mdsd
MdsdDaemon=azuremonitoragent
AMACADaemon=azuremonitor-coreagent
AMAAstExtDaemon=azuremonitor-astextension
LauncherDaemon=azuremonitor-agentlauncher
MdsdLock=/run/lock/${MdsdDaemon}.run
MdsdMgrBin=/opt/microsoft/azuremonitoragent/bin/mdsdmgr
MdsdMgrDaemon=azuremonitoragentmgr
MdsdMgrLock=/run/lock/${MdsdMgrDaemon}.run
case $1 in
install | 1)
;;
upgrade | 2)
ret=0
OpService ${MdsdDaemon} status > /dev/null || OpService ${MdsdDaemon} is-enabled > /dev/null || ret=$?
if [ $ret = 0 ]; then
# This lock file will be used by postinst to start the daemon.
touch ${MdsdLock}
OpService ${MdsdDaemon} stop > /dev/null
OpService ${AMACADaemon} stop > /dev/null
OpService ${LauncherDaemon} stop > /dev/null
OpService ${AMAKqlExtDaemon} stop > /dev/null
fi
ret=0
OpService ${MdsdMgrDaemon} status > /dev/null || OpService ${MdsdMgrDaemon} is-enabled > /dev/null || ret=$?
if [ $ret = 0 ]; then
# This lock file will be used by postinst to start the daemon.
touch ${MdsdMgrLock}
OpService ${MdsdMgrDaemon} stop > /dev/null
fi
esac
if [ "$1" = "install" -a -n "$2" ] ; then
[ -f /etc/logrotate.d/azuremonitoragent.disabled ] && mv -f /etc/logrotate.d/azuremonitoragent.disabled /etc/logrotate.d/azuremonitoragent
fi
exit 0
Now manually executing the pre-inst script in debug mode to upgrade from 1.40 to 1.41 to identify the error
azureuser@agents000004:/tmp$ sudo bash -x ama-debug/DEBIAN/preinst upgrade 1.40.3-134
+ DISTRO=
+ VER=
+ SSL_CERT_DIR=
+ SSL_CERT_FILE=
+ '[' -e /etc/system-release-cpe ']'
+ '[' -e /etc/os-release ']'
++ grep '^ID=' /etc/os-release
++ awk -F= '{print $2}'
++ sed 's/"//g'
++ awk -F- '{print $1}'
+ DISTRO=ubuntu
++ awk -F= '{print $2}'
++ grep '^VERSION_ID=' /etc/os-release
++ sed 's/"//g'
+ VER=24.04
++ GetLnxBase
++ case $DISTRO in
++ lnxBase=deb
++ echo deb
+ LNXBASE=deb
+ SetSSL
+ case $DISTRO in
+ SSL_CERT_DIR=/etc/ssl/certs
+ echo 'PRE-INSTALL: [upgrade 1.40.3-134] [ubuntu 24.04]'
PRE-INSTALL: [upgrade 1.40.3-134] [ubuntu 24.04]
+ MdsdBin=/opt/microsoft/azuremonitoragent/bin/mdsd
+ MdsdDaemon=azuremonitoragent
+ AMACADaemon=azuremonitor-coreagent
+ AMAAstExtDaemon=azuremonitor-astextension
+ LauncherDaemon=azuremonitor-agentlauncher
+ MdsdLock=/run/lock/azuremonitoragent.run
+ MdsdMgrBin=/opt/microsoft/azuremonitoragent/bin/mdsdmgr
+ MdsdMgrDaemon=azuremonitoragentmgr
+ MdsdMgrLock=/run/lock/azuremonitoragentmgr.run
+ case $1 in
+ ret=0
+ OpService azuremonitoragent status
+ servicename=azuremonitoragent
+ cmd=status
+ '[' -z azuremonitoragent ']'
+ '[' -z status ']'
+ '[' -d /run/systemd/system ']'
+ systemctl status azuremonitoragent.service
+ exit_code=3
+ '[' 3 -ne 0 ']'
+ echo 'Failed to status azuremonitoragent'
+ return 3
+ OpService azuremonitoragent is-enabled
+ servicename=azuremonitoragent
+ cmd=is-enabled
+ '[' -z azuremonitoragent ']'
+ '[' -z is-enabled ']'
+ '[' -d /run/systemd/system ']'
+ systemctl is-enabled azuremonitoragent.service
+ exit_code=0
+ '[' 0 -ne 0 ']'
+ '[' 0 = 0 ']'
+ touch /run/lock/azuremonitoragent.run
+ OpService azuremonitoragent stop
+ servicename=azuremonitoragent
+ cmd=stop
+ '[' -z azuremonitoragent ']'
+ '[' -z stop ']'
+ '[' -d /run/systemd/system ']'
+ systemctl stop azuremonitoragent.service
+ exit_code=0
+ '[' 0 -ne 0 ']'
+ OpService azuremonitor-coreagent stop
+ servicename=azuremonitor-coreagent
+ cmd=stop
+ '[' -z azuremonitor-coreagent ']'
+ '[' -z stop ']'
+ '[' -d /run/systemd/system ']'
+ systemctl stop azuremonitor-coreagent.service
+ exit_code=0
+ '[' 0 -ne 0 ']'
+ OpService azuremonitor-agentlauncher stop
+ servicename=azuremonitor-agentlauncher
+ cmd=stop
+ '[' -z azuremonitor-agentlauncher ']'
+ '[' -z stop ']'
+ '[' -d /run/systemd/system ']'
+ systemctl stop azuremonitor-agentlauncher.service
+ exit_code=0
+ '[' 0 -ne 0 ']'
+ OpService stop
+ servicename=stop
+ cmd=
+ '[' -z stop ']'
+ '[' -z ']'
+ echo 'error in OpService: command is not set'
+ return 1
+ ret=0
+ OpService azuremonitoragentmgr status
+ servicename=azuremonitoragentmgr
+ cmd=status
+ '[' -z azuremonitoragentmgr ']'
+ '[' -z status ']'
+ '[' -d /run/systemd/system ']'
+ systemctl status azuremonitoragentmgr.service
+ exit_code=3
+ '[' 3 -ne 0 ']'
+ echo 'Failed to status azuremonitoragentmgr'
+ return 3
+ OpService azuremonitoragentmgr is-enabled
+ servicename=azuremonitoragentmgr
+ cmd=is-enabled
+ '[' -z azuremonitoragentmgr ']'
+ '[' -z is-enabled ']'
+ '[' -d /run/systemd/system ']'
+ systemctl is-enabled azuremonitoragentmgr.service
+ exit_code=1
+ '[' 1 -ne 0 ']'
+ echo 'Failed to is-enabled azuremonitoragentmgr'
+ return 1
+ ret=1
+ '[' 1 = 0 ']'
+ '[' upgrade = install -a -n 1.40.3-134 ']'
+ exit 0
Follow this part from above
+ OpService stop
+ servicename=stop
+ cmd=
+ '[' -z stop ']'
+ '[' -z ']'
+ echo 'error in OpService: command is not set'
+ return 1
OpService expects two arguments:
OpService
But the script incorrectly calls it like this:
OpService stop
So:
servicename = stop
cmd = "" ← missing
Function explicitly errors out:
error in OpService: command is not set
That return 1 terminates the pre-install script, which causes:
new azuremonitoragent package pre-installation script subprocess returned error exit status 1
Usecase
We have Azure VMSS (build agents) with U24 image and installing Azure Monitor Linux Agent extension in it.
We have noticed that the agents remain unhealthy affecting the pipeline to wait for couple of hours to start.
Investigation
While checking logs after loging into a VM, it seems pre-install script is failing
Trying to install the latest extension version 1.41 manually as depicted in the above error which clearly indicates issue with pre-installation script execution
Unpacking the deb package to inspect the contents of pre-inst script in /tmp folder
Now manually executing the pre-inst script in debug mode to upgrade from 1.40 to 1.41 to identify the error
Follow this part from above
OpService expects two arguments:
OpService
But the script incorrectly calls it like this:
OpService stop
So:
servicename = stop
cmd = "" ← missing
Function explicitly errors out:
error in OpService: command is not set
That return 1 terminates the pre-install script, which causes:
new azuremonitoragent package pre-installation script subprocess returned error exit status 1