Recent changes in the Azure DevOps UI have made it somewhat difficult / not very obvious to refresh secrets on the Azure ARM service connections, here are a couple of ways you can work around this.
Azure DevOps utilizes a feature called Service Connections to allow build agents to authenticate to different services. Previously, when talking about the Azure Resource Manager service connection, you had two "main" ways of creating a Service Principal backed connection: Automatic and Manual.
The difference between these two ways was basically the ability to specify all the details yourself and even skip validation, or allow Azure DevOps to generate a project-generic service principal in Azure Active Directory scoped to a Resource Group of your selection. This automatic process utilized your personal authentication and authorization to determine if you could use it or not.
While the automatic way sounds like a great way to get going, it's' not very obvious that the management of the service principal is still on you. Meaning that there is a 2 year secret configured somewhere in Azure DevOps that you need to update when it expires.
Problem with renewal
Previously, this was a pretty easy thing to fix. Generate a new secret for the service principal, go to the Azure DevOps Edit UI for the service connection and plop that thing right in.
However, now due to "the new service connection experience" UI no longer being an opt-out feature, you can seemingly no longer edit any of the relevant data for the service connection if you created it in the automatic mode. Apparently this is by design.
The frustrating thing is that the UI still exists when you are creating a new Service Connection with the manual mode, but there's just no way of accessing it when editing existing ones.
So how do we fix this?
So I was running into this issue for a client where
- We no longer had any RBAC permissions to Azure
- We were not the initial creator of the service connection
- We still had Project Admin permissions in Azure DevOps
First thing I did was find this thread with lengthy discussion on the subject where people had actually found a couple of workarounds.
Option 1: UI based update
Even though its not at all obvious, it turns out that to fix this you can just:
- Go to the edit view of a service connection
- Not change any fields
- Click Save
Does it work? Yes, sort of...
What seems to happen on the background when you do this is a fairly similar flow of actions as with creating a totally new Service Connection with the automatic flow, a new 2 year secret included (though no new principal is created). This basically means that you need a lot of permissions to do this. Let's go through them.
(Note that our secrets had already expired, so verify that your secrets are renewed if the previous one was still valid.)
First of all, to generate the secret, you require ownership of the Service Principal & App Registration.
Second, the flow requires the user to have authorization to manage permissions of the scoped Resource Group. In practice this means either Owner or User Access Administrator-permissions. As you can imagine, getting these is a huge hassle and should be removed directly after updating the secret if you go this route.
Third, you should have (Azure DevOps) Organizational level permissions to the service connection. Either this is a fairly new feature, or I just so often had these permissions myself by default to not notice this requirement, but apparently even for single project Service Connections, organizational permission is required to change the password. This permission is given by default to the person that created the connection in the first place.
And lastly, although this might be optional, some permission that allows you to see the Service Connection object in the Azure DevOps project to actually edit it. It might be enough to limit this to a reader permission for the project and project level permissions for the Service Connection. I did not test this part thoroughly.
This was the hassle we went through to do these updates in a small scale, and I don't necessarily think you can easily scale this method to a larger amount of connections. It would be quite a bit easier if you could just get the new secret value and add it to the existing Service Connection. Maybe there's a way to do just that...
Option 2: Update using Azure DevOps APIs
While I did not ultimately test this solution, I think it might even be the better way of approaching the problem, especially if you need to do this at scale. You (or someone else) most likely still require largely the same permissions as with option 1, but can probably skip any owner permissions to the resource groups the service connection has access to. This is a pretty big distinction, as getting any owner-level permissions is often behind a large amount of bureaucracy (understandably). Again, in an ideal world you could just get a secret delivered to you (the DevOps Admin) and pop it in via the API without any extra permissions required. That is unfortunately still not the case.
This reply goes through an example of how to handle the update. In a nutshell:
- Generate a PAT token for yourself to be able to call the APIs via POSTMAN or some other tool capable of sending HTTP requests
- Get a new secret for the Azure Service Principal
- Get the ID of the service connection utilizing the Get Endpoint API
- Set the new password using the Update Endpoint API (There's also the Update Endpoints API that takes in an array in a single call)
See the example documentation and the reply linked above for an example request body. Seemingly the user still had to edit the service connection in the portal though, so your mileage may wary.
One API you could also explore is the "Get Service Endpoints with Refreshed Authentication" API.