Creating Async Custom Automated Release Gates in VSTS with REST API

One of the most exciting features we announced this year at Connect(); was automated release gates for VSTS releases. We’ve always had the ability to add manual approvers before and after every deployment environment. And now, we have the ability to also add automated gates using continuous monitoring. Out of the box, there are 4 kinds of gates.

Azure Monitor, Invoke Azure Function, Invoke REST API and Query Work Items. Azure Monitor uses Azure Monitoring to determine if a gate should pass or fail. The Work Item Query gate uses a work item query to determine if a gate should pass or fail. And the Invoke Azure Function and Invoke REST API are extensibility points for gates. These are both ways to write your own custom gates that do pretty much whatever you want. So how are these gates used in the real world? Here are three ways I’ve recently used gates.

I added a post deployment Azure Monitor gate where after deploying my app (App Service) to a test environment, I set up Azure Monitoring for 24 hours, looking for alerts. During that 24 hours, I ran all sorts of stress tests, load tests and automated UI tests. And if there were too many alerts during this 24 hour timeframe, the gate would fail, automatically stopping the release from moving on.

Another way I’ve used gates was with a work item query gate which blocks if any critical bugs were found. This way, if anybody flagged a critical bug against my system, the release would automatically be stopped.

And finally, I recently had to write my own custom gate. The project I was working on had a weird requirement where things were not allowed to be pushed into production until physical documents were signed and submitted to their document repository solution. There already existed a REST api that checked if the documents were signed so I figured, hey, this should be a piece of cake writing a custom gate using the Invoke REST API gate! Below is how I wrote my custom gate. WARNING: This feature is still in preview which means the api is still fluid and things can and probably will change. So this example works as of 12/25/2017 but be warned, things probably will change.

Now the way the REST API gate works is you create a generic endpoint that points to your REST API. If you haven’t created one in VSTS, it’s easy enough. Just click the Manage link next to the Generic Endpoint text field and that takes you to the Services endpoint page.

From here, Click New Service Endpoint, select Generic

And then give your generic connection a name, the URL and username/password/token combo if needed.

Now, go back to your release gate and select the generic connection you just made. Give your gate a name, select the method, you can tweak headers and also the body.

Notice how the Wait For Completion checkbox is not checked? That means VSTS is looking at the response code of the REST API call to determine if the gate passes or fails.However, I couldn’t use this because the REST API I was calling takes about a minute before it finishes and by that time, the call would have timed out. Selecting Wait For Completion means VSTS is expecting the rest API to be called asynchronously and when finished, the API will call back to VSTS firing the TaskCompleted event. That’s what I needed!

So I checked the checkbox

I also passed in the VSTS values needed to make the callback. The values are JobId, PlanId, TimeLineId, ProjectId, VstsURL, and Authentication token. All these values can be retrieved using VSTS variables. I also passed in the values for one and two. These are just sample variables of vsts passing values to my gate.

{    
    "JobId": "$(system.jobId)",
    "PlanId": "$(system.planId)",
    "TimelineId": "$(system.timelineId)",
    "ProjectId": "$(system.teamProjectId)",
    "VstsUrl": "$(system.CollectionUri)",
    "AuthToken": "$(system.AccessToken)",
    "one": "abc",
    "two": "def" 
}

{

"JobId": "$(system.jobId)",

"PlanId": "$(system.planId)",

"TimelineId": "$(system.timelineId)",

"ProjectId": "$(system.teamProjectId)",

"VstsUrl": "$(system.CollectionUri)",

"AuthToken": "$(system.AccessToken)",

"one": "abc",

"two": "def"

}

In my REST API, I grab these values from the body, asynchronously do whatever processing I need to do and when done, use these values to make a callback to VSTS.

I used asp.net 4.6 to write my REST API. You can use whatever language you want, that just happened to be what the original REST API was written in so that’s what I used. First, I needed to add the nuget packages my VSTS callback needs. I added

Microsoft.TeamFoundation.DistriutedTask.WebApi

Next, I wrote my REST API

        [HttpPost]
        public object PostValues(string jobId,
            string planId,
            string timelineId,
            string projectId,
            string vstsUrl,
            string authToken,
            string one,
            string two)
        {
            // launch thread that does the fake work
            var executionObj = new ExecuteObject(jobId, planId, timelineId, projectId, vstsUrl, authToken);
            var executionThread = new Thread(new ThreadStart(executionObj.Execute));
            executionThread.Start();
            // return 200 spitting backout what was passed in
            Response.ContentType = "application/json";
            var returnObj = new
            {
                JobId = jobId,
                PlanId = planId,
                TimelineId = timelineId,
                ProjecId = projectId,
                VstsUrl = vstsUrl,
                AuthToken = authToken,
                One = one,
                Two = two
            };
            var json = JsonConvert.SerializeObject(returnObj);
            return json;
        }
    }

[HttpPost]

public object PostValues(string jobId,

string planId,

string timelineId,

string projectId,

string vstsUrl,

string authToken,

string one,

string two)

{

// launch thread that does the fake work

var executionObj = new ExecuteObject(jobId, planId, timelineId, projectId, vstsUrl, authToken);

var executionThread = new Thread(new ThreadStart(executionObj.Execute));

executionThread.Start();

// return 200 spitting backout what was passed in

Response.ContentType = "application/json";

var returnObj = new

{

JobId = jobId,

PlanId = planId,

TimelineId = timelineId,

ProjecId = projectId,

VstsUrl = vstsUrl,

AuthToken = authToken,

One = one,

Two = two

};

var json = JsonConvert.SerializeObject(returnObj);

return json;

}

Here, I create an ExecuteObject based off of all the values passed from VSTS, spawn a new thread to asynchronously do the work in the ExecuteObject and then immediately return from the call. And in my execute object, I pretend to do a slow running task, and when finished, I do my VSTS call back. Here is my ExecuteObject code:

    class ExecuteObject
    {
        public readonly static string HUBNAME = "Gates";
        //public readonly static string HUBNAME = "release";
        public string JobId { get; set; }
        public string PlanId { get; set; }
        public string TimelineId { get; set; }
        public string ProjectId { get; set; }
        public string VstsUrl { get; set; }
        public string AuthToken { get; set; }
        public ExecuteObject(string jobId,
            string planId,
            string timelineId,
            string projectId,
            string vstsUrl,
            string authToken)
        {
            this.JobId = jobId;
            this.PlanId = planId;
            this.TimelineId = timelineId;
            this.ProjectId = projectId;
            this.VstsUrl = vstsUrl;
            this.AuthToken = authToken;
        }
        public void Execute()
        {
            // create guids out of strings to be used throughout this method
            var projectGuid = new Guid(this.ProjectId);
            var planGuid = new Guid(this.PlanId);
            var jobGuid = new Guid(this.JobId);
            var timelineGuid = new Guid(this.TimelineId);
            // Today we allow only 1 task to run in a job, so sending jobId works but tomorrow it will be taskId 
            // once we start supporting multiple tasks.
            var taskInstanceGuid = new Guid(this.JobId);
            // create connection to VSTS
            var connection = new VssConnection(new Uri(this.VstsUrl), new VssBasicCredential("username", this.AuthToken));
            // get task client
            var taskClient = connection.GetClient<TaskHttpClient>();
            // get the plan
            var plan = taskClient.GetPlanAsync(projectGuid, HUBNAME, planGuid).SyncResult();
            // declare a bunch of variables used in my loop
            var offlineMessageBuilder = new StringBuilder();
            List<string> liveFeedList;
            VssJsonCollectionWrapper<IEnumerable<string>> liveFeedWrapper;
            var message = "doing fake work: ";
            var completeMessage = string.Empty;
            // loop through this 5 times sleeping 2 secods between each loop logging simulate actually doing
            // something and then logging to the live feed about the task progress in VSTS
            for (int i = 0; i < 5; i++)
            {
                // building out message to send to the live feed
                completeMessage = message + i + "\n";
                offlineMessageBuilder.Append(completeMessage);
                liveFeedList = new List<string> { completeMessage };
                liveFeedWrapper = new VssJsonCollectionWrapper<IEnumerable<string>>(liveFeedList);
                // sending message to live feed
                taskClient.AppendTimelineRecordFeedAsync(projectGuid, HUBNAME, planGuid, plan.Timeline.Id, jobGuid, liveFeedWrapper);
                // sleep for 2 seconds simulating a long running work
                Thread.Sleep(2000);
            }
            // finished with long running work, will now send offline logs and then send task complete event back to vsts
            var timeLineRecords = taskClient.GetRecordsAsync(projectGuid, HUBNAME, planGuid, timelineGuid).SyncResult();
            var httpTaskTimeLineRecord = timeLineRecords.Where(record => record.ParentId != null)
                .FirstOrDefault();
            // Send the offline logs.
            var logPath = string.Format(CultureInfo.InvariantCulture, "logs\\{0:D}", httpTaskTimeLineRecord.Id);
            var tasklog = new TaskLog(logPath);
            var log = taskClient.CreateLogAsync(projectGuid, HUBNAME, planGuid, tasklog).SyncResult();
            using (var ms = new MemoryStream())
            {
                var allBytes = Encoding.UTF8.GetBytes(offlineMessageBuilder.ToString());
                ms.Write(allBytes, 0, allBytes.Length);
                ms.Position = 0;
                taskClient.AppendLogContentAsync(projectGuid, HUBNAME, planGuid, log.Id, ms).SyncResult();
            }
            // Send task completion event
            //var taskCompletedEvent = new TaskCompletedEvent(jobGuid, taskInstanceGuid, TaskResult.Succeeded);
            var jobId = HUBNAME.Equals("Gates", StringComparison.OrdinalIgnoreCase)
                ? httpTaskTimeLineRecord.Id
                : jobGuid;
            var taskCompletedEvent = new TaskCompletedEvent(jobId, taskInstanceGuid, TaskResult.Succeeded);
            taskClient.RaisePlanEventAsync(projectGuid, HUBNAME, planGuid, taskCompletedEvent).SyncResult();
        }
    }

100

class ExecuteObject

{

public readonly static string HUBNAME = "Gates";

//public readonly static string HUBNAME = "release";

public string JobId { get; set; }

public string PlanId { get; set; }

public string TimelineId { get; set; }

public string ProjectId { get; set; }

public string VstsUrl { get; set; }

public string AuthToken { get; set; }

public ExecuteObject(string jobId,

string planId,

string timelineId,

string projectId,

string vstsUrl,

string authToken)

{

this.JobId = jobId;

this.PlanId = planId;

this.TimelineId = timelineId;

this.ProjectId = projectId;

this.VstsUrl = vstsUrl;

this.AuthToken = authToken;

}

public void Execute()

{

// create guids out of strings to be used throughout this method

var projectGuid = new Guid(this.ProjectId);

var planGuid = new Guid(this.PlanId);

var jobGuid = new Guid(this.JobId);

var timelineGuid = new Guid(this.TimelineId);

// Today we allow only 1 task to run in a job, so sending jobId works but tomorrow it will be taskId

// once we start supporting multiple tasks.

var taskInstanceGuid = new Guid(this.JobId);

// create connection to VSTS

var connection = new VssConnection(new Uri(this.VstsUrl), new VssBasicCredential("username", this.AuthToken));

// get task client

var taskClient = connection.GetClient<TaskHttpClient>();

// get the plan

var plan = taskClient.GetPlanAsync(projectGuid, HUBNAME, planGuid).SyncResult();

// declare a bunch of variables used in my loop

var offlineMessageBuilder = new StringBuilder();

List<string> liveFeedList;

VssJsonCollectionWrapper<IEnumerable<string>> liveFeedWrapper;

var message = "doing fake work: ";

var completeMessage = string.Empty;

// loop through this 5 times sleeping 2 secods between each loop logging simulate actually doing

// something and then logging to the live feed about the task progress in VSTS

for (int i = 0; i < 5; i++)

{

// building out message to send to the live feed

completeMessage = message + i + "\n";

offlineMessageBuilder.Append(completeMessage);

liveFeedList = new List<string> { completeMessage };

liveFeedWrapper = new VssJsonCollectionWrapper<IEnumerable<string>>(liveFeedList);

// sending message to live feed

taskClient.AppendTimelineRecordFeedAsync(projectGuid, HUBNAME, planGuid, plan.Timeline.Id, jobGuid, liveFeedWrapper);

// sleep for 2 seconds simulating a long running work

Thread.Sleep(2000);

}

// finished with long running work, will now send offline logs and then send task complete event back to vsts

var timeLineRecords = taskClient.GetRecordsAsync(projectGuid, HUBNAME, planGuid, timelineGuid).SyncResult();

var httpTaskTimeLineRecord = timeLineRecords.Where(record => record.ParentId != null)

.FirstOrDefault();

// Send the offline logs.

var logPath = string.Format(CultureInfo.InvariantCulture, "logs\\{0:D}", httpTaskTimeLineRecord.Id);

var tasklog = new TaskLog(logPath);

var log = taskClient.CreateLogAsync(projectGuid, HUBNAME, planGuid, tasklog).SyncResult();

using (var ms = new MemoryStream())

{

var allBytes = Encoding.UTF8.GetBytes(offlineMessageBuilder.ToString());

ms.Write(allBytes, 0, allBytes.Length);

ms.Position = 0;

taskClient.AppendLogContentAsync(projectGuid, HUBNAME, planGuid, log.Id, ms).SyncResult();

}

// Send task completion event

//var taskCompletedEvent = new TaskCompletedEvent(jobGuid, taskInstanceGuid, TaskResult.Succeeded);

var jobId = HUBNAME.Equals("Gates", StringComparison.OrdinalIgnoreCase)

? httpTaskTimeLineRecord.Id

: jobGuid;

var taskCompletedEvent = new TaskCompletedEvent(jobId, taskInstanceGuid, TaskResult.Succeeded);

taskClient.RaisePlanEventAsync(projectGuid, HUBNAME, planGuid, taskCompletedEvent).SyncResult();

}

There’s a couple of interesting things in this code. First, at this moment, the taskInstanceId is the same as the JobId. Currently, only one task can be run during a job. Soon, this will change where multiple tasks can be run. When this happens, make sure you pass in the right task instance id. Next, there is that value of HUBNAME currently set to “Gates”. It’s set to “Gates” because i’m calling the rest api from a release gate. If I called the REST api from an angentless task, the HUBNAME would be “release”. And Finally, right before I create my taskCompletedEvent object, I do some wonkery to get my correct jobId. In theory, my jobId should be whatever VSTS passes to me. There’s some weirdness going on that at this moment, if i’m coming from the release gates, VSTS is expecting the timeline record id (this will change and be fixed at some point).

So now, when I run my gate, it calls my rest api, passing in all the values needed through the body. My REST API asynchronously does it’s task, and when finished, calls the VSTS callback with the TaskCompletedEvent passing the gate.

And looking at the log

Leave a Reply Cancel reply