Azure Functions for your long-lasting logic

-

Recently I started to study for the renewed Azure AZ-204 exam. My biggest pitfall while studying is not having a clear goal. Therefore, I eventually stop studying. (Please tell me that I’m not the only one…)

This blog is a try-out for my new learning routine. By writing blogs, I hope to cover all the topics of the exam and pass it. If this comes to a series of blogs, I probably won’t cover all topics of the exam. I’ll just write about the interesting topics I come across while studying.

For those interested in the exam visit the exam page. I’m currently learning by using the free learning paths from Microsoft. Additionally, I’ve watched an Udemy course for the AZ-203 exam (the predecessor of the AZ-204 exam). Finally, I was lucky enough to learn about a big chunk of the subjects in a real project at my employer.

Azure Durable Functions

Let’s start with the first subject: Azure Durable Functions. Most of you probably already know about Azure Functions, the serverless implementation of Microsoft. You may know other implementations of serverless by names as: Google Cloud Functions and AWS Lambda.

An Azure Function (and serverless in general) is stateless by design. So, when your function execution ends, your state is gone. In most cases this is not a problem at all. Nevertheless, there are scenarios where you need to maintain some state. Let’s say I’m making an application for generating thumbnails. I upload a dozen of images that need to be processed. After that, the thumbnails are generated and I need to upload them to some storage (hint: blob storage, part of the AZ-204 exam 😉).

You could make this in a single Azure Function, but you’re limited to the 5-minute maximum duration of the runtime. Sure, this is extendable up to 10 minutes but still not a desirable solution. You should create a function that receives a single picture and transforms it into a thumbnail. Another (parent / orchestrating) function should distribute the work, in case you want to create multiple thumbnails. This is where Azure Durable Functions comes into play.

A Durable function consists of three separate functions:

  • A client function, which is responsible for receiving your initial ‘start’ event. For example, the start event may be a HTTP request. It also starts the orchestrator function.
  • An orchestrator function, describes your actual long-lasting logic. It delegates work to more specific activity functions.
  • An activity function, which executes the actual work; making a thumbnail.

We’ll walk through some code examples to illustrate how these functions work together.

The thumbnail generator

We’re going to create a durable function that takes a batch of images and creates thumbnails of these images.

The client function will be a function that can be triggered with a HTTP-call. The user must provide an URL or connection string, to a blob storage container, along with this HTTP-call. I’m hosting the pictures in a blob container for this example. You’re free to use any other service to host your pictures, for example an FTP server.

The orchestrator function takes the blob storage URL, and starts a separate activity to retrieve URL’s for all images in the blob container. Once the first activity is finished, a new thumbnail activity for each image will be started. They run parallel of each other.

The thumbnail activity receives an URL to one specific picture. I’ll use Azure Cogntive Services for generating thumbnails. Azure Cognitive Services is an API service, offered by Microsoft, with different artificial intelligence functionalities. One of the APIs they offer can be used to generate thumbnails in a smart way. The API determines the point of interest in the picture using artificial intelligence, and resizes the image to a specified width and height.

The client function

For the first function, the client function, we’ll navigate to Visual Studio. Create a new Function App Project like you would normally do when making Azure functions. We’ll make our four functions within this function app. For now, choose to create an empty project.

Note: for the exam you should know your way around the Azure Portal. You could make these functions via the Azure Portal. Initially this was my plan for this blog. I really do not like development in Azure Portal though. It takes too much time due to the absence of intellisense.

Create a new Azure Function Application

Once created, I’ll add a function to the function app. I’ll do this using the context menu of the project file in my solution explorer. When you click on this, you’ll be prompted with a pop up. Yet again for choosing a template for your function. But now you’re able to choose for a ‘Durable Function Orchestration’. This template will create the base of our project. The created file contains three functions:

  • A HTTP Starter function
  • An Orchestrator function
  • An Activity function

 

Add a new functionDurable Functions Orchestration Templat

Let’s get started creating the HTTP Starter function. My final implementation of the function looks something this:

[FunctionName("Function2_HttpStart")]
public static async Task HttpStart(
    [HttpTrigger(AuthorizationLevel.Anonymous, "post")] HttpRequestMessage req,
    [DurableClient] IDurableOrchestrationClient starter,
    ILogger log)
{
    const string orchestratorFunctionName = "DurableFunctionsOrchestrator";

    // Function input comes from the request content.
    var requestBody = await req.Content.ReadAsAsync();
    var instanceId = Guid.NewGuid()
    
    await starter.StartNewAsync(orchestratorFunctionName, instanceId.ToString(), requestBody.BlobConnectionString)
    
    log.LogInformation($"Received blob url: '{requestBody.BlobConnectionString}'.");
    log.LogInformation($"Started orchestration with ID = '{instanceId}'.")
    
    return starter.CreateCheckStatusResponse(req, instanceId.ToString());
}

The orchestrator function

The orchestrator function is going to call two different activity functions. But first, I retrieve the connection string for the Azure Blob Storage from the request body. You can start new activities via the injected IDurableOrchestrationContext.

[FunctionName("DurableFunctionsOrchestrator")]
public static async Task RunOrchestrator(
    [OrchestrationTrigger] IDurableOrchestrationContext context,
    ILogger log)
{
    string connectionString = context.GetInput();

    log.LogInformation($"Start durable orchestrator.");
    var blobUrls = await context.CallActivityAsync("GetAllBlobUrlsActivity", connectionString);

    var thumbnailActivities = new List();

    foreach (var url in blobUrls)
    {
        var activity = context.CallActivityAsync("GenerateThumbnailActivity", (url, connectionString));
        thumbnailActivities.Add(activity);
    }

    var resultUrls = (await Task.WhenAll(thumbnailActivities)).ToList();

    log.LogInformation($"Finish durable orchestrator.");

    return resultUrls;
}

The image URL retrieval activitiy

The first activity is just a normal Azure Function, triggered by an ActivityTrigger. This activity reads the original images from the blob storage. For each image an absolute URI is added to a list, which is eventually returned to the orchestrator.

[FunctionName("GetAllBlobUrlsActivity")]
public static async Task GetAllBlobUrlsActivityAsync([ActivityTrigger] string blobConnectionString, ILogger log)
{
    log.LogInformation($"Start retrieving images from container");
    
    var blobContainer = GetImageBlobContainer(blobConnectionString);
    var blobs = await blobContainer.ListBlobsSegmentedAsync("original", true, BlobListingDetails.All, null, null, null, null);
    
    log.LogInformation($"Received {blobs.Results.Count()} images");
    
    return blobs.Results
        .Select(blob => blob as CloudBlockBlob)
        .Where(blob => blob != null)
        .Select(blob => blob.Uri.AbsoluteUri);
}

private static CloudBlobContainer GetImageBlobContainer(string blobConnectionString)
{
    var storageAccount = CloudStorageAccount.Parse(blobConnectionString);
    var client = storageAccount.CreateCloudBlobClient();
    return client.GetContainerReference("pictures");
}

The thumbnail generating activity

We’re going to create a second activity within our file. This is also a pretty standard Azure Function. The Azure Cognitive Service API accepts a URL for generating a thumbnail. So, no need to download the images first. The API needs some information along with the request:

  • The thumbnail dimensions
  • API key
  • Smart Cropping attribute

[FunctionName("GenerateThumbnailActivity")]
public static async Task GenerateThumbnailActivityAsync([ActivityTrigger] IDurableActivityContext context, ILogger log)
{
    var imageWidth = 400;
    var imageHeight = 200;

    var apiUrl = $"https://[your-service-name].cognitiveservices.azure.com/vision/v1.0/generateThumbnail?width={imageWidth}&height={imageHeight}&smartCropping=true";
    var apiKey = "Fill-In-Your-API-Key";

    var (imageUrl, blobConnectionString) = context.GetInput();
    var requestBody = JsonConvert.SerializeObject(new { url = imageUrl });

    using (var client = new HttpClient())
    {
        client.DefaultRequestHeaders.Add("Ocp-Apim-Subscription-Key", apiKey);
        var thumbnailResponse = await client.PostAsync(apiUrl, new StringContent(requestBody, Encoding.UTF8, "application/json"));
        var imageName = imageUrl.Split('/').Last();
        
        if (thumbnailResponse.IsSuccessStatusCode)
        {
            var imageStream = await thumbnailResponse.Content.ReadAsStreamAsync();
            var blobContainer = GetImageBlobContainer(blobConnectionString);
            var blob = blobContainer.GetBlockBlobReference($"thumnail/thumbnail-{imageName}");
            blob.Properties.ContentType = "image/jpeg";
            await blob.UploadFromStreamAsync(imageStream);

            return blob.Uri.AbsoluteUri;
        }
        else
        {
            return $"{imageName} Failed";
        }
    }
}

private static CloudBlobContainer GetImageBlobContainer(string blobConnectionString)
{
    var storageAccount = CloudStorageAccount.Parse(blobConnectionString);
    var client = storageAccount.CreateCloudBlobClient();
    return client.GetContainerReference("pictures");
}

Takeaways

That is how you create your first durable function. When creating the thumbnail generator, I first tried to retrieve the images via a blob input binding. This didn’t work because you can’t do any I/O from the orchestrator function. Therefore, I made a separate activity to retrieve all images. Creating a separate activity for this was a bit more cumbersome, but it offered a cleaner software architecture. Orchestration functions are after all only for orchestrating tasks, and not for retrieving input by themselves.

This blog provides a basic understanding of Azure Durable Functions. If you do want to study for one of the Azure exams make sure you read the full documentation.