Azure Bits #4 – Adding the Azure Web Job

Back to all blogs

This post’s main objective was originally about completing the initial skeleton of uploading an image from a web page and generating a thumbnail from an Azure Web Job using Azure Blob Storage and Azure Queues, but it turned into a pretty large refactoring in anticipation of having something a bit more realistic to eventually post to GitHub.  So, I’ll devote the first part of the post to a brief review of the most significant changes and then introduce the Azure Web Job into the mix and finally, I’ll retrieve the message from the Azure Queue and show its name in the console.  I’ll then devote Azure Bit #5 to processing the original image and generating a thumbnail to complete the initial skeleton of our Image Manipulator application.

Changing the Serialization Strategy

I’m not sure why I originally chose to go with serializing my UploadedImage to a ByteArray before inserting it into the Azure Queue, but I’ve now simplified things a bit and switched over to serializing my UploadedImage as JSON.  This allows me to drop the ByteArray extension methods that I previously added to the project and it buys me some nice auto-hydration of my UploadedImage later when my processing function is called in the Azure Web Job.  Revisiting the AddMessageToQueueAsync method in my QueueService, we now convert the UploadedImage to JSON instead of a ByteArray.  Note that you’ll need to include the Newtonsoft.JSON Nuget package for the JSON serialization if it’s not already present in your web project.

 

QueueService.cs
  1. public async Task AddMessageToQueueAsync(string messageId, T messageObject)
  2. {
  3.     var queue = GetQueue();
  5.     // Convert to JSON
  6.     var jsonMessage = JsonConvert.SerializeObject(messageObject);
  7.     // Create the actual queue message
  8.     CloudQueueMessage message = new CloudQueueMessage(jsonMessage);
  10.     // Add the message to the queue
  11.     await queue.AddMessageAsync(message);
  12. }

 

In addition, I can revert the Data property in my UploadedImage class to an Auto Property since I no longer need to bother with the [NonSerialized] attribute.  I can also remove the [Serialized] attributes from my models.  Instead, I will place a [JsonIgnore] attribute on my Data property and it will be skipped in the JSON serialization process.  I’ve also removed the previous hard-coded inclusion of one thumbnail from my constructor for UploadedImage. My models now look like this:

UploadedImage.cs
  1. public class UploadedImage
  2. {
  3.     public UploadedImage()
  4.     {
  5.         Thumbnails = new List<Thumbnail>();
  6.     }
  7.     public string Name { get; set; }
  8.     public string ContentType { get; set; }
  10.     [JsonIgnore]
  11.     public byte[] Data { get; set; }
  12.     public string Url { get; set; }
  13.     public List<Thumbnail> Thumbnails { get; set; }
  14. }

 

Thumbnail.cs
  1. public class Thumbnail
  2. {
  3.     public int Width { get; set; }
  4.     public int Height { get; set; }
  5.     public string Url { get; set; }
  6.     public string Name { get; set; }
  7. }

 

Adding Dependency Injection

I had planned to get through this Azure Bits series without Dependency Injection, but I decided I missed it and didn’t want to promote the practice of new-ing up the dependencies directly.  So, I’ve added Ninject to the solution and replaced all direct instantiation of my dependencies.  It’s super easy to add Ninject via Nuget and there’s even a Ninject.MVC5 package available to get you started on wiring the dependencies.  This package will insert a configuration file for Ninject in your App_Start directory.  The important changes I needed to make to the default Ninject configuration file were to the RegisterServices method, where the actual wiring of the dependencies takes place.  You can see that I am also injecting the appSettings and connectionString into my dependencies allowing me to mock these more easily for unit tests.

NinjectConfig.cs
  1. private static void RegisterServices(IKernel kernel)
  2. {
  3.     kernel.Bind<IImageService>().To<ImageService>()
  4.         .WithConstructorArgument(“containerName”, ConfigurationManager.AppSettings[“ImagesContainer”])
  5.         .WithConstructorArgument(“imageRootPath”, ConfigurationManager.AppSettings[“ImageRootPath”])
  6.         .WithConstructorArgument(“connectionString”, ConfigurationManager.ConnectionStrings[“BlobStorageConnectionString”].ConnectionString);
  7.     kernel.Bind<IQueueService<UploadedImage>>().To<QueueService<UploadedImage>>()
  8.         .WithConstructorArgument(“queueName”, ConfigurationManager.AppSettings[“ImagesQueue”])
  9.         .WithConstructorArgument(“connectionString”, ConfigurationManager.ConnectionStrings[“BlobStorageConnectionString”].ConnectionString);
  10. }

 

With Ninject now in place, we add a constructor to HomeController so we can inject the dependencies via Constructor Injection.  Also, I moved the creation of the default thumbnail to the controller for now.  Other than these changes, the HomeController remains unchanged.

 

HomeController.cs
  1. public class HomeController : Controller
  2. {
  3.     private readonly IImageService _imageService;
  4.     private readonly IQueueService<UploadedImage> _queueService;
  5.     public HomeController(IImageService imageService,  IQueueService<UploadedImage> queueService)
  6.     {
  7.         _imageService = imageService;
  8.         _queueService = queueService;
  9.     }
  10.     public ActionResult Index()
  11.     {
  12.         return View(new UploadedImage());
  13.     }
  15.     [HttpPost]
  16.     public async Task<ActionResult> Upload(FormCollection formCollection)
  17.     {
  18.         var model = new UploadedImage();
  19.         if (Request != null)
  20.         {
  21.             HttpPostedFileBase file = Request.Files[“uploadedFile”];
  22.             model = await _imageService.CreateUploadedImage(file);
  24.             // hard-coded adding one thumbnail for now
  25.             model.Thumbnails.Add(new Thumbnail { Width = 200, Height = 300 });
  26.             await _imageService.AddImageToBlobStorageAsync(model);
  27.             await _queueService.AddMessageToQueueAsync(model.Name, model);
  28.         }
  29.         return View(“Index”, model);
  30.     }
  31. }

 

New Format For The Message in Azure Queue

 

After making these changes, you should be able to run the application and upload an image to Azure Blog Storage and the new JSON version of your UploadedImage object should be sitting in the Azure Queue as a message waiting to be picked up by your Azure Web Job. Taking a peek at the images message queue confirms that this is indeed the case:

image

 

Creating the Azure Web Job

After that quick refactoring detour, we are now ready to get on with the business of creating our Azure Web Job which will monitor our Azure Queue for UploadedImage messages and will automatically kick off the processing of thumbnails any time a new UploadedImage is added to the Queue.  There is a nice Azure Web Job project type available in Visual Studio. Simply right-click the solution and choose Add a New Project and look under the Cloud folder for the Azure Web Job project.

 

image

This will create a console application with a few hooks in place for the Azure Web Job.  You’ll first see the standard Program.cs.  Here there will be some default code in the Main method that will create the JobHost and then run it.  This default code assumes that you have connection strings in place either in Azure or in your app.config that are named explicitly as AzureWebJobsDashboard and AzureWebJobsStorage.  If it can’t find these connection strings based on the default implementation of the Main method, you’ll get errors.  There is an overloaded constructor for JobHost that takes an instance of JobHostConfiguration that allows more control over the connection strings and other configuration information for this Web Job.  I prefer to explicitly set the connection strings and then I can name them whatever I want.  So, my Main method looks like this:

 

Web Job – Program.cs
  1. public static void Main()
  2. {
  3.     var connectionString = ConfigurationManager.ConnectionStrings[“BlobStorageConnectionString”].ConnectionString;
  5.     var config = new JobHostConfiguration
  6.     {
  7.         DashboardConnectionString = connectionString,
  8.         StorageConnectionString = connectionString
  9.     };
  10.     var host = new JobHost(config);
  11.     host.RunAndBlock();
  12. }

 

The other important file that gets created by the Azure Web Jobs template is the functions.cs file.  This is the file that contains the function (or functions) that listen to specific Azure queues and are called when new messages are added.  There is only function added in the default implementation. I am going to replace the parameters with ones more appropriate for our task at hand.  You’ll notice that the first parameter in my implementation is an UploadedImage and you’ll note that the parameter is marked with a [QueueTrigger(“images”)] attribute.  This is what tells Azure to call this method when the Queue named “images” enqueues a new message.

Moving to the second parameter, originalImageStream…  this parameter is marked with the [Blob(“images/{Name}”)] attribute.  If you recall, the UploadedImage class has a Name property that contains the name of the original image that we inserted into Azure Blob Storage.  When ProcessQueueMessageAsync is called, Azure hydrates the first parameter, UploadedImage, from the JSON message we inserted into the Queue and then it looks at the Name property (or whatever property we specify in the braces of our Blob attribute).   With this Blob attribute, we are telling Azure to go to my container named “images” and get the image that matches the uploadedImage.Name property and then deliver this to me as a stream in my processing method.  This is very helpful as the first thing you’d want to do without this automatic delivery of the source stream is go to Azure Blob Storage and fetch it.  This is what my ProcessQueueMessageAsync method looks like now.

 

Functions.cs
  1. public class Functions
  2. {
  3.     // This function will get triggered/executed when a new message is written
  4.     // to an Azure Queue called “images”.
  5.     //      Parameters:  
  6.     //          uploadedImage – automatically hydrated from Azure Queue where we placed the message as JSON object
  7.     //          originalImageStream – takes the Name property from uploadedImage and grabs image from blob storage
  8.     public async static void ProcessQueueMessageAsync(
  9.         [QueueTrigger(“images”)] UploadedImage uploadedImage,
  10.         [Blob(“images/{Name}”, FileAccess.Read)] Stream originalImageStream)
  11.     {
  12.         Console.WriteLine(“Processing image: “ + uploadedImage.Name);
  13.     }
  14. }

 

Testing the Azure Web Job

If you right-click your Web Job project and choose Publish as Azure Web Job, your Web Job should get published to your Azure Web Site.  You can double-check that in the Azure Portal by going to Web Apps and selecting Web Jobs.

 

image

 

Ideally, you would want your Web Job running continuously so that it would be called automatically when a new image is added, but depending on the type of Azure account that you have, you may not have the ability to run your Web Job continuously.  However, you can still test it by right-clicking the project in the Solution Explorer and choosing Debug > Start New Instance.  Assuming you have a message in your queue, you should see something similar to the below and you should see that the message is now removed from your Azure Queue.

 

image

 

Pausing for Now

 

Now that we are successfully fetching our UploadedImage message  and our original image, we just need to process the received image file into the thumbnail image(s) specified in our UploadedImage.Thumbnails collection.  In the next Azure Bit, I’ll introduce an ImageProcessor and add the functionality to generate a thumbnail and save it to Azure Blob Storage.  If you missed the other posts in this series, you can start with Azure Bits #1 – Up and Running.

 
Badge-Azure

Microsoft Azure and Cloud Platform
deliver the ultimate flexibility.

Not all applications and workloads are ideally suited for the cloud. The reality faced by most companies demands multiple deployments to ensure they get the reliability, performance and financial benefits from their infrastructure and applications.

Microsoft Azure

A category leader for features, global reach and pricing with both Infrastructure as a Service (IaaS) and Platform as a Service (PaaS) public Azure deployments.

Microsoft Cloud Platform

Offers an Infrastructure as a Service (IaaS) environment which is architecturally consistent with public Azure and enables private cloud deployments.

Combined

Offers customers a way to get into the cloud that meets their needs and ensures they are not locked-in to one type of deployment.

Stay Informed

Sign up for the latest blogs, events, and insights.

We deliver solutions that accelerate the value of Azure.
Ready to experience the full power of Microsoft Azure?
Start Today

Atmosera is thrilled to announce that we have been named GitHub AI Partner of the Year.

X