Azure Function: 404 Page Checker

Sometimes the simplest piece of development can be the most rewarding and I think my Azure Function that checks for broken links on a nightly basis is one of those things. The Azure Function reads from a list of links from a database table and carries out a check to determine if a 200 response is returned. If not, the link will be logged and sent to a user by email using the Sendgrid API.

Scenario

I was working on a project that takes a list of products from an API and stores them in a Hubspot HubDB table. This table contained all product information and the expected URL to a page. All the CMS pages had to be created manually and assigned the URL as stored in the table, which in turn would allow the page to be populated with product data.

As you can expect, the disadvantage of manually created pages is that a URL change in the HubDB table will result in a broken page. Not ideal! In this case, the likelihood of a URL being changed is rare. All I needed was a checker to ensure I was made aware on the odd occasion where a link to the product page could be broken.

I won't go into any further detail but rest assured, there was an entirely legitimate reason for this approach in the grand scheme of the project.

Azure Function

I have modified my original code purely for simplification.

using System;
using System.Collections.Generic;
using System.Net;
using System.Text;
using System.Threading.Tasks;
using Microsoft.Azure.WebJobs;
using Microsoft.Extensions.Logging;
using SendGrid;
using SendGrid.Helpers.Mail;

namespace ProductsSyncApp
{
  public static class ProductLinkChecker
  {
    [FunctionName("ProductLinkChecker")]
    public static void Run([TimerTrigger("%ProductLinkCheckerCronTime%"
      #if DEBUG
      , RunOnStartup=true
      #endif
      )]TimerInfo myTimer, ILogger log)
    {
      log.LogInformation($"Product Link Checker started at: {DateTime.Now:G}");

      #region Iterate through all product links and output the ones that return 404.

      List<string> brokenProductLinks = new List<string>();

      foreach (string link in GetProductLinks())
      {
        if (!IsEndpointAvailable(link))
          brokenProductLinks.Add(link);
      }

      #endregion

      #region Send Email

      if (brokenProductLinks.Count > 0)
        SendEmail(Environment.GetEnvironmentVariable("Sendgrid.FromEmailAddress"), Environment.GetEnvironmentVariable("Sendgrid.ToAddress"), "www.contoso.com - Broken Link Report", EmailBody(brokenProductLinks));

      #endregion

      log.LogInformation($"Product Link Checker ended at: {DateTime.Now:G}");
    }

    /// <summary>
    /// Get list of a product links.
    /// This would come from a datasource somewhere containing a list of correctly expected URL's.
    /// </summary>
    /// <returns></returns>
    private static List<string> GetProductLinks()
    {
      return new List<string>
      {
        "https://www.contoso.com/product/brokenlink1",
        "https://www.contoso.com/product/brokenlink2",
        "https://www.contoso.com/product/brokenlink3",
      };
    }

    /// <summary>
    /// Checks if a URL endpoint is available.
    /// </summary>
    /// <param name="url"></param>
    /// <returns></returns>
    private static bool IsEndpointAvailable(string url)
    {
      try
      {
        HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);

        using HttpWebResponse response = (HttpWebResponse)request.GetResponse();

        if (response.StatusCode == HttpStatusCode.OK)
          return true;

        return false;
      }
      catch
      {
        return false;
      }
    }

    /// <summary>
    /// Create the email body.
    /// </summary>
    /// <param name="brokenLinks"></param>
    /// <returns></returns>
    private static string EmailBody(List<string> brokenLinks)
    {
      StringBuilder body = new StringBuilder();

      body.Append("<p>To whom it may concern,</p>");
      body.Append("<p>The following product URL's are broken:");

      body.Append("<ul>");

      foreach (string link in brokenLinks)
        body.Append($"<li>{link}</li>");

      body.Append("</ul>");

      body.Append("<p>Many thanks.</p>");

      return body.ToString();
    }

    /// <summary>
    /// Send email through SendGrid.
    /// </summary>
    /// <param name="fromAddress"></param>
    /// <param name="toAddress"></param>
    /// <param name="subject"></param>
    /// <param name="body"></param>
    /// <returns></returns>
    private static Response SendEmail(string fromAddress, string toAddress, string subject, string body)
    {
      SendGridClient client = new SendGridClient(Environment.GetEnvironmentVariable("SendGrid.ApiKey"));

      SendGridMessage sendGridMessage = new SendGridMessage
      {
        From = new EmailAddress(fromAddress, "Product Link Report"),
      };

      sendGridMessage.AddTo(toAddress);
      sendGridMessage.SetSubject(subject);
      sendGridMessage.AddContent("text/html", body);

      return Task.Run(() => client.SendEmailAsync(sendGridMessage)).Result;
    }
  }
}

Here's a rundown on what is happening:

  1. A list of links is returned from the GetProductLinks() method. This will contain a list of correct links that should be accessible on the website.
  2. Loop through all the links and carry out a check against the IsEndpointAvailable() method. This method carries out a simple check to see if the link returns a 200 response. If not, it'll be marked as broken.
  3. Add any link marked as broken to the brokenProductLinks collection.
  4. If there are broken links, send an email handled by SendGrid.

As you can see, the code itself is very simple and the only thing that needs to be customised for your use is the GetProductLinks method, which will need to output a list of expected links that a site should contain for cross-referencing.

Email Send Out

When using Azure functions, you can't use the standard .NET approach to send emails and Microsoft recommends that an authenticated SMTP relay service that reduces the likelihood of email providers rejecting the message. More insight into this can be found in the following StackOverflow post - Not able to connect to smtp from Azure Cloud Service.

When it comes to SMTP relay services, SendGrid comes up favourably and being someone who uses it in their current workplace, it was my natural inclination to make use of it in my Azure Function. Plus, they've made things easy by providing a Nuget package to allow direct access to their Web API v3 endpoints.

ASP.NET Core: Using Assembly Build Date For Cache Busting

ASP.NET Core contains a variety of useful Tag Helpers to enable server-side code to participate in creating and rendering HTML elements in our Views. One Tag Helper, in particular, has the ability to cache bust links to static resources such as Image, CSS and JavaScript by appending an asp-append-version="true" attribute.

The asp-append-version attribute automatically adds a version number to the file name using a SHA256 hashing algorithm, so whenever the file is updated, the server generates a new unique version. For a deeper understanding on how ASP.NET Core performs this piece of functionality, give the following StackOverflow post a read: How does javascript version (asp-append-version) work in ASP.NET Core MVC?.

This approach works perfectly if you're linking to your static resources using the relevant HTML tag, for example img, script or link. In my scenario, I'm using a JavaScript library called LabJS - a dynamic script loader that gives the ability to control the loading and execution of different plugins. For example:

<script>
  $LAB
  .script("http://remote.tld/jquery.js").wait()
  .script("/local/plugin1.jquery.js")
  .script("/local/plugin2.jquery.js").wait()
  .script("/local/init.js").wait(function(){
      initMyPage();
  });
</script>

I need to be able to append a query string parameter to one of the JavaScript file references. One thing that came to mind was to use the applications last build-time as the cache busting value. Whenever the application is updated, this value will automatically be updated so no manual intervention is required.

I found code examples from meziantou.net that demonstrated various approaches to acquiring an applications build date. I modified the "Linker timestamp" example to return a Unix timestamp in a newly created class called AssemblyUtils.

public class AssemblyUtils
{
    #region Properties

    public int UnixTimestamp { get; set; }

    #endregion

    /// <summary>
    /// Get timestamp in Unix seconds for the last build.
    /// </summary>
    /// <returns></returns>
    public static int GetBuildTimestamp()
    {
        const int peHeaderOffset = 60;
        const int timestampOffset = 8;

        byte[] bytes = new byte[2048];

        using (FileStream file = new FileStream(Assembly.GetExecutingAssembly().Location, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
            file.Read(bytes, 0, bytes.Length);

        int headerPos = BitConverter.ToInt32(bytes, peHeaderOffset);
        int unixTime = BitConverter.ToInt32(bytes, headerPos + timestampOffset);

        return unixTime;
    }
}

The code will only return the Assembly information if your Visual Studio .csproj file (from version 15.4 onwards) includes the following setting within the <PropertyGroup> settings:

<Deterministic>False</Deterministic>

It would be a waste to constantly call the GetBuildTimestamp() method to acquire assembly information directly within the page View, when the most ideal approach would be to make this call once on application startup.

public void ConfigureServices(IServiceCollection services)
{
    #region Assembly Utils - Build Time

    Action<AssemblyUtils> assemblyBuildOptions = (opt =>
    {
        opt.UnixTimestamp = AssemblyUtils.GetBuildTimestamp();
    });

    services.Configure(assemblyBuildOptions);
    services.AddSingleton(resolver => resolver.GetRequiredService<IOptions<AssemblyUtils>>().Value);

    #endregion
}

We can access the build timestamp value using Dependency Injection within a base controller that gets inherited by all controllers.

public class BaseController : Controller
{
    private int _buildTimetamp { get; set; }

    public BaseController(AssemblyUtils assemblyUtls)
    {
        _buildTimetamp = assemblyUtls.UnixTimestamp;
    }

    public override void OnActionExecuting(ActionExecutingContext context)
    {
        base.OnActionExecuting(context);

        // Assign build timestamp to a View Bag.
        ViewBag.CacheBustingValue = _buildTimetamp;
    }
}

The timestamp is assigned to a ViewBag that can then be accessed at View level.

<script>
  $LAB
  .script("http://remote.tld/jquery.js").wait()
  .script("/local/plugin1.jquery.js")
  .script("/local/plugin2.jquery.js").wait()
  .script("/local/init.js?v=@ViewBag.CacheBustingValue").wait(function(){
      initMyPage();
  });
</script>

This will result in the following output:

<script>
  $LAB
  .script("http://remote.tld/jquery.js").wait()
  .script("/local/plugin1.jquery.js")
  .script("/local/plugin2.jquery.js").wait()
  .script("/local/init.js?v=1609610821").wait(function(){
      initMyPage();
  });
</script>

ASP.NET Core: System.FormatException: Could not parse the JSON file

Another day, another ASP.NET Core error... This time relating to JSON not being parsable. Like the error I posted yesterday, this was another strange one as it only occurred within an Azure environment.

Let me start by showing the file compilation error:

Application '/LM/W3SVC/144182150/ROOT' with physical root 'D:\home\site\wwwroot\' hit unexpected managed exception, exception code = '0xe0434352'. First 30KB characters of captured stdout and stderr logs:
Unhandled exception. System.FormatException: Could not parse the JSON file.
 ---> System.Text.Json.JsonReaderException: '0x00' is an invalid start of a value. LineNumber: 0 | BytePositionInLine: 0.
   at System.Text.Json.ThrowHelper.ThrowJsonReaderException(Utf8JsonReader& json, ExceptionResource resource, Byte nextByte, ReadOnlySpan`1 bytes)
   at System.Text.Json.Utf8JsonReader.ConsumeValue(Byte marker)
   at System.Text.Json.Utf8JsonReader.ReadFirstToken(Byte first)
   at System.Text.Json.Utf8JsonReader.ReadSingleSegment()
   at System.Text.Json.Utf8JsonReader.Read()
   at System.Text.Json.JsonDocument.Parse(ReadOnlySpan`1 utf8JsonSpan, Utf8JsonReader reader, MetadataDb& database, StackRowStack& stack)
   at System.Text.Json.JsonDocument.Parse(ReadOnlyMemory`1 utf8Json, JsonReaderOptions readerOptions, Byte[] extraRentedBytes)
   at System.Text.Json.JsonDocument.Parse(ReadOnlyMemory`1 json, JsonDocumentOptions options)
   at System.Text.Json.JsonDocument.Parse(String json, JsonDocumentOptions options)
   at Microsoft.Extensions.Configuration.Json.JsonConfigurationFileParser.ParseStream(Stream input)
   at Microsoft.Extensions.Configuration.Json.JsonConfigurationFileParser.Parse(Stream input)
   at Microsoft.Extensions.Configuration.Json.JsonConfigurationProvider.Load(Stream stream)
   --- End of inner exception stack trace ---
   at Microsoft.Extensions.Configuration.Json.JsonConfigurationProvider.Load(Stream stream)
   at Microsoft.Extensions.Configuration.FileConfigurationProvider.Load(Boolean reload)
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.Extensions.Configuration.FileConfigurationProvider.HandleException(ExceptionDispatchInfo info)
   at Microsoft.Extensions.Configuration.FileConfigurationProvider.Load(Boolean reload)
   at Microsoft.Extensions.Configuration.FileConfigurationProvider.Load()
   at Microsoft.Extensions.Configuration.ConfigurationRoot..ctor(IList`1 providers)
   at Microsoft.Extensions.Configuration.ConfigurationBuilder.Build()
   at Microsoft.Extensions.Logging.AzureAppServices.SiteConfigurationProvider.GetAzureLoggingConfiguration(IWebAppContext context)
   at Microsoft.Extensions.Logging.AzureAppServicesLoggerFactoryExtensions.AddAzureWebAppDiagnostics(ILoggingBuilder builder, IWebAppContext context)
   at Microsoft.Extensions.Logging.AzureAppServicesLoggerFactoryExtensions.AddAzureWebAppDiagnostics(ILoggingBuilder builder)
   at Microsoft.AspNetCore.Hosting.AppServicesWebHostBuilderExtensions.<>c.<UseAzureAppServices>b__0_0(ILoggingBuilder builder)
   at Microsoft.Extensions.DependencyInjection.LoggingServiceCollectionExtensions.AddLogging(IServiceCollection services, Action`1 configure)
   at Microsoft.AspNetCore.Hosting.WebHostBuilderExtensions.<>c__DisplayClass8_0.<ConfigureLogging>b__0(IServiceCollection collection)
   at Microsoft.AspNetCore.Hosting.HostingStartupWebHostBuilder.<>c__DisplayClass6_0.<ConfigureServices>b__0(WebHostBuilderContext context, IServiceCollection services)
   at Microsoft.AspNetCore.Hosting.HostingStartupWebHostBuilder.ConfigureServices(WebHostBuilderContext context, IServiceCollection services)
   at Microsoft.AspNetCore.Hosting.GenericWebHostBuilder.<.ctor>b__5_2(HostBuilderContext context, IServiceCollection services)
   at Microsoft.Extensions.Hosting.HostBuilder.CreateServiceProvider()
   at Microsoft.Extensions.Hosting.HostBuilder.Build()
   at Site.Web.Program.Main(String[] args) in C:\Development\surinder-main-website\Site.Web\Program.cs:line 11

Process Id: 2588.
File Version: 13.1.20169.6. Description: IIS ASP.NET Core Module V2 Request Handler. Commit: 62c098bc170f50feca15916e81cb7f321ffc52ff

The application was not consuming any form of JSON as part of its main functionality. The only JSON being used were three variations of appsettings.json - each one for development, staging and production. So this had to be the source of the issue. The error message also confirmed this as Program.cs was referenced and it’s at this point where the application startup code is run.

My first thought was I must have forgotten a comma or missing a closing quote for one of my values. After running the JSON through a validator, it passed with flying colours.

Solution

After some investigation, the issue was caused by incorrect encoding of the file. All the appsettings.json files were set to "UTF-8" and as a result, possibly causing some metadata to be added stopping the application from reading the files. Once this was changed to "UTF-8-BOM" through Notepad++ everything worked fine.

ASP.NET Core: Failed to start application '/LM/W3SVC/####/ROOT', ErrorCode '0x8007023e'.

You gotta love .NET core compilation errors! They provide the most ambiguous error messages known to man. I have noticed the error message and accompanying error code could be caused by a multitude of factors. This error is no different so I’ll make my contribution, hoping this may help someone else.

The error in question occurred really randomly whilst deploying a minor HTML update to a .NET Core site I was hosting within an Azure Web App. It couldn’t have been a simpler release - change to some markup in a View. When the site loaded, I was greeted with the following error:

Failed to start application '/LM/W3SVC/####/ROOT', ErrorCode '0x8007023e’.

I was able to get some further information about the error from the Event Log:

Application 'D:\home\site\wwwroot\' failed to start. Exception message:
Executable was not found at 'D:\home\site\wwwroot\%LAUNCHER_PATH%.exe'
Process Id: 10848.
File Version: 13.1.19331.0. Description: IIS ASP.NET Core Module V2.

The error could only be reproduced on Azure and not within my local development and staging environments. I created a new deployment slot to check if somehow my existing slot got corrupted. Unfortunately, this made no difference. The strange this is, the application was working completely fine up until this release. It's still unknown to me what could have happened for this error to occur all of a sudden.

Solution

It would seem that no one else on the planet experienced this issue when Googling the error message and error code. After a lot of fumbling around, the fix ended up being relatively straight-forward. The detail provided by the Event Log pointed me in the right direction and the clue was in the %LAUNCHER_PATH% placeholder. The %LAUNCHER_PATH% placeholder is set in the web.config and this is normally replaced when the application is run in Visual Studio or IIS.

In Azure, both %LAUNCHER_PATH% and %LAUNCHER_ARGS% variables need to be explicitly set. The following line in the web.config needs to be changed from:

<aspNetCore processPath="%LAUNCHER_PATH%" arguments="%LAUNCHER_ARGS%" stdoutLogEnabled="false" stdoutLogFile=".\logs\stdout" forwardWindowsAuthToken="false" startupTimeLimit="3600" requestTimeout="23:00:00" hostingModel="InProcess">

To:

<aspNetCore processPath=".\Site.Web.exe" arguments="" stdoutLogEnabled="false" stdoutLogFile=".\logs\stdout" forwardWindowsAuthToken="false" startupTimeLimit="3600" requestTimeout="23:00:00" hostingModel="InProcess">

The processPath is now pointing to the executable generated by the project. In this case, "Site.Web.exe". Also, since no arguments are being parsed in my build, the arguments attribute is left empty. When you push up your next release, the error should be rectified.

As a side note, there was one thing recommended to me by Azure support regarding my publish settings in Visual Studio. It was recommended that I should set the deployment mode from "Framework-Dependent" to "Self-Contained". This will ensure the application will always run in its current framework version on the off-chance framework changes happen at an Azure level.

ASP.NET Core: Making Your Project Production Ready For Deployment

I decided to write this post to act primarily as a reminder to myself for when I'm publishing an ASP.NET Core project ready for a production environment. Most of the ASP.NET Core projects I'm currently working on are based on pre-existing client or platform-based boilerplates and when taking these on, they vary in quality and a result, some key project settings are just not implemented.

I will be covering the following areas:

  • Ensuring the correct environment variable is set for your publish profile.
  • Setting custom error pages.
  • Switching between development and production appsetting.json files.

Setting Environment In Publish Profile

After you have created the publish profile, update the .pubxml file (found under the "/Properties/PublishProfiles" directory within your project) and add a EnvironmentName variable:

<PropertyGroup>
    <EnvironmentName>Production</EnvironmentName>
</PropertyGroup>

This variable is very much key to the whole operation. Without it, the project will be stuck in development mode and the sections, listed below, will not work.

Setting Custom Error Pages

We are only interested in seeing a custom error page when in production mode. To do this, we need to:

  1. Update the Startup.cs file to enable status code error pages.
  2. Create an error controller to render the custom error pages.

Startup.cs

To serve our custom error page, we need to declare the route using the app.UseStatusCodePagesWithReExecute() method. This method includes a placeholder {0}, which will be replaced with the status code integer - 404, 500, etc. We can then render different views depending on the error code returned. For example:

  • /Error/404
  • /Error/500
public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
    // Render full blown exception only if in development mode.
    if (env.IsDevelopment())
    {
        app.UseDeveloperExceptionPage();
    }
    else
    {
        app.UseStatusCodePagesWithReExecute("/Error/{0}");
        app.UseHsts();
    }
}

Error Controller

Based on the status code returned, different views can be rendered.

public class ErrorController : Controller
{
    [ResponseCache(Duration = 0, Location = ResponseCacheLocation.None, NoStore = true)]
    [Route("/Error/{statusCode}")]
    public ViewResult Status(int statusCode)
    {
        if (statusCode == StatusCodes.Status404NotFound)
        {
            return View("~/Views/Error/NotFound.cshtml");
        }
        else
        {
            return View("~/Views/Error/GeneralError.cshtml");
        }
    }
}

Web.config

Being a .NET Core project, there is one area that is easily overlooked as we're so focused on the Startup.cs and appsettings.json files - that is the web.config. We need to ensure the environment variable is set here also by adding the following:

<environmentVariables>
    ...
    <environmentVariable name="ASPNETCORE_ENVIRONMENT" value="Production" />
    ...
</environmentVariables>

If the "ASPNETCORE_ENVIRONMENT" value isn't set correctly at this point, this will cause issues/inconsistencies globally.

Switching To appsetting.production.json

You've probably noticed that your ASP.NET Core project contains three appsettings.json files - each one for your environment:

  • appsettings.json
  • appsettings.development.json
  • appsettings.production.json

If your ASP.NET Core project version is less than 3.0, you can switch between each appsettings.json file by adding the following code to your Startup.cs file:

public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
    IConfigurationBuilder configBuilder = new ConfigurationBuilder()
        .SetBasePath(env.ContentRootPath)
        .AddJsonFile("appsettings.json", true, true)
        .AddJsonFile($"appsettings.{env.EnvironmentName}.json", true)
        .AddEnvironmentVariables();

    Configuration = configBuilder.Build();
}

However, if running on ASP.NET Core 3.0+, you will need to use WebHost.CreateDefaultBuilder(args) method that will be added to the Programs.cs file.

public class Program
{
    public static void Main(string[] args)
    {
        CreateHostBuilder(args).Build().Run();
    }

    public static IHostBuilder CreateHostBuilder(string[] args) =>
        Host.CreateDefaultBuilder(args)
            .UseContentRoot(Directory.GetCurrentDirectory())
            .ConfigureWebHostDefaults(webBuilder =>
            {
                webBuilder.UseStartup<Startup>();
            });
}

The CreateDefaultBuilder performs the following environment-related tasks (to name a few):

  • Sets the content root to the path returned by Directory.GetCurrentDirectory().
  • Loads host configuration from environment variables prefixed with ASPNETCORE_ (for example, ASPNETCORE_ENVIRONMENT).
  • Loads application configuration settings in the following order, starting from appsettings.json and then appsettings.{Environment}.json.

As you can see, from ASP.NET Core 3.0 onwards, quite a lot is being done for you from such a minimal amount of code.

WebMarkupMin: Configuring Minification Level

When WebMarkupMin is first added to a web project, by default the minification is set very high and found that it caused my pages not to be considered valid HTML and worse, things looking slightly broken.

WebMinMarkup minified things that I didn’t even think required minification and the following things got stripped out of the page:

  • End HTML tags.
  • Quotes.
  • Protocols from attributes.
  • Form input type attribute.

The good thing is, the level of minification can be controlled by creating a configuration file inside the App_Start directory of your MVC project. I thought it was be useful to post a copy of my WebMinMarkup configuration file for reference when working on future MVC projects and might also prove useful for others as well.

public class WebMarkupMinConfig
{
    public static void Configure(WebMarkupMinConfiguration configuration)
    {
        configuration.AllowMinificationInDebugMode = false;
        configuration.AllowCompressionInDebugMode = false;
        configuration.DisablePoweredByHttpHeaders = true;

        DefaultLogger.Current = new ThrowExceptionLogger();

        IHtmlMinificationManager htmlMinificationManager = HtmlMinificationManager.Current;
        HtmlMinificationSettings htmlMinificationSettings = htmlMinificationManager.MinificationSettings;
        htmlMinificationSettings.RemoveRedundantAttributes = true;
        htmlMinificationSettings.RemoveHttpProtocolFromAttributes = false;
        htmlMinificationSettings.RemoveHttpsProtocolFromAttributes = false;
        htmlMinificationSettings.AttributeQuotesRemovalMode = HtmlAttributeQuotesRemovalMode.KeepQuotes;
        htmlMinificationSettings.RemoveOptionalEndTags = false;
        htmlMinificationSettings.RemoveEmptyAttributes = false;
        htmlMinificationSettings.PreservableAttributeList = "input[type]";

        IXhtmlMinificationManager xhtmlMinificationManager = XhtmlMinificationManager.Current;
        XhtmlMinificationSettings xhtmlMinificationSettings = xhtmlMinificationManager.MinificationSettings;
        xhtmlMinificationSettings.RemoveRedundantAttributes = true;
        xhtmlMinificationSettings.RemoveHttpProtocolFromAttributes = false;
        xhtmlMinificationSettings.RemoveHttpsProtocolFromAttributes = false;
        xhtmlMinificationSettings.RemoveEmptyAttributes = false;

        IXmlMinificationManager xmlMinificationManager = XmlMinificationManager.Current;
        XmlMinificationSettings xmlMinificationSettings = xmlMinificationManager.MinificationSettings;
        xmlMinificationSettings.CollapseTagsWithoutContent = true;

        IHttpCompressionManager httpCompressionManager = HttpCompressionManager.Current;
        httpCompressionManager.CompressorFactories = new List<ICompressorFactory>
        {
            new DeflateCompressorFactory(),
            new GZipCompressorFactory()
        };
    }
}

Once the configuration file is added to your project, the last thing you need to do is add a reference in the Global.asax file.

protected void Application_Start()
{
    // Compression.
    WebMarkupMinConfig.Configure(WebMarkupMinConfiguration.Instance);
}

Running .Net Core Site From A Subdirectory In Visual Studio

I have some websites on a production environment that need to be run from within a subdirectory and in order to carry out proper testing during development to ensure all references to CSS, JS and images files work. By default, when a .NET Core site is run from Visual Studio it will always start from the root, resulting in a broken looking page.

From .NET Core 2.0, within your Startup.cs file, you can set a sub-directory using the UsePathBase extension within the Configure method:

public void Configure(IApplicationBuilder app, IHostingEnvironment env)
{
    if (env.IsDevelopment())
        app.UsePathBase("/mydirectory");
}

Now when the site runs, it’ll be accessible from /mydirectory. In my code example, I only want to set the path base if it development mode. When released to production, the path will be configured at IIS level.

The only annoyance is when you run the site in Visual Studio, it will still start at the root and not at your newly declared subdirectory. I was surprised to see that the site is still accessible at the root, when you would expect the root path to be disabled or even greeted with a 404 response.

On first glance, I thought there was a bug in my path base declaration and perhaps I missed something. After viewing a closed Github issue raised back in 2017, it was stated that this is in fact the intended functionality. This is a minor bug bear I can live with.

Temporary ASP.NET Files Directory Compilation Error After Project Rename

After renaming my MVC project from "SurinderBhomra" to "Site.Web" (to come across less self-serving) I get the following error:

Compiler Error Message: CS0246: The type or namespace name 'xxx' could not be found (are you missing a using directive or an assembly reference?)

CS0246 Compiler Error - Temporary ASP.NET Files

But the misleading part of this compiler error is the source file reference to ASP.NET Temporary Files directory, which led me to believe that my build was been cached when it was actually to do with the fact I missed some areas where the old project name still remained.

I carried out a careful rename throughout my project by updating all the places that mattered, such as:

  • Namespaces
  • Using statements
  • "AssemblyTitle" and "AssemblyProduct" attributes in AssemblyInfo.cs
  • Assembly name and Default Namespace in Project properties (followed by a rebuild)

The key area I missed that caused the above compiler error is overlooking the the namespace section in the /Views/web.config file.

<system.web.webpages.razor>
  <host factorytype="System.Web.Mvc.MvcWebRazorHostFactory, System.Web.Mvc, Version=5.2.3.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35">
  <pages pagebasetype="System.Web.Mvc.WebViewPage">
    <namespaces>
      ...
      ...
      ...
      <add namespace="System.Web">
      ...
      ...
      ...
    </add></namespaces>
  </pages>
</host></system.web.webpages.razor>

When you first create your project in Visual Studio, it automatically adds its original namespace to this file. This also goes for any other web.config you happen to have nested in other areas inside you MVC project.

Responsive Images In ASP.NET: Converting Image Tag To Picture Tag

A picture tag allows us to serve different sized images based on different viewport breakpoints or pixel-ratios, resulting in better page load performance. Google's Pagespeed Insights negatively scores your site if responsive images aren't used. Pretty much all modern browsers support this markup and on the off chance it doesn't, an image fallback can be set.

Using the picture markup inside page templates is pretty straight-forward, but when it comes to CMS related content where HTML editors only accommodate image tags, it's really difficult to get someone like a client to add this form of markup. So the only workaround is to transform any image tag into a picture tag at code-level.

Code: ConvertImageToPictureTag Extension Method

The ConvertImageToPictureTag method will perform the following tasks:

  1. Loop through all image tags.
  2. Get the URL of the image from the "src" attribute.
  3. Get other attributes such as "alt" and "style".
  4. Generate picture markup and add as many source elements based on the viewport breakpoints required, apply the URL of the image, style and alt text.
  5. Replace the original image tag with the new picture tag.

The ConvertImageToPictureTag code uses HtmlAgilityPack, making it very easy to loop through all HTML nodes and manipulate the markup. In addition, this implementation relies on a lightweight client-side JavaScript plugin - lazysizes. The lazysizes plugin will delay the loading of the higher resolution image based on the viewport rules in the picture tag until the image is scrolled into view.

using HtmlAgilityPack;
using Site.Common.Kentico;
using System;
using System.Collections.Generic;
using System.Collections.Specialized;
using System.Linq;
using System.Text;
using System.Web;

namespace SurinderBhomra.Common.Extensions
{
    public static class ContentManipulatorExtensions
    {
        /// <summary>
        /// Transforms all image tags to a picture tag inside parsed HTML.
        /// All source image URL's need to contain a "width" query parameter in order to have a resize starting point.
        /// </summary>
        /// <param name="content"></param>
        /// <param name="percentageReduction"></param>
        /// <param name="minimumWidth">The minimum width an image has to be to warrant resizing.</param>
        /// <param name="viewPorts"></param>
        /// <returns></returns>
        public static string ConvertImageToPictureTag(this string content, int percentageReduction = 10, int minimumWidth = 200, params int[] viewPorts)
        {
            if (viewPorts?.Length == 0)
                throw new Exception("Viewport parameter is required.");

            if (!string.IsNullOrEmpty(content))
            {
                //Create a new document parser object.
                HtmlDocument document = new HtmlDocument();

                //Load the content.
                document.LoadHtml(content);

                //Get all image tags.
                List<HtmlNode> imageNodes = document.DocumentNode.Descendants("img").ToList();
                
                if (imageNodes.Any())
                {
                    // Loop through all image tags.
                    foreach (HtmlNode imgNode in imageNodes)
                    {
                        // Make sure there is an image source and it is not externally linked.
                        if (imgNode.Attributes.Contains("src") && !imgNode.Attributes["src"].Value.StartsWith("http", StringComparison.Ordinal))
                        {
                            #region Image Attributes - src, class, alt, style
                            
                            string imageSrc = imgNode.Attributes["src"].Value.Replace("~", string.Empty);
                            string imageClass = imgNode.Attributes.Contains("class") ? imgNode.Attributes["class"].Value : string.Empty;
                            string imageAlt = imgNode.Attributes.Contains("alt") ? imgNode.Attributes["alt"].Value : string.Empty;
                            string imageStyle = imgNode.Attributes.Contains("style") ? imgNode.Attributes["style"].Value : string.Empty;

                            #endregion

                            #region If Image Source has a width query parameter, this will be used as the starting size to reduce images

                            int imageWidth = 0;

                            UriBuilder imageSrcUri = new UriBuilder($"http://www.surinderbhomra.com{imageSrc}");
                            NameValueCollection imageSrcQueryParams = HttpUtility.ParseQueryString(imageSrcUri.Query);

                            if (imageSrcQueryParams?.Count > 0 && !string.IsNullOrEmpty(imageSrcQueryParams["width"]))
                                imageWidth = int.Parse(imageSrcQueryParams["width"]);

                            // If there is no width parameter, then we cannot resize this image.
                            // Might be an older non-responsive image link.
                            if (imageWidth == 0 || imageWidth <= minimumWidth)
                                continue;

                            // Clear the query string from image source.
                            imageSrc = imageSrc.ClearQueryStrings();

                            #endregion

                            // Create picture tag.
                            HtmlNode pictureNode = document.CreateElement("picture");

                            if (!string.IsNullOrEmpty(imageStyle))
                                pictureNode.Attributes.Add("style", imageStyle);

                            #region Add multiple source tags

                            StringBuilder sourceHtml = new StringBuilder();

                            int newImageWidth = imageWidth;
                            for (int vp = 0; vp < viewPorts.Length; vp++)
                            {
                                int viewPort = viewPorts[vp];

                                // We do not not want to apply the percentage reduction to the first viewport size.
                                // The first image should always be the original size.
                                if (vp != 0)
                                    newImageWidth = newImageWidth - (newImageWidth * percentageReduction / 100);

                                sourceHtml.Append($"<source srcset=\"{imageSrc}?width={newImageWidth}\" data-srcset=\"{imageSrc}?width={newImageWidth}\" media=\"(min-width: {viewPort}px)\">");
                            }

                            // Add fallback image.
                            sourceHtml.Append($"<img src=\"{imageSrc}?width=50\" style=\"width: {imageWidth}px\" class=\"{imageClass} lazyload\" alt=\"{imageAlt}\" />");

                            pictureNode.InnerHtml = sourceHtml.ToString();

                            #endregion

                            // Replace the image node with the new picture node.
                            imgNode.ParentNode.ReplaceChild(pictureNode, imgNode);
                        }
                    }

                    return document.DocumentNode.OuterHtml;
                }
            }

            return content;
        }
    }
}

To use this extension, add this to any string containing HTML markup, as so:

// The HTML markup will generate responsive images using based on the following parameters:
// - Images to be resized in 10% increments.
// - Images have to be more than 200px wide.
// - Viewport sizes to take into consideration: 1000, 768, 300.
string contentWithResponsiveImages = myHtmlContent.ConvertImageToPictureTag(10, 200, 1000, 768, 300);

Sidenote

The code I've shown doesn't carry out any image resizing, you will need to integrate that yourself. Generally, any good content management platform will have the capability to serve responsive images. In my case, I use Kentico and can resize images by adding a "width" and/or "height" query parameter to the image URL.

In addition, all image URL's used inside an image tags "src" attribute requires a width query string parameter. The value of the width parameter will be the size the image in its largest form. Depending on the type of platform used, the URL structure to render image sizes might be different. This will be the only place where the code will need to be retrofitted to adapt to your own use case.

ASP.NET Developers Who Use Eval()... Evaluate Yourself!

The title of this post might seem a tad extreme, but I just feel so strongly about it! Ever since I started learning ASP.NET those many years ago, I've never been a fan of using "Eval" in data-bound controls I primarily use, such as GridViews, Repeaters and DataList. When I see it still being used regularly in web applications I cringe a little and I feel I need to express some reasons to why it should stop being used.

I think working on an application handed down to me from an external development agency pushed me to write this post... Let's call it a form of therapy! I won't make this post a rant and will "try" to be constructive and concise. My views might come across a little one-sided, but I promise I will start with at least one good thing to say about our evil friend Eval.

Postive: Quick To Render Simple Data

If the end goal is to list out some string values as is from the database with some minor manipulation from a relatively small dataset, I almost have no problem with that, even though I still believe it can be used and abused by inexperienced developers.

Negative: Debugging


The main disadvantage of embedding code inside your design file (.aspx or .ascx) is that it's not very easy to view the output during debugging. This causes a further headache when your Eval contains some conditional statements to alter the output on a row-by-row basis.

Negative: Difficult To Carry Out Complex HTML Changes

I wouldn't recommend using Eval in scenario's where databound rows require some form of HTML change. I've seen some ugly implementations where complex conditional statements were used to list out data in a creative way. If the HTML ever had to be changed through design updates, it would be a lot more time consuming to carry when compared to moving around some form controls that are databound through a RowDataBound event.

Negative: Ugly To Look At

This point will come across very superficial. Nevertheless, what I find painful to look at is when Eval is still used to carry out more functionality by calling additional methods and potentially repeating the same functionality numerous times.

Performance/Efficiency

From my research, it's not clear if there specifically is a performance impact in using Eval alone, especially with the advances in the .NET framework over the years. A post from 2012 on StackExchange brought up a very good point:

Eval uses reflection to get the value of the relevant property/field, and using Reflection to get values from object members is very slow.

If the type of an object can be determined at runtime, you're better off explicitly declaring this. After all, it's good coding standards. In the real world, the performance impact is nominal depending on the number of records you are dealing with. Not recommended for building scalable applications. I generally notice a slow down (in milliseconds) when outputting 500 rows of data.

I have read that reflection is not as much of an issue in the most recent versions of the .NET framework when compared to say, .NET 1.1. But I am unable to find any concrete evidence of this. Regardless, I'd always prefer to use the faster approach, even if I am happening to shave off a few milliseconds in the process.

Conclusion

Just don't use Eval. Regardless of the size of the dataset I am dealing with, there would only be two approaches I'd ever use:

  1. RowDataBoundEvent: A controls RowDataBoundEvent event is triggered every time a row is databound with data. This approach enables us to modify the rows appearance and structure in a specific way depending on the type of rules we have in place.
  2. Start From Scratch: Construct the HTML markup by hand based on the datasource and render to the page.

If I were to be building a scalable application dealing with thousands of rows of data, I am generally inclined to go for option 2. As you're not relying on a .NET control, you won't be contributing to the page viewstate.

Even though I have been working on a lot more applications using MVC where I have more control on streamlining the page output, I still have to dabble with Web Forms. I feel with Web Forms, it's very easy to make a page that performs really bad, which makes it even more important to ensure you are taking all necessary steps to ensure efficiency.