Cloudflare API - Purge Files By URL In C#

Earlier this week I wrote about the reasons to why I decided to use Cloudflare for my website. I've been working on utilising Cloudflare's API to purge the cache on demand for when files need to be updated within the CDN. To do this, I decided to write a method that will primarily use one API endpoint - /purge_cache. This endpoint allows a maximum of 30 URL's at one time to be purged, which is flexible enough to fit the majority of day-to-day use cases.

To communicate with the API, we need to provide three pieces of information:

  1. Account Email Address
  2. Zone ID
  3. API Key

The last two pieces of information can be found within the dashboard of your Cloudflare account.

Code - CloudflareCacheHelper Class

The CloudflareCacheHelper class consists of a single method PurgeSelectedFiles() and the following class objects used for serializing and deserializing our responses from API requests:

  • CloudflareFileInfo
  • CloudflareZone
  • CloudflareResultInfo
  • CloudflareResponse

Not all the properties within each of the class objects are being used at the moment based on the requests I am making. But the CloudflareCacheHelper class will be updated with more methods as I delve further into Cloudflare's functionality.

public class CloudflareCacheHelper
{
    public string _userEmail;
    public string _apiKey;
    public string _zoneId;

    private readonly string ApiEndpoint = "https://api.cloudflare.com/client/v4";

    /// <summary>
    /// By default the Cloudflare API values will be taken from the Web.Config.
    /// </summary>
    public CloudflareCacheHelper()
    {
        _apiKey = ConfigurationManager.AppSettings["Cloudflare.ApiKey"];
        _userEmail = ConfigurationManager.AppSettings["Cloudflare.UserEmail"];
        _zoneId = ConfigurationManager.AppSettings["Cloudflare.ZoneId"];
    }

    /// <summary>
    /// Set the Cloudflare API values explicitly.
    /// </summary>
    /// <param name="userEmail"></param>
    /// <param name="apiKey"></param>
    /// <param name="zoneId"></param>
    public CloudflareCacheHelper(string userEmail, string apiKey, string zoneId)
    {
        _userEmail = userEmail;
        _apiKey = apiKey;
        _zoneId = zoneId;
    }
        
    /// <summary>
    /// A collection of file paths (max of 30) will be accepted for purging cache.
    /// </summary>
    /// <param name="filePaths"></param>
    /// <returns>Boolean value on success or failure.</returns>
    public bool PurgeSelectedFiles(List<string> filePaths)
    {
        CloudflareResponse purgeResponse = null;

        if (filePaths?.Count > 0)
        {
            try
            {
                HttpWebRequest purgeRequest = WebRequest.CreateHttp($"{ApiEndpoint}/zones/{_zoneId}/purge_cache");
                purgeRequest.Method = "POST";
                purgeRequest.ContentType = "application/json";
                purgeRequest.Headers.Add("X-Auth-Email", _userEmail);
                purgeRequest.Headers.Add("X-Auth-Key", _apiKey);

                #region Create list of Files for Submission In The Structure The Response Requires

                CloudflareFileInfo fileInfo = new CloudflareFileInfo
                {
                    Files = filePaths
                };

                byte[] data = Encoding.ASCII.GetBytes(JsonConvert.SerializeObject(fileInfo));

                purgeRequest.ContentLength = data.Length;

                using (Stream fileStream = purgeRequest.GetRequestStream())
                {
                    fileStream.Write(data, 0, data.Length);
                    fileStream.Flush();
                }

                #endregion

                using (WebResponse response = purgeRequest.GetResponse())
                {
                    using (StreamReader purgeStream = new StreamReader(response.GetResponseStream()))
                    {
                        string responseJson = purgeStream.ReadToEnd();

                        if (!string.IsNullOrEmpty(responseJson))
                            purgeResponse = JsonConvert.DeserializeObject<CloudflareResponse>(responseJson);
                    }
                }
            }
            catch (Exception ex)
            {
                throw ex;
            }

            return purgeResponse.Success;
        }

        return false;
    }

    #region Cloudflare Class Objects

    public class CloudflareFileInfo
    {
        [JsonProperty("files")]
        public List<string> Files { get; set; }
    }

    public class CloudflareZone
    {
        [JsonProperty("id")]
        public string Id { get; set; }

        [JsonProperty("type")]
        public string Type { get; set; }

        [JsonProperty("name")]
        public string Name { get; set; }

        [JsonProperty("content")]
        public string Content { get; set; }

        [JsonProperty("proxiable")]
        public bool Proxiable { get; set; }

        [JsonProperty("proxied")]
        public bool Proxied { get; set; }

        [JsonProperty("ttl")]
        public int Ttl { get; set; }

        [JsonProperty("priority")]
        public int Priority { get; set; }

        [JsonProperty("locked")]
        public bool Locked { get; set; }

        [JsonProperty("zone_id")]
        public string ZoneId { get; set; }

        [JsonProperty("zone_name")]
        public string ZoneName { get; set; }

        [JsonProperty("modified_on")]
        public DateTime ModifiedOn { get; set; }

        [JsonProperty("created_on")]
        public DateTime CreatedOn { get; set; }
    }

    public class CloudflareResultInfo
    {
        [JsonProperty("page")]
        public int Page { get; set; }

        [JsonProperty("per_page")]
        public int PerPage { get; set; }

        [JsonProperty("count")]
        public int Count { get; set; }

        [JsonProperty("total_count")]
        public int TotalCount { get; set; }
    }

    public class CloudflareResponse
    {
        [JsonProperty("result")]
        public CloudflareZone Result { get; set; }

        [JsonProperty("success")]
        public bool Success { get; set; }

        [JsonProperty("errors")]
        public IList<object> Errors { get; set; }

        [JsonProperty("messages")]
        public IList<object> Messages { get; set; }

        [JsonProperty("result_info")]
        public CloudflareResultInfo ResultInfo { get; set; }
    }

    #endregion
}

Example - Purging Cache of Two Files

A string collection of URL's can be passed into the method to allow for the cache of a batch of files to be purged in a single request. If all goes well, the success response should be true.

CloudflareCacheHelper cloudflareCache = new CloudflareCacheHelper();

bool isSuccess = cloudflareCache.PurgeSelectedFiles(new List<string> {
                                    "https://www.surinderbhomra.com/getmedia/7907d934-805f-4bd3-86e7-a6b2027b4ba6/CloudflareResponseMISS.png",
                                    "https://www.surinderbhomra.com/getmedia/89679ffc-ca2f-4c47-8d41-34a6efdf7bb8/CloudflareResponseHIT.png"
                                });

Rate Limits

The Cloudflare API sets a maximum of 1,200 requests in a five minute period. Cache-Tag purging has a lower rate limit of up to 2,000 purge API calls in every 24 hour period. You may purge up to 30 tags in one API call.

Useful Method To Deserialize XML or JSON To A Class Object

I have created a helper class that will allow me to consume any XML or JSON request for deserialization into a class object. As you can see from the code below, the GetJsonRequest() and GetXmlRequest() methods allow you to pass an unknown type as well as the URL to where you are getting your request from. This makes things very straight-forward when you want to easily strongly type the data.

public class ApiWebRequestHelper
{
    /// <summary>
    /// Gets a request from an external JSON formatted API and returns a deserialized object of data.
    /// </summary>
    /// <typeparam name="T"></typeparam>
    /// <param name="requestUrl"></param>
    /// <returns></returns>
    public static T GetJsonRequest<T>(string requestUrl)
    {
        try
        {
            WebRequest apiRequest = WebRequest.Create(requestUrl);
            HttpWebResponse apiResponse = (HttpWebResponse)apiRequest.GetResponse();

            if (apiResponse.StatusCode == HttpStatusCode.OK)
            {
                string jsonOutput;
                using (StreamReader sr = new StreamReader(apiResponse.GetResponseStream()))
                    jsonOutput = sr.ReadToEnd();
                    
                var jsResult = JsonConvert.DeserializeObject<T>(jsonOutput);

                if (jsResult != null)
                    return jsResult;
                else
                    return default(T);
            }
            else
            {
                return default(T);
            }
        }
        catch (Exception ex)
        {
            // Log error here.

            return default(T);
        }
    }

    /// <summary>
    /// Gets a request from an external XML formatted API and returns a deserialized object of data.
    /// </summary>
    /// <typeparam name="T"></typeparam>
    /// <param name="requestUrl"></param>
    /// <returns></returns>
    public static T GetXmlRequest<T>(string requestUrl)
    {
        try
        {
            WebRequest apiRequest = WebRequest.Create(requestUrl);
            HttpWebResponse apiResponse = (HttpWebResponse)apiRequest.GetResponse();

            if (apiResponse.StatusCode == HttpStatusCode.OK)
            {
                string xmlOutput;
                using (StreamReader sr = new StreamReader(apiResponse.GetResponseStream()))
                    xmlOutput = sr.ReadToEnd();

                XmlSerializer xmlSerialize = new XmlSerializer(typeof(T));

                var xmlResult = (T)xmlSerialize.Deserialize(new StringReader(xmlOutput));

                if (xmlResult != null)
                    return xmlResult;
                else
                    return default(T);
            }
            else
            {
                return default(T);
            }
        }
        catch (Exception ex)
        {
            // Log error here.
            return default(T);
        }
    }
}

The ApiWebRequestHelper class relies on the following namespaces:

  • Newtonsoft Json
  • System.Xml.Serialization
  • ​​System.IO;

The ApiWebRequestHelper can be used in the following way:

// Get Json Request
ApiWebRequestHelper.GetJsonRequest<MyCustomJsonClass>("http://www.surinderbhomra.com/api/result.json");

// Get XML Request
ApiWebRequestHelper.GetXmlRequest<MyCustomXMLClass>("http://www.surinderbhomra.com/api/result.xml");

JSON/XML To Strongly-Typed C# Object In Visual Studio

Today, I stumbled across a really neat feature in Visual Studio 2015 that gives you the ability to create a strongly-typed C# class directly into your class library. I'm amazed that I've happened to overlook this key feature since Visual Studio 2012!

Better late than never.

In the past, when consuming large JSON data-structures, I normally head off to json2csharp.com to help me get the initial class objects generated, which works a treat. I've blogged about my experiences using json2csharp here.

The strongly-type class generator feature is hidden away in a place I would never have thought to look - Edit > Paste Special, where you will be given the option to either generate a class object based on XML or JSON. But it's in there for a reason.

Paste Special - To Json or XML Class

All that needs to be done now is to either copy some XML or JSON ready to paste into a newly created class. I do find the generated output quite untidy, but this is a great starting point to generating complex data structures. 

Web.Config/App.Config Maintainability

Web Configuration Snippet When working on large projects whether it be websites or software applications, I like to try and make sure that settings from within my app/web configuration files are not only easily accessible within code, but also maintainable for future updates.

Over the years, I have tried different approaches in an attempt to streamline how I use ASP.NET's configuration files within my day-to-day development and this is what I currently do.

Group and Alphabetise

I have started to sort all my settings alphabetically and group common functions/settings together. This alone makes navigating through a large configuration file much more easier.

In addition, having a standard in-house development style and agreeing with your fellow developers how sections within the configuration file is expected to be structured can be useful. For example, on web projects I've worked on, all key sections are  sorted in the following order:

  1. configSections
  2. appSettings
  3. connectionStrings
  4. system.web
  5. customErrors
  6. system.webServer
  7. locations

Common Naming Conventions

I like to name my appsettings in the following format: "<Setting-Group>.<Name>". So if I were to add appsettings that related to Twitter, it would look something as the following:

<add key="Twitter.ApiUrl" value="https://api.twitter.com/1.1/statuses/show.json" />
<add key="Twitter.ApiKey" value="" />
<add key="Twitter.ApiSecret" value="" />
<add key="Twitter.AccessToken" value="" />
<add key="Twitter.AccessTokenSecret" value="" />
<add key="Twitter.Username" value="shomra" />

This provides a simple and descriptive approach to breaking down all settings into manageable chunks.

Strongly-Type AppSettings

One thing that truly annoyed me when I first started .NET development is calling configuration values from within C# code. Not only did you have to write out (the very long-winded!) "ConfigurationManager.AppSettings["Twitter.Username"]", but also cast the value to a specific type.

Using the "ConfigurationManager.AppSettings[]" call is awful. It creates the potential for typo's that aren't caught by the compiler and there's no nice intellisense to make our coding easier.

I create a static configuration wrapper class to strongly-type all my config settings. The way I name my settings (as you can see from the "Common Naming Conventions" section) compliments the structure of my wrapper class.

public class Config
{
    public static class Social
    {
        #region Twitter

        public static class Twitter
        {
            public static string TwitterApiUrl
            {
                get
                {
                    return ConfigurationManager.AppSettings["Twitter.ApiUrl"];
                }
            }

            public static string TwitterUsername
            {
                get
                {
                    return ConfigurationManager.AppSettings["Twitter.Username"];
                }
            }

            public static string TwitterApiKey
            {
                get
                {
                    return ConfigurationManager.AppSettings["Twitter.ApiKey"];
                }
            }

            public static string TwitterApiSecret
            {
                get
                {
                    return ConfigurationManager.AppSettings["Twitter.ApiSecret"];
                }
            }

            public static string TwitterAccessToken
            {
                get
                {
                    return ConfigurationManager.AppSettings["Twitter.AccessToken"];
                }
            }

            public static string TwitterAccessTokenSecret
            {
                get
                {
                    return ConfigurationManager.AppSettings["Twitter.AccessTokenSecret"];
                }
            }
        }

        #endregion
    }
}

Admittingly, this can be quite time-consuming but when compared to the long-term benefits this has saved me a lot of headache.

What Else Could Be Done?

I've seen some developers use Applications Settings Architecture within the .NET framework when building Windows applications by creating a "Settings" file within the project.

The only downside I can see with this approach is that all config values declared within the "Settings" file will be compiled at runtime and you lose the flexibility to change configuration values externally. Here's a nice StackOverflow post that describes this method in greater detail.

If any of you readers have further suggestions or advice, I am all ears! :-)

The Future...

As Nick Dyer quite rightly pointed out to me yesterday on Google+, that our beloved web.config file will no longer form part of the applications we create in ASP.NET 5. We will have the freedom to create an applications configuration the way we want, simplifying the whole process.

As I understand, there will be support for creating configuration inputs that can be placed inside JSON, XML and INI files using the new IConfiguration and ConfigurationModel sources.

Sounds very promising.

Use Your Strings Wisely

When I was first learning to code, I was always told to use my strings in applications wisely. It's very easy to use strings without much thought. I think strings are forgiving compared to any other data type...too forgiving.

I was going to write a full blown blog post on the best way to use the string data type, but I found a good blog post that explains everything and I will use for reference when guiding new developers to my somewhat infinite wisdom. :-)

Zeeshan Umar, wrote a very interesting blog post a few years ago (but still relevant) that I only happened to find, explaining some new interesting ways on how to use strings efficiently. The following approach caught my eye:

string errorCode1 = "ec-1001";

if (String.Compare(errorCode1,"ec-1001",true) == 0)
{
   //Do something...
}

The String.Compare constantly stares me in the face via Visual Studio intellisense but never thought of using it. I will be using this approach if I need to check a string whilst using an conditional statement.

If you happen to find anymore interesting ways to efficiently use strings, please leave a comment.

Easy Way to Generate Strongly-Typed C# Classes From JSON

In nearly all my previous projects I've worked on required some kind of manipulation with reading some form of JSON data.

To allow better integration into my application, I always create strongly-typed classes that mirror the structure of the incoming data feed. This in turn allows me to serialize the JSON and select specific properties I require. But sometimes, its quite difficult and confusing to the exact class structure right, especially when the JSON contains lots of fields of data.

When I find this is the case, I use json2csharp.com. By simply pasting the contents of your JSON, the site does all the hard work and creates all the strongly-typed classes for you.

C# In Depth by Jon Skeet Review

C# In Depth Third EditionWhen working as a programmer, it's really easy to continue coding in the same manner you have done since you picked up a language and made your first program.

The saying: "Why fix it if it ain't broken?" comes to mind...

I for one sometimes fail to move with the times (unknowingly to me) and find new and better ways of coding. It's only on the off chance I get introduced to different approaches through my work colleague or whilst Googling for an answer to one of my coding queries.

After reading some rave reviews on C# In Depth, written by the one and only Stackoverflow god: Jon Skeet. I decided to part with my hard earned money and make a purchase.

C# In Depth is different from other programming books I've read on C#. In fact it's really good and don't let the title of the book deter you. The contents is ideal for novice and semi-experienced programmers.

Firstly, you start off by being shown code samples on how C# has evolved through its iterations (v1 - v4). In most cases I gave myself a gratifying pat on the back when I noticed the approaches I've taken in my own projects utilised practises and features of the current language. ;-)

Secondly, unlike some programming books I've read in the past, it's not intimidating to read at all. Jon Skeet really has a great way to talk about some concepts I find difficult to comprehend in a clear a meaningful way, so I could utilise these concepts within my current applications.

The only minor niggle I have is that there were a few places where I would have liked specific chapters to go into more detail. On the other hand, it gave me the opportunity to research the nitty-gritty details for myself.

Since I purchased this book, I found myself referencing it many times and appreciating what C# has to offer along with it misconstrued and underused features.

All in all, the author truly has a gift in clearly demonstrating his understanding on the subject with finesse and if I am able to comprehend even one-tenth of his knowledge, I will be a happy man.

Generate Google Sitemap From A List of Url’s In A Text File

I had around 2000 webpage URL’s listed in a text file that needed to be generated into a simple Google sitemap.

I decided to create a quick Google Sitemap generator console application fit for purpose. The program iterates through each line of a text file and parses it to a XmlTextWriter to create the required XML format.

Feel free to copy and make modifications to the code below.

Code:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.IO;
using System.Xml;

namespace GoogleSitemapGenerator
{
    class Program
    {
        static void Main(string[] args)
        {
            string textFileLocation = String.Empty;

            if (args != null && args.Length > 0)
            {
                textFileLocation = args[0];
            }

            if (!String.IsNullOrEmpty(textFileLocation))
            {
                string fullSitemapPath = String.Format("{0}sitemap.xml", GetCurrentFileDirectory(textFileLocation));

                //Read text file
                StreamReader sr = File.OpenText(textFileLocation);

                using (XmlTextWriter xmlWriter = new XmlTextWriter(fullSitemapPath, Encoding.UTF8))
                {
                    xmlWriter.WriteStartDocument();
                    xmlWriter.WriteStartElement("urlset");
                    xmlWriter.WriteAttributeString("xmlns", "http://www.sitemaps.org/schemas/sitemap/0.9");

                    while (!sr.EndOfStream)
                    {
                        string currentLine = sr.ReadLine();

                        if (!String.IsNullOrEmpty(currentLine))
                        {
                            xmlWriter.WriteStartElement("url");
                            xmlWriter.WriteElementString("loc", currentLine);
                            xmlWriter.WriteElementString("lastmod", DateTime.Now.ToString("yyyy-MM-dd"));
                            //xmlWriter.WriteElementString("changefreq", "weekly");
                            //xmlWriter.WriteElementString("priority", "1.0");

                            xmlWriter.WriteEndElement();
                        }
                    }

                    xmlWriter.WriteEndElement();
                    xmlWriter.WriteEndDocument();
                    xmlWriter.Flush();

                    if (File.Exists(fullSitemapPath))
                        Console.Write("Sitemap successfully created at: {0}", fullSitemapPath);
                    else
                        Console.Write("Sitemap has not been generated. Please check your text file for any problems.");

                }
            }
            else
            {
                Console.Write("Please enter the full path to where the text file is situated.");
            }
        }

        static string GetCurrentFileDirectory(string path)
        {
            string[] pathArr = path.Split('\\');

            string newPath = String.Empty;

            for (int i = 0; i < pathArr.Length - 1; i++)
            {
                newPath += pathArr[i] + "\\";
            }

            return newPath;
        }
    }
}

I will be uploading a the console application project including the executable shortly.

SyndicationFeed.Load - The easy way to read an RSS feed

I always found writing code to read an RSS feed within my .NET application very time-consuming and long-winded. My RSS code was always a combination of using WebRequest, WebResponse, Stream, XmlDocument, XmlNodeList and XmlNode. That’s a lot of classes just to read an RSS feed.

Yesterday, I stumbled on an interesting piece of code on my favourite programming site StackOverflow.com, where someone asked how to parse an RSS feed in ASP.NET. The answer was surprisingly simple. RSS feeds can now be consumed using the System.ServiceModel.Syndication namespace in .NET 3.5 SP1. All you need is two lines of code:

var reader = XmlReader.Create("http://mysite.com/feeds/serializedFeed.xml");
var feed = SyndicationFeed.Load(reader);

Here’s a full example on how we can iterate through through the SyndicationFeed class:

public static List<BlogPost> Get(string rssFeedUrl)
{
    var reader = XmlReader.Create(rssFeedUrl);
    var feed = SyndicationFeed.Load(reader);

    List<BlogPost> postList = new List<BlogPost>();

    //Loop through all items in the SyndicationFeed
    foreach (var i in feed.Items)
    {
        BlogPost bp = new BlogPost();
        bp.Title = i.Title.Text;
        bp.Body = i.Summary.Text;
        bp.Url = i.Links[0].Uri.OriginalString;
        postList.Add(bp);
    }

    return postList;
}

That’s too simple, especially when compared to the 70 lines of code I normally use to do the exact same thing.

DebuggerHidden Attribute and Other Cool Debugging

Debugging a page that uses many methods from other classes can become a right pain in the neck. I find myself accidentally stepping into a method that I don’t need to debug or wanting to output specific values from my methods straight away. Being the cowboy (correction, agile) developer that I am, one of my code boffins at work showed me a two cool ways to meet my debugging needs:

1) DubuggerHidden Attribute

Using the DubuggerHidden attribute tells the Visual Studio debugger that the method is hidden from the debugging process. The simple example below, shows the DebuggerHidden attribute in use:

protected void btnDoSomething_Click(object sender, EventArgs e)
{    
    //Output random number to Textbox
    txtOutput.Text = GetNumber(1, 10).ToString();
} 
 
[DebuggerHidden]
int GetNumber(int min, int max)
{    
    System.Random random = new Random();
    return random.Next(min, max);
}

2) DebuggerDisplay Attrubute

The DebuggerDisplay attribute allows us to output variable values from a class or method to be displayed in our Visual Studio debugger. The attribute includes a single argument that is supplied as a string. The string can contain references to fields, properties and methods in the class so that the actual values from an object may be included in its description.

[DebuggerDisplay("a={a}, b={b}, ValueTotal={CalculateValues()}")]
public class AddValues
{    
    public int a { get; set; }
    public int b { get; set; }
    
    public int CalculateValues()
    {
        return a + b;
    }
}

You also have the ability to include simple expressions, like so:

[DebuggerDisplay("a={a}, b={b}, ValueTotal={CalculateValues() * 2}")]

In addition to the “DebuggerHidden” and DebuggerDisplay attributes, .NET contains some very useful attributes that modify the behaviour of a debugger. For me, they weren’t as interesting as the two debugger attributes I listed above. :-)