whitespace COMPANY whitespace SERVICES whitespace PRODUCTS whitespace PURCHASE whitespace SUPPORT whitespace CONTACTS whitespace Home whitespace Contact Us whitespace Site Map whitespace
whitespace
OVERVIEW
whitespace
TECHNOLOGIES
whitespace
CONSULTING
whitespace
DESKTOP
whitespace
WEB
whitespace
MOBILE
whitespace
DYNAMICS CRM
whitespace
whitespace

CDN integration for legacy ASP.NET websites

By Oleksandr Kucherenko, 15 May 2013

Introduction

This article is based on my personal experience and also some articles found on web (references can be found at the end of the page). I've tried my best to make code short and easy to understand. From the very beginning I would like to say that for each and every particular website, you should test and adjust the solution. Code is very close to being universal and it works perfectly on my developer environment, test server and deployment server. And those 6 different environments, in which solution is tested (IIS6, IIS7, standalone website, virtually attached folder), helps to catch a lot of issues.

CDN77 Content Delivery Network

The problem

Our website server is located in Ukraine. Internet service providers that we have are not by all means the best one's in the world and they simply can't provide fast connection for every territory in the world. And even if they can, the cost of such service won't be cheap at all.

Our main sales territories are USA and Europe. And our ISPs and server are doing pretty very well for European territory. For USA we often have feedback about slow response from our web portal. There are several factors that makes such issue well predictable:

  • ISP does not provide unlimited bandwidth internet channel. We have very good download speed, but not UPLOAD. Upload bandwidth is very important, it's actually used for providing website content.
  • Physically USA territories are far away from us. Such distance often requires several additional milliseconds.
  • Web pages itself are often pointing to a great number of additional resources: javascripts, stylesheets, images. Each webpage may produce more than 20 HTTP requests from user internet browser to our web server.

Simply multiply all these factors and you will see that on several territories your website is not as fast as you think it is. Your webserver CPU, memory and bandwidth maybe not sufficient enough to solve this issue.

But what exactly can solve all this problems? The answer is simple - CDN (Content Delivery Network).

CDN benefits:

  • CDN acts like a huge external CACHE. It can cache all the elements of the website if needed.
  • CDN network has servers on each territory. These servers are much more powerful in compare to the averafe server, which small company can afford.
  • CDN servers are connected to much more wider internet channels. They can deliver your content in much more faster way.
  • CDN servers are 99.9% stable. No power failures, no internet channel problems.

The only thing that can hold you from considering to use CDN is the cost per 1GB of traffic. In our case we choose cdn77.com which provides affordable prices, allows to test their service during 14 days. And what is even more important to us - they have servers on ex-USSR territories.

Steps to Implement

First of all we have to identify which elements of the website can be delegated to CDN and which should stay. Actually, the rule sounds easy: "static content can be delivered via CDN". And for the first iteration strong candidates for CDN are:

  • Images
  • Stylesheets
  • JavaScripts

What else? How about downloadable content like software packages that our company produces? Are all CDNs suitable for that?
Quick search shows that not all CDNs can be used for that. In addition Video and Audio - they are always a subject for special prices. We are lucky - video and audion, this is not our case.

In our particular case, we follow the idea that customer that decides to download our software can wait a second or more. The biggest package that we have is 30Mb in size. Right now we have decided to leave them as is. If in next 3-4 months CDNs will show that specific items are requested and are slow in delivery, than we will apply CDN on ZIP and PDF files.

JavaScript

In our days website without JavaScript is something strange. Almost everyone uses jQuery or common scripts. They simplify life greatky and improve usability of the web site.

For a website developer such things produce set of troubles:

  • Scripts are often updated. New versions released, bugs fixed, new components added. And as a result you have to keep a copy of script on own webserver to prevent any troubles.
  • Scripts may conflict with each other. This happens often. Quick example: jQuery UI and jQuery Tools scripts has a conflict on 'accordion' control.

Solution, found by us, was extremely easy and what is more important FREE. We've started to use public script CDNs.
I'm more than sure that you know about them - Google Hosted Libraries, Microsoft AJAX CDN (Azure Hosted Libraries) etc.
Several benefits that we've gained:

  • We use http://cdn.jsdelivr.net/ which hosts several versions of the required scripts. Support of the CDN network is very friendly and respond to our question in shortest period of time. Great service!
  • Usage of the public CDN produces less load to our internet channel and webserver. At least 10% of page size, in our case, were delegated to public CDN.

Stylesheets

Internal, external and embedded into HTML. There's only one rule: try to avoid usage of embedded styles, use more of external CSS. External CSSs are often a part of jQuery components and they are hosted by public CDNs as scripts themselves. Use this.

App_Themes

Applying the theme for a specific page on the website should not be a problem. But actually it becomes a problem when you start thinking about delegating images delivery to CDN. Stylesheets often do not contain any absolute URLs for downloading background images, they commonly use relative paths.

Issues that we've found and solved:

  • Relative to THEME folder path to background images.
  • Relative path with the use of ".." double dot notation (upper folder).
  • Loading of CSS files from several themes

  
    
    
      
    
  

      

Images

This is the biggest optimization thing for every website. On our pages 50-60% of content occupy graphics that demonstrate our solution in a nice and visual manner. Bad news for us is that large numbers of images greatly increase page "load time".

Several simple rules which I recommend to follow:

  • Think about each image as a SEO thing. Image should have ALT and TITLE attributes.
  • If possible, define for each image it's width and height. That will help internet browser render faster.
  • Avoid resizing of the images by web browser. If only thumbnails are shown to user, than loading of full image is a bad thing. Economy each byte. Use resized and prepared images for thumbnails.
  • Optimize all images by size. We've tried several tools: PNG Optimizer, SmushIt!
  • Where possible replace big PNG images with JPG version. But be very careful with transparency.
  • Use CSS sprites and background images.
  • You can use pre-fetch hints on page. That can help with loading of images from different domains.

Note: HTTP Module implementation that we show allows implementation of specific optimization module. Idea of such module is: producing of images on server in different format - for example: JPG with 65% quality. This is high quality for images, and end user will not see big difference. HTTP module will resolve each URL from HTML to corresponding file on server directory structure. If module detects existance of image in different graphics format, which is smaller by size, than module should replace old URL by a new one. For big web portals this is a huge optimization, which does not require web page content modification. Simple script can do all of the job in seconds. BTW we apply such technique on our several pages. File storage on server is cheap, traffic and "page load" speed - are things critical for success.

Combine All Togeather

To apply CDN we've use HTTP Module approach. It allows completely redefine server output with minimal efforts from developer or publisher side. In our case we've composed one class with several nested-classes, which applies CDN for us. Idea behind the module was simple: process generated HTML and CSS, and in runtime replace URLs in them. They should point to CDN server instead of our own.

Finally we've developed HTTP Module, based on code from ASP.NET community forum. Applied several significant modifications that make code more universal. Resolved several found issues, common for legacy websites.

In our case we've placed created module into App_Code website folder, for modern websites code can become a part of module (module for Orchard CMS) or part of assembly. Enabling of module can be done by merging two lines of XML into website web.config.
This all makes solution simple and transparent.

Testing

For testing we've used several nice tools. I would highly recommend to use them in day-to-day activities related to website content optimization:

  • LOADIMPACT - Stress Load our server
  • Just Ping - Detect how each territory resolves CDN server name to IP.
  • Apache JMeter - Load testing, bottlenecks search, etc.
  • Google PageSpeed Tools - helps to identify how to optimize web content
  • YSlow - Helps to optimize website content
  • Xenu - Desktop tool for website broken links search, and not only this.

We are still in the testing phase, but even after 2 days of CDN utilization we see great improvements. Google Analytics shows us that more and more potential customers haved visited our website and this is great!

Load Testing

Without CDN our server has hard times when more than 30 concurrent users try to view our pages. This produce load close to 2000 HTTP requests per second.
After applying CDN load on server become lower. The same 30 concurrent users now produce only 150 HTTP requests. All graphics, stylesheets, javascripts were delivered to the user internet browsers from CDN servers. Our server right now produce only GZIPed HTML as output on internet browser requests, and this in worst case is a 50% of total page size, for other pages - this is 10-25% of the total page size.

DNS Issues

During testing we've found interesting issue - DNS server returns for different territories different IPs of our web site. First of all this issue is only possible if more than one internet channel attached to your server. In our case we have infrastructure that has MAIN internet channel, BACKUP internet channel and additional BACKUP Gate Server configured on backup channel.

First of all you should always remember: DNS servers on all territories are configured in different ways. And for this there are millions of reasons: different DNS software in use, different hardware, different versions of the software, different configurations, etc.

In our case we've captured following issue: different DNS servers ignore priority of the DNS records and instead of getting the record with highest priority, they simply take the last record from DNS server.

Known Issues

Each solution has its own issues and solution provided by me also has several. During testing we've found them and decided to release solution with them. Cases that we capture are have little influence on our website, but in your case this can become a bigger problem. Good news - our solution shows the way of solving them. Bad news - resolving of such cases is not trivial and will require more complex implementation, but it is not as hard as someone can think.

Note: if you find that cases that I've described refer to your business, you can always hire us for implementing solution for you.

Case #1: Embedded CSS

Embedded CSS in HTML code cannot be captured by the provided solution.

Workaround: place CSS into external file. Combres solution/project can be used for optimization.

Case #2: Embedded jQuery calls or Custom JavaScript

Dynamically created DOM elements. It often happens when complex javascripts are used.

Workaround: use CSS for defining images. Use CSS sprites. If possible use jQuery scripts from CDN networks. Use external javascripts instead of embedded.

Conclusion

  • Applying of CDN for ASP.NET website is not so complex after all. We've spent only 2 days for this.
  • After applying several optimizations, suggested by tools: YSlow and Google PageSpeed - we got 10x time faster web pages.
  • Number of HTTP requests to our webserver has reduced greatly. At the beginning we had more than 66 requests per page, right now this is only 10-15 requests.
  • CDN network does not require any complex things. No uploading of graphics or content is needed on external servers.
  • Small DNS changes required, but even without them your website can works as supposed.
  • Hosting of the jQuery script and CSS is no more required on own webserver. Can be used several public CDNs, like: Google, Microosft Azure, Amazon, etc.
  • Page loading time decreased for all territories. In several cases we've got extremely good results - page load takes less than 1 second (server load was - 50 concurrent users).

P.S.

First of all I would like to say "THANK YOU!" to all those, who are composing great articles and answer to questions on forums. THANK YOU COMMUNITY! Yours articles greatly simplify life and allow others to develop faster and with great results. I hope my article will help others too.

Related Articles

#region Copyright ArtfulBits Inc. 2005 - 2013
//
//  Copyright ArtfulBits Inc. 2005 - 2013. All rights reserved.
//
//  Use of this code is subject to the terms of our license.
//  A copy of the current license can be obtained at any time by e-mailing
//  info@artfulbits.com. Re-distribution in any form is strictly
//  prohibited. Any infringement will be prosecuted under applicable laws. 
//
#endregion

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text.RegularExpressions;
using System.Web;
using System.Diagnostics;

/// 
public class CdnMappingModule : IHttpModule
{
  #region Http Module
  public CdnMappingModule()
  {
  }

  public void Dispose()
  {
  }

  public void Init( HttpApplication context )
  {
    context.BeginRequest += context_BeginRequest;
  }

  private void context_BeginRequest( object sender, EventArgs e )
  {
    HttpApplication context = ( HttpApplication )sender;

    string raw = context.Request.RawUrl.ToLowerInvariant();
    string cleanUrl = NoQuery( raw );

    if( cleanUrl.EndsWith( ".aspx" ) ||
        cleanUrl.EndsWith( ".htm" ) ||
        cleanUrl.EndsWith( ".html" ) ||
        cleanUrl.EndsWith( "/" ) )
    {
      /// extract HOST path, in several cases on web pages can be full URLs
      string url = NoQuery( HttpContext.Current.Request.Url.ToString() );
      string host = url.Replace( HttpContext.Current.Request.CurrentExecutionFilePath, "" );
      host = VirtualPathUtility.AppendTrailingSlash( host + HttpRuntime.AppDomainAppVirtualPath );
    
      var watcher = new StreamWatcher( context.Response.Filter, new HtmlReplacer( host ) );

      context.Response.Filter = watcher;
    }
    else if( cleanUrl.EndsWith( ".css" ) ) 
    {
      string theme = "";

      var match = Regex.Match( context.Request.RawUrl, @"app_themes/(.*/)(.*)\.css", 
        RegexOptions.IgnoreCase );
      
      /// captured THEME name  (actually this should be path to root dir of CSS file)
      if( match.Success )
        theme = "app_themes/" + match.Groups[ 1 ].Value;

      var watcher = new StreamWatcher( context.Response.Filter, new CssReplacer( theme ) );

      context.Response.Filter = watcher;
    }
  }
  #endregion

  #region Helper methods
  public static string NoQuery( string url ) 
  {
    int index = url.IndexOf( "?" );

    if( index >= 0 )
      return url.Substring( 0, index );

    return url;
  }
  #endregion

  }
  #endregion
}
      
#region Copyright ArtfulBits Inc. 2005 - 2013
//
//  Copyright ArtfulBits Inc. 2005 - 2013. All rights reserved.
//
//  Use of this code is subject to the terms of our license.
//  A copy of the current license can be obtained at any time by e-mailing
//  info@artfulbits.com. Re-distribution in any form is strictly
//  prohibited. Any infringement will be prosecuted under applicable laws. 
//
#endregion

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text.RegularExpressions;
using System.Web;
using System.Diagnostics;

#region Replacers
public interface IReplacer 
{
  void TransformString( ref string input );
}

public class CssReplacer : IReplacer 
{
  const string CDN = "http://527455326.r.cdn77.net/";

  private string _theme;

  public CssReplacer( string themeDir ) 
  {
    _theme = themeDir ?? "";
  }

  #region Implementation
  public void TransformString( ref string input )
  {
    string regExPatternForImgTags = @"[ \t:]url\(([^)]*)\)";

    string output = Regex.Replace( input, regExPatternForImgTags, ( match ) =>
    {
      string url = match.Groups[ 1 ].Value;

      string newUrl = url;

      if( !( url.Contains( "http://" ) || url.Contains( "https://" ) ) )
      {
        if( url.EndsWith( ".png" ) || url.EndsWith( ".jpg" ) || url.EndsWith( ".gif" ) )
        {
          // if CSS pointing on root images instead of theme images
          if( url.Contains( "../" ) )
          {
            newUrl = CDN + Regex.Replace( url, @"(\.\./)+", "" );
          }
          else
          {
            newUrl = CDN + _theme + url;
          }

          return match.Value.Replace( url, newUrl );
        }
      }

      return match.Value;
    } );

    // remove whitespaces
    output = Regex.Replace( output, @"[ \t]+", " " );

    input = output;
  }
  #endregion
}

public class HtmlReplacer : IReplacer 
{
  const string CDN = "http://527455326.r.cdn77.net/"; 
  private string _host;
    
  public HtmlReplacer( string host )
  {
    _host = host;
  }
    
  #region Implementation
  public void TransformString( ref string input )
  {
    /// this will match all the tags with src and href attribute, you may need to extend this. 
    string regExPatternForImgTags = @"[ \t]((src)|(href))=(?<o>(""|'))([^""']*)(\<o>)";

    string output = Regex.Replace( input, regExPatternForImgTags, ( match ) =>
    {
      /// extracted by RegExp groups, can be usefull for debug and URL modifications
      string attr = match.Groups[ 1 ].Value;
      string quote = match.Groups[ 4 ].Value;
      string url = match.Groups[ 5 ].Value;

      string newUrl = url.Replace( _host, "" );

      /// skip external URLs
      if( !( newUrl.Contains( "http://" ) || newUrl.Contains( "https://" ) ) )
      {
        string urlCheck = NoQuery( url );

        /// we are processing only several graphics types
        if( urlCheck.EndsWith( ".png" ) || 
            urlCheck.EndsWith( ".jpg" ) || 
            urlCheck.EndsWith( ".gif" ) || 
            urlCheck.EndsWith( ".ico" ) )
        {
          /// if CSS pointing on root image by it relative path
          if( url.Contains( "../" ) )
          {
            newUrl = CDN + Regex.Replace( newUrl, @"(\.\./)+", "" );
          }
          else
          {
            newUrl = CDN + newUrl;
          }

          return match.Value.Replace( url, newUrl );
        }
      }

      return match.Value;
    } );

    input = output;
  }
  #endregion
}
#endregion
      
#region Copyright ArtfulBits Inc. 2005 - 2013
//
//  Copyright ArtfulBits Inc. 2005 - 2013. All rights reserved.
//
//  Use of this code is subject to the terms of our license.
//  A copy of the current license can be obtained at any time by e-mailing
//  info@artfulbits.com. Re-distribution in any form is strictly
//  prohibited. Any infringement will be prosecuted under applicable laws. 
//
#endregion

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text.RegularExpressions;
using System.Web;
using System.Diagnostics;

#region Stream
public class StreamWatcher : Stream
{
  #region Memebers
  private Stream _sink;
  private long _position;
  private IReplacer _replacer;
  #endregion

  #region Properites

  public override bool CanRead
  {
    get { return true; }
  }

  public override bool CanSeek
  {
    get { return true; }
  }

  public override bool CanWrite
  {
    get { return true; }
  }

  public override void Flush()
  {
    _sink.Flush();
  }

  public override long Length
  {
    get { return 0; }
  }

  public override long Position
  {
    get { return _position; }
    set { _position = value; }
  }

  #endregion

  #region Constructor
  public StreamWatcher( Stream sink, IReplacer replacer )
  {
    if( null == replacer )
      throw new ArgumentNullException( "replacer" );

    _sink = sink;
    _replacer = replacer;
  }
  #endregion

  #region Methods

  public override int Read( byte[] buffer, int offset, int count )
  {
    return _sink.Read( buffer, offset, count );
  }

  public override long Seek( long offset, SeekOrigin origin )
  {
    return _sink.Seek( offset, origin );
  }

  public override void SetLength( long value )
  {
    _sink.SetLength( value );
  }

  public override void Close()
  {
    _sink.Close();
  }

  public override void Write( byte[] buffer, int offset, int count )
  {
    byte[] data = new byte[ count ];
    Buffer.BlockCopy( buffer, offset, data, 0, count );
    string html = System.Text.Encoding.Default.GetString( buffer );

    _replacer.TransformString( ref html );

    byte[] outdata = System.Text.Encoding.Default.GetBytes( html );
    _sink.Write( outdata, 0, outdata.GetLength( 0 ) );
  }
  #endregion
}
      




  

  
  
    
      
    
  
  
  
    
      
    
  

      
Company | Services | Practices | Technologies | Career | Contacts | Privacy
© 2005-2014 ArtfulBits. All rights reserved.