Dec 01, 2012

Fast Downloading with Parallel Requests using ASP.NET Web API

In this article, we will implement to download parts of file in parallel to get the complete file faster. It is very useful to download large file and saves a bunch of time. ASP.NET Web API supports byte range requests(available in latest nightly build) with ByteRangeStreamContent class. We will request the ranges of file parts in parallel and merge them back into a single file.

Web API:

First I would recommend to read Henrik F Nielsen’s post to understand HTTP Byte Range Support in ASP.NET Web API. We are going to implement for downloading a CSV file.


 public class DataController : ApiController
    {
        // Sample content used to demonstrate range requests     
        private static readonly byte[] _content = File.ReadAllBytes(HttpContext.Current.Server.MapPath("~/Content/airports.csv"));

        // Content type for our body
        private static readonly MediaTypeHeaderValue _mediaType = MediaTypeHeaderValue.Parse("text/csv");

        public HttpResponseMessage Get(bool IsLengthOnly)
        {
            //if only length is requested
            if (IsLengthOnly)
            {
                return Request.CreateResponse(HttpStatusCode.OK, _content.Length);
            }
            else
            {               
                MemoryStream memStream = new MemoryStream(_content);

                // Check to see if this is a range request (i.e. contains a Range header field)               
                if (Request.Headers.Range != null)
                {
                    try
                    {
                        HttpResponseMessage partialResponse = Request.CreateResponse(HttpStatusCode.PartialContent);
                        partialResponse.Content = new ByteRangeStreamContent(memStream, Request.Headers.Range, _mediaType);
                        return partialResponse;
                    }
                    catch (InvalidByteRangeException invalidByteRangeException)
                    {
                        return Request.CreateErrorResponse(invalidByteRangeException);
                    }
                }
                else
                {
                    // If it is not a range request we just send the whole thing as normal
                    HttpResponseMessage fullResponse = Request.CreateResponse(HttpStatusCode.OK);
                    fullResponse.Content = new StreamContent(memStream);
                    fullResponse.Content.Headers.ContentType = _mediaType;
                    return fullResponse;
                }
            }
        }
    }

First we need to get length of content so bool argument is used for this. if it is true then only content length is returned else content is returned based on whether the incoming request is a range request. If it is then we create a ByteRangeStreamContent and return that. Otherwise we create a StreamContent and return that.

Parallel Requests:

For simplicity, we are taking same app to consume web api. Add new MVC controller and View in the project.

Step 1: In view, First we get the length of the content


  var size;
        $.ajax({
            url: 'api/data?IsLengthOnly=true',
            async: false,
            success: function (data) {
                size = parseInt(data);
            }
        });

Step 2: To split content length in number of threads:


  var totalThreads = 4;
  var maxSize = parseInt(size / totalThreads, 10) + (size % totalThreads > 0 ? 1 : 0);

Step 3: implementing to request a particular range


		var ajaxRequests = [];
        var results = [];
        function reqToServer(reqNo) {
            var range = "bytes=" + (reqNo * maxSize) + "-" + ((reqNo + 1) * maxSize - 1);
            return $.ajax({
                url: 'api/data?IsLengthOnly=false',
                headers: { Range: range },
                success: function (data) {
                    results[reqNo] = data;
                }
            });
        }

        for (var i = 0; i < totalThreads; i++) {
            ajaxRequests.push(reqToServer(i));
        }

Step 4: combine the response of individual parallel requests and put in a hidden field(hdnData).


 var defer = $.when.apply($, ajaxRequests);
        defer.done(function (args) {
            var output = '';
            for (var i = 0; i < totalThreads; i++) {
                output = output + results[i];
            }
            $('#hdnData').val(output);
        });
http-parallel-range-requests

As in the above example, single download request takes ~6 sec while parallel requests take ~4 sec. It depends on many factors like DNS look up time, Server Bandwidth...etc.

Downlodify:

Now we have the content in the hidden field but user should be able to save as a file so we use Downloadify. It is a javascript and flash based library that allows you to download content in a file without server interaction. Download it, add files in the project and Reference to swfobject.js and downloadify.min.js.


 Downloadify.create('btnDownloadify', {
            filename: function () {
                return 'sample.csv';
            },
            data: function () {
                return document.getElementById('hdnData').value;
            },
            onComplete: function () { },
            onCancel: function () { },
            onError: function () { alert('You must put something in the File Contents or there will be nothing to save!'); },
            swf: '/Content/downloadify.swf',
            downloadImage: '/Content/download.png',
            width: 100,
            height: 30,
            transparent: true,
            append: false
        });

btnDownloadify structure is :


 <p id="btnDownloadify">
        You must have Flash 10 installed to download this file.
 </p>

You can download source code from GitHub.

Hope, you enjoy it.