Large Data and JSONP - Dealing with IE's Cross Domain Headaches

The Problem:

I created dynamic form generating application which sends form data (json object via JQuery ajax call) to an endpoint which then stores data to a database.  I’m using a third party service which appends more data to the object (company data by email address/IP address) which makes the object HUGE!  Posting the data to the endpoint works perfectly (Chrome and Firefox ), but IE, not so much.  

Here’s the issues:

• The form generator app is used on cross-domain sites, which IE isn’t too happy about.  In other words, No Posting!
• JSONP works with GET requests, but the 2083 char limit in IE chops the request, resulting in No Transport or json call fail. Json = Error: jQuery was not called

The Idea:

What if I did the following:

• Create a new endpoint called “ThanksIE”
• Chop the json object every 1500 characters and sent multiple jsonp requests to  new endpoint. Each request will also have a similar guid, and index parameter to keep order, and a parameter called “endofjson” to know when the last of the json has been reached
• The endpoint code will take all requests and save them to a staging table
• When the endpoint receives a request with endofjson=true, it will query the database for all records with the matching guid, order by index, concatenate the strings together by order to a string called “completeJson”
• Send the “completeJson” to the other endpoint (the one that Chrome and Firefox uses)
• Save the complete json to a “sent” database table for tracking.

The Bad:

•  Multiple requests to send a single form isn’t the most efficient, but according to W3Schools, 7.8% use IE (2015), so the servers probably won’t get too bogged down.
• Sessions would have been better than databases, but the multiple requests per session caused a bigger headache.

The Good:

• I can keep a record of every request sent via IE for troubleshooting purposes
• Staging DB records will show any failures
• This solution works in IE 7,8,9,10

Shut up and give me the Code:

On the JS side:

• Get the length of the serialized json and the guid (using Cradent’s cuid method)
• Create the array for the json pieces
• Slice the json object every 1500 characters and push each piece to the array
• Run a recursive method for each item in the array

var totalChars = strJson.length;

var arrRequest = [];

var guid = cuid();

while(strJson)

{

        arrRequest.push(strJson.slice(0, 1500));

        strJson = strJson.slice(maxlength);

}

function sendIEData(i, isEnd) {

        $.ajax({

                        url: apiURL + '/Applications/FormGenerator/ThanksIE,

                        data: {'cuid' : guid, 'payload' : arrRequest[i], ' endofjson': isEnd, 'index': i++, 'totalChars' : totalChars},

                        type: 'GET',

                        dataType: 'jsonp',

                        success: function(data) {

                                        if (i == len - 1) {

                                                        sendIEData(i,true);

                                                        //show confirmation page

ShowThankYou();

                                        } else if (i < len) {

                                                        sendIEData(i,false);

                                        } else {

                                                        data.data && ShowThankYou();

                                        }                                                             

                                                       

                        },

                        error: function(jqXhr, textStatus, errorThrown) {

                                        alert("oops! " + errorThrown);

                        }

        });

}

sendIEData(0,arrRequest.length == 1);         

On the Server Side (C#):

• Get the request and save it to a staging database if endOfJson != true
•  If endOfJson == true, then select all with matching guid, order them by index
• Concatenate all strings by order of index, and attach the very last request (will be complete json)
• If it’s the complete Json object, call the method that Chrome and Firefox does so well.

Logic:

var cuid = ctx.ReadParam("cuid").ToString();

var payload = ctx.ReadParam("payload").ToString();

var eof = ctx.ReadParam("endofjson'").ToString();

var index = ctx.ReadParam("index").ToString();

var charCount = ctx.ReadParam("totalChars").ToString();

int iCharCount = -1;

int.TryParse(charCount, out iCharCount);

Dictionary<string, string> results;

if (eof.ToLower() == "false")

    results = SavePartialPayload(cuid, payload, eof, index);

else

    results = ConcatAndSend(cuid, payload, eof, index, iCharCount);

Worker Methods:

public Dictionary<string, string> SavePartialPayload(string cuid, string payload, string eof, string index)

{

    Data d = new Data();

    bool ret = d.SavePartialPayload(cuid, payload, eof, index);

    Dictionary<string, string> results = new Dictionary<string,string>();

    results.Add("results", ret.ToString());

    return results;

}

public Dictionary<string, string> ConcatAndSend(string cuid, string payload, string eof, string index, int charCount)

{

    Data d = new Data();

    StringBuilder completeJson = new StringBuilder();

    Dictionary<string, string> results = new Dictionary<string, string>();

    List<Partial> partials = d.SelectPartialByCuid(cuid);

    foreach (Partial p in partials){

completeJson.Append(p.payload);

   }

    completeJson.Append(payload);

    if (charCount == completeJson.Length)

    {

results = ProcessFGRequest(completeJson.ToString());

if (!results.ContainsKey("error"))

    d.DeletePartials(cuid, completeJson.ToString(), true, null);

    }

    else

results.Add("error", "character count do not match: Should be " + charCount.ToString() + ", but is " + completeJson.Length);

    return results;

}

Hope this helps somebody with the same problems as I did.  Not the easiest method, but I it’s working perfectly so far

Shout outs:  

Clark Inada and Corey Hadden