Batch Posting Asynchronously

On the tail end of our WPF client project, we started getting a ‘Task is cancelled’ exception from a method that posts JSON data to a REST API. I know that this exception is just a pointer to the actual problem and I’m confident that it’s not a code issue because the last modification made to the method was 4 weeks ago.

It turns out, since we started testing rigorously, we are sending large amount of JSON to the API. Our knee-jerk reaction was to set  the amount of JSON we can post to the maximum value (of course):

protected virtual string Serialize(object obj, bool maxResult) {
            var javaScriptSerializer            = new JavaScriptSerializer();
            javaScriptSerializer.MaxJsonLength  = Int32.MaxValue;
            return javaScriptSerializer.Serialize(obj);
}

We’ve mistakenly thought that this solved the problem. Few days later, we started getting the same error again so we decided to take a long-term approach on the issue. We cannot hope that there will be less data. We can, however, chop the data and send them by batch:

public virtual async Task PostBatch<T> (string path, IEnumerable<T> collection, SyncServiceContainer<T> container, int take) {
            int max               = (collection.Count() / take) + 1;
            var contents          = new List();
 
            for (int i = 0; i < max; i++) {
                var slicedCollection   = collection.Skip(take * i).Take(take).ToList();
                container.Data         = slicedCollection;
                contents.Add(SerializeContent(container));
            }
 
            foreach (var content in contents) {
                await Post(path, content);
            }
}

A few things can be said about the PostBatch method. First, the collection parameter is data that needs to be chopped. We used IEnumerable<T> so it can accept any type. Second, the container parameter is just an object that wraps the collection. That class has a property called Data which will contain the chopped collection. The API that receives the data should have an object parameter the same as SyncServiceContainer object.

This resolved our problem completely.