Fetch more than 5000 records in one go…paging cookie way…but there’s a catch….

Hi Folks,

Hope some day or the other, every CRM Consultant will face this issue where they need to retrieve more than 5000 records in one go.

As every one in the internet suggests, paging cookie is the easiest way to achieve the same.

However here there is a catch…if you will use distinct=”false” no-lock=”true” along with your fetch query, you will never achieve the expected result and the timeout error keeps on popping up.

Pretty surely Microsoft have had documented this one, but I couldn’t find any relevant Microsoft article stating this. So thought of sharing this tip over my blog so that some day it will help some one. Here is the complete solution…

//Call the method to use Paging cookie and fetch multiple records in one go....
EntityCollection fetchDonationTransactionRecords = RetrieveAll(service, fetch);

/// <summary>
        /// RetrieveAll
        /// </summary>
        /// <param name="service"></param>
        /// <param name="fetchXml"></param>
        /// <returns></returns>
        public static EntityCollection RetrieveAll(this IOrganizationService service, string fetchXml)
        {
            EntityCollection finalCollection = new EntityCollection();
            int fetchCount = 5000;
            // Initialize the page number.
            int pageNumber = 1;
            // Specify the current paging cookie. For retrieving the first page, 
            // pagingCookie should be null.
            string pagingCookie = null;

            while (true)
            {
                // Build fetchXml string with the placeholders.
                string xml = CreateXml(fetchXml, pagingCookie, pageNumber, fetchCount);
                EntityCollection recordsCollection = new EntityCollection();
                FetchExpression expression = new FetchExpression(xml);
                recordsCollection = service.RetrieveMultiple(expression);

                // * Build up results here *

                // Check for morerecords, if it returns 1.
                if (recordsCollection.MoreRecords)
                {
                    // Increment the page number to retrieve the next page.
                    pageNumber++;
                    pagingCookie = recordsCollection.PagingCookie;

                    foreach (var record in recordsCollection.Entities)
                    {
                        finalCollection.Entities.Add(record);
                    }
                }
                else
                {
                    // If no more records in the result nodes, exit the loop. 
                    if (finalCollection.Entities.Count >= 5000)
                    {
                        foreach (var record in recordsCollection.Entities)
                        {
                            finalCollection.Entities.Add(record);
                        }
                    }
                    else
                    {
                        finalCollection = recordsCollection;
                    }
                    break;
                }
            }
            return finalCollection;
        }

Hope this helps…that’s it for today…be safe & happy CRM’ing….

Cheers,

PMDY