Pagination

Pagination allows you to control the result set that comes back from Cloud CMS for any operations that produce a list of items. It is called pagination because it gives you a way to organize your record set into pages and then get those pages back, one at a time.

Pagination gives you a way to adjust the result set before it is sent over the wire (and back to you) so that you can have smaller "paged" payloads. Pagination is useful because, most of the time, unless the result set is very small, you're not going to be interested in dealing with all of the items at once.

For example, with pagination, you can do things like query for all of the users in a domain. You might be interested in presenting them on the screen. Let's imagine there are 10,000 users. Well, chances are you probably don't want to show 10,000 users on the screen all at once. Instead, maybe you'd like to only show 50 at a time. Pagination lets you tell Cloud CMS to hand back 50 starting at index 0. Or jump ahead to index 9017 and give back the next 50 (i.e. users 9017 through 9066). That way, you're only getting 50 users at a time. However, you can get back a different 50 each time, as you web application user clicks forward and back.

Fundamentally, pagination lets you adjust two important properties:

skip - how many records you'd like to skip ahead in the total result set before handing back.
limit - how many records in total you'd like to have sent over the wire.

Suppose you ran a query and wanted to retrieve the first 10 items. You might specify:

skip = 0
limit = 10

You could then later retrieve the next 10 items by specifying:

skip = 10
limit = 10

The record set response from Cloud CMS contains information that you can use to determine the total number of results as well as offset information. For example, a response might come back that essentially looks something like this:

{
    "rows": [{
        "_doc": "d0977d2194dfcb8eec61",
        ...
    }, {
        "_doc": "d0977d2194dfcb8eec62",
        ...
    }],
    "size": 10,
    "total_rows": 45,
    "offset": 0
}

Where:

size is the number of records returned (i.e. the size of the rows array)
total_rows is the total number of matching results in the query
offset is the index into the results where things were handed back (i.e. the skip you passed in)

For more information on how this works, please read up on the Collections Envelope in the API.

Sorting

Pagination also lets you define how the record set should be sorted before it is handed back. The sorting is taken into account during query or lookup time so that the actual cursor into the underlying database is used to optimize performance.

You can sort on zero or more fields. For each field, you can choose to sort in ascending (1) or descending (-1) order. You specify this by passing a JSON object that defines this for any and all fields.

For example, the following could be used to sort users first by their name and second by a nested custom property:

{
    "name": 1, 
    "company.name": -1 
}

By default, sorting is performed using a fast ASCII comparison that is not sensitive to locale. Each value being compared is essentially treated as a string and the internal ASCII values for each character are compared. As such, the default sort behavior is case-sensitive and not particular to any given locale.

For many cases, this will provide you with what you want. However, if you require locale-specific treatment of strings and numeric values, please read on.

Sorting Options

This section describes features that are coming in 4.0

Starting with Cloud CMS version 4.0 and beyond, advanced sorting options are available that you can use to finely control locale-specific sort order and sort behavior.

These options are:

locale - (required) the locale code to assume when dealing with strings and numbers on any sorted fields
numericOrdering - true or false - whether to treat numbers in strings as their numeric values (instead of strings)
caseFirst - either upper or lower indicating that you wish for case-sensitive sorting with preference to either uppercase or lowercase characters.

Written as JSON, these options may look like:

{
    "locale": "en_US",
    "numeringOrdering": true,
    "caseFirst": "lower"
}

The only required option is locale. If only locale is specified, the sorting will be case-insensitive and carried out according to the expectations of that locale.

Locale

These advanced sorting options fundamentally require that you tell Cloud CMS the locale to assume for any data that it needs to sort on. This is necessary since Cloud CMS allows you to store content in many locales all at once. For example, two nodes stored in Cloud CMS could be in two entirely different locales but may have the same content type and same properties. Or you might have a single node with multiple properties, each with content of a different locale.

By telling Cloud CMS the locale to assume during the sort, Cloud CMS can apply some locale-specific logic to the sort. This is especially useful when working with character sets (such as Swedish or French) where accented characters are present and those characters may be required to be sorted in an expected order.

For example, in French, we might have the following words:

cote, coté, côte, côté

If sorted descending using a normal ASCII approach (i.e. not using Options), we'd get the default order. This order would be:

côté, côte, coté, cote

If we tell Cloud CMS to be mindful of the French locale (fr), we will get back a different order. This is the expected order of results taking into consideration the accents of the French language.

côté, coté, côte, cote

Some languages require words to be ordered on the secondary level according to the last accent difference, as opposed to the first accent difference. Canadian French happens to be one of these cases. If we tell Cloud CMS to sort results for Canadian French (fr_CA), we'll get back a different result as well:

côté, côte, coté, cote

You can pass any locale you wish using the locale attribute. In most cases, the locale will be stripped down to just its language code. However, in some cases, there are country and language code combinations that have specific sorting rules (such as with Canadian French above). Cloud CMS will make a best effort to determine the locale that is applicable based on what you pass in.

For a full list of supported locales, see: https://docs.mongodb.com/manual/reference/collation-locales-defaults/#supported-languages-and-locales

Cloud CMS also supports variants as noted in the link above. Where applicable, you can specify a variant of a locale by using the following structure:

<locale code>@collation=<variant>

For example, to sort in Chinese using the pinyin variant, you could set locale to zh@collation=pinyin.

Once you've specified the locale, there are a few additional optimizations you can provide as listed below.

Numeric Ordering

You may opt to turn on Numeric Ordering in order to have any numbers represented in your sorted fields treated as numerals instead of strings.

For example, you might have the following data:

1 James St
5 James St
10 James St
2 James St
20 James St
50 James St

If you sorted this ascending using the normal ASCII approach, you'd get back a result set that treats each element as a string. The numbers inside the string are not taken into consideration and you'd get back something like this:

1 James St
10 James St
2 James St
20 James St
5 James St
50 James St

However, if you tell Cloud CMS that the locale is en (English) and enable numericOrdering (set to true), you will get back the following results:

1 James St
2 James St
5 James St
10 James St
20 James St
50 James St

Which is what any normal, every day human being in this sector of the milky way galaxy would expect when sorting the aforementioned data. The numericOrdering option makes it so that the numbers inside of the string are treated individually and introduces a numeric comparison between values.

Case First

You may opt to turn on Case First behaviors in order to adjust the sort results for cases where there are collisions on case between strings.

For example, suppose you have the following:

PIRATES OF THE CARIBBEAN
pirates of the caribbean
Pirates of the Caribbean
Thunder Mountain Railroad
thunder mountain railroad
THUNDER MOUNTAIN RAILROAD
HAUNTED MANSION
haunted mansion
Haunted Mansion

If you sorted this ascending using the normal ASCII approach, you'd get back this:

haunted mansion
Haunted Mansion
HAUNTED MANSION
pirates of the caribbean
Pirates of the Caribbean
PIRATES OF THE CARIBBEAN
thunder mountain railroad
Thunder Mountain Railroad
THUNDER MOUNTAIN RAILROAD

If you sort ascending but set caseFirst to upper, you would get back the following:

HAUNTED MANSION
Haunted Mansion
haunted mansion
PIRATES OF THE CARIBBEAN
Pirates of the Caribbean
pirates of the caribbean
THUNDER MOUNTAIN RAILROAD
Thunder Mountain Railroad
thunder mountain railroad

If you sort ascending but set caseFirst to lower, you would get back the following:

haunted mansion
Haunted Mansion
HAUNTED MANSION
pirates of the caribbean
Pirates of the Caribbean
PIRATES OF THE CARIBBEAN
thunder mountain railroad
Thunder Mountain Railroad
THUNDER MOUNTAIN RAILROAD

Note that, in this case, the default ASCII and lower sorts are the same. This will be the case for cases where the character keycodes (ASCII) are already naturally sorted via keycode.

For non-English languages that have accented characters and larger character sets, the expected sort orders will differ. Be sure to use lower in this case to get proper lowercase sorting for the desired locale.

REST API

For methods in the REST API that produce lists of items, you can pass pagination information via request parameters.

For example, the following would hand back a list of 10 principals from a domain starting at index 5 in the result set. There may 1,000 users overall, but you'd only get back 10 (numbers 5 through 14).

GET http://api.cloudcms.com/domains/domainId/principals?skip=5&limit=10

We can also pass sorting information along (as a serialized JSON object):

GET http://api.cloudcms.com/domains/domainId/principals?skip=5&limit=10&sort={"name":1,"company.name":-1}

We can also pass pagination options along (as a serialized JSON object):

GET http://api.cloudcms.com/domains/domainId/principals?skip=5&limit=10&options={"locales": "fr_FR"}
GET http://api.cloudcms.com/domains/domainId/principals?skip=5&limit=10&options={"locales": "fr_FR", "caseFirst": "lower"}
GET http://api.cloudcms.com/domains/domainId/principals?skip=5&limit=10&options={"locales": "fr_FR", "caseFirst": "lower", "numericOrdering": true}

Driver methods

You can also pass pagination information via driver methods. Most drivers support pagination as an optional argument into methods that produce lists of objects.

Here we retrieve the same list of 10 users from a domain starting from index 5.

var domain = ...;

domain.listUsers({
    "skip": 5,
    "limit": 10
});

Here we retrieve the same list but we sort by name and company.name:

var domain = ...;

domain.listUsers({
    "skip": 5,
    "limit": 10,
    "sort": {
        "name": 1,
        "company.name": -1
    }
});

We can also perform custom queries and paginate the results:

var domain = ...;

domain.queryUsers({
    "company.name": "Assan Motors"
},{
    "skip": 5,
    "limit": 10,
    "sort": {
        "name": 1,
        "company.name": -1
    }
});