From the Rough Cut
From the Rough Cut
From the Rough Cut
Paging
The idea behind “paging” is that when you’ve got a large data set, you’re going to want to retrieve it in manageable chunks. The client has control of that via query options using $count, $skip and $top. For example, we can count the number of TinyLinks in our data:
http://.../TinyLinkService.svc/TinyLinks/$count
If the result is 100, getting them all at once is probably fine; if the result is 10,000, probably not. When you don’t want to consume all of a set of data at once, you can use $skip and $top. For example, you can get 300 links in three gulps using the following:
http://.../TinyLinkService.svc/TinyLinks?$top=100 http://.../TinyLinkService.svc/TinyLinks?$skip=100&$top=100 http://.../TinyLinkService.svc/TinyLinks?$skip=200&$top=100
Likewise, if you’re using LINQ, you can do the same thing:
int count = service.TinyLinks.Count(); var page1Links = service.TinyLinks.Take(100); var page2Links = service.TinyLinks.Skip(100).Take(100); var page3Links = service.TinyLinks.Skip(200).Take(100);
This code generates the same URLs as shown previously, but of course, once you have the count, it’s better to write the loop, e.g.
int chunk = 100;
int count = service.TinyLinks.Count();
for (int skip = 0; skip < count; skip += chunk) {
var pageLinks = service.TinyLinks.Skip(skip).Take(chunk);
}
This all works well in a perfect world with polite clients, but even in the real-world with greedy clients, a Data Services endpoint can set the maximum page size no matter how much data the client asks for:
public class TinyLinkService : DataService<TinyLinkContainer> {
public static void InitializeService(DataServiceConfiguration config) {
config.SetEntitySetAccessRule("TinyLinks", EntitySetRights.All);
config.SetEntitySetAccessRule("TinyUsers", EntitySetRights.All);
config.SetEntitySetPageSize("TinyLinks", 100);
config.SetEntitySetPageSize("TinyUsers", 100);
config.DataServiceBehavior.MaxProtocolVersion = DataServiceProtocolVersion.V2;
}
}
Notice the call to SetEntitySetPageSize: this lets the server decide the maximum of how many entities of each type you’d like to return. With this in place, a query will contain a continuation query at the bottom of each result:
<!-- http://localhost:1676/TinyLinkService.svc/TinyUsers?$top=1000 -->
<feed ...>
...
<entry>...</entry>
...
<link rel="next"
href="http://localhost:1676/TinyLinkService.svc/TinyUsers?$top=900&$skiptoken=100"
/>
</feed>
You can check for the presence of the continuation query by wrapping your LINQ statement in a DataServiceCollection object like so:
var links = new DataServiceCollection<TinyLink>(
from link in service.TinyLinks.Expand("User")
where link.IsActive
orderby link.Url descending
select link);
// Get the links by page
while (links.Continuation != null) {
links.Load(service.Execute<TinyLink>(links.Continuation));
}
foreach (var link in links) {
Console.WriteLine("Url= {0}, IsActive= {1}, User.Name= {2}",
link.Url, link.IsActive, link.User.Name);
}
In this code, we’re doing the least efficient thing – in the presence of a non-null Continuation property on the DataServiceCollection instance (which indicates the possibility of another page of data), we call the Load method immediately, passing in the result of calling the Execute method on the Continuation data. The Execute method is a low-level method that will execute any OData query and we’ll see it again before this chapter is done.
Executing the continuation query exposed from the DataServiceCollection[20] will populate the results with more results and executing the loop until there is no more continuation query will guarantee that you have all the results, but this isn’t the best way to go – instead, in the presence of paging, you should attempt to only request the results that you actually need.
Notice that once we have whatever pages of results that make us happy that using the collection works just as before, i.e. we can pull out each entry as an object, inspect its values, make updates, etc.
Paging is one way that the server has to influence the service endpoint behavior on a per-type basis. If you like, it can also do so on a per-call basis with interceptors.
Footnote
[20] If you’d like to manage paging on the client-side without the use of the DataServiceCollection, you can do so with QueryOperationResponse: http://blogs.msdn.com/b/astoriateam/archive/2010/02/02/server-paging-in-data-services.aspx (http://tinysells.com/147).