QueryReadStore breaking Grid scrolling

[haysmark]

See Maine's first comment, with the code sample, in: http://dojotoolkit.org/book/dojo-book-0-9-1-0/part-2-dijit-dojo-widget-library/advanced-editing-and-display/grid-1-0/sortin#comment-9112

If you create a QueryReadStore? that pages data from a server app, it tells Grid that there are only as many data items as are in the first page or two. Consequently, the virtual scrolling doesn't kick in because Grid thinks it has all of the items. QueryReadStore? needs to report more items so Grid can scroll more.

[jaredj]

The problem here is tha QueryReadStore? never reports the total matching size in addition to returning the subset requested. I know how to generally fix this, just need to allocate some time to do so.

[wolfram]

There is no problem here, please see the examples http://archive.dojotoolkit.org/nightly/dojotoolkit/dojox/data/demos/demo_QueryReadStore.html http://archive.dojotoolkit.org/nightly/dojotoolkit/dojox/data/demos/demo_QueryReadStore_sort.html http://archive.dojotoolkit.org/nightly/dojotoolkit/dojox/data/demos/demo_QueryReadStore_filter.html I made them to exactly show how to do this.

Of course its the server side that has to take care of returning the numRows field, in order for the grid to handle it! So I would not say this is a bug

[guest]

Connected to: Handle numRows sent by the server. Maybe the right question is that what should be done to the DojoData? model that it doesn't have customized for using it like this with QueryReadStore?.

[wolfram]

Please clarify: is there a question/bug?

What are the examples above missing?

To what I understood the question is what the server has to do. It just has to send the field "numRows" in order for the grid to know how to size the grid and when to page, etc. The examples show this too.

[guest]

Sorry for a confusing post. What I meant that the fix solves this issue nicely from the point of view of the QueryReadStore?. The demos prove this.

As a separate issue: The demos also illustrate that the DojoData? model is not usable this way out-of-the-box, but it still has to be customized. The question being: Can some of these changes in DojoDataSortable? be included in standard DojoData? in such a way that a) it doesn't break anything and b) would enable server side sorting out-of-the-box?

[wolfram]

Good point. I will investigate and try to suggest a patch if it is reasonable, since that sounds logical to me too. Thanks for the input!

[jaredj]

QueryReadStore? is the item with the issue here, not the model itself. I've built datastores that do serverside paging fine. The problem is QueryReadStore? is not reporting the total match size along with the subset of items. It needs the total match size reported from the server in order to be able to know how big to make the grid. effectively, the data object returned should supply:

totalMatched: <some number> as an attribute in its array, much like how you can pass identifer and such back.

If QueryReadStore? provided that, then the grid would work fine on sorting and paging.

I'll try to patch this tomorrow to note what needs to be changed.

[jaredj]

And ... it looks like QueryReadStore? does provide it. It's the numRows option of the data. As long as the server properly populates that value with the total data size, not just the size of the page of data, then it should properly scale and page in the grid.

[jaredj]

Which the testfile does. So ... I'm not sure what the problem is here. The current version of QueryReadStore? should work properly with grid's virtual scrolling as long as the serverside returns the right value for numRows in the data object.

[guest]

jaredj, is it so that with the current versions of DojoData? and QueryReadStore? there's no need for DojoDataSortable? (or any other extension)? Shouldn't the test pages then be updated accordingly?

[jaredj]

Well, I need to understand what is still broken. As long as the service is supplying the right value for onBegin (which comes from the numRows internal var that QueryReadStore? looks for in its data treturn from the server), then virtual scrolling should be fine.

Though, I don't think QueryReadStore? implementsits sorting serverside (which it probably should), and that may be where the other problem lies. Sorting for a server backed store should really be handled in the same place the query is, in order to get the right offset/size/order in a huge dataset. Trying to do it on the client where it only ever has a small view of that data is going to be very, very, problematic. Sorting *is* supposed to be handled by the store itself, so having that 'DojoDataSortable?' external to the store goes against the dojo.data API design.

I'll see if I can mock up changes to QueryReadStore? so that it can support serverside sorting. I just don't know PHP very well, so it will take some time to port over the JS sorter functions that are used clientside to get a basic sortable QueryReadStore?.php working in test.

[jaredj]

It's actually simpler to get sorting/paging working fine with grid with the current dojo.data grid model. The promary problem was that QueryReadStore? had no mechanism active to pass the sort details to the serverside, even though the pho script to demo QueryReadStore? had simple sorting logic already in it. Once I put some code into the store to passthrough to server when 'doClientSorting' was false, paging/sorting in the grid worked perfectly fine. I even have a simple test file showing it.

I'll be attaching a patch for that I'd like Wolfram's opinion on.

[jaredj]

Patch to do passthrough to server for proper sorting and paging.

[jaredj]

Wolfram wishes to push this to 1.2 and do more investigation ... so, moving!

[wolfram]

(In [13168]) + committing jaredj patch, thx. Make QueryReadStore? automatically handle "sorting" (as set by the grid) fixes #5843 + simplify the according examples + add grid example + fix some CS issues

[guest]

Replying to wolfram:

There is no problem here, please see the examples http://archive.dojotoolkit.org/nightly/dojotoolkit/dojox/data/demos/demo_QueryReadStore.html http://archive.dojotoolkit.org/nightly/dojotoolkit/dojox/data/demos/demo_QueryReadStore_sort.html http://archive.dojotoolkit.org/nightly/dojotoolkit/dojox/data/demos/demo_QueryReadStore_filter.html I made them to exactly show how to do this. Of course its the server side that has to take care of returning the numRows field, in order for the grid to handle it! So I would not say this is a bug

These URL's are broken - where have they moved to? Please cc josh@…

[wolfram]

Replying to guest:

Replying to wolfram:

There is no problem here, please see the examples http://archive.dojotoolkit.org/nightly/dojotoolkit/dojox/data/demos/demo_QueryReadStore.html http://archive.dojotoolkit.org/nightly/dojotoolkit/dojox/data/demos/demo_QueryReadStore_sort.html http://archive.dojotoolkit.org/nightly/dojotoolkit/dojox/data/demos/demo_QueryReadStore_filter.html I made them to exactly show how to do this. Of course its the server side that has to take care of returning the numRows field, in order for the grid to handle it! So I would not say this is a bug

These URL's are broken - where have they moved to? Please cc josh@…

http://archive.dojotoolkit.org/nightly/dojotoolkit/dojox/data/demos/demo_QueryReadStore_grid.html

[guest]

Replying to wolfram:

Replying to guest:

Replying to wolfram:

There is no problem here, please see the examples http://archive.dojotoolkit.org/nightly/dojotoolkit/dojox/data/demos/demo_QueryReadStore.html http://archive.dojotoolkit.org/nightly/dojotoolkit/dojox/data/demos/demo_QueryReadStore_sort.html http://archive.dojotoolkit.org/nightly/dojotoolkit/dojox/data/demos/demo_QueryReadStore_filter.html I made them to exactly show how to do this. Of course its the server side that has to take care of returning the numRows field, in order for the grid to handle it! So I would not say this is a bug

These URL's are broken - where have they moved to? Please cc josh@…

http://archive.dojotoolkit.org/nightly/dojotoolkit/dojox/data/demos/demo_QueryReadStore_grid.html

Thank you - looks like it combined the filtering/sorting too.