Huge Recordsets

Hello,
I presented the Active Widgets component to my company, and they loved it until we took a look at the performance drawbacks. Our internal site is using huge recordsets, thousands of rows in some cases. Does anybody have a good solution for dealing with the performance degredation associated with creating a javascript array with so many rows??
I realize the virtual rows thing is out there, and I am going to play with it over the next couple of days, but I am not sure if this will work for us.

One more thing, My company is very interested in freezing one or more columns, but this seems to really hose the performance of the grid.

Jim,

I believe virtual mode with client-side sorting will be acceptable solution for up to 10 000 rows. Grid rendering performance in virtual mode does not depend on how much data you have, so initial image will always be quick. Javascript array sorting is relatively fast and takes 5-15 sec for 10000 rows on IE (2-5 sec on FF) on average 1-2 GHz processor.

For larger sets you should probably do sorting on the server-side as the client will contain only (visited) parts of the dataset.

The disadvantage of virtual mode is that when you scroll too fast - the new data will appear after some delay (repainting the screen at the new viewing posiiton). You can check it quickly with this demo:

http://www.activewidgets.com/preview/2.0/examples/virtual.htm

The current (beta1) implementation assumes that all data is already available in the data model. In reallity you would rather load only small portion of data (only rows requested by the grid). On the first moment the data model will not have the requested data, so it should return proper status code to the grid and send request for this portion of data to the server. Then, when the data comes back, refresh grid rows.

In addition to all that there is always good old paging approach. I don't have paging example for 2.0 yet but I am going to make one soon.

Hey Alex, is it possible to incorporate virtual grid, and frozen columns?

In addition, I have been working on this large dataset thing all day, and it appears that the grid comes up really fast with about 900 rows, but my other dataset (4000 rows) takes a couple of minutes to render.
I am building the javascript by iterating through the recordset, and assigning the row data for each row. Is it possible to just give it the recordset and have it do all this for me?

Any other ideas on faster processing of the recordset to display the grid.
I would love to use paging, but my bosses wont let me.

Also, in the asp 1.0 example, you used multiple instances of the replace function to remove line breaks and format quotes, which seems to be a resource hog, so I am working on a regular expression that will accomodate the removal or formatting of characters or linebreaks which will break the javascript.

It sounds like your slowness is caused by your function providing the data. In the QuickRef directory, there is a grid.htm example that shows you the different ways to assign data to the grid. You can simply pass in a JavaScript Array, without doing a function call like in 1.0. The demo Alex set up has a ton of rows and it's extremely fast so I think your slowness is caused externally.

Right on Jim, I messed around with it quite a bit, and finally got around to using an xml page, and it's pretty fast with about 5000 rows. Very Stoked..