Large dataset makes the grid really slow

Hi! I am trying the xml - simple.htm example with a large dataset(500+ rows). I notice some performance degradation and increased memory consumption. Is a some sort of a virtual mode available for this grid ?
Anything I can do to optimize for the large dataset ?

Thank you for your help

Hey I was buying the library for commercial use, but If this is true, I need it for large datasets visualization (like 100+ rows and 20 or more columns).

Any thought about the slowdown ?
Maybe latest 1.0 version fixed this leeks ?

Regards

I use a 200 row limit and have not had a problem. I limit the data that comes back to 200 rows. Takes under 4 seconds to load (including download data on a local intranet) and about 3 seconds to sort.

I give users the tools to filter their searches and figure if they max out the number of rows then that they can learn to filter better.

I don't think you will find another component that does what this does.

I am planning to add 'virtual' loading at some point, but the current version does not have this feature. If your dataset exceeds 100 rows you have to implement some sort of paging otherwise the data loading and sorting will be quite slow (approx. 0.5-1 sec per 1000 cells). There is a simple paging example in /patches/paging1.htm which might help to start with paging implementation.

The memory leaks is a different issue. Every effort had been taken to eliminate possibility of memory leaks in ActiveWidgets library. So far I am not aware of any memory leaks in the current version. If anyone sees memory leaks related to ActiveWidgets code - please let me know.

I use the paging - and right now am working with 30 columns, and about 700 rows

It takes maybe a second on my system right now to sort, which is fine with me (considering I have more things open than NASA right now)

I think javascript native array sorting algorthm is very efficient. It usually takes more time to repaint (subset of) rows than sort them.

I agree. I don't use Alex's grid but I have a 300+ row table which is sortable on primary, secondary and tertiary colums. It was having speed issues and investigation showed that the sort was near instantaneous while the rendering took forever.

Those timings are approximate. ;-)

If I have a data of say, 1000 rows and if I want to display 50 rows at a time, what would be the method to call which takes from and to values as parameters and prints the rows depending on the parameters?

Thanks in Advance.

I'm evaluating the product and have data requirements of 2 to 8 columns and between 20 and 75,000 (!) rows. I'm loading the data from a client-side ActiveX control (no URL for the XML) which caches XML data on the client for ready access. So, the (significant) load time is definitely in these two calls:

awTable.setXML(xmlDomDoc);
-or-
theDiv.innerHTML = awObj;

Alex, This really is the best Web client grid I've seen in my years and I'd love to be able to purchase it for use in my application if I can get it to meet my needs. Do you have any suggestions for how to use your AW Grid with data requirements of such a large size?

Any help would be greatly appreciated...

Here is a trick to help speed things up when you have a lot of rows. First, before displaying the grid, set the rowCount to 10. Display the grid then change the rowCount to the actual number of rows and do a refresh(). You are going to find that it SIGNIFICANTLY speeds up load/display time of the grid. I use a window.setTimeout to trigger a function to do the dirty work. I set the timeout to 500 ms and it seems to work great for us.

Jim, would you be so kind as to post some code to this? It would help if I had an example (with and without your "trick") so that I can fully investigate the impact of this.

Would be great if what you say is true.

The trick which Jim has described should work fine. What it does - it first renders the grid with a small subset of data (top 10 rows), so the user sees the result without delay. Then it renders the grid again with full data set which would take as much time as normally, but this is invisible for the user because the top rows are already on screen. The downside is that browser is still locked during the long second rendering.

I would say that the only real solution for large datasets is paging - but this trick can significantly improve 'perception of speed'.

Alex,

If you do some testing you are actually going to find that the total time to load the page goes down using my trick. We brought a 20 second total load time down to about 7 seconds. It surprised us, but we did many tests and they all bore similar results.

The paging patch is the way to go, I must agree. The only problem with the patch is when one select a colomn for sorting, the page label is not updated.

Guys,
I kinda dissagree with paging as the ultimate solution to the strategy. The two best ways to truly solve the problem are:

1) Virtual Mode: Don't render rows until they come into view. Look at the stats, it take 95% of the time to render the rows, and 5% to load and sort the data. Domapi (www.domapi.com) v4 has a listgrid that is very mature and supports this functionality. A quick test of Domapi's listgrid with Virtual Mode On vs. Off shows a DRAMATIC improvement speedwise from only rendering rows in view.

2) Seemless Paging: Use AJAX calls for data everytime the grid is loaded,scrolled or sorted. ive seen random implementations of this in custom grid components online, and its surprisingly effective, and enables the grid to work with infinite size data sets .

#2 Is complex to implement, but virtual mode is very implementable. Alex if you want some ideas how to do it, i recommend taking a look at the domapi source.

Guys, this widget is great for small recordsets, I large recordsets, it's better to use simple html tables, manipulating the DOM.
I get a 2000 rows 20 cols, render in les than 1 sec. suing simple DOM techniques.