How we make 5000 row tables fast in Streak(blog.streak.com)
blog.streak.com
How we make 5000 row tables fast in Streak
http://blog.streak.com/2012/07/butter-performance.html
13 comments
It's great to read some of these advanced optimization techniques, especially for us web devs that aren't working on these larger projects.
I'm interested to know more about how and why you are using this "Levinshtein distance" (Fast Levinshtein) algorithm. I looked at the Wikipedia and the explanation makes sense; it's just not sinking in as to where I might think about making use of it in my own code.
I'm interested to know more about how and why you are using this "Levinshtein distance" (Fast Levinshtein) algorithm. I looked at the Wikipedia and the explanation makes sense; it's just not sinking in as to where I might think about making use of it in my own code.
Is there a way to view one of these tables without having to log in?
No, but I'll make a video right now.
edit: here's the video http://www.youtube.com/watch?v=o1O7HkBj74c
edit: here's the video http://www.youtube.com/watch?v=o1O7HkBj74c
[deleted](2)
Open source it!
Please.
Please.
FYI, the right-side menu makes the scrollbar impossible to grab at the top of the page. However, I can still scroll with the wheel or bottom of the scrollbar.
Ironically, when I move the scrollbar on this blog post (in Chrome), the scrolling is noticeably not "buttery", probably because of the JS-heavy interface...
Something is very strange with scrolling using a mouse wheel, too. It seems to scroll about a third to half of the distance it usual does for each "tick" of the mouse wheel (Ubuntu, Firefox). I guess it doesn't recognize the OS setting.
The whole page lags like a bitch on ipad3. Irony, for sure.
I love articles about web design that are completely blank if you don't have JavaScript activated.
I love how there are people who have to point out that they have JavaScript disabled in 2012.
I use requestpolicy with fairly strict settings, and I have since grown to hate websites that have the content and styles loaded from two or more different domains. The vastly faster load times and vastly less junk on the page definitely makes it worth it.
Funny, because loading assets from different domain actually can help improve page performance: for one it allows to work around maximum-parallel-request-per-domain restriction, for other you may serve resources from cookie-less domain shaving a few more milliseconds.
It absolutely can. Unfortunately the stuff usually loaded is a crapload of antisocial share buttons and tracking crap that more that destroy any gain that the techniques you mention bring.
How about a phone browser where it loads but freaks out and makes a glitchy mess that can't be scrolled.
And on my desktop Javascript uses most of the browser's memory. No thanks.
And on my desktop Javascript uses most of the browser's memory. No thanks.
We do something similar to render large tables, but we use divs to layout our data. We establish the height of the scrollable area with a large div, and then absolutely position the rows within it. We have just enough real physical rows to fill the screen, plus a buffer of a few rows. The virtual rows are assigned to physical rows via modular arithmetic, so only a couple of rows change their data on scroll. (And are repositioned accordingly.)
This technique (particularly calculating the positions of the rows) requires fixed height rows. (There is a solution for variable height rows, but it's a bit more complicated.)
I originally got the idea by looking at google books years ago.
Here is a quick and dirty example as a self-contained HTML file:
This technique (particularly calculating the positions of the rows) requires fixed height rows. (There is a solution for variable height rows, but it's a bit more complicated.)
I originally got the idea by looking at google books years ago.
Here is a quick and dirty example as a self-contained HTML file:
https://gist.github.com/3086578
If anyone wants to have a play with this code, http://jsfiddle.net/SukX5/1/
This is awesome. I like your technique a lot better actually!
Sadly--and please correct me if I'm wrong on this--there is no way to use that same trick on Android/iOS without painfully-and-likely-poorly reimplementing the browser's scrolling mechanism (as you don't get events and cannot render while the user is manipulating the native viewport).
Yeah, I get the events on iOS if I turn off touch scrolling, but I don't get the nice, smooth scrolling. I don't know how it works on Android.
However, on iOS, I can't seem to disable the bounce scrolling of the entire screen if I have any scrollable areas on the screen. So I ended up writing a decorator for overflow:auto elements that re-implements touch scrolling. And does a preventDefault() as necessary to prevent the bounce of the app.
Here is a gist of the decorator:
It works on the iPad, but I can't vouch for android. (This code is from a little weekend project.)
Edit: I didn't implement rubber banding or anything. It's just a quick and dirty scroll with some inertia based on a weighted average velocity of the last two touchmove events. It doesn't feel too bad, but it isn't perfect.
However, on iOS, I can't seem to disable the bounce scrolling of the entire screen if I have any scrollable areas on the screen. So I ended up writing a decorator for overflow:auto elements that re-implements touch scrolling. And does a preventDefault() as necessary to prevent the bounce of the app.
Here is a gist of the decorator:
https://gist.github.com/3086981
It works on the iPad, but I can't vouch for android. (This code is from a little weekend project.)
Edit: I didn't implement rubber banding or anything. It's just a quick and dirty scroll with some inertia based on a weighted average velocity of the last two touchmove events. It doesn't feel too bad, but it isn't perfect.
Check this out: http://johanbrook.com/browsers/native-momentum-scrolling-ios...
It can cause some quirky behavior so test it out carefully but you get much better performance with long lists if using it.
It can cause some quirky behavior so test it out carefully but you get much better performance with long lists if using it.
Qooxdoo, my favorite opensource RIA tool, has virtual lists and grids as core building blocks in both the desktop and mobile variants.
However, i would not be surprised to find out, that they emulate their own scroll containers. But it is indeed possible and works well.
See http://qooxdoo.org
However, i would not be surprised to find out, that they emulate their own scroll containers. But it is indeed possible and works well.
See http://qooxdoo.org
Just checked that one out, and I'm somewhat floored that it isn't obvious to you that it reimplemented the scrolling behavior: the rubber-banding and acceleration curves are wrong, and it scrolls to logical pixel boundaries (the native scrolling can move one physical pixel, while this framework moves by two as the minimum step). It is much better than what I've seen previously, though; makes me wonder if the CPU on the 4S combined with the latest WebKit+Nitro really is now enough to emulate scrolling well (assuming someone attended better to the details). Thanks!
It was quite obvious it emulates the scrollbars on the desktop, but i have no iphone, and made no such assumption about what it actually does on mobile.
Ive been following Qooxdoo for quite some time, and the level of browser normalisation ( from keyboard events to how scrollbars render ) is insane.
And so is the performance, and the tricks they pull. For example: they recyle dom nodes. They have different array looping constructs based on the target platform. They emulate their own event bubbling system. There is a whole range of performance tests you can try on their website as well.
The real question is, can we get this kind of performance with more idiomatic html/css.
Ive been following Qooxdoo for quite some time, and the level of browser normalisation ( from keyboard events to how scrollbars render ) is insane.
And so is the performance, and the tricks they pull. For example: they recyle dom nodes. They have different array looping constructs based on the target platform. They emulate their own event bubbling system. There is a whole range of performance tests you can try on their website as well.
The real question is, can we get this kind of performance with more idiomatic html/css.
[deleted]
iScroll [1] does a pretty reasonable job of rebuilding the scrolling events.
[1] http://cubiq.org/iscroll
[1] http://cubiq.org/iscroll
This is really great, thanks. Could you shed some light on the variable height row solution? Do you still use absolutely positioned rows, or do you do something different?
This sounds pretty similar to how UITableView on the iPhone keeps its performance up. If I recall correctly, iOS actually recycles a small number of discrete cells for use in the table and shuffles in the necessary data.
I'm always surprised (though it happens fairly frequently) when I run into a UI toolkit with any kind of fixed-size item list view that doesn't recycle the child widgets and handle the actual item list as a sliding-window data provider.
Not to take anything away from any of the mentioned projects, but this pattern is not new or novel.
Not to take anything away from any of the mentioned projects, but this pattern is not new or novel.
No, its not. Its even more relevant for Javascript, because creating a dom node is way more expensive, thsn just altering it.
We actually did the exact same thing (concept from Slickgrid but modular rows using backbone models and what we call Modules for the view of that row).
I recently jumped through a lot of similar hoops in making an Activity Feed for my company. We use a few finesses to cut down on the number of repaints the browser likes to do when dealing with huge tables like these.
More on that here: http://layervault.tumblr.com/post/26117253199/making-a-new-t...
More on that here: http://layervault.tumblr.com/post/26117253199/making-a-new-t...
That slow multisort doesn't do multisort at all, generally. If it did, it would be relying on (1) the sort algorithm being stable and (2) the properties being in the reverse order from what you used in the second multisort algorithm.
Eg with properties a,b it sorts on a, then sorts the list again based on b. The list is now in b-order (not a-order) and b-equal blocks are only in a-order if the sort is stable. Meanwhile, the 'fast' algortithm is in a-order with b-ordered a-equal blocks.
Eg with properties a,b it sorts on a, then sorts the list again based on b. The list is now in b-order (not a-order) and b-equal blocks are only in a-order if the sort is stable. Meanwhile, the 'fast' algortithm is in a-order with b-ordered a-equal blocks.
GWT had a PagingScrollTable - I used it 3-4 years ago. It has been a while since I built a GWT project, so I'm sure it has improved dramatically, but I was able to get lightning fast tables using PagingScrollTable. Here's a demo http://zenoconsulting.biz/com.example.Application/Applicatio... -- try 5,000 rows. It is pretty speedy.
One of their requirements was no paging - everything on one page.
5000 rows or 5000 columns? 5000 rows isn't a lot.
Edit: the article says 5000 rows, 10 columns. Hm.
Edit: the article says 5000 rows, 10 columns. Hm.
1. position:relative; or absolute on the ccell or within a cell causes same slowdown as overflow:hidden.
2. On iOS, overflow:hidden and position:relative on cells can cause serious slowdowns later when an absolute div later dynamically over document.
3. Beware of tables in IE8 - changing className of div in cell or other changes can cause complete table reflow calc - found out using commercial profiler that gives information about reflow/redraw times.