Evaluating Javascript Performance

There’s a lot of factors that you have to take into account when you pick a javascript framework. Picking the right tool is a balance of developer familiarity, the task at hand, and performance. Measuring a framework’s performance is tricky. The reality is that no benchmark will account for what your app will do in real life. Making the task harder is that each framework has different features, making a straight head-to-head comparison impossible. However, that doesn’t mean we don’t need to know the relative speed of different frameworks. I found myself in that position recently. I attempted to gauge the relative speed of Mootools, JQuery, Prototype, and Appcelerator. The rational behind picking these four frameworks is that I understood them enough to create a simple test.

the benchmark

The bechmark that I wrote is a rails app and available on GitHub. The README describes what it does:

This pretty simple test attempts to measure the time it takes to load the framework and do some basic tasks. The first is to load the page with the W3C css selector page pre-loaded into a hidden div. It then dynamically loads in the Georgia 2008 Presidential election results, again in a hidden div. Next it loads in the Georgia 2008 Senate results and attaches a click event to all the red elements. Finally, it loads in the Georgia 2008 election summary. All files are served from the local server to avoid mistimes due to network latency. Because of the way Appcelerator works, instead of using a selector to attach an event, I’ve created a different version of the “dynamic” html file with “on” expressions set for the red elements. I feel this test is more fair, since there’s no reason to select through the dom when Appcelerator processes each element by default. This is a rails app, so you should just be able to download it, then run script/server. Navigate to localhost:3000 and you’ll be good to go. Feel free to fork and improve as you see fit.

results

I ran the benchmarks on my Windows XP vm, with 384MB of RAM running within VMWare Fusion on a 2.2GHz Macbook Pro. These are the times. Appcelerator is included twice. The first is an internal patched version. The second is the latest 2.2.1 release. Columns in Red indicate that an error occurred while running the test.

Test (all times in ms)	Firefox	Safari	Chrome	IE
Prototype
Framework Loaded in	80	359	224	31
DOM Loaded in	127	1567	155	109
Load Large File	247	79	164	141
Dynamic Large File	179	104	135	703
Crazy-Large File	3828	572	1363	985
Total	4493	2681	2042	1984
JQuery
Framework Loaded in	111	60	517	141
DOM Loaded in	58	10	28	31
Load Large File	234	198	249	563
Dynamic Large File	474	209	227	1187
Crazy-Large File	1055	861	2218	3922
Total	1968	1338	3240	5876
Mootools
Framework Loaded in	48	54	147	485
DOM Loaded in	45	11	44	62
Load Large File	319	142	190	3813
Dynamic Large File	275	170	124	2329
Crazy-Large File	1697	932	1293	24877
Total	2413	1310	1799	31598
Appcelerator (internal)
Framework Loaded in	190	234	timeout	281
DOM Loaded in	870	341	timeout	2485
Load Large File	1241	501	timeout	24237
Dynamic Large File	1086	578	timeout	31331
Crazy-Large File	20196	2772	timeout	595237
Total	23658	4426	timeout	653664
Appcelerator (2.2.1)
Framework Loaded in	343	425	timeout	Error
DOM Loaded in	1067	748	timeout	Error
Load Large File	3145	2053	timeout	Error
Dynamic Large File	2420	1253	timeout	Error
Crazy-Large File	17814	6780	timeout	Error
Total	24841	11265	timeout	Error

conclusions

Safari is incredibly fast. If all your users used Safari, the framework you use doesn’t really matter. I’m pretty surprised with how fast Prototype was in IE. It was the only one that was quicker in IE than in FF. Mootools and JQuery had similar performance, right up until it came time to load a very large file in IE. No framework was the clear winner on speed. Hopefully this benchmark provides a little guidance on which framework to pick. If all you’re doing is catering to Safari users, feel free to go with something like Appcelerator. If you need to target IE users or the general web population, you can’t go wrong with Prototype, Mootools or JQuery, unless you’re loading in very large files, at which point Mootools is out. Otherwise, pick for which one you like better.