JSON Decoding Speed Comparison
Meet Karl - Karl is a very cute wind up Inchworm that was put in my stocking for x-mas. What does Karl have to do with JSON? Probably nothing, but he is very photogenic, and I like having a picture in my blog posts.
Now that our new buddy Karl has been properly introduced, lets move on to the boring stuff. I have been trying to optimize decoding of a large chunk of JSON data by testing decoding speeds between the three main methods of decoding JSON and how that applied to the type of data that was being decoded, along with the environment, ie: the browser. My data has been gathering for over a month now, and has run on 3500 unique clients, so I feel like I can come to a decent conclusion.
The Decoded Data
I figured for my testing purposes that there were five main types of data that would be commonly decoded.
- Empty Array
- Empty Object
- Array Data Only
- Object Data Only
- Mixed Array and Object Data
The Decoding Methods
Generally speaking, you will find three methods of decoding JSON used in the wild, though eval is by far the most common. I am not using any decoding/encoding libraries because I don't want to taint the results with any browser targeted workarounds that might be present in them, I just want to know what the actual browser does.
eval - This is the old standard for decoding JSON, which is considered dangerous to use because of its ability to create functions.
native - The latest and greatest method of decoding JSON, recent browsers include this built in feature. The 'safe' way to decode data.
new function - This method is essentially the same as eval, but uses the evaluating mechanism of the Function constructor to decode data.
In The Wild
I wanted my testing to represent actual usage in the wild, so I used a tool that allowed my tests to run in the browsers of unsuspecting internet surfers. So the results I have can be considered an accurate depiction of what will actually be encountered on the intertubes. If you're creating intranet sites targeted to a particular browser, then these results should be interpreted differently. Any browser that cannot handle all three methods of decoding JSON was not included in the test, so you IE6/7 guys are out of luck. Also, any browser that has an active console (console object present) was excluded due to the overhead of debuggers (ie: FireBug).
The Outcome
The results are quite interesting, and confusing in some cases. Im guessing there is room for some browsers to optimize their JSON decoding routines here. This first chart shows results grouped by browser make. The X axis in all of the following results shows milliseconds per 1000 repetitions, so smaller is better. Legend colors used in the first graph will continue through all graphs.
This is all good and fine if your a browser vendor and want to know how you're browser stacks up against others, but in web development we don't have the option of picking our clients browser. What we want to see is what programming techniques we can use to take advantage of browser efficiency across the board. Which leads us to the next chart, showing results grouped by decoding method
That's more like it, I now have a clearer idea of how the methods themselves stack up against each other, the red line indicates the average of all the data types.
The Conclusion
Even though the results seem to indicate that native JSON decoding is only mildly faster than eval, its actually a bit faster if you exclude the strange handling of empty objects in Firefox. For the purpose of choosing a method, I think we can draw the conclusion that native JSON decoding is the clear winner here, followed closely by eval, and the function constructor being a general bad idea.
Comparing the browsers for native decoding, Safari is a clear winner here, with our new buddy Chrome losing the race by quite a lot.
Quirks
By far the biggest quirk is Firefox's slowness in decoding an empty Object natively, which skews the results, making it appear that native JSON decoding is only mildly faster than eval.
Another interesting result was decoding using eval ran 500% slower with FireBug enabled, while native decoding suffered no degradation at all when FireBug was enabled.
Safari apparently does not handle the function constructor very efficiently.
As a general rule of thumb the straight array data decoded the slowest, so when considering system design, array data should only be used when there is a clear benefit to it.
Feedback
If there are any particular ways you would like to see the data broken out, just let me know in the comments and ill see what I can do.
What version of Firefox were you using there?
And I still can’t get MSIE 8.0′s native JSON to work.
Though, I see documents about it on the web, so I know it’s just me.
All versions that support native JSON, which is 3.1 and up. This test was in the wild, so it includes any version that is still in use.
Remember that native JSON decoding requires that it be 100% valid JSON. Run your JSON through http://www.jsonlint.com to find out.
nice post, kinda expected tho. First of all, eval in that way is unsafe. You should compare json2.js from Mr D. and see how things go with all those “let’s see if it’s safe” RegExps.
Secondly, there is no reason on earth to use new in front of Function since whatever function that returns an instance of object, does not need new in front of it. In few words, the usage of new Function creates an instance that is completely useless.
Example:
function Function(){
// here we got an instance that will never be use
return function () {};
// overload the returned value, not “this” but the function
};
new Function();
Function();
I am sure that this won’t make any consistent difference in above results in any case, but specially when Firebug is enabled and we need to handle big results, Function does not pass throught the console, so I usually appreciate this to avoid problems/massive delays while debugging.
Regards
I already noted that eval is unsafe in the post, along with my reasoning for not testing a json decoding library. This post has nothing to do with ‘safe’ methods.
I disagree that the result was “expected”, since native JSON has the requirement of safely decoding JSON, the expectation is that would take longer than the other unsafe methods. This was the assumption of many people that I discussed this with before starting on these tests.
Thanks for the tip on leaving off the new operator, though I agree that it likely would not have changed the test results.
? What is the measurement for the values on the X axis? Is lower better? Is it trans/sec,? Not sure how to understand the data without a label telling what data is being shown???
Lower is better, the measurement is milliseconds per 1000 repetitions.