Curl Blog: Comparing the performance of Curl and Flex 3

Curl outperforms ActionScript by a factor of 8 to 10

Curl was designed from the beginning so that Curl programs could be compiled to very high quality native code. We have always been proud of the performance of Curl programs, and have made claims of "unmatched performance".

Performance claims should be substantiated by repeatable tests. So we decided to do a performance comparison to quantify how two platforms compare when executing some computationally demanding code.

One of the new components in the Flex 3 Framework is the JPEGEncoder class. That functionality is used to good effect in the AIR SalesBuilder demo to drop a "snapshot" of the dashboard display onto the users desktop. However, the Salesbuilder demo script does warn about a delay when you start dragging.

JPEG encoding seems like an ideal test case since it's a computation that many people do all the time, which makes this computation more interesting than any synthetic benchmark. At the same time, this computation really measures the inherent performance of the language that the algorithm is written in, because even the inner loops are all written in the same language: there is no escape to library routines that might be written in a different language with different performance properties.

In fact, the original motivation for implementing a JPEG encoding algorithm in ActionScript was to illustrate the substantial performance improvements achieved in ActionScript 3 compared to earlier versions.

So we translated this ActionScript program to Curl in the straightforward way and compared the resulting performance for three images ranging from small to moderately large. These are the results that we observed:

	Curl time	Flex time	Image size	Megapixels	Output file size
Small image	0.16 seconds	1.72 seconds	700 x 933	0.65	72 kB
Medium image	0.46	4.43	1170 x 1560	1.83	195 kB
Large image	1.36	11.69	2560 x 1920	4.92	511 kB

The performance comparisons were done between Curl version 6.0 and Flex 3, running on a Dell XPS M170 computer (2.26 GHz, 2 GB RAM, Windows XP SP2).

These results show that Curl retains a substantial advantage in raw execution speed, by a factor of about 8 to 10. We attribute this mainly to the fact that ActionScript is derived from a language that was not architected to be able to be compiled to efficient code, while the Curl language was.

This difference will be important in any application that needs to do data processing or visualization specified by logic in the application itself, which cannot be delegated to predefined, optimized libraries that are already included in the underlying platform.

From the viewpoint of programmer productivity, it is also interesting to note that compiling the JPEG encoder (only about 600 lines of code) took about 8 seconds for the ActionScript implementation using the Flex 3 SDK, and produced a 170 kB SWF file. The Curl implementation requires no compilation step before the 22 kB Curl file is deployed to a Web site, and the just-in-time compilation that occurs when the Curl application is used took less than a quarter of a second.

We've attached the Curl and Flex sources so you can see the exact source code that we compared, if you are curious.

We look forward to your feedback and to any suggestions how this comparison study can be improved.

References

The JPEGEncoder is now part of the Flex 3 framework. The SalesBuilder demo uses an earlier, open source version, from the as3corelib project. The Curl code was transliterated from as3corelib; the reported Flex timings used the Flex3 framework.

The as3corelib code originated in an internal experiment intended to illustrate the performance advantage of the (then, new) AS3 virtual machine. That experiment adapted a free C implementation written by a Rumanian student, Cristi Cuturicu. In fact, the Flex 3 documentation references his paper about the JPEG algorithm.

Flex 3: http://livedocs.adobe.com/flex/3/langref/mx/graphics/codec/JPEGEncoder.html
as3corelib: http://code.google.com/p/as3corelib/source/browse/trunk/src/com/adobe/images/JPGEncoder.as
Original Actionscript: http://www.kaourantin.net/2005/10/more-fun-with-image-formats-in-as3.html
Original C: http://www.yov408.com/html/codespot.php?gg=47
Salesbuilder: http://coenraets.org/air/salesbuilder/salesbuilder_script.pdf

Attachments:

jpeg_080509.zip (1.6 MB)

Tags: curl_blog

Add a comment Leave a comment on this blog post.

May 12, 2008 6:37 PM Reply

friedger

Interesting to see.
I heard that at http://www.jamesward.org/census/ they are working on adding Curl to the benchmark.
Friedger

May 12, 2008 6:41 PM Reply

friedger in response to: friedger

What about MS Silverlight?
Would it be possible to do a similar comparison?
Friedger

May 12, 2008 7:57 PM Reply

RMH in response to: friedger

Our engineers are working on that right now. We hope to make Curl a part of the census soon!

May 12, 2008 7:59 PM Reply

Our engineers did this with Curl and ActionScript because the ActionScript code was already written. If someone in the MS Silverlight crowd wants to write a version in Silverlight we're all for it!

May 13, 2008 8:23 AM Reply

cbarber

I should also say that we are actively working on compiler optimization at this time and expect to improve on our existing performance both for this benchmark and for many other real-world coding situations.

Jun 4, 2008 4:40 PM Reply

Darren

Cherry-picking an example where Curl is vastly superior to Flex is all well and good - and clearly some parts of Curl are more optimised than Flex - but don't you think you went a little overboard with the heading and tagline. Shouldn't they be "Comparing the performance of Curl and Flex 3 in JPEG encoding" and "Curl outperforms ActionScript by a factor of 8 to 10 in JPEG encoding". Otherwise, it's like saying "Hillary outperforms Obama" and leaving out "in Puerto Rico". Unless, you were just trying to be sensationalist, which you clearly were...

Jun 4, 2008 4:48 PM Reply

friedger in response to: Darren

Darren, do you have other result in other examples?

Jun 4, 2008 5:48 PM Reply

rhh in response to: Darren

Darren, I don't think we're cherry-picking at all. The point of this article is that Curl (the language) outperforms ActionScript (the language) by an order of magnitude. The discussion is very clear that this is about the performance of application code written in the language (either Curl or ActionScript), not about the performance of libraries that are included in the Curl RTE or the Flash Player. The discussion also explains the reasons why we think this is significant, which I expanded on in my subsequent post at

http://developers.curl.com/blogs/community_blog/2008/05/30/does-ria-platform-performance-matter

As Friedger said, we'd welcome other interested parties to put forward their own evidence about the situation.

-Bert

Jun 5, 2008 8:01 AM Reply

dmccrae in response to: Darren

Darren, Your point is well taken about the title. I'm glad it got your attention, though!

Despite what we like to say when wearing our marketing hats, performance comparisons (and technology comparisons) are complicated, and contingent. My post earlier this week about different aspects of platform performance has a more abstract title. I hope it got your attention, too.

http://developers.curl.com/blogs/community_blog/2008/06/03/when-ria-language-performance-matters

Jun 5, 2008 10:17 PM Reply

I used to be a curl developer back in 2002 (or was it 2003?). I loved the language, dropped it in favour of a Beta version of Flex called Royale(2004) and have to say Flex/ Flash player is generally "better" technology.

Note: I have said "better technology"... why? at the end of the day they curl, flex, or even silverlight are just tools to help use developers solve a business problem. Curl is fast. Flex is Fast too. but apps developed in any language are no good if they can't be "seen" by anyone.

Rendering a JPG ? or Encoding a JPG ? though it is important to a degree, you have to ask yourself why bench mark this?
when Technology reach, and ease of development and deployment are vastly more important. that's my thoughts.

Jun 6, 2008 7:00 AM Reply

cbarber in response to: Al

It seems that you are saying that Flex is better technology simply because it has wider deployment. That is definitely an important issue for some, but it is really more a matter of marketing reach than technology. I personally don't see the wider deployment as being that big of an issue. No one hesitates to publish documents in pdf form despite the fact that many people do not have the Acrobat reader and it now requires a 22MB+ download, so I don't think it is all that silly to assume that people might be willing to download the much smaller Curl RTE.

In any case, I might be willing to concede that Flash/Flex/AIR is better technology than Curl circa 2002, but a lot of time has passed since then, and we have done a lot to make Curl better, faster, easier to develop in and easier to deploy and will continue to do so.