Silveright 2 GUIMark - Big Performance Boost in Silverlight 2 RTW
10/23/2008 Update: Updated the code (rendering and MaxFrameRate) and some notes below
Silverlight has had a few iterations since the Silverlight 1.1 Alpha Days. We started at Silverlight 1.1 Alpha in 2007. This year we had two beta releases and RC0 before Silverlight 2 RTW was released. Silverlight and JavaFX are two new RIA technologies that were introduced this year competing with Adobe's Flash/Flex. In comparing the different technologies, it is natural to compare the performance of the different RIA products and their AJAX/Java Applet cousins.
One of the big performance indicators was GUIMark. GUIMark is a performance test designed by Sean Christmann to gauge the performance of the rendering subsytem of the given technologies. One of the claims of this benchmark is that it uses a variety of tests to give a more true test of the rendering pipeline. There are a variety of different visuals that truly test the performance of the rendering engine. No test is perfect, but I found this one to be pretty good.
GUIMark and Silverlight 2 Beta 1 & 2
Silverlight 2 Beta 1 at the time the results were published the technology was obviously not finished. Why was the rendering so poor? Alex Grenier in the GUIMark website pointed out that the text rendering was the bottleneck. Sure enough, I tried the test myself and removing the text rendering resulted in Silverlight dramatically having the same FPS as Flex. Microsoft obviously had some work to do in order to improve the text rendering for Silverlight.
GUIMark and Silverlight 2 RTW Code
Silverlight 2 RTW has been out for a couple of weeks now. GUIMark has not updated the source code since Beta 1 so I decided to upgrade the code and see how how the performance has increased. I essentially took Sean's Silverlight Beta 1 code and upgraded it for Silverlight 2 RTW in its bare essentials. (The code can be downloaded below)
- I updated the dependency properties to use double data types instead of ints.
- After Beta 1, Silverlight apparently does not support binding from an object that is subclassed from the DependencyObject. You can read more about this here: http://silverlight.net/forums/t/13593.aspx To me, this is really weird and it is worth a discussion outside this topic.
- I changed the namespace and added a web project for deployment and testing
I did NOT TOUCH the rendering loop which is totally inefficient and can now leverage the Silverlight 2 RTW. This can easily add several FPS to the project. See the 10/23 update below, I updated the rendering loop to use the CompositionTarget.Rendering
Silverlight 2 RTW Performance Test
In one of the changes from the betas to RC0/RTW was the text glyph rendering. We knew this was a bottleneck before, lets see how well this performs now. The best way to judge the performance is to see the code run yourself. Click the links below to run the test. Open each link individually and make sure nothing processor intensive is running. If you click the "Run Test" button, just click it once to receive an average FPS over 10 seconds. (If you keep clicking Run Test, the test will fail).
Results - On my Vista Dual Core IE 7 (Your results will vary depending on your processors and browser):
- Flex 3: ~43 FPS
- Silverlight 2 RTW: ~43 FPS (updated code with MaxFrameRate set to 500)
The performance is almost identical between Silverlight and Flex. Microsoft definitetly fixed the rendering of text glyphs. That change is primarily responsible for the great performance improvement. Remember that render loop still can be improved (I might do an update to the code) for extra FPS. One thing to note about FPS is that in a normal production system, FPS is ususally limited. In Flash/Silverlight, the default is about 60 FPS. A lot of developers/architects (including myself) limit the FPS even further. The human eye can't tell the difference between 40-60 FPS anyway, so trying to eke out 400 FPS is completely uncessary.
Silverlight 2 RTW Code: SilverlightHack_GUIMarkSilverlight2RTM.zip (795.85 kb)
I updated the code to use the new CompositionTarget.Rendering event which is new in Silverlight 2 RC0/RTW. This is the best practice for things such as gameplay loops. Some people noticed that Flash was performing siginigantly better than Silverlight. I thought it was the rendering loop being inefficient, however this change by itself did not change the FPS at all. This rendering loop is directly tied to the MaxFrameRate which can be set programatically at runtime by setting the Application.Current.Settings.MaxFrameRate property. Playing with this increased the Silverlight FPS by 15!
The default for the MaxFrameRate is 60. That means that if you have no code or little code if you just put a FPS counter you will get 60 FPS (I tested that and that works fine). Obviously, if you have code in your loop that code could potentially drop. So, I updatd the code/demo with a Frame Rate selector to show what is happening here.
- Setting the MaxFrameRate 10-40 the FrameRate increases. (for example if I set it to 10, I get 10 FPS)...all this makes sense
- Setting the MaxFrameRate 40-60 the FrameRate drops to 35 FPS. This is probably because of the increased overhead of the events. This kind of makes sense.
- Setting the MaxFrameRate 60-500 the FrameRate increases up to 46 FPS ??? This is really weird. I don't understand why setting the FrameRate to 500 actually nets the best results. To me this doesn't make sense.
I posted this question on the Silverlight.net forums here: http://silverlight.net/forums/p/41045/115696.aspx#115696 (When I get an answer, I will update this post)
This does some interesting questions on the MaxFrameRate setting. Oviously one thing I can say 100%...if you are writing a Silvelight game or heavy animations, you need to be very aware of this setting for performance. Furthermore, if writing any benchmarks for Silverlight you may want to play with the optimal MaxFrameRate setting in order to get the best possible results.