Lower the cost of Azure pricing through Tools, Promotions and Technology Decisions
Abstract: In this article I will show some techniques to lower the costs of using the Windows Azure Platform. We will look at three different methods of lowering the cost of Azure: via tools, promotions and technology architecture decisions that can dramatically impact costs of using the Azure cloud platform.
Tools: Azure Platform Pricing Calculator
The first step in understanding your costs and how to lower them in Azure is to forecast the costs of the services in Azure. Until recently this was largely a manual effort and an architect or technical business owner would have to sift through pricing grids (like this one) and tally the numbers yourself.
Microsoft has released the Azure Platform Pricing Calculator that allows you to easily select the services you need and determine their costs very easily. This not only allows you to make planning easier, but it also can dramatically lower your costs by showing you cheaper options available (i.e. promotions or smaller instances of Compute). The Azure Platform Pricing Calculator can also show you the different costs associated with architectural options (i.e. having a 100gig SQL Server database is about $1,000/month versus table storage of the same size is $15/month + minimal costs with storage transactions).
Promotions & AzureOffers: Look at All the different Microsoft Azure options
Microsoft is really pushing Windows Azure. A lot of their core evangelists, program managers and sales teams are pushing the platform to the enterprise. Furthermore, there are a lot of deals to attract business owners, consultants, developer hobbyists as well. Make sure you shop around for the BEST deal.
These will probably change, but there are several different FREE trials available to try Azure and lower your costs.
- Microsoft offers a no credit card required 30-day trial of Windows Azure Platform trial here.
- Microsoft offers a credit card required offer until September 30th, 2011 FREE trial here.
MSDN Subscribers have the ability to get components and Azure services for FREE at a certain limit for FREE as well. The higher level of your MSDN Subscription, the more is alotted to you (for example a MSDN Ultimate Subscription has 1,500 hours/month of a compute instance available).
Other Business & Partner Agreements:
If you are a large customer from Microsoft, you can obviously negotiate better enterprise pricing for Azure ("You can negotiate?...yes"
). However, if you are a small business or a startup you can also participate in either the "Cloud Essentials" Microsoft partner program
or the Microsoft BizSpark program
, which allows you to take advantage of the Windows Azure platform at lower costs as well.
You can compare all of the offers from Microsoft in a nice grid here
Technology Architecture Considerations:
SQL Azure Alternatives
SQL Azure is EXPENSIVE (especially for very large sites). Even for a small site like mine, which has a read-only SQL Server database...I did not want to pay $10/month ($120/year for a 1 gig database). Here are some alternatives:
- Use SQL Server CE 4. It is a new embedded database that can be deployed to the web role in Azure. The nice thing is that you will not have to pay for storage or transactions, because it is part of the web role. The downside is that you will have to be responsible for your own disaster recovery. This would not work great for mission critical or very transactional databases. However, small databases with read-only historical data (i.e. baseball stats), aggregate caching databases (i.e. map reduce business intelligence) or low transaction databases (i.e. a blog) would be great candidates and save you a lot of money.
- Combine Data Content Data Sources. If you have multiple data sources...look for ways to combine them into a single data source. For example, if you have a small blog and a membership of ASP.NET..instead of paying $9.99/month x 2 for two databases...why not combine them into a single database and pay $9.99/month? In those cases, proper naming conventions in SQL will save you and allow you to do this.
- Use NOSQL Databases. Not every single web site needs a relational database behind the scenes. Sites like Google, Facebook or Reddit use NOSQL storage almost exclusively to power their entire platforms. For example, a .NET Lucene library port for Azure exists, which uses local caching heavily as well as Azure Blob storage which is significantly cheaper than SQL Azure. Using the Azure infrastructure/SDK you can even run NOSQL databases like MongoDB on Azure as well. Azure Table storage is another "NOSQL like" alternative in the Azure platform that is much cheaper than SQL Azure.
- Use cheaper cloud storage databases. If you are a .NET developer or prefer working with the Microsoft cloud tools nothing is stopping you from using different cloud content alternatives. For example, you can host your database on Amazon EC2 and have the front-end or compute instances deployed to Windows Azure.
- Sometimes SQL Azure is not a good choice (costs-wise). For example, if I have a large data warehouse (50-75 gig), I would strongly reconsider surfacing that content in Azure directly in SQL Server...the costs per month would be astronomical. I would consider a NOSQL database or not move the data warehouse to the cloud.
I know this might be an obvious one, but remember you are not only paying for storing your content in the cloud...you also pay a very small amount each time someone access it. On a very busy site with lots of images, icons, scripts etc. this can add up very quickly. Having the content pulled from the web compute role will drastically lower the costs of accessing other storage mechanisms such as BLOB or Queue storage.
Two types of caching:
- Server content caching: This content is cached on the server, therefore the content does not have to be retrieved from a SQL Azure, BLOB Storage or from another remote location that is not on the local server. For example, caching the same results in memory instead of performing the same SQL Server t-sql query will save costs because you are mitigating the number of transations from the SQL Server to the web server.
Use the Enterprise CDNs for Scripts (i.e. jQuery, Ajax)
(I am not talking about the Azure CDN, which is a different topic). Microsoft and Google both provide hosted access to things like jQuery. Instead of having that script downloaded from your Azure web role, the client browser or mobile device downloads it from Microsoft or Google directly. This saves you both transactional and bandwidth costs. Here is a good article describing
the pros/cons of both. Putting those scripts on Microsoft's or Google's network will decrecy the network latency of the calls and also allow your pages to load faster (since client browsers can only make a limited amount of requests to the same domain). The main benefits are described in this article here
Use the RIGHT VM Sizes
Azure includes several different VMs you can use to deploy your web or worker roles. Ensure that you pick the right one for the job. For example, Microsoft positions the "Extra Small" VM as a way to test code and not really meant for production. However, that role is perfectly OK for running a blog, small corporate site, a restaurant site etc with no problems. I am running my www.silverlightbusinessintelligence.com
site on the Extra Small instance and it works really well and have no need to pay for a more expensive VM.
- Note: Remember that the defaults and best practices Microsoft perscribes are not necessarily what is needed for you and there are lower options available. You can always quickly scale higher as you need more horsepower. There are techniques to poll every minute your CPU/Memory utilization to determine if you need a better cloud VM infrastructure.
Multiple Web Sites Per Web Role
In the Azure 1.3 SDK update, now you can host multiple web sites per single web role. For example, if you have a Medium VM you might be able to put on 3 or 4 different web sites per single web role. This will obviously cut down costs as you are not paying for 4 different VMs just one. The Windows Azure Platform has detailed instructions on how to support this here
- Note: Architecturally you can also stagger different sites for traffic reasons (according to year/time of day/week etc). For example, if you had an tax website (high traffic beginning of the year) and place that with a web sites for NFL Football (popular towards the end of the year).
Use IIS 7.x HTTP Compression (dynamic and static content)
In Azure, you have a choice to deploy to either an Windows 2008 SP1 or Windows 2008 R2 Server. Both servers use IIS 7.x as the web server, which makes configuring things like static compression or even dynamic compression pretty easy. This will save you a lot of money on larger sites, since most of the web traffic is string based that can be compressed with dramatic ratios.
- Note: It appears in either the new Azure SDK or the new VM image, IIS compression is turned on by default for dynamic content. For example, in order to have my web site (silverlightbusinessintelligence.com) to take advantage of IIS 7.x compression..I did not have to do anything. However, keep in mind that if you take full control of your serialization, media formats or APIs etc. you will have to configure IIS Compression.
In this article I showed you different tools, promotions and technology architecture choices you can use to lower the costs of using the Azure Cloud. Remember that every service you choose to use from Azure has a cost associated with it.
- Plan using the Azure Platform Pricing Calculator
- Look at the available promotions and offers.
- An offer may not include everything, but it can dramatically subsidize the costs of doing business in Azure
- Architectural decisions can end up costing you a lot of money
Azure Service Deployment Package Inefficiency (Bug) - Content Size Added Twice
I spent this past week migrating my companion site for my book www.silverlightbusinessintelligence.com to the Azure cloud (http://slbusinessintelligence.cloudapp.net/). The site has a lot of demos and source code associated with it, therefore the Azure Service deployment package (cspkg file) was pretty big.
Problem: I actually ended up running into the maximum allowed size for an Azure service package, which is 600 megs (which I find odd, since you can easily get around that limitation by adding static content to the web role by remoting in or doing a web deploy afterwards). I deployed to an "Extra Small" instance, which is all I really need to a simple site like this and you get a lot of storage (shown below). The reason Microsoft caps the deployment at 600 megs is that your deploying to the E:\ logical drive, which has only 1 gig of space. However, as you can see there is PLENTY of space on both the d:\ logical drive. I know space is required for the OS, future patches and changes etc. but this limitation struck me as odd.
My three logical drives on the Azure Web Role:
Solution: One of the obvious solutions was the move the three source code zip files (which are over 100 megs) to the Azure blob storage (which costs extra) and server the links from there. I have my static source code zip files on the Azure blob storage for example: http://bartczernickiblogs.blob.core.windows.net/sourcecode/SilverlightBI_SourceCode_Version1.zip.
The other nice benefit of moving your static content to blob storage (and enabling public read access) is that your content loads faster since it is coming from a different domain in a browser. For example, if you had a lot of images to load on a web site.
Azure Deployment Package Inefficiency: After doing this I thought it was curious that the Azure service package size was so high as I didn't think I had over 600 megs of code, XAP filess or content. Granted my three different versions of the source code are 137 megs in total and the total Azure service package is: 630 megs. However, after I removed the three source code zip files (figuring it would make the package 137 megs less) it made the newer package a total of 351 megs almost 279 megs smaller! So it looks like Azure deployment packaging is adding the content twice!?
Digging in....an Azure Service Deployment package (cspkg file) is just a zip file. Inside it you can find the main content file, which is stored inside a file with the csnsx extension. The csnsx file is just another zip file that is encrypted. In order to decrypt it, add the _CSPACK_FORCE_NOENCRYPT_ environment variable and set it to true (follow this blogpost in detail). I decrypted my package and low and behold the entire site is replicated basically twice in two folders both APPROOT and SITEROOT (shown below):
AppRoot source code files of the Azure Deployment package:
SiteRoot source code files of the Azure Deployment package:
There obviously is a technical reason this was done this way, by having the same files duplicated in both APPROOT and SITEROOT folders. However, I look at this as a big optimization opportunity for the next version of the Azure SDK. For example, Microsoft could derive the APPROOT or SITEROOT from each other after it is deployed to the web role (?). In my opinion, if Microsoft is going to limit the Azure deployment package size to 600 megs, static content should not be added twice!
My example is not very common, but there are reasons why to make this faster:
- Bandwidth costs: If you count the amount of content uploaded to Azure, having content double package sizes costs the Azure user and Microsoft both money. Also for large packages like this, with the new world of bandwidth caps this costs me money too.
- Deployment time: Uploading large packages like this takes time. It almost took 45 minutes from upload to instantiation of the package. Making the packages smaller and more efficient will speed up the deployment process. One of the biggest selling points of AppHarbor (which is .NET cloud on Amazon EC2) is how much faster it is to deploy to the cloud....and it is by factors of 10x or more!).
Step by step instructions to deploy ASP.NET MVC 3 to Windows Azure
Abstract: This article will show you the steps to overcome the lack of Visual Studio 2010/Azure SDK tooling to deploy ASP.NET MVC 3 as a web role to the Azure cloud. As of the time this article was written, there are many articles out there that overcomplicate the steps and some are even outdated with the new Azure 1.4 SDK release. This article will allow you to work with the full ASP.NET MVC 3 tooling. This article will be relevant until Microsoft releases an update to the Visual Studio 2010 tooling for Azure (probably not until the Azure 1.5 SDK). The Visual Studio 2010 project/source code is included below.
10/01/2011 Update: The latest Windows Azure SDK (September 2011 Update) now includes ASP.NET MVC 3 by default. However, if you are interested in including ASP.NET MVC4 in Azure, these steps can be followed below (with some changes).
For example, this week a Microsoft employee Avkash Chauhan posted an article called "ASP.NET MVC3 Web Role Deployment in Windows Azure (SDK 1.4) - Detailed steps and Full Sample". While the article is correct and the process outlined will work, it overcomplicates the steps necessary and there are some bugs in his solution (i.e. you do not get the MVC 3 tooling support, since it is a ASP.NET MVC 2 web role).
Deploying ASP.NET MVC 3 to Azure 1.4 should not include so many steps.
Steps to Deploy an ASP.NET MVC 3 web role to the Azure Cloud
The list below are the high level steps that that this article will follow.
- Create a new ASP.NET MVC 3 project
- this can be an Internet or Empty application (or an existing ASP.NET MVC 3 project)
- Add a Windows Azure Cloud project
- Add the Azure Web Role components to the ASP.NET MVC 3 Project
- Add the ASP.NET MVC 3 Assemblies to the bin Folder
- Deploying via the new Azure 1.4 start up tasks is another method
- Publish to Windows Azure
Note: You will need to have some experience with Windows Azure, since I do not cover publishing to the Azure cloud (creating certificates, management console, remote desktop etc.). You will also need:
- Visual Studio 2010 SP1
- ASP.NET MVC 3 with tools update (Released in May 2011: http://www.asp.net/mvc)
- Azure 1.4 SDK installed
- Active Windows Azure hosted service account (there is a promotion going on to receive Windows Azure for FREE until September 30th 2011, much better than the FREE 30 day trial if you just want to test things out or run a small blog: http://www.microsoft.com/windowsazure/free-trial/)
1) Create a new ASP.NET MVC 3 Project
- Open Visual Studio 2010 (as an Administrator)
- The Azure Cloud emulator needs to run in elevated privaledges
- Inside Visual Studio 2010, go to File -> New Project
- Select "ASP.NET MVC 3 Web Application"
- I named the project "AspNetMVC3DeployToAzure" *you can call it whatever you like)
- Select "Internet Application" from the next dialog windo
- Note that we will not be deploying the ASP.NET membership provider to SQL Azure (that will be for another time)
- Just as long as you do not click Log On the application will function correctly
- If you build the application and debug it, you should see the generic ASP.NET MVC 3 startup screen
2) Add a Windows Azure Cloud Project
- In the Visual Studio 2010 Solution Explorer, right-click on the AspNetMvc4DeployToAzure solution and select Add -> New Project...
- In the New Project Dialog,
- select "Cloud" from the Installed Templates
- select the "Windows Azure Project"
- I named the project "WindowsAzureAspNetMvc3Deploy" (you can name it whatever you prefer)
- In the next dialog, do NOT select any Windows Azure roles. Leave it empy and click OK.
3) Add the Azure Web Role components to the ASP.NET MVC 3 Project
- In the Solution Explorer, highlight the ASP.NET MVC 3 project by clicking on it (this will tell Visual Studio the context you are in(
- In Visual Studio 2010 (this step will only work if the ASP.NET MVC 3 project is highlighted) go to Tools -> "Library Package Manager" -> "Add Library Package Reference..."
- Click "Online" to search the available packages Online
- Type in "windowsazure" all one word NO spaces
- Select the WindowsAzure.WebRole package and click Install. Close the dialog after you are done (VS 2010 will ask you to reload the project...click OK) The following changes will be made to the ASP.NET MVC 3 Project:
- The package will add the necesary Windows Azure SDK 1.4 assemblies
- The package will make changes necessary in the web.config for Windows Azure
- The package will add a class that facilitates the new Azure SDK 1.4 change for performign a task when the role is deployed
- In the Solution Explorer, right-click on the Roles folder of the Windows Azure project. (this will tell our Windows Azure cloud project that we want to associate to the ASP.NET MVC 3 web role)
- Select Add -> "Web Role Project in Solution..."
- In the previous step, we ran the NuGet package that "converted" our traditional ASP.NET MVC 3 application to a web role by adding all the necessary assemblies. Therefore, this step is telling Azure that we want to add an existing Web Role from our current solution.
- In the "Associate with Web Role" dialog, click OK (as we only have one available project)
- Your solution should now look like the screen shot below.
- Note that we have additional Azure assemblies referenced in the ASP.NET MVC 3 project, we have a new class WebRole.cs, the web.config has Azure elements in it and inside the Azure cloud project we have a new web role added
- You can now buiild and debug the solution. If you did everything properly, this will launch the Windows Azure emulator and run the application.
4) Add the ASP.NET MVC 3 assemblies to the bin folder
Note: the emulator is just an emulator, so it is still utilizing resources on your workstation (i.e. ASP.NET MVC 3, Web Matrix assemblies). In a real Azure Virtual Machine, you will ONLY have the basic .NET 4.0 assemblies (Microsoft only keeps the VMs to a bare minimum). Therefore, we need to add those assemblies as part of the deployment into Azure VM.
Note: An alternative approach is to utilize the ASP.NET MVC 3 installer as a Azure start up task (each time the VM is restarted or re-provisioned in case of an error or recovery process) or manually set each assembly to "Copy Local" to True.
- In the ASP.NET MVC 3 Tools Update Microsoft provides a very easy way to add all the required ASP.NET MVC 3 assemblies automatically for bin deployment. (thanks to Mike Brown from http://azurecoding.net/ for pointing this out).
- Right-click on the ASPNetMvc3DeployToAzure MVC 3 project and select "Add Deployable Dependencies..."
- Select ASP.NET MVC checkbox and click OK
- This will add the necessary ASP.NET MVC 3 dependency assemblies.
- You Solution Explorer in Visual Studio should look like this after adding the ASP.NET MVC 3 Dependency assemblies.
5) Publish to Windows Azure
- Now you are ready to publish to Windows Azure. Right-click on the Windows Azure project and select Publish. Select your Azure credentials and storage information.
- Note: If you do not have the Azure hosted service, storage or credentials set up you will have to create it. Step by step instructions can be found here (Microsoft has better instructions than I will write on this)
- I highly recommed you also select the "Enable Intellitrace for .NET 4 Roles". This will show you any errors that you get if you don't have the proper ASP.NET MVC 3 assemblies installed. Here is a good article on how you can view the IntelliTrace logs: http://www.nickharris.net/2010/09/how-to-enable-intellitrace-on-windows-azure/. For exampe, if you forget to enable "Copy Local" to true for the System.Web.Mvc.dll the Azure publish process will cycle and you will not know why.
- If you forget a certain assembly, you will also get an error like this (if you turn off custom errors in the web.config)
Visual Studio 2010 Source Code For Project
This source code for this project includes some minor changes unrelated to the steps above:
AspNetMVC3DeployToAzure.zip (13.85 mb)
- in the web.config I turned custom errors off, so you can see the errors if you receive any in deploying
- I changed the Windows Azure instance to be "Extra Small", this way you get charged less playing with this and it is covered under the FREE Windows Azure trial
- I changed the OS family to 2 in the Windows Azure configuration, so that we receive a Windows Server 2008 R2 VM...not sure why that is not the default (?)
In this article, I hope you learned how to deploy an ASP.NET MVC 3 application to the Azure cloud easily. The key things to remember are:
- using the WindowsAzure.WebRole NuGet package to convert your ASP.NET MVC 3 application to a web role
- Associating the ASP.NET MVC 3 project with the Windows Azure Cloud project
- Using the new "Add Deployment Dependencies.." feature in the ASP.NET MVC 3 Tools Update to deploy all the required ASP.NET MVC 3 assemblies
In the future, Microsoft will no doubt make this process easier and include ASP.NET MVC 3 tooling for Azure deployment.
IaaS, PaaS and SaaS Terms Clearly Explained and Defined
I love it when semi-abstract terms and sometimes confusing technologies have an easy way to describe them in a terse/quick manner. I found a great explanation from the "Windows Azure Platform: Cloud Development Jump Start" series in the first video (the video series is available both on iTunes and Zune and it covers the latest 1.3.x version of the SDK). The series is provided free of charge. Below are a couple of slides from the presentation:
Easy to Understand one-word descriptions of each term (i.e. IaaS means host)
Data Visualization of Vendor Management Responsibility of Services
I really like the last slide, because it clearly breaks down hierarchy of software layers into what is each vendor service (IaaS, PaaS and SaaS) is responsible for and what you are responsible for. Also in the video one of the speakers (Manu Cohen-Yashar) breaks each service down by the "unit you are gaining" from the service:
- IaaS - the unit you are gaining is a computer/server. Therefore IaaS is essentially a "physical server box". An example of this would be going to RackSpace or SoftLayer and leasing a physical box from them. The vendor manages the networking, hard drives (if they fail), hardware of the box, virtualization O/S (if the box is virtualized). You can remote desktop to the box and you manage everything else (shown in the screenshot above). Windows Azure provides IaaS in the form the VM Role (you upload a Windows Server 2008 R2 image and manage the server yourself).
- PaaS - the unit you are gaining is an application/framework. Therefore PaaS is a "hosted application/framework/tools that you can leverage to build something on. That application is configured on IIS/SQL Server etc and runs on a hardware/virtual system that the vendor manages. An example of PaaS would be Windows Azure (excluding the VM Role) services like web role, worker role, Reporting Services etc.
- SaaS - the unit you are gaining is business functionality. For example, Gmail is a type of a SaaS mail provider because you don't have to manage any service yourself and its all done by the vendor (Google in this example).
I really like the clear examples and distinct definitions that are made in the videos. Hopefully, you can use these examples in explaining this to your boss or a non-technical person. I highly recommend you check out the "Windows Azure Platform: Cloud Development Jump Start" series if you are interested on learning about Azure or gain general information on the cloud.