Wednesday, May 11, 2011

Performance Testing in Azure Dev Fabric

I'm running an application at Azure that runs in Full IIS Mode with Dev Fabric version 1.4 on x64 Windows 7. I set out to run some performance tests, since Memory\Page Faults/sec is high during a few scenarios in production.

Unfortunately, the Visual Studio tooling has not caught up to Azure, so it takes a bit of hacking and Powershell goodness to get the data you need. On the positive side, the Visual Studio tooling is just a UI for the same command line tools we're about to use. Plus, analysis happens exactly the same way that you're used to in Visual Studio.

This tutorial assumes you want Memory and Performance metrics using instrumented assemblies. If you don't want/need Memory, skip the relevant bullet.

NB: Be sure you're using the 64-bit versions of these commands if that is your platform. Also run Powershell as an Administrator if you use UAC. Also, oddly, vsperfcmd doesn't seem to return from the ISE, so you have to use good ol' powershell.exe.

  • Install x64 Performance Tools for Visual Studio 2010 SP1
  • (Optional) Enable CLR memory profiling (allocations, or allocations and lifetimes (shown))
    • & "C:\Program Files (x86)\Microsoft Visual Studio 10.0\Team Tools\Performance Tools\x64\VSPerfCLREnv.cmd" /globaltracegclife
    • (You may have to reboot for the command to take effect)
    • Important: When you're done, be sure to run the same command with /Off and /GlobalOff switches, otherwise EVERY .NET app on your system will be publishing these metrics and run like dog crap.
  • Build your application
  • Instrument your assemblies in-place
    • $vsinstr = "C:\Program Files (x86)\Microsoft Visual Studio 10.0\Team Tools\Performance Tools\x64\vsinstr.exe"
      dir ".\MyCloudApp\bin\Debug\MyCloudApp.csx\roles\MyWebRole\approot\bin\MyNamespace*.dll" | foreach-object {
          & $vsinstr $_.FullName;
  • Start the performance profiler; it seems this must start before your application (Note I'm naming the .vsp output with a timestamp for easier scripting and history tracking)
    • $vsperfcmd = "C:\Program Files (x86)\Microsoft Visual Studio 10.0\Team Tools\Performance Tools\x64\vsperfcmd.exe"
      $date = get-date -F yyyyMMddHHmmss
      &$vsperfcmd /START:TRACE /OUTPUT:perf_$date.vsp /user:Everyone
  • Start your application via the Azure cloud tool csrun.exe
    • $csrun = "C:\Program Files\Windows Azure SDK\v1.4\bin\csrun.exe"
      &$csrun /devstore:start
      &$csrun .\MyCloudApp\bin\Debug\MyCloudApp.csx .\MyCloudApp\bin\Debug\ServiceConfiguration.cscfg
  • Wait for the appropriate process to start and grab its PID. In my case, since I'm using Full IIS Mode and interested in capturing metrics from RoleEntryPoint and deeper, that process is WaIISHost.exe. If you are running a worker role, that process is WaWorkerHost.exe; "old" web roles use WaWebHost; web sites running under Full IIS Mode use the regular w3wp.exe (though getting the right app pool may be more difficult).
    • while(-not $process) {
          $process = Get-Process | where { $_.ProcessName -ieq "waiishost" };
          start-sleep 0.1;
  • When ready, start capturing instrumented data
    • & $vsperfcmd /processOn:$($process.Id)
  • When done, disconnect and shut down
    • & $vsperfcmd /processOff:$($process.Id)
    • & $vsperfcmd /shutdown

Now open the .vsp and—there you have it—a performance report generated from code running in Azure dev fabric!


1 comment:

Adam said...

That's *kind of* awesome - to be able to leverage the tools we all know and love to diagnose and optimize is incredibly powerful. It also sounds as though the UI integration can't be far behind, but in the meantime, you're a thought leader! Very nice job, I look forward to trying this out myself.