Portable Server Drive Performance

I was wondering what the performance of the RAID5 setup in my new portable server and I found this nifty little HD benchmark tool at http://www.datamarck.com/.

I ran it and it showed that the average throughput of my RAID5 array was about 77 MB/s. In contrast, my older desktop machine ran about 46 MB/s and my older laptop was a measly 36 MB/s. Not too shabby.

I haven't done any video or CPU benchmarking yet but I wonder how the video performs with Win2k8 server and the nVidia 8800M single card.

I'm currently installing two instances of Win2k3 standard under Hyper-V to play around with xfering IIS metabase and some other automated tasks and the beauty of this is that I can take snapshots and restore those snapshots while playing around.

But I did find a bummer about Hyper-V. You can bind it to a NIC as long as it is not a wireless NIC. I found that rather strange that there would be a differentiation of interfaces as both are ethernet devices. Go figure.

Another thing I noticed is that the network card is reported by Win2k8 as a 10 GB/s card which I find hard to believe. It's prolly just a reporting err. No biggie as I don't have 10 GB infrastructure in the house.

All in all, pretty nifty for a portable device.

Hyper-V is nifty!

I haven't had much of a chance to play with Hyper-V but I did a little bit of fiddling tonight. I loaded up a VM with the latest Ubuntu and got it up and running and all is well with the world.

I made a snapshot and then did the classic n00b mistake of:

`cd / && rm -rf *`

Yeah. Not a good thing to do and yes, it destroyed the installation as expected.

I applied the snapshot that I had taken a few minutes before and within a few seconds my Ubuntu was up and running exactly the way it was when I took the snapshot. How stinkin' cool is that?

I have a need to build out an installation process that builds installs and configures a Win2k3 machine with users, applications, IIS, vdirs, etc and this will be the perfect environment to test. Build up a base machine, get it configured and snap shot.

Then I can try the process and tweak as much as a I need by restoring the snapshot. Too stinkin' cool!

The Beast!

I got my new laptop in today. It is a rebadged Clevo 901 sporting a Quad Xeon 2.83 GHz proc, RAID5 with 3 200 gig spindles at 7200 RPM and 4 gig of RAM. I can upgrade to 8 gig of RAM when the memory becomes cheap enough. It also has a nVidia 8800M video card.

The sucker is huge with a 17 inch wide screen display (1680x1050) and is almost three inches thick. It's definately not a laptop but a desktop replacement. In my case, it is a mobile server.

I was able to order it without an OS as I can get a license through my employer. Today I loaded up Win2k8 64-bit Standard (no need for Enterprise as I am not clustering my "laptop." I also loaded SQL Server 2005 64-bit and then VS2008 Team Suite Testing Edition. On top of that I threw in Office 2007 and spent a lot of the day twiddling with installations and settings.

After using the Win2k8 interface for a while it seems that I am going to just have to bite the bullet and get used to the Vista UI as Win2k8 uses it as well. *sigh* It was good knowing ya, XP Pro.

But, for what it's worth, Win2k8 did seem pretty damn snappy. Of course, it might have something to do with the quad core Xeon and 4 gig of RAM and a 512 meg video card. :)

I also enabled IIS and Hyper-V after I enabled VMM and NX in the machine's BIOS configuration. I'm stoked about Hyper-V but I've already found out one bummer with Hyper-V and that is enabling the Hyper-V role disables the ability to hibernate the machine. I guess I can understand why. If you are hosting a bunch of VMs with Hyper-V you really don't need the ability to hibernate the server. So, it's a minor irritation that but is all.

I did enable some eye-candy such as smoothing fonts and thumbnails instead of icons. That is one Vista thing that I really liked. That and the use of c:\users instead of the iritating "documents and settings." If you are a keyboard commando like me, those spaces get to be damned annoying. Of course, Microsoft still uses spaces in a lot of their subdirs so the issue isn't totally settled.

And I learned something from my previous Vista Ultimate experience on my older laptop (15.4 inch more conventional laptop, not a UberPowah machine like my current big rig) and that is to avoid the Microsoft Unix tools and just run with the GNU Unix Tools for Win32. I found a lot of unix commands lacking with the MS implementation and it is easier to unzip the GNU tools and add the subdir to the path. Of course, I can't compile any X code like I supposedly could with the MS unix services but I didn't need to compile any anyway so it is no great loss.

Another side effect is that I had to buy a new laptop bag and this time instead of a backpack I went with a Swiss Army gear rolling laptop bag that let's me put both my new MobileServer and Acer Ferrari in the bag at the same time and roll it along. With both machines and accessories (power bricks, extra battery, typical junque) that bag feels like it weighs in at 45 - 50 pounds. That would have been a bit tough to carry around an airport. I spent more than I wanted to but I think it'll be worth it.

On other good news fronts last week work gave me two physical boxen to actually build up my LT environment and I think that I'll actually be doing some dry runs tomorrow measuring up initial testing capacity with JMeter testing against FTP chunking my GPG encrypted files for the service to munch on. Finally!

I've been doing a lot of hopping around at work but my goal is still the same, to get daily automagic load tests running and in place with our continuous integration work flow.

Transaction Transaction, what's your action?

I noticed something this morning as I was going over my SP that I wrote on Thursday night. I realized that I needed to also gonkulate how many transactions had failed. Seems simple enough, right? If a page (or dependent request) fails inside a transaction the whole transaction gets marked as a failure, right? That's the way that LoadRunner handles the transaction issue.

In LoadRunner I would normally leave the determination of the status of the transaction by using LR_AUTO. There were a few times that I would explicitly call EndTransaction with a LR_FAIL.

I realized looking over the schema of the Load Test Repository Store that there does not appear to be any logging of failures on the transaction level. The page level? Sure. Not a problem. LoadTestMessage table contains those goodies for us but I cannot find a way to link LoadTestMessage with WebLoadTestRequestMap. Going back over some of the online documentation for running VS2008 load tests it appears that the tool itself does not count transaction failures and only counts failed pages.

Yeah... I'm not too happy about that. With my analysis I want to see what transactions failed as I might have multiple page hits to a single transaction. Hopefully this gets fixed in a future version of the MS LT tool.

Collecting daily LT Metrics

Last night I wrote a stored procedure to automagically collect the latest LT results and insert the values into a database that I created on my local SQL Server 2005 Express.

This SP finds the latest LT that has been run, crunches the numbers and inserts them into two tables, LoadTestHistory and LoadTestHistoryDetails. I used the GUID generated by VS2008 in the Load Test Repository Store as the primary/foreign key between the two tables.

It's a bit rough and could probably use some more polish:

   1:  set ANSI_NULLS ON

   2:  set QUOTED_IDENTIFIER ON

   3:  GO

   4:  ALTER PROCEDURE [dbo].[UpdateLoadTestHistory]

   5:  AS

   6:  BEGIN

   7:   

   8:    SET NOCOUNT ON;

   9:   

  10:    declare @AlreadyExists    as int

  11:   

  12:    declare @latestLoadTest   as int

  13:    declare @loadTestGUID     as nvarchar(36)

  14:    declare @StartTime        as datetime

  15:    declare @EndTime          as datetime

  16:    declare @Duration         as int

  17:   

  18:    select @latestLoadTest = max(LoadTestRunId)

  19:    from LoadTest.dbo.LoadTestRun

  20:    where EndTime is not null

  21:   

  22:    select @loadTestGUID   = RunId,

  23:           @StartTime      = StartTime,

  24:           @EndTime        = EndTime,

  25:           @Duration       = RunDuration

  26:    from LoadTest.dbo.LoadTestRun

  27:    where LoadTestRunId = @latestLoadTest

  28:   

  29:    select @AlreadyExists = count(*)

  30:    from dbo.LoadTestHistory

  31:    where LoadTestGUID = @loadTestGUID

  32:   

  33:    if (@AlreadyExists = 0) 

  34:      begin

  35:   

  36:        begin transaction

  37:   

  38:        begin try

  39:   

  40:          insert into dbo.LoadTestHistory

  41:          values (@LoadTestGUID,

  42:                  @StartTime,

  43:                  @EndTime,

  44:                  @Duration)

  45:   

  46:          SELECT @loadTestGUID                                as LoadTestGUID,

  47:                 WebLoadTestTransaction.TransactionName, 

  48:                 LoadTestTransactionSummaryData.Average, 

  49:                 STDEV(LoadTestTransactionDetail.ElapsedTime) as StdDev, 

  50:                 LoadTestTransactionSummaryData.Minimum, 

  51:                 LoadTestTransactionSummaryData.Maximum, 

  52:                 LoadTestTransactionSummaryData.Percentile90, 

  53:                 LoadTestTransactionSummaryData.Percentile95, 

  54:                 LoadTestTransactionSummaryData.TransactionCount

  55:          INTO   #tmpLoadTestHistory

  56:          FROM   LoadTest.dbo.LoadTestTransactionDetail INNER JOIN

  57:                 LoadTest.dbo.WebLoadTestTransaction ON LoadTest.dbo.LoadTestTransactionDetail.LoadTestRunId = LoadTest.dbo.WebLoadTestTransaction.LoadTestRunId AND 

  58:                 LoadTest.dbo.LoadTestTransactionDetail.TransactionId = LoadTest.dbo.WebLoadTestTransaction.TransactionId INNER JOIN

  59:                 LoadTest.dbo.LoadTestTransactionSummaryData ON LoadTest.dbo.WebLoadTestTransaction.LoadTestRunId = LoadTest.dbo.LoadTestTransactionSummaryData.LoadTestRunId AND 

  60:                 LoadTest.dbo.WebLoadTestTransaction.TransactionId = LoadTest.dbo.LoadTestTransactionSummaryData.TransactionId

  61:          WHERE  (LoadTestTransactionDetail.LoadTestRunId = @latestLoadTest)

  62:          GROUP BY WebLoadTestTransaction.TransactionName, LoadTestTransactionSummaryData.Average, LoadTestTransactionSummaryData.Minimum, 

  63:                   LoadTestTransactionSummaryData.Maximum, LoadTestTransactionSummaryData.Percentile90, LoadTestTransactionSummaryData.Percentile95, 

  64:                   LoadTestTransactionSummaryData.TransactionCount

  65:          ORDER BY WebLoadTestTransaction.TransactionName

  66:   

  67:          insert dbo.LoadTestHistoryDetails

  68:          select LoadTestGUID, TransactionName, Average, StdDev, Minimum, Maximum, Percentile90, Percentile95, TransactionCount

  69:          from #tmpLoadTestHistory

  70:   

  71:          drop table #tmpLoadTestHistory

  72:   

  73:          commit transaction

  74:        end try

  75:   

  76:        begin catch

  77:          rollback transaction

  78:        end catch

  79:      end

  80:  END

And it works like a champ!

I tested it with this bit of SQL:

   1:  use LoadTestResults

   2:  go

   3:   

   4:  truncate table dbo.LoadTestHistory

   5:  truncate table dbo.LoadTestHistoryDetails

   6:  go

   7:   

   8:  dbo.UpdateLoadTestHistory

   9:  go

  10:   

  11:  select dbo.LoadTestHistory.LoadTestGUID, 

  12:         StartTime, 

  13:         EndTime, 

  14:         Duration, 

  15:         TransactionName, 

  16:         Average, 

  17:         StdDev,

  18:         Minimum,

  19:         Maximum,

  20:         [90th],

  21:         [95th],

  22:         TransactionCount

  23:  from dbo.LoadTestHistory, dbo.LoadTestHistoryDetails

  24:  where dbo.LoadTestHistory.LoadTestGUID = dbo.LoadTestHistoryDetails.LoadTestGUID

And the results come out for all to enjoy:

   1:  LoadTestGUID                         StartTime               EndTime                 Duration TransactionName            Average          StdDev            Minimum Maximum 90th  95th  TransactionCount

   2:  0ebdf821-454f-4c50-8e3a-a82a291adb97 2008-07-03 16:21:33.280 2008-07-03 16:31:33.280 600      someTransaction.Details    3.4155221238938  0.583565372475793 1.526   4.952   4.161 4.35  226

   3:  0ebdf821-454f-4c50-8e3a-a82a291adb97 2008-07-03 16:21:33.280 2008-07-03 16:31:33.280 600      someTransaction.FirstHit   3.42468584070796 0.79565610855916  1.636   11.888  4.053 4.285 226

   4:  0ebdf821-454f-4c50-8e3a-a82a291adb97 2008-07-03 16:21:33.280 2008-07-03 16:31:33.280 600      someTransaction.LookupName 3.46306194690266 0.606631255863035 2.087   5.15    4.285 4.411 226

Retrieving Standard Deviation for Transaction Response times with VS2008

One of my plans for the firm for which I work is to integrate a nightly automagic LT with the Continuous Integration effort. I want to be able to automagically compare results of the previous days results and see if anything is amiss. One of the tests I would like to apply is the two population mean test which requires having the average response time, standard deviation and the number of transactions.

VS2008 doesn't provide all of these statistics by default in their LT summary. LoadRunner includes this information by default in their summary results and extracting the values are quite simple if you publish the results to an Excel Spreadsheet.

It's not so quite as simple as that with VS2008 but it can be done with a little effort.

I have configured my LT to record individual metrics to the LoadTest Data Store so that I can query the tables and extract the information that I want. In the examples below I have identified the LoadTestRunId for a given run where I have plenty of metrics to number crunch. In my nightly build and test scenario I envision a simple query to pull the max(LoadTestRunId) to get the latest run to run queries to extract the required information.

I wrote this query today to get the information that I wanted:

   1:  SELECT WebLoadTestTransaction.TransactionName, 

   2:         LoadTestTransactionSummaryData.Average, 

   3:         STDEV(LoadTestTransactionDetail.ElapsedTime) AS StdDev, 

   4:         LoadTestTransactionSummaryData.Minimum, 

   5:         LoadTestTransactionSummaryData.Maximum, 

   6:         LoadTestTransactionSummaryData.Percentile90, 

   7:         LoadTestTransactionSummaryData.Percentile95, 

   8:         LoadTestTransactionSummaryData.TransactionCount

   9:  FROM   LoadTestTransactionDetail INNER JOIN

  10:                    WebLoadTestTransaction ON LoadTestTransactionDetail.LoadTestRunId = WebLoadTestTransaction.LoadTestRunId AND 

  11:                    LoadTestTransactionDetail.TransactionId = WebLoadTestTransaction.TransactionId INNER JOIN

  12:                    LoadTestTransactionSummaryData ON WebLoadTestTransaction.LoadTestRunId = LoadTestTransactionSummaryData.LoadTestRunId AND 

  13:                    WebLoadTestTransaction.TransactionId = LoadTestTransactionSummaryData.TransactionId

  14:  WHERE  (LoadTestTransactionDetail.LoadTestRunId = 33)

  15:  GROUP BY WebLoadTestTransaction.TransactionName, LoadTestTransactionSummaryData.Average, LoadTestTransactionSummaryData.Minimum, 

  16:                    LoadTestTransactionSummaryData.Maximum, LoadTestTransactionSummaryData.Percentile90, LoadTestTransactionSummaryData.Percentile95, 

  17:                    LoadTestTransactionSummaryData.TransactionCount

  18:  ORDER BY WebLoadTestTransaction.TransactionName

The QBE table entries look this this:


And I get the results I want:

   1:  TransactionName            Average             StdDev               Minimum Maximum 90th  95th  TransactionCount

   2:  someTransaction.Details    0.47510353043101411 0.090671477518885088 0.416   3.695   0.522 0.552 4617

   3:  someTransaction.FirstHit   0.44759085986571417 0.077896506427873255 0.384   3.956   0.502 0.526 4617

   4:  someTransaction.LookupName 0.51917002382499466 0.082360597958219192 0.45    2.844   0.577 0.605 4617

Huzzah!

I'll end up setting up an automagic comparison routine. A number of years ago I wrote a C# class for doing statistical tests so I'll probably end up using that for my left/right comparisons and number crunching.

It's finally been ordered!

I got news today that my new laptop has finally been official ordered. Here is the basic specs as it stands right now:

Quad Core Xeon 2.83 GHz proc
4 gig RAM
17 inch display at 1680x1080
RAID5 with 3 drives spinning 7200 RPM with 200 gig each for 400 gig of filespace
nVidia 8800M 512 meg video card (only 1 but it can support 2)

I was able to order it without an operating system so I will be installing either Win2k3 or Win2k8 for my load testing needs.

Hopefully tomorrow I'll find out when it is arriving. It's like Christmas in July!