Asynchronous Invocation of a Load Test from a Continuous Integration System

Why the fancy pants title? A Microsoft recruiter asked for my resume for some load testing related stuff and my blog is on my resume so I figured I needed some high falutin' titles. Hopefully they don't read anything but the titles. :-)

Anyway...

I'm getting to the point where I need to be able to invoke a load test after the nightly build and automagically run a load test. Now, I know, right now you're saying to yourself, "Hey, a nightly build doesn't constitute a continuous integration system!" And I agree, however, "Continuous Integration System" sounds much more impressive than just "nightly build." ;-)

Back to the important stuff: How am I going to invoke a Load Test on another machine after the nightly build is done? Right now I am working everything out with that super-duper handy utility, psexec from the UberBrains at SysInternals (now part of Microsoft).

I have recently build a stand alone server that I had installed SQL Server 2005, VS2008 TSTE and my custom LoadTestResults database where I store summary information from runs that I will be reporting from. On that server which I will refer to as my Load Test Controller, I setup a machine account user account by the name of TestRunner. During my trial and errors I found that I had to add TestRunner to the Administrators group but I need to go back and play around with that some to see if there is a more secure way to invoke the nightly automagic load test.

On the Load Test Controller I have two batch files: trigger.bat and loadtest.bat.

trigger.bat:

8<----------------------------
@echo off
echo Cry havoc and let slip loose the dogs of loadtest!
start C:\LoadTest\Trigger\loadtest.bat
---------------------------->8

The start command allows the async execution of the second batch file loadtest.bat, where the load test is really started.

loadtest.bat:

8<----------------------------
call "C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\vcvarsall.bat"
cd "C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\IDE\"
MSTest.exe /testcontainer:c:\LoadTest\LoadTests\MyLoadTest\MyLoadTest\MyReallyCoolLoadTest.loadtest
---------------------------->8

I can call trigger.bat from psexec with any automated process that can shell out to the command line or execute a batch file. Here is an example of starting a load test on the Load Test Controller using psexec from the command line:

8<----------------------------
C:\>psexec \\loadtestbox -u testrunner -p testrunner cmd /c c:\loadtest\trigger\trigger.bat

PsExec v1.92 - Execute processes remotely
Copyright (C) 2001-2007 Mark Russinovich
Sysinternals - www.sysinternals.com


Cry havoc and let slip loose the dogs of loadtest!
c:\loadtest\trigger\trigger.bat exited on loadtestbox with error code 0.
---------------------------->8

Even though psexec exit back to the command line, the load test is actually running on my Load Test Controller:



I was having problems getting results to be automagically saved to the Load Test Results Repository. I figured the SQL connection string was stored in the VS2008 solution but I had to open up VS2008 on my Load Test Controller machine and manually add the connection string for my TestRunner profile. After that, the LoadTest database is updated with each execution. w00t!

Now it's off to add the automagic reporting SP calls via osql.

Extracting Perfmon data from the VS2008 Load Test Repository

Ok. You are performing load tests with VS2008 TSTE and you've enabled the logging of perfmon metrics in your test by tweaking the "Timing Details Storage" property of the Run Settings in your load test. Every 5 seconds the perfmon metrics you specified are being stored to the load test repository and after the run you get some metrics. That's great and all but what if you want more detailed statistics such as average AND standard deviation.

Or better yet! You want at that data to generate graphs of the metrics over the duration of the test. Sure, you can do this by firing up good 'ol Perfmon and logging the values or you can just get them from your load test repository with some SQL. Why have Perfmon logging when VS2008 already does this for you?

I have a need for just such an action and I've been working on the SQL statements to extract the data I need. It's not complete, but this is a major step forward to allowing me to finish up the SPs I want to collect the data dynamically for consumption by managers and other QA folks.

Looking at the handy dandy schema of the load test repository (located here for all to gander) I have created some SQL statements that take care of the leg work for me.

   1:  SELECT LoadTestPerformanceCounterInstance.InstanceId, 

   2:         LoadTestPerformanceCounterCategory.CategoryName, 

   3:         LoadTestPerformanceCounter.CounterName, 

   4:         LoadTestPerformanceCounterInstance.InstanceName, LoadTestPerformanceCounterCategory.MachineName, 

   5:         AVG(LoadTestPerformanceCounterSample.ComputedValue) AS AverageValue,

   6:         stdev(LoadTestPerformanceCounterSample.ComputedValue) as StdDev,

   7:         count(LoadTestPerformanceCounterSample.ComputedValue) as Count

   8:  FROM  LoadTestPerformanceCounterCategory INNER JOIN

   9:                 LoadTestPerformanceCounter INNER JOIN

  10:                 LoadTestPerformanceCounterInstance ON LoadTestPerformanceCounter.LoadTestRunId = LoadTestPerformanceCounterInstance.LoadTestRunId AND 

  11:                 LoadTestPerformanceCounter.CounterId = LoadTestPerformanceCounterInstance.CounterId ON 

  12:                 LoadTestPerformanceCounterCategory.CounterCategoryId = LoadTestPerformanceCounter.CounterCategoryId AND 

  13:                 LoadTestPerformanceCounterCategory.LoadTestRunId = LoadTestPerformanceCounter.LoadTestRunId INNER JOIN

  14:                 LoadTestPerformanceCounterSample ON LoadTestPerformanceCounterInstance.LoadTestRunId = LoadTestPerformanceCounterSample.LoadTestRunId AND 

  15:                 LoadTestPerformanceCounterInstance.InstanceId = LoadTestPerformanceCounterSample.InstanceId

  16:  WHERE (LoadTestPerformanceCounter.LoadTestRunId = 83)

  17:  GROUP BY LoadTestPerformanceCounterCategory.CategoryName, LoadTestPerformanceCounter.CounterName, LoadTestPerformanceCounterInstance.InstanceName, 

  18:                 LoadTestPerformanceCounterInstance.InstanceId, LoadTestPerformanceCounterCategory.MachineName

  19:  ORDER BY LoadTestPerformanceCounterInstance.InstanceId

In the above example, the reference to LoadTestRunId is hard coded to a previous run that I have in my results repository. For all my other SPs I use the GUID of the test and will convert the above SQL statement to reference GUID all in good time. The query above returns results like this:

   1:  0     .NET CLR Memory     Gen 0 heap size                             w3wp     10.0.0.11     944824777.209431     185400315.203616     721

   2:  1     .NET CLR Memory     Large Object Heap size                      w3wp     10.0.0.11     86994000.5298197     29625345.9051784     721

   3:  2     .NET CLR Memory     # Gen 0 Collections                         w3wp     10.0.0.11     132.608876560333     69.5129454689118     721

   4:  3     .NET CLR Memory     Gen 2 heap size                             w3wp     10.0.0.11     197119053.001387     78824416.0154383     721

   5:  4     .NET CLR Memory     Allocated Bytes/sec                         w3wp     10.0.0.11     63521791.4160888     91335026.9218956     721

   6:  5     .NET CLR Memory     # Gen 2 Collections                         w3wp     10.0.0.11     29.5104022191401     12.1351287581502     721

   7:  6     .NET CLR Memory     Promoted Memory from Gen 0                  w3wp     10.0.0.11     42540355.2815534     13356693.0175676     721

   8:  7     .NET CLR Memory     # Induced GC                                w3wp     10.0.0.11     0                    0                    721

   9:  8     .NET CLR Memory     Gen 0 Promoted Bytes/Sec                    w3wp     10.0.0.11     418557.084366331     1120132.98732717     721

  10:  9     .NET CLR Memory     Promoted Memory from Gen 1                  w3wp     10.0.0.11     21740368.0721221     25126733.8606902     721

  11:  10    .NET CLR Memory     # GC Handles                                w3wp     10.0.0.11     3750.73370319001     216.191892870829     721

  12:  11    .NET CLR Memory     # Gen 1 Collections                         w3wp     10.0.0.11     70.0693481276005     35.4698332819129     721

  13:  12    .NET CLR Memory     Gen 1 heap size                             w3wp     10.0.0.11     62394980.9237171     31034211.4233198     721

  14:  13    .NET CLR Memory     Finalization Survivors                      w3wp     10.0.0.11     8217.1040221914      3024.02383909951     721

  15:  14    .NET CLR Memory     Promoted Finalization-Memory from Gen 0     w3wp     10.0.0.11     11415345.6768377     11434973.2174913     721

  16:  15    .NET CLR Memory     % Time in GC                                w3wp     10.0.0.11     3.48545611573799     6.73310861817717     721

The first column is the InstanceId of the perfmon counter and the information for the counter is split up into three separate columns (CategoryName, CounterName and InstanceName) but the column that ties them all together is the InstanceId.

Ok. That's fine and dandy but what about graphing the metrics over the duration of the load test?

Well, that's another SQL query I wrote that is in the alpha stage of development. In the above example I displayed 15 out of 466 metrics that were logged. Let's choose to graph the .NET CLR Memory Gen 2 heap size for w3wp. This particular metric happens to be InstanceId #3 this time around and we can use it in the SQL statement below to extract the data that can eventually be used to graph the results.

   1:  SELECT LoadTestPerformanceCounterInstance.InstanceName,

   2:         LoadTestPerformanceCounterSample.ComputedValue

   3:  FROM  LoadTestPerformanceCounterInstance INNER JOIN

   4:        LoadTestPerformanceCounterSample ON LoadTestPerformanceCounterInstance.LoadTestRunId = LoadTestPerformanceCounterSample.LoadTestRunId AND 

   5:        LoadTestPerformanceCounterInstance.InstanceId = LoadTestPerformanceCounterSample.InstanceId

   6:  WHERE (LoadTestPerformanceCounterInstance.LoadTestRunId = 83) AND (LoadTestPerformanceCounterInstance.InstanceId = 3)

   7:  ORDER BY LoadTestPerformanceCounterSample.TestRunIntervalId

This query yields the results of:

   1:  w3wp    0

   2:  w3wp    0

   3:  w3wp    96

   4:  w3wp    3231264

   5:  w3wp    2.170597E+07

   6:  w3wp    2.170597E+07

   7:  w3wp    2.170597E+07

   8:  w3wp    2.976022E+07

   9:  w3wp    2.976022E+07

  10:  w3wp    2.976022E+07

  11:  w3wp    2.976022E+07

  12:  w3wp    4.127677E+07

  13:  w3wp    4.127677E+07

  14:  w3wp    4.127677E+07

and so on and so on until all 722 metrics are displayed. We can take those 722 metrics and copypasta into Excel and generate a quick and dirty graph:



Viola! Quick and dirty graph from Excel!

I'll need to change the column output to show more details on the metrics to display all three columns to better identify the perfmon metric and also display the time that the metric was taken.

After I do all that I'll need to write some code to generate graphs on the fly. I'm thinking that this project on SourceForge will fit the bill nicely and there seems to be a bunch of examples out on the 'net how to use ZedGraph to it's fullest.

Oh yeah! Almost forgot. I need to double check the results between the first query and the individual information for InstanceId 3 metric (the .NET CLR Gen 2 heap size for w3wp). The first query output says the average value is 197119053.001387. Take the average of output that I did the copypasta into Excel gonkulates to an average of 197119053.4. The difference between the two? A measly 0.355. I'm thinking that is good 'nuff for government work.

Now I just need to finish up the queries and get them into SPs and integrate them with the other code that I've written that does automagic before/after comparison of transaction response times (more on that at a later date).

Coding like Ike and Tina...

I'm working on a WebTestRequest that changes an address and in the post back the POST data contains a bunch of information that is displayed in the HTML and I need to extract data from text boxes, check boxes and drop down lists. Extracting the textboxes and checkboxes are pretty simple, it is the drop down boxes that are a pain.

The .NET HTML parser treats the HtmlTags that reside inside the SELECT tag separately so I cannot just get the attributes for the select and then slice and dice and get the default selected value for the drop in.

I tried to be "elegant" in my solution and use the HTML Agility Pack and a XPath pulled from FireBug but alas, there was no love to be found. I'm not sure what the exact problem is other than me being a total XPath n00b (I normally use an open source Java project to extract XPaths from XML docs) but something was amiss. After wasting several hours trying to get the elegant solution I decided to be like Ike and Tina and just brute force the solution of identifying the default selected value for the drop down lists. I populate the Context of the WebTestRequest with all the values that are extracted and just reference them by element name in the address change POST ala the HIDDEN1$ that VS2008 does for hidden field extraction and it seems to work pretty good.

Here is the extraction code that I wrote to pull the fields that I needed to reference in the Context:

   1:  void ExtractCustomerDataToContext(object sender, ExtractionEventArgs e) {

   2:    Dictionary<string, string> customerData = new Dictionary<string, string>();

   3:    string lastDropDown = null;

   4:    foreach (HtmlTag tag in e.Response.HtmlDocument.HtmlTags) {

   5:      string type = tag.GetAttributeValueAsString("type");

   6:      if (type != null) {

   7:        lastDropDown = null ;

   8:        if (type == "text" || type == "checkbox") {

   9:          if (tag.GetAttributeValueAsString("checked") != null) {

  10:            if (tag.GetAttributeValueAsString("checked") == "checked") {

  11:              customerData.Add(tag.GetAttributeValueAsString("name"), "on");

  12:            } else {

  13:              customerData.Add(tag.GetAttributeValueAsString("name"), "off");

  14:            };

  15:          } else {

  16:            customerData.Add(tag.GetAttributeValueAsString("name"), tag.GetAttributeValueAsString("value"));

  17:          };

  18:        };

  19:      } else {

  20:        if (tag.GetAttributeValueAsString("class") != null) {

  21:          if (tag.GetAttributeValueAsString("class") == "dropDownListDefault") {

  22:            lastDropDown = tag.GetAttributeValueAsString("name");

  23:          };

  24:        } else {

  25:          if (tag.GetAttributeValueAsString("selected") != null) {

  26:            if (lastDropDown != null) {

  27:              customerData.Add(lastDropDown, tag.GetAttributeValueAsString("value"));

  28:            };

  29:          };

  30:        };

  31:      };

  32:    };

  33:    foreach (KeyValuePair<string, string> kvp in customerData) {

  34:      if (this.Context.ContainsKey("CUSTOMER_UPDATE_DATA." + kvp.Key)) {

  35:        this.Context.Remove("CUSTOMER_UPDATE_DATA." + kvp.Key);

  36:      };

  37:      if (kvp.Value == null) {

  38:        this.Context.Add("CUSTOMER_UPDATE_DATA." + kvp.Key, "");

  39:      } else {

  40:        this.Context.Add("CUSTOMER_UPDATE_DATA." + kvp.Key, kvp.Value);

  41:      };

  42:    };

  43:  }

So, if the previous page HTML has a text field by the name of "txtCustomerAccountNumber" I can simply add that information to the post with:

   1:  changeAddressRequestBody.FormPostParameters.Add("txtCustomerAccountNumber", this.Context["CUSTOMER_UPDATE_DATA.txtCustomerAccountNumber"].ToString());

As long as the information was displayed via HTML in the prior hit, I should be good to go.

Revisiting CSV Data Binding in VS2008 Coded Web Tests

I've been going through the WinDBG labs located here. Tess has done a great job with the labs and done all the hard work, so w00t for her!

I needed to add another .CSV data file for my main VUser that I am currently working on so I did a copypasta of a pre-existing DataBinding and DataSource. When I first created the coded web test I had added so much custom code that I deleted the web test and left only the coded web test. The fear was that I might accidently re-gen the code from the web test and over write my work. That in and of itself shouldn't be too bad since we all know to use a code repository, right? Of course we do.

Anyhoo, I made the code changes and was greeted by an error when my file couldn't be loaded. Huh? How rude!

I finally got it to work, but I wanted to note the way .CSV data binding works in a coded web test in greater detail just to remind myself next time this happens. There are plenty of examples of binding a .CSV file to a web test, but few on a pre-existing coded web test. So, here goes!

First we have to have a DataSource declaration that takes five parameters for a .CSV file:

1: dataSourceName
This is the name we are giving the DSN for our .CSV file.
2: providerName
In my case for a .CSV file I am using "Microsoft.VisualStudio.TestTools.DataSource.CSV"
3: connectionString.
For my use, this is the path with escaped back-whacks to the .CSV file. In my case, it is "c:\\data\\streetNames.csv"
4: DataBindingAccessMethod
For my use, I want a random selection of data from the .CSV file so I am using "Microsoft.VisualStudio.TestTools.WebTesting.DataBindingAccessMethod.Random"
5: tableName
I thought that I could provide my own table name to reference the .CSV file but that didn't work and was forced to use the form of #csv, like this: "streetNames#csv". When I tried to use the name "streetNameTable" I would get back an error of "cannot find streetTableName.txt" from VS2008. But, the #csv form works, so no biggie.

Here is what I am using for my entire Data Source:

   1:    [DataSource("streetNameDataSource",

   2:              "Microsoft.VisualStudio.TestTools.DataSource.CSV",

   3:              "C:\\Data\\streetNames.csv",

   4:              Microsoft.VisualStudio.TestTools.WebTesting.DataBindingAccessMethod.Random,

   5:              "streetNames#csv")]

Next, we have to define what column we are pulling from the .CSV file. In my case, it is a single column CSV file but we still have to have a column name defined. For my CSV file I am using the form of:

8<---------------------------
streetNames
"ABBINGTON"
"ALDRICH"
"ALEXANDER"
"ALICE"
"ALLISON"
"ALMOND"
"ALTA"
"AMHERST"
--------------------------->8

These are all the street names in Grover's Mill, NJ as reported by Melissa Data.

The DataBinding declaration that I am using takes four parameters.

1: dataSourceName
This is the DSN that we previously defined.
2: tableName
This is the tableName that we defined in the form of #csv.
3: columnName
This is the column name of the CSV that we are extracting. I've found this to be case insensitive.
4: contextVariableName
This is the key name of the entry that will be created in the Context for the iteration.

Here is what I am using for my DataBinding:

   1:    [DataBinding("streetNameDataSource", "streetNames#csv", "streetNames", "streetNames")]

Now with each iteration of the vuser I can reference the Context for the randomized street name, ala:

   1:  Debug.WriteLine("Street name is " + this.Context["streetNames"].ToString());

Works like a champ!

Let's be deviant with our think times!

I mentioned in another post that I stopped using the WebTestRequest.ThinkTime as the think time is reported in the Load Test Repository and that annoys me, especially because the default option for think time is for the MS tools to use a normally distributed variable think time on the time requested.

Now, I think the normally distributed think time is awesome to help prevent VUsers from getting into a lock step situation and most of the time we want the VUsers to be in route step.

I created a ThinkTime routine the other day that just used a slightly randomized think time but of course, it was not normally distributed and I really like the idea of a normally distributed think time. So, off to the Internets and I found a simple to implement algorithm by the name of Box-Muller that I was able to use for my VUsers think times.

Below is a graph with an example of a 5 second think time with multiple values of deviation, s.



Below is the main code that I used to write the class to be invoked from inside the coded web test and of course, think time must be invoked outside of any named transactions otherwise you end up in the same situation before with variable think time being reported in the transaction time.

Because of my slackard like nature, I went ahead and made all the methods static so that I don't have in instantiate any objects for later referencing.

I simply call the code like this:

   1:        string transactionName = "Login.PageHit";

   2:        this.BeginTransaction(transactionName);

   3:        WebTestRequest loginPageHit = new WebTestRequest("http://" + targetWebServer.ToString() + "/SomeWebApp/login.aspx");

   4:        ExtractHiddenFields loginExtractionRule = new ExtractHiddenFields();

   5:        loginExtractionRule.Required = true;

   6:        loginExtractionRule.HtmlDecode = true;

   7:        loginExtractionRule.ContextParameterName = "1";

   8:        loginPageHit.ExtractValues += new EventHandler<ExtractionEventArgs>(loginExtractionRule.Extract);

   9:        yield return loginPageHit;

  10:        loginPageHit = null;

  11:        this.EndTransaction(transactionName);

  12:        #if THINKTIME

  13:          LoadTestThinkTime.ThinkTime(5, 0.1);

  14:        #endif 

Below is the actual code that I wrote:

   1:  using System;

   2:  using System.Threading;

   3:  namespace LoadTestThinkTime {

   4:    public class LoadTestThinkTime {

   5:      /// <summary>

   6:      /// This routine will generate a randomized variable that is in a standard distribution with a deviation of 0

   7:      /// ± s/2 for given s and then invoke Thread.Sleep for the given amount of seconds converted to milliSeconds.

   8:      /// </summary>

   9:      /// <param name="secondsToSleep"></param>

  10:      /// <param name="s"></param>

  11:      static private void RandomizedSleep(int secondsToSleep, double s) {

  12:        if (s > 0) {

  13:          if (s <= 1.5) {

  14:            if (secondsToSleep > 0) {

  15:              if (secondsToSleep < (int.MaxValue / 1000)) {

  16:                double u  = 0;

  17:                double v  = 0;

  18:                double w  = 0;

  19:                double z0 = 0;

  20:                double z1 = 0;

  21:                Random randomizer = new Random();

  22:                do {

  23:                  u = 2.0 * randomizer.NextDouble() - 1.0;

  24:                  v = 2.0 * randomizer.NextDouble() - 1.0;

  25:                  w = u * u + v * v;

  26:                } while (w >= 1.0);

  27:                w = Math.Sqrt(((-2.0 * Math.Log(w)) / w));

  28:                z0 = u * w;

  29:                z1 = v * w; /* I don't use z1, but it is part of the original algorithm so here it is. */

  30:                /* When using small time values, i.e, 1 second, large values of s can generate negative values. In that

  31:                 * situation go ahead and take the absolute value and invoke the call with that to prevent errors during

  32:                 * loadtests. */

  33:                int milliSecondSleepTime = (int)((secondsToSleep + z0 * s) * 1000.0);

  34:                if (milliSecondSleepTime < 0) {

  35:                  milliSecondSleepTime = Math.Abs(milliSecondSleepTime);

  36:                };

  37:                Thread.Sleep(milliSecondSleepTime);

  38:              } else {

  39:                throw new Exception("LoadTestThinkTime: Maximum sleep time is " + ((int)int.MaxValue / 1000).ToString() + " seconds.");

  40:              };

  41:            } else {

  42:              throw new Exception("LoadTestThinkTime: Sleep time cannot be negative.");

  43:            };

  44:          } else {

  45:            throw new Exception("LoadTestThinkTime: Deviation cannot be larger than 1.5");

  46:          };

  47:        } else {

  48:          throw new Exception("LoadTestThinkTime: Deviation cannot be negative");

  49:        }

  50:      } 

  51:      /// <summary>

  52:      /// Public method to invoke variable sleep time with a given deviation of s where s is in the domain of [0, 1.5].

  53:      /// </summary>

  54:      /// <param name="secondsToSleep"></param>

  55:      /// <param name="s"></param>

  56:      static public void ThinkTime(int secondsToSleep, double s) {

  57:        RandomizedSleep(secondsToSleep, s);

  58:      }

  59:      /// <summary>

  60:      /// Public method to invoke variable sleep time with a default deviation of 0.25.

  61:      /// </summary>

  62:      /// <param name="secondsToSleep"></param>

  63:      static public void ThinkTime(int secondsToSleep) {

  64:        RandomizedSleep(secondsToSleep, 0.25);

  65:      }

  66:    }

  67:  }

Extracting Transaction Response Time by Transaction Name per Load Test Run

Despite being annoyed that think time is included in the VS2008 web test response time I would like to be able to graph the response times for a given transaction name. I load my Load Test History database by the unique GUID and forgo the integer test ID to prevent possible duplicates in the future.

I created a query that takes transaction name and test GUID to pull all the transaction times.

   1:  SELECT LoadTestTransactionDetail.TimeStamp, LoadTestTransactionDetail.ElapsedTime

   2:  FROM  LoadTest.dbo.LoadTestRun INNER JOIN

   3:                 LoadTest.dbo.WebLoadTestTransaction ON LoadTestRun.LoadTestRunId = WebLoadTestTransaction.LoadTestRunId INNER JOIN

   4:                 LoadTest.dbo.LoadTestTransactionDetail ON WebLoadTestTransaction.TransactionId = LoadTestTransactionDetail.TransactionId AND 

   5:                 LoadTest.dbo.LoadTestRun.LoadTestRunId = LoadTestTransactionDetail.LoadTestRunId

   6:  WHERE (WebLoadTestTransaction.TransactionName = 'Some_Transaction_Name') AND (LoadTestRun.RunId = '987cce52-7608-40a7-ac85-96714af1ac6d')

   7:  ORDER BY LoadTestTransactionDetail.TimeStamp

   8:   

   9:   

There we go! All the transaction times for "Some_Transaction_Name:"

   1:  2008-07-29 15:58:04.467    7.944

   2:  2008-07-29 15:58:37.633    6.138

   3:  2008-07-29 15:58:55.930    5.243

   4:  2008-07-29 15:59:04.977    4.615

   5:  2008-07-29 15:59:25.607    6.461

   6:  2008-07-29 15:59:36.493    5.026

   7:  2008-07-29 15:59:40.820    5.256

   8:  2008-07-29 16:00:00.507    5.573

   9:  2008-07-29 16:00:04.940    5.457

  10:  2008-07-29 16:00:08.977    5.577

  11:  2008-07-29 16:00:16.547    5.506

  12:  2008-07-29 16:00:34.303    5.758

Uh-Oh. This isn't good.

I found out something today that I really don't like with VS2008 for load testing. I created my virtual user and setup think time ala WebTestRequest.ThinkTime with reasonable think times. After the testing was done I moved the results over to my LoadTestHistory database with a SP that I wrote about earlier in my blog. When I examined the results it turned out that the think time was included in the transaction response time. Nope, I don't like that. I dislike that about as much as I don't like having a failed counter for transactions.

I guess I can't be really surprised as the Begin/End transaction is wrapped around the WebTestRequest object and the think time is handled by the WebTestRequest object and there is no communication between the transaction begin/end and the WTR object (AFAIK).

I looked in the schema for the Load Test Repository and didn't see any fields where response time alone (without think time) would be recorded but there is definitely response time without sleep time recorded during the execution of the VS2008 loadtest. I figured there would be a field in the database with this value but I didn't see it.

Luckily, though, this is pretty simple to get around. If you aren't going to be using variable think times you can simple perform a Thread.Sleep(new TimeSpan(0, 0, 0, 5, 0)) which is similar to the LoadRunner idea behind lr_think_time(). Don't forget to add the "using System.Threading;" to your code to reference the Thread object since it is not referenced by default.

If you want to use variable think time then you'll have to come up with your own mechanism for varying the randomness of the think time. It's not that difficult, it is just a pain.

Clearing cookies in VS2008

In a coded web test that I am working on I have a need to clear all pre-existing cookies for a login page. I don't want the page identifying my vuser with a cookie that is assigned later on in the page hits.

While VS2008 has a CookieContainer that is a container for cookies, there is not a .Clear() method to clear out all the cookies like can be done with LoadRunner with the web_cleanup_cookies() API call.

I have found the way to do this is to use assign a new CookieContainer during the PreWebTest event to clear out all pre-existing cookies like this:

   1:  public Some_Coded_Web_Test() {

   2:    this.PreAuthenticate = true;

   3:    this.PreWebTest += new EventHandler<PreWebTestEventArgs>(ClearCookies);

   4:  }

   5:   

   6:  void ClearCookies(object sender, PreWebTestEventArgs e) {

   7:    Debug.WriteLine("Ha!Ha! I am clearing cookies!");

   8:    this.Context.CookieContainer = new System.Net.CookieContainer();

   9:  }

No more tasty, tasty cookies on script iterations.

Transaction Transaction, what's your action?

I noticed something this morning as I was going over my SP that I wrote on Thursday night. I realized that I needed to also gonkulate how many transactions had failed. Seems simple enough, right? If a page (or dependent request) fails inside a transaction the whole transaction gets marked as a failure, right? That's the way that LoadRunner handles the transaction issue.

In LoadRunner I would normally leave the determination of the status of the transaction by using LR_AUTO. There were a few times that I would explicitly call EndTransaction with a LR_FAIL.

I realized looking over the schema of the Load Test Repository Store that there does not appear to be any logging of failures on the transaction level. The page level? Sure. Not a problem. LoadTestMessage table contains those goodies for us but I cannot find a way to link LoadTestMessage with WebLoadTestRequestMap. Going back over some of the online documentation for running VS2008 load tests it appears that the tool itself does not count transaction failures and only counts failed pages.

Yeah... I'm not too happy about that. With my analysis I want to see what transactions failed as I might have multiple page hits to a single transaction. Hopefully this gets fixed in a future version of the MS LT tool.

Collecting daily LT Metrics

Last night I wrote a stored procedure to automagically collect the latest LT results and insert the values into a database that I created on my local SQL Server 2005 Express.

This SP finds the latest LT that has been run, crunches the numbers and inserts them into two tables, LoadTestHistory and LoadTestHistoryDetails. I used the GUID generated by VS2008 in the Load Test Repository Store as the primary/foreign key between the two tables.

It's a bit rough and could probably use some more polish:

   1:  set ANSI_NULLS ON

   2:  set QUOTED_IDENTIFIER ON

   3:  GO

   4:  ALTER PROCEDURE [dbo].[UpdateLoadTestHistory]

   5:  AS

   6:  BEGIN

   7:   

   8:    SET NOCOUNT ON;

   9:   

  10:    declare @AlreadyExists    as int

  11:   

  12:    declare @latestLoadTest   as int

  13:    declare @loadTestGUID     as nvarchar(36)

  14:    declare @StartTime        as datetime

  15:    declare @EndTime          as datetime

  16:    declare @Duration         as int

  17:   

  18:    select @latestLoadTest = max(LoadTestRunId)

  19:    from LoadTest.dbo.LoadTestRun

  20:    where EndTime is not null

  21:   

  22:    select @loadTestGUID   = RunId,

  23:           @StartTime      = StartTime,

  24:           @EndTime        = EndTime,

  25:           @Duration       = RunDuration

  26:    from LoadTest.dbo.LoadTestRun

  27:    where LoadTestRunId = @latestLoadTest

  28:   

  29:    select @AlreadyExists = count(*)

  30:    from dbo.LoadTestHistory

  31:    where LoadTestGUID = @loadTestGUID

  32:   

  33:    if (@AlreadyExists = 0) 

  34:      begin

  35:   

  36:        begin transaction

  37:   

  38:        begin try

  39:   

  40:          insert into dbo.LoadTestHistory

  41:          values (@LoadTestGUID,

  42:                  @StartTime,

  43:                  @EndTime,

  44:                  @Duration)

  45:   

  46:          SELECT @loadTestGUID                                as LoadTestGUID,

  47:                 WebLoadTestTransaction.TransactionName, 

  48:                 LoadTestTransactionSummaryData.Average, 

  49:                 STDEV(LoadTestTransactionDetail.ElapsedTime) as StdDev, 

  50:                 LoadTestTransactionSummaryData.Minimum, 

  51:                 LoadTestTransactionSummaryData.Maximum, 

  52:                 LoadTestTransactionSummaryData.Percentile90, 

  53:                 LoadTestTransactionSummaryData.Percentile95, 

  54:                 LoadTestTransactionSummaryData.TransactionCount

  55:          INTO   #tmpLoadTestHistory

  56:          FROM   LoadTest.dbo.LoadTestTransactionDetail INNER JOIN

  57:                 LoadTest.dbo.WebLoadTestTransaction ON LoadTest.dbo.LoadTestTransactionDetail.LoadTestRunId = LoadTest.dbo.WebLoadTestTransaction.LoadTestRunId AND 

  58:                 LoadTest.dbo.LoadTestTransactionDetail.TransactionId = LoadTest.dbo.WebLoadTestTransaction.TransactionId INNER JOIN

  59:                 LoadTest.dbo.LoadTestTransactionSummaryData ON LoadTest.dbo.WebLoadTestTransaction.LoadTestRunId = LoadTest.dbo.LoadTestTransactionSummaryData.LoadTestRunId AND 

  60:                 LoadTest.dbo.WebLoadTestTransaction.TransactionId = LoadTest.dbo.LoadTestTransactionSummaryData.TransactionId

  61:          WHERE  (LoadTestTransactionDetail.LoadTestRunId = @latestLoadTest)

  62:          GROUP BY WebLoadTestTransaction.TransactionName, LoadTestTransactionSummaryData.Average, LoadTestTransactionSummaryData.Minimum, 

  63:                   LoadTestTransactionSummaryData.Maximum, LoadTestTransactionSummaryData.Percentile90, LoadTestTransactionSummaryData.Percentile95, 

  64:                   LoadTestTransactionSummaryData.TransactionCount

  65:          ORDER BY WebLoadTestTransaction.TransactionName

  66:   

  67:          insert dbo.LoadTestHistoryDetails

  68:          select LoadTestGUID, TransactionName, Average, StdDev, Minimum, Maximum, Percentile90, Percentile95, TransactionCount

  69:          from #tmpLoadTestHistory

  70:   

  71:          drop table #tmpLoadTestHistory

  72:   

  73:          commit transaction

  74:        end try

  75:   

  76:        begin catch

  77:          rollback transaction

  78:        end catch

  79:      end

  80:  END

And it works like a champ!

I tested it with this bit of SQL:

   1:  use LoadTestResults

   2:  go

   3:   

   4:  truncate table dbo.LoadTestHistory

   5:  truncate table dbo.LoadTestHistoryDetails

   6:  go

   7:   

   8:  dbo.UpdateLoadTestHistory

   9:  go

  10:   

  11:  select dbo.LoadTestHistory.LoadTestGUID, 

  12:         StartTime, 

  13:         EndTime, 

  14:         Duration, 

  15:         TransactionName, 

  16:         Average, 

  17:         StdDev,

  18:         Minimum,

  19:         Maximum,

  20:         [90th],

  21:         [95th],

  22:         TransactionCount

  23:  from dbo.LoadTestHistory, dbo.LoadTestHistoryDetails

  24:  where dbo.LoadTestHistory.LoadTestGUID = dbo.LoadTestHistoryDetails.LoadTestGUID

And the results come out for all to enjoy:

   1:  LoadTestGUID                         StartTime               EndTime                 Duration TransactionName            Average          StdDev            Minimum Maximum 90th  95th  TransactionCount

   2:  0ebdf821-454f-4c50-8e3a-a82a291adb97 2008-07-03 16:21:33.280 2008-07-03 16:31:33.280 600      someTransaction.Details    3.4155221238938  0.583565372475793 1.526   4.952   4.161 4.35  226

   3:  0ebdf821-454f-4c50-8e3a-a82a291adb97 2008-07-03 16:21:33.280 2008-07-03 16:31:33.280 600      someTransaction.FirstHit   3.42468584070796 0.79565610855916  1.636   11.888  4.053 4.285 226

   4:  0ebdf821-454f-4c50-8e3a-a82a291adb97 2008-07-03 16:21:33.280 2008-07-03 16:31:33.280 600      someTransaction.LookupName 3.46306194690266 0.606631255863035 2.087   5.15    4.285 4.411 226

Retrieving Standard Deviation for Transaction Response times with VS2008

One of my plans for the firm for which I work is to integrate a nightly automagic LT with the Continuous Integration effort. I want to be able to automagically compare results of the previous days results and see if anything is amiss. One of the tests I would like to apply is the two population mean test which requires having the average response time, standard deviation and the number of transactions.

VS2008 doesn't provide all of these statistics by default in their LT summary. LoadRunner includes this information by default in their summary results and extracting the values are quite simple if you publish the results to an Excel Spreadsheet.

It's not so quite as simple as that with VS2008 but it can be done with a little effort.

I have configured my LT to record individual metrics to the LoadTest Data Store so that I can query the tables and extract the information that I want. In the examples below I have identified the LoadTestRunId for a given run where I have plenty of metrics to number crunch. In my nightly build and test scenario I envision a simple query to pull the max(LoadTestRunId) to get the latest run to run queries to extract the required information.

I wrote this query today to get the information that I wanted:

   1:  SELECT WebLoadTestTransaction.TransactionName, 

   2:         LoadTestTransactionSummaryData.Average, 

   3:         STDEV(LoadTestTransactionDetail.ElapsedTime) AS StdDev, 

   4:         LoadTestTransactionSummaryData.Minimum, 

   5:         LoadTestTransactionSummaryData.Maximum, 

   6:         LoadTestTransactionSummaryData.Percentile90, 

   7:         LoadTestTransactionSummaryData.Percentile95, 

   8:         LoadTestTransactionSummaryData.TransactionCount

   9:  FROM   LoadTestTransactionDetail INNER JOIN

  10:                    WebLoadTestTransaction ON LoadTestTransactionDetail.LoadTestRunId = WebLoadTestTransaction.LoadTestRunId AND 

  11:                    LoadTestTransactionDetail.TransactionId = WebLoadTestTransaction.TransactionId INNER JOIN

  12:                    LoadTestTransactionSummaryData ON WebLoadTestTransaction.LoadTestRunId = LoadTestTransactionSummaryData.LoadTestRunId AND 

  13:                    WebLoadTestTransaction.TransactionId = LoadTestTransactionSummaryData.TransactionId

  14:  WHERE  (LoadTestTransactionDetail.LoadTestRunId = 33)

  15:  GROUP BY WebLoadTestTransaction.TransactionName, LoadTestTransactionSummaryData.Average, LoadTestTransactionSummaryData.Minimum, 

  16:                    LoadTestTransactionSummaryData.Maximum, LoadTestTransactionSummaryData.Percentile90, LoadTestTransactionSummaryData.Percentile95, 

  17:                    LoadTestTransactionSummaryData.TransactionCount

  18:  ORDER BY WebLoadTestTransaction.TransactionName

The QBE table entries look this this:


And I get the results I want:

   1:  TransactionName            Average             StdDev               Minimum Maximum 90th  95th  TransactionCount

   2:  someTransaction.Details    0.47510353043101411 0.090671477518885088 0.416   3.695   0.522 0.552 4617

   3:  someTransaction.FirstHit   0.44759085986571417 0.077896506427873255 0.384   3.956   0.502 0.526 4617

   4:  someTransaction.LookupName 0.51917002382499466 0.082360597958219192 0.45    2.844   0.577 0.605 4617

Huzzah!

I'll end up setting up an automagic comparison routine. A number of years ago I wrote a C# class for doing statistical tests so I'll probably end up using that for my left/right comparisons and number crunching.

Another neat use for PostRequest

I have a page that I am testing against that comes back as a failure because a .js file hasn't been migrated yet so I don't want my coded web test to hit that page and fail. I took care of the problem with PostRequest for the WebTestRequest call like this:

   1:  void preventCallsToJS(object sender, PostRequestEventArgs e) {

   2:    List<WebTestRequest> requestsToRemove = new List<WebTestRequest>()

   3:    foreach (WebTestRequest linkToRemove in e.Request.DependentRequest

   4:      if (linkToRemove.Url.EndsWith("someJavaScriptFile.js")) {

   5:        requestsToRemove.Add(linkToRemove);

   6:      }

   7:    };

   8:    foreach (WebTestRequest linkToRemove in requestsToRemove) {

   9:      e.Request.DependentRequests.Remove(linkToRemove);

  10:    };

  11:  }

For each request I have added the call to the above PostRequest like this:

   1:  request4.PostRequest += new EventHandler<PostRequestEventArgs>(preventCallsToJS);

No more problems hitting those pesky not yet deployed .js files. w00t!

Saving sessions as webtests with Fiddler2

Today I was trying to save some saved sessions with Fiddler2 to a webtest since I cannot record webtests with VS2008 with my common user profile that I use on my laptop.

When I tried to save the file I got a very annoying error that a specified assembly could not be found. Fiddler2 is trying to load version 8.0.0.0 of Microsoft.VisualStudio.QualityTools.WebTestFramework.dll but alas, I do not have VS2005 install, I have VS2008. What to do? I'm already bummed that I cannot record webtests as I should be able to. I find this to be really annoying as recording webtests shouldn't be this annoying.

At this point you're probably asking, "So, Mr. Auswipe-Load-Tester-Dude, does this shake your faith in the VS2008 framework for load testing?" Not yet is my reply. Why you ask when these two issues are clearly impediments? Well, with Load Runner I was running into problems recording virtual users and had the problem for over three weeks where I could not record vusers on my work machine. When I tried to record vusers both FireFox and IE would detonate and crash. I worked with Level I and Level II techs for over three weeks trying to correct the problem and the issues was never solved. It turns out I got my current job offer and accepted it. I told my former coworkers that I figured the solution was to take the fdisk quiz and reload the OS and start again. I certainly hope the same solution is not required for the MS solution.

Anyway, back to the issue at hand...

I did some Googling and I found this entry on CodeProf.com with a solution by Ed Glas. Ed basically says to modify the fiddler.exe.config and use the tag but as documented here but did not include a full fledged example, just the link to the .NET config file documentation. So, I'm not a .NET expert by any stretch of the imagination (but am working on it slowly but surely) so I wasn't quite sure what all config tags were required for the redirect. Here is what I used and it appears to be working *cross fingers*.

   1:  <configuration> 

   2:    <runtime> 

   3:      <legacyUnhandledExceptionPolicy enabled="1" /> 

   4:      <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">

   5:        <dependentAssembly>

   6:          <assemblyIdentity name="Microsoft.VisualStudio.QualityTools.WebTestFramework"

   7:                                  publicKeyToken="b03f5f7f11d50a3a"

   8:                                  culture="neutral" />

   9:              <bindingRedirect oldVersion="8.0.0.0"

  10:                               newVersion="9.0.0.0"/>

  11:           </dependentAssembly>

  12:        </assemblyBinding>

  13:    </runtime> 

  14:  </configuration> 

After doing some recording it appears that the code that is generated from the converted Fiddler2 to .webtest to coded web test looks good and I see the automagic correlation of data extracted from form fields and the ever dreaded VIEWSTATE.

This oughta be good enough for me right now until I resolve the situation with VS2008. I have no personal problems with recording a session, saving as a .webtest and then added the .webtest to a project and converting to a coded web test so it's all good with me.

The Good, The Bad and The Ugly.

I've been real busy with non load testing stuff at work and haven't had much of a chance to continue playing around with VS2008 for LT purposes but I have found out some interesting stuff.

The Good:

You can specify the data store (either SQL Server or SQL Server Express) for storing all the individual hits from web sites and then query that information which renders that I was doing in the earlier blog post as moot. Yay! By default, VS2008 TSTE keeps these results in SQL Server Express that is installed when Visual Studio is installed. It's pretty cool that you can choose to store results in another location for historical reasons.

Looking around the data in the tables I'm not sure if you can group page hits together by transaction ID. But, I haven't had much of a chance to play around with querying data and am not for sure if this is the case. If that is the case, I can still use a method similar to a previous blog entry for collecting the metrics myself for catching page response time as it relates to individual transactions. A bit of a PITA but fantastic that I can do it if I want. Yay for a flexible framework that allows me as a test with development background to run around and do this kind of stuff.

One thing I know for sure though is with the ability to query the databases I should now have the ability to crunch the numbers for transactions and now get average, standard deviation, hit counts, etc. The idea is that in a nicely controlled environment I should be able to generate an automagic report for day to day automated load test execution and be able to do something like a mean population comparison of transactions times from one test to the next and pick out statistically significant deviations for alpha = 0.05. That'll be kinda neat if it works out.

Now that I think about it, the VS2008 tool should have been reporting these metrics all along. LoadRunner defaults with basic statistical information and VS2008 should report the same as well. It is kind of annoying, but I like the fact that I have a published schema that I can use to extract the information that I want (see link to schema information for VS2008 further down below). So, another plus in the LoadRunner column. [I might really like the VS2008 product, but I do have to be fair in my comparisons of Pro/Con of VS2008 versus LoadRunner.]

I wasn't able to use the statistical modeling at my previous employer because our systems were just too large with too many variables involved. We had communication from the web server to the database server (which was on the same switch so not much of a problem) but we also had a bunch of communications to the Host. The Host was required for testing the web sites and was actually hitting production systems that merely disregarded the requests and sent back bogus results so that we wouldn't kill the mainframe.

Many years ago I tried to get rid of this variability by writing what I called a "parroting proxy server." It was a bit of C# code that was essentially a protocol agnostic proxy server that would pass messages from point A to point C where the intermediate Point B was the proxy. It would store responses with a in memory hash and if the same request was caught being requested it would go ahead and send back the hashed response to eliminate the variance that we would see. It worked out pretty good but we never got around to implementing it. Unfortunately in a Fortune 50 company the bureaucracy can be too great and with Empire Building as it is the other managers didn't see what was in this for them and I suppose I didn't do a very good sales job. So be it.

To flip the switch to log all data so that you can query the data for fun and profit is to modify the "Timing Details Storage Property" in the Run Setting node of the load test editor to "All individual details."

And unlike the LoadRunner database (AFAIK, I could be wrong!), this schema is published with all sorts of handy information located at http://blogs.msdn.com/billbar/articles/529874.aspx. How great is that?

The Bad:

I haven't had enough time to continue my little load testing adventure. I'm finding that my new employer needs to update their processes as it relates to building production final machines. The current steps are extremely painful. But I suspect it'll be just fine in the long run as I've had some experience in this in the past and now that this process can be fully automated and integrated into our build process to be painless.

The Ugly:

Ok, this is ugly. I tried to write a simple script to augment some stuff that I did last week only to find that I couldn't bring up an instance of IE with the Web Recorder active and it is frustrating as hell. I found several posts that essentially say that the "solution" is to delete your login profile and start again. Yeah, that really sucks, Microsoft. I'd like to have a solution that doesn't require me to blow away my profile and have me spend hours getting my desktop back into the shape that I like it. That is a real PITA for sure.

I have three profiles on my machine and I can only generate recorded web tests with one of these profiles. I blew away one of the smaller profiles (not my main profile where I have 2.5+ gig of files) and gave it a try and it was no-go. I still could not record web tests. Yeah, that is bad. Very bad.

I'm not sure what the solution on this is going to be. In the short term I'm going to have to use my Administrator login to create scripts and then xfer that over to my main login script. How craptacular is that? Once I get some free time I'll have to look into this further but I am dreading it. No fun!

Here are some MS blog entries related to the problem. Perhaps they could help somebody else but I haven't seen any love for me, yet.

http://blogs.msdn.com/mtaute/archive/2007/11/09/diagnosing-and-fixing-web-test-recorder-bar-issues.aspx
http://blogs.msdn.com/edglas/archive/2007/11/01/web-test-recorder-bar-not-showing-up.aspx

It appears that Fiddler2 might be an interim solution from what I read here in this MS blog post located at http://blogs.msdn.com/slumley/pages/enhanced-web-test-support-in-fiddler.aspx.

A PITA? Sure, but it beats not having any solution at all. Fiddler is purt near cool and I've "fiddled" with it a little bit and it is nifty.

Loading parameter data from a CSV in a VS2008 coded web test

Today I've been working on how to load correlated data from a CSV file into a coded web test. VS2008 has support for a wide range of data sources for correlation but I still prefer the good 'ol CSV format.

I rag on LR a lot, but selecting a data file and accessing it with the "{pSomeParam}" format is much easier in LR than VS2008. Details below:

In my little webtest I recorded a simple script hitting Google and I wanted to be able to pass different search terms to Google. Nothing fancy, just enough to figure out how to do what I wanted. This task in LR is pretty trivial.

I created a file by the name of searchTerms.csv that contains the following:

8<------------------
column1
loadrunner
perl
superbase
vs2008
------------------>8

I then added a data source to the web test and converted it over to code to see how I need to code things in the future and this is how it looks:

   1:  [DataSource("searchTerms", 

   2:              "Microsoft.VisualStudio.TestTools.DataSource.CSV", 

   3:              "|DataDirectory|\\databindingtowebtest\\searchTerms.csv", 

   4:              Microsoft.VisualStudio.TestTools.WebTesting.DataBindingAccessMethod.Sequential,

   5:              "searchTerms#csv")]

   6:  [DataBinding("searchTerms", "searchTerms#csv", "column1", "GoogleSearchTerm")]

The context for the current web test should now be automagically updated with an index entry by the name of "GoogleSearchTerm" that is referenced like this:

   1:  request2.QueryStringParameters.Add("q", this.Context["GoogleSearchTerm"].ToString(), false, false);

I don't mind the accessing of the correlated data via the context. That is no big deal, but setting up the databinding is kind of a complicated PITA. I have to give nods to LR at this point for the ease of creating params. But, on the other hand, VS2008 gives me a lot more control of slicing and dicing of return HTML data so there had to be a trade off someplace and I guess this is one of those situations.

Something important that I've learned is that numeric data needs to be double quoted to prevent mangling of the values. For example, I have the following entries now:

  [DataSource("lastNames", 

              "Microsoft.VisualStudio.TestTools.DataSource.CSV", 

              "c:\\work\\data\\LRData\\lastNames.csv", 

              Microsoft.VisualStudio.TestTools.WebTesting.DataBindingAccessMethod.Random, //   .Sequential, 

              "lastNames#csv")]

  [DataSource("targetServer",

            "Microsoft.VisualStudio.TestTools.DataSource.CSV",

            "c:\\work\\data\\LRData\\targetServer.csv",

            Microsoft.VisualStudio.TestTools.WebTesting.DataBindingAccessMethod.Sequential,

            "targetServer#csv")]

  [DataBinding("lastNames", "lastNames#csv", "lastName", "lastName")]

  [DataBinding("targetServer", "targetServer#csv", "TargetServer", "targetServer")]

The IP address that I want to hit is bound to the context entry for "targetServer."

I found that if I have the entry as:

8<----------------------
TargetServer
10.0.0.100
---------------------->8

I find that if it is not quoted it will be manged into the value "10.001." Just adding the double quotes around the value will allow me to use the IP address as I originally intended.

string targetWebServer = this.Context["targetServer"].ToString();

WebTestRequest request1 = new WebTestRequest("http://" + targetWebServer.ToString() + "/someVDir/somePage.aspx");

Knowing is half the battle. (GI Joe!)

Selecting a random button in a grid with VS2008

I am finally getting into writing virtual users with the VS2008 VSTE tools and the learning curve is pretty steep but the amount of control that I get is pretty darn cool so I think the pain points will be worth it.

Something that has always been a pain with LR in the past is randomly selecting some parsed HTML to randomize the next step in a vuser script. Sure, you can use web_reg_save_param with "ord=all" ala:

web_reg_save_param ("someParam", "LB= value=\"", "RB=\"", "Ord=All", LAST);

and pull it all into an array parameter and then iterate over the entries in the parameter by utilizing sprintf() ala:

   1:  web_reg_save_param ("someParam", "LB= value=\"", "RB=\"", "Ord=All", LAST);

   2:   

   3:  for (i = 1; i <= atoi(lr_eval_string("{someParam_count}")); i++) {

   4:    sprintf(someParamValue, "{someParam_%d}", i);

   5:    lr_output_message("%s", lr_eval_string(someParamValue));

   6:  }

That way of accessing the parameter array info always seemed like a PITA to me (maybe I'm in the minority with this thought?). Want to substring the output even further? More of a PITA since C doesn't have any quick and friendly string handling functions (substr(), left(), ltrim(), etc).

But, you have to be able to do it to get the job done.

Today I needed to basically select a random customer view button from a grid that was returned from this WebTestRequest hit:

   1:  WebTestRequest request3 = new WebTestRequest("http://xxx.yyy.zzz.iii/someWebPage.aspx");

   2:  request3.Method = "POST";

   3:  FormPostHttpBody request3Body = new FormPostHttpBody();

   4:  request3Body.FormPostParameters.Add("__EVENTTARGET", this.Context["$HIDDEN1.__EVENTTARGET"].ToString());

   5:  request3Body.FormPostParameters.Add("__EVENTARGUMENT", this.Context["$HIDDEN1.__EVENTARGUMENT"].ToString());

   6:  request3Body.FormPostParameters.Add("__LASTFOCUS", this.Context["$HIDDEN1.__LASTFOCUS"].ToString());

   7:  request3Body.FormPostParameters.Add("__VIEWSTATE", this.Context["$HIDDEN1.__VIEWSTATE"].ToString());

   8:  request3Body.FormPostParameters.Add("__VIEWSTATEENCRYPTED", this.Context["$HIDDEN1.__VIEWSTATEENCRYPTED"].ToString());

   9:  request3Body.FormPostParameters.Add("__EVENTVALIDATION", this.Context["$HIDDEN1.__EVENTVALIDATION"].ToString());

  10:  request3.Body = request3Body;

  11:  ExtractHiddenFields extractionRule2 = new ExtractHiddenFields();

  12:  extractionRule2.Required = true;

  13:  extractionRule2.HtmlDecode = true;

  14:  extractionRule2.ContextParameterName = "1";

  15:  request3.ExtractValues += new EventHandler<ExtractionEventArgs>(selectRandomButton);

  16:  request3.ExtractValues += new EventHandler<ExtractionEventArgs>(extractionRule2.Extract);

  17:  yield return request3;

  18:  request3 = null;

The WebTestRequest has an event for allowing coders to write their own code for extracting data from the HTML output. I wrote the code below is referenced in the above code at line # 15:

   1:  void selectRandomButton(object sender, ExtractionEventArgs e) {

   2:    if (e.Response.HtmlDocument != null) {

   3:      List<String> buttonNameList = new List<String>();

   4:      foreach (HtmlTag tag in e.Response.HtmlDocument.GetFilteredHtmlTags(new string[] { "input" })) {

   5:        if (tag.GetAttributeValueAsString("type") == "submit") {

   6:          buttonNameList.Add(tag.GetAttributeValueAsString("name"));

   7:        };

   8:      };

   9:      if (buttonNameList.Count > 0) {

  10:        Random randomizer = new Random();

  11:        e.WebTest.Context.Add("RANDOMCUSTOMERBUTTON",

  12:                               buttonNameList[(int)randomizer.Next(buttonNameList.Count) - 1]);

  13:        e.Success = true;

  14:      } else {

  15:        e.Success = false;

  16:        e.Message = "No entries were found for customer view buttons.";

  17:      };

  18:    } else {

  19:      e.Success = false;

  20:      e.Message = "No HTML to work against!";

  21:    };

  22:  }

Take a gander at the loop at line #4. In that loop I'm inspected a group of elements searching for the data that I want. No need to specify a dozen calls of web_reg_save_param only to reference said params via lr_eval_string and strcmp(). That's right, nice simple easy to read code! It's all there. I can spin, fold and mutilate the information all I want! That and I have access to .NET Containers. I don't have to write my own double linked list for storing data for declaring the square array in advanced (I did a lot of square arrays because I am a slacker like that). How sweet is that?!

While it might seem more complex at first, I believe that the extra control that I get over the selection of data is fantastic. I can think of several times in the past that I've had to write more complicated code than I have wanted to for the slicing and dicing of HTML and this method would have made things so much easier for me.

And I found a cool code formatting site today located at http://www.manoli.net/csharpformat/format.aspx. It took a little work getting the CSS stuff taken care of (after all, I'm not a GUI whiz) but it was worth it for sure.