Revisiting CSV Data Binding in VS2008 Coded Web Tests

I've been going through the WinDBG labs located here. Tess has done a great job with the labs and done all the hard work, so w00t for her!

I needed to add another .CSV data file for my main VUser that I am currently working on so I did a copypasta of a pre-existing DataBinding and DataSource. When I first created the coded web test I had added so much custom code that I deleted the web test and left only the coded web test. The fear was that I might accidently re-gen the code from the web test and over write my work. That in and of itself shouldn't be too bad since we all know to use a code repository, right? Of course we do.

Anyhoo, I made the code changes and was greeted by an error when my file couldn't be loaded. Huh? How rude!

I finally got it to work, but I wanted to note the way .CSV data binding works in a coded web test in greater detail just to remind myself next time this happens. There are plenty of examples of binding a .CSV file to a web test, but few on a pre-existing coded web test. So, here goes!

First we have to have a DataSource declaration that takes five parameters for a .CSV file:

1: dataSourceName
This is the name we are giving the DSN for our .CSV file.
2: providerName
In my case for a .CSV file I am using "Microsoft.VisualStudio.TestTools.DataSource.CSV"
3: connectionString.
For my use, this is the path with escaped back-whacks to the .CSV file. In my case, it is "c:\\data\\streetNames.csv"
4: DataBindingAccessMethod
For my use, I want a random selection of data from the .CSV file so I am using "Microsoft.VisualStudio.TestTools.WebTesting.DataBindingAccessMethod.Random"
5: tableName
I thought that I could provide my own table name to reference the .CSV file but that didn't work and was forced to use the form of #csv, like this: "streetNames#csv". When I tried to use the name "streetNameTable" I would get back an error of "cannot find streetTableName.txt" from VS2008. But, the #csv form works, so no biggie.

Here is what I am using for my entire Data Source:

   1:    [DataSource("streetNameDataSource",

   2:              "Microsoft.VisualStudio.TestTools.DataSource.CSV",

   3:              "C:\\Data\\streetNames.csv",

   4:              Microsoft.VisualStudio.TestTools.WebTesting.DataBindingAccessMethod.Random,

   5:              "streetNames#csv")]

Next, we have to define what column we are pulling from the .CSV file. In my case, it is a single column CSV file but we still have to have a column name defined. For my CSV file I am using the form of:

8<---------------------------
streetNames
"ABBINGTON"
"ALDRICH"
"ALEXANDER"
"ALICE"
"ALLISON"
"ALMOND"
"ALTA"
"AMHERST"
--------------------------->8

These are all the street names in Grover's Mill, NJ as reported by Melissa Data.

The DataBinding declaration that I am using takes four parameters.

1: dataSourceName
This is the DSN that we previously defined.
2: tableName
This is the tableName that we defined in the form of #csv.
3: columnName
This is the column name of the CSV that we are extracting. I've found this to be case insensitive.
4: contextVariableName
This is the key name of the entry that will be created in the Context for the iteration.

Here is what I am using for my DataBinding:

   1:    [DataBinding("streetNameDataSource", "streetNames#csv", "streetNames", "streetNames")]

Now with each iteration of the vuser I can reference the Context for the randomized street name, ala:

   1:  Debug.WriteLine("Street name is " + this.Context["streetNames"].ToString());

Works like a champ!

Tasty memory! Om Nom Nom!

So, I created a scenario and had 45 virtual users doing nothing but browsing the application that I am testing. After one hour I noticed that the w3wp private bytes are running around 1.5 gigs of memory. That's right, just browsing. I was hitting it at a rate about 26 hits per second and running about 90% CPU utilization and 1.5 gigs of private bytes being consumed with almost 700 megs of hit in Gen 0.

Yeah, something ain't right.

But, that gives me an excuse to brush up on my WinDBG skills. I fired up adplus.vbs and took a -hang dump that resulted in a 1.5 gig dump. Woo! So tonight I've been re-learning the art of poking around in a dump file with a debugger.

Along the way I found a most excellent blog for a MS dudette in Sweden that really knows her stuff and has some great tutorials for debuggery:

http://blogs.msdn.com/tess/default.aspx

Check out the most excellent labs located here.

Let's be deviant with our think times!

I mentioned in another post that I stopped using the WebTestRequest.ThinkTime as the think time is reported in the Load Test Repository and that annoys me, especially because the default option for think time is for the MS tools to use a normally distributed variable think time on the time requested.

Now, I think the normally distributed think time is awesome to help prevent VUsers from getting into a lock step situation and most of the time we want the VUsers to be in route step.

I created a ThinkTime routine the other day that just used a slightly randomized think time but of course, it was not normally distributed and I really like the idea of a normally distributed think time. So, off to the Internets and I found a simple to implement algorithm by the name of Box-Muller that I was able to use for my VUsers think times.

Below is a graph with an example of a 5 second think time with multiple values of deviation, s.



Below is the main code that I used to write the class to be invoked from inside the coded web test and of course, think time must be invoked outside of any named transactions otherwise you end up in the same situation before with variable think time being reported in the transaction time.

Because of my slackard like nature, I went ahead and made all the methods static so that I don't have in instantiate any objects for later referencing.

I simply call the code like this:

   1:        string transactionName = "Login.PageHit";

   2:        this.BeginTransaction(transactionName);

   3:        WebTestRequest loginPageHit = new WebTestRequest("http://" + targetWebServer.ToString() + "/SomeWebApp/login.aspx");

   4:        ExtractHiddenFields loginExtractionRule = new ExtractHiddenFields();

   5:        loginExtractionRule.Required = true;

   6:        loginExtractionRule.HtmlDecode = true;

   7:        loginExtractionRule.ContextParameterName = "1";

   8:        loginPageHit.ExtractValues += new EventHandler<ExtractionEventArgs>(loginExtractionRule.Extract);

   9:        yield return loginPageHit;

  10:        loginPageHit = null;

  11:        this.EndTransaction(transactionName);

  12:        #if THINKTIME

  13:          LoadTestThinkTime.ThinkTime(5, 0.1);

  14:        #endif 

Below is the actual code that I wrote:

   1:  using System;

   2:  using System.Threading;

   3:  namespace LoadTestThinkTime {

   4:    public class LoadTestThinkTime {

   5:      /// <summary>

   6:      /// This routine will generate a randomized variable that is in a standard distribution with a deviation of 0

   7:      /// ± s/2 for given s and then invoke Thread.Sleep for the given amount of seconds converted to milliSeconds.

   8:      /// </summary>

   9:      /// <param name="secondsToSleep"></param>

  10:      /// <param name="s"></param>

  11:      static private void RandomizedSleep(int secondsToSleep, double s) {

  12:        if (s > 0) {

  13:          if (s <= 1.5) {

  14:            if (secondsToSleep > 0) {

  15:              if (secondsToSleep < (int.MaxValue / 1000)) {

  16:                double u  = 0;

  17:                double v  = 0;

  18:                double w  = 0;

  19:                double z0 = 0;

  20:                double z1 = 0;

  21:                Random randomizer = new Random();

  22:                do {

  23:                  u = 2.0 * randomizer.NextDouble() - 1.0;

  24:                  v = 2.0 * randomizer.NextDouble() - 1.0;

  25:                  w = u * u + v * v;

  26:                } while (w >= 1.0);

  27:                w = Math.Sqrt(((-2.0 * Math.Log(w)) / w));

  28:                z0 = u * w;

  29:                z1 = v * w; /* I don't use z1, but it is part of the original algorithm so here it is. */

  30:                /* When using small time values, i.e, 1 second, large values of s can generate negative values. In that

  31:                 * situation go ahead and take the absolute value and invoke the call with that to prevent errors during

  32:                 * loadtests. */

  33:                int milliSecondSleepTime = (int)((secondsToSleep + z0 * s) * 1000.0);

  34:                if (milliSecondSleepTime < 0) {

  35:                  milliSecondSleepTime = Math.Abs(milliSecondSleepTime);

  36:                };

  37:                Thread.Sleep(milliSecondSleepTime);

  38:              } else {

  39:                throw new Exception("LoadTestThinkTime: Maximum sleep time is " + ((int)int.MaxValue / 1000).ToString() + " seconds.");

  40:              };

  41:            } else {

  42:              throw new Exception("LoadTestThinkTime: Sleep time cannot be negative.");

  43:            };

  44:          } else {

  45:            throw new Exception("LoadTestThinkTime: Deviation cannot be larger than 1.5");

  46:          };

  47:        } else {

  48:          throw new Exception("LoadTestThinkTime: Deviation cannot be negative");

  49:        }

  50:      } 

  51:      /// <summary>

  52:      /// Public method to invoke variable sleep time with a given deviation of s where s is in the domain of [0, 1.5].

  53:      /// </summary>

  54:      /// <param name="secondsToSleep"></param>

  55:      /// <param name="s"></param>

  56:      static public void ThinkTime(int secondsToSleep, double s) {

  57:        RandomizedSleep(secondsToSleep, s);

  58:      }

  59:      /// <summary>

  60:      /// Public method to invoke variable sleep time with a default deviation of 0.25.

  61:      /// </summary>

  62:      /// <param name="secondsToSleep"></param>

  63:      static public void ThinkTime(int secondsToSleep) {

  64:        RandomizedSleep(secondsToSleep, 0.25);

  65:      }

  66:    }

  67:  }

Extracting Transaction Response Time by Transaction Name per Load Test Run

Despite being annoyed that think time is included in the VS2008 web test response time I would like to be able to graph the response times for a given transaction name. I load my Load Test History database by the unique GUID and forgo the integer test ID to prevent possible duplicates in the future.

I created a query that takes transaction name and test GUID to pull all the transaction times.

   1:  SELECT LoadTestTransactionDetail.TimeStamp, LoadTestTransactionDetail.ElapsedTime

   2:  FROM  LoadTest.dbo.LoadTestRun INNER JOIN

   3:                 LoadTest.dbo.WebLoadTestTransaction ON LoadTestRun.LoadTestRunId = WebLoadTestTransaction.LoadTestRunId INNER JOIN

   4:                 LoadTest.dbo.LoadTestTransactionDetail ON WebLoadTestTransaction.TransactionId = LoadTestTransactionDetail.TransactionId AND 

   5:                 LoadTest.dbo.LoadTestRun.LoadTestRunId = LoadTestTransactionDetail.LoadTestRunId

   6:  WHERE (WebLoadTestTransaction.TransactionName = 'Some_Transaction_Name') AND (LoadTestRun.RunId = '987cce52-7608-40a7-ac85-96714af1ac6d')

   7:  ORDER BY LoadTestTransactionDetail.TimeStamp

   8:   

   9:   

There we go! All the transaction times for "Some_Transaction_Name:"

   1:  2008-07-29 15:58:04.467    7.944

   2:  2008-07-29 15:58:37.633    6.138

   3:  2008-07-29 15:58:55.930    5.243

   4:  2008-07-29 15:59:04.977    4.615

   5:  2008-07-29 15:59:25.607    6.461

   6:  2008-07-29 15:59:36.493    5.026

   7:  2008-07-29 15:59:40.820    5.256

   8:  2008-07-29 16:00:00.507    5.573

   9:  2008-07-29 16:00:04.940    5.457

  10:  2008-07-29 16:00:08.977    5.577

  11:  2008-07-29 16:00:16.547    5.506

  12:  2008-07-29 16:00:34.303    5.758

Uh-Oh. This isn't good.

I found out something today that I really don't like with VS2008 for load testing. I created my virtual user and setup think time ala WebTestRequest.ThinkTime with reasonable think times. After the testing was done I moved the results over to my LoadTestHistory database with a SP that I wrote about earlier in my blog. When I examined the results it turned out that the think time was included in the transaction response time. Nope, I don't like that. I dislike that about as much as I don't like having a failed counter for transactions.

I guess I can't be really surprised as the Begin/End transaction is wrapped around the WebTestRequest object and the think time is handled by the WebTestRequest object and there is no communication between the transaction begin/end and the WTR object (AFAIK).

I looked in the schema for the Load Test Repository and didn't see any fields where response time alone (without think time) would be recorded but there is definitely response time without sleep time recorded during the execution of the VS2008 loadtest. I figured there would be a field in the database with this value but I didn't see it.

Luckily, though, this is pretty simple to get around. If you aren't going to be using variable think times you can simple perform a Thread.Sleep(new TimeSpan(0, 0, 0, 5, 0)) which is similar to the LoadRunner idea behind lr_think_time(). Don't forget to add the "using System.Threading;" to your code to reference the Thread object since it is not referenced by default.

If you want to use variable think time then you'll have to come up with your own mechanism for varying the randomness of the think time. It's not that difficult, it is just a pain.

Clearing cookies in VS2008

In a coded web test that I am working on I have a need to clear all pre-existing cookies for a login page. I don't want the page identifying my vuser with a cookie that is assigned later on in the page hits.

While VS2008 has a CookieContainer that is a container for cookies, there is not a .Clear() method to clear out all the cookies like can be done with LoadRunner with the web_cleanup_cookies() API call.

I have found the way to do this is to use assign a new CookieContainer during the PreWebTest event to clear out all pre-existing cookies like this:

   1:  public Some_Coded_Web_Test() {

   2:    this.PreAuthenticate = true;

   3:    this.PreWebTest += new EventHandler<PreWebTestEventArgs>(ClearCookies);

   4:  }

   5:   

   6:  void ClearCookies(object sender, PreWebTestEventArgs e) {

   7:    Debug.WriteLine("Ha!Ha! I am clearing cookies!");

   8:    this.Context.CookieContainer = new System.Net.CookieContainer();

   9:  }

No more tasty, tasty cookies on script iterations.

Thinkin' about ditching Hyper-V

Seriously thinking about ditching Hyper-V for it's smaller brother, Virtual PC 2007. Sure, Virtual PC doesn't give the VM as much horsepower or as much contact with the metal of the host machine but by golly gosh the networking doesn't try to kill off Client for Microsoft Network.

I'm sure Hyper-V works great in the intended role of hosting guest VMs while racked in a server room someplace but the network issue that I am having is really irking me. Perhaps Hyper-V isn't ready for the type of prototyping work I want to do and Virtual PC 2007 will gimp along for my prototyping work.

Of course, the only issue that I have so far is that I have four VMs created and I'd have to go through all that trouble of reloading the OS. Well, I'm just about done with my load balancing prototyping so blowing away the VMs and removing the Hyper-V role wouldn't be that painful...

Of course, the biggest thing that I'd miss is the snapshot feature of Hyper-V. For prototyping and messin' around with a VM that is the greatest thing since sliced bread. I don't suspect we'll see that in a free Virtual PC anytime soon...