Showing posts with label Thinktime. Show all posts
Showing posts with label Thinktime. Show all posts

Friday, August 1, 2008

Let's be deviant with our think times!

I mentioned in another post that I stopped using the WebTestRequest.ThinkTime as the think time is reported in the Load Test Repository and that annoys me, especially because the default option for think time is for the MS tools to use a normally distributed variable think time on the time requested.

Now, I think the normally distributed think time is awesome to help prevent VUsers from getting into a lock step situation and most of the time we want the VUsers to be in route step.

I created a ThinkTime routine the other day that just used a slightly randomized think time but of course, it was not normally distributed and I really like the idea of a normally distributed think time. So, off to the Internets and I found a simple to implement algorithm by the name of Box-Muller that I was able to use for my VUsers think times.

Below is a graph with an example of a 5 second think time with multiple values of deviation, s.



Below is the main code that I used to write the class to be invoked from inside the coded web test and of course, think time must be invoked outside of any named transactions otherwise you end up in the same situation before with variable think time being reported in the transaction time.

Because of my slackard like nature, I went ahead and made all the methods static so that I don't have in instantiate any objects for later referencing.

I simply call the code like this:



   1:        string transactionName = "Login.PageHit";

   2:        this.BeginTransaction(transactionName);

   3:        WebTestRequest loginPageHit = new WebTestRequest("http://" + targetWebServer.ToString() + "/SomeWebApp/login.aspx");

   4:        ExtractHiddenFields loginExtractionRule = new ExtractHiddenFields();

   5:        loginExtractionRule.Required = true;

   6:        loginExtractionRule.HtmlDecode = true;

   7:        loginExtractionRule.ContextParameterName = "1";

   8:        loginPageHit.ExtractValues += new EventHandler<ExtractionEventArgs>(loginExtractionRule.Extract);

   9:        yield return loginPageHit;

  10:        loginPageHit = null;

  11:        this.EndTransaction(transactionName);

  12:        #if THINKTIME

  13:          LoadTestThinkTime.ThinkTime(5, 0.1);

  14:        #endif 



Below is the actual code that I wrote:



   1:  using System;

   2:  using System.Threading;

   3:  namespace LoadTestThinkTime {

   4:    public class LoadTestThinkTime {

   5:      /// <summary>

   6:      /// This routine will generate a randomized variable that is in a standard distribution with a deviation of 0

   7:      /// ± s/2 for given s and then invoke Thread.Sleep for the given amount of seconds converted to milliSeconds.

   8:      /// </summary>

   9:      /// <param name="secondsToSleep"></param>

  10:      /// <param name="s"></param>

  11:      static private void RandomizedSleep(int secondsToSleep, double s) {

  12:        if (s > 0) {

  13:          if (s <= 1.5) {

  14:            if (secondsToSleep > 0) {

  15:              if (secondsToSleep < (int.MaxValue / 1000)) {

  16:                double u  = 0;

  17:                double v  = 0;

  18:                double w  = 0;

  19:                double z0 = 0;

  20:                double z1 = 0;

  21:                Random randomizer = new Random();

  22:                do {

  23:                  u = 2.0 * randomizer.NextDouble() - 1.0;

  24:                  v = 2.0 * randomizer.NextDouble() - 1.0;

  25:                  w = u * u + v * v;

  26:                } while (w >= 1.0);

  27:                w = Math.Sqrt(((-2.0 * Math.Log(w)) / w));

  28:                z0 = u * w;

  29:                z1 = v * w; /* I don't use z1, but it is part of the original algorithm so here it is. */

  30:                /* When using small time values, i.e, 1 second, large values of s can generate negative values. In that

  31:                 * situation go ahead and take the absolute value and invoke the call with that to prevent errors during

  32:                 * loadtests. */

  33:                int milliSecondSleepTime = (int)((secondsToSleep + z0 * s) * 1000.0);

  34:                if (milliSecondSleepTime < 0) {

  35:                  milliSecondSleepTime = Math.Abs(milliSecondSleepTime);

  36:                };

  37:                Thread.Sleep(milliSecondSleepTime);

  38:              } else {

  39:                throw new Exception("LoadTestThinkTime: Maximum sleep time is " + ((int)int.MaxValue / 1000).ToString() + " seconds.");

  40:              };

  41:            } else {

  42:              throw new Exception("LoadTestThinkTime: Sleep time cannot be negative.");

  43:            };

  44:          } else {

  45:            throw new Exception("LoadTestThinkTime: Deviation cannot be larger than 1.5");

  46:          };

  47:        } else {

  48:          throw new Exception("LoadTestThinkTime: Deviation cannot be negative");

  49:        }

  50:      } 

  51:      /// <summary>

  52:      /// Public method to invoke variable sleep time with a given deviation of s where s is in the domain of [0, 1.5].

  53:      /// </summary>

  54:      /// <param name="secondsToSleep"></param>

  55:      /// <param name="s"></param>

  56:      static public void ThinkTime(int secondsToSleep, double s) {

  57:        RandomizedSleep(secondsToSleep, s);

  58:      }

  59:      /// <summary>

  60:      /// Public method to invoke variable sleep time with a default deviation of 0.25.

  61:      /// </summary>

  62:      /// <param name="secondsToSleep"></param>

  63:      static public void ThinkTime(int secondsToSleep) {

  64:        RandomizedSleep(secondsToSleep, 0.25);

  65:      }

  66:    }

  67:  }

Tuesday, July 29, 2008

Uh-Oh. This isn't good.

I found out something today that I really don't like with VS2008 for load testing. I created my virtual user and setup think time ala WebTestRequest.ThinkTime with reasonable think times. After the testing was done I moved the results over to my LoadTestHistory database with a SP that I wrote about earlier in my blog. When I examined the results it turned out that the think time was included in the transaction response time. Nope, I don't like that. I dislike that about as much as I don't like having a failed counter for transactions.

I guess I can't be really surprised as the Begin/End transaction is wrapped around the WebTestRequest object and the think time is handled by the WebTestRequest object and there is no communication between the transaction begin/end and the WTR object (AFAIK).

I looked in the schema for the Load Test Repository and didn't see any fields where response time alone (without think time) would be recorded but there is definitely response time without sleep time recorded during the execution of the VS2008 loadtest. I figured there would be a field in the database with this value but I didn't see it.

Luckily, though, this is pretty simple to get around. If you aren't going to be using variable think times you can simple perform a Thread.Sleep(new TimeSpan(0, 0, 0, 5, 0)) which is similar to the LoadRunner idea behind lr_think_time(). Don't forget to add the "using System.Threading;" to your code to reference the Thread object since it is not referenced by default.

If you want to use variable think time then you'll have to come up with your own mechanism for varying the randomness of the think time. It's not that difficult, it is just a pain.