Taking some time to keep on learning.

Since I am unemployed and not having anything better to do with my time I decided I would check out "Apache JMeter" by Emily H. Halili (Packt, 2008).

It's a light read and I found that Chapter 7 (Advanced Features) was the most helpful to myself as I am not a n00b to JMeter. However, I felt that Chapter 7 would have been better served if BeanShell processing would have been tackled. I've found from my load testing experience in the past that at some point the return HTML is gonna have to be sliced and diced and data extracted that cannot be done with a simple RegEx extraction (like what is covered in Chapter 7 of the book).

Other than that, I think that if you are a total n00b to JMeter it isn't half bad as a simple introduction to using JMeter for performance/load testing. There's a lot more to performance/load testing than the book covers such as metrics collection, number crunching, et cetera but the book doesn't purport itself to be the end all be all of explaining performance/load testing so I can't complain.

Three stars.

Abusing SMTP Servers for fun and profit with JMeter and the SMTP plug-in

Let’s Be Spam Kings with JMeter! (aka Abusing SMTP Servers for fun and profit)

The other day I came across a SMTP Plug-in for JMeter from http://www.beolink.org/index/jmeter-plug-in.

Two extra .jars are needed that are not shipped with JMeter and those are mail.jar and activation.jar. There were no MD5 sum values so I assume that these two .jars are adequate as the solution appears to work. I am not aware of any possible versioning conflicts as I am no Java guru. I did write “Hello, World!” code once in Java back in 1995.

To install the plug-in I had to copy both mail.jar and activation.jar to the jakarta-jmeter-2.3.1\lib\ sub-dir and copied the SmtpSampler-0.9.jar to the jakarta-jmeter-2.3.1\lib\ext sub-dir.

Let’s create a brand new JMeter project:


Let’s add a thread for our spam chunking:


I’ve setup 5 threads with 200 iterations for a total of 1,000 iterations to be executed in total as this is the number of files that I want to chunk at our poor SMTP server.

Like in my FTP application, I have a CSV file that contains all the information that JMeter will require for chunking spam. It is pretty similar to the FTP data:

8<----------------------------------------------------------------- user1@someserversomeplace.com,10.0.1.4,C:\work\smtpLoadAgent\data\dataFile1.txt.gpg user2@someserversomeplace.com,10.0.1.4,C:\work\smtpLoadAgent\data\dataFile2.txt.gpg user3@someserversomeplace.com,10.0.1.4,C:\work\smtpLoadAgent\data\dataFile3.txt.gpg user4@someserversomeplace.com,10.0.1.4,C:\work\smtpLoadAgent\data\dataFile4.txt.gpg user2@someserversomeplace.com,10.0.1.4,C:\work\smtpLoadAgent\data\dataFile5.txt.gpg user1@someserversomeplace.com,10.0.1.4,C:\work\smtpLoadAgent\data\dataFile6.txt.gpg user5@someserversomeplace.com,10.0.1.4,C:\work\smtpLoadAgent\data\dataFile7.txt.gpg user4@someserversomeplace.com,10.0.1.4,C:\work\smtpLoadAgent\data\dataFile8.txt.gpg user5@someserversomeplace.com,10.0.1.4,C:\work\smtpLoadAgent\data\dataFile9.txt.gpg ------------------------------------------------------------------>8

The columns are pretty easy to figure out: e-mail address, IP address of the SMTP server in question and the path to the file that is going to be chunked. In this case, just like the FTP server example, it’s a GPG’d text file containing random data.

Just like the other example, add a CSV Data Set Config to JMeter defining the variables that will be used:


For some reason, the name of the plug-in comes across as [res_key=smtp_title]. I don’t know if this a config issue on my side or something not set properly in the .jar. I emailed the person that was nice enough to create this plug-in but haven’t heard anything back. It appears to be a cosmetic issue at this point and doesn’t cause any problems that I’ve seen so far.


Now we need to fill out the information for the SMTP sampler so we can abuse the SMTP server:


In this example, I replaced the default [res_key=smtp_title] with “smtpSampler” and set the Server field to the dynamic variable that I defined earlier as ${ipAddress}.

In my particular case, the SMTP server is running on port 35 and not the standard port 25. It’s great that the SMTP plug-in gives us this option as I couldn’t find an option to change the port number in the perl code that I had written with MIME::Lite and Net::SMTP. I had done some testing with a freeware port redirection tool that got the job done but I never got far enough to see how much overhead was consumed by the port redirection. Now I don’t have to worry about it!

I set the Address From field to be a (I think) non-existant e-mail address of joe@blow.com.

Address To is set to the dynamic variable ${emailAddress}

The file that I want to send is set to be ${fileToAttach} which is the full path to the file in question. Since the CSV file has a different file for each line, we won’t be sending the exact same file with each iteration.

For the subject line I used “${fileToAttach} ” and then checked the “Include timestamp in subject” checkbox. The timestamp that will be appended to the filename being sent is actually epoch time and not human readable form. For most people I doubt that this will be a problem. I also checked “Calculate message size” so I can see the number of bytes that were transferred when logging output.

Let’s go ahead and throw some Listeners so that we can see the results of our testing. I’m not going to use the spline graph as I prefer the scatter plot that Graph Results show. Even it isn’t my preferred way of showing data but it is easy enough to dump the output to an XML/CSV file and use some of my perl code to generate “pretty” graphs:


I have found that I prefer to watch the “Average”, “Deviation” and “Throughput” data points in the scatter plot.

In my particular case, I am pointing at an SMTP server running on my local laptop in a Virtual PC 2007 VM hosting a Win2k3 server running hMailServer. For a free e-mail server, hMailServer seems to be a fairly full featured product and includes IMAP support and integration with blacklists and WinClam virus protection and I even saw some entries for tar pitting and even AD support. Pretty darn cool!

Here is a shot of hMailServer waiting to receive our mass spamming of attachments:



Let’s rip loose with JMeter and see what happens:





Well, look at that! 1,000 e-mails chunked at a rate of about 300 per minute or about 5 per second. Not too shabby.

Logging into Thunderbird in the VM we see that the e-mail attachment was sent without issue:



How cool is that?

I found that the JMeter with the SMTP plug-in was faster than my perl code using MIME::Lite and Net::SMTP by about 1.5 e-mails per sec. I also had some issues with the MIME::Lite issuing a die() when a problem was encountered instead of returning a false with $msg->send_by_smtp() which would cause a worker thread to die off which was pretty annoying.

So, with the SMTP plug-in I get access to non-standard ports (as seen above) and better over all performance.

Something that I did find is that if on the SMTP Sampler screen the “Check for Failure” checkbox is checked that each of the e-mail xfers are marked as “warning”. Example is below:



If the log file that I specified is examined you will see the following:

“org.beolink.jmeter.protocol.smtp.UnexpectedSuccessException: Expected failure but got success...”

To checkout the 500 error return I went ahead and did some tests and captured the TCP stream with Wireshark (nee ethereal) and saw that there was no difference between the perl chunking and the JMeter/SMTP plug-in chunking and there were no error codes returned.

My solution? Uncheck the “Check for failure” checkbox and keeping on trucking.

Despite the two issues noted above, the SMTP plug-in for JMeter does a great job and I will be using it instead of my home grown perl solution.

It's on it's way!

I got news from management today that my portable server, the Billy Bad Ass quad core RAID5 notebook has been approved. It's barely squeaking by the $4K limit but it should be on it's way soon. Yay!

Can't wait to fire that bad boy up with Win2k[38] Server and get all four cores chunking load at a target.

I also got the SMTP Plugin for JMeter up and running and will make an entry about that tomorrow if I have time. It does a pretty good job and is faster than my perl code that makes use of MIME::Lite and Net::SMTP for chunking attachments at an e-mail server.

More details in the next post.

Let's abuse a FTP server with JMeter!

Let's use JMeter to chunk files at a FTP server!

For my job I had written some multi-threaded perl code using Net::FTP to chunk files at an FTP server. The data is generated by another chunk of code and during the generation I generate a file by the name of sourceList.txt that has various elements of data used by my perl code.

The CSV file is in the form of:

8<-------------------------------------
user1,pass1,C:\work\ftpLoadAgent\data\dataFile1.txt.gpg,dataFile1.txt.gpg user1,pass1,C:\work\ftpLoadAgent\data\dataFile2.txt.gpg,dataFile2.txt.gpg user1,pass1,C:\work\ftpLoadAgent\data\dataFile3.txt.gpg,dataFile3.txt.gpg user3,pass3,C:\work\ftpLoadAgent\data\dataFile4.txt.gpg,dataFile4.txt.gpg user1,pass1,C:\work\ftpLoadAgent\data\dataFile5.txt.gpg,dataFile5.txt.gpg user3,pass3,C:\work\ftpLoadAgent\data\dataFile6.txt.gpg,dataFile6.txt.gpg user3,pass3,C:\work\ftpLoadAgent\data\dataFile7.txt.gpg,dataFile7.txt.gpg user2,pass2,C:\work\ftpLoadAgent\data\dataFile8.txt.gpg,dataFile8.txt.gpg user2,pass2,C:\work\ftpLoadAgent\data\dataFile9.txt.gpg,dataFile9.txt.gpg user1,pass1,C:\work\ftpLoadAgent\data\dataFile10.txt.gpg,dataFile10.txt.gpg ------------------------------------->8

The first column contains the username on the FTP server and the second column contains the account password. The third column contains the path to the file that I want to chunk upstream to the FTP server and the final column contains the password that the file is saved as on the FTP server.

Let's build a project to crush our FTP server!

Start with a blank JMeter project:


Let’s add a thread group to the test plan and name it “ftpLoadTest."


Let’s add a CSV Data Set Config under the ftpLoadTest thread group:


I set the name of the CSV group to “ftpLoadData” and gave it a path to my .TXT file (which is actually a CSV file with a .TXT extension) and give the column names of the columns of data (username, password, localFileName, remoteFileName).

As the thread executes the FTP sampler the record being pointed at will advanced. It’s my understand that if there are multiple threads they will alternate records so that multiple threads won’t pull from the same record. Sounds good to me so far.

Now it is time to add the FTP sampler that will do the FTP work for us:


Now we have the entry for the FTP sampler:


Now we need to populate the fields with the data that is being pulled from the .TXT file in CSV format. The method of using the referenced variable data is the same way that a variable can be referenced in perl code, that is $ sigil with open brace, variable name and close brace like this: ${someVariableName}.

In this case, I am going to hard code the IP address of the FTP server (ever be it so humble, there is no place like 127.0.0.1). When filling out the password portion of the data entry all the characters will be masked but go ahead and use the ${someVariableName} format. I also set the put (STOR) radio button and select binary checkbox. In my particular case, the data that I am sending is not an ASCII file but a GPG encrypted file and sending the file as a ASCII file could corrupt the data and I wouldn’t want that.

Even though you can’t see it in the password field, it reads “${password}.”


We want to see the results of the FTP chunking so let’s add a “View Results Tree” listener.


Now, in my particular case for this example I am running a local FTP server on my laptop and I am using the super handy stand alone FTP server “Quick’n Easy FTP Server 3.1 Lite.” This FTP server great for prototype testing or just sharing files at a LAN party. The config files are saved in .XML files and you can just xcopy to another machine and fire ‘er up. Good stuff, that!


I’ve already created the usernames and subdirs in advance and won’t go over that in this blog entry. But suffice it to say that setting up the users with the interface is easy enough that even *I* can do it.

The server allows us the see server statistics in real time:


And has the option to allow us to watch the FTP server log in real time:

So, let’s hit the run button and let’s see what happens!


Look at that! 10 successful runs (I only have 10 files to chunk at this time).

Let’s go ahead and add an listener to show results in tabular format and hit run again:


Here we can see more details such as response times, file size (in my case the file sizes are random) and success/failures. Good stuff!

Let’s go ahead and add a spine graph visualizer and see how it looks:


I generated more files so that I’d have more than 10 files to work with. In this case, I generated 10,000 of 5K byte files and instead of random upper case characters I used an uppercase X. The resultant GPG’d files were only 640 to 641 bytes in size.

Appropriately I change the number of iterations to 10,000 and execute the test.


I didn’t like how the spline graph looked with so many data entries so I went ahead and added a graph listener which does a scatter graph of the results. Below are the first 2,458 iterations:


The graph isn’t very readable so when I am executing this test in the future I’ll go ahead and just record the results to file and generate files either with some perl code that I have already coded or with Excel. But, the graph is good enough for watching while a test is executing. To be truthful, I am too much of a slacker to modify the original code to graph the way I'd like to see so I really can't complain.

The rate of chunking the files is about the same as my perl code but in the future I am more likely to use JMeter to run my FTP test as all my other fellow coworkers might not be perl fluent.

All in all, cool stuff!

Today sniffing around the ‘net I found a SMTP sampler and I might be able to use JMeter for my SMTP chunking as well. I hope it works out the same as the FTP JMeter project. I like my perl code and all, but this is a better solution for those folks that aren’t perl coders.

The only major difference between my perl code and the JMeter results is that when an FTP failure occurs, my perl code pushes the data back onto a stack to be picked up by another thread. That is the only major difference.

If this was a LoadRunner scenario with FTP users using a .DAT file in a similar manner the skipping of records with failed iterations would be the same so it’s no biggie in the end and good enough for my purposes.