Wednesday, May 28, 2008

Let's abuse a FTP server with JMeter!

Let's use JMeter to chunk files at a FTP server!

For my job I had written some multi-threaded perl code using Net::FTP to chunk files at an FTP server. The data is generated by another chunk of code and during the generation I generate a file by the name of sourceList.txt that has various elements of data used by my perl code.

The CSV file is in the form of:

8<-------------------------------------
user1,pass1,C:\work\ftpLoadAgent\data\dataFile1.txt.gpg,dataFile1.txt.gpg user1,pass1,C:\work\ftpLoadAgent\data\dataFile2.txt.gpg,dataFile2.txt.gpg user1,pass1,C:\work\ftpLoadAgent\data\dataFile3.txt.gpg,dataFile3.txt.gpg user3,pass3,C:\work\ftpLoadAgent\data\dataFile4.txt.gpg,dataFile4.txt.gpg user1,pass1,C:\work\ftpLoadAgent\data\dataFile5.txt.gpg,dataFile5.txt.gpg user3,pass3,C:\work\ftpLoadAgent\data\dataFile6.txt.gpg,dataFile6.txt.gpg user3,pass3,C:\work\ftpLoadAgent\data\dataFile7.txt.gpg,dataFile7.txt.gpg user2,pass2,C:\work\ftpLoadAgent\data\dataFile8.txt.gpg,dataFile8.txt.gpg user2,pass2,C:\work\ftpLoadAgent\data\dataFile9.txt.gpg,dataFile9.txt.gpg user1,pass1,C:\work\ftpLoadAgent\data\dataFile10.txt.gpg,dataFile10.txt.gpg ------------------------------------->8

The first column contains the username on the FTP server and the second column contains the account password. The third column contains the path to the file that I want to chunk upstream to the FTP server and the final column contains the password that the file is saved as on the FTP server.

Let's build a project to crush our FTP server!

Start with a blank JMeter project:


Let’s add a thread group to the test plan and name it “ftpLoadTest."


Let’s add a CSV Data Set Config under the ftpLoadTest thread group:


I set the name of the CSV group to “ftpLoadData” and gave it a path to my .TXT file (which is actually a CSV file with a .TXT extension) and give the column names of the columns of data (username, password, localFileName, remoteFileName).

As the thread executes the FTP sampler the record being pointed at will advanced. It’s my understand that if there are multiple threads they will alternate records so that multiple threads won’t pull from the same record. Sounds good to me so far.

Now it is time to add the FTP sampler that will do the FTP work for us:


Now we have the entry for the FTP sampler:


Now we need to populate the fields with the data that is being pulled from the .TXT file in CSV format. The method of using the referenced variable data is the same way that a variable can be referenced in perl code, that is $ sigil with open brace, variable name and close brace like this: ${someVariableName}.

In this case, I am going to hard code the IP address of the FTP server (ever be it so humble, there is no place like 127.0.0.1). When filling out the password portion of the data entry all the characters will be masked but go ahead and use the ${someVariableName} format. I also set the put (STOR) radio button and select binary checkbox. In my particular case, the data that I am sending is not an ASCII file but a GPG encrypted file and sending the file as a ASCII file could corrupt the data and I wouldn’t want that.

Even though you can’t see it in the password field, it reads “${password}.”


We want to see the results of the FTP chunking so let’s add a “View Results Tree” listener.


Now, in my particular case for this example I am running a local FTP server on my laptop and I am using the super handy stand alone FTP server “Quick’n Easy FTP Server 3.1 Lite.” This FTP server great for prototype testing or just sharing files at a LAN party. The config files are saved in .XML files and you can just xcopy to another machine and fire ‘er up. Good stuff, that!


I’ve already created the usernames and subdirs in advance and won’t go over that in this blog entry. But suffice it to say that setting up the users with the interface is easy enough that even *I* can do it.

The server allows us the see server statistics in real time:


And has the option to allow us to watch the FTP server log in real time:

So, let’s hit the run button and let’s see what happens!


Look at that! 10 successful runs (I only have 10 files to chunk at this time).

Let’s go ahead and add an listener to show results in tabular format and hit run again:


Here we can see more details such as response times, file size (in my case the file sizes are random) and success/failures. Good stuff!

Let’s go ahead and add a spine graph visualizer and see how it looks:


I generated more files so that I’d have more than 10 files to work with. In this case, I generated 10,000 of 5K byte files and instead of random upper case characters I used an uppercase X. The resultant GPG’d files were only 640 to 641 bytes in size.

Appropriately I change the number of iterations to 10,000 and execute the test.


I didn’t like how the spline graph looked with so many data entries so I went ahead and added a graph listener which does a scatter graph of the results. Below are the first 2,458 iterations:


The graph isn’t very readable so when I am executing this test in the future I’ll go ahead and just record the results to file and generate files either with some perl code that I have already coded or with Excel. But, the graph is good enough for watching while a test is executing. To be truthful, I am too much of a slacker to modify the original code to graph the way I'd like to see so I really can't complain.

The rate of chunking the files is about the same as my perl code but in the future I am more likely to use JMeter to run my FTP test as all my other fellow coworkers might not be perl fluent.

All in all, cool stuff!

Today sniffing around the ‘net I found a SMTP sampler and I might be able to use JMeter for my SMTP chunking as well. I hope it works out the same as the FTP JMeter project. I like my perl code and all, but this is a better solution for those folks that aren’t perl coders.

The only major difference between my perl code and the JMeter results is that when an FTP failure occurs, my perl code pushes the data back onto a stack to be picked up by another thread. That is the only major difference.

If this was a LoadRunner scenario with FTP users using a .DAT file in a similar manner the skipping of records with failed iterations would be the same so it’s no biggie in the end and good enough for my purposes.

1 comment:

Not Available said...

How very interesting. I would have never thought Load Testing could be so fun.





You == Über Nerd