- Why We Need Scatter Charts
- The Analysis Process
- Filter Out Your Variables
- Change the Dimensions of the Test
- Correlate Multiple Sources of Data for the Same Run
- Applying These Principles on a Real Project
- Tips for Formatting Your Data
Like this article? We recommend
Like this article? We recommend
Like this article? We recommend
Applying These Principles on a Real Project
Take a look at the scatter chart in Figure 6 and think about what you see before you read on.
)
Figure 6 Unpolished scatter chart from a project.
Notice that, even at the end of the run, the server wasn’t dead. It was still returning some information fine, but other information never came back. You should be able to distinguish the three banding patterns. The transactions in both of the bottom bands turned out to be trivial (images, CSS, and so on). There was no consistency between what forced something from the bottom band up to the second band. This was confusing because it meant that sometimes an image was served in less than a second, and at other times the same image was served in around two minutes.
It turned out that some scripts were set to time out after two minutes—thus the band around two minutes in the figure. The third band (at the top) represents transactions that never came back. After about 30 minutes, I ended the run, and the third band represents all of those transactions stopping. Everything between the third and second band represents responses coming in. Oddly enough, some of those responses seemed to be working correctly—they were just really slow.
The testing uncovered a problem with the application server settings. Even though users were logging out of the application properly, their server sessions didn’t end. It wasn’t until the session timed out (after half an hour) that the server ended the session. Thus, my test of 200 users was really testing 200 active users (the first run of 50, the second run of 100, and the third run of 150) and 300 idle users. This problem would have been difficult to identify without pushing past the "150 users" test, even though I knew that at 150 users my testing was invalid. I needed to see the end pattern for it to all click into place for me.