Status for Andrew DeFaria

Pulse

I decided to make the pulse script part of a larger project so I created a perf subdirectory under cc. The idea here is that pulse is a sort of performance test - a "sanity performance test" if you will. In general the script does the following:

• Creates a logfile (pulse.log)
• Creates a new view (default) if required
• Sets context to that view
• Changes directory to the view to the vob (/vobs/adm by default)
• Performs 10 checkouts and checkins of a relatively large file (a copy of firefox.exe which is ~7 Meg)
• Compares the time that those checkouts and checkins took against a threshold (default 60) and reports an error if it took too long

Here is a usage:

ccase-rmna-3:export SITE_PERL_LIBPATH=~/SCM/lib
ccase-rmna-3:pulse -u
Usage:  pulse (v1.0) [-u] [-v] [-d] [-view ] [-vob ]
        [-element ] [-t ] [-i ]

Where:

  -u:       Display usage
  -v:       Turn on verbose mode
  -d:       Turn on debug mode
  -view:    View tag to create/use (Default: default)
  -vob:     Vob tag to use (Default /vobs/adm)
  -element: Vob relative path to element to checkout/in (Default: firefox.exe)
  -t :   Threshold of what is "too long" (Default 60 seconds)
  -i :   Number of iterations (default 10)

Here's an example of the log file produced:

pulse: 12/30/2005 @ 12:27: Performing 10 checkout/ins in view default vob /vobs/adm of element firefox.exe
pulse: 12/30/2005 @ 12:28: Finished in 36 seconds
pulse: 12/30/2005 @ 12:28: Performing 10 checkout/ins in view default vob /vobs/adm of element firefox.exe
pulse: 12/30/2005 @ 12:29: Finished in 35 seconds

Several issues came up when attempting to code this script. For example:

• Which view should this operate in?
• Which vob should this operate in?
• How will we assure that this vob exists at any site that we want to run pulse? Perhaps we should make a vob for performance testing only (e.g. /vobs/perf).
• Checking out and in a single element creates many versions on the version tree. If this pulse script was run every hour then 240 versions would be created in only one day. It would be better to more properly "setup" and "teardown" the whole environment but this would be considerably more complex and time consuming. For example, pulse could create the view, create the vob, formulate some elements and do mkelems on them, do the checkouts and checkins, do rmelems and remove the vob and view, etc. That's a lot! Alternately it could simply do the mkelems, the checkouts and checkins, then do rmelems but that brings up the issue of where does the source come for the mkelems? Also, typically there's a trigger to prevent rmelem...
• Currently the script is pretty Unix centric. For example, it uses cleartool lsview -stgloc -auto. This works on Unix but for some reason -stgloc doesn't work on Windows. I also suspect that at the various sites usage of -stgloc might not be that reliable.
• Also the view used is a dynamic view. Should a snapshot version be coded?
• Current the script uses /vobs/ for a vob tag. In fact it uses /vobs/adm. This is not a Windows vob tag so the script will not work on Windows
• The script does sport paramaters to change things such as the view used, the vob used and the element to checkout and in. The defaults are great but then again we don't have a performance testing environment setup and replicated yet
• Currently the script logs it's activity to a log file but it's not in a great format for analysis. The TimeUtils module I'm using only as 1 second resolution. I'm not sure how much analysis of this logfile is desired
• There is no facility for emailing alerts when the script determines that things are taking too long