2 ////////////////////////////////////////////////////////////////////////////////
4 // File: Changelog0.9.php
6 // Description: Change log for Rantest 0.9
7 // Author: Andrew@ClearSCM.com
8 // Created: Mon Apr 28 15:20:06 MST 2008
12 // (c) Copyright 2008, General Dynamics, all rights reserved.
14 // All rights reserved except as subject to DFARS 252.227-7014 of contract
15 // number CP02H8901N issued under prime contract N00039-04-C-2009.
17 // Warning: This document contains technical data whose export is restricted
18 // by the Arms Export Control Act (Title 22, U.S.C., Sec 2751, et seq.) or the
19 // Export Administration Act of 1979, as amended, Title, 50, U.S.C., App. 2401
20 // et seq. Violations of these export laws are subject to severe criminal
21 // penalties. Disseminate in accordance with provisions of DoD Directive
24 ////////////////////////////////////////////////////////////////////////////////
25 $script = basename ($_SERVER["PHP_SELF"]);
27 include_once "$_SERVER[DOCUMENT_ROOT]/php/Utils.php";
28 include_once "$_SERVER[DOCUMENT_ROOT]/php/RantestDB.php";
30 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"
31 "http://www.w3.org/TR/html4/strict.dtd">
34 <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
35 <link rel="stylesheet" type="text/css" media="screen" href="/css/Testing.css">
36 <link rel="stylesheet" type="text/css" media="screen" href="/css/Tables.nohover.css">
37 <title>RANTEST: ChangeLog 0.9</title>
43 <?php print banner ();?>
45 <h1 align=center>RANTEST 0.9 ChangeLog</h1>
48 <li><a href="#0.9.9c">Version 0.9.9c</a></li>
50 <li><a href="#0.9.9b">Version 0.9.9b</a></li>
52 <li><a href="#0.9.9a">Version 0.9.9a</a></li>
54 <li><a href="#0.9.9">Version 0.9.9 - Code Complete</a></li>
56 <li><a href="#0.9.8d">Version 0.9.8d</a></li>
58 <li><a href="#0.9.8c">Version 0.9.8c</a></li>
60 <li><a href="#0.9.8b">Version 0.9.8b</a></li>
62 <li><a href="#0.9.8a">Version 0.9.8a</a></li>
64 <li><a href="#0.9.8">Version 0.9.8</a></li>
66 <li><a href="#0.9.7a">Version 0.9.7a</a></li>
68 <li><a href="#0.9.7">Version 0.9.7 - "The Rantvl release"</a></li>
70 <li><a href="#0.9.6">Version 0.9.6</a></li>
72 <li><a href="#0.9.5">Version 0.9.5</a></li>
75 <p>This is the ChangeLog for RANTEST for versions 0.9 and up.</p>
80 <li><a href="ChangeLog1.0.php">RANTEST 1.0 ChangeLog</a></li>
82 <li><a href="ChangeLog1.1.php">RANTEST 1.1 ChangeLog</a></li>
84 <li><a href="ChangeLog.php">RANTEST 1.2 ChangeLog</a></li>
87 <h2><a name="0.9.9c">Version 0.9.9c</a></h2><hr>
92 <li>Changed MAX_ATTEMPTS to 4</li>
94 <li>Moved capturing of $startTime into the do/until statement</li>
96 <li>Added some LogDebug statements to trace what's happening with
97 this auto-rerun stuff.</li>
99 <li>Changed if statement such that the warning would come out on the
100 first re-run. I suspect impatient engineers were not waiting for the
101 3 iteration when the first warning would come out.</li>
103 <li>Changed to use /vobs/nms_log for NMS logs. Vob's been replicated
104 but not put into project yet. This should happen soon however.</li>
110 <li>Changed to handle the situation where -pattern is not
113 <li>Added [OUTPUT][/OUTPUT] and [STATUS][/STATUS] tags.</li>
119 <li>Fixed bug with finding ranscrub's conf file.</li>
125 <li>Added <i>Testcase Per Version</i> report.</li>
127 <li>Added support for multiple nightly logs</li>
129 <li>Added link to <i>Testcase Per Version</i> report.</li>
132 <h2><a name="0.9.9b">Version 0.9.9b</a></h2><hr>
137 <li>Added runValidation subroutine that handles running of a
138 validation and also adds to the rantest database the validation test
141 <li>Fixed up some errors in the execution of valMsgs.pl
142 validation. This included missing the testlogs component of the
143 path, adding the leading "/" before Rantvl, using $_opts{type}
144 instead of the missing $_opts{unitType} and adding on the ">" for
145 proper redirection.</li>
151 <li>Added logging to ranscrub. It will now log to testlogs.</li>
157 <li>Will now accept a parameter for the DUT in question. This allows
158 a poor man's way of doing parallelization by creating .suite or
159 .test files that have the DUT as part of the filename and adding
160 multiple lines in cron(1).</li>
162 <li>Because of the above: Removed the updating or
163 rantest.stats.csv.</li>
169 <li>Added new report: Test Case Per version.</li>
171 <li>Removed useless reports on many page</li>
173 <li>Added average run time to Test History Report but only for when
174 all tests are selected.</li>
177 <h2><a name="0.9.9a">Version 0.9.9a</a></h2><hr>
182 <li>Changed to not spawn extra processes when -timeout -1 was
185 <li>At Roman and Doug's request, added code to print out all of the
186 logfiles produced under logpath.</li>
192 <li>RantestDB.php: Added new Error function</li>
194 <li>RantestDB.php: Added getStatus function like
195 RantestDB::getStatus function. This routine is the driver for the
196 new TestStats.php page.</li>
198 <li>TestHistory.php: Added total row.</li>
200 <li>index.php: Activated TestStats report</li>
202 <li>TestStats.php: Added TestStats report</li>
208 <li>Added formatSize to format the size into a more human readable
211 <li>Added scrubbing of Rantvl</li>
213 <li>Added option handling for testlogs, rantvl and rantestdb
214 scrubbing as well as all - default all.</li>
216 <li>Added rantvl_user to config file and updated scrub_days and
217 db_scrub to 45 days.</li>
223 <li>Changed update_view to use -x to avoid xauth problems</li>
226 <h2><a name="0.9.9">Version 0.9.9 - Code Complete</a></h2><hr>
228 <p>Reached code complete!</p>
233 <li>Changed code to automatically create test case ID directories if
234 they are not present for <i>Run for Record</i></li>
236 <li>Changed to handle error output a little better.</li>
239 <h2><a name="0.9.8d">Version 0.9.8d</a></h2><hr>
244 <li>Added code to capture output from validation scripts into a log
251 <li>Added code to put out the date, failures and successes into
252 .../auto/test/nightly/rantest.stats.csv for graphing purposes.</li>
258 <li>Changed handling of log collection again to handle run for
259 record capturing better.</li>
261 <li>Checking for /^\[LOG\]/ instead of /\[LOG\]/. Sometimes [LOG]
262 was appearing in ASCII dumps!</li>
265 <h2><a name="0.9.8c">Version 0.9.8c</a></h2><hr>
270 <li>Emergency fix! Changing the flow of when log files are collected
271 after disconnection is fine. However when in suite mode we
272 disconnect for each .test file we run. Since disconnect no logner
273 automatically calls collectLogFiles we need to call it right after
274 each time we disconnect. We also need to run validations at that
277 <li>Changed collectLogFiles to not exit rantest merely because we
278 couldn't find a logfile. In suite mode there may be more to do.</li>
281 <h2><a name="0.9.8b">Version 0.9.8b</a></h2><hr>
286 <li>I believe the persistent bug regarding running in cron is
289 <li>Removed the wait for processes to finish. This was done only
290 because GenericDesktopMonitor would be running for up to 10 seconds
291 after we stopped. Before, when we used to tar up things, this was
292 important. Sometimes the tar would report that the files changed
293 while tarring them. Since we don't make tars anymore this should not
294 be a problem. Might be a problem for Run for Record though...</li>
296 <li>Added verbose output to tell the user where the log files are
297 stored under testlogs. This will be helpful for manual validations
298 that are still required.</li>
300 <li>Fixed bug with grepping for "Simulator version is". Needed to
301 put quotes around that!</li>
303 <li>Changed name of rantest build logs vob to
304 rantest_build2_log. There is no rantest_log_build2 vob!</li>
306 <li>Changed running of validations to occur after collection of log
307 files. Otherwise it doesn't make sense to validate against logfiles
308 that haven't be collected.</li>
314 <li>Renamed decode.test.FQT to aal2val</li>
316 <li>Created msgdefs directory</li>
322 <li>Checked in bare bones ranscrub and config/ranscrub.conf</li>
325 <h2><a name="0.9.8a">Version 0.9.8a</a></h2><hr>
330 <li>Fixed borken link in copyright version</li>
336 <li>Fixed bug with handling of ReadLine properly in interactive and
339 <li>Fixed bug with errorneously calling validation routines when
340 there were none specified in test</li>
342 <li>Added progress indication in log collection</li>
344 <li>Fixed bug with keeping track of stats - errorneously adding file
345 failure count to test failure count</li>
351 <li>Changed bugcatcher to only log to /tmp/rantest.debug.log</li>
354 <h2><a name="0.9.8">Version 0.9.8</a></h2><hr>
359 <li>Changed copyright block to highlight and link only the current
362 <li>Added Last Modified into copyright block</li>
368 <li>Added comment about suite files</li>
370 <li>Now properly returning $suiteFailures from runSuiteFile</li>
372 <li>Added runValidation subrountine. Now does rudimentary
373 validations for valMsgs.pl and aalval</li>
375 <li>Changed to use -t STDIN to distingish if we are running with a
376 tty. If not we are in cron and we have to behave differently for the
379 <li>Now properly returning errors from runFile subroutine</li>
381 <li>Now ouputs a message telling the user if it is busy terminating
382 processes and collecting logfiles at the end of execution</li>
388 <li>Changed bug catcher (for ls -t problem) to log all java
389 processes running to /tmp</lI>
391 <li>Added getCollectLogFiles method.</li>
393 <li>Refixed bugs in calls to error with join in subroutines like
394 _checkoutElement</li>
396 <li>Totally changed the way collectLogFiles work. For one, it no
397 longer creates tar files, except for run for record. The new
398 ranscrub (in development) will be responsible for cleaning up space
399 in testlogs and other areas. Doing this also removes the disposition
400 of logfiles from rantest's concern</li>
402 <li>Plus run for record runs now first store in testlogs then only
403 made as a tar directly into the appropriate Clearcase vob.</li>
405 <li>Based on the log type (EAST|TM500|NMS|RANTVL) the run for record
406 tar image is put into simdev_log, tm500_log, rantest_log_build2. For
407 that last one both NMS and RANTVL is stored into
408 rantest_log_build2. Yes I don't like the 2 in build 2! :-(</li>
411 <h2><a name="0.9.7a">Version 0.9.7a</a></h2><hr>
413 <p>Roll up fixes for <a href="#0.9.7">0.9.7</a>:</p>
416 <li>Changed handling of errors in collectLogFiles. Since this is
417 happening from when disconnecting from EAST there's no sense in
418 attempting to return status or error messages, just die! Also
419 changed the internal methods that collectLogFiles calls
420 (e.g. _makeTar, _checkOutElement, etc.) simply die too.</li>
422 <li>Now setting umask correctly to 002 (was setting it to 022!,
423 which did nothing different). With new umask setting we no longer
424 need to attempt to chmod directories to 775.</li>
426 <li>Removed DESTROY subroutine. Seems if we attempt to call
427 disconnect from DESTROY then it doesn't work well. Not really sure
428 why this happens but rantest now calls $east->disconnect
429 directly. We should investigate this further because it would be
430 much better if DESTROY did the right thing</li>
432 <li>Implemented new testtimeout option. This option, entered into
433 the .test file, will set an overall limit of how long rantest will
434 wait for all test steps to complete. This is done via an alarm
435 signal. So rantest sets the alarm at the start of execution of the
436 test steps and a signal handler to catch the alarm signal. If caught
437 then rantest disconnects from EAST. Not sure how smoothly things
438 proceed from there (does rantest continue to the next .test
441 <li>Created runSuiteFile as a separate subroutine. This subroutine
442 will now parse off options after the .test file that override what's
443 in the .test file. This will allow us to specify "mytest.test -unit
444 6" so that even if mytest.test said to use unit 3, during this suite
445 run it would use unit 6.</li>
447 <li>Added user ID of who ran the test to the Test Step web page.</li>
449 <li>Added "news" link to this page</li>
451 <li>Added link to main page to the Nightly Log</li>
453 <li>Added/fixed p6000c and p4781c's ssh keys to authorized_keys</li>
455 <li>Added this ChangeLog</li>
458 <h2><a name="0.9.7">Version 0.9.7 - "The Rantvl release"</a></h2><hr>
462 <p>A word about rantvl. This is a rantest_tools tool so it resides on
463 the RAN, specifically under /prj/muosran/SWIT/tools/bin/rantvl. This
464 Perl script contacts the DUT and sets up logging via a moshell script
465 (which is why it needs to run on a RAN machine since moshell doesn't
466 work from the Linux EAST environment). It then kicks off a number of
467 processes that run in the background. All of these processes are
468 spawned off as children of rantvl. Rantvl then wait(3)'s for either
469 the children to finish (bad, bad - this is an error since the logging
470 should never finish) or for it to be terminated.</p>
472 <p>Rantest will open up a channel to RANHOST as RANUSER and run rantvl
473 supplying it the -[rnc|rbs] [n] parameters as well as a -logpath
474 parameter. The -logpath parameter is a path relative to
475 ranray:/export/rantvl, since this filesystem seems to have space. We
476 probably should verify and OK that. So then rantest will supply the
477 logpath of <testname>/<DUT>/<timestamp>. (Note to
478 Ken, while the logfiles may grow large, rantest/rantvl is pretty
479 efficient in that once the test is completed these files are copied to
480 our testlogs areas on seast1 and removed from
481 ranray:/export/rantvl).</p>
483 <p>Rantvl then will use that logpath to write it's logfiles. When
484 rantest is done running tests in a .test file it collects the rantvl
485 logfiles by scp'ing them from
486 <RANHOST>:/export/rantvl/<logpath>/* to seast1's LOGBASE -
487 /east/seast1/testlogs/<logpath>/Rantvl/*. Rantest will then
488 remove the logfiles from the source area
489 (<RANHOST>:/export/rantvl/<logpath>/*) to conserve space.
490 Rantvl is then added to the list of directories to tar up into a
492 href="http://rantestweb/testlogs/b2_l3_rnc_irt_001.rantvl/rnc5/20080527@14:20:59/">http://rantestweb/testlogs/b2_l3_rnc_irt_001.rantvl/rnc5/20080527@14:20:59/</a>).</p>
497 <li>Replaced bash script, update_view, with Perl script. Now does
498 -overwrite by default. Cronjob now calls it with -nooverwrite.</li>
500 <li>Added copy of authorized_keys to config. This is the ssh
501 authorized_keys file for all of the rantm50x machines as well as the
502 NMS simulator machines. This allows testers to ssh as pswit onto
503 these machines without needing a password. This facilitates
510 <li>Added support for rantvl.</li>
512 <li>Added LOGBASE</li>
514 <li>Added constants for RANHOST (ranray) and RANUSER (pswit). Both
515 Moshell and Rantvl need to connect remotely to a RAN machine as a
516 generic user. These constants facilitate that.</li>
518 <li>Changed to only attempt collection of log files if there log
519 files to collected. Rantest will set the member collectLogFiles to
520 true when there are log files to collect (i.e. when a test step is
521 attempted). It will also turn this off if the test run is
522 interrupted (Control-C) and abort is selected.</li>
524 <li>Added new set/getCollectLogFiles methods for the above.</li>
526 <li>Changed the logic in collectLogFiles a little bit to accommodate
527 collection of Rantvl log files too. When collecting Rantvl log files
528 they are scp'ed from RANHOST to the local host (in this case seast1)
529 under LOGBASE and then removed from RANHOST to conserve space.</li>
531 <li>Added pod documentation for new East methods.</li>
537 <li>Put web pages formally under Clearcase control.
538 cclinux:/var/www/html now points directly to
539 ./pswit_auto/vobs/rantest_auto/web.</li>
541 <li>php/Utils.php now puts out the version of the web pages (0.9.7)
542 so that they stay in line with the version of rantest and
545 <li>Added logic for new "test selectors" on the main page to
546 TestHistory.php.</li>
548 <li>Renamed links in the copyright section to shorter names. Also
549 added in Gantry's nice User's Guide: Home | Wiki | Users Guide |
552 <li>Changed RantestDB.php to categorize "Logging started" results
553 (from a rantvl log) in a similar fashion to "In progress" results as
554 the running of rantvl is really a backgrounded operation...</li>
560 <li>Changed $east object to a global since the INT signal handler
561 now needs to set state in it and there's no other way to get to the
562 object except to make it global. This required removing $east from
563 all subroutines it used to be passed to.</li>
565 <li>Added logic to bail out on a .test run if one could not connect
566 to EAST. Rantest will now go to the next test in the case of suite
567 execution (since the next case may have a different rnc, etc.)</li>
569 <li>Added code to handle new rantvl type steps.</li>
571 <li>Added code to treat "Logging started" like "In progress".</li>
573 <li>Error out if we cannot create our "saveTo" directory.</li>
575 <li>Changed to set umask to 022 for reduce the problems created by
576 not creating objects as group writable.</li>
578 <li>Changed to classify both Failed's and Timeout's as error and
579 exit with a total of both of them. IOW only if there are no Failed's
580 and no Timeout's will an exit 0 be done.</li>
582 <li>Updated rantest wiki page to 0.9.7. Also includes link to
586 <h2><a name="0.9.6">Version 0.9.6</a></h2><hr>
588 <p>I'm delivering rantest 0.9.6 with the following set of features in
589 anticipation of working on integrating rantvl for 0.9.7.</p>
594 <li>Now properly records the full path to the log files for this run
595 in the databse as testruns.eastlogs.</li>
597 <li>Errors in creation of paths are reported better.</li>
599 <li>New directories that are created are chmod 0775 to facilitate
600 sharing. Note that not every component is chmoded - only the leaf
603 <li>Disposition of log directories that are tarred is set to "keep"
604 now. This means that there will be a tar file and the source
605 directory for easy navigation through the web. It also means that
606 our testlogs directory will get fuller, faster.</li>
608 <li>bin/donightly has come back with a similar purpose but different
609 implementation. It now runs all of the .test and .suite files in
610 ../auto/test/nightly.</li>
612 <li>Changed testruns.eastlogs and testruns.cmlogs from tinytext to
613 just text. These log file paths are pretty long and what's a few
614 bytes between friends?</li>
616 <li>Changed rantest to save original command line options and
617 restore them at the start of each .test run.</li>
619 <li>Changed rantest to save the testStartTime so that the start
620 times reported on the web match those in the path to the
621 logfiles. This allows the web loglinks to work.</li>
623 <li>Reorganized a lot of the web stuff and put it firmly under
624 Clearcase control.</li>
626 <li>Fixed problem with drop downs in Firefox (1.0) when doing hover
627 rollovers on main page.</li>
629 <li>Rearranged front page and added "test selectors" to a few of the
630 reports. Also made the Test History report properly respond to these
631 new test selectors. if all boxes are "*" then Test History does an
632 "All Tests" report, otherwise it filters the qualifying entries and
633 reports on them.</li>
635 <li>Added Loglinks to web site. Engineers can now drill down to the
636 actual tests and then into the logs themselves to diagnois
639 <li>Removed Run By from Daily Test Report.</li>
641 <li>Changed Daily Test Report to report Unit and Version.</li>
643 <li>Added version number to copyright block on the web page.</li>
645 <li>Added getBuildNbr function to RantestDB.php.</li>
647 <li>Changed DBError calls to use __FUNCTION__.</li>
650 <h2><a name="0.9.5">Version 0.9.5</a></h2><hr>
653 <li>Takes two extensions for -file, .suite or .test. The .test is
654 like the old .suite files (in fact, I've renamed the old .suite
655 files -> test). Therefore the new .suite file is different - it's
656 simply a file listing a bunch of .test's to test.</li>
658 <li>Now uses LOGBASE, set to $ENV{MNT_DIR}/testlogs. All logs are
659 now stored under that.</li>
661 <li>Now uses TESTBASE, set to "/local/server/auto/test". This is the
662 base relative directory that rantest uses to look for tests and
665 <li>Verbose output improved. Since rantest can run suites, which
666 contain other tests, we needed to properly represent this in the
669 <li>The -saveto has been removed. Instead logs are saved to LOGBASE
670 (with appropriate subdirectory structure).</li>
672 <li>Added pattern to elock subcommand. Allows you to filter out
673 stuff. So "rantest elock rnc" will show only rnc's. Also, elock is a
674 short circut commands such that if it is given on the command line
675 as the above hints at, then elock is done and rantest exits. Note
676 elock still works inside rantest in interactive mode. Finally
677 colorized output.</li>
679 <li>recordRun changed to announceTestRun and is used before each
680 test run of a suite or individual test.</li>
682 <li>Added code to support new rantest DB format. Now we call
683 start[type]run and end[type]run (where type is Suite, Test, Step) to
684 record to the database the starting and ending of suites, tests and
687 <li>Changed runFile to call runTestFile for tests. runFile now
688 handles suite runs.</li>
690 <li>Changed East.pm to disconnect from East (i.e. shutdown
691 GenericDesktop and friends) after each execution of a test. This
692 requires that we start another one up for the next test in a
695 <li>Added methods to set TestCaseID and saveTo directory so that
696 saveLogFiles can be called in the East object destructor.</li>
698 <li>saveLogFiles now properly copies log files from their source
699 locations to our new LOGBASE location before tarring them up.</li>
701 <li>saveLogFiles now copies subdirs of EASTLogs as per Ross'
704 <li>saveLogFiles now removes the copied subdirs after the tar to
708 <p>One outstanding problem remains that occasionally rantest is unable
709 to find the logfile. I have a bugcatcher in 0.9.5 however it see ms
710 like there is no logfile directory to find log files in. Let me
711 restate, when we are done with testing we attempt to locate the
713 $MNT_DIR/$EAST_REL/DUT/<dut>/data/logs/regression/testcase/<testcase>.
714 At this point there should be a time stamped directory. So I'm doing
715 an ls -t and picking off the first entry.</p>
717 <p>What I've been finding is that all directories done the path to
718 <testcase> exist but the <testcase> directory is no there!
719 I can think of 3 reasons why this might be the case:</p>
722 <li>The directory just doesn't exist and rantest is right to
725 <li>The directory does not exist yet because some background process
726 has not yet created it. Well when I was debugging this I was sitting
727 in the debugger looking around for quite some time. In that time no
728 background process came along to create this directory. Besides all
729 testing processing has terminated long ago.</li>
731 <li>The directory used to exist and some process, thinking it was
732 done with this, decided to get rid of the directory.</li>