Opera: Free, Fast and Safe Browsing Valid XHTML 1.1! Valid CSS! this page on perl.org | this page on cmve.net | this page on hpux.ws
FAQ's | Full statistics | Summaries | Congratulations

Merijn's HP-UX software for ITRC members

Here's some answers to the FAQ's about the statistics I post on the HP ITRC Forum, and on my site.


Frequently Asked Questions


Full statistics

These are the statistics I put on my site. It's a full ordered list all members that have at least one point assigned to in the ITRC forum. Taking the top 22 of the ranked table of end April 2004 as example:

Rank PointsID Name Country AsignAnswrRateSinceLst postBunnyDiffRspnss
1 ! 50518CA275236A. Clay Stephenson Usa 67 67100.01998200404202769 +4054352
2 38289CA114842James R. Ferguson Usa 82 82100.01997200309052386 4141
3 31031BR458 Stefan Farrelly Uk 244 288 84.7199620040407 994 2119
4 ! 29737CA569821Steven E Protter Usa 15321530100.12001200404211453 +4765710
5 28050CA13182 Pete Randall Usa 14691458100.81996200404201320 3250
6 ! 26576CA362973Michael Tully Australia 664 673 98.71999200404211440 +1634325
7 23868CA18242 Patrick Wallek Usa 994 993100.11996200404201423 3029
8 23457CA592260harry d brown jr Usa 641 643 99.72000200404201033 4459
9 20806CA520085S.K. Chan Usa 57 57100.02000200310141263 2179
10 20538CA675588Sridhar Bhaskarla Usa 0 0 -2001200403141150 2478
11 ! 19756CA752641Jeff Schussele Usa 434 435 99.82002200404201053 +2262582
12 ! 18528CA486994Bill Hassell Usa 64 64100.02000200404201204 +2553046
13 16727CA591749T G Manikandan India 405 405100.0200020040420 836 2660
14 ! 16606CA632734Sanjay Usa 24 24100.0200120040421 897 +3082868
15 15720CA503958Shiju Wilson Usa 370 370100.0200020040419 840 2057
16 12967BR661700Leif Halvarsson Sweden 280 289 96.9200220040420 675 2208
17 12900BR154011Alex Glennie Uk 129 133 97.0200020040416 735 1965
18 11508BR8935 Wodisch Germany 256 258 99.2199720040417 550 2713
19 ! 11278BR148616Robert-Jan GoossensNetherlands11551154100.1200020040420 610 +3141885
20 ! 11160BR41967 procura Netherlands 955 946101.0199720040420 412 +991198
21 10837BR170990Kurt Beyers Belgium 377 374100.8200020040420 563 2136
22 10416BR180643Bill McNAMARA France 15671592 98.4200020040304 464 2187
ColumnHeaderDescription
1Rank This is the position in the list of all members that have at least one point in ITRC sorted on points, bunny, rate, since, and ID using this code in the stat scripts.
foreach $id (map  { $_->[0] }
	     sort { $b->[1] <=> $a->[1] || # points
	            $b->[2] <=> $a->[2] || # bunny
	            $b->[3] cmp $a->[3] || # rate
	            $b->[4] <=> $a->[4] || # since
	            $b->[0] cmp $a->[0] }  # ID
	     map  { [ $_, $db{ptsitrc}{$_}, $db{bunny}{$_},
		      $db{rate}{$_}, $db{since}{$_} ] }
	     map  { foreach my $cat (qw( ptsitrc bunny rate since )) {
			unless (exists $db{$cat}{$_}) {
			    printf STDERR "%-12s has no %s\n", $_, $cat;
			    next;
			    }
			unless (defined $db{$cat}{$_}) {
			    printf STDERR "%-12s has undefined %s\n", $_, $cat;
			    next;
			    }
			}
		    $_;
		    }
	     @id) {
    write;
    }
2Fast This (!) is an indication that this member made it to the fast movers list in the period between the previous report and this one. The list currently has a minimum of 24 entries, which is extended to include at least all century breaks. I also cheat: If I am not on the list already, but the last entry on the regular list is less than 8 points away from the score I had, the list will extend to include me, like:
foreach my $diff (sort { $b <=> $a } keys %diff) {
    $diff < $bound and last;
    foreach $id (@{$diff{$diff}}) {
	printf STDERR " %-12s %+5d %5d %4d %7s %8s %s\n",
	    $id, $diff,
	    $db{ptsitrc}{$id},
	    $db{bunny}{$id},
	    $db{rate}{$id},
	    $db{lastc}{$id},
	    $db{name}{$id};
	$id{$id}{fast} = "!";
	$id{$id}{diff} = "+$diff";
	$donef{$id}++;
	$nfast++;
	}
    # When to stop:
    #  at least $maxf (20) entries
    $nfast > $maxf or next;
    #  at century breaks
    if ($diff > 100) {
	$bound = 100;
	next;
	}
    #  and me if I'm within reach of the last
    if (!exists $donef{BR41967} && $mydiff >= $diff - 7) {
	$bound = $mydiff;
	next;
	}
    last;
    }
3Points Until 20 September 2005, this was the grand total of all points the member earned in the combined forums (ITRC and BSF).
Now this column represents the points a member has earned in the ITRC forum only. The BSF points and the grand total are now represented in two new columns on the right. (unlike the totals seen on the HP forum pages, which are per-forum), like:
while ($ml =~ m{<tr\b.*?href="/service/forums/.*?userId=([A-Z]+\d+)"
		.*?>\s*(.*?)\s*</a>
		.*?>\s*(\d+)
		.*?>[A-Z][a-z]+\s+\d+,\s+(\d+)
		.*?</tr
		}gxis) {
    print STDERR "\r$base $1\t";
    my ($id, $name, $points, $since) = ($1, $2, $3, $4);
    exists $banned{$id} and next;
    $since =~ s/(\S+) (\d+),\s*(\d+)/sprintf "%04d%02d%02d", $3, $month{$1}, $2/e;
    if ($points) {
	exists $db{points}{$id} or $db{points}{$id} = $points;
	if ($points < $db{points}{$id}) {
	    print STDERR "WARN: $id gone backwards from $db{points}{$id} to $points! (restored)\n";
	    $points = $db{points}{$id}--;
	    }
	if ($points > $db{ptsitrc}{$id} or
	    !exists $db{lastc}) {
	    push @x, $id;
	    }
	}
    $db{name}   {$id} = $name;
    $db{points} {$id} = $points;
    $db{since}  {$id} = $since;

As you can read in the script, statistical refreshes are only performed when a member has registered a higher number of points than the current points in my pseudo database. This means that if you only get points in other parts of the forum (those where the points will not add to the user profile I check in the code as shown), there will be no update to the stats. Once the member makes a single point increase in the main forum, an update will occur, and all points are added again.

4ID This is the member's user ID, which is the key for all member related data.
5Name This is the member's name as is shows on posts in the Forum. A members name does not have to be unique. It's more like an alias to one's ID. Fetched in one sweep with points. See for script code there.
6Country This is the country the member is registered to be registered from. No guarantee that he's posting from there.
7Asign This is the total number of messages that the user has assigned points to.
8Answr This is the total number of posts (answers) that a member has received on his/her posted questions.
9Rate This is the percentage of answers that already have been assigned points to.
If the rate is "-", the user has never posted any questions.
If the rate is below 100.0%, the user must visit his/her profile page and follow the link labeled "questions or topics with unassigned points" in the "my question(s)" section.
If the rate is above 100.0%, the member has received points in threads that have been deleted by the Forum moderators after the points were assigned (or they were assigned multiple times way back in time when the software allowed on to go back to the assign page and hit submit again :)
10Since This is a timestamp by the HP Forum of the year, and if more recent also month and date of since when the member was registered. This has no relation whatsoever to the start of activity of the member, which can start months or even years later.
11Lst post This is the date of the last post that was submitted by the member.
12Bunny This is the total number of threads where bunnies (answers that were assigned 8, 9, or 10 points) were received on answers of the member. Since April 2004, this count is only counting threads, so if a member received more bunnies in a single thread, it still counts only one.
13Diff For fast movers this field shows the point gain since last report.
14Rspnss This is the total number of threads the member has responded in.
15Eff Efficiency. Until April 2004, this number indicated the average score the member got for all his answers, but now that for bunnies and answers HP only counts the threads, the usefulness of this figure is debatable. If you are really interested in the average score you earned, use the full_fetch script from the statistics script collection.
16PtBSF This is the total number of points gathered in the BSF forum.
17PtTotal This is the total number of points gathered in the ITRC and BSF forum added.

A full interactive version of all historical data is available on Alex Lavrov's site.


Statistic summaries

These are the posts I make to the Forum every Wednesday morning MET, and they look a bit different. The database updates are done through cron, and only fetch data for members that have gained points since last report.

Introduction / Summary

Fast movers are compared to archive/id-20040414.txt.gz
Pre-scanning 51735 members, 9325 have points  prev 20040413

Fast Movers List

Fast Movers:

 ID           Diff  Point Bunn  Rate   Last Msg Name
 ============ ===== ===== ==== ======= ======== =========================
 CA569821      +476 29737 1453 100.1 % 20040421 Steven E Protter
 CA275236      +405 50518 2769 100.0 % 20040420 A. Clay Stephenson
 CA954298      +390   978   57     -   20040420 Rudy
 CA615696      +363  7089  290  99.4 % 20040421 Roger Faucher
 BR680842      +357  8286  432 100.0 % 20040421 Mark Grant
 BR776558      +349  1999   85 100.0 % 20040420 Uwe Zessin
 BR148616      +314 11278  610 100.1 % 20040420 Robert-Jan Goossens
 CA632734      +308 16606  897 100.0 % 20040421 Sanjay
 CA486994      +255 18528 1204 100.0 % 20040420 Bill Hassell
 CA963445      +253  2759  122 100.0 % 20040420 Martin P.J. Zinser
 CA1035813     +248   946   24  20.0 % 20040421 Cheryl
 BR41967       +235 11296  420 100.9 % 20040427 procura
 CA752641      +226 19756 1053  99.8 % 20040420 Jeff Schussele
 CA460479      +223  1548   78 100.5 % 20040420 Kapil Raj
 CA888381      +216  1638   48 117.2 % 20040421 Ganesh Babu
 CA1093611     +190   535   33     -   20040420 Ramesh Pakkath
 BR655479      +189  4439  216 100.0 % 20040420 G. Vrijhoeven
 CA893598      +182  1940   95  90.9 % 20040421 Ernest Ford
 CA648840      +176  5060  256 100.0 % 20040420 Geoff Wild
 CA753474      +174  4315  220 100.0 % 20040421 Stuart Browne
 BR760764      +171  1765   53 100.0 % 20040420 Bruno Ganino
 BR757936      +166  1725   97  96.2 % 20040420 Jeroen Peereboom
 CA362973      +163 26576 1440  98.7 % 20040421 Michael Tully
 CA899580      +161  3362  157 100.0 % 20040421 Frank Kaufman
 CA947949      +160  3172  113 100.8 % 20040421 JOHN YELTON
 CA874247      +158  9321  531 100.0 % 20040420 drew dimmick
 CA500989      +157  2781  129 100.1 % 20040420 Sanjiv Sharma
 CA1035862     +153  1690   72 100.0 % 20040420 Sanjay Kumar Suri
 CA638194      +151   420   25     -   20040421 SS
 BR141494      +141   251   13  65.8 % 20040420 Gary Cooper
 CA984331      +126  5890  239 100.9 % 20040420 Michael Schulte
 BR778774      +122  2986  153 100.0 % 20040416 Elmar P. Kolkman
 CA982285      +115   699   31 100.0 % 20040420 Nicolas Dumeige
 BR640407      +114  2360  131 100.0 % 20040420 Marino Meloni
 CA1038666     +107  1194   56     -   20040420 Frans
 CA278572      +104  3228  183 100.0 % 20040420 Jim Donovan
 CA871531      +104   737   41  50.0 % 20040420 Bob Roell
 BR53811       +102  3981  258 100.0 % 20040420 Charles Guiocheau
 CA861818      +100  1678  111   0.0 % 20040420 David Claypool

Marked 40 fast movers in 9325 members

Hat count

   10 Olympian's
   22 Pharaoh's
  117 Royalty's
  174 Wizard's
  190 Graduate's
  238 Pro's

Promotees

Promoted to Wizard:
        1028 20040420   198    34  79.3 % CA649974  Dan Gazzaniga

Promoted to Graduate:
         543 20040420   135    28 200.0 % CA1012689 Kim Torkelson
         536 20040420   321    31   0.0 % CA898307  e4Services.com
         535 20040420   181    33     -   CA1093611 Ramesh Pakkath
         528 20040420   134    34     -   CA857800  Doug de Werd
         518 20040420   150    26     -   CA772559  Rory R Hammond
         512 20040415   147    25  84.4 % BR635031  malvin drakley
         504 20040419    92    24     -   BR761173  Andre van Beuzekom
         503 20040415   132    25 100.0 % CA439752  Seth Parker
         502 20040412    93    24 100.0 % CA1049963 Dave Hutton
         502 20040420   136    26  93.8 % CA1095325 Shaikh Imran

Promoted to Pro:
         263 20040420    55    14  25.9 % CA637313  Matt Bender
         254 20040418    33    15  53.8 % CA963429  Craig A. Berry
         253 20040420    95    16     -   CA938006  Brian Murdoch
         251 20040420    65    13  65.8 % BR141494  Gary Cooper

Almost Promoters

Almost Pharaoh:
        7417 20040420  1216   324 101.5 % BR89052   Massimo Bianchi

Almost Wizard:
         978 20040420   340    57     -   CA954298  Rudy
         955 20040415   394    53  25.0 % CA467151  Terri Harris
         946 20040421    81    24  20.0 % CA1035813 Cheryl

Almost Graduate:
         499 20040416   124    24  67.4 % CA822220  Olivier Drouin
         479 20040414   106    31     -   CA1007312 Jeff Brown

Almost Pro:
         245 20040402    79    14 100.0 % CA40300   Denis St-Amand
         240 20040419    94    13 100.0 % CA944939  Larry Straile
         238 20040415    64     0     -   BR793404  F. Beest
         234 20040420    56     8 100.0 % CA621405  Celso Medina Kern
         233 20040420    71     0 100.0 % CA441690  Jov
         232 20040330    76    12     -   CA467660  R. Allan Hicks

This section causes many questions. Here's the code that determines if someone is to be mentioned in the almost list:

if ($db{lastc}{$id} >= $almost) { # must have posted within the last month
       if ($points >= 80_000) { $points >= 160_000 and push @{$almost{Frantic} }, $id }
    elsif ($points >= 40_000) { $points >=  80_000 and push @{$almost{OlymStr2}}, $id }
    elsif ($points >= 20_000) { $points >=  39_750 and push @{$almost{OlymStar}}, $id }
    elsif ($points >=  7_500) { $points >=  19_800 and push @{$almost{Olympian}}, $id }
    elsif ($points >=  2_500) { $points >=   7_350 and push @{$almost{Pharaoh} }, $id }
    elsif ($points >=  1_000) { $points >=   2_400 and push @{$almost{Royalty} }, $id }
    elsif ($points >=    500) { $points >=     915 and push @{$almost{Wizard}  }, $id }
    elsif ($points >=    250) { $points >=     460 and push @{$almost{Graduate}}, $id }
    elsif ($points >=    230) {                        push @{$almost{Pro}     }, $id }
    }

And, no, Frantic is not an official level bound. It's a concoction of mine while writing the code.

Blame Log

  00 ..   09 %  1267
  10 ..   19 %   253
  20 ..   29 %   353
  30 ..   39 %   351
  40 ..   49 %   297
  50 ..   59 %   563
  60 ..   69 %   496
  70 ..   79 %   382
  80 ..   89 %   414
  90 ..   99 %   385
         100 %  1151
 101 ..  109 %    36
 110 ..  119 %     3
 130 ..  139 %     3
 140 ..  149 %     2
 150 ..  159 %     1
 200 ..  209 %     3
Average: 53.7 %

Since these statistics are all about point assignment, and many members often complain about points not being assigned or points are being assigned in unfair ways, I once started a full blame list: The same list as the full statistics, but now ordered by rate, answers, bunny, points, and ID using this beautiful Schwarzian Transform:

my ($sum, $cnt, $p100, %rt) = (0, 0, 0);
foreach $id (map  { $_->[3] == 100.0 && $p100++;
		    $sum += $_->[3];
		    $rt{int ($_->[3] / 10)}++;
		    $_->[0] }
	     sort { $a->[3] <=> $b->[3] || # rate, asc
		    $b->[4] <=> $a->[4] || # answers, desc
		    $a->[2] <=> $b->[2] || # bunny, asc
		    $a->[1] <=> $b->[1] || # points
		    $a->[0] cmp $b->[0] }  # ID
	     map  { [ $_, $db{ptsitrc}{$_}, $db{bunny}{$_},
		      substr ($db{rate}{$_}, 0, 5), $db{answers}{$_} ] }
	     grep { exists $db{answers}{$_} and $db{answers}{$_} > 0}
	     @id) {
    write;
    }
foreach my $rt (sort { $a <=> $b } keys %rt) {
    if ($rt == 10) {
	printf "         100 %% %5d\n", $p100;
	printf "%3d1 .. %3d9 %% %5d\n", $rt, $rt, $rt{$rt} - $p100;
	}
    else {
	printf "%3d0 .. %3d9 %% %5d\n", $rt, $rt, $rt{$rt};
	}
    $cnt += $rt{$rt};
    }
printf "Average: %4.1f %%\n", $sum / $cnt;

Personal Congratulations

Personal congratulations are done by other members, who start a thread to congratulate a member with the newly acquired hat. It's from Patrick's etiquette that congrats are started only for Wizards and above.
Threads are usually started in the forum where the member is most active.

Since it's rather obvious that the members in the Buisiness Forum do not appreciate these metrics, I will not post there.

Table

A table is generated showing the progress the member has made over time. Only for the archived data, which is static.

20040107    20   9545 BR41967   procura                  Netherlands      833   824 101.1     1997 20040105   471      0   2007  3.85
20040114    20   9620 BR41967   procura                  Netherlands      842   833 101.1     1997 20040114   474     75   2016  3.86
20040121    20   9673 BR41967   procura                  Netherlands      847   838 101.1     1997 20040119   477     53   2028  3.86
20040128    20!  9790 BR41967   procura                  Netherlands      850   841 101.1     1997 20040127   480   +117   2052  3.87
20040204    20   9875 BR41967   procura                  Netherlands      865   856 101.1     1997 20040203   484     85   2074  3.86
20040211    20! 10037 BR41967   procura                  Netherlands      867   858 101.0     1997 20040210   493   +162   2090  3.89
20040218    20! 10180 BR41967   procura                  Netherlands      879   870 101.0     1997 20040217   500   +143   2116  3.89
20040225    19! 10421 BR41967   procura                  Netherlands      897   888 101.0     1997 20040224   382   +241   1127  6.91
20040303    19! 10542 BR41967   procura                  Netherlands      915   906 101.0     1997 20040302   386   +121   1138  6.92
20040310    19  10625 BR41967   procura                  Netherlands      924   915 101.0     1997 20040310   392     83   1142  6.93
20040317    19  10641 BR41967   procura                  Netherlands      928   919 101.0     1997 20040315   392     16   1146  6.92
20040324    19  10716 BR41967   procura                  Netherlands      936   927 101.0     1997 20040323   395     75   1153  6.92
20040331    19! 10923 BR41967   procura                  Netherlands      939   930 101.0     1997 20040330   406   +207   1164  6.96
20040407    19  11018 BR41967   procura                  Netherlands      940   931 101.0     1997 20040406   408     95   1175  6.96
20040414    19  11061 BR41967   procura                  Netherlands      940   931 101.0     1997 20040406   408     43   1175  6.99
20040421    20! 11160 BR41967   procura                  Netherlands      955   946 101.0     1997 20040420   412    +99   1198  6.93
DB              11296 BR41967   procura                  Netherlands      969   960 100.9     1997 20040427   420    136   1204  6.96
Average points per report: 115.21

The columns are almost the same as with the full statistics:

  1. Date of statistics (DB is current state in my database);
  2. Rank;
  3. Fast Mover;
  4. ITRC Points;
  5. ID;
  6. Name;
  7. Country;
  8. Assigned answers;
  9. Answers received;
  10. Rate;
  11. Member since;
  12. Date last post;
  13. Number of Bunny threads;
  14. Point gain since previous report. If it has a plus, you will also see an exclamation mark in column 3, as it indicates that it was reported in the fast movers list. The plus-less diff's are calculated on the fly;
  15. Number of threads with Responses;
  16. Efficiency;
  17. BSF Points;
  18. Total Points.

Graph

A graphical representation is generated over the collected data for the member and the members close to the member. The congratulated ID is usually represented with the Green line which is just a bit wider than the other graph lines.

The black text shown in the top area of the graph is the member's personal quote, if he/she has one. I've not gone through the trouble of trying to fit it if it is too long.

The legenda on the right, shows the members ID's and names for the graph lines included in the picture. Note that points with the names might not be the actual situation, since with congrats only the data for the congratulated member are updated to the database. The data for the other members reflect the last time their data was fetched.

Graphs are creted with Perl/Tk and convert. If you're interested, look at graph.pl script from the statistics script collection.

Alex also provides graphs on his site. For example, see mine


Enjoy, Have FUN! H.Merijn

Opera: Free, Fast and Safe Browsing Valid XHTML 1.1! Valid CSS! © H.Merijn Brand [ 18 Aug 2008 ]