defaria.com/blogs/MAPS/index.html

   1 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
   2     "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
   3 <html xmlns="http://www.w3.org/1999/xhtml" id="sixapart-standard">
   4 <head>
   5     <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
   6 <meta name="generator" content="Movable Type 5.2.3" />
   7 <link rel="stylesheet" href="http://defaria.com/blogs/MAPS/styles.css" type="text/css" />
   8 <link rel="start" href="http://defaria.com/blogs/MAPS/" title="Home" />
   9 <link rel="alternate" type="application/atom+xml" title="Recent Entries" href="http://defaria.com/blogs/MAPS/atom.xml" />
  10 <script type="text/javascript" src="http://defaria.com/blogs/MAPS/mt.js"></script>
  11
  12
  13     <link rel="EditURI" type="application/rsd+xml" title="RSD" href="http://defaria.com/blogs/MAPS/rsd.xml" />
  14     <title>MAPS</title>
  15 </head>
  16 <body id="classic-blog" class="mt-main-index layout-wtt">
  17     <div id="container">
  18         <div id="container-inner">
  19
  20
  21             <div id="header">
  22     <div id="header-inner">
  23         <div id="header-content">
  24
  25
  26             <h1 id="header-name"><a href="http://defaria.com/blogs/MAPS/" accesskey="1">MAPS</a></h1>
  27             <h2 id="header-description"></h2>
  28
  29         </div>
  30     </div>
  31 </div>
  32
  33
  34
  35             <div id="content">
  36                 <div id="content-inner">
  37
  38
  39                     <div id="alpha">
  40                         <div id="alpha-inner">
  41
  42
  43                             <div id="entry-651" class="entry-asset asset hentry">
  44     <div class="asset-header">
  45         <h2 class="asset-name entry-title"><a href="http://defaria.com/blogs/MAPS/archives/2007/07/hit-count-and-l.html" rel="bookmark">Hit Count and Last Hit</a></h2>
  46         <div class="asset-meta">
  47             <span class="byline">
  48
  49                 <abbr class="published" title="2007-07-13T17:26:08-06:00">July 13, 2007  5:26 PM</abbr>
  50
  51             </span>
  52
  53
  54
  55
  56         </div>
  57     </div>
  58     <div class="asset-content entry-content">
  59
  60         <div class="asset-body">
  61             <p>After having left MAPS pretty much alone for a long time, though I did implement "nuke" which helps nuke whole domains, I finally have implemented new functionality for MAPS. You see over this last year or so I've been getting a lot of spam mostly oriented in which the domain name was wildly varying. So lots of mail loops were happening. Whacking whole domains became a favorite past time of mine and hence the "nuke" script (not an official part of MAPS mind you). Still it seemed like a losing battle.</p>
  62
  63 <p>Then I got to thinking - something I've been thinking for a while - how effective are all of my null list entries, now numbering in the 17K area! I thought that I could easily add some fields to the list table, namely a hit_count and a last_hit field, and then during scrubbing time see if these null list entries were at all effective, and if not then to automatically remove them.</p>
  64
  65 <p>So I added the fields and changed MAPS such that it records the hits and when the last hit happened. Then in mapsscrub check to see if last_hit is older than the number of days the user is keeping history. If it's 0 then simply remove that entry. Note this happens only for the null list. White list and black list are to be managed by the user. You purposely add somebody to the white or black list for a reason. Do we want to auto age these off? Probably not.</p>
  66
  67 <p>If the hit count is not 0 then how do we age off these apparently no longer effective null list entries? Well the though was to age them by subtracting 1 from the hit count. Thus if we had a null list entry that was fairly active for a little while and say accumulated a hit_count of 10 but then was largely un-hit, the hit_count would go 9, 8, 7, 6, 5... down to 0 and then the entry would be removed. I might want to change this as some null list entries are getting hit_counts in the thousands now! Assuming the user has a 30 day history then perhaps after 30 days of inactivity I should age any null list entry off the list in 7 days? Note that any activity in those 30 days would immediately reset the clock for at least the next 30 days (assuming a history of 30 days).</p>
  68
  69 <p>With this in place I decided to change the mail loop procedure to auto add the sender to the null list. Previously I was whacking off the whole domain - here we are just auto adding the individual sending address. But it's automatically added, automatically tracked for effectiveness and automatically aged off. Beside I still have the option of perusing the top returning domains and can whack whole domains still. Now any hits on any users in the domain will continue to insure that that domain null list entry will remain and not automatically age away...</p>
  70         </div>
  71
  72
  73
  74     </div>
  75     <div class="asset-footer"></div>
  76 </div>
  77
  78
  79
  80                             <div id="entry-542" class="entry-asset asset hentry">
  81     <div class="asset-header">
  82         <h2 class="asset-name entry-title"><a href="http://defaria.com/blogs/MAPS/archives/2006/04/no-more-defaria.html" rel="bookmark">No more &lt;anything&gt;@defaria.com</a></h2>
  83         <div class="asset-meta">
  84             <span class="byline">
  85
  86                 <abbr class="published" title="2006-04-15T22:58:40-06:00">April 15, 2006 10:58 PM</abbr>
  87
  88             </span>
  89
  90
  91
  92
  93         </div>
  94     </div>
  95     <div class="asset-content entry-content">
  96
  97         <div class="asset-body">
  98             <p>Due to an influx of spam getting through MAPS with a format of &lt;something&gt;@defaria.com I finally decided to fix MAPS to handle that better. Now if the from is from defaria.com (or the current domain) then the user name is looked up in /etc/passwd using getpwent and if there is no user of that name it is simply nulllisted. This is an obvious attempt by spammers to forge from email addresses.</p>
  99
 100 <p>Also removed logging of when the message is obviously garbled or not conforming to standards. Plus I no longer log if the user uses andrew@defaria.com but fails to have a name such as "Andrew DeFaria &lt;andrew@defaria.com&gt;". The former turns out to be mostly Nigerian spam anyway.</p>
 101
 102 <p>FInally, replaced /usr/local/amps with a direct checkout form CVS!</p>
 103         </div>
 104
 105
 106
 107     </div>
 108     <div class="asset-footer"></div>
 109 </div>
 110
 111
 112
 113                             <div id="entry-469" class="entry-asset asset hentry">
 114     <div class="asset-header">
 115         <h2 class="asset-name entry-title"><a href="http://defaria.com/blogs/MAPS/archives/2005/11/maps-scrubber.html" rel="bookmark">MAPS Scrubber</a></h2>
 116         <div class="asset-meta">
 117             <span class="byline">
 118
 119                 <abbr class="published" title="2005-11-06T10:17:58-06:00">November  6, 2005 10:17 AM</abbr>
 120
 121             </span>
 122
 123
 124
 125
 126         </div>
 127     </div>
 128     <div class="asset-content entry-content">
 129
 130         <div class="asset-body">
 131             <p>Updated mapsscrub to also report total number of user emails in the database. Also fixed header.</p>
 132         </div>
 133
 134
 135
 136     </div>
 137     <div class="asset-footer"></div>
 138 </div>
 139
 140
 141
 142                             <div id="entry-468" class="entry-asset asset hentry">
 143     <div class="asset-header">
 144         <h2 class="asset-name entry-title"><a href="http://defaria.com/blogs/MAPS/archives/2005/11/added-page-drop.html" rel="bookmark">Added page drop down</a></h2>
 145         <div class="asset-meta">
 146             <span class="byline">
 147
 148                 <abbr class="published" title="2005-11-05T10:44:37-06:00">November  5, 2005 10:44 AM</abbr>
 149
 150             </span>
 151
 152
 153
 154
 155         </div>
 156     </div>
 157     <div class="asset-content entry-content">
 158
 159         <div class="asset-body">
 160             <p>Added a page drop down to list.php. This is useful when one wants to say <i>go to the last page</i>.</p>
 161         </div>
 162
 163
 164
 165     </div>
 166     <div class="asset-footer"></div>
 167 </div>
 168
 169
 170
 171                             <div id="entry-150" class="entry-asset asset hentry">
 172     <div class="asset-header">
 173         <h2 class="asset-name entry-title"><a href="http://defaria.com/blogs/MAPS/archives/2004/03/move-list-entry.html" rel="bookmark">Move list entry bug</a></h2>
 174         <div class="asset-meta">
 175             <span class="byline">
 176
 177                 <abbr class="published" title="2004-03-08T09:17:23-06:00">March  8, 2004  9:17 AM</abbr>
 178
 179             </span>
 180
 181
 182
 183
 184         </div>
 185     </div>
 186     <div class="asset-content entry-content">
 187
 188         <div class="asset-body">
 189             <p>Found a bug today. While adding to the blacklist an entry was on the white list, MAPS correctly adds the entry to the black list and deletes the entry on the white list. However it fails to resequence the white list causing problems down the line.</p>
 190         </div>
 191
 192
 193
 194     </div>
 195     <div class="asset-footer"></div>
 196 </div>
 197
 198
 199
 200                             <div id="entry-139" class="entry-asset asset hentry">
 201     <div class="asset-header">
 202         <h2 class="asset-name entry-title"><a href="http://defaria.com/blogs/MAPS/archives/2004/02/login-bug.html" rel="bookmark">Login bug</a></h2>
 203         <div class="asset-meta">
 204             <span class="byline">
 205
 206                 <abbr class="published" title="2004-02-17T23:25:05-06:00">February 17, 2004 11:25 PM</abbr>
 207
 208             </span>
 209
 210
 211
 212
 213         </div>
 214     </div>
 215     <div class="asset-content entry-content">
 216
 217         <div class="asset-body">
 218             <p>Squashed a nasty bug, well actually a lack of implementation. After converting some of the web pages to PHP I neglected to implement Login in PHP! Now implemented.</p>
 219         </div>
 220
 221
 222
 223     </div>
 224     <div class="asset-footer"></div>
 225 </div>
 226
 227
 228
 229                             <div id="entry-138" class="entry-asset asset hentry">
 230     <div class="asset-header">
 231         <h2 class="asset-name entry-title"><a href="http://defaria.com/blogs/MAPS/archives/2004/02/space.html" rel="bookmark">Space</a></h2>
 232         <div class="asset-meta">
 233             <span class="byline">
 234
 235                 <abbr class="published" title="2004-02-16T23:12:26-06:00">February 16, 2004 11:12 PM</abbr>
 236
 237             </span>
 238
 239
 240
 241
 242         </div>
 243     </div>
 244     <div class="asset-content entry-content">
 245
 246         <div class="asset-body">
 247             <p>Added new "Space Usage" report. Not sure if I like this because 1) it's too slow and 2) the resulting report is not very impressive - it's just a line saying "User is using X bytes in the databsase". Might wish to implement a space counter in the user table. This would require that all procedures updating the database update the space counter. The intent for this space counter is to judge how much space a user is using in the database for the purposes of quotas or perhaps to charge people if they want more space.</p>
 248
 249 <p>Also, currently this space report is only counting space used in the email table. Might wish to add in the log table, which can add up, and the user and useropts tables. The latter two are minimal but including them would make the report more accurate.</p>
 250         </div>
 251
 252
 253
 254     </div>
 255     <div class="asset-footer"></div>
 256 </div>
 257
 258
 259
 260
 261                             <div class="content-nav">
 262                                 <a href="http://defaria.com/blogs/MAPS/archives.html">Archives</a>
 263                             </div>
 264
 265
 266                         </div>
 267                     </div>
 268
 269
 270                     <div id="beta">
 271     <div id="beta-inner">
 272
 273
 274
 275     <div class="widget-archive widget-archive-category widget">
 276     <h3 class="widget-header">Categories</h3>
 277     <div class="widget-content">
 278
 279     </div>
 280 </div>
 281
 282
 283
 284
 285     <div class="widget-archive-monthly widget-archive widget">
 286     <h3 class="widget-header">Monthly <a href="http://defaria.com/blogs/MAPS/archives.html">Archives</a></h3>
 287     <div class="widget-content">
 288         <ul>
 289
 290             <li><a href="http://defaria.com/blogs/MAPS/archives/2007/07/">July 2007 (1)</a></li>
 291
 292
 293
 294             <li><a href="http://defaria.com/blogs/MAPS/archives/2006/04/">April 2006 (1)</a></li>
 295
 296
 297
 298             <li><a href="http://defaria.com/blogs/MAPS/archives/2005/11/">November 2005 (2)</a></li>
 299
 300
 301
 302             <li><a href="http://defaria.com/blogs/MAPS/archives/2004/03/">March 2004 (1)</a></li>
 303
 304
 305
 306             <li><a href="http://defaria.com/blogs/MAPS/archives/2004/02/">February 2004 (3)</a></li>
 307
 308
 309
 310             <li><a href="http://defaria.com/blogs/MAPS/archives/2003/11/">November 2003 (3)</a></li>
 311
 312
 313
 314             <li><a href="http://defaria.com/blogs/MAPS/archives/2003/10/">October 2003 (9)</a></li>
 315
 316         </ul>
 317     </div>
 318 </div>
 319
 320
 321
 322
 323
 324 <div class="widget-pages widget">
 325     <h3 class="widget-header">Pages</h3>
 326     <div class="widget-content">
 327         <ul>
 328
 329
 330         </ul>
 331     </div>
 332 </div>
 333
 334 <div class="widget-syndication widget">
 335     <div class="widget-content">
 336         <ul>
 337             <li><img src="http://defaria.com/mt/mt-static/images/status_icons/feed.gif" alt="Subscribe to feed" width="9" height="9" /> <a href="http://defaria.com/blogs/MAPS/atom.xml">Subscribe to this blog's feed</a></li>
 338
 339         </ul>
 340     </div>
 341 </div>
 342 <div class="widget-powered widget">
 343     <div class="widget-content">
 344         <a href="http://www.movabletype.com/"><img src="http://defaria.com/mt/mt-static/images/bug-pbmt-white.png" alt="Powered by Movable Type 5.2.3" width="120" height="75" /></a>
 345     </div>
 346 </div>
 347
 348
 349
 350     </div>
 351 </div>
 352
 353
 354
 355
 356 <div id="gamma">
 357     <div id="gamma-inner">
 358     <div class="widget-search widget">
 359     <h3 class="widget-header">Search</h3>
 360     <div class="widget-content">
 361         <form method="get" action="http://defaria.com/mt/mt-search.cgi">
 362             <input type="text" id="search" class="ti" name="search" value="" />
 363
 364             <input type="hidden" name="IncludeBlogs" value="3" />
 365
 366             <input type="hidden" name="limit" value="20" />
 367             <input type="submit" accesskey="4" value="Search" />
 368         </form>
 369     </div>
 370 </div>
 371
 372 <div class="widget-recent-entries widget-archives widget">
 373     <h3 class="widget-header">Recent Entries</h3>
 374     <div class="widget-content">
 375         <ul>
 376
 377             <li><a href="http://defaria.com/blogs/MAPS/archives/2007/07/hit-count-and-l.html">Hit Count and Last Hit</a></li>
 378
 379
 380
 381             <li><a href="http://defaria.com/blogs/MAPS/archives/2006/04/no-more-defaria.html">No more &lt;anything&gt;@defaria.com</a></li>
 382
 383
 384
 385             <li><a href="http://defaria.com/blogs/MAPS/archives/2005/11/maps-scrubber.html">MAPS Scrubber</a></li>
 386
 387
 388
 389             <li><a href="http://defaria.com/blogs/MAPS/archives/2005/11/added-page-drop.html">Added page drop down</a></li>
 390
 391
 392
 393             <li><a href="http://defaria.com/blogs/MAPS/archives/2004/03/move-list-entry.html">Move list entry bug</a></li>
 394
 395
 396
 397             <li><a href="http://defaria.com/blogs/MAPS/archives/2004/02/login-bug.html">Login bug</a></li>
 398
 399
 400
 401             <li><a href="http://defaria.com/blogs/MAPS/archives/2004/02/space.html">Space</a></li>
 402
 403
 404
 405             <li><a href="http://defaria.com/blogs/MAPS/archives/2004/02/delete-bug.html">Delete bug</a></li>
 406
 407
 408
 409             <li><a href="http://defaria.com/blogs/MAPS/archives/2003/11/rounded-tables.html">Rounded Tables</a></li>
 410
 411
 412
 413             <li><a href="http://defaria.com/blogs/MAPS/archives/2003/11/css-site.html">CSS Site</a></li>
 414
 415         </ul>
 416     </div>
 417 </div>
 418
 419
 420
 421
 422
 423
 424
 425
 426     </div>
 427 </div>
 428
 429
 430
 431
 432                 </div>
 433             </div>
 434
 435
 436             <div id="footer">
 437     <div id="footer-inner">
 438         <div id="footer-content">
 439             <div class="widget-powered widget">
 440                 <div class="widget-content">
 441                     Powered by <a href="http://www.movabletype.org/">Movable Type</a>
 442                 </div>
 443             </div>
 444
 445         </div>
 446     </div>
 447 </div>
 448
 449
 450
 451         </div>
 452     </div>
 453 </body>
 454 </html>