cross-approval

a scheme for arranging for netnews articles,
cross-posted to more than one moderated newsgroup,
to be approved by all relevant moderators independently

The Usenet/netnews moderation system was originally designed for each moderated newsgroup to be an isolated entity and it doesn't work too well for cross-posting between moderated newsgroups. But just as there's nothing to stop people cross-posting to any newsgroups they like (apart from the Breidbart Index-driven spam cancellers) there's nothing to stop two or more of those newsgroups being moderated. As usage of the newsgroup system matures there is an increasing need for cross-posting between moderated newsgroups as if they were the same as any others. This is particularly true for the more "social" newsgroups (I used to moderate uk.religion.christian) where long-term regulars stop seeing moderation as the exception & so are less likely to make allowances for limitations.

Posting to a moderated newsgroup triggers a mechanism whereby the post is intercepted before it gets into the newspool of the local newserver, and is instead sent as a mail to the designated moderator of the newsgroup. That person (or team of people, or moderbot) turns the mail back into a post & does something ("giving approval") to it to make sure it actually goes out as a post this time. The problem is that that something is not newsgroup-specific and once done it works for all moderated newsgroups in the Newsgroups header. The moderators for those other newsgroups never get to apply their particular moderation policies, the post simply appears in their newsgroups when posted by the first moderator.

What's supposed to happen is that the moderator the post first goes to notices the other moderated newsgroup(s) in the Newsgroups header, and instead of the normal action sends the post, as a mail, on to the moderator of the next moderated newsgroup. The next moderator has to work out that they weren't the first moderator, and also whether they're the last, and either do what they normally do or send the post on again. There are three problems: moderators fail to do this at all, or they get it wrong because it's fiddly and an exception to the usual routine, and it slows everything down because it's not automated.

Many moderated newsgroups use a "moderbot" - an automatically run program to do the bulk of the moderation process by it automatically giving approval to trusted regulars. This not only saves the human moderators a lot of work and removes much scope for error but considerably speeds things up. This page is a proposal for a scheme for how to respond to the situation described above in such a way that it too can be automated.

The problem has already been partially solved by the author of PGPMoose, a system for PGP-signing approvals, to create the possibility that forged approvals can be detected & those articles cancelled. He included a way of being compatible with PGPMoose without actually having PGP signatures, which this proposal uses.

Assume a moderated newsgroup called news.stuff moderated by John Doe with an address of JohnDoe@some-place.com, and an article has just arrived at the moderator's mail-spool:

A way of minimizing the work to do this is to maintain manually a list of newsgroups previously seen in cross-posts. Then the automated search along the Newsgroups header for moderated newsgroups can give up if it finds a newsgroup not in the list, & forward the post to the human moderator for manual intervention. The moderator can find out its moderation status & the name/status added to the list. The volume of work in doing this is minimal wrt the usage of cross-posting in the newsgroup, and most cross-posting from moderated newsgroups is to well-known newsgroups not to endless lists of rarely encountered ones. If a newsgroup becomes moderated when already listed as unmoderated then you can be sure of getting complaints quickly, which will means that problem won't stay around for long.

An alternative would be to use Dave Lawrence's chkgroups posts in news.announce.newgroups as up-to-date sources of this information. That group also has changes to moderated statuses in the Big-8, uk.net.news.announce doing the same for uk.*.

If an X-Auth header has been generated by PGPMoose it will have the particular newsgroup name as the last blank-delimited word on the first newline-delimited line. comp.os.linux.announce is an example of a newsgroup that uses PGPMoose. Sometimes a newsreader will show the X-Auth contents wrapped without regard to the newlines put in by PGPMoose, but what matters is what arrives by mail, which should have the original formatting [3].

If an intermediate server mucks up the formatting so you can't work out automatically which newsgroup an X-Auth header is for, one solution is to trigger a manual intervention by the human moderator.

Here is an example of a PGPMoose signature for a particular newsgroup:

X-Auth: PGPMoose V1.1 PGP comp.os.linux.announce
        iQCVAgUBOYx/51rUI/eHXJZ5AQH91QP/R1pN71k9zAMwaGZ6173up1x4oNJ7xwOD
        K9V6pdesr162LwqL8o9sDt2oQSm0ggFfOY44JHPB5pc5ixlYbe5DbCYVwgzjv1Rk
        RNPG0WcNStjhrQBeM+GL0TvpHynqAuDi+zUllEOB+Fn+FhJHMkifBstXnOgvFMdh
        AhY53SPX6eE=
        =R0+1

Here's one that is compatible with PGPMoose without using PGP signatures:

X-Auth: None JohnDoe@some-place.com news.stuff

Note the number of blank-delimited words on the first line in each case, which shows the necessity of using the last word on the first line to find out which newsgroup an X-Auth header is for.

Logically this ought to be the one that was found in the first place that triggered going down this side of the if...then, but it doesn't have to be.

either by knowing where to look e.g. in a list of moderator addresses, or by knowing how to construct a valid forwarding address for the newsgroup name e.g. news-stuff@moderators.isc.org. The latter should work for all moderated newsgroups though isn't guaranteed by anything formal.

This is an example implementation in the REXX language from a currently running moderbot. It uses the "non-PGP" option & guesses the addresses of moderators.

The story so far: checks for cross-posting volume and integrity have already been done which means all the groups posted to will be known to the moderbot (else they would have triggered a forwarding to the human moderator who would have updated the config file), so the array NewsgroupsStatus.* will contain an entry for all the newsgroups posted to. Any X-Auth headers found in the incoming mail are in the array Mail.X_AUTH.*. Mail.HEADER.NEWSGROUPS, ThisNewsgroup, ModeratorAddress, ModeratorProperName, ThisNewsgroupCharterURL, PostOutFile, NewLine already have the appropriate values; other variables and code should be obvious I hope.

Included at the start is boilerplate code to prevent the system looping between two or more moderbots.

  /* abend if we've already approved this post */

  do i = 1 to Mail.X_AUTH.0
    parse var Mail.X_AUTH.j X_AUTH_FirstLine ( NewLine ) .
    if   lower( word( X_AUTH_FirstLine, words( X_AUTH_FirstLine ) ) ),
       = lower( ThisNewsgroup ) then do
      call abend 'we''ve already approved this one - it needs a manual fix'
      end
  end

  /* find out if any other group is moderated and not yet approved for */

  NewsgroupsPostedTo = translate( Mail.HEADER.NEWSGROUPS, ' ', ',' )

  NeedMoreApprovals = 0
  do i = 1 to words( NewsgroupsPostedTo )
    NewsgroupPostedTo = word( NewsgroupsPostedTo, i )
    if   NewsgroupPostedTo \= ThisNewsgroup,
       & NewsgroupStatus.NewsgroupPostedTo = 'moderated' then do
      NeedMoreApprovals = 1
      /* find out if it's been approved for */
      do j = 1 to Mail.X_AUTH.0
        parse var Mail.X_AUTH.j X_AUTH_FirstLine ( NewLine ) .
        if   lower( word( X_AUTH_FirstLine, words( X_AUTH_FirstLine ) ) ),
           = lower( NewsgroupPostedTo ) then do
          say NewsgroupPostedTo 'is approved for'
          NeedMoreApprovals = 0
          leave j
          end
      end
      if NeedMoreApprovals then do
        NewsgroupNeedingApproval = NewsgroupPostedTo
        say NewsgroupNeedingApproval 'isn''t approved for'
        leave i
        end
      end
  end

  /* do X-Auth header or our own approval header */

  if NeedMoreApprovals then do
    call lineout PostOutFile, 'X-Auth: None' ModeratorAddress ThisNewsgroup
    end
  else do
    call lineout PostOutFile, 'Approved:' ModeratorAddress '('ModeratorProperName
    call lineout PostOutFile, '          by charter at' ThisNewsgroupCharterURL')'
    end

  /* preserve existing X-Auth headers */

  do i = 1 to Mail.X_AUTH.0
    call charout PostOutFile, 'X-Auth:' Mail.X_AUTH.i || NewLine
  end

  /* then follows the writing out of the rest of the headers & body from
     the original post */

  ...

  /* post or send on to next moderator */

  if NeedMoreApprovals then do
    /* construct next moderator's address, changing '.' to '-' */
    NextModeratorViaIsc =    translate( NewsgroupNeedingApproval, '-', '.' ),
                          || '@moderators.isc.org'
    call mailArticle NextModeratorViaIsc, PostOutFile
    end
  else do
    call postArticle PostOutFile
    end

  

If you want to contribute or have something clarified you can mail me, or post to any or all of news.{ groups, admin.censorship, admin.misc, software.misc }, uk.net.news.{ moderation, management }. I'm not likely to see your post without you sending me a CC. Only one of those newsgroups is actually spot-on for this topic (uk.net.news.moderation), so don't be surprised if people moan.

The moderators of the following newsgroups have said they want to use this scheme, either by hand or automatically, in the future ( >add to the list):

The moderators of the following newsgroups are using this scheme, but only by hand, not automatically ( >add to the list):

The moderators of the following newsgroups are using this scheme, by automatically run program with manual intervention if necessary ( add to the list):

• uk.religion.christian

this page.

to be mailed when changes occur.

Patrick Herring, except where otherwise acknowledged eg. in "alt" clauses.