Mailer Contrib

<--
   Contributions to this TWiki extension are appreciated. Please update the page at
   http://twiki.org/cgi-bin/view/Plugins/MailerContrib or provide feedback at
   http://twiki.org/cgi-bin/view/Plugins/MailerContribDev.
   If you are a TWiki contributor please update the extension in the SVN repository.
-->

Introduction

The Mailer Contrib allows users to "subscribe" to regularly scheduled e-mails containing either:

• A report on changes to all topics that have changed within a particular TWiki web.
• A report on changes to a specific topic or set of topics the user can define flexibly.
• The entire content of a specific topic or set of topics. This is referred to as "news mode."
• The companion plugin (TWiki:Plugins.SubscribePlugin) lets you trivially add a "Subscribe to changes" button to topics

tools/mailnotify

The central component of MailerContrib is a script, tools/mailnotify, that generates and sends out the emails based on analysis of

1. users' subcriptions listed in the WebNotify topic in each web, and
2. changes within the respective webs.

The script collates the changes emails so that each subscriber only receives one changes notification for all changes in all webs in the TWiki. Furthermore, users can elect to receive just summaries of changes, or the entire content of topics that have changed.

Each web can optionally contain a topic called WebNotify.

<-- Included by WebChangesAlert -->

Users subscribe to email notifications using their WikiName or an alternative email address, and can specify the webs/topics they wish to track, Whole groups of users can also be subscribed for notification.

The general format of a subscription is:

three spaces * subscriber [ : topics ]

Where subscriber can be a WikiName, an E-mail address, or a group name. If subscriber contains any characters that are not legal in

• ... without a Web. prefix
• ...that exist in this web.

• Using wild-card character in topic names - You can use * in a topic name, where it is treated as a wildcard character. A * will match zero or more other characters - so, for example, Fred* will match all topic names starting with Fred, *Fred will match all topic names ending with Fred, and * will match all topic names.
• Unsubscribing to specific topics - Each topic may optionally be preceded by a '+' or '-' sign. The '+' sign means "subscribe to this topic". The '-' sign means "unsubscribe" or "don't send notifications regarding this particular topic". This allows users to elect to filter out certain topics. Topic filters ('-') take precedence over topic includes ('+') i.e. if you unsubscribe from a topic it will cancel out any subscriptions to that topic.
• Including child-topics in subscription - Each topic may optionally be followed by an integer in parentheses, indicating the depth of the tree of children below that topic. Changes in all these children will be detected and reported along with changes to the topic itself. Note This uses the TWiki "Topic parent" feature.
• Subscribing to entire topic ("news mode") - Each topic may optionally be immediately followed by an exclamation mark ! and/or a question mark ? with no intervening spaces, indicating that the topic (and children if there is a tree depth specifier as well) should be mailed out as complete topics instead of change summaries. ! causes the full topic to be mailed every time even if there have been no changes, and ? will mail the full topic only if there have been changes. One can limit the content of the subscribed topic to send out by inserting %STARTPUBLISH% and %STOPPUBLISH% markers within the topic. Note that "news mode" subscriptions require a corresponding cron job that includes the "-news" option (see details).

Examples: Subscribe Daisy to all changes to topics in this web.

   * daisy.cutter@flowers.com

   * daisy.cutter@flowers.com : Web*

   * DaisyCutter: Petal* (1) WeedKillers (3) Pretty*Flowers

   * StarTrekFan: Star* - *Wars - *sInTheirEyes - *shipTroopers

   * daisy@flowers.com: NewsLetter?

   * buttercup@flowers.com: NewsLetter! (1)

   * GardenGroup: AllNewsLetters? (3)
   * petunia@flowers.com: - ManureNewsLetter

   * 'IT:admins' : Web*

A user may be listed many times in the WebNotify topic. Where a user has several lines in WebNotify that all match the same topic, they will only be notified about changes that topic once (though they will still receive individual mails for news topics).

If a group is listed for notification, the group will be recursively expanded to the e-mail addresses of all members.

Tip:

In the future it is intended that individual users will be able to control the frequency with which they are notified of topic changes, by changing a schedule specification in their home topic. However at present, the notification schedule is controlled by the frequency of activation of the cron job that runs the mailnotify script.

Note that when using the "news mode" ! or ? specifiers the entire topic text is mailed out as HTML. The newsletter template is used to generate the content in this mail, using whatever skin is selected in the topic being mailed.

In addition, the %STARTPUBLISH% and %STOPPUBLISH% markers used by TWiki:Plugins.PublishContrib to delimit the text to be published are respected in news mode.

TWiki/Contrib/MailerContrib code library

The second part of the module is a code library that provides the services for other applications to modify the subscription topics through a clean, well documented API. This allows (for example) plugin developers to add (for example) a "Register me for this newsletter" button to their pages. Developers should refer to the POD documentation for the WebNotify class as their starting point.

Installation Instructions

You do not need to install anything in the browser to use this extension. The following instructions are for the administrator who installs the extension on the server where TWiki is running.

Like many other TWiki extensions, this module is shipped with a fully automatic installer script written using the BuildContrib.

• If you have TWiki 4.2 or later, you can install from the configure interface (Go to Plugins->Find More Extensions)
  • See the installation supplement on TWiki.org for more information.
• If you have any problems, then you can still install manually from the command-line:
  1. Download one of the .zip or .tgz archives
  2. Unpack the archive in the root directory of your TWiki installation.
  3. Run the installer script ( perl <module>_installer )
  4. Run configure and enable the module, if it is a plugin.
  5. Repeat for any missing dependencies.
• If you are still having problems, then instead of running the installer script:
  1. Make sure that the file permissions allow the webserver user to access all files.
  2. Check in any installed files that have existing ,v files in your existing install (take care not to lock the files when you check in)
  3. Manually edit LocalSite.cfg to set any configuration variables.

• To make sure the installation was successful, run the mailnotify script from the command line, with no parameters. In this case it will print out what it would have done to STDOUT.

• You can change the regular expression that matches email addresses in WebNotify using the {MailerContrib}{EmailFilterIn} setting in =configure. This allows you to limit the domains to which emails can be sent, or even block email addresses altogether.

Setting up your cron job(s)

You need to set up a cron (or equivalent) job to run the tools/mailnotify perl script.

The script is used as follows: perl -I bin mailnotify [-q] [-news] [ web1 web2 ... webN ]

0 0 * * * cd /usr/local/twiki && perl -I bin tools/mailnotify -q Public Private

0 0 * * * cd /usr/local/twiki && perl -I bin tools/mailnotify -q -Sandbox

0 0 * * 0 cd /usr/local/twiki && perl -I bin tools/mailnotify -news

Note: Multiple instances of mailnotify script are not allowed to be executed simultaneously. If you need to run the script multiple times with different options, make sure the cron jobs are scheduled so a previous run has finished before the next starts. You can also write a small script that runs mailnotify in sequence as described in TWiki:Support.DuplicateNotificationsFromMailerCon.

<-- 

 Set STUB = TWiki::Contrib::Mailer
 
 Set SHORTDESCRIPTION = Send e-mail notification of changes to topics in TWiki webs
 
-->

Developer Notes

The changes mails sent to subscribers are based on a TWiki template called mailnotify. This template must contain the following definitions.

The default template sends multipart mails containing both HTML and plaintext versions. You can easily provide a custom version of this template using a local skin.

Newsletters are sent after formatting using the standard view template, using whatever skin is selected in the topic being mailed.

Contrib Info

Many thanks to the following sponsors for supporting this work:

• Wind River
• WikiGardens

15 Oct 2008	TWikibug:Item: generalised code to enable TWiki:Codev.SubscribePlugin to delegate parsing to MailerContrib - TWiki:Main.SvenDowideit
14 Oct 2008	TWikibug:Item6066: fixed issue for user mappings where cuid = login - TWiki:Main.SvenDowideit
27 Aug 2008	TWikibug:Item5949: fixed problem with unsubscribe that should result in an empty subscription
03 Aug 2008	TWiki 4.2.1 release version
27 Jul 2008	TWikibug:Item5776: TWiki:Main.KennethLavrsen added note to warn against running multiple instances of mailnotify at the same time.
15 May 2008	TWikibug:Item5232: TWiki:Main.CrawfordCurrie added support for non-ascii alphanumerics in topic names to WebNotify. TWikibug:Item5630: TWiki:Main.SvenDowideit fixed some spelling errors
25 Jan 2008	TWikibug:Item4812: added TWiki:Main.BobGoldstein's noexpand patch for groups
2 Nov 2007	TWikibug:Item4818: added quotes to support non-alphabetic and other wierd group names TWikibug:Item4887: corrected minor rendering error TWikibug:Item4917: removed dependence on symbolic web names
9 Sep 2007	TWikibug:Item4326 workaround for possible error in WebNotify API in old releases, Should not affect most users.
6 Sep 2007	TWikibug:Item4488 doc tweaks
14550	TWikibug:Item4461 - 'Changed' link now points to most recent changes, not the entire history
22 Jun 2007	TWikibug:Item4284 - added access control checks and email filter
21 May 2007	TWikibug:Item3969 - 8bit email fix (TWiki:Main.WillNorris)
13623	TWikibug:Item4014 no changes was resetting the notify time to 0. Thanks to TWiki:Main.JeffCrawford for nailing this down.
12496	TWikibug:Item3415 mailnotify did not send notifications to intranet users because of wrong call to findUser.
11672	Added newsletter support, after much harassment from TWiki:Main.LynnwoodBrown
11534	TWikibug:Item2153 Clarified docs. TWikibug:Item2698 Improved error reporting.
8808	TWikibug:Item1654 mailnotify must enter the command_line context
8625	TWikibug:Item1508 Making the dashes in the separatator clearer
8606	TWikibug:Item1508 MailerContrib: Brushing up HTML mailnotify template
8602	TWikibug:Item1508 MailerContrib: Cleaning up plaintext e-mail template, removing TEXTAREA
8522	TWikibug:Item1511 arguments to getScriptUrl in wrong order
8434	TWikibug:Item1465 Fix 'TWiki.' to '%TWIKIEB%.'
8398	TWikibug:Item1460 polished up the comment a bit
8308	TWikibug:Item1362 moving mailnotify cron script
7848	TWikibug:Item1167 forced all mail operations to generate absolute URLs
7568	TWikibug:tem910 use SCRIPTURL{view} instead of complex url expr
6864	TWikibug:tem624 mailer templates moved the the right places
6861	TWikibug:tem624 Added proper templates support for plain text mails
6809	TWikibug:tem623 don't print anything if verbosity is switched off.
6659	TWikibug:tem528 Updated MailerContrib. it's working and the sendmail parameter is used.
6474	TWikibug:tem420 removed spurious remove_obsolete_locks from MailerContrib
5924	TWikibug:tem153 fix mail URL-fixing scheme
5269	Minor doc fixes
5266	Doc tidy-ups, added filtering of _ webs, added obsolete lock script
5264	Changed default to add web name to user name (I hope)
5263	Minor doc tidyups
5261	Documentation changes, and fixed to scan all webs.
5253	runnable as CGI script, minor bugfixes, removed dependency on DBCacheContrib
5234	Minor doc changes
5231	Made a change an object, added unit tests to CVS, lots of testing.
4 March 2005	1.010 Dakar release ready.
12 Oct 2004	1.004 Added support for anti-subscriptions. Doc fixes from TWiki:Main.PeterThoeny. Bug fixes to permissions code. Back-off and retry if the mailer can't be reached (should really be in Net::sendEmail)
6 Oct 2004	1.003 Excluded _ webs from processing, added bin/remove_obsolete_locks for full reverse-compatibility
1 Oct 2004	1.002 PeterThoeny provided additional documentation
27 Sep 2004	1.001 runnable as CGI script, minor bugfixes, removed dependency on DBCacheContrib
8 Sep 2004	1.000 Initial version
Home:	http://TWiki.org/cgi-bin/view/Plugins/MailerContrib
Feedback:	http://TWiki.org/cgi-bin/view/Plugins/MailerContribDev
Appraisal:	http://TWiki.org/cgi-bin/view/Plugins/MailerContribAppraisal

Related Topics: TWikiPreferences

Mailer Contrib

<--
   Contributions to this TWiki extension are appreciated. Please update the page at
   http://twiki.org/cgi-bin/view/Plugins/MailerContrib or provide feedback at
   http://twiki.org/cgi-bin/view/Plugins/MailerContribDev.
   If you are a TWiki contributor please update the extension in the SVN repository.
-->

Introduction

The Mailer Contrib allows users to "subscribe" to regularly scheduled e-mails containing either:

• A report on changes to all topics that have changed within a particular TWiki web.
• A report on changes to a specific topic or set of topics the user can define flexibly.
• The entire content of a specific topic or set of topics. This is referred to as "news mode."
• The companion plugin (TWiki:Plugins.SubscribePlugin) lets you trivially add a "Subscribe to changes" button to topics

tools/mailnotify

The central component of MailerContrib is a script, tools/mailnotify, that generates and sends out the emails based on analysis of

1. users' subcriptions listed in the WebNotify topic in each web, and
2. changes within the respective webs.

The script collates the changes emails so that each subscriber only receives one changes notification for all changes in all webs in the TWiki. Furthermore, users can elect to receive just summaries of changes, or the entire content of topics that have changed.

Each web can optionally contain a topic called WebNotify.

<-- Included by WebChangesAlert -->

Users subscribe to email notifications using their WikiName or an alternative email address, and can specify the webs/topics they wish to track, Whole groups of users can also be subscribed for notification.

The general format of a subscription is:

three spaces * subscriber [ : topics ]

Where subscriber can be a WikiName, an E-mail address, or a group name. If subscriber contains any characters that are not legal in an email address, then it must be enclosed in 'single' or "double" quotes.

topics is an optional space-separated list of topics:

• ... without a Web. prefix
• ...that exist in this web.

• Using wild-card character in topic names - You can use * in a topic name, where it is treated as a wildcard character. A * will match zero or more other characters - so, for example, Fred* will match all topic names starting with Fred, *Fred will match all topic names ending with Fred, and * will match all topic names.
• Unsubscribing to specific topics - Each topic may optionally be preceded by a '+' or '-' sign. The '+' sign means "subscribe to this topic". The '-' sign means "unsubscribe" or "don't send notifications regarding this particular topic". This allows users to elect to filter out certain topics. Topic filters ('-') take precedence over topic includes ('+') i.e. if you unsubscribe from a topic it will cancel out any subscriptions to that topic.
• Including child-topics in subscription - Each topic may optionally be followed by an integer in parentheses, indicating the depth of the tree of children below that topic. Changes in all these children will be detected and reported along with changes to the topic itself. Note This uses the TWiki "Topic parent" feature.
• Subscribing to entire topic ("news mode") - Each topic may optionally be immediately followed by an exclamation mark ! and/or a question mark ? with no intervening spaces, indicating that the topic (and children if there is a tree depth specifier as well) should be mailed out as complete topics instead of change summaries. ! causes the full topic to be mailed every time even if there have been no changes, and ? will mail the full topic only if there have been changes. One can limit the content of the subscribed topic to send out by inserting %STARTPUBLISH% and %STOPPUBLISH% markers within the topic. Note that "news mode" subscriptions require a corresponding cron job that includes the "-news" option (see details).

Examples: Subscribe Daisy to all changes to topics in this web.

   * daisy.cutter@flowers.com

   * daisy.cutter@flowers.com : Web*

   * DaisyCutter: Petal* (1) WeedKillers (3) Pretty*Flowers

   * StarTrekFan: Star* - *Wars - *sInTheirEyes - *shipTroopers

   * daisy@flowers.com: NewsLetter?

   * buttercup@flowers.com: NewsLetter! (1)

   * GardenGroup: AllNewsLetters? (3)
   * petunia@flowers.com: - ManureNewsLetter

   * 'IT:admins' : Web*

A user may be listed many times in the WebNotify topic. Where a user has several lines in WebNotify that all match the same topic, they will only be notified about changes that topic once (though they will still receive individual mails for news topics).

If a group is listed for notification, the group will be recursively expanded to the e-mail addresses of all members.

Warning: Because an email address is not linked to a user name, there is no way for TWiki to check access controls for subscribers identified by email addresses. A subscriber identified by an email address alone will only be sent change notifications if the topic they are subscribed to is readable by guest users. You can limit what email addresses can be used in %NOTIFYTOPIC%, or even block use of emails altogther, using the {MailerContrib}{EmailFilterIn} setting in =configure.

Tip: List names in alphabetical order to make it easier to find the names.

In the future it is intended that individual users will be able to control the frequency with which they are notified of topic changes, by changing a schedule specification in their home topic. However at present, the notification schedule is controlled by the frequency of activation of the cron job that runs the mailnotify script.

Note that when using the "news mode" ! or ? specifiers the entire topic text is mailed out as HTML. The newsletter template is used to generate the content in this mail, using whatever skin is selected in the topic being mailed.

In addition, the %STARTPUBLISH% and %STOPPUBLISH% markers used by TWiki:Plugins.PublishContrib to delimit the text to be published are respected in news mode.

TWiki/Contrib/MailerContrib code library

The second part of the module is a code library that provides the services for other applications to modify the subscription topics through a clean, well documented API. This allows (for example) plugin developers to add (for example) a "Register me for this newsletter" button to their pages. Developers should refer to the POD documentation for the WebNotify class as their starting point.

Installation Instructions

You do not need to install anything in the browser to use this extension. The following instructions are for the administrator who installs the extension on the server where TWiki is running.

Like many other TWiki extensions, this module is shipped with a fully automatic installer script written using the BuildContrib.

• If you have TWiki 4.2 or later, you can install from the configure interface (Go to Plugins->Find More Extensions)
  • See the installation supplement on TWiki.org for more information.
• If you have any problems, then you can still install manually from the command-line:
  1. Download one of the .zip or .tgz archives
  2. Unpack the archive in the root directory of your TWiki installation.
  3. Run the installer script ( perl <module>_installer )
  4. Run configure and enable the module, if it is a plugin.
  5. Repeat for any missing dependencies.
• If you are still having problems, then instead of running the installer script:
  1. Make sure that the file permissions allow the webserver user to access all files.
  2. Check in any installed files that have existing ,v files in your existing install (take care not to lock the files when you check in)
  3. Manually edit LocalSite.cfg to set any configuration variables.

• To make sure the installation was successful, run the mailnotify script from the command line, with no parameters. In this case it will print out what it would have done to STDOUT.

• You can change the regular expression that matches email addresses in WebNotify using the {MailerContrib}{EmailFilterIn} setting in =configure. This allows you to limit the domains to which emails can be sent, or even block email addresses altogether.

Setting up your cron job(s)

You need to set up a cron (or equivalent) job to run the tools/mailnotify perl script.

The script is used as follows: perl -I bin mailnotify [-q] [-news] [ web1 web2 ... webN ]

0 0 * * * cd /usr/local/twiki && perl -I bin tools/mailnotify -q Public Private

0 0 * * * cd /usr/local/twiki && perl -I bin tools/mailnotify -q -Sandbox

0 0 * * 0 cd /usr/local/twiki && perl -I bin tools/mailnotify -news

Note: Multiple instances of mailnotify script are not allowed to be executed simultaneously. If you need to run the script multiple times with different options, make sure the cron jobs are scheduled so a previous run has finished before the next starts. You can also write a small script that runs mailnotify in sequence as described in TWiki:Support.DuplicateNotificationsFromMailerCon.

<-- 

 Set STUB = TWiki::Contrib::Mailer
 
 Set SHORTDESCRIPTION = Send e-mail notification of changes to topics in TWiki webs
 
-->

Developer Notes

The changes mails sent to subscribers are based on a TWiki template called mailnotify. This template must contain the following definitions.

The default template sends multipart mails containing both HTML and plaintext versions. You can easily provide a custom version of this template using a local skin.

Newsletters are sent after formatting using the standard view template, using whatever skin is selected in the topic being mailed.

Contrib Info

Many thanks to the following sponsors for supporting this work:

• Wind River
• WikiGardens

2010-07-09:	TWikibug:Item6518 - Fix for notification e-mail showing %MAKETEXT{"Web"} in subject
2010-06-12:	TWikibug:Item6489 - Separate WIKITOOLNAME from WEB name; escape WEB name to avoid auto-linking in case web is WikiWord
2010-05-26:	TWikibug:Item6474 - Fix for mailnotify failing due to non-existing _alert method
2010-05-21:	TWikibug:Item6433 - more doc fixes
2010-04-25:	TWikibug:Item6433 - doc fixes, no code changes
15 Oct 2008	TWikibug:Item: generalised code to enable TWiki:Codev.SubscribePlugin to delegate parsing to MailerContrib - TWiki:Main.SvenDowideit
14 Oct 2008	TWikibug:Item6066: fixed issue for user mappings where cuid = login - TWiki:Main.SvenDowideit
27 Aug 2008	TWikibug:Item5949: fixed problem with unsubscribe that should result in an empty subscription
03 Aug 2008	TWiki 4.2.1 release version
27 Jul 2008	TWikibug:Item5776: TWiki:Main.KennethLavrsen added note to warn against running multiple instances of mailnotify at the same time.
15 May 2008	TWikibug:Item5232: TWiki:Main.CrawfordCurrie added support for non-ascii alphanumerics in topic names to WebNotify. TWikibug:Item5630: TWiki:Main.SvenDowideit fixed some spelling errors
25 Jan 2008	TWikibug:Item4812: added TWiki:Main.BobGoldstein's noexpand patch for groups
2 Nov 2007	TWikibug:Item4818: added quotes to support non-alphabetic and other wierd group names TWikibug:Item4887: corrected minor rendering error TWikibug:Item4917: removed dependence on symbolic web names
9 Sep 2007	TWikibug:Item4326 workaround for possible error in WebNotify API in old releases, Should not affect most users.
6 Sep 2007	TWikibug:Item4488 doc tweaks
14550	TWikibug:Item4461 - 'Changed' link now points to most recent changes, not the entire history
22 Jun 2007	TWikibug:Item4284 - added access control checks and email filter
21 May 2007	TWikibug:Item3969 - 8bit email fix (TWiki:Main.WillNorris)
13623	TWikibug:Item4014 no changes was resetting the notify time to 0. Thanks to TWiki:Main.JeffCrawford for nailing this down.
12496	TWikibug:Item3415 mailnotify did not send notifications to intranet users because of wrong call to findUser.
11672	Added newsletter support, after much harassment from TWiki:Main.LynnwoodBrown
11534	TWikibug:Item2153 Clarified docs. TWikibug:Item2698 Improved error reporting.
8808	TWikibug:Item1654 mailnotify must enter the command_line context
8625	TWikibug:Item1508 Making the dashes in the separatator clearer
8606	TWikibug:Item1508 MailerContrib: Brushing up HTML mailnotify template
8602	TWikibug:Item1508 MailerContrib: Cleaning up plaintext e-mail template, removing TEXTAREA
8522	TWikibug:Item1511 arguments to getScriptUrl in wrong order
8434	TWikibug:Item1465 Fix 'TWiki.' to '%TWIKIEB%.'
8398	TWikibug:Item1460 polished up the comment a bit
8308	TWikibug:Item1362 moving mailnotify cron script
7848	TWikibug:Item1167 forced all mail operations to generate absolute URLs
7568	TWikibug:tem910 use SCRIPTURL{view} instead of complex url expr
6864	TWikibug:tem624 mailer templates moved the the right places
6861	TWikibug:tem624 Added proper templates support for plain text mails
6809	TWikibug:tem623 don't print anything if verbosity is switched off.
6659	TWikibug:tem528 Updated MailerContrib. it's working and the sendmail parameter is used.
6474	TWikibug:tem420 removed spurious remove_obsolete_locks from MailerContrib
5924	TWikibug:tem153 fix mail URL-fixing scheme
5269	Minor doc fixes
5266	Doc tidy-ups, added filtering of _ webs, added obsolete lock script
5264	Changed default to add web name to user name (I hope)
5263	Minor doc tidyups
5261	Documentation changes, and fixed to scan all webs.
5253	runnable as CGI script, minor bugfixes, removed dependency on DBCacheContrib
5234	Minor doc changes
5231	Made a change an object, added unit tests to CVS, lots of testing.
4 March 2005	1.010 Dakar release ready.
12 Oct 2004	1.004 Added support for anti-subscriptions. Doc fixes from TWiki:Main.PeterThoeny. Bug fixes to permissions code. Back-off and retry if the mailer can't be reached (should really be in Net::sendEmail)
6 Oct 2004	1.003 Excluded _ webs from processing, added bin/remove_obsolete_locks for full reverse-compatibility
1 Oct 2004	1.002 PeterThoeny provided additional documentation
27 Sep 2004	1.001 runnable as CGI script, minor bugfixes, removed dependency on DBCacheContrib
8 Sep 2004	1.000 Initial version
Home:	http://TWiki.org/cgi-bin/view/Plugins/MailerContrib
Feedback:	http://TWiki.org/cgi-bin/view/Plugins/MailerContribDev
Appraisal:	http://TWiki.org/cgi-bin/view/Plugins/MailerContribAppraisal

Related Topics: TWikiPreferences

Mailer Contrib

<--
   Contributions to this TWiki extension are appreciated. Please update the page at
   http://twiki.org/cgi-bin/view/Plugins/MailerContrib or provide feedback at
   http://twiki.org/cgi-bin/view/Plugins/MailerContribDev.
   If you are a TWiki contributor please update the extension in the SVN repository.
-->

Introduction

The Mailer Contrib allows users to "subscribe" to regularly scheduled e-mails containing either:

• A report on changes to all topics that have changed within a particular TWiki web.
• A report on changes to a specific topic or set of topics the user can define flexibly.
• The entire content of a specific topic or set of topics. This is referred to as "news mode."
• The companion plugin (TWiki:Plugins.SubscribePlugin) lets you trivially add a "Subscribe to changes" button to topics

tools/mailnotify

The central component of MailerContrib is a script, tools/mailnotify, that generates and sends out the emails based on analysis of

1. users' subcriptions listed in the WebNotify topic in each web, and
2. changes within the respective webs.

The script collates the changes emails so that each subscriber only receives one changes notification for all changes in all webs in the TWiki. Furthermore, users can elect to receive just summaries of changes, or the entire content of topics that have changed.

Each web can optionally contain a topic called WebNotify.

<-- Included by WebChangesAlert -->

Users subscribe to email notifications using their WikiName or an alternative email address, and can specify the webs/topics they wish to track, Whole groups of users can also be subscribed for notification.

The general format of a subscription is:

three spaces * subscriber [ : topics ]

Where subscriber can be a WikiName, an E-mail address, or a group name. If subscriber contains any characters that are not legal in an email address, then it must be enclosed in 'single' or "double" quotes.

topics is an optional space-separated list of topics:

• ... without a Web. prefix
• ...that exist in this web.

• Using wild-card character in topic names - You can use * in a topic name, where it is treated as a wildcard character. A * will match zero or more other characters - so, for example, Fred* will match all topic names starting with Fred, *Fred will match all topic names ending with Fred, and * will match all topic names.
• Unsubscribing to specific topics - Each topic may optionally be preceded by a '+' or '-' sign. The '+' sign means "subscribe to this topic". The '-' sign means "unsubscribe" or "don't send notifications regarding this particular topic". This allows users to elect to filter out certain topics. Topic filters ('-') take precedence over topic includes ('+') i.e. if you unsubscribe from a topic it will cancel out any subscriptions to that topic.
• Including child-topics in subscription - Each topic may optionally be followed by an integer in parentheses, indicating the depth of the tree of children below that topic. Changes in all these children will be detected and reported along with changes to the topic itself. Note This uses the TWiki "Topic parent" feature.
• Subscribing to entire topic ("news mode") - Each topic may optionally be immediately followed by an exclamation mark ! and/or a question mark ? with no intervening spaces, indicating that the topic (and children if there is a tree depth specifier as well) should be mailed out as complete topics instead of change summaries. ! causes the full topic to be mailed every time even if there have been no changes, and ? will mail the full topic only if there have been changes. One can limit the content of the subscribed topic to send out by inserting %STARTPUBLISH% and %STOPPUBLISH% markers within the topic. Note that "news mode" subscriptions require a corresponding cron job that includes the "-news" option (see details).

Examples: Subscribe Daisy to all changes to topics in this web.

   * daisy.cutter@flowers.com

   * daisy.cutter@flowers.com : Web*

   * DaisyCutter: Petal* (1) WeedKillers (3) Pretty*Flowers

   * StarTrekFan: Star* - *Wars - *sInTheirEyes - *shipTroopers

   * daisy@flowers.com: NewsLetter?

   * buttercup@flowers.com: NewsLetter! (1)

   * GardenGroup: AllNewsLetters? (3)
   * petunia@flowers.com: - ManureNewsLetter

   * 'IT:admins' : Web*

A user may be listed many times in the WebNotify topic. Where a user has several lines in WebNotify that all match the same topic, they will only be notified about changes that topic once (though they will still receive individual mails for news topics).

If a group is listed for notification, the group will be recursively expanded to the e-mail addresses of all members.

Warning: Because an email address is not linked to a user name, there is no way for TWiki to check access controls for subscribers identified by email addresses. A subscriber identified by an email address alone will only be sent change notifications if the topic they are subscribed to is readable by guest users. You can limit what email addresses can be used in %NOTIFYTOPIC%, or even block use of emails altogther, using the {MailerContrib}{EmailFilterIn} setting in =configure.

Tip: List names in alphabetical order to make it easier to find the names.

In the future it is intended that individual users will be able to control the frequency with which they are notified of topic changes, by changing a schedule specification in their home topic. However at present, the notification schedule is controlled by the frequency of activation of the cron job that runs the mailnotify script.

Note that when using the "news mode" ! or ? specifiers the entire topic text is mailed out as HTML. The newsletter template is used to generate the content in this mail, using whatever skin is selected in the topic being mailed.

In addition, the %STARTPUBLISH% and %STOPPUBLISH% markers used by TWiki:Plugins.PublishContrib to delimit the text to be published are respected in news mode.

TWiki/Contrib/MailerContrib code library

The second part of the module is a code library that provides the services for other applications to modify the subscription topics through a clean, well documented API. This allows (for example) plugin developers to add (for example) a "Register me for this newsletter" button to their pages. Developers should refer to the POD documentation for the WebNotify class as their starting point.

Installation Instructions

You do not need to install anything in the browser to use this extension. The following instructions are for the administrator who installs the extension on the server where TWiki is running.

Like many other TWiki extensions, this module is shipped with a fully automatic installer script written using the BuildContrib.

• If you have TWiki 4.2 or later, you can install from the configure interface (Go to Plugins->Find More Extensions)
  • See the installation supplement on TWiki.org for more information.
• If you have any problems, then you can still install manually from the command-line:
  1. Download one of the .zip or .tgz archives
  2. Unpack the archive in the root directory of your TWiki installation.
  3. Run the installer script ( perl <module>_installer )
  4. Run configure and enable the module, if it is a plugin.
  5. Repeat for any missing dependencies.
• If you are still having problems, then instead of running the installer script:
  1. Make sure that the file permissions allow the webserver user to access all files.
  2. Check in any installed files that have existing ,v files in your existing install (take care not to lock the files when you check in)
  3. Manually edit LocalSite.cfg to set any configuration variables.

• To make sure the installation was successful, run the mailnotify script from the command line, with no parameters. In this case it will print out what it would have done to STDOUT.

• You can change the regular expression that matches email addresses in WebNotify using the {MailerContrib}{EmailFilterIn} setting in =configure. This allows you to limit the domains to which emails can be sent, or even block email addresses altogether.

Setting up your cron job(s)

You need to set up a cron (or equivalent) job to run the tools/mailnotify perl script.

The script is used as follows: perl -I bin mailnotify [-q] [-news] [ web1 web2 ... webN ]

0 0 * * * cd /usr/local/twiki && perl -I bin tools/mailnotify -q Public Private

0 0 * * * cd /usr/local/twiki && perl -I bin tools/mailnotify -q -Sandbox

0 0 * * 0 cd /usr/local/twiki && perl -I bin tools/mailnotify -news

Note: Multiple instances of mailnotify script are not allowed to be executed simultaneously. If you need to run the script multiple times with different options, make sure the cron jobs are scheduled so a previous run has finished before the next starts. You can also write a small script that runs mailnotify in sequence as described in TWiki:Support.DuplicateNotificationsFromMailerCon.

<-- 

 Set STUB = TWiki::Contrib::Mailer
 
 Set SHORTDESCRIPTION = Send e-mail notification of changes to topics in TWiki webs
 
-->

Developer Notes

The changes mails sent to subscribers are based on a TWiki template called mailnotify. This template must contain the following definitions.

The default template sends multipart mails containing both HTML and plaintext versions. You can easily provide a custom version of this template using a local skin.

Newsletters are sent after formatting using the standard view template, using whatever skin is selected in the topic being mailed.

Contrib Info

Many thanks to the following sponsors for supporting this work:

• Wind River
• WikiGardens

Change History:	<-- specify latest version first -->
2010-07-09:	TWikibug:Item6518 - Fix for notification e-mail showing %MAKETEXT{"Web"} in subject
2010-06-12:	TWikibug:Item6489 - Separate WIKITOOLNAME from WEB name; escape WEB name to avoid auto-linking in case web is WikiWord
2010-05-26:	TWikibug:Item6474 - Fix for mailnotify failing due to non-existing _alert method
2010-05-21:	TWikibug:Item6433 - more doc fixes
2010-04-25:	TWikibug:Item6433 - doc fixes, no code changes
15 Oct 2008	TWikibug:Item: generalised code to enable TWiki:Codev.SubscribePlugin to delegate parsing to MailerContrib - TWiki:Main.SvenDowideit
14 Oct 2008	TWikibug:Item6066: fixed issue for user mappings where cuid = login - TWiki:Main.SvenDowideit
27 Aug 2008	TWikibug:Item5949: fixed problem with unsubscribe that should result in an empty subscription
03 Aug 2008	TWiki 4.2.1 release version
27 Jul 2008	TWikibug:Item5776: TWiki:Main.KennethLavrsen added note to warn against running multiple instances of mailnotify at the same time.
15 May 2008	TWikibug:Item5232: TWiki:Main.CrawfordCurrie added support for non-ascii alphanumerics in topic names to WebNotify. TWikibug:Item5630: TWiki:Main.SvenDowideit fixed some spelling errors
25 Jan 2008	TWikibug:Item4812: added TWiki:Main.BobGoldstein's noexpand patch for groups
2 Nov 2007	TWikibug:Item4818: added quotes to support non-alphabetic and other wierd group names TWikibug:Item4887: corrected minor rendering error TWikibug:Item4917: removed dependence on symbolic web names
9 Sep 2007	TWikibug:Item4326 workaround for possible error in WebNotify API in old releases, Should not affect most users.
6 Sep 2007	TWikibug:Item4488 doc tweaks
14550	TWikibug:Item4461 - 'Changed' link now points to most recent changes, not the entire history
22 Jun 2007	TWikibug:Item4284 - added access control checks and email filter
21 May 2007	TWikibug:Item3969 - 8bit email fix (TWiki:Main.WillNorris)
13623	TWikibug:Item4014 no changes was resetting the notify time to 0. Thanks to TWiki:Main.JeffCrawford for nailing this down.
12496	TWikibug:Item3415 mailnotify did not send notifications to intranet users because of wrong call to findUser.
11672	Added newsletter support, after much harassment from TWiki:Main.LynnwoodBrown
11534	TWikibug:Item2153 Clarified docs. TWikibug:Item2698 Improved error reporting.
8808	TWikibug:Item1654 mailnotify must enter the command_line context
8625	TWikibug:Item1508 Making the dashes in the separatator clearer
8606	TWikibug:Item1508 MailerContrib: Brushing up HTML mailnotify template
8602	TWikibug:Item1508 MailerContrib: Cleaning up plaintext e-mail template, removing TEXTAREA
8522	TWikibug:Item1511 arguments to getScriptUrl in wrong order
8434	TWikibug:Item1465 Fix 'TWiki.' to '%TWIKIEB%.'
8398	TWikibug:Item1460 polished up the comment a bit
8308	TWikibug:Item1362 moving mailnotify cron script
7848	TWikibug:Item1167 forced all mail operations to generate absolute URLs
7568	TWikibug:tem910 use SCRIPTURL{view} instead of complex url expr
6864	TWikibug:tem624 mailer templates moved the the right places
6861	TWikibug:tem624 Added proper templates support for plain text mails
6809	TWikibug:tem623 don't print anything if verbosity is switched off.
6659	TWikibug:tem528 Updated MailerContrib. it's working and the sendmail parameter is used.
6474	TWikibug:tem420 removed spurious remove_obsolete_locks from MailerContrib
5924	TWikibug:tem153 fix mail URL-fixing scheme
5269	Minor doc fixes
5266	Doc tidy-ups, added filtering of _ webs, added obsolete lock script
5264	Changed default to add web name to user name (I hope)
5263	Minor doc tidyups
5261	Documentation changes, and fixed to scan all webs.
5253	runnable as CGI script, minor bugfixes, removed dependency on DBCacheContrib
5234	Minor doc changes
5231	Made a change an object, added unit tests to CVS, lots of testing.
4 March 2005	1.010 Dakar release ready.
12 Oct 2004	1.004 Added support for anti-subscriptions. Doc fixes from TWiki:Main.PeterThoeny. Bug fixes to permissions code. Back-off and retry if the mailer can't be reached (should really be in Net::sendEmail)
6 Oct 2004	1.003 Excluded _ webs from processing, added bin/remove_obsolete_locks for full reverse-compatibility
1 Oct 2004	1.002 PeterThoeny provided additional documentation
27 Sep 2004	1.001 runnable as CGI script, minor bugfixes, removed dependency on DBCacheContrib
8 Sep 2004	1.000 Initial version
Home:	http://TWiki.org/cgi-bin/view/Plugins/MailerContrib
Feedback:	http://TWiki.org/cgi-bin/view/Plugins/MailerContribDev
Appraisal:	http://TWiki.org/cgi-bin/view/Plugins/MailerContribAppraisal

Related Topics: TWikiPreferences

Mailer Contrib

<--
   Contributions to this TWiki extension are appreciated. Please update the page at
   http://twiki.org/cgi-bin/view/Plugins/MailerContrib or provide feedback at
   http://twiki.org/cgi-bin/view/Plugins/MailerContribDev.
   If you are a TWiki contributor please update the extension in the SVN repository.
-->

Introduction

The Mailer Contrib allows users to "subscribe" to regularly scheduled e-mails containing either:

• A report on changes to all topics that have changed within a particular TWiki web.
• A report on changes to a specific topic or set of topics the user can define flexibly.
• The entire content of a specific topic or set of topics. This is referred to as "news mode."
• The companion plugin (TWiki:Plugins.SubscribePlugin) lets you trivially add a "Subscribe to changes" button to topics

tools/mailnotify

The central component of MailerContrib is a script, tools/mailnotify, that generates and sends out the emails based on analysis of

1. users' subcriptions listed in the WebNotify topic in each web, and
2. changes within the respective webs.

The script collates the changes emails so that each subscriber only receives one changes notification for all changes in all webs in the TWiki. Furthermore, users can elect to receive just summaries of changes, or the entire content of topics that have changed.

Each web can optionally contain a topic called WebNotify.

<-- Included by WebChangesAlert -->

Users subscribe to email notifications using their WikiName or an alternative email address, and can specify the webs/topics they wish to track, Whole groups of users can also be subscribed for notification.

The general format of a subscription is:

three spaces * subscriber [ : topics ]

Where subscriber can be a WikiName, an E-mail address, or a group name. If subscriber contains any characters that are not legal in an email address, then it must be enclosed in 'single' or "double" quotes.

topics is an optional space-separated list of topics:

• ... without a Web. prefix
• ...that exist in this web.

• Using wild-card character in topic names - You can use * in a topic name, where it is treated as a wildcard character. A * will match zero or more other characters - so, for example, Fred* will match all topic names starting with Fred, *Fred will match all topic names ending with Fred, and * will match all topic names.
• Unsubscribing to specific topics - Each topic may optionally be preceded by a '+' or '-' sign. The '+' sign means "subscribe to this topic". The '-' sign means "unsubscribe" or "don't send notifications regarding this particular topic". This allows users to elect to filter out certain topics. Topic filters ('-') take precedence over topic includes ('+') i.e. if you unsubscribe from a topic it will cancel out any subscriptions to that topic.
• Including child-topics in subscription - Each topic may optionally be followed by an integer in parentheses, indicating the depth of the tree of children below that topic. Changes in all these children will be detected and reported along with changes to the topic itself. Note This uses the TWiki "Topic parent" feature.
• Subscribing to entire topic ("news mode") - Each topic may optionally be immediately followed by an exclamation mark ! and/or a question mark ? with no intervening spaces, indicating that the topic (and children if there is a tree depth specifier as well) should be mailed out as complete topics instead of change summaries. ! causes the full topic to be mailed every time even if there have been no changes, and ? will mail the full topic only if there have been changes. One can limit the content of the subscribed topic to send out by inserting %STARTPUBLISH% and %STOPPUBLISH% markers within the topic. Note that "news mode" subscriptions require a corresponding cron job that includes the "-news" option (see details).

Examples: Subscribe Daisy to all changes to topics in this web.

   * daisy.cutter@flowers.com

   * daisy.cutter@flowers.com : Web*

   * DaisyCutter: Petal* (1) WeedKillers (3) Pretty*Flowers

   * StarTrekFan: Star* - *Wars - *sInTheirEyes - *shipTroopers

   * daisy@flowers.com: NewsLetter?

   * buttercup@flowers.com: NewsLetter! (1)

   * GardenGroup: AllNewsLetters? (3)
   * petunia@flowers.com: - ManureNewsLetter

   * 'IT:admins' : Web*

A user may be listed many times in the WebNotify topic. Where a user has several lines in WebNotify that all match the same topic, they will only be notified about changes that topic once (though they will still receive individual mails for news topics).

If a group is listed for notification, the group will be recursively expanded to the e-mail addresses of all members.

Warning: Because an email address is not linked to a user name, there is no way for TWiki to check access controls for subscribers identified by email addresses. A subscriber identified by an email address alone will only be sent change notifications if the topic they are subscribed to is readable by guest users. You can limit what email addresses can be used in %NOTIFYTOPIC%, or even block use of emails altogther, using the {MailerContrib}{EmailFilterIn} setting in =configure.

Tip: List names in alphabetical order to make it easier to find the names.

In the future it is intended that individual users will be able to control the frequency with which they are notified of topic changes, by changing a schedule specification in their home topic. However at present, the notification schedule is controlled by the frequency of activation of the cron job that runs the mailnotify script.

Note that when using the "news mode" ! or ? specifiers the entire topic text is mailed out as HTML. The newsletter template is used to generate the content in this mail, using whatever skin is selected in the topic being mailed.

In addition, the %STARTPUBLISH% and %STOPPUBLISH% markers used by TWiki:Plugins.PublishContrib to delimit the text to be published are respected in news mode.

TWiki/Contrib/MailerContrib code library

The second part of the module is a code library that provides the services for other applications to modify the subscription topics through a clean, well documented API. This allows (for example) plugin developers to add (for example) a "Register me for this newsletter" button to their pages. Developers should refer to the POD documentation for the WebNotify class as their starting point.

Installation Instructions

You do not need to install anything in the browser to use this extension. The following instructions are for the administrator who installs the extension on the server where TWiki is running.

Like many other TWiki extensions, this module is shipped with a fully automatic installer script written using the BuildContrib.

• If you have TWiki 4.2 or later, you can install from the configure interface (Go to Plugins->Find More Extensions)
  • See the installation supplement on TWiki.org for more information.
• If you have any problems, then you can still install manually from the command-line:
  1. Download one of the .zip or .tgz archives
  2. Unpack the archive in the root directory of your TWiki installation.
  3. Run the installer script ( perl <module>_installer )
  4. Run configure and enable the module, if it is a plugin.
  5. Repeat for any missing dependencies.
• If you are still having problems, then instead of running the installer script:
  1. Make sure that the file permissions allow the webserver user to access all files.
  2. Check in any installed files that have existing ,v files in your existing install (take care not to lock the files when you check in)
  3. Manually edit LocalSite.cfg to set any configuration variables.

• To make sure the installation was successful, run the mailnotify script from the command line, with no parameters. In this case it will print out what it would have done to STDOUT.

• You can change the regular expression that matches email addresses in WebNotify using the {MailerContrib}{EmailFilterIn} setting in =configure. This allows you to limit the domains to which emails can be sent, or even block email addresses altogether.

Setting up your cron job(s)

You need to set up a cron (or equivalent) job to run the tools/mailnotify perl script.

The script is used as follows: perl -I bin mailnotify [-q] [-news] [ web1 web2 ... webN ]

0 0 * * * cd /usr/local/twiki && perl -I bin tools/mailnotify -q Public Private

0 0 * * * cd /usr/local/twiki && perl -I bin tools/mailnotify -q -Sandbox

0 0 * * 0 cd /usr/local/twiki && perl -I bin tools/mailnotify -news

Note: Multiple instances of mailnotify script are not allowed to be executed simultaneously. If you need to run the script multiple times with different options, make sure the cron jobs are scheduled so a previous run has finished before the next starts. You can also write a small script that runs mailnotify in sequence as described in TWiki:Support.DuplicateNotificationsFromMailerCon.

<-- 

 Set STUB = TWiki::Contrib::Mailer
 
 Set SHORTDESCRIPTION = Send e-mail notification of changes to topics in TWiki webs
 
-->

Developer Notes

The changes mails sent to subscribers are based on a TWiki template called mailnotify. This template must contain the following definitions.

The default template sends multipart mails containing both HTML and plaintext versions. You can easily provide a custom version of this template using a local skin.

Newsletters are sent after formatting using the standard view template, using whatever skin is selected in the topic being mailed.

Contrib Info

Many thanks to the following sponsors for supporting this work:

• Wind River
• WikiGardens

2010-05-26:	TWikibug:Item6474 - Fix for mailnotify failing due to non-existing _alert method
2010-05-21:	TWikibug:Item6433 - more doc fixes
2010-04-25:	TWikibug:Item6433 - doc fixes, no code changes
15 Oct 2008	TWikibug:Item: generalised code to enable TWiki:Codev.SubscribePlugin to delegate parsing to MailerContrib - TWiki:Main.SvenDowideit
14 Oct 2008	TWikibug:Item6066: fixed issue for user mappings where cuid = login - TWiki:Main.SvenDowideit
27 Aug 2008	TWikibug:Item5949: fixed problem with unsubscribe that should result in an empty subscription
03 Aug 2008	TWiki 4.2.1 release version
27 Jul 2008	TWikibug:Item5776: TWiki:Main.KennethLavrsen added note to warn against running multiple instances of mailnotify at the same time.
15 May 2008	TWikibug:Item5232: TWiki:Main.CrawfordCurrie added support for non-ascii alphanumerics in topic names to WebNotify. TWikibug:Item5630: TWiki:Main.SvenDowideit fixed some spelling errors
25 Jan 2008	TWikibug:Item4812: added TWiki:Main.BobGoldstein's noexpand patch for groups
2 Nov 2007	TWikibug:Item4818: added quotes to support non-alphabetic and other wierd group names TWikibug:Item4887: corrected minor rendering error TWikibug:Item4917: removed dependence on symbolic web names
9 Sep 2007	TWikibug:Item4326 workaround for possible error in WebNotify API in old releases, Should not affect most users.
6 Sep 2007	TWikibug:Item4488 doc tweaks
14550	TWikibug:Item4461 - 'Changed' link now points to most recent changes, not the entire history
22 Jun 2007	TWikibug:Item4284 - added access control checks and email filter
21 May 2007	TWikibug:Item3969 - 8bit email fix (TWiki:Main.WillNorris)
13623	TWikibug:Item4014 no changes was resetting the notify time to 0. Thanks to TWiki:Main.JeffCrawford for nailing this down.
12496	TWikibug:Item3415 mailnotify did not send notifications to intranet users because of wrong call to findUser.
11672	Added newsletter support, after much harassment from TWiki:Main.LynnwoodBrown
11534	TWikibug:Item2153 Clarified docs. TWikibug:Item2698 Improved error reporting.
8808	TWikibug:Item1654 mailnotify must enter the command_line context
8625	TWikibug:Item1508 Making the dashes in the separatator clearer
8606	TWikibug:Item1508 MailerContrib: Brushing up HTML mailnotify template
8602	TWikibug:Item1508 MailerContrib: Cleaning up plaintext e-mail template, removing TEXTAREA
8522	TWikibug:Item1511 arguments to getScriptUrl in wrong order
8434	TWikibug:Item1465 Fix 'TWiki.' to '%TWIKIEB%.'
8398	TWikibug:Item1460 polished up the comment a bit
8308	TWikibug:Item1362 moving mailnotify cron script
7848	TWikibug:Item1167 forced all mail operations to generate absolute URLs
7568	TWikibug:tem910 use SCRIPTURL{view} instead of complex url expr
6864	TWikibug:tem624 mailer templates moved the the right places
6861	TWikibug:tem624 Added proper templates support for plain text mails
6809	TWikibug:tem623 don't print anything if verbosity is switched off.
6659	TWikibug:tem528 Updated MailerContrib. it's working and the sendmail parameter is used.
6474	TWikibug:tem420 removed spurious remove_obsolete_locks from MailerContrib
5924	TWikibug:tem153 fix mail URL-fixing scheme
5269	Minor doc fixes
5266	Doc tidy-ups, added filtering of _ webs, added obsolete lock script
5264	Changed default to add web name to user name (I hope)
5263	Minor doc tidyups
5261	Documentation changes, and fixed to scan all webs.
5253	runnable as CGI script, minor bugfixes, removed dependency on DBCacheContrib
5234	Minor doc changes
5231	Made a change an object, added unit tests to CVS, lots of testing.
4 March 2005	1.010 Dakar release ready.
12 Oct 2004	1.004 Added support for anti-subscriptions. Doc fixes from TWiki:Main.PeterThoeny. Bug fixes to permissions code. Back-off and retry if the mailer can't be reached (should really be in Net::sendEmail)
6 Oct 2004	1.003 Excluded _ webs from processing, added bin/remove_obsolete_locks for full reverse-compatibility
1 Oct 2004	1.002 PeterThoeny provided additional documentation
27 Sep 2004	1.001 runnable as CGI script, minor bugfixes, removed dependency on DBCacheContrib
8 Sep 2004	1.000 Initial version
Home:	http://TWiki.org/cgi-bin/view/Plugins/MailerContrib
Feedback:	http://TWiki.org/cgi-bin/view/Plugins/MailerContribDev
Appraisal:	http://TWiki.org/cgi-bin/view/Plugins/MailerContribAppraisal

Related Topics: TWikiPreferences

Mailer Contrib

• A report on changes to all topics that have changed within a particular TWiki web.
• A report on changes to a specific topic or set of topics the user can define flexibly.
• The entire content of a specific topic or set of topics. This is referred to as "news mode."
• The companion plugin (TWiki:Plugins.SubscribePlugin) lets you trivially add a "Subscribe to changes" button to topics

tools/mailnotify

The central component of MailerContrib is a script, tools/mailnotify, that generates and sends out the emails based on analysis of

1. users' subcriptions listed in the WebNotify topic in each web, and
2. changes within the respective webs.

The script collates the changes emails so that each subscriber only receives one changes notification for all changes in all webs in the TWiki. Furthermore, users can elect to receive just summaries of changes, or the entire content of topics that have changed.

Each web can optionally contain a topic called WebNotify.

three spaces * subscriber [ : topics ]

topics is an optional space-separated list of topics:

• ... without a Web. prefix
• ...that exist in this web.

• Using wild-card character in topic names - You can use * in a topic name, where it is treated as a wildcard character. A * will match zero or more other characters - so, for example, Fred* will match all topic names starting with Fred, *Fred will match all topic names ending with Fred, and * will match all topic names.
• Unsubscribing to specific topics - Each topic may optionally be preceded by a '+' or '-' sign. The '+' sign means "subscribe to this topic". The '-' sign means "unsubscribe" or "don't send notifications regarding this particular topic". This allows users to elect to filter out certain topics. Topic filters ('-') take precedence over topic includes ('+') i.e. if you unsubscribe from a topic it will cancel out any subscriptions to that topic.
• Including child-topics in subscription - Each topic may optionally be followed by an integer in parentheses, indicating the depth of the tree of children below that topic. Changes in all these children will be detected and reported along with changes to the topic itself. Note This uses the TWiki "Topic parent" feature.
• Subscribing to entire topic ("news mode") - Each topic may optionally be immediately followed by an exclamation mark ! and/or a question mark ? with no intervening spaces, indicating that the topic (and children if there is a tree depth specifier as well) should be mailed out as complete topics instead of change summaries. ! causes the full topic to be mailed every time even if there have been no changes, and ? will mail the full topic only if there have been changes. One can limit the content of the subscribed topic to send out by inserting %STARTPUBLISH% and %STOPPUBLISH% markers within the topic. Note that "news mode" subscriptions require a corresponding cron job that includes the "-news" option (see details).

Examples: Subscribe Daisy to all changes to topics in this web.

   * daisy.cutter@flowers.com

   * daisy.cutter@flowers.com : Web*

   * DaisyCutter: Petal* (1) WeedKillers (3) Pretty*Flowers

   * StarTrekFan: Star* - *Wars - *sInTheirEyes - *shipTroopers

   * daisy@flowers.com: NewsLetter?

   * buttercup@flowers.com: NewsLetter! (1)

   * GardenGroup: AllNewsLetters? (3)
   * petunia@flowers.com: - ManureNewsLetter

   * 'IT:admins' : Web*

A user may be listed many times in the WebNotify topic. Where a user has several lines in WebNotify that all match the same topic, they will only be notified about changes that topic once (though they will still receive individual mails for news topics).

If a group is listed for notification, the group will be recursively expanded to the e-mail addresses of all members.

Tip:

In the future it is intended that individual users will be able to control the frequency with which they are notified of topic changes, by changing a schedule specification in their home topic. However at present, the notification schedule is controlled by the frequency of activation of the cron job that runs the mailnotify script.

Note that when using the "news mode" ! or ? specifiers the entire topic text is mailed out as HTML. The newsletter template is used to generate the content in this mail, using whatever skin is selected in the topic being mailed.

In addition, the %STARTPUBLISH% and %STOPPUBLISH% markers used by TWiki:Plugins.PublishContrib to delimit the text to be published are respected in news mode.

TWiki/Contrib/MailerContrib code library

Installation Instructions

Like many other TWiki extensions, this module is shipped with a fully automatic installer script written using the BuildContrib.

• If you have TWiki 4.2 or later, you can install from the configure interface (Go to Plugins->Find More Extensions)
  • See the installation supplement on TWiki.org for more information.
• If you have any problems, then you can still install manually from the command-line:
  1. Download one of the .zip or .tgz archives
  2. Unpack the archive in the root directory of your TWiki installation.
  3. Run the installer script ( perl <module>_installer )
  4. Run configure and enable the module, if it is a plugin.
  5. Repeat for any missing dependencies.
• If you are still having problems, then instead of running the installer script:
  1. Make sure that the file permissions allow the webserver user to access all files.
  2. Check in any installed files that have existing ,v files in your existing install (take care not to lock the files when you check in)
  3. Manually edit LocalSite.cfg to set any configuration variables.

• To make sure the installation was successful, run the mailnotify script from the command line, with no parameters. In this case it will print out what it would have done to STDOUT.

• You can change the regular expression that matches email addresses in WebNotify using the {MailerContrib}{EmailFilterIn} setting in =configure. This allows you to limit the domains to which emails can be sent, or even block email addresses altogether.

Setting up your cron job(s)

The script is used as follows: perl -I bin mailnotify [-q] [-news] [ web1 web2 ... webN ]

0 0 * * * cd /usr/local/twiki && perl -I bin tools/mailnotify -q Public Private

0 0 * * * cd /usr/local/twiki && perl -I bin tools/mailnotify -q -Sandbox

0 0 * * 0 cd /usr/local/twiki && perl -I bin tools/mailnotify -news

Developer Notes

Newsletters are sent after formatting using the standard view template, using whatever skin is selected in the topic being mailed.

Contrib Info

• Wind River
• WikiGardens

Mailer Contrib

Allows users to "subscribe" to regularly scheduled e-mails containing either:

• A report on changes to all topics that have changed within a particular TWiki web.
• A report on changes to a specific topic or set of topics the user can define flexibly.
• The entire content of a specific topic or set of topics. This is referred to as "news mode."
• The companion plugin (TWiki:Plugins.SubscribePlugin) lets you trivially add a "Subscribe to changes" button to topics

<--

   PLEASE DO NOT EDIT THIS TOPIC

   It is automatically generated from the subversion repository, and any changes
   you make will simply be overwritten the next time a release is generated.

   Instead, you could check your fix in, raise a bug in the Bugs web, or mail the author.
-->

tools/mailnotify

The central component of MailerContrib is a script, tools/mailnotify, that generates and sends out the emails based on analysis of

1. users' subcriptions listed in the WebNotify topic in each web, and
2. changes within the respective webs.

The script collates the changes emails so that each subscriber only receives one changes notification for all changes in all webs in the TWiki. Furthermore, users can elect to receive just summaries of changes, or the entire content of topics that have changed.

Each web can optionally contain a topic called WebNotify.

<-- Included by WebChangesAlert -->

Users subscribe to email notifications using their WikiName or an alternative email address, and can specify the webs/topics they wish to track, Whole groups of users can also be subscribed for notification.

The general format of a subscription is:

three spaces * subscriber [ : topics ]

Where subscriber can be a WikiName, an E-mail address, or a group name. If subscriber contains any characters that are not legal in an email address, then it must be enclosed in 'single' or "double" quotes.

topics is an optional space-separated list of topics:

• ... without a Web. prefix
• ...that exist in this web.

• Using wild-card character in topic names - You can use * in a topic name, where it is treated as a wildcard character. A * will match zero or more other characters - so, for example, Fred* will match all topic names starting with Fred, *Fred will match all topic names ending with Fred, and * will match all topic names.
• Unsubscribing to specific topics - Each topic may optionally be preceded by a '+' or '-' sign. The '+' sign means "subscribe to this topic". The '-' sign means "unsubscribe" or "don't send notifications regarding this particular topic". This allows users to elect to filter out certain topics. Topic filters ('-') take precedence over topic includes ('+') i.e. if you unsubscribe from a topic it will cancel out any subscriptions to that topic.
• Including child-topics in subscription - Each topic may optionally be followed by an integer in parentheses, indicating the depth of the tree of children below that topic. Changes in all these children will be detected and reported along with changes to the topic itself. Note This uses the TWiki "Topic parent" feature.

   * daisy.cutter@flowers.com

   * daisy.cutter@flowers.com : Web*

   * DaisyCutter: Petal* (1) WeedKillers (3) Pretty*Flowers

   * StarTrekFan: Star* - *Wars - *sInTheirEyes - *shipTroopers

   * daisy@flowers.com: NewsLetter?

   * buttercup@flowers.com: NewsLetter! (1)

   * GardenGroup: AllNewsLetters? (3)
   * petunia@flowers.com: - ManureNewsLetter

   * 'IT:admins' : Web*

A user may be listed many times in the WebNotify topic. Where a user has several lines in WebNotify that all match the same topic, they will only be notified about changes that topic once (though they will still receive individual mails for news topics).

If a group is listed for notification, the group will be recursively expanded to the e-mail addresses of all members.

Tip:

In the future it is intended that individual users will be able to control the frequency with which they are notified of topic changes, by changing a schedule specification in their home topic. However at present, the notification schedule is controlled by the frequency of activation of the cron job that runs the mailnotify script.

Note that when using the "news mode" ! or ? specifiers the entire topic text is mailed out as HTML. The newsletter template is used to generate the content in this mail, using whatever skin is selected in the topic being mailed.

In addition, the %STARTPUBLISH% and %STOPPUBLISH% markers used by TWiki:Plugins.PublishContrib to delimit the text to be published are respected in news mode.

TWiki/Contrib/MailerContrib code library

Installation Instructions

Like many other TWiki extensions, this module is shipped with a fully automatic installer script written using the BuildContrib.

• If you have TWiki 4.2 or later, you can install from the configure interface (Go to Plugins->Find More Extensions)
  • See the installation supplement on TWiki.org for more information.
• If you have any problems, then you can still install manually from the command-line:
  1. Download one of the .zip or .tgz archives
  2. Unpack the archive in the root directory of your TWiki installation.
  3. Run the installer script ( perl <module>_installer )
  4. Run configure and enable the module, if it is a plugin.
  5. Repeat for any missing dependencies.
• If you are still having problems, then instead of running the installer script:
  1. Make sure that the file permissions allow the webserver user to access all files.
  2. Check in any installed files that have existing ,v files in your existing install (take care not to lock the files when you check in)
  3. Manually edit LocalSite.cfg to set any configuration variables.

• To make sure the installation was successful, run the mailnotify script from the command line, with no parameters. In this case it will print out what it would have done to STDOUT.

• You can change the regular expression that matches email addresses in WebNotify using the {MailerContrib}{EmailFilterIn} setting in =configure. This allows you to limit the domains to which emails can be sent, or even block email addresses altogether.

Setting up your cron job(s)

The script is used as follows: perl -I bin mailnotify [-q] [-news] [ web1 web2 ... webN ]

0 0 * * * cd /usr/local/twiki && perl -I bin tools/mailnotify -q Public Private

0 0 * * * cd /usr/local/twiki && perl -I bin tools/mailnotify -q -Sandbox

0 0 * * 0 cd /usr/local/twiki && perl -I bin tools/mailnotify -news

Note: Multiple instances of mailnotify script are not allowed to be executed simutaneously. If you need to run the script multiple times with different options, make sure the cron jobs are scheduled so a previous run has finished before the next starts. You can also write a small script that runs mailnotify in sequence as described in TWiki:Support.DuplicateNotificationsFromMailerCon.

<-- 

 Set STUB = TWiki::Contrib::Mailer
 
 Set SHORTDESCRIPTION = Supports e-mail notification of changes.
 
-->

Developer Notes

Newsletters are sent after formatting using the standard view template, using whatever skin is selected in the topic being mailed.

Contrib Info

Another great TWiki extension from the WikiRing - working together to improve your wiki experience!

Many thanks to the following sponsors for supporting this work:

• Wind River
• WikiGardens

Mailer Contrib

Allows users to "subscribe" to regularly scheduled e-mails containing either:

• A report on changes to all topics that have changed within a particular TWiki web.
• A report on changes to a specific topic or set of topics the user can define flexibly.
• The entire content of a specific topic or set of topics. This is referred to as "news mode."
• The companion plugin (TWiki:Plugins.SubscribePlugin) lets you trivially add a "Subscribe to changes" button to topics

<--

   PLEASE DO NOT EDIT THIS TOPIC

   It is automatically generated from the subversion repository, and any changes
   you make will simply be overwritten the next time a release is generated.

   Instead, you could check your fix in, raise a bug in the Bugs web, or mail the author.
-->

tools/mailnotify

1. users' subcriptions listed in the WebNotify topic in each web, and

The script collates the changes emails so that each subscriber only receives one changes notification for all changes in all webs in the TWiki. Furthermore, users can elect to receive just summaries of changes, or the entire content of topics that have changed.

Each web can optionally contain a topic called WebNotify.

<-- Included by WebChangesAlert -->

three spaces * subscriber [ : topics ]

topics is an optional space-separated list of topics:

• ... without a Web. prefix
• ...that exist in this web.

   * daisy.cutter@flowers.com

   * daisy.cutter@flowers.com : Web*

   * DaisyCutter: Petal* (1) WeedKillers (3) Pretty*Flowers

   * StarTrekFan: Star* - *Wars - *sInTheirEyes - *shipTroopers

   * daisy@flowers.com: NewsLetter?

   * buttercup@flowers.com: NewsLetter! (1)

   * GardenGroup: AllNewsLetters? (3)
   * petunia@flowers.com: - ManureNewsLetter

   * 'IT:admins' : Web*

A user may be listed many times in the WebNotify topic. Where a user has several lines in WebNotify that all match the same topic, they will only be notified about changes that topic once (though they will still receive individual mails for news topics).

If a group is listed for notification, the group will be recursively expanded to the e-mail addresses of all members.

__ Warning: Because an email address is not linked to a user name, there is no way for TWiki to check access controls for email addresses. A user identified by an email address will only be sent change notifications if the topic they are asubscribed to is readable by guest users. You can limit what email addresses can be used in WebNotify, or even block use of emails altogther, using the {MailerContrib}{EmailFilterIn} setting in =configure.

Tip: List names in alphabetical order to make it easier to find the names.

In the future it is intended that individual users will be able to control the frequency with which they are notified of topic changes, by changing a schedule specification in their home topic. However at present, the notification schedule is controlled by the frequency of activation of the cron job that runs the mailnotify script.

Note that when using the "news mode" ! or ? specifiers the entire topic text is mailed out as HTML. The newsletter template is used to generate the content in this mail, using whatever skin is selected in the topic being mailed.

In addition, the %STARTPUBLISH% and %STOPPUBLISH% markers used by TWiki:Plugins.PublishContrib to delimit the text to be published are respected in news mode.

TWiki/Contrib/MailerContrib code library

Installation Instructions

Like many other TWiki extensions, this module is shipped with a fully automatic installer script written using the BuildContrib.

• If you have TWiki 4.2 or later, you can install from the configure interface (Go to Plugins->Find More Extensions)
  • See the installation supplement on TWiki.org for more information.
• If you have any problems, then you can still install manually from the command-line:
  1. Download one of the .zip or .tgz archives
  2. Unpack the archive in the root directory of your TWiki installation.
  3. Run the installer script ( perl <module>_installer )
  4. Run configure and enable the module, if it is a plugin.
  5. Repeat for any missing dependencies.
• If you are still having problems, then instead of running the installer script:
  1. Make sure that the file permissions allow the webserver user to access all files.
  2. Check in any installed files that have existing ,v files in your existing install (take care not to lock the files when you check in)
  3. Manually edit LocalSite.cfg to set any configuration variables.

• To make sure the installation was successful, run the mailnotify script from the command line, with no parameters. In this case it will print out what it would have done to STDOUT.

• You can change the regular expression that matches email addresses in WebNotify using the {MailerContrib}{EmailFilterIn} setting in =configure. This allows you to limit the domains to which emails can be sent, or even block email addresses altogether.

Setting up your cron job(s)

The script is used as follows: perl -I bin mailnotify [-q] [-news] [ web1 web2 ... webN ]

0 0 * * * cd /usr/local/twiki && perl -I bin tools/mailnotify -q Public Private

0 0 * * * cd /usr/local/twiki && perl -I bin tools/mailnotify -q -Sandbox

0 0 * * 0 cd /usr/local/twiki && perl -I bin tools/mailnotify -news

<-- 

 Set STUB = TWiki::Contrib::Mailer
 
 Set SHORTDESCRIPTION = Supports e-mail notification of changes.
 
-->

Developer Notes

Newsletters are sent after formatting using the standard view template, using whatever skin is selected in the topic being mailed.

Contrib Info

Another great TWiki extension from the WikiRing - working together to improve your wiki experience!

Many thanks to the following sponsors for supporting this work:

• Wind River
• WikiGardens

• A report on changes to all topics that have changed within a particular TWiki web.
• A report on changes to a specific topic or set of topics the user can define flexibly.
• The entire content of a specific topic or set of topics. This is referred to as "news mode."

<--

   PLEASE DO NOT EDIT THIS TOPIC

   It is automatically generated from the subversion repository, and any changes
   you make will simply be overwritten the next time a release is generated.

   Instead, you could check your fix in, raise a bug in the Bugs web, or mail the author.
-->

1. users' subcriptions listed in the WebNotify topic in each web, and
2. changes within the respective webs.

The script collates the changes emails so that each subscriber only receives one changes notification for all changes in all webs in the TWiki. Furthermore, users can elect to receive just summaries of changes, or the entire content of topics that have changed.

Each web can optionally contain a topic called WebNotify.

<-- Included by WebChangesAlert -->

• Each topic may optionally be followed by an integer in parentheses, indicating the depth of the tree of children below that topic. Changes in all these children will be detected and reported along with changes to the topic itself. Note This uses the TWiki "Topic parent" feature.

   * daisy.cutter@flowers.com

   * DaisyCutter: Petal* (1) WeedKillers (3) Pretty*Flowers

   * StarTrekFan: Star* - *Wars - *sInTheirEyes - *shipTroopers

   * daisy@flowers.com: NewsLetter?

   * buttercup@flowers.com: NewsLetter! (1)

   * GardenGroup: AllNewsLetters? (3)
   * petunia@flowers.com: - ManureNewsLetter

Tip:

In the future it is intended that individual users will be able to control the frequency with which they are notified of topic changes, by changing a schedule specification in their home topic. However at present, the notification schedule is controlled by the frequency of activation of the cron job that runs the mailnotify script.

1. 1. Check in any installed files that have existing ,v files in your existing install (take care not to lock the files when you check in)
   2. Manually edit LocalSite.cfg to set any configuration variables.

0 0 * * * cd /usr/local/twiki && perl -I bin tools/mailnotify -q Public Private

<-- 

 Set STUB = TWiki::Contrib::Mailer
 
 Set SHORTDESCRIPTION = Supports e-mail notification of changes.
 
-->

Newsletters are sent after formatting using the standard view template, using whatever skin is selected in the topic being mailed.

MailerContrib

<--

   PLEASE DO NOT EDIT THIS TOPIC

   It is automatically generated from the subversion repository, and any changes
   you make will simply be overwritten the next time a release is generated.

   Instead, you could check your fix in, raise a bug in the Bugs web, or mail the author.
-->

WARNING: TWiki-4 only. If you want to use this extension with an earlier version of TWiki, please see here

<-- Included by WebChangesAlert -->

• Specify topics without a Web. prefix
• Topics must exist in this web.
• Topics may be specified using * wildcards

• Each topic may optionally be followed by an integer in parentheses, indicating the depth of the tree of children below that topic. Changes in all these children will be detected and reported along with changes to the topic itself. Note This uses the TWiki "Topic parent" feature.

Tip: List names in alphabetical order to make it easier to find the names.

In the future it is intended that individual users will be able to control the frequency with which they are notified of topic changes, by changing a schedule specification in their home topic. However at present, the notification schedule is controlled by the frequency of activation of the cron job that runs the mailnotify script.

• To make sure the installation was successful, run the mailnotify script from the command line, with no parameters. In this case it will print out what it would have done to STDOUT.

Contrib Info

<-- Included by WebChangesAlert -->

Subscribers are listed in WebNotify following one of these bullet list formats:

three spaces * [ webname . ] wikiName - SMTP mail address
three spaces * [ webName . ] wikiName
three spaces * SMTP mail address
three spaces * SMTP mail address : topics
three spaces * [ webname . ] wikiName : topics

where topics is a space-separated list of topic names.

• Specify topics without a Web. prefix
• Topics must exist in this web.
• Topics may be specified using * wildcards
• Each topic may optionally be followed by an integer in parentheses, indicating the depth of the tree of children below that topic. Changes in all these children will be detected and reported along with changes to the topic itself. Note This uses the TWiki "Topic parent" feature.
• Each topic may optionally be preceded by a '+' or '-' sign. The '+' sign means "subscribe to this topic" (the same as not putting anything). The '-' sign means "don't send notifications regarding this topic". This allows users to elect to filter out changes to certain topics (and their children, to an arbitrary depth). Topic filters ('-') take precedence over topic includes ('+').

For example:

   * daisy@flowers.com
   * daisy@flowers.com: Web*
   * DaisyCutter: Petal* (1) WeedKillers (3) Red*Phlox
   * StarTrekFan: * - *Wars - *sInTheirEyes - *shipTroopers

If a TWiki group is listed for notification, the group will be recursively expanded to the e-mail addresses of all members.

Tip: List names in alphabetical order to make it easier to find the names.

In the future it is intended that individual users will be able to control the frequency with which they are notified of topic changes, by changing a schedule specification in their home topic. However at present, the notification schedule is controlled by the frequency of activation of the cron job that runs the mailnotify script.

Note mailnotify ignores permissions in webs. It is entirely possible for a user to get added to a WebNotify topic in a web, when they are not authorised to view the topics in that web. This could result in them having limited access to sensitive information (the topic summaries).

• Download the ZIP file from the Plugin web (see below)
• Unzip MailerContrib.zip in your twiki installation directory.
• Run the installer script MailContrib_intaller.pl or alternatively resolve all dependencies manually.
• To make sure the installation was successful, run the mailnotify script from the command line, with no parameters. In this case it will print out what it would have done to STDOUT.

Summary of Contents

tools/mailnotify

<-- Included by WebChangesAlert -->

Subscribers are listed in WebNotify following one of these bullet list formats:

three spaces * [ webname . ] wikiName - SMTP mail address
three spaces * [ webName . ] wikiName
three spaces * SMTP mail address
three spaces * SMTP mail address : topics
three spaces * [ webname . ] wikiName : topics

where topics is a space-separated list of topic names.

If a TWiki group is listed for notification, the group will be recursively expanded to the e-mail addresses of all members.

Tip: List names in alphabetical order to make it easier to find the names.

In the future it is intended that individual users will be able to control the frequency with which they are notified of topic changes, by changing a schedule specification in their home topic. However at present, the notification schedule is controlled by the frequency of activation of the cron job that runs the mailnotify script.

Note mailnotify ignores permissions in webs. It is entirely possible for a user to get added to a WebNotify topic in a web, when they are not authorised to view the topics in that web. This could result in them having limited access to sensitive information (the topic summaries).