CiviMail Configuration

Fen Labalme's picture

Setting Up CiviMail

This process should be streamlined and scripted; it requires work to be done on the mail vhost, the web vhost, in qmail admin and via the Drupal/CiviCRM admin GUI interfaces. But this description of the process is a good start. (With a bit of generalization, this might be a good blog post and/or contribution back to the CiviCRM community.)

The old imap2soap configuration page (pre-CiviCRM 2.1) can still be found at CiviMail Configuration: Old

Variables Used in this Section

Variable Name Value & Description
CIVIMAIL The Drupal user name. Used for handling bounces, etc. and for authentication
PASSWORD The password used by the CIVIMAIL Drupal user and by the CIVIMAIL IMAP account
CLIENT the client vhost username without the initial "w_"
SITENAME as in SITENAME.civicactions.net - often the same as CLIENT
CLIENT_MX_HANDLER This is set in DNS, and will usually be something like 'che-mail.civicactions.net' (for clients on a shared vhost like 'che-web') or the name of the VE when a client has a dedicated virtual environment
CIVIMAIL_MAIL_DOMAIN The mail domain the client would like to use for CiviMail. Often, if they are currently using an external mail handler for their organization's general mail (which is recommended; if they are not, suggest they use Google Apps for Your Domain as CivicActions does not support spam filtering and has limited mail capabilities) they may want to prepend 'civimail.' or perhaps 'lists.' to their org's URL, such as 'civimail.savetheearth.org'

System Tasks

This section will get folded into vhost creation whenever civimail is requested; see System ticket #729 "create-civimail" mail vhost script. The one necessary output of this section - that will need to be communicated back to the Project's TL and/or PM - is the PASSWORD for the CIVIMAIL account.

Requirements

  • a mail domain with webmail admin interface
  • ssh login creds for client's mail vhost
  • Drupal admin creds for client site

Determine CLIENT's Mail Handler (CLIENT_MX HANDLER)

  • This command will return the mail handler
    host ntr.civicactions.net | grep 'mail is handled by' | cut -d ' ' -f 7
    

Create the Mail Domain (Should Already be Done)

  • As root on CLIENT's MX mail handler, run:
    /opt/vpopmail/current/bin/vadddomain -r -u m_CLIENT -d ~m_CLIENT CIVIMAIL_MAIL_DOMAIN -e 'delete'
    
  • Configure Mail Delivery Defaults

    • From the CLIENT's MX mail handler, enter the mail domain vhost and take note of the postmaster's password (you'll need it later):
      su - m_CLIENT
      cat .passwd
      
    • Create the CIVIMAIL user's password and save it in .passwd
      PASSWORD=`passgen`
      echo "Password for CIVIMAIL@CIVIMAIL_MAIL_DOMAIN = $PASSWORD" >> .passwd
      
    • Ensure the .qmail-default file will delete any messages that do no match an endpoint
      cd ~/${CLIENT_MAIL_DOMAIN}
      cat .qmail-default
      
    • This file should look like:
      | /opt/vpopmail/current/bin/vdelivermail '' delete
      
    • Set up the CiviMail message handlers:
      echo "&CIVIMAIL@CLIENT.civicactions.net" > .qmail-bounce-default
      for i in b c confirm o optout r reply s subscribe u unsubscribe re e resubscribe; do ln -s .qmail-bounce-default .qmail-$i-default; done
      

    QMailAdmin: Create IMAP Mailbox

    • Log into the CLIENT webmail admin interface at http://webmail.CLIENT.civicactions.net/cgi-bin/qmailadmin
      • The postmaster password in in ~m_CLIENT/.passwd on the mailhost (saved in a step above)
    • Create an account for user "CIVIMAIL" using the PASSWORD generated above
    • Set the account to be a "CatchAll"
      • Then unset this by clicking "Set catchall email deleted"

    Drupal/CiviCRM: Configure CiviCRM

    Create 'CIVIMAIL' Drupal Account

    Create a Drupal user account named CIVIMAIL (typically 'civimail') using PASSWORD

    Configure CiviMail Mail Account

    • Go to /civicrm/admin/mailSettings&reset=1 (CiviMail >> Mail Accounts)
      • Name: CLIENT Back Channel Default
      • Server: CLIENT_MX_HANDLER
      • Username: CIVIMAIL@CIVIMAIL_MAIL_DOMAIN
      • Password: PASSWORD
      • Localpart:
      • Domain: CLIENT_MX_HANDLER
      • Return-Path:
      • Protocol: IMAP
      • Source:
      • Use SSL?
      • Default Option? checked
    • Save

    Configure CiviMail Mailer Settings

    • Go to /civicrm/admin/mail?reset=1 (CiviMail >> Mailer Settings)
      • VERP Separator: - (a single hyphen)
    • Save

    CLIENT Vhost: Create Cron Jobs

    Ensure htaccess Allows CLIENT Access

    • Log into CLIENT vhost
    • The following script should output two essentially identical lines:
      • Note: on some sites, CLIENT != SITENAME; on those sites, please set SITENAME manually
      CLIENT=`whoami | cut -f 2 -d _`
      SITENAME=${CLIENT}
      IPADDR=`host ${SITENAME}.civicactions.net | head -1 | cut -f 4 -d ' '`
      echo -e "\nAllow from ${IPADDR}"; grep ${IPADDR} ~/sites/${SITENAME}-dev.civicactions.net/htaccess ; echo ""
      
    • If not found, edit, save and commit the CLIENT/trunk/htaccess from your sandbox
    • Then, back on the CLIENT vhost:
      cd ~/sites/${SITENAME}-dev.civicactions.net
      svn up
      
    • ...and re-run the above script

    Ensure CiviCRM settings_location.php Exists

    • If ~/sites/${SITENAME}.civicactions.net/modules/civicrm/settings.location.php doesn't exist, then:
      echo -e "" > ~/sites/${SITENAME}.civicactions.net/modules/civicrm/settings.location.php
      

    Create SITEKEY

    • Ensure that CIVICRM_SITE_KEY is not defined in civicrm.settings.inc
    • Ensure there is a line in civicrm.settings.php that looks like:
      define('CIVICRM_SITE_KEY', 'SITEKEY');
      
    • Then run this script:
      SITEKEY=`php-cli -r 'echo md5(uniqid(rand(), true));'`
      cd  ~/sites/SITENAME.civicactions.net/
      chmod 711 .
      replace SITEKEY ${SITEKEY} -- civicrm.settings.php
      

    Test Configuration

    • Set CIVI_USER, CIVI_PASS and SITE_KEY to appropriate values for your install (SITENAME should already be defined from above).
    • The result should show the INBOX being opened and checked:
      cd ~
      CIVI_USER=civimail
      CIVI_PASS=xxxxxxxx
      SITE_KEY=yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
      BASE_URL="http://${SITENAME}.civicactions.net/sites/${SITENAME}.civicactions.net/modules/civicrm/bin"
      AUTH_STR="-d name=${CIVI_USER} -d pass=${CIVI_PASS} -d key=${SITE_KEY}"
      curl ${AUTH_STR} ${BASE_URL}/CiviMailProcessor.php
      

    Edit the local crontab

    • Add the appropriate sections from the following to the local crontab - replacing the ${VARIABLES} - using 'crontab -e' (launches the 'vi' text editor):
    MAILTO=cron-notify@civicactions.com,CLIENT-int@lists.civicactions.net
    PATH=/bin:/usr/bin:/usr/local/bin
    
    # Run Drupal cron.php job (updates search indexes, other housekeeping)
    */20 * * * *   cd public_html/SITENAME/sites/$SITENAME.civicactions.net && COLUMNS=80 drush -q cron
    11 */2 * * * cd public_html/SITENAME-qa/sites/SITENAME-qa.civicactions.net && COLUMNS=80 drush -q cron
    31 */6 * * * cd public_html/SITENAME-dev/sites/SITENAME-dev.civicactions.net && COLUMNS=80 drush -q cron
    
    ## CiviCRM CiviMail configuration (only needed if CiviMail is active)
    CIVI_USER=civimail
    CIVI_PASS=xxxxxxxx
    SITE_KEY=yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
    
    BASE_URL="http://SITENAME.civicactions.net/sites/SITENAME.civicactions.net/modules/civicrm/bin"
    AUTH_STR="-d name=${CIVI_USER} -d pass=${CIVI_PASS} -d key=${SITE_KEY}"
    
    # Check for queued mail to send every 15 minutes (for no output, change civimail.cronjob.out to /dev/null)
    */15 * * * * curl -s -o civimail.cronjob.out ${AUTH_STR} ${BASE_URL}/civimail.cronjob.php
    
    # Process bounces every other hour (for no output, change CiviMailProcessor.out to /dev/null)
    50 */2 * * * curl -s -o CiviMailProcessor.out ${AUTH_STR} ${BASE_URL}/CiviMailProcessor.php
    
    ## Alternate method of calling CiviMail cron job using drush (currently unused)
    # cd public_html/SITENAME/sites/SITENAME.civicactions.net && COLUMNS=80 drush civicrm-civimail -u ${CIVI_USER} -p ${CIVI_PASS} -s ${SITE_KEY}
    

Tags: 

Groups audience: 

Openness: 

Public - accessible to all