TailWatch

Introduction

TailWatch is a process that handles the tasks previously handled by 4 separate processes: eximstats, chchkservd, antirelayd and cpbandwd. Each of those older processes performed very similar functions. By merging the common functionality into a single process, system resource requirements are reduced.

How it works

As suggested by the name, TailWatch monitors a log file for certain activity and takes action based upon the activity. The specifics, such as which log to monitor and what to do with the information, is handled by a driver module. The stock drivers as shipped by cPanel are:

• antirelayd
• cpbandwd
• chkservd
• eximstats

These drivers are much improved versions of the prior stand-alone processes, building on the features of the older implementations.

Upgrading to TailWatch

Upgrading to a version of cPanel that uses TailWatch should be a seamless experience. TailWatch is started, then the older processes are stopped. The older scripts, such as /usr/local/cpanel/bin/eximstats, are modified to no longer function after the upgrade. These scripts will eventually be removed by a later update. Attempting to start the older process will result in failure and a message being displayed informing the user to use TailWatch instead.

The chkservd monitors for antirelayd, cpbandwd and eximstats are removed by the upgrade. The chkservd monitor then simply monitors the tailwatchd service to ensure all enabled drives are loaded and running. You can control which drivers are enabled via WHM -> Service Configuration -> Service Manager.

chkservd Overview

chkservd is the driver used to determine if a process is online and whether it needs to be restarted. You can decide which services are monitored via the WHM -> Service Configuration -> Service Manager interface.

chkservd checks each service once every 5 minutes to determine if it is online. It will detail it's findings to the chkservd log file. This file can be found at:

• /var/log/chkservd.log

If a service is online, chkservd will move onto the next. If a service is offline, it will attempt to restart it via the cPanel restart scripts located in:

• /scripts/restartsrv_*

If you so choose, you can have the results emailed to you. The results are also displayed in the WHM Service Status Panel under Server Information

Common Debugging

One of the most common issues occurs when a service is not monitored or has been disabled and is still listed on the service status page as offline. To fix this error, you may need to remove and/or edit a few files. For the purposes of this tutorial, assume that the service is clamd.

Remember: In the example below, clamd can be replaced with any other service.

• Remove the run file:
  • rm -f /var/run/chkservd/clamd
• Remove the chkservd restart file:
  • rm -f /etc/chkserv.d/clamd
• Edit the chkservd configuration file with your preferred text editor. You may need to remove the line for clamd. The chkservd configuration file can be found at:
  • /etc/chkserv.d/chkservd.conf
• Restart chkservd:
  • /scripts/restartsrv_chkservd

After you have made the necessary changes, simply return to the Service Manager interface and click Save at the bottom without making any changes.

Process Control and Status

The TailWatch process can be controlled with various command line flags passed to /usr/local/cpanel/bin/tailwatchd. The full list of flags is obtainable using the -help flag:

/usr/local/cpanel/bin/tailwatchd - Driver based real time log processing 

--help This screen 

--version Show version of Cpanel::TailWatch  

--version=long Same as --version but also include active driver modules version 

information also. 

--status Status information  

--status=long Same as --status but also include information about the PID if its running  

--stop Stop current tailwatch daemon  

--start Start tailwatch daemon  

--restart Start tailwatch daemon, stopping it first if necessary  

--trace log the first 10,000 lines the drivers collectively process  

--trace=n log the first 'n' lines the drivers collectively process  

Trace only takes effect on --[re]start.  

--start --trace (equivalent to --start --trace=10000)  

--restart --trace='perl -e 'print 12+3-4+5+67+8+9;''  

--start --trace=1729  

--status --trace (--trace has no effect here)  

--debug This tells all driver's to log() debug messages.  

Debug only takes effect in --[re]start. Typically the message should be prefaced with 

‘[debug]’ and are about processed lines that result in an entry of some sort 

--perldoc This shows the POD for Cpanel::TailWatch 

Customization

It is possible to add custom functionality by providing a new driver. Full documentation and examples are provided in the PerlDoc for the TailWatch Module. To access this documentation, execute:

• /usr/local/cpanel/bin/tailwatchd --perldoc

These drivers need to be placed in /usr/local/cpanel/Cpanel/TailWatch. Once the drivers are in place, restarting TailWatch will load the new drivers, providing the functionality they provide.