Apparent Discrepancies in Bandwidth Usage Statistics
For WHM version 11.40
Often, when users compare bandwidth
usage statistics provided by log analysis programs such as AWStats
, Logaholic, and Webalizer
statistics provided by cPanel & WHM, they are surprised to find apparent discrepancies.
These apparent discrepancies arise as a result of the way different programs measure bandwidth
The purpose of this document is to explain the differences in measurement methods. Our goal is to help web hosts and website owners understand how to obtain the most accurate bandwidth
Where to find bandwidth data in cPanel & WHM
Bandwidth data in cPanel & WHM comes from 2 sources: data stored in summary files and the data stored in RRDtool databases.
Data stored in summary files
Several cPanel & WHM screens display (mainly numeric) bandwidth
usage information stored in summary files. These screens include:
- WHM >> Account Information >> View Bandwidth Usage.
- WHM >> Account Functions >> Limit Bandwidth Usage.
- cPanel >> Stats >> Monthly Bandwidth Transfer.
- cPanel >> Logs >> Bandwidth >> monthly bandwidth usage pie charts.
- cPanel >> Logs >> Bandwidth >> select the domain >> Bandwidth By Day.
All of the information on these pages is generated by the same bandwidth
In contrast, AWStats
, Logaholic, and Webalizer
according to a different method
. This is why sometimes it appears that there is no correlation between the data shown on the bandwidth
usage information screens and the data that AWStats
, Logaholic, and Webalizer
Data stored in RRDtool databases
graphs are generated from usage information stored in RRDtool databases. These graphs include:
- cPanel >> Logs >> Bandwidth >> Traffic for Today graph.
- cPanel >> Logs >> Bandwidth >> Traffic for Last 7 Days graph.
- cPanel >> Logs >> Bandwidth >> Traffic for Last 12 Months graph.
Data for these bandwidth
graphs are calculated using a different method
from the summary file data above.
What causes the discrepancies?
Differences in measuring daily bandwidth statistics
The information presented by cPanel & WHM is likely to differ from the day's usage as shown by log processing programs, such as AWStats
, Logaholic, or Webalizer
. There are three possible reasons for this:
- The server administrator can set log processing and bandwidth processing cycles independently. This means that the 2 statistics can be generated hours apart, in which case, they may vary widely.
- Even if the server administrator attempts to synchronize log and bandwidth processing cycles, the system will always process the statistics at least 8 minutes apart.
- The statistics are processed such that usage data from 5 minutes before the time of processing, up to the time of processing, is always held until the next processing cycle. Therefore, those statistics would appear in a later report.
In short, daily bandwidth
usage numbers are far too volatile for the bandwidth
and log processing results to match.
Overall differences in measuring bandwidth
There are further differences between the methods used by log programs and cPanel & WHM to process bandwidth
How the log processing programs measure bandwidth
The log processing programs extract the amount of bytes transferred from the Apache
combined access log. The Apache
combined access log records incoming data requests. One of the pieces of data recorded is the size of the requested file, measured in bytes.
The size amount of the requested file:
- only shows the full size of the content of the file being requested. It does not reflect the actual number of bytes that were transferred.
- measures the content of the file only. This does not include the size of the headers of the response.
- does not include the size of the request.
In other words, the log processing programs measure bandwidth
by showing the size of the data that was requested
, not the amount the data that was actually transferred
How cPanel & WHM measures bandwidth
cPanel & WHM measures bandwidth
by combining the amount of bytes of an incoming data request with the outgoing transferred
data (i.e., the response to that request). cPanel & WHM obtains this data from the Apache
The amount of bandwidth
- the total bandwidth used by the incoming request that solicited the content.
- the total bandwidth used by the complete response: the sum of the size of the content plus the header.
How this affects the results
For most websites, these differences will be relatively small. The headers are a few hundred bytes; the request itself is usually much less. Given reasonably sized output content, this overhead works out to a few percentage points' difference.
There are a certain types of request/response combinations in which these differences begin to make a significant impact on the bandwidth
- Large numbers of non-success responses — Because redirects and errors have virtually no content, and are mostly composed of headers, the bandwidth reported by the log processing programs will be much smaller than that reported by cPanel & WHM.
- Large numbers of very small responses — For transferred content below 1 kilobyte, the log processing programs will show a smaller bandwidth usage amount than that reported by cPanel & WHM. This is because the combined size of the response headers plus the request can often be larger than 1 kilobyte.
- Large requests cancelled by the user — As stated above, log processing programs measure bandwidth by showing the size of the data requested, not the amount the data that was actually transferred. This means that these programs do not make a distinction between completed and cancelled requests. Therefore, a high number of requests cancelled by the user may cause the amount of bandwidth reported by cPanel & WHM to look substantially smaller than the bandwidth reported by the log processing program. This situation occurs often with sites that serve a large number of video or audio streaming requests.
- For example, 100 people request a 1GB video, but cancel the request after only 10% of the video has downloaded. cPanel & WHM would report that less than 10 GB of data were transferred. However, because of the way they obtain their information, the log processing programs would report that 100 GB were transferred.
Which report should I use?
You will want to use the more complete bandwidth
measurement reported by cPanel & WHM, for example on the cPanel Bandwidth
screen, or the WHM View Bandwidth Usage
screen. Log analysis programs, while not as accurate in reporting bandwidth
, are worth using for information about pageview statistics, referrals, and other information.
You should set bandwidth
processing cycles as desired on the Server Configuration >> Statistics Software Configuration
screen of WHM, in the Bandwidth Processing Frequency
area of the Schedule Configuration
This will allow you to balance server performance with reporting latency and, given those constraints, will provide the most accurate bandwidth
usage statistics possible.
For up-to-date usage information, you may process an individual user's statistics at any given moment. You can do this on the Server Configuration >> Statistics Software Configuration
screen of WHM, under the Process Statistics for User