Interpreting Log Files

This page provides the information required to interpret a Web Appliance sophos_log. This file is saved as part of a system backup that is configured on the Configuration > System > Backup page, if you have the Transaction log files at least once daily at midnight option selected, and you have chosen to back up the logs in the Sophos format. If you have chosen to back up the logs in the Squid format, see the Squid log format page.

Introduction

The appliance keeps a log (called sophos_log) of all requests it processes. The following is an example of a sophos_log entry:

h=10.99.115.13 u="DOMAIN\\johnsmith" s=200 X=- t=1336666489 T=284453
Ts=0 act=1 cat="0x220000002a" app="-" rsn=- threat="-" type="text/html" ctype="text/html"
sav-ev=4.77 sav-dv=2012.5.10.4770003 uri-dv=- cache=- in=1255 out=26198
meth=GET ref="-" ua="Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20100101 Firefox/12.0"
req="GET http://www.google.ca/ HTTP/1.1" dom="google.ca" filetype="-" rule="0"
filesize=25815 axtime=0.048193 fttime=0.049360 scantime=0.011 src_cat="0x2f0000002a"
labs_cat="0x2f0000002a" dcat_prox="-" target_ip="74.125.127.94" labs_rule_id="0"
reqtime=0.027 adtime=0.001625 ftbypass=- os=Windows authn=53 auth_by=portal_cache 
dnstime=0.000197 quotatime=- sandbox=-
      
h=192.168.98.38 u="SILKNET2\\t\xc3\xb5m\xc3\xa4sj\xc3\xb3n\xc3\xa9s" s=200 X=X
t=1178921655 T=3444378 Ts=3 act=1 cat="0x220000001a" rsn=- threat="-"
type="application/x-exe" ctype="application/x-msdos-program" sav-ev=4.17
sav-dv=2007.5.9.417008 uri-dv=2007.5.9.6031 cache=MISS in=905 out=236936 meth=GET
ref="http://funnel-web.ca.sophos.com/mime/" ua="Mozilla/4.0 (compatible; MSIE 6.0;
Windows NT 5.1; SV1; .NET CLR 1.1.4322)"
req="GET http://funnel-web.ca.sophos.com/mime/exe.exe HTTP/1.1" dom="sophos.com"
filetype="exe.exe" rule="3479751" filesize=266360 axtime=0.001234 fttime=0.000235
scantime=0.010 src_cat="0x3200001d53" labs_cat="0x0200000012" dcat_prox="-"
target_ip="192.168.3.125" labs_rule_id="3479751" reqtime=0.056 adtime=0.000003
      

Data Fields

The following table explains the keys used in the sophos_log file.

Field Description
ep This setting is optional and is only displayed if you are using Endpoint Web Control. A value of ep=1 means the browsing occurred on the endpoint computer, and that this entry was then uploaded to the appliance.
sxl This setting is optional. An entry of sxl=y or sxl=n indicates if an SXL lookup for a particular transaction was successful or not.
h Remote host (the IP address that sent the request).
u Remote user who made the request (null if user authentication is off). Note that the second entry example above shows how UTF-8 usernames are encoded in the log file.
s HTTP status code sent back to the client.
X

The connection status when the response was completed:
X = connection aborted before the response completed,
+ = connection may be kept alive after the response is sent,
- = connection will be closed after the response is sent.

t Timestamp (in seconds) of when the request was first received since the UNIX Epoch, i.e. 1970-01-01 00:00:00 UTC).
T Time in microseconds required to serve this request.
Ts Time required (in seconds) to serve this request.
act

Action code that identifies the outcome of the request:
-7 = User is shown a sandbox analysis page .
-6 = User attempted to proceed on a quota page, but the request was blocked.
-5 = Block page displayed: daily quota time exceeded.
-4 = Quota time warning displayed.
-3 = User proceeded but request was blocked.
-2 = Request was warned.
-1 = Request was blocked.
1 = Request was allowed.
2 = Request was warned and user decided to proceed.
3 = User proceeded.
4 = User accepts a quota time and proceeds.
5 = Requested proceeded after quota accepted.

cat Matched URI category ID (e.g. 0x2n00000034). The n indicates the risk level: 0=unclassified, 1=trusted, 2=low, 3=medium, and 4=high. For a full listing of the values used with the cat key, see the section Category Codes below.
app Application identified by the appliance.
rsn

Reason code that identifies why a particular request was blocked. The supported codes are listed below; however, this list is subject to change.
1401 = Blocked because request contains a virus,
1402 = Blocked by Local or Sophos URI list,
1403 = Blocked by file type,
1404 = Blocked because the request is encrypted and could not be scanned,
1405 = Blocked because the virus scanner timed out when trying to scan the request,
1406 = Blocked by policy,
1407 = Blocked because the originating server failed SSL certificate validation,
1408 = Blocked ‘Range’ requests,
1409 = Blocked by tag.
1410 = Blocked (Lookup failed)
1411 = Blocked because of application control.
1412 = Blocked by Sandstorm

threat Malware/Virus name detected by the scanner.
type MIME type identified by the appliance.
ctype Content-Type indicated by the originating server.
sav-ev Sophos Anti-virus engine version used for this request.
sav-dv Sophos Anti-virus data version used for this request.
uri-dv Sophos URI list version used for this request.
cache Cache HIT or MISS (whether the request was served from the appliance cache).
in Amount of data (including headers, in bytes) received from the client by the appliance for this request.
out Amount of data (including headers, in bytes) sent out by the appliance to the client for this request.
meth HTTP request method (i.e. POST/GET/CONNECT).
ref HTTP "referrer" field, populated if this request was referred by another (e.g. request for an image that is part of a web page).
ua User-Agent of the client that’s making this request.
req HTTP request string (including request method, URL requested, and request protocol).
dom Domain portion of the request URI.
filetype Sophos filetype category (e.g. both ‘application/x-gzip’ & ‘application/x-bzip2’ belong to the category ‘archive.compress’).
rule Policy rule ID matched for this request. Local Site List categorization rules are prefixed with 'LSL-'. Sophos categorization rules are not documented.
filesize Size of the file served for this request (does not include HTTP protocol overhead).
axtime Amount of time (in seconds) it took to perform access checks.
fttime Amount of time (in seconds) it took to perform file-typing.
scantime Amount of time (in seconds) it took to perform scanning.
src_cat Internal use only.
labs_cat Internal use only.
dcat_prox Internal use only.
target_ip The IP address that the request resolves to.
labs_rule_id Internal use only.
reqtime Amount of time (in seconds) that a web resource is in the queue for scanning.
adtime Amount of time (in seconds) it took to perform Active Directory or eDirectory authentication.
ftbypass Internal use only.
os If you have specified a connection profile for authentication, this is the type of operating system or device that was detected.
auth_by The form of authentication that succeeded (for example, "bypass," "portal," "kerb").
authn Internal use only.
dnstime Amount of time (in seconds) it took to get a DNS response. This can help troubleshoot DNS latency issues.
quotatime The number of quota minutes used. This relates to the number of minutes allowed for this quota. This is configured in the Naming & Scheduling tab of the Additional Policy wizard: Configuration > Group Policy > Additional Policies.
sandbox Identifies whether a download should be sent to the sandbox component of Sophos Sandstorm.

Special Notes

The basic format is [key]=[value] where there is no whitespace between the key, the equals character or the value. The value may be enclosed in quotes, e.g. [key]="[value]". Values that contain embedded whitespace will always use quote delimiters. Implementers are encouraged to check for, and remove if found, surrounding unescaped quote characters for each value.

Each log line is terminated by a linefeed character (ASCII LF, 0x10). Since these log files may be moved between computers that could reformat the text file, implementers are encouraged to recognize and accept log lines terminated by any of the standard text line termination schemes: linefeed, carriage return (ASCII CR, 0x0D) or LF+CR as used by Windows/DOS.

Quotes ( " ) and backslashes ( \ ) within a value are escaped by prepending a backslash. Keys will never contain such characters.

Null values may be represented by an empty string (e.g. [key]= or [key]="") or a dash character (e.g. [key]=- or [key]="-"). Any value containing only a dash character should be treated as if the value was not specified. Some fields will contain a null string if the value would otherwise be undefined (e.g. for a blocked request, the filetype field will be meaningless).

The appliance supports Unicode usernames when authenticating users to an Active Directory or eDirectory server. In these cases the user field will contain a UTF-8 string; the non-printable bytes are escaped using the ‘\x’ prefix followed by the hexadecimal representation of the raw bytes (e.g. \xAF). In the example above, ‘SILKNET2\\t\xc3\xb5m\xc3\xa4sj\xc3\xb3n\xc3\xa9s’ translates to ‘SILKNET2\tõmäsjónés’, where the username is:

t	U+0074, Latin Small Letter T
õ	U+00F5, Latin Small Letter O with Tilde
m	U+006D, Latin Small Letter M
ä	U+00E4, Latin Small Letter A with Diaeresis
s	U+0073, Latin Small Letter S
j	U+006A, Latin Small Letter J
ó	U+00F3, Latin Small Letter O with Acute
n	U+006E, Latin Small Letter N
é	U+00E9, Latin Small Letter E with Acute
s	U+0073, Latin Small Letter S

The list of supported fields will change. Implementers are encouraged to silently ignore fields containing an unrecognized key.

The order of fields contained in each log line will change. Implementers are encouraged to parse fields using methods that do not rely on specific ordering.

Category Codes

The following table explains the values used with the cat key. The n indicates the risk level: 0=unclassified, 1=trusted, 2=low, 3=medium, and 4=high.

Category ID User-visible category name Category ID User-visible category name
0x0n00000000 Uncategorized 0x2n0000001C Kid's Sites
0x2n00000001 Adult/Sexually Explicit 0x2n0000001D Motor Vehicles
0x2n00000002 Advertisements & Pop-Ups 0x2n0000001E News
0x2n00000003 Alcohol & Tobacco 0x2n0000001F Peer-to-Peer
0x2n00000004 Arts 0x2n00000020 Personals and Dating
0x2n00000005 Blogs & Forums 0x2n00000021 Philanthropic & Professional Orgs.
0x2n00000006 Business 0x2n00000022 Phishing & Fraud
0x2n00000007 Chat 0x2n00000023 Photo Searches
0x2n00000008 Computing & Internet 0x2n00000024 Politics
0x2n00000009 Criminal Activity 0x2n00000025 Proxies & Translators
0x2n0000000A Downloads 0x2n00000026 Real Estate
0x2n0000000B Education 0x2n00000027 Reference
0x2n0000000C Entertainment 0x2n00000028 Religion
0x2n0000000D Fashion & Beauty 0x2n00000029 Ringtones/Mobile Phone Downloads
0x2n0000000E Finance & Investment 0x2n0000002A Search Engines
0x2n0000000F Food & Dining 0x2n0000002B Sex Education
0x2n00000010 Gambling 0x2n0000002C Shopping
0x2n00000011 Games 0x2n0000002D Society & Culture
0x2n00000012 Government 0x2n0000002E Spam URLs
0x2n00000013 Hacking 0x2n0000002F Sports
0x2n00000014 Health & Medicine 0x2n00000030 Spyware
0x2n00000015 Hobbies & Recreation 0x2n00000031 Streaming Media
0x2n00000016 Hosting Sites 0x2n00000032 Tasteless & Offensive
0x2n00000017 Illegal Drugs 0x2n00000033 Travel
0x2n00000018 Infrastructure 0x2n00000034 Violence
0x2n00000019 Intimate Apparel & Swimwear 0x2n00000035 Weapons
0x2n0000001A Intolerance & Hate 0x2n00000036 Web-based email
0x2n0000001B Job Search & Career Development 0x2n10000037 Custom

Sandbox codes

The following table explains the values used with the sandbox key.

Sophos log Description
sandbox=- Engine reports sandboxing is not needed.
sandbox=1 Engine reports sandboxing is needed, file is not sent for analysis.
sandbox=2 File is sent to cloud to be analyzed
sandbox=3 Sandbox fast response: file is clean.
sandbox=4 Sandbox cloud response: file is clean.
sandbox=-1 Sandbox fast response: file is malicious.
sandbox=-2 Sandbox fast response: error occurred.
sandbox=-3 Sandbox cloud response: file is malicious.
sandbox=-4 Sandbox cloud response: error occurred.