Chapter 4. Preprocessing: An Introduction

Introduction

Snort has several components other than the rules engine. For example, some packets and applications have to be decoded into plain text for Snort rules to trigger. The component that handles the packets before they get to the rules engine is called the preprocessor. The available preprocessors and their functions as of Snort 2.2.0 are listed in Table 4-1.

Table 4-1. Snort 2.2.0 preprocessors

Preprocessor name

Function

Flow

This preprocessor helps keep a state flow log of packets passing through the Snort engine. The only preprocessor to use this engine so far is the new flow-portscan.

Frag2

This preprocessor detects and reassembles fragmented packets attempting to bypass detection. This preprocessor also detects a denial of service (DoS) attack using fragmented packets at a high rate of speed.

There is a patch to this preprocessor that detects the Rose Attack. The patch file and instructions are found later in this chapter.

stream4

This preprocessor reassembles TCP packets and inspects them to detect attempted IDS evasion attacks from tools such as snot or stick using stateless attacks. This preprocessor also detects port scans, state problems with a session, and records session information.

stream4_reassemble

This is the second part of the stream4 engine. It reassembles packets into meaningful sessions for the Snort rules engine and for the preprocessors loaded after Snort. It can also specify reassembly of the client side, server side, or both sides of a connection over all ports or a select set of ports.

Http_inspect

This is a new preprocessor that handles all HTTP traffic to help speed it through to the rules engine. This preprocessor serves several purposes such as: HTTP traffic normalization; HTTP traffic profiling and normalization, possibly for each web server in your organization; and the ability to detect proxy usage.

rpc_decode

This is actually only an application decoder. It listens for RPC protocol packets on certain ports, and then decodes the traffic on those ports to ASCII to be passed back to the Snort rules engine for comparison.

telnet_decode

This is also an application decoder. It decodes all traffic on several ports, including 23/tcp, and then passes it back to the Snort engine.

bo_decode

This preprocessor detects when the popular Trojan horse program Back Orifice is in use on your network. This highly popular Trojan has its own protocol that Snort is able to quickly detect and pass on to the rules engine for detailed inspection to determine the commands in use. Subseven and several other Trojan tools have surpassed this Trojan. Depending on the network you are on, you might not want to run this preprocessor.

Flow-portscan

This is the only preprocessor that has to have the flow preprocessor enabled to work. It takes flow data and finds the port scans in that data.

Arpspoof

This preprocessor is fed a list of IP:MAC addresses. When it detects a layer-2 attack, it triggers an alarm for a layer-2 event, such as multiple MAC addresses from a single IP.

Perfmonitor

This is a new preprocessor that generates statistical information on the load Snort is under, sensor load, and several network performance measurements.

4.1. Detecting Stateless Attacks and Stream Reassembly

Problem

I have read about the Snort DoS, stateless-attack tools snot, and stick. How can I protect my sensor from this type of attack?

Solution

There are several options available to help defeat stateless-attack tools. Among these are two parts to the stream4 preprocessor: stream4 and stream4_reassemble.

The first is a simple command-line option for Snort: -z. This option forces Snort to alert only on streams that have established a full three-way handshake or that have shown some data in transit. This effectively blocks all the stick/snot/sneeze stateless attacks.

Snort -my -other -options -z

Stream4

Another option is to use the snort.conf file to tweak the stream4 preprocessor to be more effective on your network. Following are some examples of the types of attacks and traffic that the stream4 preprocessor can detect.

If you want stream4 to detect scans that are not full connection scans such as SYN, FIN/SYN, and other TCP-based scans, use the code in Example 4-1.

Example 4-1. Use stream4 to detect stealth scanning activity

"preprocessor stream4: detect_scans"

If you’re trying to detect problems with a connection, such as bad or out-of-order sequence numbers, use Example 4-2 .

Example 4-2. Use stream4 to detect IP state problems, such as IP overlapping

"preprocessor stream4: detect_state_problems"

The stream4 preprocessor is useful in detecting possible evasion attempts through the code in Example 4-3. Note that if you’re monitoring an asynchronous link or some high-speed networks, such as those used by some of the larger Internet Service Providers (ISP), detection is quite noisy. This is because asynchronous links see only part of a TCP connection—i.e., only the client side or the server side. If you are trying to use the evasion alarms, they will fire for just about every connection, as the link sees only, for example, a TCP ACK without the TCP SYN/ACK of a session.

Example 4-3. Don’t alarm on high noise levels from possible evasion attacks

"preprocessor stream4:disable_evasion_alerts"

Another use of the stream4 preprocessor is to determine the amount of data transferred in a connection. This can be extremely helpful in cases of some of the more advanced exploits to determine either data loss or hostile code uploads. For more information, see later in the chapter. Example 4-4 shows how much traffic was sent and received. This is stored in session.log in your Snort log directory. To enable the session logging, use Example 4-5. The information is kept in the flat session.log text files to ease scripted searches through the file(s) with the help of the code in Example 4-6, which tells Snort to log the information one line at a time instead of multiple lines. However, if you are using Barnyard or some other log-unification system, and no logging or alerting from the stream4 preprocessor is wanted, simply enable Example 4-6 to prevent the preprocessor from outputting any data. Finally, like all the preprocessors, you can combine the options to be more effective on your network(s), such as in Example 4-7.

Example 4-4. Provide session information about your connections

"preprocessor stream4:keepstat"

Example 4-5. Format the stats information into single line entries

"preprocessor stream4:keepstat machine"

Example 4-6 makes searching through with tools like grep or custom scripts easier. Binary logging is used for binary installs. Baynard takes the steam4 logs in binary mode to increase speed.

Example 4-6. Format the stats information into binary entries

"preprocessor stream4: keepstats binary"

The following code affects only alarms from the preprocessor and not the signature engine. It should be invoked only while testing a preprocessor or some other part of the Snort engine(s).

"preprocessor stream4:noinspect"

Example 4-7 combines Example 4-4 through Example 4-6, which, depending on sensor placement and load, will be more effective for your network(s).

Example 4-7. Combine the options to be more effective for your network

"preprocessor stream4:disable_evasion_alerts,detect_scans,keepstats machine"

Stream4_reassemble

This part of the preprocessor determines how much of a session to reassemble for analysis. Depending on your unique requirements, you may want to enable/disable some of these options. Things such as network location, speed, and load of the sensor all should be considered when enabling these options. In Example 4-8, all alerts from the reassemble preprocessor have been disabled. This configuration might be found enabled on either highly loaded perimeter sensors or in testing environments where filtering of event data is used to test other portions of the Snort engine.

Example 4-8. Turn off all alerts from this preprocessor component

    "preprocessor stream4_reassemble:noalert"

If you want to tax your sensor(s), try enabling full session reassembly on both the client and server sides of connections over the common ports, as in Example 4-9. The common or “default” ports used for the reassembly preprocessor are: 21/tcp, 23/tcp, 25/tcp, 53/tcp, 80/tcp, 143/tcp, 110/tcp, 111/tcp, 513/tcp, and 1433/tcp. However, for most events, the default configuration will reassemble client-side only over the “default” ports.

Example 4-9. Reassemble client and server sided events on common ports

     "preprocessor stream4_reassemble:both,ports default"

If you’re running common applications on nonstandard ports, the ports option might be helpful for assembling attacks against your applications. For example, you might want to change the ports option to reflect your NAT (network address translation) or PAT (port address translation) port ranges for those common applications. The ports option is a comma-separated list for your applications. The reassembly preprocessor can handle applications over these nonstandard ports with a simple snort.conf option, as in Example 4-10. Reassemble client side-only sessions for specific ports. The ports option will reassemble the port you provide within a bracket list. The preprocessor doesn’t care what the application(s) running on the port(s) are being used for. However, it will still create the pseudopackets for the stream to hand back to the Snort rules engine for analysis.

Example 4-10. Reassemble client side-only sessions for specific ports

    "preprocessor stream4_reassemble:clientonly,ports [2121,27,25,53,8080,1443]"

All the options in Example 4-10. can be combined for more effectiveness on your networks; for example, on a RAS or VPN sensor, you might want to monitor all ports and both sides of connections, as in the Example 4-11.

Example 4-11. Combining the options in Example 4-10. Reassemble client side-only sessions for specific ports to be more effective for your network(s)

  "preprocessor stream4_reassemble: both, ports all"

However, note that as you’re now attempting to put together sessions from encrypted traffic, you won’t be able to determine any session information. The only reason to capture ports such as 443/tcp (HTTPS) or 22/tcp (ssh) is to use the session information to take an educated guess on the amount of data transferred. For example, if you have an SSH session that has 2 GB of packet data, there is a good chance that you might have a problem with exfiltration of data.

Discussion

For the first part of the stream4 preprocessor, stream4, we might want to adjust a couple of options for our network. All options are comma-delimited values and can be combined to be more effective.

In Example 4-1, we used the detect_scans keyword. This option allows Snort to alert on several types of stealth scans, such as those used by Nmap, to try to hide from other detection tools. This option is disabled by default in the snort.conf file.

In Example 4-2, we used the detect_state_problems keyword. This option allows us to trigger lots of alarms for events such as data sent in a SYN flagged TCP packet with window and ACK numbers out of sequence. Be very careful using this option, as on a core network, it can cause a flood of alarms due to poorly written IP stack implementations. This option is disabled by default in the snort.conf file.

In Example 4-3, we used the disable_evasion_alerts keyword. This option disables alerts on “possible” IDS evasion packets, such as IP overlapping or TCP RST flooding. This option is enabled by default to help cut down on the noise from a new Snort implementation.

In Example 4-4, we used a very useful keyword, keepstats. This keyword has two subkeywords that may be useful to an IDS team. This option takes the data passing through the stream4 preprocessor and creates a log of that information in the file session.log. This file is automatically created and placed in your Snort log directory. This log contains information to help determine if a file was transferred during an exploit by showing the size of the connection. For example, the following is a copy out of a session.log file and the data available to search on:

[*] Session => Start: 08/24/04-10:35:57 End Time: 08/24/04-10:36:22
[Server IP: 10.0.4.45 port: 21  pkts: 14  bytes: 3339] [Client IP: 
10.0.4.2 port: 2147 pkts: 13 bytes: 112]

For example, in this session log, we can determine the amount of data transferred and by whom as well as the time and duration.

This connection was pretty much just banners; check maybe an attempted and failed login. Not enough to actually push out too much. As well, this connection had the server sending out only 14 packets and the client in the connection sent only 13. So this is pretty much only enough for the banner and possibly the login prompt of an FTP connection. In Example 4-5 and Example 4-6, we used the subkeywords machine and binary. The machine keyword causes the stream4 preprocessor to output each session onto a single line instead of multiple lines. This will make sorting and gathering data out of the session.log file much easier.

The binary keyword causes the stream4 preprocessor to output in the machine-readable unified format. This can then be read by something like Barnyard for detailed postprocessing of the data.

In Example 4-6, we used the keyword noinspect. This option would be used if, for example, you weren’t getting any useful information back from the stream4 preprocessor or wanted to temporarily disable it.

In Example 4-7, we demonstrated combining several options to be more effective on our network. In our example, we turned off the noise evasion alarms while enabling detection of stealth scans. Finally, we also turned on session logging, writing to a new file session.log, formatted with each new entry as a single line. Using this example, we have new alarms to show to the analysts as well as a record of the size and duration of each connection.

Lastly, the min_ttl, ttl_limit, and log_flushed_streams keywords should almost never be adjusted. If you would like to learn about them, check out the Snort documentation that comes with the source code snort_manual.pdf in the doc subdirectory.

stream4_reassemble

This component takes packets and reassembles them into server-side, client-side, or both-sided connections. Snort’s default configuration reassembles client-sided connections on only a short list of ports common applications.

Example 4-8 enabled the noalert keyword to prevent triggers of an event on either client-side or server-side evasion and insertion attacks. This option should be disabled only during testing, or if you are using a nonregenerative tap for your IDS sensor.

Example 4-9 enabled the stream4_reassemble preprocessor to reassemble and find alarms for both client- and server-sided connections over the default ports. Those ports are 21, 23, 25, 53, 80, 110, 111, 143, 513, and 1433, which apply for both TCP and UDP ports.

Example 4-10 reassembled client side-only sessions for specific ports demonstrated a way to specify new ports to use. This could be helpful when running common applications on nonstandard ports, such as using a proxy for all network traffic. The ports option is a comma-separated list for your applications.

Finally, Example 4-11 combined the options to increase effectiveness, albeit a slight hit on your Snort sensor’s performance. This might be an effective solution a slower or less-used connection, such as on a RAS or VPN sensor where you might want to monitor all ports and both sides of connections for clarity.

4.2. Detecting Fragmentation Attacks and Fragment Reassembly with Frag2

Problem

How can Snort help me detect attacks that try to use small fragmented packet streams or fragmented network scans to try and get through my weak ACLs?

Solution

Use the frag2 preprocessor to help detect fragmentation attacks. These are DoS attacks from tools like Teardrop or Jolt to a network probe using hping2 or fragroute. The frag2 preprocessor isn’t very useful for a more in-depth analysis, but here are a couple examples you might find useful. In Example 4-13, the time that packets are kept in the preprocessor has been shortened, as well as the memory allocated to this preprocessor. This might be used on a high-speed sensor, such as at a perimeter sensor where fragmented attacks such a denial of service (Dos) might happen. Another example of this type of configuration is at a core or network aggregation point, such as a speed throttling or proxy point, as shown in Example 4-12.

Example 4-12. Tweak the time limit and memory size for a core or high-traffic sensor

# Default timeout 60 seconds and memory buffer of 4MB
Preprocessor frag2: timeout 15, memcap 2097152

Example 4-13 would be good for sensors deep inside a network core that only should have certain packets coming and going through it.

Example 4-13. Tweak the TTL limit to alarm if packets are outside a set range

 Preprocessor frag2: min_ttl 5, ttl_limit 15

The preprocessor sets the default minimum TTL to 0 to help detect even local network attacks. This also sets the default highest count on the TTL that it will count as 55 hops away. If you are monitoring a specific network segment that should be accepting packets only from certain route points, you can use this limit to automatically detect when packets are not coming through your specific route points.

Preprocessor frag2:ttl_limit 10, min_ttl 5

Example 4-14 will help us detect use of probing tools, such as fragroute and hping2. However, as this can be a very noisy alarm on most high-speed or asynchronous networks, this is disabled by default in the Snort configuration.

Example 4-14. Turn on detect_state_problems

preprocessor frag2: detect_state_problems

Discussion

Attack tools such as Teardrop, Jolt, and fragroute all had one similarity: they all used some form of fragmentation or irregular packet lengths to successfully exploit and/or identify their targets. The frag2 preprocessor detects this type of attack by analyzing the fragmented packets in terms of TTL, time, and even duration of the flow. However, another form of attack that can slip by border firewalls is a fragmented network scan. This sends fragmented packets that are either smaller than usual or otherwise out of spec to gain entry past a border firewall and get responses back from internal hosts. Hping2 is one tool that can launch this type of attack. However, for simplicity, the following example uses the Teardrop exploit itself.

      This is what the example attacker might launch 
     "./teardrop_frag.exec 10.0.4.100 10.0.4.2 -s 4321 -t 80 -n 80"

With the frag2 preprocessor enabled, Snort would detect the attack and send out the following alarm.

"[**] [113:2:1] (spp_frag2) Teardrop attack [**]
08/16-01:19:44.445492 10.0.4.100 -> 10.0.4.2
UDP TTL:64 TOS:0x0 ID:242 IpLen:20 DgmLen:24
Frag Offset: 0x0003   Frag Size: 0x0004"

These tools are widely available and actively in use “in the wild.” Having seen several of the tools, what they can do to bypass a Cisco router ACL with a simple RST scan is pretty scary. However, with this preprocessor and several other signatures to help identify this type of attack, you can protect your network.

The rose attack is one method of attacking an entire range of network devices, from workstations to routers and switches. This attack would come in the form of a two-packet attack, so the response time is severely limited. The attacker can also modify the original code to make detection much harder. However, there is a small patch to the frag2 preprocessor that enables the preprocessor to detect this type of attack. The following patch file will enable you to add an option to the frag2 preprocessor in your snort.conf file causing several alarms for rose like attacks. (Special thanks to Marty Roesch, who developed this patch, along with helping to bring it up to Snort 2.2.x version support.) To install this patch, simply copy the following code into a file, and then follow the instructions.

# The  rose_attack_detection.patch   
----------START OF PATCH -----------------------
diff -ur snort-2.2.0/src/generators.h snort-2.2.0.rose/src/generators.h
--- snort-2.2.0/src/generators.h        Mon Oct 20 11:03:19 2003
+++ snort-2.2.0.rose/src/generators.h   Fri Apr  9 21:54:26 2004
@@ -109,6 +109,7 @@
 #define     FRAG2_IPOPTIONS                       8
 #define     FRAG2_EMERGENCY                       9
 #define     FRAG2_SUSPEND                         10
+#define     FRAG2_ROSE_ATTACK                     11
   
 #define GENERATOR_SPP_FNORD         114
 #define     FNORD_NOPSLED                         1
@@ -240,6 +241,7 @@
 #define FRAG2_TTL_EVASION_STR "(spp_frag2) TTL Limit Exceeded (reassemble) 
detection"
 #define FRAG2_EMERGENCY_STR "(spp_frag2) Shifting to Emergency Session Mode"
 #define FRAG2_SUSPEND_STR "(spp_frag2) Shifting to Suspend Mode"
+#define FRAG2_ROSE_STR "(spp_frag2) Interfragment gap threshold exceeded, 
possible Rose attack"
   
   
diff -ur snort-2.2.0/src/preprocessors/spp_frag2.c snort-2.2.0.rose/src/
preprocessors/spp_frag2.c
--- snort-2.2.0/src/preprocessors/spp_frag2.c   Mon Oct 20 11:03:37 2003
+++ snort-2.2.0.rose/src/preprocessors/spp_frag2.c      Fri Apr  9 22:58:18 2004
@@ -134,6 +134,8 @@
   
     char state_protection;
   
+    int gap_threshold;  /* alerting threshold for max gap (rose attack) */
+
     SPMemControl frag_sp_data; /* self preservation data */
 } Frag2Data;
   
@@ -175,6 +177,7 @@
     u_int8_t complete;
     u_int8_t teardrop;
     u_int8_t outoforder;
+    int max_gap;
 } CompletionData;
   
 typedef struct _F2Emergency
@@ -343,8 +346,14 @@
     else if(frag->offset > next_offset)
     {
         DEBUG_WRAP(DebugMessage(DEBUG_FRAG2, "Holes in completion check... (%u > %u)\n",
-                                frag->offset, next_offset););
+                    frag->offset, next_offset););
         comp->complete = 0;
+
+        if(comp->max_gap < (frag->offset - next_offset))
+        {
+            comp->max_gap  = frag->offset - next_offset;
+            printf("recomputing maxgap!  size: %d\n", comp->max_gap);
+        }
     }
   
     return;
@@ -468,6 +477,7 @@
         f2data.frag_sp_data.mem_usage = 0;
         f2data.frag_sp_data.fault_count = 0;
         f2data.frag_sp_data.sp_func = Frag2SelfPreserve;
+        f2data.gap_threshold = 0;
   
         if(!pv.quiet_flag)
         {
@@ -483,6 +493,7 @@
             LogMessage("    Self preservation period: %d\n", f2data.sp_period);
             LogMessage("    Suspend threshold: %d\n", f2data.suspend_threshold);
             LogMessage("    Suspend period: %d\n", f2data.suspend_period);
+            LogMessage("    Max frag gap threshold: %d\n", f2data.gap_threshold);
   
         }
   
@@ -647,7 +658,13 @@
             {
                 f2data.state_protection = 1;
             }
-
+            else if(!strcasecmp(stoks[0], "gap_threshold"))
+            {
+                if(isdigit((int)stoks[1][0]))
+                {
+                    f2data.gap_threshold = atoi(stoks[1]);
+                }
+            }
   
             mSplitFree(&stoks, s_toks);
   
@@ -674,6 +691,7 @@
         LogMessage("    Self preservation period: %d\n", f2data.sp_period);
         LogMessage("    Suspend threshold: %d\n", f2data.suspend_threshold);
         LogMessage("    Suspend period: %d\n", f2data.suspend_period);
+        LogMessage("    Max frag gap threshold: %d\n", f2data.gap_threshold);
   
     }
 }
@@ -876,6 +894,7 @@
         compdata.complete = 0;
         compdata.teardrop = 0;
         compdata.outoforder = 0;
+        compdata.max_gap = 0;
   
         if(FragIsComplete(ft, &compdata))
         {
@@ -903,7 +922,23 @@
             }
   
             RebuildFrag(ft, p);
-        } else {
+        }
+        else
+        {
+            if(((ft->frag_flags & (FRAG_GOT_FIRST|FRAG_GOT_LAST)) =  =
+                        (FRAG_GOT_FIRST|FRAG_GOT_LAST)) &&
+                    (f2data.gap_threshold != 0) &&
+                    (compdata.max_gap > 0) &&
+                    (compdata.max_gap > f2data.gap_threshold))
+            {
+                SetEvent(&event, GENERATOR_SPP_FRAG2,
+                        FRAG2_ROSE_ATTACK, 1, 0, 5, 0);
+                CallAlertFuncs(p, FRAG2_ROSE_STR, NULL, &event);
+                CallLogFuncs(p, FRAG2_ROSE_STR, NULL, &event);
+                ft->alerted = 1;
+                DisableDetect(p);
+            }
+
             DEBUG_WRAP(DebugMessage(DEBUG_FRAG2, "Fragment not complete\n"););
         }
     }

If you would like to enable snort-2.2.x to use this patch, you are only going to be able to:

  • Compile Snort from source code

  • Use it on a Linux or BSD sensor

Next, to build this patch, simply follow the instructions. Create a directory for the patch file and the Snort source code.

Root# pwd 
/opt/ROSE
Root# ls 
Rose_attack.patch snort-2.2.x.tar.gz

Extract the Snort code and, using the patch command, apply the rose attack file.

Root# tar xvfz snort-2.2.x.tar.gz 
Root# patch -p < Rose_attack.patch

If you get no errors, simply compile Snort as you would normally.

Finally, add the following extra options to your snort.conf file:

# Snort.conf file example 
Preprocessor frag2: gap_threshold 32768

These options are the default number of bytes that are in the proof-of-concept code outlining the use of the rose attack. You can change or adjust the gap_threshold value as you want, however, as this attack has yet to be seen in the wild. There is little need to adjust this setting.

See Also

The author of the rose attack proof-of-concept code and the creator of the attack, available at http://digital.net/~gandalf/Rose_Frag_Attack_Explained.htm

Snort-devel mailing list

4.3. Detecting and Normalizing HTTP Traffic

Problem

With all the available HTTP evasion attacks, how can Snort help detect these types of attacks? Can I create unique profiles for each of my web servers? Can I detect HTTP proxy usage?

Solution

As of Snort 2.0, there is a preprocessor to handle all the HTTP traffic coming through the Snort engine. This preprocessor has grown in flexibility and features and now has two parts: a global and a server section. As there are four variables for the global section and three server default server profiles, you have 24 specific alarms and variations from which to choose.

Global examples

These options are set for all the server(s) you create.

Snort has a default language interpreter of English. This means Snort can translate Unicode characters to English for ASCII comparison in rules. However, for organizations that need to have other language support, there is a tool in the Snort source code distribution, called ms-unicode-generator.c, that needs to be compiled to run. Once compiled, it will build a new Unicode map for Snort to use from a new language file.

preprocessor http_inspect: global iis_unicode_map unicode.map 1252

The following example turns on the proxy detection for our server to use. If this is not enabled here, proxy through your web server will not be detected.

Preprocessor http_inspect: global iis_unicode_map unicode.map \ 
1252 proxy_alert

The following example detects new web servers coming online. Be warned that this works through stateless detection, which makes it highly unusable on your core network. The reason for the flood of alarms is that the detection method at this point in Snort is stateless. This means that every packet that has your HTTP_PORTS variable port is going to be checked as a new web server! On even a medium-sized network, that means that the moment you turn on this detection, every workstation shows up as a new web server! The reason for this “flaw” is that the current implementation of the web server detection code is stateless. This means that the preprocessor doesn’t differentiate between who started the HTTP connection; it will determine that whoever gets port 80/tcp packets must be a web server!

Preprocessor http_inspect: global iis_unicode_map unicode.map 1252 
detect_anomalous_servers

Server examples

The server portion of the http_inspect engine gets very granular and specific for web servers you want to monitor.

The first option is the default server config. This will apply to all HTTP servers not specifically named in a server <IP> configuration. The following example is the default build from the snort.conf file. It uses the server profile all and listens for HTTP on ports 80, 8080, and 8180 TCP. It alerts on all the events turned on by the profile all option. It also detects when URL directories are larger than 500 characters long.

preprocessor http_inspect_server: server default \
    profile all ports { 80 8080 8180 } oversize_dir_length 500

The following solution might be good for your IIS web servers. Please note that all uncommented options for a profiled server are actively used.

Preprocessor http_inspect_server: server <IP_of_IIS_server> profile iis \
ports {80 8080 }  \   # The ports to filter for HTTP traffic to/from this server
flow_depth 200 \       # How many bytes to down down into the server response
inspect_uri_only       # Performance improvement to only look at the url field 
#
# oversize_dir_length <number> 
## number of characters outside of the web root that this will trigger an alarm on. 
# iis_unicode_map <unicode file> <number in the file to use> 
## can be specified for other language servers within your organization 
# allow_proxy_use 
## turn off alarms for HTTP proxying through the server 
# no_alerts  
## disables all http_inspect alarms for this server

As you might have noticed, when you use a server profile, you lose most of the flexibility to enable and disable http_inspect alarms. Mimicking all the options in the profile iis while allowing the flexibility to change would look like this:

Preprocessor http_inspect_server: server <IP_of_IIS_server> \
ports {80 8080} \
flow_depth 300 \
ascii no \
multi_slash no
directory no \
webroot
double_decode yes \ 
u_encode yes \ 
bare_byte yes \ 
iis_unicode yes \
iis_backslash no \
iis_delimiter no \
apache_whitespace no \ 
non_strict

Creating the same server configuration for the Apache profile is smaller, as it has less application data to decode and normalize under normal conditions:

# Remember the yes/no option only turns on or off alerting
Preprocessor http_inspect_server: server <IP_of_apache_server> \
ports {80 8008}  \
flow_depth 300 \
non_strict \
chunk_length 500000 \
acsii no \
multi_slash no \ 
directory no \ 
apache_whitespace yes \
webroot \
utf_8 no

However, if you want to just place your server’s IPs and ports in use with no other options, this is what you will get:

Preprocessor http_inspect_server: server <IP_of_your_server> \
ports {80 8080 }  \ # If not specified defaults to only port 80 
flow_depth 300 \
chunk_length 500000 \
ascii no \
utf_8 no \
multi_slash no \
directory no \
webroot \
apache_whitespace no \
iis_delimiter no \
non_strict

Discussion

The http_inspect preprocessor breaks down into two parts: global and server. The global portion enables some of the server options, such as proxy detection for each server and Unicode-to-English mappings. The Unicode mapping just requires a Unicode map file, and then a number to the proper mapping for your language. For most situations, the unicode.map file found in the etc directory of the Snort installation will serve most users. However, should you need to create your own Unicode mapping, there is a file called ms_unicode_generator.c that is found with the Snort source code in the contrib directory. If you are on a non-English version of Windows, you can compile this tool that will create your map file for that specific language.

The proxy_alert keyword allows defined web servers to alarm when they’re being used in an HTTP proxy. This can be helpful in determining when users are bypassing a defined proxy server. For example, turn on the global variable proxy_alert, and then enable by proxy server a defined host with proxy traffic allowed through it. If you leave the default server to alert on proxy HTTP traffic, this tells you when users are using an unauthorized proxy server.

# Global 
Preprocessor http_inspect: global iis_unicode_map unicode.map 1252 proxy_alert 
# Proxy Server
preprocessor http_inspect_server: server <IP_of_proxy_server> \
ports {80 3128 }  \
allow_proxy_use  
# Everything else 
preprocessor http_inspect_server: server default \
ports { 80 }  \
profile all

The server portion of the http_inspect preprocessor can be tailored to most of the common web server configurations. As shown in the previous code, this preprocessor can handle out of the box some of the idiosyncrasies of the Microsoft IIS web server. For example, it handles the successful translation of the Unicode characters into normalized data. Table 4-2 should help show all the Unicode characteristics the preprocessor can handle.

Table 4-2. Http_inspect Unicode keywords and their meanings

Keyword

Detection

ascii

This keyword decodes ASCII characters, such as “ping.” It can help in detecting unencoded attacks such as a default directory traversal attack.

iis_unicode

This keyword decodes Microsoft IIS Unicoded characters such as “%2fscripts.” It uses theunicode.map file for proper unicode to ascii translations..

double_decode

This handles the fact that IIS runs two passes through each URI request: the first for encoding (Unicode, UTF), and the second for plain text.

bare_byte

This handles the use of non-ASCII characters to decode UTF-8 characters. There are no known legitimate uses of this encoding. However, it does seem to be popular with the way certain IIS served web sites communicate with IE browsers, so be aware of a high false-positive hit count.

Finally, along with the server-side features is the normalization of any HTTP traffic that passes the preprocessor. For example, with the multi_slash keyword, any HTTP traffic that comes in looks flaky, such as this URL:

GET  /etc///////////passwd HTTP/1.0

This would get normalized back to the rules engine as:

GET /etc/passwd HTTP/1.0

Then correctly trigger the following rule:

alert tcp $EXTERNAL_NET any -> $HTTP_SERVERS $HTTP_PORTS (msg:"WEB-MISC 
/etc/passwd"; flow:to_server,established; content:"/etc/passwd"; nocase; 
classtype:attempted-recon; sid:1122; rev:5;)

Without the help of the http_inspect preprocessor, that attack would have probably gone right past Snort without triggering an alarm.

These attacks aren’t limited to IIS servers. Apache servers benefit from normalization as well. For example, a while ago there was a “chunked-encoding” exploit in Apache that could be detected by the HTTP preprocessor. It didn’t detect the exploit based on the content of the packet—as this string was used by valid applications—but rather by the size of the request. In the previous solutions, we see our use of the chunk_length keyword. In the previous example, we filtered out normal-sized chunks of data. Chunk is a base size of an HTTP session payload that the preprocessor will examine for any given HTTP session. When we set it to 500,000 bytes long, this will successfully detect the buffer overflow portion of the chunked encoding exploit. Another example of Apache normalization would be the use of the apache_whitespace keyword to help normalize the use of a tab versus a space keyboard key to be handled as a space within a URL string.

Finally, if you have http_inspect normalization enabled, certain signature rules are never going to alert. This is because in the Snort rules language, there are two keywords that deal with payload data: content and uricontent. The content keyword looks through the raw data being handed back to the Snort rules engine. The uricontent keyword handles only the normalized data headed back after the http_inspect preprocessor handles it. From the Snort documentation, see this example:

For example, the URI:
/scripts/..%c0%af../winnt/system32/cmd.exe?/c+ver
will get normalized into:
/winnt/system32/cmd.exe?/c+ver

This example illustrates what the http_inspect preprocessor normalizes in an HTTP session. For example, the attacker is going to use the actual directory traversal attack, so you write a rule to detect that part of the attack /..%. However, when you wrote your rule, you used the uricontent keyword to look only within the HTTP Get statements, not realizing that uricontent strips off the Unicode characters from an HTTP session. This act makes your rule useless, as it will never trigger. When creating your HTTP rules, be aware of when you might be stepping on the functions of one of the preprocessors.

See Also

Snort-devel mailing list

Beale, Jay. Snort 2.1 Intrusion Detection. Rockland, MA: Syngress, 2004.

Snort documentation (http://www.snort.org)

4.4. Decoding Application Traffic

Problem

How do I decode and normalize application traffic such as Telnet or rpc-encoded traffic so that the rules engine can analyze it?

Solution

Using the rpc_decode and telnet_decode decoders, Snort can decode the plain-text content inside of these applications.

The Telnet decoder has no options for you to set. This decoder takes the application encoding on Telnet (port 23/tcp), FTP (port 21/tcp), SMTP (25/tcp), and NNTP (119/tcp) connections then remove the application data and sends what’s left back to the Snort engine for rules comparison of the plain-text content.

# enable this line in your snort.conf file
Preprocessor telnet_decode

The rpc_decode decoder has more options and some ability to change its alerts.

You can select on which ports it should decode rpc traffic. For example, if you want to enable rpc_decode in your snort.conf file to decode only Unix SUNRPC traffic ports, use the following line:

preprocessor rpc_decode: 111 32771

If you also want to normalize Microsoft rpc traffic on port 135/tcp and detect when an rpc request is fragmented, add the extra port and the alert_fragmentation option. However, this combination can generate loads of traffic.

preprocessor rpc_decode: 111 32771 135 alert_fragmentation

A smarter solution would be to filter our rpc events. The following example filters out when more than one rpc message or query is in a single rpc packet stream.

Preprocessor rpc_decode: 111 32771 no_alert_multiple_requests

The no_alert_large_fragments option tells rpc_decode not to alert on fragmented queries when they are larger than a single packet.

Preprocessor rpc_decode: 111 32771 no_alert_large_fragments

Finally, the no_alert_incomplete option tells rpc_decode not to alert on rpc messages that span more than one stream. This can be helpful for handling large rpc queries.

Preprocessor rpc_decode: 111 32771 no_alert_incomplete

Discussion

Snort has the ability to decode and normalize certain application traffic. This plain-text data can then be passed back to Snort so the rules engine can analyze it. The Telnet and rpc preprocessors are not full-blown preprocessors, but just protocol decoders. There is very little to note other than the Telnet and rpc preprocessors are of the few components of the Snort engine that actually rewrite the packets into pseudopackets. This is actually true even in the new 2.2.0 version of Snort. As the packets are handed in raw form from the stream4_reassembly preprocessor, they are normalized and stripped of their actual payload. Then they are reassembled into plain-text representations of the actual payload data stream.

For further discussion of this issue—or nonissue, depending on your network environment—check out the snort-devel mailing list archives of this building process.

See Also

Snort-devel mailing list

Beale, Jay. Snort 2.1 Intrusion Detection. Rockland, MA: Syngress, 2004.

4.5. Detecting Port Scans and Talkative Hosts

Problem

How do I detect when hosts on my network(s) are performing port scans and host scans?

Solution

There are actually a couple of answers to that question. This is because Snort developers have gone through several iterations of port scan detectors. The most common is the portscan preprocessor, while the newest is the flow-portscan preprocessor. Finally, portscan2 was supposed to address some of the problems with the portscan preprocessor, such as detection of SYN floods as port scans instead of DoS attacks. All these preprocessors are still compiled into Snort by default, even as late as Version 2.2.0. However, the trend is toward the flow-portscan preprocessor, as this is the first preprocessor to use the flow engine for its data. This section gives some example configurations for all three. The most effort is on the flow-portscan preprocessor, as the other two are no longer part of the default snort.conf file.

Portscan

This is the oldest and most commonly used of the three preprocessors. However, if you are using ACID (Chapter 5), you might want to pull some port scan information into ACID with little changes. To enable this in your snort.conf file, simply enter this example into the file right below the flow preprocessor.

Preprocessor flow: stats_interval  0 hash 2 
# 
# Legacy Support - Porscan Preprocessor from snort 1.x 
preprocessor portscan: $HOME_NET 4 3 /path/to/logs/portscan.log

When enabled, this preprocessor detects when a source host other than the one in the HOME_NET variable starts more that four port connections within three seconds. When that happens, two events are written: one in the Snort alert file, and the other in the portscan.log file. The alert file notifies the analysts of a possible port scan against one of their resources.

[**] [100:2:1] spp_portscan: portscan status from 10.0.4.100: 1150 
connections across 1 hosts: TCP(1150), UDP(0) [**]

The portscan.log file displays the ports targeted and their respective source port(s), as in the next example:

# quick display of an nmap scan (nmap -sT -F 10.0.4.45)
Aug 29 03:05:48 10.0.4.100:4530 -> 10.0.4.45:9535 SYN ******S*
Aug 29 03:05:48 10.0.4.100:4531 -> 10.0.4.45:1347 SYN ******S*
Aug 29 03:05:48 10.0.4.100:4532 -> 10.0.4.45:9992 SYN ******S*
Aug 29 03:05:48 10.0.4.100:4533 -> 10.0.4.45:8009 SYN ******S*
Aug 29 03:05:48 10.0.4.100:4534 -> 10.0.4.45:583 SYN ******S*
Aug 29 03:05:48 10.0.4.100:4535 -> 10.0.4.45:5713 SYN ******S*
Aug 29 03:05:48 10.0.4.100:4536 -> 10.0.4.45:2043 SYN ******S*
Aug 29 03:05:48 10.0.4.100:4537 -> 10.0.4.45:12345 SYN ******S*
Aug 29 03:05:48 10.0.4.100:4538 -> 10.0.4.45:6141 SYN ******S*
Aug 29 03:05:48 10.0.4.100:4539 -> 10.0.4.45:518 SYN ******S*

One concern of this preprocessor was how to blanket ignore hosts such as your DNS servers that often appeared as portscan attackers. The solution came in the form of another component of the portscan preprocessor: portscan-ignorehosts. This component simply tells the portscan preprocessor to not alert on any traffic from the host(s) and/or network(s) in a given list. An example of that is as follows; more than one entry into this list is space separated.

# Goes in snort.conf file below "preprocessor portscan" line 
Preprocessor portscan-ignorehosts: 10.0.4.1 10.0.4.105

This example filters out any port scans coming from either the DNS or web server.

Portscan2

As we mentioned, the portscan preprocessor had some limitations that another group of Snort developers tried to remedy with a rewrite and some added functionality. The portscan2 preprocessor relies on the old conversation tracking preprocessor and can’t be enabled when the flow preprocessor is active. Following is an example of a typical conversation and portscan2 configuration.

# First Disable the flow preprocessor 
# preprocessor flow: stats_interval 0 hash 2
# 
# Enable the conversation preprocessor 
preprocessor conversation: allowed_ip_protocols all, timeout 60, \
max_conversations 50000 
# the arguments are:
# allowed IP protocols, either a list of protocol numbers or word "all"
# timeout (seconds) before connections or conversations are rolled 
# out of the preprocessor 
#the max number of conversations that the preprocessor should see 
   
# Enable the portscan2 preprocessor 
preprocessor portscan2: scanners_max 256, targets_max 256, \
target_limit 3, port_limit 10, timeout 60
# arguments are:
# the max number of scanning hosts to support at once
# the max number of target hosts to support at once
# the number of hosts a scanner must touch before a scan is triggered
# number of ports a scanner must touch before a scan is triggered
# the timeout period (seconds) before a scanners activity is rolled
# out of the preprocessor

When this is enabled, a scan would look like this in your alert file:

[**] [117:1:1] (spp_portscan2) Portscan detected from 10.0.4.100: 
1 targets 11 ports in 0 seconds [**]
08/xx-13:27:32.464097 10.0.4.100:3537 -> 10.0.4.1:5232
Tcp TTL:64 TOS:0x0 ID:11424 IpLen:20 DgmLen:60 DF
******S* Seq: 0xEA6B7F8E  Ack: 0x0  Win: 0x16D0  TcpLen: 40
TCP Options (5) => MSS: 1460 SackOK TS: 83186478 0 NOP WS: 0

The type of data logged into the default file scan.log in your Snort log directory is much more detailed:

08/xx-13:27:32.464097  TCP src: 10.0.4.100 dst: 10.0.4.1 sport: 3537 \
dport: 5232 tgts: 1 ports: 11 flags: ******S* event_id: 0
08/xx-13:27:32.464177  TCP src: 10.0.4.100 dst: 10.0.4.1 sport: 3538 \
dport: 5002 tgts: 1 ports: 12 flags: ******S* event_id: 7
08/xx-13:27:32.464256  TCP src: 10.0.4.100 dst: 10.0.4.1 sport: 3539 \
dport: 780 tgts: 1 ports: 13 flags: ******S* event_id: 7
08/xx-13:27:32.465642  TCP src: 10.0.4.100 dst: 10.0.4.1 sport: 3540 \
dport: 1484 tgts: 1 ports: 14 flags: ******S* event_id: 7
08/xx-13:27:32.465722  TCP src: 10.0.4.100 dst: 10.0.4.1 sport: 3541 \
dport: 2002 tgts: 1 ports: 15 flags: ******S* event_id: 7
08/xx-13:27:32.465802  TCP src: 10.0.4.100 dst: 10.0.4.1 sport: 3542 \
dport: 214 tgts: 1 ports: 16 flags: ******S* event_id: 7

From this logfile you can immediately determine several facts about this scan:

  • This is a TCP Syn scan, the ******S* is the snort flagging for Syn only packets.

  • The source port is going up and changing every connection, possibly from a tool such as nmap.

  • How many ports per hit our victim was taking from the “ports” tag.

Flow-portscan

This is the newest preprocessor to detect port scans. This preprocessor is the first to take advantage of the flow preprocessor data. While that is the case, this preprocessor remains one of the hardest preprocessors for people to configure and use.

# for a single IP cable/DSL connection detection config 
# The would be useful for a network that doesn't server many or any services to
# the outside world. 
#

Note that talkers are hosts that are active on your network such as your workstations for browsing the Web, file sharing, etc. Scanners are hosts that have started communicating with one of your hosts within the learning time of the host over a previously unused port.

                  preprocessor flow-portscan: \
# the IP space to use for our allowed/learned network(s)
server-watchnet [10.0.4.0/24] \
# The number of seconds to keep port information on your watchnet for example this
# will keep the ports in use on each host for a 1-minute interval before refreshing 
server-learning-time 60 \
# the number of requests a port on a host in the watchnet must see before 
# it's treated as a talker rather than a scanner 
server-scanner-limit 50  \
# If you have hosts or networks that you want to ignore and not 
# count into the learning time place here
# src-ignore-net [10.0.4.1/32] \
# If you have destination networks or hosts that you want to ignore
# such as your DNS server or POP mail server place here
# dst-ignore-net [10.0.4.1/32] \
# This sets how the alarms will be sent out. The default setting is 
# display the alerts "once" per scan. However, in this case we are going 
# to alarm every time the points go above the threshold.
alert-mode all \
# The tells the preprocessor to send the alarm out in a text message 
# mode as seen below. However, if you want there is "pktkludge" option
# that you can use as well to send to the snort logging system. 
output-mode msg \
# This turns on detection much like the stream4 preprocessor for invalid
# or odd tcp flows. Such as a SYN/FIN flagged flow. 
tcp-penalties on

These settings log something like the following in the alert file. This correctly identifies the nmap host—in this case, 10.0.4.100. However, you don’t see what ports it’s been probing or the targets.

[**] [121:2:1] Portscan detected from 10.0.4.100 Talker(fixed: 1 sliding: 1) 
Scanner(fixed: 10 sliding: 40) [**] 08/xx-14:29:14.676834
[**] [121:1:1] Portscan detected from 10.0.4.100 Talker(fixed: 1 sliding: 1) 
Scanner(fixed: 15 sliding: 5) [**] 08/xx-14:29:14.676904
[**] [121:1:1] Portscan detected from 10.0.4.100 Talker(fixed: 1 sliding: 1) 
Scanner(fixed: 15 sliding: 20) [**] 08/xx-14:29:14.677102

For monitoring a larger network, you might try the following configuration example:

                  preprocessor flow-portscan: \
# Network to monitor
server-watchnet [192.168.1.0/24,192.168.2.0/24] \
# Ignore traffic coming from the routers
#src-ignore-net [192.168.1.1/32,192.168.2.1/32] \
# Ignore traffic going for the DNS servers 
#dst-ignore-net [192.168.1.2/32,192.168.2.2/32] \
# the number of requests to a single port such as 80/tcp that a hosts
# in the watchnet must recieve before the port is ignored for portscans
#server-ignore-limit 200 \
# Time (seconds) to keep there watchnet servers ports before resetting
server-learning-time 3600 \
# the number of requests a port on a host in the watchnet must see before 
# it's treated as a talker rather than a scanner 
server-scanner-limit 50 \
# sets the alert mode to alarm on every event over the threshold
alert-mode all \
# Sends a text message to the alert file 
output-mode msg \
# alarm on odd flow tcp flag settings 
tcp-penalties on \
# Used for debugging to dump the contents of all of the flow-portscan
# 3 "tables" of data to the screen on snort exit. Set to 0 to disable.
Dumpall 1

Discussion

As you might have seen, the portscan preprocessor is still useful when detecting port scans from the flow preprocessor. This, combined with the fact that it’s one of the simplest preprocessors to set up, makes it a viable preprocessor, especially if you are using a Snort frontend such as ACID (Chapter 5).

However, the portscan2 preprocessor takes quite a bit of memory and requires disabling and reenabling preprocessors. The worst of these is disabling the flow preprocessor. This causes problems even with the Snort rules engine, as quite a few of the new rules use the flow keyword in their detection patterns. The other concern about this preprocessor is the requirement of the conversation preprocessor, which flow was built to replace. The conversation preprocessor didn’t handle state very well. However, one useful keyword that the conversation preprocessor had was alert_odd_protocols. The following conversation preprocessor configuration detects when protocols other than TCP, UDP, or ICMP are in use on your network.

Preprocessor conversation: preprocessor conversation: \
allowed_ip_protocols, 1 6 17,timeout 60, max_conversations 50000, \ 
alert_odd_protocols

Finally, with the new flow-portscan preprocessor, we used a small network and larger network configuration example that should at least get you started on detecting port scans on your network(s). However, the flow-portscan can be tweaked for your network. If you want data from the preprocessor’s output, you can apply a patch to Snort to get more data back.

You can change from a port scan log entry like this:

[**] [121:3:1] Portscan detected from 10.0.4.100 Talker(fixed: 15 sliding: 
15) Scanner(fixed: 0 sliding: 0) [**] 08/xx-15:38:21.619113

To a more detailed log like this:

[**] [121:3:1] (flow_ps) Portscan detected from 10.0.4.100 Talker(fixed: 
15 sliding: 15) Scanner(fixed: 0 sliding: 0) [**]
08/xx-16:10:08.174184 10.0.4.100:42027 -> 10.0.4.95:80
TCP TTL:41 TOS:0x0 ID:16080 IpLen:20 DgmLen:40
***A**** Seq: 0xFB200EDB  Ack: 0x8FFD88F7  Win: 0x800  TcpLen: 20

If you want to enable this type of logging, just follow these directions to patch and remake your Snort build.

# copy this code into your system  
# create file "flow-portscan_output.patch"
### START OF PATCH
diff -urN snort-2.2.0_orig/src/generators.h snort-2.2.0/src/generators.h
--- snort-2.2.0_orig/src/generators.h 2003-10-20 15:03:19.000000000 +0000
+++ snort-2.2.0/src/generators.h 2004-05-22 23:01:52.000000000 +0000
@@ -316,6 +316,7 @@
 #define DECODE_BAD_TRHMR_STR "(snort_decoder) WARNING: Bad Token Ring MR Header!"
   
   
+#define FLOWPS_PREFIX_STR "(flow_ps) Portscan detected from "
 #define SCAN2_PREFIX_STR "(spp_portscan2) Portscan detected from "
   
 #define CONV_BAD_IP_PROTOCOL_STR "(spp_conversation) Bad IP protocol!"
diff -urN snort-2.2.0_orig/src/preprocessors/flow/portscan/flowps_snort.c 
snort-2.2.0/src/preprocessors/flow/portscan/flowps_snort.c
--- snort-2.2.0_orig/src/preprocessors/flow/portscan/flowps_snort.c 2004-03-31 
18:09:47.000000000 +0000
+++ snort-2.2.0/src/preprocessors/flow/portscan/flowps_snort.c 2004-05-22 
23:04:00.000000000 +0000
@@ -811,6 +811,8 @@
     char buf[1024 + 1];
     u_int32_t event_id;
     u_int32_t event_type; /* the sid for the gid */
+    Event event;
+
     /*  Assign an event type to the display
      */
     if(sep->flags & ALERT_FIXED_SCANNER)
@@ -837,18 +839,21 @@
     switch(output_type)
     {
     case PKTKLUDGE:
+ DEBUG_WRAP(DebugMessage(DEBUG_FLOWSYS, FLOWPS_PREFIX_STR "%s %s\n",
+     inet_ntoa(*(struct in_addr *) address), "logged using pktkludge."););
         /* log a packet to the output system */
         p = flowps_mkpacket(sep, orig_packet, address, cur);
     case VARIABLEMSG:
-        snprintf(buf, 1024,
-                 "Portscan detected from %s Talker(fixed: %u sliding: %u) 
Scanner(fixed: %u sliding: %u)",
+        snprintf(buf, 1024, FLOWPS_PREFIX_STR
+                 "%s Talker(fixed: %u sliding: %u) Scanner(fixed: %u sliding: %u)",
                  inet_ntoa(*(struct in_addr *) address),
                  sep->fixed_talker.score, sep->sliding_talker.score,
                  sep->fixed_scanner.score, sep->sliding_scanner.score);
         buf[1024] = '\0';
   
-        /* p is NULL w/ the VARIABLEMSG fmt */
event_id = GenerateSnortEvent(p,
+ DEBUG_WRAP(DebugMessage(DEBUG_FLOWSYS, "%s\n", buf););
+
+        event_id = GenerateSnortEvent(orig_packet,
   GENERATOR_FLOW_PORTSCAN,event_type, 1,/* revision */
### END OF PATCH

Once you have this patch in place, patch your Snort code placing this patch file in the same directory where you extracted Snort.

patch -p0 < flow-portscan_output.patch

Check for errors, and then make Snort with your options.

make

Rerun Snort and run a scan to see the new output format!

An explanation of the how the flow-portscan preprocessor works might prove helpful in understanding how it detects and scores traffic.

First, the preprocessor has three main parts: scoreboards, a uniqueness tracker, and the server statistics tracker. The scoreboards—one for talkers and one for scanners—keep information about each IP address that’s come through the preprocessor and the points/scores associated with each IP since Snort started. The uniqueness tracker determines if a flow is new. If the source/destination IP, destination port, or the IP protocol changes, the flow is marked as new and passed to the server statistics tracker for scoring. The server statistics determines each flow’s score and place as either talker or scanner.

The server-learning-time setting determines how unique a connection is. For example, with a small network or SOHO connection, if you set the learning time to one minute, this will help detect port scans by making most connections new to the preprocessor. This keeps the port scans that are typically fast and hard when coming through networks very noticeable. If you were on a large network, you might want to adjust that learning time to an hour (3,600 seconds). This allows for dynamic port allocation on such things as file servers to keep them from appearing as scanners.

You might also adjust the server-ignore-limit to a high enough number that your real servers never hit the limit unless under attack/scan. The fine line to dance is how to set your server-scanner-limit low enough to have a scan marked as a scan without hitting the high limit of server-ignore-limit. For example, following are two suggestions, one for small networks and one for high-traffic networks.

# small networks - low traffic and not to many servers/services

Add scanner points to a flow/IP when the number of ports in use is more than 1 and less than 500 within the learning time when the connection is destined for a host in the watchnet.

server-scanner-limit 1 
server-ignore-limit 500 
#
# larger networks - high traffc high volume of services/servers

Add scanner points to a flow/IP when the number of ports in use is more than 5 and less than 5,000 within the learning time when the connection is destined for a host in the watchnet

server-scanner-limit 5 
server-ignore-limit 5000

If you are still having trouble with the flow-portscan preprocessor, you can always use the dumpall keyword. Setting it to 1 enables it and 0 disables it. This keyword, when enabled, dumps the contents of all three tables to the screen: server, uniqueness, and scoreboards. Using the techniques covered in this section, you could record the information from the screen to a file. If you are having trouble seeing port scans in the Snort logfile, this might help show where your scanning host is getting scored. It also might help show how effective your current configuration is.

See Also

Snort-devel mailing list

4.6. Getting Performance Metrics

Problem

I want to use the new perfmonitor preprocessor to help gather some passive statistics on my network load and sensor loads. How can I do that?

Solution

The perfmonitor preprocessor has to be enabled at Snort compile time to enable all the performance counters.

./configure --enable-perfmonitor -my -other -options

You can have more than one perfmonitor operational in your snort.conf file, but be aware that this causes a noticeable load increase on Snort and possibly on your sensor platform for disk reads/writes as well. Depending on how you invoke Snort, you may find the following scenario useful.

This example will output to the screen every five minutes:

preprocessor perfmonitor: flow console time 300

This will log the raw data to a logfile every 10 minutes instead of outputting to the screen:

preprocessor perfmonitor: flow snortfile time 600

Then run Snort like this example under /bin/sh.

This will send all STDOUT (standard output, the screen) and STDERR (standard error, also usually the screen) to the file console.log. However, remember that this works only under the shell interpreter sh.

sh <enter>
/path/to/snort -c /path/to/snort.conf -i SNIFF_INT -l /path/to/log -z > 
snort_console.log 2>&1

This will create the comma-separated values (CSV) file for you to parse through later with snort-rrd.pl, snortgraph, or other tools to graph the data, while at the same time displaying to a file some nicely formatted statistics every five minutes (time 300 value is seconds) much like this.

# displayed by running snort as in example above 
# with a perfmonitor setting of 
# preprocessor perfmonitor: snortfile perfstats.log console flow \
#  time 60 events pktcnt 100
# 
Snort Realtime Performance  : Sun Aug XX 00:42:12 2004
--------------------------
Pkts Recv:   1602
Pkts Drop:   0
% Dropped:   0.00%
   
KPkts/Sec:   0.03
Bytes/Pkt:   1052
   
Mbits/Sec:   0.12 (wire)
Mbits/Sec:   0.10 (rebuilt)
Mbits/Sec:   0.22 (total)
   
PatMatch:    171.00%
   
CPU Usage:   0.11% (user)  0.03% (sys)  99.87% (idle)
   
Alerts/Sec      :  0.0
Syns/Sec        :  0.0
Syn-Acks/Sec    :  0.0
New Sessions/Sec:  0.0
Del Sessions/Sec:  0.0
Total Sessions  :  3
Max Sessions    :  3
Stream Flushes/Sec :  10.2
Stream Faults/Sec  :  0
Stream Timeouts    :  1
Frag Completes()s/Sec:  0.0
Frag Inserts()s/Sec  :  0.0
Frag Deletes/Sec     :  0.0
Frag Flushes/Sec     :  0.0
Frag Timeouts        :  0
Frag Faults          :  0
   
....more available to the user

There are a several options to the type of data the perfmonitor preprocessor logs. These options will have to be outputted as console or screen data, as seen earlier. Otherwise, the data logged with the file/snortfile keyword is output as CSV-formatted logs written to a logfile for later analysis, as in this example:

1093754532,0.000,0.1,0.0,0.0,1052,171.00,0.0,0.0,0.0,0.0,3,3,10.2,0,1,
0.0,0.0,0.0,0.0,0,0,0.1,0.0,99.9
1093755035,0.000,0.3,0.0,0.1,919,164.21,2.7,2.7,2.8,2.7,6,12,41.2,0,1,
0.0,0.0,0.0,0.0,0,0,0.2,0.1,99.6
1093755096,0.000,0.1,0.0,0.0,1048,171.03,0.0,0.0,0.0,0.1,3,12,14.0,0,2,
0.0,0.0,0.0,0.0,0,0,0.1,0.0,99.8
1093755180,0.000,0.1,0.0,0.0,749,156.76,1.5,1.5,1.5,1.5,3,7,21.8,0,1,
0.0,0.0,0.0,0.0,0,0,0.2,0.0,99.8

For a quick meaning of this data, you can compile perfstats.c, which comes with the Snort source code, in the contrib directory.

    "gcc -o perfstats.exec perfstats.c"

Then using your newly compiled program, you can get a quick idea of what kind of data is being logged with the preprocessor, as you can see in the following code. The following output is available through the perfstats program.

# Run the program like this to get a quick statistics page 
# The "-q" flag just tells the perfstats program to only display a 
# summary of the information. With no options the perfstats program
# displays the summary information for each line its analyzing then
# the full summary like with "-q" at the bottom. 
   
cat <snort_perfmon_log_file>.log | ./perfstats.exec -q 
4 statistics lines read
   
         Mbits/Sec:        0.1       0.0       0.3
         Drop Rate:    0.0000%   0.0000%   0.0000%
        Alerts/Sec:        0.0       0.0       0.0
        K-Pkts/Sec:        0.0       0.0       0.1
     Avg Bytes/Pkt:      942.0     749.0    1052.0
       Pat-Matched:      165.8     156.8     171.0
          Syns/Sec:        1.1       0.0       2.7
       SynAcks/Sec:        1.1       0.0       2.7
           New/Sec:        1.1       0.0       2.8
           Del/Sec:        1.1       0.0       2.7
            Active:        3.8
        Max Active:       15.0
       Flushes/Sec:      21.8      10.2      41.2
            Faults:       0.0
          Timeouts:       1.2
Frag-Completes/Sec:       0.0       0.0       0.0
  Frag-Inserts/Sec:       0.0       0.0       0.0
  Frag-Deletes/Sec:       0.0       0.0       0.0
  Frag-Flushes/Sec:       0.0       0.0       0.0
     Frag-Timeouts:       0.0
       Frag-Faults:       0.0
              Usr:        0.2       0.1       0.2
              Sys:        0.0       0.0       0.1
             Idle:       99.8      99.6      99.9

If you find this type of data useful, the program supports specifying the number of lines to read back out of the logfile. For example, if you are writing to this file every five minutes, run perfstats every hour to get the statistics for the past hour by passing it the -c flag. You could call the following shell script out of a cron job on the sensor to write a status update of the load on the sensor and that portion of the network.

#!/bin/sh 
# 
# Create an hourly report from the sensor
#
# Variables 
# Time or date stamp on the file anyone? This can be useful to 
# determine if you have failure as to the last run time. 
mydate='date "+DATE: %Y-%m-%d%nTIME: %H:%M:%S"'
   
# Clean the old file and create a blank new one 
rm /path/to/status.txt 
touch /path/to/status.txt 
   
# ROUGH formatting of the new file 
echo " " > /path/to/status.txt 
echo " " >> /path/to/status.txt 
echo "THIS THE LAST HOURS performance data for: " >> /path/to/status.txt
echo "$mydate" >> /path/to/status.txt 
echo " " >> /path/to/status.txt
echo " " >> /path/to/status.txt 
   
# Execute the perfstats program only showing the data for the past 
# hour by using the -c to only analysis the last 12 lines in the 
# log file 
   
cat /path/to/perfstats.log | ./path/to/perfstats.exec -q -c 12 >> 
/path/to/status.txt

Discussion

There are some other options that you can enable in the perfmonitor preprocessor to help you gather better data for your organization. The previous examples can be tweaked with some options to the perfmonitor preprocessor with some keywords to the preprocessor.

One example is the flow option; this can be replaced with the events option or combined. The flow option on the Snort perfmonitor preprocessor calculates the protocol and traffic distribution going past the sensor. This was shown earlier in the example of real-time performance data. However, the events keyword turns on Snort ruleset measurements. Following is a partial example of the events keyword’s data.

Snort Setwise Event Stats
-------------------------
Total Events:           2444
Qualified Events:       466
Non-Qualified Events:   1978
%Qualified Events:      19.0671%
%Non-Qualified Events:  80.9329%

When the events keyword is enabled, it tracks qualified events. Qualified events are packets that have triggered a rule, while nonqualified events are packets that either didn’t match up to a rule or were found to be non-hostile. The Snort documentation from the author explains it as:

This prints out statistics as to the number of signatures that were matched by the set wise pattern matcher and the number of those matches that were verified with the signature flags. We call these non-qualified and qualified events. It shows the user if there is a problem with the rule set that they are running.”

In other words, this means that for about 80 percent of the traffic, signatures are not matching on this sensor! This could point out a problem with your signatures or that you might want to consider looking at the flow data to determine if this is correct for the network segment this sensor is monitoring.

Another tweak of the perfmonitor preprocessor is to improve performance. By now, you can see that when not used properly, this preprocessor might cause some severe load on Snort and the sensor. Two more keywords can help with this task: time and pktcnt.

The time keyword tells the perfmonitor when intervals in the counts should take place, while the pktcnt (packet count) keyword counts the number of packets that have passed the sensor since the last time the file or console was written to. The default for this is 10,000 packets, which might either be too many or too few for some of the higher-speed networks, even within a 5 to 10 minute time frame. Feel free to adjust as needed as in our previous example.

preprocessor perfmonitor: snortfile perfstats.log console flow time 60
events pktcnt 100

This example tells Snort to log to a CSV-formatted file perfstats.log, while at the same time writing nicely formatted output to the screen. It tracks traffic distribution and signature distribution information on a one-minute window of time after passing only 100 packets. This configuration works for a cable modem or DSL line, so adjust it to better suit your sensor’s network segment.

One last keyword helps calculate the theoretical limits of your network segment based on the performance data within each data set or write. However, since many multiprocessor machines don’t keep accurate kernel statistics—or in some cases, any kernel statistics at all—this helps only on single processor sensors. The following is an example of the max keyword turned on in the previous perfmonitor configuration:

#preprocessor perfmonitor: snortfile perfstats.log console flow \
#  time 60 events pktcnt 100 max
   
Snort Maximum Performance
-------------------------
   
Mbits/Second
----------------
Snort:       109.16
Sniffing:    477.56
Combined:    88.85
   
uSeconds/Pkt
----------------
Snort:       41.06
Sniffing:    9.39
Combined:    50.45
   
KPkts/Second
------------------
Snort:       24.35
Sniffing:    106.55
Combined:    19.82

See Also

Beale, Jay. Snort 2.1 Intrusion Detection. Rockland, MA: Syngress, 2004.

Snort source code documentation

Snort-devel mailing list

4.7. Experimental Preprocessors

Problem

The preprocessor arpspoof is not officially supported by Snort developers yet. How do I implement it?

How do I use the arpspoof preprocessor to detect layer 2 attacks?

Solution

The experimental arpspoof preprocessor can detect layer 2 attacks. These attacks include man-in-the-middle attacks between an important host such as a web server and core router(s). This is one of the few preprocessors that can actually have several instances running simultaneously.

This example monitors ARP cache changes for two machines on the same layer 2 segments as our sensor. As this code is still not fully supported, you cannot use it for monitoring an entire network.

preprocessor arpspoof_detect_host: 10.0.4.1 00:01:03:30:3f:c8
preprocessor arpspoof_detect_host: 10.0.4.100 00:06:29:30:16:2f

Another option is to use this preprocessor for detecting when unsolicited ARP Unicasts are sent out over your layer 2 segments.

Preprocessor arpspoof: -unicast

Discussion

This preprocessor, while experimental, has an active author and is actively patched and discussed on the snort-devel mailing list. One of the configuration issues of this preprocessor is that it can only track MAC addresses on the same layer 2 segment that the sensor is on. For example, if you are sniffing on your RAS link, this preprocessor can only be used to monitor for traffic either on that same segment or no further than the first routable hop. That is because after the packet is passed through its first hop, it no longer has its MAC address but the address of the first hop, usually the router’s MAC address. For example, if you wanted to monitor layer 2 changes to a router and the next closest switch/router, that would count as the first hop away.

Another concern with this preprocessor is that to detect your MAC changes, you have to enter manually each IP and its proper MAC address you want to use. One question that has been raised by the community is how this sensor will react to high-traffic networks such as those that use Hot Switch Routing Protocol (HSRP) for a redundant network.

Another concern with the detection of Unicast ARP requests and replies is that it can generate a lot of alarms. Apparently, most Linux and Cisco systems perform active ARP table updates of their cache files to keep the most up-to-date MAC information.

See Also

Beale, Jay. Snort 2.1 Intrusion Detection. Rockland, MA: Syngress, 2004.

Snort-devel mailing list

4.8. Writing Your Own Preprocessor

Problem

None of these preprocessors do what I want. How do I write my own?

Solution

Actually, this really depends on what you are trying to accomplish with your new preprocessor. Are you trying to create an application decoder to pass plain text data back the rules engine for analysis? Are you trying to create an anomaly detection tool out of nonstandard rules?

Discussion

For a really long, hard look at how to create your own preprocessor, see Jay Beale’s entire chapter on the subject in the Snort 2.1 book from Syngress Publishing.

See Also

Beale, Jay. Snort 2.1 Intrusion Detection. Rockland, MA: Syngress, 2004.

Snort-devel mailing list

Get Snort Cookbook now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.