Merge branch 'master' of homeproj.cesnet.cz:warden

src/warden-client/doc/README.cesnet

@@ -36,15 +36,13 @@ B. Registration
       - hostname of the machine, where client runs,
       - name of the detection service (for example 'ScanDetector'),
       - client type = sender,
-      - description tags of sent events (more at 
-        https://homeproj.cesnet.cz/projects/warden/wiki/Typy_udalosti), 
+      - description tags of sent events (see below)
       - CIDR from which client will communicate with Warden server.
 
     * For receiver client:
       - hostname of the machine, where client runs,
       - client type = receiver,
-      - type of requested events (for example 'portscan', more at
-        https://homeproj.cesnet.cz/projects/warden/wiki/Typy_udalosti),
+      - type of requested events (for example 'portscan', see below)
       - receiving of sent events from my organization = yes/no (organizations
         are separated based on the top-level and second-level domain),
       - CIDR from which client will communicate with Warden server.
@@ -60,19 +58,97 @@ B. Registration
       https://tcs.cesnet.cz/
 
 --------------------------------------------------------------------------------
-C. Configuration
+C. Description tags
+
+   Tags are case insensitive alphanumeric strings, designed to allow event
+receivers to do more general filtering according to event source. Receiver
+can for example decide to use only events originating at honeypots, or
+filter out events, generated by human conclusions or correlation engines.
+
+   Sender client specifies its descriptive tags during registration, it is
+up to client administrator's judgment to select or omit any particular tag.
+   Currently tags fall into four general categories - based on event medium,
+data source, detection methodology and detector or analyzer product name.
+   Product name tag is free to choose if same product name was not yet
+accepted by registrar, otherwise existing form must be used (registrar will
+notify about such cases).
+   Categories list is certainly not complete.  Therefore if new client's
+administrator feels that name or type of important feature of his (or
+others) detector is not covered, providers of Warden server are glad to
+discuss it at registration address or at Warden project mailing list. 
+However, it may or may not be accepted, as aim is to keep the list of
+categories possibly unambiguous, short and usable.
+
+   Following is grouped list of tags together with closer description and
+examples.
+
+ 1. Detection medium
+
+    * Network - network data based (Snort, Suricata, Bro, FTAS, LaBrea, Kippo)
+    * Host - host based (Swatch, Logcheck)
+    * Correlation - corellation engines (Prelude, OSSIM)
+    * External - credible external sources (incident reporting, ticket
+                 systems, human verified events)
+
+ 2. Data source
+
+    * Content - datagram content based detectors (Snort, Bro)
+    * Flow - netflow based (FTAS, FlowMon)
+    * Connection - connection data (portscan, portsweep)
+    * Data - application data based (SpamAssassin, antiviruses)
+    * Log - based on system logs, where more specific source is not
+            applicable (Swatch, Logcheck, SSH scans)
+    * IR - incident reporting, ticket systems, human verified events
+
+ 3. Detection methodology
+
+    * Honeypot (LaBrea, Kippo, Dionaea)
+    * Antispam (SpamAssassin, Bogofilter, CRM114, Policyd, greylisting)
+    * Antivirus (ClamAV)
+    * IDS - IDS/IPS, Snort, Suricata, Bro
+
+ 4. Detector/analyzer product name examples
+
+    * Snort, FTAS, SpamAssassin, LaBrea, Swatch, Prelude
+
+--------------------------------------------------------------------------------
+D. Types of events
+
+   Event types purpose is to allow event receivers to filter and/or
+categorise particular events according to attack characteristics. Types are
+loosely chosen as list of common security incidents nowadays observed. List
+is by no means complete, however it was created based on expected use cases
+at receiving places. Possibility of a new type is also open to discussion.
+
+   * portscan - TCP/UDP port scanning/sweeping
+   * bruteforce - dictionary/bruteforce attack to services authentication
+   * spam - unsolicited commercial email (except phishing)
+   * phishing - email, trying to scam user to revealing personal information
+     (possibly by some other channel)
+   * botnet_c_c - botnet command & control master machine
+   * dos - (possibly distributed) denial of service attack
+   * malware - virus/malware sample
+   * copyright - copyright infringement
+   * webattack - web application attack
+   * other - the rest, uncategorizable yet
+
+   In case of complex scenarios with structured info more events with
+particular parts of information can be created.
+
+--------------------------------------------------------------------------------
+E. Configuration
 
     CESNET Warden server resides at URI 'https://warden.cesnet.cz:443/Warden'.  
 
 --------------------------------------------------------------------------------       
-D. Testing
+F. Testing
 
     For testing purposes of sender clients, event type 'test' can be used.
     These events will end up in server database, but will not be taken
     further into consideration.
 
 --------------------------------------------------------------------------------
-E. Authors of this document
+G. Authors of this document
 
     Pavel Kacha     <ph@cesnet.cz>
     Jan Soukal      <soukal@ics.muni.cz>

src/warden-server/doc/README

@@ -6,16 +6,20 @@ Content
 
  A. Overall Information
  B. Installation Dependencies
- C. Registration
- D. Installation
- E. Integration with Local Applications
- F. Client Upgrade
- G. Functions, Arguments and Calls
- H. Authors
+ C. Installation
+ D. Configuration
+ E. Update
+ F. Init Scripts
+ G. Registration of Clients
+ H. Status Info
+ I. Nagios Integration
+ J. Authors
 
 --------------------------------------------------------------------------------
 A. Overall Information
 
+ /*TODO*/Upravit pro kontext warden serveru
+
  1. About Warden Client
  
     Warden is a client-based architecture service designed to share detected
@@ -46,7 +50,9 @@ A. Overall Information
 
 --------------------------------------------------------------------------------
 B. Installation Dependencies
-  
+ 
+ /*TODO*/Zkontrolovat, zdali plati...
+
     Perl 5.10.1  
     SOAP::Lite
     IO::Socket::SSL
@@ -54,47 +60,9 @@ B. Installation Dependencies
     FindBin
 
 --------------------------------------------------------------------------------
-C. Registration
+C. Installation
 
-    Any client attempting to communicate with Warden server must be registered
-    on this server. Unknown (not registered) clients are not allowed to exchange
-    any data with server.
-  
-    Registration of your client is provided by Warden server administrator.
-    Usually via e-mail.
-  
-    Clients need to have valid client certificate to prove their identity to
-    the Warden server. 
-  
-    Each client is defined by its hostname, service name, type of client, type
-    of requested events and CIDR the client is allowed to communicate from only.
-  
-    Hostname                  - hostname of client to be registered
-    Service name              - Text string. Unique name of the service
-                                the client is integrated in.
-                                E.g. 'ScanDetector_1.0'. This is mandatory for
-                                'Sender' client. Default value null is used for
-                                'Receiver' client.
-    Type of client            - Either 'Sender' or 'Receiver'.
-    Type of requested events  - Type of events the client only accepts from
-                                Warden server. This is mandatory only for
-                                'Receiver' client. Default value null is used
-                                for 'Sender' client. Brief information about
-                                event types is provided in section G. Functions
-                                arguments and calls.  
-    CIDR                      - CIDR stands for IP address or IP (sub)net
-    				the client is going to communicate from. Any
-				communications between the client and Warden
-				Server must be performed from IP address from
-				a range stated in CIDR.
-				Examples: '123.123.0.0/16', '123.123.123.123/32'
-
-  
-    For complete information about client attributes and/or event types see
-    Warden project documentation.
-   
---------------------------------------------------------------------------------
-D. Installation
+  /*TODO*/Zkontrolovat, co z klienta plati i pro server a doplnit chybejici...
 
  1. Check SHA1 checksum of corresponding Warden client package archive
 
@@ -168,170 +136,89 @@ D. Installation
 	-V                        print script version number and exit
 	
     Example: $ ./install.sh -d /opt -u detector -k /etc/ssl/private/client.key
-    			    -c /etc/ssl/certs/client.pem -a /etc/ssl/certs"
+ 
+--------------------------------------------------------------------------------
+D. Configuration
+
+  /*TODO*/Doplnit konfiguraci (warden.conf) - mozna to v klientske verzi
+  zasahuje do predchozi sekce, zkontrolovat
 
 --------------------------------------------------------------------------------       
-E. Integration with Local Applications
- 
- (Note: Clients need to be registered on server to be able to communicate with
-        server properly. See section C. Registration for more information about
-        client registration.)
+E. Update
  
- 1. Client sender (this type of client reports events to Warden server)
-  
-    Client functionality is included as a Perl module (WardenClientSend.pm)
-    into Perl code of local detection application.   
-   
-    See warden-client/doc/example-sender.pl.txt for example how to use
-    warden-client sender functionality.
-    
-    Brief information about syntax of sending functions and functionality is
-    provided in section G. Functions arguments and calls.
- 
- 2. Client receiver (this type of clients uploads events from Warden server)
- 
-    Client functionality is included as a perl module (WardenClientReceive.pm)
-    into perl code of local 'reaction' application or may be used as as core of
-    standalone local application.
-      
-    See warden-client/doc/example-receiver.pl.txt for example how to use
-    warden-client receiver functionality.   
-    
-    Brief information about syntax of receiving functions and functionality is
-    provided in section G. Functions arguments and calls. 
+ /*TODO*/Doplnit, jak se dela update...
+
+   To upgrade a client, install a new version.
 
 --------------------------------------------------------------------------------
-F. Client Upgrade
+F. Init Scripts
 
-   To upgrade a client, install a new version.
+  /*TODO*/Doplnit init scripty
+
+ 1. Start
+
+    /*TODO*/Doplnit...
+
+ 2. Stop
+
+    /*TODO*/Doplnit...
+
+ 3. Restart
+
+    /*TODO*/Doplnit...
+
+ 4. Status
+
+    /*TODO*/Doplnit...
+
+ 5. Force-stop
+
+    /*TODO*/Doplnit...
 
 --------------------------------------------------------------------------------
-G. Functions, Arguments and Calls
+G. Registration of Clients
 
- 1. WardenClientSend::saveNewEvent
- 
-    Function to upload one event on the Warden server. See example 'Sender'
-    client in warden-client/doc/example-sender.pl.txt
-    
-    Function call (Perl):
-    
-    # Path to warden-client folder
-    $warden_path = '/opt/warden-client';
-    
-    # Inclusion of warden-client sender module
-    require $warden_path . '/lib/WardenClientSend.pm';
-    
-    # Sending event to Warden server
-    WardenClientSend::saveNewEvent($warden_path, \@event);
-    
-    Event array is defined as (perl):
-    
-    @event = ($service, $detected, $type, $source_type, $source, $target_proto,
-              $target_port, $attack_scale, $note, $priority, $timeout );
-              
-    Event array attributes with example value and explanation on the right
-    (Perl):
-
-    # SERVICE - VARCHAR (64)
-    # Name of a service detecting this event. Service must be the same with this
-    # provided in 'Sender' client registration. See more about this issue in
-    # section C. Registration.    
-    $service      = "ScanDetector";
-
-    # DETECTED - TIMESTAMP in UTC, ISO 8601
-    # Date and time when was event detected.
-    $detected     = "2011-07-16T19:20:30.45";
-    
-    # TYPE - VARCHAR (64)
-    # Type of reported event. Currently supported values are:
-    # darkspace   - access into honeypot segment
-    # portscan    - scannig of TCP/UDP ports
-    # bruteforce  - bruteforce/dictionary attack against authentication
-    #               service(s)
-    # spam        - unsolicited e-mail that does not have phishing-like
-    #               character
-    # phishing    - e-mail attempting to gather sensitive data
-    # botnet_c_c  - command and control center of botnet
-    # dos         - (distributed) denial of service attack
-    # malware     - virus sample
-    # copyright   - copyright infringement issue
-    # webattack   - attack against web application
-    # other       - anything that does not match any of previous categories
-    $type         = "portscan";
-    
-    # SOURCE_TYPE - VARCHAR 64
-    # Type of source of reported attack/issue. Currently supported values are:
-    # IP, URL, Reply-To:, null
-    $source_type  = "IP";
-    
-    # SOURCE - VARCHAR 256
-    # identification of attack source/origin according to source_type
-    $source       = "123.123.123.123";
-    
-    # TARGET_PROTO - VARCHAR 16
-    # Protocol type of reported attack/issue target. Supported are all L3 and L4
-    # protocols and null 
-    $target_proto = "TCP";
-    
-    # TARGET_PORT - INT 2
-    # Port number of reported attack/issue target or null.
-    $target_port  = "22";
-    
-    # ATTACK_SCALE - INT 4
-    # Definition of attack scale, e.g. number of affected targets. Null is also
-    # possible when attack scale is not known or clear enough.
-    $attack_scale = "1234567890";
-    
-    # NOTE - TEXT
-    # Some important(!) note or comment or null. Also, it may contain virus
-    # sample, phishing e-mail with headers and other accordingly to event type.
-    $note         = "this threat is dangerous";
-    
-    # PRIORITY - INT 1
-    # Subjective definition of incident severity. Values 0-255 or null are
-    # possible where 0 is the lowest priority.
-    $priority     = "null";
-    
-    # TIMEOUT - INT 2
-    # Subjective time (in minutes) or null. After this time event might be
-    # considered timeouted.
-    $timeout      = "20";
+  /*TODO*/Popsat registraci klientu
 
- 2.  WardenClientReceive::getNewEvents
+ 1. Register Sender
 
-    Function to download batch of events from the Warden server. Downloaded
-    events are stored in @events array. See example 'Receiver' client in
-    warden-client/doc/example-receiver.pl.txt
-    
-    Function call (perl):
-    
-    # Path to warden-client directory
-    my $warden_path = '/opt/warden-client';
-
-    # Inclusion of warden-client receiving functionality
-    require $warden_path . '/lib/WardenClientReceive.pm';
-
-    # Definition of requested event type. Type must be the same with this
-    # provided in 'Receiver' client registration. See more about this issue in
-    # section C. Registration. See more about event types in section
-    # G. 1. WardenClientSend::saveNewEvent
-    $requested_type = "botnet_c_c";
-
-    # Download batch of new events from Warden server
-    @new_events = WardenClientReceive::getNewEvents($warden_path,
-                                                    $requested_type);
-                                                    
-    Structure of each received event in the event array equals to this explained
-    in section G. 1. WardenClientSend::saveNewEvent. It has one additional
-    attribute ID - unique id of this particular event (BIGINT).
+    /*TODO*/Doplnit...
+
+ 2. Register Receiver
+
+   /*TODO*/Doplnit...
+
+ 3. Unregister Client
+
+   /*TODO*/Doplnit...
+
+--------------------------------------------------------------------------------
+H. Status Info
+
+  /*TODO*/Popsat praci s administrativnimi/dohledovymi funkcemi
+
+  1. Get Status
+
+     /*TODO*/Doplnit...
+
+  2. Get Clients
+
+     /*TODO*/Doplnit...
+
+--------------------------------------------------------------------------------
+I. Nagios Integration
+
+  /*TODO*/Doplnit...
+
+  Is available via Nagios plugin /opt/warden-server/bin/warden-alive.
 
 --------------------------------------------------------------------------------
-H. Authors
+J. Authors
 
 Development:	Tomas PLESNIK   <plesnik@ics.muni.cz>
 		Jan SOUKAL      <soukal@ics.muni.cz>
 
-Copyright (C) 2011 Cesnet z.s.p.o
+Copyright (C) 2012 Cesnet z.s.p.o
 
 Special thanks go to Martin Drasar from CSIRT-MU for his help and support
 in the development of Warden system.