Descriptions of common parameters should be in the documentation, so that the updates do not have to be done on so many places. Additional documentation should be present only for script/daemon specific parameters. (Redmine issue: #3387)

This commit brings in considerable amount of work. When working on this issue everything was so tied together, that I have abandoned all attempts to split it into multiple commits. There are following changes and more:

* Implementation of lightweight chroot for containing the application in single directory tree.
* Design changes in scripts and daemons to implement the lightweight chroot.
* Many configuration file changes to make everything work properly.
* Makefile additions, new targets related to running
* Updated development documentation page.

(Redmine issue: #4216)

This commit also contains following related improvements:

* documentation enhancements
* configuration file cleanup
* runlog enhancement, better information about items created and deleted
* code simplification, the status and update command now share the same code
* coding style improvements using pylint tool

(Redmine issue: #3752)

Exception files can be used to override internal whois resolving for designated networks to different abuse groups without modifying the original database records. This new feature enables users to generate whois exception files from simple text files containing lists of networks, which can be easily generated by anyone. (Redmine issue: #3385, #3365)

Migration, refactoring and code style unification to support latest PyZenKit and Pynspect libraries.

This huge update represents a lot of work on many parts of the system. After major upgrade in PyZenKit and Pynspect libraries it was necessary to bring this project up to date. There were many changes in the code, lot of refactoring and while all this was done, code quality was improved, documentation written and unit tests modified. Many of these changes simply make sure, that the code works with latest versions of both libraries. This commit is supposed to be a single major feature upgrade, after this only smaller commits will follow to polish this huge amount of work.

(Redmine issues: #3374, #1017, #3361, #3387, #3443)

Status display command is fully working, the update command still needs few tweaks and more testing before it can be considered production tool. There is an issue with storing network records into database, because the database is not capable to store IPv6 as 16 byte integers. Because of this the existing query interface in Hawat will not work. For the necesary conversions the typedcol library could be used.

(Redmine issue: #3385)

(Redmine issue: #3364)

This commit introduces finished working prototype of mentat-storage.py real-time message processing module including appropriate unit tests and basic documentation work. Key features are possible customization of target database and collection, usage of core database configuration file, which can be overriden with local config file, or command line options. Messages are currently stored in database one by one, however batch processing will possibly be implemented in the future.

(Redmine issues: #3364, #1017 and #3361)