(Redmine issue: #7041)

(Redmine issue: #7041)

(Redmine issue: #3387)

The makefile target 'develop' now pre-builds web interface artifacts. There is a new example configuration file for Apache2 that demonstrates the necessary configuration for Git installations. There is a custom WSGI script file for easier web interface installation from Git. (Redmine issue: #4216)

(Redmine issue: #4800)

Currently only built-in Flask commands run, shell, etc. are available. It can be used to launch development web server, which is now documented in development documentation page. (Redmine issue: #4216)

After this all code related to MongoDB is gone and database migration from MongoDB to PostgreSQL is no longer possible. I have kept the documentation page about migration for now, so that the users can still at least find the information. (Redmine issue: #4225)

(Redmine issue: #4213,#1017,#3361,#3443)

This commit introduces the Pythonic reimplementation of mentat-reporter-ng module. It is complete from roughly 70 percent, there are many features that stil need to be implemented as well as some necessary polishing.

The most important next implementation steps:
* thresholding and relapse
* email template improvements
* report section legends with translations
* report web interface improvements
* unauthorized access to reports
* unit testing and bulletproofing

(Redmine issue: #3378)

The mentat-informant.py module is a Python replacement for previous mentat-briefer.py module. It is designed to provide periodical statistical overviews via email. The email reports are localized using babel package and can be translated into any language. Additionally, the reporting engine uses Jinja2 templating system for message rendering, so it enables very easy customization. (Redmine issue: #3377)

(Redmine issue: #3443)

The hawat application factory now returns production level interface by default instead of development level.

The default configuration object for Hawat application factory is now hawat.config.ProductionConfig. (Redmine issue: #3734)

This commit represents huge amount of work and introduces new Python-based version of Hawat web user interface for Mentat system. It is built on top of the great Flask framework. Currently it is still work in progress and many features are implemented only as dummy stubs, however the core functionality is almost finished. Following is a list of current key features:

  * Focus on modular architecture using Flask`s blueprint feature.
  * Multi-layered configurability (defaults, multiple file overrides).
  * Plugin system.
  * Internationalization support.
  * Support for multiple authentication systems.

This commit provides working albeit hollow starting point for future web interface development. IT IS NOT READY FOR PRODUCTION USAGE!!!

(Redmine issue: #3734,#3361)

This module is capable of precaching certain sets of database values, that can be later used for example for generating select boxes for the user interface. (Redmine issue: #3386)

Improved code style using pylint tool and adjusted related documentation for module mentat.daemon.piper, which is a base module for all real-time message processing daemons. (Redmine issue: #3374, #3361)

(Redmine issue: #3361)

Code for each Mentat module now resides in valid Python module in mentat.module and all apropriate launcher scripts are very minmalistic. This is because names of executable files are not valid Python names for Python modules and this new approach will improve code testing and documentation processes. (Redmine issue: #3374, #3361, #3443)

(Redmine issue: #3361)

Migration, refactoring and code style unification to support latest PyZenKit and Pynspect libraries.

This huge update represents a lot of work on many parts of the system. After major upgrade in PyZenKit and Pynspect libraries it was necessary to bring this project up to date. There were many changes in the code, lot of refactoring and while all this was done, code quality was improved, documentation written and unit tests modified. Many of these changes simply make sure, that the code works with latest versions of both libraries. This commit is supposed to be a single major feature upgrade, after this only smaller commits will follow to polish this huge amount of work.

(Redmine issues: #3374, #1017, #3361, #3387, #3443)

Statistician is message post-processing module performing statistical calculations for given time frame and storing them to persistent storage and RRD databases. (Redmine issue: #3379)

(Redmine issue: #3385)

Enricher is real-time message processing module with modular architecture and with support for easy implementation and usage of arbitrary enrichment plugins. Plugins can be developed based on predefined base class and dynamically loaded and instantinated upon module startup. (Redmine issue: #3365)

Wrong string literal used when generating backup shell command.

Greatly improved code quality using pylint and pyflakes. All files now rank above 9 points with custom .pylintrc-bin file. Following exceptions were added to default pylint configuration:

  * line-too-long - I do not want to have calls to logging service spanning over multiple lines and distracting from code flow. I want them to be only single line, no matter how long, because it is not important.
  * bad-whitespace - I like to add additional whitespaces in places, where they should not be, mostly to align related items vertically, which in my opinion improves readability.
  * logging-format-interpolation - In Python3 I prefer string.format() to string % (args).

Additionally the pattern for module name validation was altered, because of our executable file naming policy (added suppot for '-' and '.' characters in name).

(Redmine issue: #3443)

Default value must be none, otherwise it is not possible to turn the output queue off. (Redmine issues: #3364 and #3387)

Status display command is fully working, the update command still needs few tweaks and more testing before it can be considered production tool. There is an issue with storing network records into database, because the database is not capable to store IPv6 as 16 byte integers. Because of this the existing query interface in Hawat will not work. For the necesary conversions the typedcol library could be used.

(Redmine issue: #3385)

(Redmine issues: #3376 and 3361)

This commit introduces finished working prototype of mentat-storage.py real-time message processing module including appropriate unit tests and basic documentation work. Key features are possible customization of target database and collection, usage of core database configuration file, which can be overriden with local config file, or command line options. Messages are currently stored in database one by one, however batch processing will possibly be implemented in the future.

(Redmine issues: #3364, #1017 and #3361)

(Redmine issue: #3361)

Improvements include:
* removed default hardcoded values for 'collections' and 'caches' configuration
* rewritten initialization and setup parts of the script
* improved comments
* improved config file

Now it is possible to stop|kill component in any 'RUNNING_...' state.

Script will try to determine, if the startup was successfull, or not. It also exits with appropriate exit code.