Shuffle directories and filenames to get rid of historic cruft and to better reflect contrib/production status

Radko Krkoš authored 6 years ago and Pavel Kácha committed 6 years ago

Signed-off-by: Pavel Kácha <ph@cesnet.cz>

Václav Bartoš authored 6 years ago and Pavel Kácha committed 6 years ago

read_cfg() no longer add directory of warden_client.py in front of the config file name. It simply opens the path/filename passed. This allows to install warden_client.py to system paths and use relative paths to config files, but IT MAY BROKE SOME EXISTING INSTALLATIONS if they use relative path to config file and run script from different directory than where warden_client.py is located.

Providing certificates to HTTPSConnection was changed in Py 2.7.9, the old
way is still supported, but there is a memory leak (fixed in 2.7.12). The
code was changed to use the new way (SSLContext object) if it's
available, thus avoiding the leak.

Some Python/ssl module versions incorrectly initialize hashlib, causing weird openssl errors. Importing hashlib explicitly helps.

Logging module in Python 2.6 does not accept unicode socket path - but strings from JSON configuration are always unicode. Applied workaround, now on 2.6 Client accepts at least ASCII filenames for syslog socket.

 * Added format_time (to avoid time conversions where not necessary, for example when parsed from logs)
 * Added utc arg for format_timestamp, to signal, whether input epoch needs adjustment to local time (defaults to True, so yes)

* Fixed initialization logging

* filenames now have .idea extension (Mentat requires that :) )
* bugfix: use list range instead of generator xrange on event index list generation (also in warden_client)

Brown paper bag commit

This reverts commit 1fc362c3.

* warden_server: Added "events_id" key into errors along with "events" - makes error messages bigger, but allows client operators to identify offending messages by stable identifiers based on logs.
* Removed too much internal info from database errors.
* Removed format type checking from Draft4Validator, using only explicit schema regexps - Draft4Validator raised FormatError istead of ValidationError, rendering iter_errors unusable.
* warden_client: Implemented retries on server errors.
* Client now honours send_events_limit from server - too long list of events is split and sent in chunks.
* getInfo and sendEvents update local send_events_limit according to one sent by server in info or error message.
* Logging is now explicit, not authomatic in Error class, allowing to log only single time at top level methods.
* More sane default client name.
* Errors from server are now checked and ensure correct format.
* warden_filer: Ditched error and retry handling, warden_client now does for us.

Reworked error handling in server, client and filer (again :( ). We definitely have to support multiple errors so let's do it right.  Error now contains list of errors.  Got rid of "detail" section everything now goes directly within particular error.  Error can bear "events" attribute for list of sent events concerned (or none if all sent events erred). Streamlined error logging.
Server now retries if MySQL went away.
Response result now gets properly encoded if Unicode appears somehow (however that's just fighting symptoms, should get more thorough fix).

syntax highlighting. :) (deja-vu)

syntax highlighting. :)

  consistently. Possible keeping and reopening where possible will need more
  work and testing, I'm not keen to try to clench in now. So - changed to
  reopening connection each time, which will possibly affect performance (on
  small queries).
- Both server and client library now defaults to get_events_limit instead of 1

Fixed Recon id in catmap
Fixed isoformat calling on None
Shortened validation error message
Fixed rejecting of Test messages

Added optional client auth by shared secret (also modified schema, client, curl client). Modified Client __str__ method for log display without disclosing secret. Modified wsgi server logging and parameter disposing accordingly.

Clients are now authorized by client= argument in URL, schema and client lib is updated accordingly. Method authentication defined in "expose" decorator. Added Client container class for deterministic logging. Schema update to consistent InnoDB and utf8mb4 everywhere.

Error is now raised via req.error helper, which automatically adds already known info (method, request id), thus calls are terser.
Client's Error communicates request id, if received from server.
exc_info made explicit everywhere. Automatic exc_info gathering sometimes returned bogus info about unrelated exception from far before.