No support for format

[saadqc]

I have a custom method to return logstash format. Although initializing it doesn't seem to work.

handler = CMRESHandler(hosts=[{'host': 'localhost', 'port': 9200}],
                               auth_type=CMRESHandler.AuthType.NO_AUTH,
                               es_index_name="my_logs")
handler.setFormatter(LogstashFormatterVersion1())

I noticed in emit method in handlers.py self.format(record) is being called but return dict is not stored anywhere.

[cmanaha]

Thanks a lot for the feedback.

I'm not entirely sure why do you need a formatter for this handler in particular? Could you give more context of what are you trying to achieve so we get a better understanding ?

This handler is intended to pick up the full record (including the internal dict), serialise into a json doc and sent/store into elastic in bulk format.

Formatters (please read https://docs.python.org/3.5/library/logging.html#formatter-objects) for more reference are responsible for converting a LogRecord to a string (usually) which can be interpreted by either a human or an external system.

For the CMRESHandler, you do not need a formatter, the library will pick up the internal logrecord and inject directly into elastic.

[stephenpaulger]

@saadqc I'm not sure what your use case is but you may be able to use filters to achieve what you're trying to do while using this log handler.

https://docs.python.org/2/howto/logging-cookbook.html#filters-contextual

[saadqc]

@stephenpaulger @cmanaha : I switched to a different package after struggling with this.

I will reproduce this issue and share my findings here. Otherwise close this PR.

Thanks

[stephenpaulger]

@saadqc Interested to hear what package you've switched to. I've had a different issue with this one when using ElasticSearch 6.5.

[saadqc]

@stephenpaulger : it's python-logstash with python's default logging.

        fh = FileHandler(log_file_path)
        fh.setLevel(logging.INFO)
        fh.setFormatter(LogstashFormatterVersion1())

[michielvoo]

@cmanaha One use case for using a formatter is Elastic Common Schema. The official Python support for ECS in elastic/ecs-logging-python is implemented as a formatter.