Capture long running method execution in Spring Boot using annotation
It is crucial to capture long running method execution in your application (whether it is an API or traditional mvc application). You should also …
Recently, I had to listen kafka consumer log to identity bug I have found in my web application. But listening kafka consumer logs via console output was a pain and reading logs on the console was so hard. Then I was just starting to search “how to pretty print for the kafka logs”. After all search, finally I created a util python script to listen the logs.
I also didn’t want to see all fields inside the kafka record. I only wanted to see specific fields I was looking for.
If you faced the same problem, you can use this script as well
Script itself is self-explanatory, I will not explain it. You can run this script from your IDE (such as PyCharm).
from subprocess import Popen, PIPE
import json
import datetime
def listen_console(command):
process = Popen(command, stdout=PIPE, shell=True)
while True:
line = process.stdout.readline().rstrip()
if not line:
break
yield line
def pretty_print(kafka_json):
creation_date = datetime.datetime.fromtimestamp(float(kafka_json["{timestamp_field}"]) / 1000)
print("{")
print("Any field: ", kafka_json["field_name"])
# ...
print("}")
if __name__ == "__main__":
kafka_consumer_log = "sh /path/to/kafka/bin/kafka-console-consumer.sh --topic {kafka_topic_name} --bootstrap-server {ip_address}:9092"
for item in listen_console(kafka_consumer_log):
kafka_json = json.loads(item)
pretty_print(kafka_json)