I wrote a Kibana plugin - https://github.com/sivasamyk/logtrail and a few graylog plugins which opened many doors for me. It also helped me land my last job at Sematext. I lost interest in maintaining the plugin after Elastic's open-source license changes.
At Sematext we replaced our HBase based metrics datastore with Clickhouse. We are happy with the performance gain and flexibility. We also added support for Clickhouse monitoring - https://sematext.com/blog/clickhouse-monitoring-sematext/
https://github.com/sivasamyk/logtrail - A Kibana plugin to view and tail logs. Started as project to scratch my own itch and now used by many. Technology : AngularJS and NodeJS. Currently working on a major enhancement for this project :)
I submitted logtrail (https://github.com/sivasamyk/logtrail) 312 days ago (more of a side project of mine) and made it to front page. Got lots of stars :). Helped me boost my confidence and currently has active users and made multiple releases with new features. Happy I did it.
Agreed, there are not lots of tutorials out there explaining plugin development in Kibana. Mostly I used Timelion and Kibana source code as reference and also I kept the Kibana API dependency to minimal since it is not documented and may change anytime (and also I am new to Javascript, so had tough time in understanding the code :)).
Yes the whole view inspired by PapertrailApp ( as mentioned in the first line of README ) . I liked the Papertrail App UI, but didn't want to use cloud based solution.
This view is tailor made to view log files. Personally I didn't like using the Kibana table-views to view log files. Features like
1) Filter logs files from particular machine
2) Live tail
3) Seek to any time in log file (e.g. you can enter 'last friday' in seek window)
4) filter by program/app
makes logtrail suitable for viewing and debugging log files.
Moreover I always wanted a interface like this in Kibana to view all my log files in a single view :)
I assume these companies were honoring NSA requests for long time. These requests are made public only after Snowden revelation, just because these companies are scared about losing their customers.