Distributed grep client in Go(github.com)

https://github.com/irtefa/sherlog

7 comments

I don't know if this is meant to be actually used, but clearly ssh with shared keys and a trivial bash script would be better than just shelling out to grep from go itself. Fairly pointless really.

tsenart13 anni fa

Without the context of why this was developed, I have to agree with easytiger's point of pointlessness. Perhaps it was just an exercise in curiosity and exploration of the language. If not, I would use pdsh instead http://sourceforge.net/projects/pdsh/ $ pdsh -b -w '10.0.0.1,...,10.0.0.n' 'cat /var/log/your_log | grep "your_pattern"'

easytiger13 anni fa

Yea i dont mind things like this being posted, but some context would be good. It seems to present itself like a serious tool which might dupe some inexperienced people.

irtefa13 anni fa

I was not intending to confuse people. This is the first time I posted something on hn. I will give more context when I post something next time. Thanks!

easytiger13 anni fa

no worries. HN does have quite a big love affair with go.

How long have you been writing in it? I have an idea for autility id like to implement in it for a bit of fun.

maguireb13 anni fa

Nice one on pdsh! Hadn't heard of it before but will definitely be using it from now on. No more bash for loops with ssh :)

microtonal13 anni fa

This doesn't really particularly safe to me (assuming that exec.Command allows spaces in arguments):

  cmd := exec.Command("grep", s, logName)

Since you are searching for the expression in just one log file, why not use Go's excellent regexp package?

cmccabe13 anni fa

exec.Command doesn't run a shell; it just executes the command. So yes, it's perfectly safe to have spaces and other shell metacharacters in there. Whether or not it's efficient here is another question, of course.

microtonal13 anni fa

So yes, it's perfectly safe to have spaces and other shell metacharacters in there.

You could (for instance) probe content of system files by passing

  -f/some/file

And matching that to the log. If you have some control over the log, you could use this to read content from system files. Of course, it's all indirect, since the file is interpreted as containing regular expressions. But it's certainly not 'perfectly safe'.

cmccabe13 anni fa

I could also read content from system files by using the advanced hacker tool "cat".

I don't see what this has to do with exec.Command.

irtefa13 anni fa

It started off as a class assignment where it was required to execute "grep" on whatever shell we were running the program on. That's why, instead of using the regexp package, we went for the above (somewhat shitty) approach. But, that's something I am definitely looking into changing!

enneff13 anni fa

You might have to work pretty hard to write a grep that's as good as grep. It pulls a lot of swift moves in the name of speed.

[deleted]13 anni fa

willvarfar13 anni fa

I've just got a little python script that sshs around my boxes running grep on each, writing it to the a log file with the box name prefixed, and sorting them all by date (as my log lines have date prefixes and this makes sense for me).

Using ssh instead of running a service on each server seems more straightforward to my thinking.

satish8013 anni fa

Golang dependency on each machine can be removed by building the executable. The command to execute the server cannot be run remotely. Since the grep is confined to a single log file, it is not very useful. It shows how easy to use go routines to make concurrent requests.

[deleted]13 anni fa

mcot213 anni fa

This isn't really 'distributed grep' as far as I can tell. It is simply running grep individually on each machine.

[deleted]13 anni fa