The Art of Command Line(github.com)
github.com
The Art of Command Line
https://github.com/jlevy/the-art-of-command-line
29 comments
The discussion here has tended to focus on either one or the other of two different facts:
You absolutely need to be aware of and manage language and locale settings diligently. This is a big topic, but generally, whenever possible, use UTF8. And you generally don't want to change your desktop OS or applications' locale or language settings other than that.
However, if you are processing data files and are aware of the implications, and can get away with treating data as binary instead of character streams -- for example, any of the command sequences that use sort/uniq for uniqueness or set union/intersection/difference -- then using C locale is way faster. As in, the difference between doing something in an hour on a big machine and rewriting a whole pipeline in Hadoop. A good fact to know.
It's obvious I (the author) didn't make the simultaneous truth of both these points clear. I'll rewrite.
You absolutely need to be aware of and manage language and locale settings diligently. This is a big topic, but generally, whenever possible, use UTF8. And you generally don't want to change your desktop OS or applications' locale or language settings other than that.
However, if you are processing data files and are aware of the implications, and can get away with treating data as binary instead of character streams -- for example, any of the command sequences that use sort/uniq for uniqueness or set union/intersection/difference -- then using C locale is way faster. As in, the difference between doing something in an hour on a big machine and rewriting a whole pipeline in Hadoop. A good fact to know.
It's obvious I (the author) didn't make the simultaneous truth of both these points clear. I'll rewrite.
I have to reiterate this; don't do it -- sort order changes and you'll run into other mysterious issues.
It's not worth the "performance improvement".
It's not worth the "performance improvement".
There are good reasons to use LC_ALL=C for certain commands, especially if you want sorting that's stable across different systems. Sometimes you want "aa" to be before "b" even though the Norwegian UTF-8 locale puts it after "å" (which comes after "…zæø"):
$ echo $'a\naa\nb\nå'| LC_ALL=C sort
a
aa
b
å
$ echo $'a\naa\nb\nå'| LC_ALL=nn_NO.UTF-8 sort
a
b
å
aa
$ echo $'∨\n∧'|LC_ALL=C sort -u
∧
∨
$ echo $'∨\n∧'|sort -u
∨
I'm well aware of that, but in general, don't do it still applies :-)
In general, I'd never do it for performance only, it would have to involve one of the issues you've pointed out (there are some components I build that require it, erroneously IMO).
In general, I'd never do it for performance only, it would have to involve one of the issues you've pointed out (there are some components I build that require it, erroneously IMO).
In some cases, it certainly is worth the improvement. Performance improvements translate into $$$.
Knowing that "LC_ALL=C can improve performance but cause UTF problems" is awesome information to base a design on. A blanket statement like "never use LC_ALL=C" is not.
Knowing that "LC_ALL=C can improve performance but cause UTF problems" is awesome information to base a design on. A blanket statement like "never use LC_ALL=C" is not.
Short of a bug or other pathological issue, if you resort to messing with locale just to avoid a performance problem, you're likely to hide other issues that will cause problems whenever another user attempts to do the same thing and it breaks.
Again, not worth it -- if only for performance sake, generally speaking.
Again, not worth it -- if only for performance sake, generally speaking.
The question is, do you need to handle UTF-8? In most cases of sever management having LATIN1 would suffice, or appropriate localized 8-byte encoding.
You can apply the same argument to any other obscure encoding scheme, not just UTF-8. The answer is really to pick the encoding which will suite the most usecases and won't be painful to use.
You can apply the same argument to any other obscure encoding scheme, not just UTF-8. The answer is really to pick the encoding which will suite the most usecases and won't be painful to use.
How is ASCII or Latin-1 somehow not an obscure encoding scheme by your parlance? Latin-1 is less common by quite a bit on the web than UTF-8, and ASCII might be if UTF-8 wasn't a superset.
Fact of the matter is people have to get out of the habit of going "oh, that will never be relevant for me", because a) that's unlikely to be true for everybody (for example in a server environment you can't even write perfectly good North American names like Étienne let alone perfectly good older-than-the-country names like Þórr) b) it's unlikely to be true even for you as you will discover the first time you try to read a file written by somebody else c) it promotes intellectual laziness that makes later internationalization efforts difficult to impossible, because they're trying to convince people like you to care about stuff they find inconvenient but that is of vital importance to your customers.
I know of a man whose first name is Þórr who outright refuses to deal with any organization that will not let him write his name, and I have a hard time disagreeing with him.
Fact of the matter is people have to get out of the habit of going "oh, that will never be relevant for me", because a) that's unlikely to be true for everybody (for example in a server environment you can't even write perfectly good North American names like Étienne let alone perfectly good older-than-the-country names like Þórr) b) it's unlikely to be true even for you as you will discover the first time you try to read a file written by somebody else c) it promotes intellectual laziness that makes later internationalization efforts difficult to impossible, because they're trying to convince people like you to care about stuff they find inconvenient but that is of vital importance to your customers.
I know of a man whose first name is Þórr who outright refuses to deal with any organization that will not let him write his name, and I have a hard time disagreeing with him.
>I know of a man whose first name is Þórr who outright
>refuses to deal with any organization that will not let him
>write his name, and I have a hard time disagreeing with him.
How does he fly? All flights I've ever been on have required me to ASCII-fy my name.
How does he fly? All flights I've ever been on have required me to ASCII-fy my name.
He mentioned being somewhat forgiving when people invariably screwed it up to 'rr' or 'orr', but I think he was Norwegian. So this may be a much more common problem with accepted solutions in his area of Europe. What drove him up the wall was when he typed his name in and the web site would be all "illegal characters in name".
The original question was about the command line. When managing servers it's perfectly accessible to use LATIN1 if one does not expect to deal with UTF-8 data (examining processes, VM states, changing configuration, etc). If that's not the case, one can use UTF-8 or some other encoding for that matter (i.e. I routinely have to deal with non-UTF non-ASCII filenames and text, using UTF-8 won't cut here either).
> if one does not expect to deal with UTF-8 data
This is a horrible assumption. UTF8 can show up just about anywhere. 99% of the utilities mentioned in the article support UTF8, which means they will output UTF8 if they think that's a good idea. And that can show up at any point. Exotic filenames, error messages, fancy boxing or indicative characters in the output...
Don't expect Latin-1, ever. Respect specified encodings, and default to UTF8. Never, ever think you'll be fine with Latin-1 when dealing with an UTF-8 output just because you speak english.
The trick the original article specified is a hack to get better performance in very controlled situations, it should never be put in a rc file like it recommends because it will break and you won't know that is what's causing it.
Edit: Here's a few examples of very real potential breakage, to show you how bad an idea this is: You're debugging a python script. You open a python shell and start pasting some of its lines to figure out what's happening. OOPS, there's UTF-8 in there.
Should I keep going? See how "awk '{print tolower($0)}'" behaves when it's dealing with utf-8 input. Maybe that awk is deeply hidden in a 5000 line script. Maybe it deals with web data, filenames, anything. And suddenly you have unicode escapes in data that doesn't expect any.
Now how's that one line in your rc file, which you wrote maybe months or years ago, working out for you? Are you going to know the issue is your LC_ALL variable?
This is a horrible assumption. UTF8 can show up just about anywhere. 99% of the utilities mentioned in the article support UTF8, which means they will output UTF8 if they think that's a good idea. And that can show up at any point. Exotic filenames, error messages, fancy boxing or indicative characters in the output...
Don't expect Latin-1, ever. Respect specified encodings, and default to UTF8. Never, ever think you'll be fine with Latin-1 when dealing with an UTF-8 output just because you speak english.
The trick the original article specified is a hack to get better performance in very controlled situations, it should never be put in a rc file like it recommends because it will break and you won't know that is what's causing it.
Edit: Here's a few examples of very real potential breakage, to show you how bad an idea this is: You're debugging a python script. You open a python shell and start pasting some of its lines to figure out what's happening. OOPS, there's UTF-8 in there.
[4:08:48] adys@azura ~ % python -c 'print("I like the letter Š")'
I like the letter Š
[4:08:54] adys@azura ~ % LANG=C python -c 'print("I like the letter Š")'
Unable to decode the command from the command line:
UnicodeEncodeError: 'utf-8' codec can't encode character '\udcc5' in position 25: surrogates not allowed
Should I keep going? See how "awk '{print tolower($0)}'" behaves when it's dealing with utf-8 input. Maybe that awk is deeply hidden in a 5000 line script. Maybe it deals with web data, filenames, anything. And suddenly you have unicode escapes in data that doesn't expect any.
Now how's that one line in your rc file, which you wrote maybe months or years ago, working out for you? Are you going to know the issue is your LC_ALL variable?
There are multiple issues you are dealing with here. First, no tools "will output UTF8 if they think that's a good idea". All properly designed application are required to respect the LANG setting (or equivalent LC_XXX environment variables) and use character encoding specified in them. This is the basis of character encoding support in most (all?) modern UNIXes. There's no risk running with LANG set to something other than en_EN.UTF-8 when using properly implemented software (most of it), and as a matter of fact a lot of UNIX variants has C as default (actually, the only two I know which don't are Linux and OS X). I have been running with LANG set to C (or KOI8-R on OS X) on FreeBSD, Linux and OS X for more than 10 years, and yet to hit the problem. Now, there are always issues with dealing with UTF-8 data (or different 1 byte encoding to a lesser extent), but see below.
Second, you keep treating UTF-8 as some special encoding (ein OS, ein encoding?), but it's certainly not. Even if you are using UTF-8 as a system encoding, you are going to have problems dealing with non-UTF8 data, and it's actually going to be worse than dealing with UTF-8 when using LATIN1. Mind you, most systems don't use UTF-8 as default encoding: most Linux distributions do, but Windows uses UTF-16, Mac OS X has it's own somewhat incompatible version of UTF-8, Java is UTF-16, and so on. One will have problems dealing with all these systems if his encoding is set to UTF-8.
Lastly, the Python example is not really representative. Some language implementations have better support for multi-byte encodings, and some worse. Ruby, for example, has a very good support for dealing with UTF-8 data even when not using it a system encoding.
Second, you keep treating UTF-8 as some special encoding (ein OS, ein encoding?), but it's certainly not. Even if you are using UTF-8 as a system encoding, you are going to have problems dealing with non-UTF8 data, and it's actually going to be worse than dealing with UTF-8 when using LATIN1. Mind you, most systems don't use UTF-8 as default encoding: most Linux distributions do, but Windows uses UTF-16, Mac OS X has it's own somewhat incompatible version of UTF-8, Java is UTF-16, and so on. One will have problems dealing with all these systems if his encoding is set to UTF-8.
Lastly, the Python example is not really representative. Some language implementations have better support for multi-byte encodings, and some worse. Ruby, for example, has a very good support for dealing with UTF-8 data even when not using it a system encoding.
> I know of a man whose first name is Þórr who outright refuses to deal with any organization that will not let him write his name, and I have a hard time disagreeing with him.
I wish I had that luxury sometimes. Here's a pet peeve. I live in France, where people expect everyone to have a one-word family name (it seems). It's really common for people and especially computer systems to "correct" my name (which is of the form Me van der Somebody) to something like Me VANDERSOMEBODY or Me Van Der Somebody. A particular web page (my employer's, actually -- where I have to fill in my name for my "profile page") does the latter, and refuses to allow me to fix it. I have submitted bug reports, and just get WONTFIX back. :(
It's really rude, so I can imagine that for a person called þórr it's a nightmare everywhere except Iceland.
I wish I had that luxury sometimes. Here's a pet peeve. I live in France, where people expect everyone to have a one-word family name (it seems). It's really common for people and especially computer systems to "correct" my name (which is of the form Me van der Somebody) to something like Me VANDERSOMEBODY or Me Van Der Somebody. A particular web page (my employer's, actually -- where I have to fill in my name for my "profile page") does the latter, and refuses to allow me to fix it. I have submitted bug reports, and just get WONTFIX back. :(
It's really rude, so I can imagine that for a person called þórr it's a nightmare everywhere except Iceland.
I have the following set in my .cshrc:
setenv LC_ALL en_US.UTF-8
setenv LC_COLLATE C # use the ASCII sort order
This won't work because LC_ALL overrides LC_COLLATE (and all the LC_* variables actually). It's LANG that you want to use. LANG is overriden by LC_* which are in turn overriden by LC_ALL. Don't use LC_ALL in your config files, use it only for one shot command lines or in scripts that need to be sure of the environment they run in.
Thank you both for the clue. I used to have LANG set, but I'm just going to unset them all. I think I put them in originally to change how directory listings got sorted, but I'm not sure I care any more. :)
You're not breaking anything on a technical level with that. Be wary of anything that deals with string or character comparison however (including regexes).
Some environments support a locale named "C.UTF-8" - with the same familiar sorting and formatting rules as the C locale, but with proper UTF-8 support.
Using C locale is useful when processing very large data files in a shell, e.g. on a server. For example, uniquing or set difference will work many times faster (and doesn't require the slow/fancy collation calculations). But yes, it's a good point, of course it shouldn't be used on your desktop globally or apps will break. I'll clarify that.
Setting LC_COLLATE=C and leaving the rest to a proper locale can often win you what you want without losing much.
Thanks! Didn't think of filing it as an issue.
> StrictHostKeyChecking=no
And welcome to MITM haven. This is awful advice if you care about the first S in SSH.
And welcome to MITM haven. This is awful advice if you care about the first S in SSH.
Depends on your circumstances, really. If you're in an environment with lots of VM's floating around, being created, destroyed, and so on, dealing with SSH's paranoia (why can't I just dismiss a warning about a host/key mismatch instead of having to edit .known_hosts?) quickly becomes an exercise in frustration for dubious security benefit. (If the enemy is on your LAN and able to manipulate your DNS, you've already lost)
On your home box, sure. But let's not pretend there's not a reason for the option to exist.
On your home box, sure. But let's not pretend there's not a reason for the option to exist.
I use an alias to select when to, and not to, use host keys;
alias ssht='ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no'
ssht floaty.vm # does not use host key checking
ssh my.bastion.host # validates host key
Really I ought to get SSHFP records populated when my vm's are created...
alias ssht='ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no'
ssht floaty.vm # does not use host key checking
ssh my.bastion.host # validates host key
Really I ought to get SSHFP records populated when my vm's are created...
In these environments you can use server certificates
https://www.digitalocean.com/community/tutorials/how-to-crea...
https://www.digitalocean.com/community/tutorials/how-to-crea...
In which case, just use telnet. Seriously.
What would be good is an interactive confirmation when a host fingerprint has changed, instead of a flatout abortion and having to edit your `known_host` file. I have to SSH into hundreds/thousands of VM that reuse the same IPs/FQDN sometimes, and the strict host checking is a major PITA. I haven't turned it off tho, I prefer the pain to MITM.
Don't use SSH in a trust-on-first-use configuration in that situation.
There are two other modes to use. Either DNSSEC with SSHFP fingerprints, or sign your host keys with a CA key that you install on all your machines.
A decent configuration management tool could also keep known_hosts up to date, but it tends not to be worth the trouble. Do the above first.
There are two other modes to use. Either DNSSEC with SSHFP fingerprints, or sign your host keys with a CA key that you install on all your machines.
A decent configuration management tool could also keep known_hosts up to date, but it tends not to be worth the trouble. Do the above first.
I wonder if SSHFP records on Avahi/mDNS would work?
If you learn bash, and you learn vi, then the next most glorious addition is:
set -o vi
Then you have vi keys in your shell. And it is marvelous.
set -o vi
Then you have vi keys in your shell. And it is marvelous.
That's nice, but it only affects the command line.
Instead of that, create the following file:
~/.inputrc (tilde slash dot-inputrc)
And in that file put at least this:
set editing-mode vi
Now every program that you run that a) has its own command line, and b) uses the readline library (there are a lot) has vi commandline editing. psql, mysql, telnet, lftp, and many more.
man bash has a section on READLINE. For example, in man bash search for
editing-mode
Instead of that, create the following file:
~/.inputrc (tilde slash dot-inputrc)
And in that file put at least this:
set editing-mode vi
Now every program that you run that a) has its own command line, and b) uses the readline library (there are a lot) has vi commandline editing. psql, mysql, telnet, lftp, and many more.
man bash has a section on READLINE. For example, in man bash search for
editing-mode
or if you learn emacs you have emacs shortcuts by default in shells.
which goes against his childish comment on emacs:
> Learn Vim (vi). There's really no competition for random Linux editing (even if you use Emacs, a big IDE, or a modern hipster editor most of the time).
which goes against his childish comment on emacs:
> Learn Vim (vi). There's really no competition for random Linux editing (even if you use Emacs, a big IDE, or a modern hipster editor most of the time).
I would argue it's not childish and is a strong point considering emacs is not typically installed by default. Vim isn't usually installed by default either, gotta go with good ol' vi, unless your distro maintainers are nice and symlink vim to vi for you until you install vim.
At first I thought the same as you, but then I realised that he's not talking about your average editor, but about which editor to use if you SSH into a random web server etc. The thing you use in a foreign environment. That's what "random Linux editing" probably means.
This. On your home computer, use whatever you want. If you're using all sorts of foreign environments, you're going to want to use whatever is common to all of them so that you don't fumble around badly trying to do basic tasks. Learn vi. You don't have to like it, you don't have to use it on your home system, and you don't have to learn the eleventy-billion vim-efficiency-hacks floating around the Internet. But learn it well enough that if you have to ssh into some server that only has vi installed on it, you aren't rendered completely helpless. Even "Notepad Capability" is fine.
Can you share a little more info about this? I tried it but found it rather confusing.
Also, for more readline maneuvers: http://unix.stackexchange.com/questions/21788/how-to-delete-...
Also, for more readline maneuvers: http://unix.stackexchange.com/questions/21788/how-to-delete-...
`set -o vi` allows us to use `vi` bindings instead of the default `emacs` bindings.
The way to think about it is that the prompt starts out in `vi insert mode` so whatever we type in will just be inserted at the prompt. However, we can hit the`Escape` key and move into `vi normal mode` where the keys are interpreted as normal `vi` commands. So, for example, once we hit `esc` we can navigate around the line using `h` and `l` and navigate history (previous lines) using `k` and `j`.
Commands I find very useful are:
`jk` : for history navigation
`bw`: for moving in the current line. These are word movement commands that are faster to navigate with.
`cw` : similarly, changing a particular word in the current line is also great. This works particularly well with history - retrieve a previous command and change a word or option
`dw` : for deleting the current word
`/` : is a fantastic command for searching history. `/cd` will bring up our last change directory command and just hitting `n` will cycle through all our change directory commands that are in history.
`AI` : to insert at the beginning/end of line
...and so on. If you have `vi` muscle memory it's a fantastic option for working on the shell.
(edit: Formatting)
The way to think about it is that the prompt starts out in `vi insert mode` so whatever we type in will just be inserted at the prompt. However, we can hit the`Escape` key and move into `vi normal mode` where the keys are interpreted as normal `vi` commands. So, for example, once we hit `esc` we can navigate around the line using `h` and `l` and navigate history (previous lines) using `k` and `j`.
Commands I find very useful are:
`jk` : for history navigation
`bw`: for moving in the current line. These are word movement commands that are faster to navigate with.
`cw` : similarly, changing a particular word in the current line is also great. This works particularly well with history - retrieve a previous command and change a word or option
`dw` : for deleting the current word
`/` : is a fantastic command for searching history. `/cd` will bring up our last change directory command and just hitting `n` will cycle through all our change directory commands that are in history.
`AI` : to insert at the beginning/end of line
...and so on. If you have `vi` muscle memory it's a fantastic option for working on the shell.
(edit: Formatting)
Agreed. I made a pull-request for this.
https://github.com/jlevy/the-art-of-command-line/pull/9
https://github.com/jlevy/the-art-of-command-line/pull/9
Learning about this was a revelation for me. I can't live without it now.
> Fluency on the command line is a skill that is in some ways archaic...
I don't see this to be the case at all. Plus, having this be the opening line may cause people to equate archaic=I don't need this.
I'd suggest (would pull request but afk) that you remove "is a skill that is in some ways archaic".
I don't see this to be the case at all. Plus, having this be the opening line may cause people to equate archaic=I don't need this.
I'd suggest (would pull request but afk) that you remove "is a skill that is in some ways archaic".
Agree. I've been hearing people saying the command line is archaic since Windows 95, and probably before that. Yet somehow I (any everyone I work with, and have worked with over the past 25 years) still works in a shell daily. I don't understand how someone on the technical end of running large internet services could avoid it.
Sure, GUI tools have crept in here and there. And I understand that some toolchains mandate all-singing, all-dancing IDEs. I don't work in those areas (the closest I've come was doing Java work for about five years and I still lived in vi, because Eclipse makes me homicidal). And I still see our iOS developers using `find' and `sed'.
Which isn't to say GUIs can't improve on the command line for some things. For example, the MacOS tool "A Better Finder Rename" (despite its inability to rename itself something less clunky) is a great tool for mass-renaming, e.g., photos. I have written such critters before as shell/perl/python scripts, but pulling EXIF data and renaming files programmatically is ticky and potentially destructive enough that I tend to write it once and then not modify it unless it breaks. The GUI preview and regex capture display makes this much less annoying.
Sure, GUI tools have crept in here and there. And I understand that some toolchains mandate all-singing, all-dancing IDEs. I don't work in those areas (the closest I've come was doing Java work for about five years and I still lived in vi, because Eclipse makes me homicidal). And I still see our iOS developers using `find' and `sed'.
Which isn't to say GUIs can't improve on the command line for some things. For example, the MacOS tool "A Better Finder Rename" (despite its inability to rename itself something less clunky) is a great tool for mass-renaming, e.g., photos. I have written such critters before as shell/perl/python scripts, but pulling EXIF data and renaming files programmatically is ticky and potentially destructive enough that I tend to write it once and then not modify it unless it breaks. The GUI preview and regex capture display makes this much less annoying.
Comparing your first sentence and that of the author, you're missing what I see as the key word: "skill". I use command line everyday. Probably most developers do as well. But I definitely don't have "skill" in it. I think the point is that most developers can get by with nothing more than the basics (whatever "basics" are in the context of their work). The command line isn't archaic, but command line as a fluency might be.
It's a fun experience trying to introduce command lines to people who have lived in IDE-land all their lives.
"Command Line?! HAHAHAH"
"Command Line?! HAHAHAH"
Just throwing this http://mywiki.wooledge.org/BashGuide (it's listed 2 indirections deep so will probably be missed)
Every time you have an issue with the shell, go there.
Every time you have an issue with the shell, go there.
The Linux Command Line (free pdf): http://linuxcommand.org/tlcl.php
Hosted on Sourceforge which is now blocked almost everywhere.
You're exactly right tomaac. Thank you for your comment.
The reason: http://arstechnica.com/information-technology/2015/05/source...
The reason: http://arstechnica.com/information-technology/2015/05/source...
The author should prepare to catch some hell for advocating 'ForwardAgent=yes' in ssh configs.
I had meant to take another look at ProxyCommand (http://heipei.github.io/2015/02/26/SSH-Agent-Forwarding-cons...), so thanks for the reminder!
If you're into the command line, I'd recommend getting a physical copy of Unix Power Tools [1] and spending some time with it. This is a nice article, but Unix Power Tools is better in almost every way for learning the basics (and more) of the Unix command line. This article mentions a few more modern tools, but Unix Power Tools has a far better explanation of what's going on.
[1] http://www.amazon.com/Unix-Power-Tools-Third-Edition/dp/0596...
[1] http://www.amazon.com/Unix-Power-Tools-Third-Edition/dp/0596...
One really useful shortcut I learned in bash is Ctrl-/ which is a sort of undo for bash when composing a command. Can't really explain how it works, you should try it out yourself.
Also really handy is Ctrl-Y, which pastes the last cut characters by Ctrl-W, Ctrl-U or Ctrl-K.
Also really handy is Ctrl-Y, which pastes the last cut characters by Ctrl-W, Ctrl-U or Ctrl-K.
C-/ and C-y are from readline which mimicked them (and many more) from Emacs.
Unix tools are very powerful, but they are not as intuitive as I would like. It would be nice if we can have an httpie for process management, one for text manipulation, another for system stats, and so on.
Powerful tools are rarely instinctive. Quite often, one person's obvious is another's opaque.
So this is tangentially related by why do people seem to push Vi? Nano is a perfectly serviceable editor. If I need to do extensive editing I always end up loading it into Atom, Sublime or some other editor anyway.
I've never really had to use a console editor for more than 10s of lines.
I've never really had to use a console editor for more than 10s of lines.
Nano is fine for 10s of lines, but Vim is great for much much more than that. It's also nice to have the same keybindings for every console app (gosh I love readline), and it's repeat commands make editing text an absolute breeze. Why open another editor on my host if I can edit the file in my console on a remote server?
vi keybindings are ubiquitous, so familiarity with the editor is beneficial outside the editor. Just offhand, both ncdu and tig (which I have used in the past hour) respond to vi keystrokes.
It's not as hard as it first seems to learn, either. Well worth the time spent, vi is worlds more powerful than nano.
It's not as hard as it first seems to learn, either. Well worth the time spent, vi is worlds more powerful than nano.
Nano is not as ubiquitous as vi. Every single *nix box will have vi on it, so even if you prefer something else, it's still in your best interest to learn the basics of vi.
> If I need to do extensive editing I always end up loading it into Atom, Sublime or some other editor anyway.
So you're saying "why would you use X, when you could avoid that by using Y"?
Well, because people prefer X, for a variety of reasons. Not to mention vi is installed everywhere, while sublime and atom are not.
So you're saying "why would you use X, when you could avoid that by using Y"?
Well, because people prefer X, for a variety of reasons. Not to mention vi is installed everywhere, while sublime and atom are not.
nano is super easy to use, I agree. I tried vim and gave up. Will try again.
I find it a bit irksome that when discussing command line editors, the dichotomy appears to be be "nano vs vim".
There are other CLI editors. I'm quite partial to ne (the Nice Editor), which has more manageable shortcuts than emacs, a command line (it's still a non-modal editor, to be clear), macros and syntax highlighting. Also jed is not bad. While vim (and nvim) have advantages, and certainly benefit from being ubiquitous and having ther shortcuts replicated in other programs, users should shop around, even on the command line.
There are other CLI editors. I'm quite partial to ne (the Nice Editor), which has more manageable shortcuts than emacs, a command line (it's still a non-modal editor, to be clear), macros and syntax highlighting. Also jed is not bad. While vim (and nvim) have advantages, and certainly benefit from being ubiquitous and having ther shortcuts replicated in other programs, users should shop around, even on the command line.
In modern Bourne shell derivatives (Bash, ksh, some POSIX shells), the command:
set -o emacs
Will give command line editing bindings compatible with Emacs (which is also the default editing keys in MS-Windows and many IDE's BTW).
HTH
set -o emacs
Will give command line editing bindings compatible with Emacs (which is also the default editing keys in MS-Windows and many IDE's BTW).
HTH
Emacs bindings are not at all compatible with MS Windows.
You're right. I had misremembered thinking that the control keys often exposed in various GUI's were the same as the cmd ones. Which they are not:
http://ss64.com/nt/syntax-keyboard.html
Also, the original comment was on "console editors" and not on "command line editing", which is what I thought the subject was about. Looks like I struck out on the whole comment.
http://ss64.com/nt/syntax-keyboard.html
Also, the original comment was on "console editors" and not on "command line editing", which is what I thought the subject was about. Looks like I struck out on the whole comment.
"I don't understand why everyone doesn't do things exactly like I do using only tools I like" is a really stupid way to look at the world.
Ironically, it's unclear whether you're countering the comment's argument, or the article's argument.
"Use vi, nothing beats it." == "I use x and I like it just fine."
If that was your intention, my mistake. It's just not apparent.
"Use vi, nothing beats it." == "I use x and I like it just fine."
If that was your intention, my mistake. It's just not apparent.
Fluency on the command line is a skill now often neglected or considered archaic
By whom? (honest question: even the most GUI oriented people I know reckon the power of command line skills)
By whom? (honest question: even the most GUI oriented people I know reckon the power of command line skills)
I work in a .Net house. When we had to move to front end development which required using grunt shock from the command line there was plenty of dissent at how we were being forced back into the "dark ages".
I'm glad the author took time to mention sl - every sysadmin should make sure that its installed on all the boxes they manage.
Good ole' steam locomotive command...
The new csysdig curses CLI [1] unifies and replaces a bunch of the system debugging tools listed here.
[1] https://github.com/draios/sysdig/wiki/Csysdig%20Overview
[1] https://github.com/draios/sysdig/wiki/Csysdig%20Overview
I'd never even thought about commenting a line I was half way through writing, when suddenly realising I'd forgotten the correct arguments.
I normally end up opening another terminal and using man there.
I normally end up opening another terminal and using man there.
I do this sometimes when I want to run some command first from earlier in my history, then the commented out command also stays in history so I can continue with it after I've re-run the earlier command first.
Much, much more can be found at:
http://www.commandlinefu.com/commands/browse
http://www.commandlinefu.com/commands/browse
Fun fact: alt+# is impossible on a Spanish keyboard, because alt+3 = #. So you actually need to hold alt to type the character in the first place.
> To locate a file by name in the current directory, find -iname something . (or similar). To find a file anywhere by name, use locate something (but bear in mind updatedb my not have indexed recently created files).
I think this will glob unless you escape the asterisks. Also on my system (debian 8) I need to put the directory to search first, or not at all:
find . -iname \something\
find -iname \something\
edit: hackernews ate all my asterisks
I think this will glob unless you escape the asterisks. Also on my system (debian 8) I need to put the directory to search first, or not at all:
find . -iname \something\
find -iname \something\
edit: hackernews ate all my asterisks
> I think this will glob unless you escape the asterisks.
The majority of times, this is the case. At least one exception is when the shell pattern does not match one or more files in the directory which find(1) is executed. However, this can be "surprising" so it's best practice to either use escapes or ensure interpolation is disabled via single quotes (in Bourne shell syntax).
> Also on my system (debian 8) I need to put the directory to search first...
FWIW, you can specify more than one directory if desired. The primaries will be applied to the results of each.
The majority of times, this is the case. At least one exception is when the shell pattern does not match one or more files in the directory which find(1) is executed. However, this can be "surprising" so it's best practice to either use escapes or ensure interpolation is disabled via single quotes (in Bourne shell syntax).
> Also on my system (debian 8) I need to put the directory to search first...
FWIW, you can specify more than one directory if desired. The primaries will be applied to the results of each.
> The majority of times, this is the case. At least one exception is when the shell pattern does not match one or more files in the directory
Thankfully zsh has the NULLGLOB option which will do the right thing and expand to an empty list, of the zero matching files.
Thankfully zsh has the NULLGLOB option which will do the right thing and expand to an empty list, of the zero matching files.
This depends a lot on the terminal, and my advise is to always suround the glob with quotes:
find . -iname "*file*"
edit: hackernews ate all my asterisks
cat hosts | xargs -I{} ssh root@{} hostname
echo y | xargs -Ix echo x
I agree, this is one of my favorite tricks. I like to think of xargs as a `map` function over stdin lines.
As the article mentions, especially useful when combined with the -P flag for paralellism.
As the article mentions, especially useful when combined with the -P flag for paralellism.
cat urlpaths.txt | xargs -I{} -P wget domain.com/{}
cat dirs.txt | xargs -I{} sh -c 'cd {}; touch file;'Seems like a good introduction. I'm an amateur who's been using IDEs up until now. I'm just starting to become dimly aware of the power at the command line, and slowly learning in a haphazard way. This could be very helpful.
Another one for obscure but useful: jot
# print 10 values from 1 to 10, with leading zeros:
for n in `jot -w "%04d" 10 1 10`; do echo $n; done
# print 10 values from 1 to 10, with leading zeros:
for n in `jot -w "%04d" 10 1 10`; do echo $n; done
Just use Bash's builtin things:
for n in {0001..0010}; do echo ${n}; done
for n in {0001..0010}; do echo ${n}; done
Nice, but this needs an actual version of bash, with that oldie shipped with OSX Yosemite the result is not padded with zeros.
Cool, thanks.
Or
seq -f %04g 1 10
Sure seq is OK, but jot does everything seq does and more, so I prefer jot.
# generate 10 random numbers between 1 and 1 million
jot -r 10 1 1000000
# generate 10 random numbers between 1 and 1 million
jot -r 10 1 1000000
I meant to go all "just use coreutils, there is shuf available already, no need for some third-party tool" but then I realised that jot is an integral part of BSD. Rock on! :)
For us Linuxers: shuf -i 1-1000000 -n 10
For us Linuxers: shuf -i 1-1000000 -n 10
No mention of autojump? :-p
I googled about autojump, then found fasd and decided to go with it.
https://github.com/clvv/fasd
https://github.com/clvv/fasd
pretty good writeup. Learning nohup was a life changing experience for me.
there's also disown. It's a bash builtin, so you'll have to type "help disown" ("man disown" gets you nothing). E.g. to run touchegg and close your terminal, keeping touchegg running:
touchegg & disown; exitI use an alias for it:
function launch {
type $1 >/dev/null || { print "$1 not found" && return 1 }
$@ &>/dev/null &|
}
alias launch="launch "why the alias as well as the function?
ah, thanks for that tip :-)
I learned a dozen new things in the first 100 lines or so, that will save me hours, and I've been living in bash for 20 years.
Thanks for this!
Thanks for this!
There's no reason to learn vi if you know Emacs. If you claim that Emacs isn't installed everywhere, guess what: Neither is vi.
If you want an "installed everywhere" editor, learn ed. If you're willing to take an editor with you (or otherwise make sure a specific editor is everywhere you are), there's no reason it has to be vi.
If you want an "installed everywhere" editor, learn ed. If you're willing to take an editor with you (or otherwise make sure a specific editor is everywhere you are), there's no reason it has to be vi.
I can't remember ever encountered a unix-like system without "vi" installed. I've used (and set up) machines running FreeBSD, DragonflyBSD, OSX, Ubuntu, Debian, RHEL, ...
Plenty of those didn't have emacs, especially on first boot, when you're most likely to need to modify network configuration files in order to get online to install new packages in the first place...
Edit: By "vi" I mean that somewhere in the path there is a program called vi. Of course, usually this is nvi, vim, evil, etc; I don't mean the original UNIX vi binary.
Plenty of those didn't have emacs, especially on first boot, when you're most likely to need to modify network configuration files in order to get online to install new packages in the first place...
Edit: By "vi" I mean that somewhere in the path there is a program called vi. Of course, usually this is nvi, vim, evil, etc; I don't mean the original UNIX vi binary.
I can't remember ever encountered a unix-like system without "vi" installed.
It's often/always part of a full install, but it isn't always there when you end up in rescue mode and/or situations where /usr won't mount.
It's often/always part of a full install, but it isn't always there when you end up in rescue mode and/or situations where /usr won't mount.
Not on OpenBSD on rescue mode, or when booting from bsd.rd. Pretty sure the same applies to several others.
However, ed is available, and vi is closer to ed than emacs is. And emacs won't be available either.
However, ed is available, and vi is closer to ed than emacs is. And emacs won't be available either.
> I can't remember ever encountered a unix-like system without "vi" installed.
Default Ubuntu install.
Default Ubuntu install.
Ubuntu 14.04, the default Ubuntu on EC2, definitely comes with vi.
Wow, seriously? I stand corrected -- still much more likely to find than emacs.
Ubuntu (and Debian) comes with vi (not vim) by default. It's just not the default editor.
vim.tiny is installed by default, though I don't believe the vi symlink is installed by default.
vi is a posix requirement, emacs is not. If the system you're working on has ed, it'll have some version of vi or vim as well.
and not having to know ed is an excellent reason to learn vi :)
obCompulsoryEdJoke: https://www.gnu.org/fun/jokes/ed-msg.html
obCompulsoryEdJoke: https://www.gnu.org/fun/jokes/ed-msg.html
a
I learned ed for a joke and found myself actually liking some features so much that I started porting them to emacs.
.
w
I learned ed for a joke and found myself actually liking some features so much that I started porting them to emacs.
.
w
I've used ed(1) to do complex(ish) programmatic edits to files that I think would have been extremely painful otherwise. An interesting editor worth spending at least a small amount of brainpower on.
That was pretty damm amusing.
I don't think this is true. I've certainly been in a bunch of hairy situations over the years where I had nothing fancier than ed available to me because I was in some sort of recovery mode. The single user mode that FreeBSD comes with is lacking vi if, for example, you don't have / mounted yet. Thankfully ed is in the ramdisk.
V7/x86 [1] has both vi and ed, but ed is easier to get to.
"Of course, we are not absolutely compelled to use ed, since V7/x86 comes with the full-screen editor vi (as well as the related line editor ex), but getting access to vi in this context would involve mounting the /usr filesystem and changing various environment variables and settings to allow editing in full-screen mode. Some basic ed skills are particularly useful in the V7 world, and most of all when one is involved on system administration tasks." [2]
[1] http://www.nordier.com/v7x86/
[2] http://www.nordier.com/v7x86/doc/v7x86intro.pdf
"Of course, we are not absolutely compelled to use ed, since V7/x86 comes with the full-screen editor vi (as well as the related line editor ex), but getting access to vi in this context would involve mounting the /usr filesystem and changing various environment variables and settings to allow editing in full-screen mode. Some basic ed skills are particularly useful in the V7 world, and most of all when one is involved on system administration tasks." [2]
[1] http://www.nordier.com/v7x86/
[2] http://www.nordier.com/v7x86/doc/v7x86intro.pdf
Not in Ubuntu.
Anyone considering using vi should be warned away by this practice of "disagreement by downvoting". If they do it here, imagine what they would do to a bug report!
I much prefer emacs and at least emacs shortcuts are by default in most shell (and everywhere in osx!) but no, every servers I install (debian, ubuntu, osx...) comes with vi but not emacs.
Before that I was not so familiar with cmd. After that I was.
Do not. Having a non-utf8 locale means you won't be able to handle utf-8 sanely ("that's why it's faster") and it will break at the most inexplicable times. Any non-latin1 character appearing in your prompt or command line with this will mess its spacing up for example. Do not do not do not.
Hell I even check for it in my .zshrc: https://github.com/jleclanche/dotfiles/blob/master/.zshrc#L3...
Good post otherwise.