0

I am new to learning Linux cli and I encountered an interesting problem.

I created a 1.txt file such:

1
1
2
2
3

When I use uniq 1.txt it emits 1 2 3. I decided to output this result to 1.txt, so this small trick would delete all the duplicating elements in the file and save it. To do that I used this command : uniq 1.txt > 1.txt. But in the end it shows that 1.txt file is totally empty. Can anyone help me to understand what happened?

Charles Duffy
  • 280,126
  • 43
  • 390
  • 441
Hayk Arabyan
  • 37
  • 3
  • That's not just uniq, *all* commands behave that way; redirections start before everything else, so they wipe the content before they can be read. – Charles Duffy Sep 23 '19 at 21:25

1 Answers1

0

Typically in bash, you should avoid piping to a file being used in a preceding command. The piping (from >) is often working in parallel with the command (in this case, uniq), so unexpected behavior can occur. Some commands like sed and perl have options like -i that allows inplace file manipulation, but generally it's usually safest just to assume it's not allowed. The dirty workaround is something like so:

uniq 1.txt > tmpfile
mv tmpfile 1.txt
Jason K Lai
  • 1,500
  • 5
  • 15
  • Please flag duplicates as such rather than answering them; see the third bullet point in the *Answer Well-Asked Questions* section of [How to Answer](https://stackoverflow.com/help/how-to-answer). – Charles Duffy Sep 23 '19 at 21:26
  • Part of the value of redirecting to existing questions is that they have answers which have already been vetted for correctness, commented and voted on, etc. As currently written, this answer is incorrect: In a simple command such as `foo > bar`, `foo` is started only **after** `bar` is open attached to its stdout. Thus, the redirection isn't "in parallel" (and thus subject to a race condition which might be won or lost); the redirection (truncating the output file) happens *first* 100% of the time. – Charles Duffy Sep 23 '19 at 21:27
  • (There are other cases where you might have a race -- in `foo baz | bar > baz`, some shells might create a FIFO for the stdout of `foo`, start `foo`, and in parallel open `baz` for output, truncating it, and then -- after `baz` is opened -- run `bar` with the other side of the FIFO attached to stdin; in that case, it's unlikely but theoretically possible for `foo` to read some of the original file contents before the shell truncates it before starting up `bar`; however, no such race condition exists in `uniq 1.txt >1.txt`, in which case the file will **always** be truncated before it is read). – Charles Duffy Sep 23 '19 at 21:34