In an earlier post I wrote about pv -d. Here are some more tricks that can be even easier to use.

Reading to stdout with pv instead of cat

Reading files to stdout can be done with pv instead of cat. pv will also print transfer rate, ETA and a progress bar to stderr.

For example, counting packets in a pcap file:

➜  ls -lh
total 1.0G
-rw-rw-r-- 1 assafmo assafmo 1.0G Jun 10 20:51 my.pcap
➜  pv my.pcap | tcpdump -r /dev/stdin -qn | wc -l
reading from file /dev/stdin, link-type EN10MB (Ethernet)
512MiB 0:00:10 [26.3MiB/s] [================>                 ] 50% ETA 0:00:09

Result:

➜  pv my.pcap | tcpdump -r /dev/stdin -qn | wc -l
reading from file /dev/stdin, link-type EN10MB (Ethernet)
1023MiB 0:00:19 [44.8MiB/s] [================================>] 100%
8635943

Using pv -s in between pipes

Placing pv in between pipes will make it read from stdin and print to stdout (just like cat), and also print the transfer rate to stderr.

By using the -s flag we can tell pv the total size of the data being transferred, thus pv will also print progress and ETA to stderr.

For example, merging multiple pcap files together:

➜  ls -lh
total 3.0G
-rw-rw-r-- 1 assafmo assafmo 1.0G Jun 10 20:51 1.pcap
-rw-rw-r-- 1 assafmo assafmo 1.0G Jun 10 20:51 2.pcap
-rw-rw-r-- 1 assafmo assafmo 1.0G Jun 10 20:51 my.pcap
➜  joincap *.pcap | pv -s $(du -bc *pcap | tail -1 | cut -f 1) > merged.pcap
1.16GiB 0:00:04 [ 275MiB/s] [============>                    ] 38% ETA 0:00:06

Note: du -bc *pcap | tail -1 | cut -f 1 gives us an approximation for the size of merged.pcap. This is good enough, as pv -s won’t crash if the data is bigger or smaller, it’ll just be wrong (exit before or go over 100%).

Using parallel --bar to count done jobs

Finally, GNU parallel can print how many jobs are done and an ETA with the --bar flag.

For example, counting packets in multiple pcaps:

➜  ls *pcap | parallel --bar 'tcpdump -r "{}" -qn | wc -l > "{}".count'
66% 2:1=1s my.pcap