top | item 33263004

(no title)

JayGuerette | 3 years ago

pv is a great tool. One of it's lesser known features is throttling; transfer a file without dominating your bandwidth:

pv -L 200K < bigfile.iso | ssh somehost 'cat > bigfile.iso'

Complete with a progress bar, speed, and ETA.

discuss

order

smcl|3 years ago

Oh damn that's neat I never thought to use `ssh` directly when transferring a file, I always used `scp bigfile.iso name@server.org:path/in/destination`

fbergen|3 years ago

Also see `scp -l 200 bigfile.iso name@server.org:path/in/destination`

from man page:

-l limit

Limits the used bandwidth, specified in Kbit/s.

tyingq|3 years ago

A similar trick that's nice is piping tar through ssh. Handy if you don't have rsync or something better around. Even handy for one file, since it preserves permissions, etc.

tar -cf - some/dir | ssh remote 'cd /place/to/go && tar -xvf -'

dspillett|3 years ago

Similarly, though useful less often these days, using -B/--buffer-size to increase the amount that it can buffer. If reading data from traditional hard drives, piping that data through some process, and writing the result back to the same drives, this option can increase throughput significantly by reducing head movements. It can help on other storage systems too, but usually not so much so.