cp from usbdisk using devstream writes until no space left

Sun, 4 Oct 2015 20:16:41 EDT
qu7uux@[REDACTED]

Using cp(1) with devstream to copy a file of a certain size from a usb disk to a local fs keeps copying bytes until there is no space left.

Tested on changeset 4917:e540b0bb61c8 with hjfs. Setup: – stream added to /sys/src/9/pc64/pc64 under dev – bind -a ‘#¶’ /fd – using cp(1) (not fcp or dircp, mv, etc.) – file size >= 16386 – copy file from usbdisk to local filesystem – usb disk accessed through shr(3) as /shr/sdU*/

Results (further below): – [1] if file size is >= 16386, cp always writes bytes until local disk is full. – [2], [3] if file size < 16386 or devstream not used, copy happens normally.

Notes: – can replicate this with 2 different usb sticks, on 2 different terminals – same result on amd64 and 386 both, arm not tested – cp to usbdisk works fine

creating example file containing a known pattern, size >= 16386

% dd -bs 16386 -count 1 </dev/zero >/shr/sdUce48e/asd 1+0 records in 1+0 records out % dd -bs 16 -count 1 -trunc 0 </dev/random >/shr/sdUce48e/asd 1+0 records in 1+0 records out % xd -r /shr/sdUce48e/asd 0000000 eb56839b 0b0454c7 204b594e 44dc078e 0000010 00000000 00000000 00000000 00000000 * 0004000 00000000 0004002

namespace

% pwd /usr/glenda % bind -a ‘#¶’ /fd % ns | grep fd bind ‘#d’ /fd bind -a ‘#¶’ /fd

[1] cp with devstream from usbdisk –> loop until disk full

% cp /shr/sdUce48e/asd . # interrupted with Kdel after a few secs % du -n asd 47898624 asd % xd -r asd | sed 100q 0000000 00eb5683 9b0b0454 c7204b59 4e44dc07 0000010 8e000000 00000000 00000000 00000000 0000020 00000000 00000000 00000000 00000000 * 0002000 eb56839b 0b0454c7 204b594e 44dc078e 0002010 00000000 00000000 00000000 00000000 * 0004000 eb56839b 0b0454c7 204b594e 44dc078e 0004010 00000000 00000000 00000000 00000000 * 0006000 eb56839b 0b0454c7 204b594e 44dc078e 0006010 00000000 00000000 00000000 00000000 * 0008000 eb56839b 0b0454c7 204b594e 44dc078e 0008010 00000000 00000000 00000000 00000000 * 000a000 eb56839b 0b0454c7 204b594e 44dc078e 000a010 00000000 00000000 00000000 00000000 * 000c000 eb56839b 0b0454c7 204b594e 44dc078e 000c010 00000000 00000000 00000000 00000000 * 000e000 eb56839b 0b0454c7 204b594e 44dc078e 000e010 00000000 00000000 00000000 00000000 * 0010000 eb56839b 0b0454c7 204b594e 44dc078e 0010010 00000000 00000000 00000000 00000000 * 0012000 eb56839b 0b0454c7 204b594e 44dc078e 0012010 00000000 00000000 00000000 00000000 * 0014000 eb56839b 0b0454c7 204b594e 44dc078e 0014010 00000000 00000000 00000000 00000000 * 0016000 eb56839b 0b0454c7 204b594e 44dc078e 0016010 00000000 00000000 00000000 00000000 * 0018000 eb56839b 0b0454c7 204b594e 44dc078e 0018010 00000000 00000000 00000000 00000000 * 001a000 eb56839b 0b0454c7 204b594e 44dc078e 001a010 00000000 00000000 00000000 00000000 * 001c000 eb56839b 0b0454c7 204b594e 44dc078e 001c010 00000000 00000000 00000000 00000000 * 001e000 eb56839b 0b0454c7 204b594e 44dc078e 001e010 00000000 00000000 00000000 00000000 * 0020000 eb56839b 0b0454c7 204b594e 44dc078e 0020010 00000000 00000000 00000000 00000000 * 0022000 eb56839b 0b0454c7 204b594e 44dc078e 0022010 00000000 00000000 00000000 00000000 * 0024000 eb56839b 0b0454c7 204b594e 44dc078e 0024010 00000000 00000000 00000000 00000000 * 0026000 eb56839b 0b0454c7 204b594e 44dc078e 0026010 00000000 00000000 00000000 00000000 * 0028000 eb56839b 0b0454c7 204b594e 44dc078e 0028010 00000000 00000000 00000000 00000000 * 002a000 eb56839b 0b0454c7 204b594e 44dc078e 002a010 00000000 00000000 00000000 00000000 * 002c000 eb56839b 0b0454c7 204b594e 44dc078e 002c010 00000000 00000000 00000000 00000000 * 002e000 eb56839b 0b0454c7 204b594e 44dc078e 002e010 00000000 00000000 00000000 00000000 * 0030000 eb56839b 0b0454c7 204b594e 44dc078e 0030010 00000000 00000000 00000000 00000000 * 0032000 eb56839b 0b0454c7 204b594e 44dc078e 0032010 00000000 00000000 00000000 00000000 * 0034000 eb56839b 0b0454c7 204b594e 44dc078e 0034010 00000000 00000000 00000000 00000000 * 0036000 eb56839b 0b0454c7 204b594e 44dc078e 0036010 00000000 00000000 00000000 00000000 * 0038000 eb56839b 0b0454c7 204b594e 44dc078e 0038010 00000000 00000000 00000000 00000000 * 003a000 eb56839b 0b0454c7 204b594e 44dc078e 003a010 00000000 00000000 00000000 00000000 * 003c000 eb56839b 0b0454c7 204b594e 44dc078e 003c010 00000000 00000000 00000000 00000000 * 003e000 eb56839b 0b0454c7 204b594e 44dc078e 003e010 00000000 00000000 00000000 00000000 * 0040000 eb56839b 0b0454c7 204b594e 44dc078e 0040010 00000000 00000000 00000000 00000000 * % rm asd

[2] cp without devstream from usbdisk –> ok

% unmount ‘#¶’ /fd % cp /shr/sdUce48e/asd . % xd -r asd 0000000 eb56839b 0b0454c7 204b594e 44dc078e 0000010 00000000 00000000 00000000 00000000 * 0004000 00000000 0004002 % rm asd

new example file, this time size < 16386

% dd -bs 16385 -count 1 </dev/zero >/shr/sdUce48e/asd 1+0 records in 1+0 records out % dd -bs 16 -count 1 -trunc 0 </dev/random -of /shr/sdUce48e/asd 1+0 records in 1+0 records out % xd -r /shr/sdUce48e/asd 0000000 24b52d58 1d7c0554 f1802c4a 6a424a6b 0000010 00000000 00000000 00000000 00000000 * 0004000 00000000 0004001

[3] cp with devstream from usbdisk, but file size < 16386 –> ok

% bind -a ‘#¶’ /fd % cp /shr/sdUce48e/asd . % xd -r asd 0000000 24b52d58 1d7c0554 f1802c4a 6a424a6b 0000010 00000000 00000000 00000000 00000000 * 0004000 00000000 0004001 %


Sun, 4 Oct 2015 22:56:49 EDT
cinap_lenrek@[REDACTED]

was bug in dossrv, which ignored the upper 32 bit of the 64 bit file offset. as devstream tests if a file interprets the offset by seeking to 0x7fffffffffffffff and if read returns data, assumes the file is a pipe like stream where we need to continue reading until we get eof (passing in 0 as the file offset).

fixed in dossrv with commit 8053b8f00635