Commit Graph

13632 Commits

Author SHA1 Message Date
James Peach cef4decf04 unshare: add --keep-caps option
Add the --keep-caps option to unshare to preserve capabilities that
are granted when creating a new user namespace. This allows the child
process to retain privilege within the new user namespace without also
being UID 0.
2019-09-09 20:23:08 -07:00
Karel Zak a1790d265d bash-completion: (unshare) add --map-current-user
Signed-off-by: Karel Zak <>
2019-09-09 12:58:03 +02:00
Karel Zak 646bba41fd dmesg: add --noescape
We have no way how to print the kernel message buffer in really raw
way. The new option --noescape disables all \x<hex> translations.

Signed-off-by: Karel Zak <>
2019-09-09 12:56:53 +02:00
Karel Zak a6cc5c16f4 Merge branch 'unshare/map-current' of
* 'unshare/map-current' of
  unshare: add --map-current-user option
2019-09-09 11:31:04 +02:00
Karel Zak b008e4d668 tests: (sfdisk) update move output
Signed-off-by: Karel Zak <>
2019-09-09 11:27:03 +02:00
James Peach 4175f29e62 unshare: add --map-current-user option
Add the --map-current-user option to unshare. This option maps the
current effective UID and GID in the new user namespace so that the
inner and outer credentials match.

Signed-off-by: James Peach <>
2019-09-08 15:00:00 -07:00
Karel Zak 925f2d4f4f sfdisk: (--move-data) add speed to progress bar, don't use POSIX_FADV_DONTNEED
posix_fadvise() in the loop is pretty expensive.

Signed-off-by: Karel Zak <>
2019-09-06 16:57:40 +02:00
Karel Zak 4bf424188d sfdisk: (--move-data) add simple progress bar
Signed-off-by: Karel Zak <>
2019-09-06 15:07:59 +02:00
Karel Zak e4e0b1a6f0 libfdisk: cleanup fdisk_deassign_device() docs
Signed-off-by: Karel Zak <>
2019-09-06 15:07:43 +02:00
Karel Zak 7942ba8a79 sfdisk: add --move-use-fsync, disable fsync() by default
It's slow, so slooow...

Signed-off-by: Karel Zak <>
2019-09-06 15:07:43 +02:00
Karel Zak 69f30c3126 sfdisk: (--move-data) make log optional
The log may be pretty huge and very probably not used by many users.
Let's make it optional.

The patch also clean up move-data output messages.

Signed-off-by: Karel Zak <>
2019-09-06 15:07:30 +02:00
Karel Zak 4ce4675cce sfdisk: (--move-data) keep step size based on optimal I/O
The current implementation is too paranoid and tries to keep in
memory only data which are on disk too. It means very small step size
when move partition only a few sectors left/right.

This patch enlarge the buffer to at least 1MiB and aligned to optimal

Signed-off-by: Karel Zak <>
2019-09-06 15:05:59 +02:00
Karel Zak 1116884cf7 libfdisk: use grain as small as possible
The current implementation does not allow to move partition for
example in +/-1 sector range, because free space analyze is by default
based on regular grain used for partitioning (=1MiB).

Signed-off-by: Karel Zak <>
2019-09-06 13:09:30 +02:00
Karel Zak 45aaa8f3db sfdisk: make --no-act usable for --move-data too
Signed-off-by: Karel Zak <>
2019-09-06 12:37:26 +02:00
Karel Zak 492a5c70b4 Merge branch 'xfs-v5' of
* 'xfs-v5' of
  tests: Add test for current version (v5) of XFS filesystem
2019-09-06 10:33:45 +02:00
Karel Zak 91099dea35 tests: update sfdisk wipe output
- stderr is used for the warning (like in fdisk)

Signed-off-by: Karel Zak <>
2019-09-05 16:46:14 +02:00
Karel Zak f227757c77 fstrim: ignore non-directory mountpoints
It seems better to silently ignore mount binds on file (= mountpoint
is not a directory).

This patch also fixes use-after-free bug from commit 402006fa6e.

Signed-off-by: Karel Zak <>
2019-09-05 16:32:10 +02:00
Karel Zak 40af0db4cd fdisk: fix quit dialog for non-libreadline version
We need to clear stdin errors otherwise it returns EOF forever after

Reported-by: Lukas Czerner <>
Signed-off-by: Karel Zak <>
2019-09-05 12:34:01 +02:00
Karel Zak fff85eb577 fdisk: make quit question more usable
Reported-by: Lukas Czerner <>
Signed-off-by: Karel Zak <>
2019-09-05 12:06:27 +02:00
Karel Zak 6cd671d427 fdisk: cleanup wipe warning
Let's remove 'old' from the sentence, add man page reference to

Signed-off-by: Karel Zak <>
2019-09-04 14:54:54 +02:00
Anatoly Pugachev 037bc610e5 tests: Add test for current version (v5) of XFS filesystem
Old xfs.img cannot be used with current version (5.0) of xfsprogs, so create
a new version of xfs-v5.img.

Since I wasn't able to find logs on how was created old xfs.img, saving
create log with this commit message:

$ fallocate -l 16M xfs-v5.img
$ mkfs.xfs -L test-xfs-v5 xfs-v5.img
$ ./blkid -p -o udev xfs-v5.img | sort -V > tests/expected/blkid/low-probe-xfs-v5
$ xz -c xfs-v5.img > tests/ts/blkid/images-fs/xfs-v5.img.xz

Signed-off-by: Anatoly Pugachev <>
2019-09-03 19:08:22 +03:00
Karel Zak 03154d2cf2 sfdisk: add -J between mutually exclusive options
Signed-off-by: Karel Zak <>
2019-09-03 16:22:21 +02:00
Karel Zak 987ec2a696 libmount: fix typo
Signed-off-by: Karel Zak <>
2019-09-03 16:18:48 +02:00
Karel Zak 2771d40b88 libblkid: (xfs) fix sector size calculation
Reported-by: Anatoly Pugachev <>
Signed-off-by: Karel Zak <>
2019-09-03 15:10:35 +02:00
Karel Zak ce16f04efa fdisk: add hint about --wipe to warning
Signed-off-by: Karel Zak <>
2019-09-03 11:57:23 +02:00
Karel Zak eabcbfc449 sfdisk: mark --dump and --list-free as mutually exclusive
Reported-by: Bertrand Jacquin <>
Signed-off-by: Karel Zak <>
2019-09-02 13:10:44 +02:00
Karel Zak e7d318a9dd tests: (blkid) update regression tests (due to BLOCK_SIZE)
Signed-off-by: Karel Zak <>
2019-09-02 12:57:11 +02:00
Mikulas Patocka cd129b7d2f blkid: retport block size of a filesystem
This patch extends libblkid, so that it reports filesystem block size.

When blkid returns a specific number in the BLOCK_SIZE attribute, it
guarantees that all the bios submitted by the filesystem are aligned on
this boundary.

We need this because when we want to enable dm-integrity or dm-writecache
on an existing filesystem, we need to know filesystem block size, so that
dm-integrity or dm-writecache is initialized with matching block size.

We could always use block size 512 for dm-integrity and dm-writecache, but
that would cause metadata overhead and performance degradation. On the
other hand, if we used block size 4096, it would fail if the filesystem
has smaller blocksize.

[ - move vfat BLOCK_SIZE to probing function
		  - remove unwanted debug fprintf from ZFS prober]

Signed-off-by: Mikulas Patocka <>
Signed-off-by: Karel Zak <>
2019-09-02 12:55:53 +02:00
Patrick Steinhardt 530220b6bf include/closestream: fix assignment to read-only standard streams
In order to avoid closing standard streams multiple times, commit
52aa1a661 (include/closestream: avoid close more than once, 2019-06-13)
introduced code to set the standard output and error streams to `NULL`.
As musl libc defines standard streams as constant pointers, the change
causes compiler errors on systems with that libc. According to ISO C89,
being able to assign to the standard text streams is not a requirement
for any C implementation, see footnote 238 in chapter §

    The primary use of the freopen function is to change the file
    associated with a standard text stream (stderr, stdin, or stdout),
    as those identifiers need not be modifiable lvalues to which the
    value returned by the fopen function may be assigned.

This commit implements a new function `flush_standard_stream` that tries
to reliably flush standard streams without actually closing them. By not
calling fclose(3P), we can neatly avoid the issue of accessing standard
streams in an unspecified state and thus remove the infringing `NULL`

Properly flushing standard streams without fclose(3P) proves to be more
intricate than one may expect, though, as some filesystems like NFS may
defer flushing until they see a close(3P) of the underlying descriptor.
One may call fsync(3P) to remedy that, but this may incur a heavy
performance penalty in some scenarios. To work around the issue and
still get proper errors, we duplicate the stream's file descriptor and
close that one instead, which is sufficient to cause a flush.

Note that both `close_stdout` and `close_stdout_atexit` are misnamed
after this change, as we do not actually close the streams now. In order
to avoid unnecessary code churn, we still retain their current names.

Signed-off-by: Patrick Steinhardt <>
2019-09-02 11:55:56 +02:00
Karel Zak d356507207 colcrt: make seek to \n more robust
Signed-off-by: Karel Zak <>
2019-09-02 11:53:10 +02:00
Karel Zak 50c77b6afe tests: (colcrt) use env to set locale
Signed-off-by: Karel Zak <>
2019-08-30 20:46:58 +02:00
Karel Zak e3803ff869 tests: (getopt) remove unwanted paths from error output
Signed-off-by: Karel Zak <>
2019-08-30 20:46:44 +02:00
Karel Zak eb61e95f1c tests: remove unbuffered ts_run feature
It's unnecessary as stdout and stderr are split in the tests.

Signed-off-by: Karel Zak <>
2019-08-30 16:30:35 +02:00
Karel Zak 18c1f9e643 tests: add remaining stderr outputs
Signed-off-by: Karel Zak <>
2019-08-30 16:10:01 +02:00
Karel Zak b161f810b0 libdisk: write sample output to stdout
Signed-off-by: Karel Zak <>
2019-08-30 16:05:50 +02:00
Karel Zak cbf858aa08 tests: split stdout and stderr
* add $TS_ERRLOG for script stderr output
* add optional $TS_EXPECTED_ERR which points to expected/*/*.err

This change allows to keep track about stderr output from our commands
(already found bug in sfdisk...).

We do not have to depend on fragile stdout vs. stderr order (due to
different buffering semantic in different libc, etc.).

Signed-off-by: Karel Zak <>
2019-08-30 15:50:07 +02:00
Karel Zak 7837cd491a sfdisk: write all message to stdout
The code writes message to stdout and \n to stderr.

Signed-off-by: Karel Zak <>
2019-08-30 15:40:06 +02:00
Karel Zak ee551c909f libmount; fix and improve read+poll mountinfo
* fix read() buffer size (stupid bug...)

* split read() EINTR and poll() based attempts

* use 100 attempts

* wait 10000 usec between attempts, but first 10 attempts are without
  this delay. It makes the function usable for usual use-cases as well
  as on very busy systems (successfully tested with 300 concurrent
  mount/umount processes)

Signed-off-by: Karel Zak <>
2019-08-29 15:50:58 +02:00
Karel Zak e4925f591c libmount: improve mountinfo reliability
The standard way how we read mount table is not reliable because
during the read() syscalls the table may be modified by some another
process. The changes in the table is possible to detect by poll()
event, and in this case it seems better to lseek to the begin of the file
and read it again. It's expensive, but better than races...

This patch does not modify mountinfo parser, but it reads all file to
memory (by read()+poll()) and than it creates memory stream
from the buffer and use it rather than a regular file stream.

It means the parser is still possible to use for normal files
(e.g. fstab) as well as for mountinfo and it's also portable to
systems where for some reason is no fmemopen().

Note that re-read after poll() event is limited to 5 attempts (but
successful read() without event zeroize the counter). It's because we
do not want to wait for consistent mountinfo for ever. It seems better
to use old (less reliable) way than hang up in read()+poll()

Reported-by: Zbigniew Jędrzejewski-Szmek <>
Signed-off-by: Karel Zak <>
2019-08-29 14:27:11 +02:00
Patrick Steinhardt 59a46f36b6 tests: use env and support both unbuffer/stdbuf
Triggered by commit f612c4c67 (tests: fix --unbuffered mode with
ASAN, 2019-08-27), which says:

    Well, this patch sucks. It would be nice to have things in
    the way how it has been original expected by Patrick's patch,
    but ...

So this commit here effectively reverts it and instead tries to
improve the shortcomings of the original patch. First, it uses
env(1) to set ASAN_OPTIONS instead of directly adding it to the
args array to fix execution of "${args[@]}" "$@".

Second, it now supports both unbuffer(1) and stdbuf(1). The
latter uses LD_PRELOAD tricks, which doesn't play nicely with
ASAN, so it will not be used if ASAN has been requested. It's
still valuable to have support for both, as many more systems
will have stdbuf(1) from coreutils installed but not unbuffer(1)
from expect.

Signed-off-by: Karel Zak <>
2019-08-28 08:56:24 +02:00
Karel Zak 71210f5bab tests: don't show diff for TS_KNOWN_FAIL
Signed-off-by: Karel Zak <>
2019-08-27 14:38:21 +02:00
Karel Zak 751fae0d96 tests: improve unbuffer check
* remove obsolete stdbuf check

* check for unbuffer command in ts_run() than skip all test

Reported-by: Patrick Steinhardt <>
Signed-off-by: Karel Zak <>
2019-08-27 14:28:11 +02:00
Patrick Steinhardt 2ede7ec214 tests: (fdisk) avoid hardcoding of errno string
The test fdisk/oddinput hardcodes strings returned by strerror(3P) for
both the errors ENOENT and ENOTTY. As these strings are unportable,
convert the tests to use the test_strerror helper instead to convert
them with sed(1).

Signed-off-by: Patrick Steinhardt <>
2019-08-27 14:25:32 +02:00
Karel Zak f612c4c674 tests: fix --unbuffered mode with ASAN
Unfortunately, ASAN is pretty sensitive to LD_PRELOAD, but stdbuf from
coreutils is based on LD_PRELOAD. So, I have replaced stdbuf with
unbuffer (from expect pkg).

The another problem is "${args[@]}" "$@" which does not work as expected.

Well, this patch sucks. It would be nice to have things in the way
how it has been original expected by Patrick's patch, but ...

Signed-off-by: Karel Zak <>
2019-08-27 13:02:38 +02:00
Karel Zak f7f25faae0 Merge branch '2019wk33' of
* '2019wk33' of
  docs: try to find broken man references and fix them
  docs: correct su.1 runuser reference from section 8 to 1
  po: remove possibility to translate static option arguments
2019-08-27 10:11:02 +02:00
Patrick Steinhardt eebc9e4dc2 tests: (col) avoid hardcoding of errno string
The col/multibyte test has a hardcoded error string as part of its
expected output that is returned by glibc's strerror(3P) function. Even
though many of these strings are the same across libc implementations,
they are not standardiced and some are certainly different. One example
is the string for EILSEQ on musl libc.

To fix this, we introduce a new test helper "test_strerror". The helper
can be invoked with an error code like "EILSEQ", which will cause it to
print out the the respective error message for that code. Note that
"test_strerror" cannot act on the error's value (e.g. 84 for EILSEQ), as
these aren't standardized either. Instead, we thus need to have an array
of the error's string representation ("EILSEQ") to its respective error
code (the define EILSEQ). The array can trivially be extended as
required, thus it is only sparsely populated with EILSEQ right now.

To fix the col/multibyte test, we introduce a call to sed(1) to replace
the strerror(3P) message from EILSEQ with "EILSEQ". Furthermore, as
we're running tests with the POSIX locale by default which treats all
bytes as valid multibyte sequences, we have to change to the C.UTF-8
locale instead to actually get an error.

Signed-off-by: Patrick Steinhardt <>
2019-08-27 09:37:01 +02:00
Patrick Steinhardt 6ef9a9e184 tests: (column) use actually invalid multibytes to test encoding
If reading an invalid multibyte sequence, column(1) will encode the byte
as "\x<hex>" instead. The tests try to verify that by piping "£" into
column(1). As the tests run with LC_ALL=POSIX by default, though, libc
implementations strictly adhering to the POSIX standard will treat all
characters as valid multibyte characters. As a consequence, no EILSEQ is
raised by mbtowc(3P) and the character is not encoded as hex, breaking
the test.

Fix this by setting LC_ALL=C.UTF-8. As "£" is a valid UTF-8 character,
we also change the test to use a proper illegal multibyte sequence.

Signed-off-by: Patrick Steinhardt <>
2019-08-27 09:36:50 +02:00
Patrick Steinhardt 5718f8d96b tests: (colcrt) fix reliance on EILSEQ in POSIX locale
The input file "crash1" in the colcrt/regressions test contains the
illegal byte sequence "\x94\x7e". While "\x7e" is '~', "\x94" is not a
valid character. Thus, the test assumes that getwc(3P) will return
`WEOF` and set `errno=EILSEQ`, causing colcrt(1) to abort reading the
stream and thus not print the trailing '~'.

This assumption holds just fine for glibc as it will dutifully report
EILSEQ, but musl libc will happily read the complete stream without
complaining about the illegal character. But in fact, as tests run with
LC_ALL=POSIX by default, glibc's behaviour is wrong while musl is right.
Quoting mbrtowc(3P) from POSIX.1-2017:

    [EILSEQ] An invalid character sequence is detected. In the POSIX locale an
             [EILSEQ] error cannot occur since all byte values are valid

Fix the issue by running the colcrt tests with C.UTF8 locale.

Signed-off-by: Patrick Steinhardt <>
2019-08-27 09:36:38 +02:00
Patrick Steinhardt 8892f94c57 tests: (libfdisk) remove reliance on buffer behaviour of standard streams
The tests in libfdisk/mkpart-full all rely on the buffering behaviour of
standard output and standard error streams, most importantly that stderr
is non-buffering and stdout is buffering. This doesn't hold on all libc
implementations when redirecting to a file, breaking the test suite on
such platforms.

Use `ts_run --unbuffered` to stop buffering of the standard output
stream to fix this.

Signed-off-by: Patrick Steinhardt <>
2019-08-27 09:36:26 +02:00
Patrick Steinhardt 801d689e47 tests: remove reliance on buffer behaviour of stderr/stdout streams
In the test cases "rename::exit_codes" and "rename::exit_codes", we rely
on the flushing behaviour of stderr and stdout streams relative to each
other. Streams in glibc will not flush on newlines if stdout is pointing
to a non-TTY file descriptor, but relying on this is fragile and may
break on systems with a different behaviour like musl libc.

Fix this by introducing a new parameter "--unbuffered" to `ts_run`. If
this parameter is passed and stdbuf(1) from coreutils is available, then
it will use it to disable buffering of standard output completely. Like
this, we can selectively run tests with this if ordering of messages
from stdout and stderr is being checked.

Signed-off-by: Patrick Steinhardt <>

2019-08-27 09:36:05 +02:00