Add linux-next 5.9 cpuinfo flag for Intel enqcmd/s instructions:
x86/cpufeatures: Enumerate ENQCMD and ENQCMDS instructions:
Work submission instruction comes in two flavors. ENQCMD can be called
both in ring 3 and ring 0 and always uses the contents of a PASID MSR
when shipping the command to the device. ENQCMDS allows a kernel driver
to submit commands on behalf of a user process. The driver supplies the
PASID value in ENQCMDS. There isn't any usage of ENQCMD in the kernel as
of now.
The CPU feature flag is shown as "enqcmd" in /proc/cpuinfo.
If __WITH_AF_UNIX is defined when Cygwin is built, then a named
AF_UNIX socket is represented by a reparse point with a
Cygwin-specific tag and GUID. Make such files recognizable as reparse
points (but not as sockets) even if __WITH_AF_UNIX is not defined.
That way utilities such as 'ls' and 'rm' still behave reasonably.
This requires two changes:
- Define the GUID __cygwin_socket_guid unconditionally.
- Make check_reparse_point_target return PATH_REP on a reparse point
of this type if __WITH_AF_UNIX is not defined.
Commit aa467e6e, "Cygwin: add AF_UNIX reparse points to path
handling", changed check_reparse_point_target so that it could return
a positive value on a known reparse point that is not a symlink. But
some of the code in check_reparse_point that handles this positive
return value was executed unconditionally, when it should have been
executed only for symlinks.
As a result, posixify could be called on a buffer containing garbage,
and check_reparse_point could erroneously return a positive value on a
non-symlink. This is now fixed so that posixify is only called if the
reparse point is a symlink, and check_reparse_point returns 0 if the
reparse point is not a symlink.
Also fix symlink_info::check to handle this last case, in which
check_reparse_point returns 0 on a known reparse point.
The following Windows system calls currently fail with
STATUS_IO_REPARSE_TAG_NOT_HANDLED when called on an AF_UNIX socket:
- NtOpenFile in get_file_sd
- NtOpenFile in set_file_sd
- NtCreateFile in fhandler_base::open
Fix this by adding the FILE_OPEN_REPARSE_POINT flag to those calls
when the file is a known reparse point.
../../../../src/winsup/cygwin/pinfo.cc: In member function 'DWORD pinfo::status_exit(DWORD)':
../../../../src/winsup/cygwin/ntdll.h:21:68: error: narrowing conversion of '-536870295' from 'NTSTATUS' {aka 'int'} to 'unsigned int' [-Wnarrowing]
../../../../src/winsup/cygwin/pinfo.cc:136:10: note: in expansion of macro 'STATUS_ILLEGAL_DLL_PSEUDO_RELOCATION'
../../../../src/winsup/cygwin/sigproc.cc: In member function 'DWORD child_info::proc_retry(HANDLE)':
../../../../src/winsup/cygwin/ntdll.h:21:68: error: narrowing conversion of '-536870295' from 'NTSTATUS' {aka 'int'} to 'unsigned int' [-Wnarrowing]
../../../../src/winsup/cygwin/sigproc.cc:1120:10: note: in expansion of macro 'STATUS_ILLEGAL_DLL_PSEUDO_RELOCATION'
NT error statuses seem to be variously DWORD (unsigned) or NTSTATUS
(signed)? So use the one which doesn't cause problems here.
../../../../src/winsup/cygwin/fhandler_console.cc: In member function 'const unsigned char* fhandler_console::write_normal(const unsigned char*, const unsigned char*)':
../../../../src/winsup/cygwin/fhandler_console.cc:2782:8: error: narrowing conversion of '-2' from 'int' to 'long unsigned int' [-Wnarrowing]
../../../../src/winsup/cygwin/fhandler_console.cc:2786:8: error: narrowing conversion of '-1' from 'int' to 'long unsigned int' [-Wnarrowing]
../../../../src/winsup/cygwin/fhandler_console.cc:2836:8: error: narrowing conversion of '-2' from 'int' to 'long unsigned int' [-Wnarrowing]
../../../../src/winsup/cygwin/fhandler_console.cc:2840:8: error: narrowing conversion of '-1' from 'int' to 'long unsigned int' [-Wnarrowing]
A mbtowc_p function returns an int, so that seems the correct type to use here.
../../../../src/winsup/cygwin/fhandler_socket_inet.cc: In member function 'ssize_t fhandler_socket_wsock::send_internal(_WSAMSG*, int)':
../../../../src/winsup/cygwin/fhandler_socket_inet.cc:1381:69: error: suggest parentheses around assignment used as truth value [-Werror=parentheses]
Commit aa467e6e, "Cygwin: add AF_UNIX reparse points to path
handling", changed the return values of check_reparse_point_target.
Update the comment accordingly.
This was added as a temporary measure in commit e18f7f99 because it
wasn't yet in the mingw-w64 headers. With one exception, it is now in
the current release of the headers (version 8.0.0), so we don't need
it in winlean.h.
The exception is that VirtualAlloc2 is declared conditionally in
<w32api/memoryapi.h>, but the compilation of Cygwin requires it to
always be declared, even though it will only be executed on systems
that support it. So retain the declaration in winlean.h. And add
"WINAPI" to the declaration, as in memoryapi.h.
Add a check that version >= 8 of the mingw-w64 headers is intalled.
Also revert commit 3d136011, which was a related temporary workaround.
- Charset conversion for UTF-7, ISO-2022 and ISCII, which are not
supported in cygwin, does not work properly as a result. At the
expense of the above, the code has been simplified a bit.
- If pseudo console is disabled, non-cygwin apps do not detect
console device. In this case, some apps output UTF-8 regardless
of the locale setting. At least git-for-windows, rust-based apps
and node.js do that. This patch provides backward compatibility
as default behaviour by setting console codepage to the charset of
the locale. Even in the cases above, garbled output is prevented
with this patch in most cases because mintty uses UTF-8 by default.
I beleave this is not really a problem in cygwin side but that in
app side, however, some users complain about garbled output with
existing apps in MSYS2 (which is based on cygwin) in which pseudo
console is disabled by default.
Setting up the pty in the master constructor ends up creating a new pty
on every stat(2) call on /dev/ptmx. Only do this when actually opening
the device, not when using the device class in another, non-opening
context.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
- Use tmp_pathbuf instead of HeapAlloc()/HeapFree().
- Remove mb_str_free() function.
- Consider the case where the multibyte string stops in the middle
of a multibyte char.
fhandler_process::exists is called when we are checking a path
starting with "/proc/<pid>/fd". If it returns virt_none and sets an
errno, there is no need for further checking. Just set 'error' and
return.
The incoming path is allowed to have the form "$PID/fd/[0-9]*/.*"
provided the descriptor symlink points to a directory. Check that
this is indeed the case.
The new function __eval_codepage_from_internal_charset
is a simplified version of the former code in
fhandler_tty.cc. It probably needs some extension,
but the gist is to use knowledge of internals to
be as quick as possible.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Both flags are outdated and collide with official flags in
sys/_default_fcntl.h, which may result in weird misbehaviour
of file functions.
O_NOSYMLINK is not used anyway.
O_DIROPEN is used in fhandler_virtual and derived classes.
The collision with O_NOFOLLOW results in spurious EISDIR
errors when, e. g., reading files in the registry.
fhandler_base::open_fs uses O_DIROPEN in the call to
fhandler_base::open, but it's not used in this context
further down the road.
Drop both flags and create an alternative "diropen" bool
flag in fhandler_virtual.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
...as on glibc right now. This is supposed to support autoconf scripts
checking for existence of these symbols in libpthread.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
GLibc will change this code in the forseeable future to align more
with FreeBSD, so this hack is not actually desired.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
- Pseudo console internally sends escape sequence CSI6n (query cursor
position) on startup of non-cygwin apps. If the terminal does not
support CSI6n, CreateProcess() hangs waiting for response. To prevent
hang, this patch disables pseudo console if the terminal does not
have CSI6n. This is checked on the first execution of non-cygwin
app using the following steps.
1) Check if the terminal support ANSI escape sequences by looking
into terminfo database. If terminfo has cursor_home (ESC [H),
the terminal is supposed to support ANSI escape sequences.
2) If the terminal supports ANSI escape sequneces, send CSI6n for
a test and wait for a responce for 40ms.
3) If there is a responce within 40ms, CSI6n is supposed to be
supported.
Also set-title capability is checked, and removes escape sequence
for setting window title if the terminal does not have the set-
title capability.
Currently, when using CYGWIN's error_start facility, the faulting
process isn't stopped while the error_start process is started when the
fault is caused by an exception. (it even seems possible in theory that
the faulting process could have exited before the error_start process
attaches).
This leads to e.g. the core dump written by CYGWIN='error_start=dumper'
in response to an exception being non-deterministic.
Remove the waitloop argument from try_to_debug(), only used in the
exception case, so the faulting process busy-waits until the error_start
process attaches.
Code archaeology to determine why the code is this way didn't really
turn up any answers, but this seems a low-risk change, as this only
changes the behaviour when:
- a debugger isn't already attached
- an error_start is specified in CYGWIN env var
- an exception has occurred which will be translated to a signal
If error_start invokes something which doesn't attach using
DebugActiveProcess(), we will spin indefinitely, but that will also
currently occur for any of the existing other uses of try_to_debug(),
which default to waitloop=TRUE.
On 32 bit x86, clang seems to miss loading input parameters based
on asm constraints for inline assembly that uses the x87 floating
registers, unless the snippet has got the volatile keyword.
Signed-off-by: Martin Storsjö <martin@martin.st>
Code taken from FreeBSD, which implements C11 threads as
wrapper around pthreads. Fix up machine/_threads.h which
is called from newlib's machine-independent threads.h to
match Cygwin's pthreads types.
Add the FreeBSD source files to libc subdir and take
opportunity to define LIBC_OFILES var in Makefile.
Add new symbols to common.din and sort symbols.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
In preparation of importing FreeBSDs stdthreads functions,
change the way pthread_yield is exported, so that the symbol
can be used internally as well.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
pthread_yield was only declared under GNU visibility,
but the function should be available under BSD visibility
as well.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
POSIX requires that key destructors are called in a loop
for each key with a non-NULL value until all values are
NULL, or until all destructors for non-NULL values
have been called at least PTHREAD_DESTRUCTOR_ITERATIONS
(per POSIX: 4) times.
Cygwinonly called all destructors with non-NULL values
exactly once. This patch fixes Cygwin to follow POSIX.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
We need one more entry than max children in the arrays.
There's no reason to do this for the static array, though.
One more entry in the overflow array is sufficient.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
On PROC_EXEC_CLEANUP, the pinfo's in chld_procs are removed.
This is done in a loop always removing the child with index 0.
This, however, results in copying the last child's pinfo in
chld_procs to position 0. Do this for 100 children and you
get 99 entirely useless copy operations.
Fix this by calling remove_proc in reverse order.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
256 children per process is a bit tight in some scenarios.
Fix this by revamping the `procs' array. Convert it to an
extensible class child_procs and rename procs to chld_procs.
Fix code throughout to use matching class methods rather than
direct access.
To allow a lot more child processes while trying to avoid
allocations at DLL startup, maintain two arrays within class
child_procs, one using a default size for 255 (i686) or 1023
(x86_64) children, the other, dynamically allocated on overflowing
the first array, giving room for another 1023 (i686) or 4095
(x86_64) processes.
On testing with a simple reproducer on a x86_64 machine with
4 Gigs RAM, a system memory overflow occured after forking
about 1450 child processes, so this simple dynamic should
suffice for a while.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
pinfo::remember with the detach parameter set to true is
the only way to call proc_subproc with PROC_DETACHED_CHILD.
This call is exclusively used in spawn to set up a pinfo for
a detached child, and that pinfo goes out of scope right
afterwards without any further action.
Drop the flag and drop the detach parameter from pinfo::remember.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
The return value is used in a numerical context and remove_proc
already returned inconsistently "true" vs. 0.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
After patch 23a779bf3d
"Cygwin: pinfo: stop remember doing reattach",
PROC_ADDCHILD actually just sets up a new child, mirroring
PROC_DETACHED_CHILD. The actual attaching of the child is
performed by action PROC_REATTACH_CHILD or pinfo::reattach
respectively.
To better reflect what's going on, rename PROC_REATTACH_CHILD
to PROC_ATTACH_CHILD and rename pinfo::reattach to pinfo::attach.
For better readability change PROC_ADDCHILD to PROC_ADD_CHILD.
Fix comments accordingly.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
- In this implementation, pseudo console is created for each native
console app. Advantages and disadvantages of this implementation
over the previous implementation are as follows.
Advantages:
1) No performance degradation in pty output for cygwin process.
https://cygwin.com/pipermail/cygwin/2020-February/243858.html
2) Free from the problem caused by difference of behaviour of control
sequences between real terminal and pseudo console.
https://cygwin.com/pipermail/cygwin/2019-December/243281.htmlhttps://cygwin.com/pipermail/cygwin/2020-February/243855.html
3) Free from the problem in cgdb and emacs gud.
https://cygwin.com/pipermail/cygwin/2020-January/243601.htmlhttps://cygwin.com/pipermail/cygwin/2020-March/244146.html
4) Redrawing screen on executing native console apps is not necessary.
5) cygwin-console-helper is not necessary for the pseudo console
support.
6) The codes for pseudo console support are much simpler than that
of the previous one.
Disadvantages:
1) The cygwin program which calls console API directly does not work.
2) The apps which use console API cannot be debugged with gdb. This
is because pseudo console is not activated since gdb uses
CreateProcess() rather than exec(). Even with this limitation,
attaching gdb to native apps, in which pseudo console is already
activated, works.
3) Typeahead key inputs are discarded while native console app is
executed. Simirally, typeahead key inputs while cygwin app is
executed are not inherited to native console app.
4) Code page cannot be changed by chcp.com. Acctually, chcp works
itself and changes code page of its own pseudo console. However,
since pseudo console is recreated for another process, it cannot
inherit the code page.
5) system_printf() does not work after stderr is closed. (Same with
cygwin 3.0.7)
6) Startup time of native console apps is about 3 times slower than
previous implemenation.
7) Pseudo console cannot be activated if it is already activated for
another process on same pty.
This is necessary in order to be consistent with the following comment
in the definition of _Unwind_RaiseException() in the GCC source file
libgcc/unwind-seh.c:
The exception handler installed in crt0 will continue any GCC
exception that reaches there (and isn't marked non-continuable).
Previously we failed to do this and, as a consequence, the C++ runtime
didn't call std::terminate after an unhandled exception.
This fixes the problem reported here:
https://cygwin.com/pipermail/cygwin/2019-October/242795.htmlhttps://sourceware.org/pipermail/cygwin/2020-August/245897.html
- After commit 095972ce5b, charset
conversion in mintty is broken if charset is set to other than
UTF-8. This seems to be caused because mintty does not set locale
yet at fork() call. This patch changes the timing of set_locale()
call again to avoid this issue.
- If native app is exec()'ed in a new pty, setup_locale() loses the
chance to be called. For example, with "mintty -e cmd", charset
conversion does not work as expected. This patch fixes the issue.
- If a lot of mintty are started in a short time from a mintty, some
of them hang with empty screen, crash immediately or hang on exiting
mintty. The following report seems to be related to this issue.
https://cygwin.com/pipermail/cygwin/2020-August/245751.html
The cause is not clear at all, but this patch seems to solve the
issue.
This patch has been inspired by the Linux kernel patch
294f69e662d1 compiler_attributes.h: Add 'fallthrough' pseudo keyword for switch/case use
written by Joe Perches <joe AT perches DOT com> based on an idea from
Dan Carpenter <dan DOT carpenter AT oracle DOT com>. The following text
is from the original log message:
Reserve the pseudo keyword 'fallthrough' for the ability to convert the
various case block /* fallthrough */ style comments to appear to be an
actual reserved word with the same gcc case block missing fallthrough
warning capability.
All switch/case blocks now should end in one of:
break;
fallthrough;
goto <label>;
return [expression];
continue;
In C mode, GCC supports the __fallthrough__ attribute since 7.1,
the same time the warning and the comment parsing were introduced.
Cygwin-only: add an explicit -Wimplicit-fallthrough=5 to the build
flags.
Currently raw_read makes two passes through the list of clients. On
the first pass it tries to read from the client from which it last
read successfully. On the second pass it tries to read from all
connected clients.
Add a new pass in between these two, in which raw_read tries to read
from all clients that are in the fc_input_avail case. This should be
more efficient in case select was previously called and detected input
available.
Slightly tweak the first pass. If a client is marked as having the
last successful read but reading from it now finds no input, don't
unmark it unless we successfully read from a different client on one
of the later passes.
The fifo_reader thread function and the function select.cc:peek_fifo()
can both change the state of a fifo_client_handler. These changes are
made under fifo_client_lock, so there is no race, but the changes can
still be incompatible.
Add code to make sure that only one of these functions can change the
state from its initial fc_listening state. Whichever function does
this calls the fhandler_fifo::record_connection method, which is now
public so that peek_fifo can call it.
Slightly modify that method to make it suitable for being called by
peek_fifo.
Make a few other small changes to the fifo_reader thread function to
change how it deals with the STATUS_PIPE_CLOSING value that can
(rarely) be returned by NtFsControlFile.
Add commentary to fhandler_fifo.cc to explain fifo_client connect
states and where they can be changed.
Don't try to read from fifo_client_handlers that are in the fc_closing
state. Experiments have shown that this always yields
STATUS_PIPE_BROKEN, so it just wastes a Windows system call.
Re-order the values in enum fifo_client_connect_state to reflect the
new status of fc_closing.
Rename the existing set_state() to query_and_set_state() to reflect
what it really does. (It queries the O/S for the pipe state.) Add a
new set_state() method, which is a standard setter, and a
corresponding getter get_state().
fhandler_fifo::take_ownership() is called from select.cc::peek_fifo
and fhandler_fifo::raw_read and could potentially block indefinitely
if something goes wrong. This is always undesirable in peek_fifo, and
it is undesirable in a nonblocking read. Fix this by adding a timeout
parameter to take_ownership.
Arbitrarily use a 1 ms timeout in peek_fifo and a 10 ms timeout in
raw_read. These numbers may have to be tweaked based on experience.
Replace the call to cygwait in take_ownership by a call to WFSO.
There's no need to allow interruption now that we have a timeout.
fhandler_fifo::take_ownership() tacitly assumes that the current
owner's fifo_reader_thread will be woken up from WFMO when
update_needed_evt is signaled. But it's possible that the the current
owner's fifo_reader_thread is at the beginning of its main loop rather
than in its WFMO call when that event is signaled.
In this case the owner will never see that the event has been
signaled, and it will never update the shared fifo_client_handlers.
The reader that wants to take ownership will then spin its wheels
forever.
Fix this by having the current owner call update_shared_handlers at
the beginning of its loop, if necessary.
microcode is unsigned long long, printed by _small_sprintf using %x;
Cygwin32 used last 4 bytes of microcode for next field MHz, printing 0;
use correct _small_sprintf format %X to print microcode, producing
correct MHz value under Cygwin32
CPUID 7:0 EDX[14] serialize added in linux-next 5.8 by Ricardo Neri-Calderon:
The Intel architecture defines a set of Serializing Instructions (a
detailed definition can be found in Vol.3 Section 8.3 of the Intel "main"
manual, SDM). However, these instructions do more than what is required,
have side effects and/or may be rather invasive. Furthermore, some of
these instructions are only available in kernel mode or may cause VMExits.
Thus, software using these instructions only to serialize execution (as
defined in the manual) must handle the undesired side effects.
As indicated in the name, SERIALIZE is a new Intel architecture
Serializing Instruction. Crucially, it does not have any of the mentioned
side effects. Also, it does not cause VMExit and can be used in user mode.
This new instruction is currently documented in the latest "extensions"
manual (ISE). It will appear in the "main" manual in the future.
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/arch/x86/include/asm/cpufeatures.h?id=85b23fbc7d88f8c6e3951721802d7845bc39663d
Newlib's posix_spawn has been taken from FreeBSD. The code relies on
BSD-specific behaviour of vfork, namely the fact that vfork blocks
the parent until the child exits or calls execve as well as the fact
that the child shares parent memory in non-COW mode.
This behaviour can't be emulated by Cygwin. Cygwin's vfork is
equivalent to fork. This is POSIX-compliant, but it's lacking BSD's
vfork ingrained synchronization of the parent to wait for the child
calling execve, or the chance to just write a variable and the parent
will see the result.
So this requires a Cygwin-specific solution. The core function of
posix_spawn, called do_posix_spawn is now implemented twice, once using
the BSD method, and once for Cygwin using Windows synchronization under
the hood waiting for the child to call execve and signalling errors
upstream. The Windows specifics are hidden inside Cygwin, so newlib
only calls internal Cygwin functions.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
It's working on 32 bit OSes only anyway. It even fails on WOW64.
Drop unsupported NtMapViewOfSection flags.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Commit 605bdcd410 enabled mapping beyond
EOF in 64 bit environments. But the variable 'orig_len' did not get
rounded up to a multiple of 64K. This rounding was done on 32 bit
only. Fix this by rounding up orig_len on 64 bit, in the same place
where 'len' is rounded up.
Rounding up is needed to make sigbus_page_len a multiple of the
allocation granularity.
In addition, failing to round up could cause orig_len to be smaller
than len. Since these are both unsigned values, the statement
'orig_len -= len' could then cause orig_len to be huge, and mmap would
fail with errno EFBIG.
I observed this failure while debugging the problem reported in
https://sourceware.org/pipermail/cygwin/2020-July/245557.html.
The failure can be seen by running the test case in that report under
gdb or strace.
- After commit 0365031ce1, key input
becomes not working by following steps.
1) Start cmd.exe in mintty.
2) Open another mintty.
3) Execute "echo AAA > /dev/pty*" (pty* is the pty opened in 1.)
This patch fixes the issue.
Use cygwait in take_ownership to allow interruption while waiting to
become owner. Return the cygwait return value or a suitable value to
indicate an error.
raw_read now checks the return value and acts accordingly.
If update_my_handlers fails to duplicate one or more handles, just
mark the corresponding handlers as being in an error state.
But if update_my_handlers is unable to open the process of the
previous owner, it's likely that something serious has gone wrong, so
we continue to make that a fatal error.
When a reader takes ownership in fifo_reader_thread, it now goes
directly to the part of the main loop that listens for a connection.
Previously it went back to the beginning of the loop.
Also, if the reader has to delay taking ownership because the previous
owner has not finished updating the shared fifo_client handlers, it
now checks to see if cancel_evt has been set. Previously it might
have had to spin its wheels unnecessarily only to eventually find that
its thread had been canceled.
Add a bool member 'last_read' to the fifo_client_handler structure,
which is set to true on a successful read. This is used by raw_read
as follows.
When raw_read is called, it first locates the writer (if any) for
which last_read is true. raw_read tries to read from that writer and
returns if there is input available. Otherwise, it proceeds to poll
all the writers, as before.
The effect of this is that if a writer writes some data that is only
partially read, the next attempt to read will continue to read from
the same writer. This should reduce the interleaving of output from
different writers.
When a reader opens, it needs to block if there are no writers open
(unless is is opened with O_NONBLOCK). This is easy for the first
reader to test, since it can just wait for a writer to signal that it
is open (via the write_ready event). But when a second reader wants
to open, all writers might have closed.
To check this, use a new '_nwriters' member of struct fifo_shmem_t,
which keeps track of the number of open writers. This should be more
reliable than the previous method.
Add nwriters_lock to control access to shmem->_nwriters, and remove
reader_opening_lock, which is no longer needed.
Previously only readers had access to the shared memory, but now
writers access it too so that they can increment _nwriters during
open/dup/fork/exec and decrement it during close.
Add an optional 'only_open' argument to create_shmem for use by
writers, which only open the shared memory rather than first trying to
create it. Since writers don't need to access the shared memory until
they have successfully connected to a pipe instance, they can safely
assume that a reader has already created the shared memory.
For debugging purposes, change create_shmem to return 1 instead of 0
when a reader successfully opens the shared memory after finding that
it had already been created.
Remove check_write_ready_evt, write_ready_ok_evt, and
check_write_ready(), which are no longer needed.
When opening a writer and looping to try to get a connection, recheck
read_ready at the top of the loop since the number of readers might
have changed.
To slightly speed up the process of opening the first reader, take
ownership immediately rather than waiting for the fifo_reader_thread
to handle it.
When the owning reader closes and there are still readers open, the
owner needs to wait for a new owner to be found before closing its
fifo_client handlers. This involves a loop in which dec_nreaders is
called at the beginning and inc_nreaders is called at the end. Any
other reader that tries to access shmem->_nreaders during this loop
will therefore get an inaccurate answer.
Fix this by adding an nreaders method and using it instead of
dec_nreaders and inc_nreaders. Also add nreaders_lock to control
access to the shmem->_nreaders.
Make various other changes to improve the reliability of finding a new
owner.
Since FD_CONNECT is only given once, we manually need to set
FD_WRITE for connection failed sockets to have consistent
behaviour in programs calling poll/select multiple times.
Example test to non-listening port: curl -v 127.0.0.1:47
If the acl_t struct was at or above 0x80000000 then the pointer was
sign-extended to 0xffff_ffff_8000_0000 and so the index was lost.
Signed-off-by: David Allsopp <david.allsopp@metastack.com>
This should slightly speed up especially path conversions,
given there's one less function call rearranging all function
arguments in registers/stack (and less stack pressure).
For clarity, rename overloaded sys_wcstombs to _sys_wcstombs
and sys_cp_mbstowcs to _sys_mbstowcs.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Re: CPU microcode reported wrong in /proc/cpuinfo
https://sourceware.org/pipermail/cygwin/2020-May/245063.html
earlier Windows releases used different registry values to store microcode
revisions depending on the MSR name being used to get microcode revisions:
add these alternative registry values to the cpuinfo registry value lookup;
iterate thru the registry data until a valid microcode revision is found;
some revision values are in the high bits, so if the low bits are all clear,
shift the revision value down into the low bits
sys_mbstowcs is called with the destination buffer length
set to MaximumLength from the receiving UNICODE_STRING buffer.
This is twice as much as the actual size of the buffer in
wchar_t units, which is the unit expected by sys_mbstowcs.
sys_mbstowcs always attaches a NUL, within the destination
buffersize given. But if the string is exactly one wchar_t
less than the actual buffer, and the buffersize is given too
large, sys_mbstowcs writes a NUL one wchar_t beyond the buffer.
This has only been exposed with Cygwin 3.1.5 because alloca
on newer gcc 9 apparently allocates more tightly. The alloca
buffer here is requested with 16 bytes, which is exactly the
number of bytes required for the string L"cmd.exe". Older gcc
apparently allocated a few more bytes on the stack, while gcc 9
allocates in 16 byte granularity...
Fix this by giving the correct destination buffer size to
sys_mbstowcs.
Fixes: https://cygwin.com/pipermail/cygwin/2020-June/245226.html
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Use WSAIoctl(SIO_KEEPALIVE_VALS) on older systems.
Make sure that keep-alive timeout is equivalent to
TCP_KEEPIDLE + TCP_KEEPCNT * TCP_KEEPINTVL on older systems,
even with TCP_KEEPCNT being a fixed value on those systems.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
- Drop definitions from <cygwin/sockets.h>
- Drop options only available on BSD
- Fix value of TCP_MAXSEG. It was still defined as the BSD value
while WinSock uses another value
- Handle the fact that TCP_MAXSEG is a R/O value in WinSock
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
- Return value of eat_readahead() is redefined. The return values
of fhandler_termios::eat_readahead() and fhandler_pty_slave::
eat_readahead() were little bit different. This patch unifies
them to number of bytes eaten by eat_readahead().
- Considerration for raixget() is added to fhandler_pty_master::
accept_input() code.
- Transfering contents of read ahead buffer in
fhandler_pty_master::transfer_input_to_pcon() is removed since
it is not necessary.
- fhandler_pty_slave::eat_readahead() ckecks EOL only when ICANON
is set.
- Guard for _POSIX_VDISABLE is added in checking EOL.
- If vim is started from WSL (Ubuntu) which is executed in pseudo
console in mintty, shift key and ctrl key do not work. Though
this issue is similar to the issue resolved by commit
4527541ec6, that commit is not
effective for this issue. This patch fixes the issue by discarding
"CSI > Pm m" in fhandler_pty_master::pty_master_fwd_thread().
- Commit c4b060e3fe seems to be not
enough. Moreover, it does not work as expected at all in Win10
1809. This patch essentially reverts that commit and add another
fix. After all, the cause of the problem was a race issue in
switch_to_pcon_out flag. That is, this flag is set when native
app starts, however, it is delayed by wait_pcon_fwd(). Since the
flag is not set yet when less starts, the data which should go
into the output_handle accidentally goes into output_handle_cyg.
This patch fixes the problem more essentially for the cause of
the problem than previous one.
- If the output of non-cygwin apps is browsed using less, screen is
ocasionally distorted after less exits. This frequently happens
if cmd.exe is executed after less. This patch fixes the issue.
- In current pty, the window title can not be set from non-cygwin
program due to the code which prevents overwriting the window
title to "cygwin-console-helper.exe" in fhandler_pty_master::pty_
master_fwd_thread(). This patch fixes the issue.
- If the cygwin vim is started from a non-cygwin process which is
executed in pseudo console, shift key and ctrl key do not work.
In this case, vim is executed under /dev/cons*. If vim outputs
escape sequence which is not supported by pseudo console, the
escape sequence is leaked into the parent pty. This causes
unexpected results. This patch fixes the issue by discarding
"CSI > Pm m". "OSC 10;? BEL/ST" and "OSC 11;? BEL/ST" are
discarded as well.
- After commit 29431fcb5b, the issue
reported in https://cygwin.com/pipermail/cygwin/2020-May/245057.html
occurs. This is caused by the following mechanism. Cygwin less
called from non-cygwin git is executed under /dev/cons* rather
than /dev/pty* because parent git process only inherits pseudo
console handle. Therefore, less sets ICANON flag for /dev/cons*
rather than original /dev/pty*. When pty is switched to non-cygwin
git process, line_edit() is used in fhandler_pty_master::write()
only to set input_available_event and read ahead buffer is supposed
to be flushed in accept_input(). However, ICANON flag is not set
for /dev/pty*, so accept_input() is not called unless newline
is entered. As a result, the input data remains in the read ahead
buffer. This patch fixes the issue.
Disable -std option since gnu++14 is default anyway, but keep
it available as comment.
Update dynamic exception specifications deprecated with
C++11 to C++11-introduced noexcept expression.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
This partially reverts commit
f36262d56a. That commit incorrectly
made the st_mode of a fifo reflect the Windows permissions of the disk
file underlying the fifo.
Rename localtime.c.patched to localtime.patched.c to keep the correct
language suffix.
Create localtime.patched.c in the build dir rather than in the source
dir. Decouple the build rule for creating localtime.patched.c from
the rule to build localtime.o, so we don't have to rebuild the patched
source file all the time.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
That also requires a small tweak to localtime.c.patch, otherwise
GCC complains about the position of the 'trydefrules' label.
Also, simplify includes.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Add tz_posixrules.h with data generated from most recent Cygwin tzdata
package. Establish localtime.cc as primarily a wrapper around a patched
copy of localtime.c. See README for more information.
Modifies winsup/cygwin/Makefile.in to build localtime.o from items in
new winsup/cygwin/tzcode subdirectory. Compiler option "-fpermissive"
is used to accept warnings about missing casts on the return values of
malloc() calls. This patch also removes existing localtime.cc and
tz_posixrules.h from winsup/cygwin as they are superseded by the
subsequent patches in this set.
Both functions are declared as extern "C" functions in
sys/smallprint.h, but as C++ funcs in winsup.h and in the
source itself.
Add extern "C to definitions, remove declarations in winsup.h
and include sys/smallprint.h instead.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
- The number of slaves attached to pseudo console is used only for
triggering redraw screen. Counting was not only needless, but also
did not work as expected. This patch removes the code for counting.
This reverts commit 39a9cd9465.
There is no need to explicitly take ownership in fixup_after_exec; if
ownership transfer is needed, it will be taken care of by
fhandler_fifo::close when the parent closes. Moreover, closing the
parent's fifo_reader_thread can cause problems, such as the one
reported here:
https://cygwin.com/pipermail/cygwin-patches/2020q2/010235.html
- After commit 071b8e0cbd, the problem
reported in https://cygwin.com/pipermail/cygwin/2020-May/244873.html
occurs. This is due to freeing console device accidentally rather
than pseudo console. This patch makes sure to call FreeConsole()
only if the process is attached to the pseudo console of the current
pty.
During a blocking read, we sleep for 1 ms after each iteration through
the connected writers. Currently we do this by calling Sleep (1).
Remove this call to Sleep and instead change the timeout in the
cygwait call from 0 to 1, so that raw_read can be interrupted while
sleeping.
There are currently three functions that call NtQueryInformationFile
to determine the state of a pipe instance. Do this only once, in a
new fifo_client_handler::set_state () function, and call that when
state information is needed.
Remove the fifo_client_handler methods pipe_state and get_state, which
are no longer needed.
Make fhandler_fifo::get_fc_handler return a reference, for use in
select.cc:peek_fifo.
Make other small changes to ensure that this commit doesn't change any
decisions based on the state of a fifo_client_handler.
The tricky part is interpreting FILE_PIPE_CLOSING_STATE, which we
translate to fc_closing. Our current interpretation, which is not
changing as a result of this commit, is that the writer at the other
end of the pipe instance is viewed as still connected from the point
of view of raw_read and determining EOF.
But it is not viewed as still connected if we are deciding whether to
unblock a new reader that is trying to open.
fifo_reader_id_t::operator == and != have been defined without type
accidentally. For some weird reason, only x86 gcc complains about
this problem, not x86_64 gcc.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
The beginning of fhandler_fifo.cc contains a long comment giving an
overview of the FIFO implementation. This is now updated to describe
the support for multiple readers.
Although we can have multiple readers open because of dup/fork/exec,
the current code does not support multiple readers opening a FIFO by
explicitly calling 'open'.
The main complication in supporting this is that when a blocking
reader tries to open and there's already one open, it has to check
whether there any writers open. It can't rely on the write_ready
event, whose state hasn't changed since the first writer opened.
To fix this, add two new named events, check_write_ready_evt and
write_ready_ok_evt, and a new method, check_write_ready().
The first event signals the owner's reader thread to call
check_write_ready(), which polls the fc_handler list to check for
connected writers. If it finds none, it checks to see if there's a
writer in the process and then sets/resets write_ready appropriately.
When check_write_ready() finishes it sets write_ready_ok_evt to signal
the reader that write_ready has been updated.
The polling is done via fifo_client_handler::pipe_state(). As long as
it's calling that function anyway, check_write_ready() updates the
state of each handler.
Also add a new lock to prevent a race if two readers are trying to
open simultaneously.
Add a take_ownership method, used by raw_read and select.cc:peek_fifo.
It wakes up all fifo_reader_threads and allows the caller to become
owner. The work is done by the fifo_reader_threads.
For synchronization we introduce several new fhandler_fifo data
members and methods:
- update_needed_evt signals the current owner to stop listening for
writer connections and update its fc_handler list.
- shared_fc_handler() gets and sets the status of the fc_handler
update process.
- get_pending_owner() and set_pending_owner() get and set the reader
that is requesting ownership.
Finally, a new 'reading_lock' prevents two readers from trying to take
ownership simultaneously.
If the owning reader is closing, wait for another reader (if there is
one) to take ownership before closing the owner's pipe handles.
To synchronize the ownership transfer, add events owner_needed_evt and
owner_found_evt, and add methods owner_needed and owner_found to
set/reset them.
Modify the fifo_reader_thread function to wake up all non-owners when
a new owner is needed.
Make a cosmetic change in close so that fhandler_base::close is called
only if we have a write handle. This prevents strace output from
being littered with statements that the null handle is being closed.
If fixup_after_exec is called on a non-close-on-exec reader whose
parent is the owner, transfer ownership to the child. Otherwise the
parent's pipe handles will be closed before any other reader can
duplicate them.
To help with this, make the cancel_evt and thr_sync_evt handles
inheritable, so that the child can terminate the parent's
fifo_reader_thread (and the parent will update the shared fc_handler
list).
Add an optional argument 'from_exec' to update_my_handlers to simplify
its use in this case; no handle duplication is required.
This is in a new shared memory section. We will use it for temporary
storage of the owner's fc_handler list when we need to change owner.
The new owner can then duplicate the pipe handles from that list
before taking ownership.
Add several shared data members and methods that are needed for the
duplication process
Add methods update_my_handlers and update_shared_handlers that carry
out the duplication.
Allow the shared list to grow dynamically, up to a point. Do this by
initially reserving a block of memory (currently 100 pages) and only
committing pages as needed.
Add methods create_shared_fc_handler, reopen_shared_fc_handler, and
remap_shared_fc_handler to create the new shared memory section,
reopen it, and commit new pages. The first is called in open, the
second is called in dup/fork/exec, and the third is called in
update_shared_handlers if more shared memory is needed.
Modify the fifo_reader_thread function to call update_my_handlers when
it finds that there is no owner. Also make it call
update_shared_handlers when the owner's thread terminates, so that the
new owner will have an accurate shared fc_handler list from which to
duplicate.
For convenience, add new methods cleanup_handlers and
close_all_handlers. And add an optional arg to add_client_handler
that allows it to create a new fifo_client_handler without creating a
new pipe instance.
Make fc_handler a pointer to malloc'd memory instead of a fixed-size
array. The size is now a new data member 'shandlers'. Call realloc
in add_client_handler if we need to grow the array.
free fc_handler in close. As long as we're touching that code, also
remove an unneeded lock.
Among all the open readers of a FIFO, one is declared to be the owner.
This is the only reader that listens for client connections, and it is
the only one that has an accurate fc_handler list.
Add shared data and methods for getting and setting the owner, as well
as a lock to prevent more than one reader from accessing these data
simultaneously.
Modify the fifo_reader_thread so that it checks the owner at the
beginning of its loop. If there is no owner, it takes ownership. If
there is an owner but it is a different reader, the thread just waits
to be canceled. Otherwise, it listens for client connections as
before.
Remove the 'first' argument from create_pipe_instance. It is not
needed, and it may be confusing in the future since only the owner
knows whether a pipe instance is the first.
When opening a reader, don't return until the fifo_reader_thread has
time to set an owner.
If the owner closes, indicate that there is no longer an owner.
Clear the child's fc_handler list in dup, and don't bother duplicating
the handles. The child never starts out as owner, so it can't use
those handles.
Do the same thing in fixup_after_fork in the close-on-exec case. In
the non-close-on-exec case, the child inherits an fc_handler list that
it can't use, but we can just leave it alone; the handles will be
closed when the child is closed.
This uniquely identifies an fhandler_fifo open for reading in any
process.
Add a new data member 'me' of this type, which is set in open, dup,
fork, and exec.
Add data and methods to the shared memory that keep track of the
number of open readers.
Increment this number in open, dup, fork, and exec. Decrement it in
close. Reset read_ready if there are no readers left.
Even though we currently allow a FIFO to be opened for reading only
once, we can still have more than one reader open because of dup and
fork. Add a named shared memory section accessible to all readers of
a given FIFO. In future commits we will add information needed by all
readers to this section
Add a class fifo_shmem_t that lets us access this information.
Add a method create_shmem that is called when a reader opens, and add
a method reopen_shmem that is called by dup, fork, and exec. (Each
new reader needs its own view of the shared memory.)
This will simplify future work.
Rename the thread from "listen_client_thread" to "fifo_reader_thread"
because it will be used for more than just listening.
Remove the fixup_before stuff, which won't be needed after future
changes to fixup_after_fork and fixup_after_exec.
- Make read_ready a manual-reset event.
- Signal read_ready in open instead of in the listen_client_thread.
- Don't reset read_ready when the listen_client thread terminates;
instead do it in close().
- Rearrange open and change its error handling.
- Add a wait_open_pipe method that waits for a pipe instance to be
available and then calls open_pipe. Use it when opening a writer if
we can't connect immediately. This can happen if the system is
heavily loaded and/or if many writers are trying to open
simultaneously.
According to Posix, a FIFO open for reading is at EOF if it is empty
and there are no writers open.
The only way to test this is to poll the fifo_client_handlers as in
raw_read and select.cc:peek_fifo. The current hit_eof instead relies
on the value of nconnected, which can be out of date. On the one
hand, it doesn't take into account writers that were connected but
have since closed. On the other hand, it doesn't take into account
writers that are in the process of opening but haven't yet connected.
Fix this by introducing a maybe_eof method that tentatively assumes
EOF if there are no connected writers after polling. Then check for
writers currently opening (via a new 'writer_opening' event), and wait
for the fifo_reader_thread to record any new connection that was made
while we were polling.
To handle the needs of peek_fifo, replace the get_fc_handle method
by a get_fc_handler method, and add a fifo_client_handler::get_state
method.
Remove the is_connected method, which was used only in peek_fifo and
is no longer needed.
Remove the nconnected data member, which was used only for the flawed
hit_eof.
Add some comments about events to fhandler.h.
Always return 0; no one is doing anything with the return value
anyway.
Remove the return value from stop_listen_client.
Make the connection event auto-reset, so that we don't have to reset
it later.
Simplify the process of connecting a bogus client when thread
termination is signaled.
Make some failures fatal.
Remove the unnecessary extra check for thread termination near the end
of listen_client_thread.
Make the values correspond to the possible return values of
fifo_client_handler::pipe_state().
When cleaning up the fc_handler list in listen_client_thread(), don't
delete handlers in the fc_closing state. I think the pipe might still
have input to be read in that case.
Set the state to fc_closing later in the same function if a connection
is made and the status returned by NtFsControlFile is
STATUS_PIPE_CLOSING.
In raw_read, don't error out if NtReadFile returns an unexpected
status; just set the state of that handler to fc_error. One writer in
a bad state doesn't justify giving up on reading.
Replace the 'fhandler_base *' member by a HANDLE to the server side of
the Windows named pipe instance. Make the corresponding
simplifications throughout.
When `cygwin-console-helper.exe` is either missing, or corresponds to a
different Cygwin runtime, we currently wait forever while setting up
access to the pseudo console, even long after the process is gone that
was supposed to signal that it set up access to the pseudo console.
Let's handle that more gracefully: if the process exited without
signaling, we cannot use the pseudo console. In that case, let's just
fall back to not using it.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The workaround to access the full disk required since Vista
and described in http://support.microsoft.com/kb/942448
(NOT ACCESSIBLE at the time of writing this commit message)
is required on floppy drives as well.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
The cast to generate the return value uses a DWORD variable
as test and set value. The error case is the constant -1.
Given the type of the other half of the conditioal expression,
-1 is cast to DWORD as well.
On 64 bit, this results in the error case returning a 32 bit
-1 value which is equivalent to (ssize_t) 4294967295 rather
than (ssize_t) -1.
Add a fixing cast.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
The return type of fhandler write methods is ssize_t. Don't
use an int to store the return value, use ssize_t. Use ptrdiff_t
for the buffer size.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
The implicit assumption seemed to be that any subsequent occurence of
the same setting in nsswitch.conf is supposed to rewrite the previous
ones completely. This was not the case if the third or any further
schema was previously defined and the last line defined less than that
(but at least 2), for example:
```
db_home: windows cygwin /myhome/%U
db_home: cygwin desc
```
Let's document this behavior as well.
Signed-off-by: David Macek <david.macek.0@gmail.com>
If the cygdrive prefix is /, then the following happens right now:
$ ln -s /tmp/foo .
$ ls -l foo
lrwxrwxrwx 1 user group 12 Apr 15 23:44 foo -> /mnt/tmp/foo
Fix this by skipping cygdrive prefix conversion to WSL drive
prefix "/mnt", if the cygdrive prefix is just "/". There's no
satisfying way to do the right thing all the time in this case
anyway. For a description and the alternatives, see
https://cygwin.com/pipermail/cygwin-developers/2020-April/011859.html
Also, fix a typo in a comment.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
If the first scheme in db_* was invalid, the code would think there
were no schemes specified and replace the second scheme with
NSS_SCHEME_DESC.
Signed-off-by: David Macek <david.macek.0@gmail.com>
So far Cygwin was jumping through hoops to restrict memory
allocation to specific regions. With the advent of VirtualAlloc2
and MapViewOfFile3 (and it's NT counterpart NtMapViewOfSectionEx),
we can skip searching for free space in the specific regions
and just call these functions and let the OS do the job more
efficiently and less racy.
Use VirtualAlloc2 on W10 1803 and later in thread stack allocation.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
So far Cygwin was jumping through hoops to restrict memory
allocation to specific regions. With the advent of VirtualAlloc2
and MapViewOfFile3 (and it's NT counterpart NtMapViewOfSectionEx),
we can skip searching for free space in the specific regions
and just call these functions and let the OS do the job more
efficiently and less racy.
Use the new functions on W10 1803 and later in mmap.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Windows 10 1803 introduced an extended memory API allowing
to specify memory regions allocations are to be taken off.
In preparation of using this API, define the struct
MEM_EXTENDED_PARAMETER and friends. Declare and allow to
autoload the functions VirtualAlloc2 and NtMapViewOfSectionEx.
Introduce a wincap flag has_extended_mem_api.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Otherwise big stacks have a higher probability to collide with
randomized PEBs and TEBs after fork.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Unfortunately Windows doesn't understand WSL symlinks,
despite being a really easy job. NT functions trying
to access paths traversing WSL symlinks return the status
code STATUS_IO_REPARSE_TAG_NOT_HANDLED. Handle this
status code same as STATUS_OBJECT_PATH_NOT_FOUND in
symlink_info::check to align behaviour to traversing
paths with other non-NTFS type symlinks.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
WSL symlinks are reparse points containing a POSIX path in UTF-8.
On filesystems supporting reparse points, use this symlink type.
On other filesystems, or in case of error, fall back to the good
old plain SYSTEM file.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Commit 4a36897af3 allowed to convert /mnt/<drive> path
prefixes to Cygwin cygdrive prefixes on the fly. However,
the patch neglected WSL symlinks pointing to the /mnt
directory. Rearrange path conversion so /mnt is converted
to the cygdrive prefix path itself.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Treat WSL symlinks just like other symlinks. Convert
absolute paths pointing to Windows drives via
/mnt/<driveletter> to Windows-style paths <driveletter>:
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Jon Turney
Brian Inglis
Ken Brown
Takashi Yano
Takashi Yano via Cygwin-patches
Corinna Vinschen
Martin Storsjö
Liu Hao
Marc Hoersken
David Allsopp
Mark Geisert
Johannes Schindelin
Biswapriyo Nath