The new function __eval_codepage_from_internal_charset
is a simplified version of the former code in
fhandler_tty.cc. It probably needs some extension,
but the gist is to use knowledge of internals to
be as quick as possible.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Both flags are outdated and collide with official flags in
sys/_default_fcntl.h, which may result in weird misbehaviour
of file functions.
O_NOSYMLINK is not used anyway.
O_DIROPEN is used in fhandler_virtual and derived classes.
The collision with O_NOFOLLOW results in spurious EISDIR
errors when, e. g., reading files in the registry.
fhandler_base::open_fs uses O_DIROPEN in the call to
fhandler_base::open, but it's not used in this context
further down the road.
Drop both flags and create an alternative "diropen" bool
flag in fhandler_virtual.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
...as on glibc right now. This is supposed to support autoconf scripts
checking for existence of these symbols in libpthread.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
GLibc will change this code in the forseeable future to align more
with FreeBSD, so this hack is not actually desired.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
- Pseudo console internally sends escape sequence CSI6n (query cursor
position) on startup of non-cygwin apps. If the terminal does not
support CSI6n, CreateProcess() hangs waiting for response. To prevent
hang, this patch disables pseudo console if the terminal does not
have CSI6n. This is checked on the first execution of non-cygwin
app using the following steps.
1) Check if the terminal support ANSI escape sequences by looking
into terminfo database. If terminfo has cursor_home (ESC [H),
the terminal is supposed to support ANSI escape sequences.
2) If the terminal supports ANSI escape sequneces, send CSI6n for
a test and wait for a responce for 40ms.
3) If there is a responce within 40ms, CSI6n is supposed to be
supported.
Also set-title capability is checked, and removes escape sequence
for setting window title if the terminal does not have the set-
title capability.
Currently, when using CYGWIN's error_start facility, the faulting
process isn't stopped while the error_start process is started when the
fault is caused by an exception. (it even seems possible in theory that
the faulting process could have exited before the error_start process
attaches).
This leads to e.g. the core dump written by CYGWIN='error_start=dumper'
in response to an exception being non-deterministic.
Remove the waitloop argument from try_to_debug(), only used in the
exception case, so the faulting process busy-waits until the error_start
process attaches.
Code archaeology to determine why the code is this way didn't really
turn up any answers, but this seems a low-risk change, as this only
changes the behaviour when:
- a debugger isn't already attached
- an error_start is specified in CYGWIN env var
- an exception has occurred which will be translated to a signal
If error_start invokes something which doesn't attach using
DebugActiveProcess(), we will spin indefinitely, but that will also
currently occur for any of the existing other uses of try_to_debug(),
which default to waitloop=TRUE.
On 32 bit x86, clang seems to miss loading input parameters based
on asm constraints for inline assembly that uses the x87 floating
registers, unless the snippet has got the volatile keyword.
Signed-off-by: Martin Storsjö <martin@martin.st>
Code taken from FreeBSD, which implements C11 threads as
wrapper around pthreads. Fix up machine/_threads.h which
is called from newlib's machine-independent threads.h to
match Cygwin's pthreads types.
Add the FreeBSD source files to libc subdir and take
opportunity to define LIBC_OFILES var in Makefile.
Add new symbols to common.din and sort symbols.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
In preparation of importing FreeBSDs stdthreads functions,
change the way pthread_yield is exported, so that the symbol
can be used internally as well.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
pthread_yield was only declared under GNU visibility,
but the function should be available under BSD visibility
as well.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
POSIX requires that key destructors are called in a loop
for each key with a non-NULL value until all values are
NULL, or until all destructors for non-NULL values
have been called at least PTHREAD_DESTRUCTOR_ITERATIONS
(per POSIX: 4) times.
Cygwinonly called all destructors with non-NULL values
exactly once. This patch fixes Cygwin to follow POSIX.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
We need one more entry than max children in the arrays.
There's no reason to do this for the static array, though.
One more entry in the overflow array is sufficient.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
On PROC_EXEC_CLEANUP, the pinfo's in chld_procs are removed.
This is done in a loop always removing the child with index 0.
This, however, results in copying the last child's pinfo in
chld_procs to position 0. Do this for 100 children and you
get 99 entirely useless copy operations.
Fix this by calling remove_proc in reverse order.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
256 children per process is a bit tight in some scenarios.
Fix this by revamping the `procs' array. Convert it to an
extensible class child_procs and rename procs to chld_procs.
Fix code throughout to use matching class methods rather than
direct access.
To allow a lot more child processes while trying to avoid
allocations at DLL startup, maintain two arrays within class
child_procs, one using a default size for 255 (i686) or 1023
(x86_64) children, the other, dynamically allocated on overflowing
the first array, giving room for another 1023 (i686) or 4095
(x86_64) processes.
On testing with a simple reproducer on a x86_64 machine with
4 Gigs RAM, a system memory overflow occured after forking
about 1450 child processes, so this simple dynamic should
suffice for a while.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
pinfo::remember with the detach parameter set to true is
the only way to call proc_subproc with PROC_DETACHED_CHILD.
This call is exclusively used in spawn to set up a pinfo for
a detached child, and that pinfo goes out of scope right
afterwards without any further action.
Drop the flag and drop the detach parameter from pinfo::remember.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
The return value is used in a numerical context and remove_proc
already returned inconsistently "true" vs. 0.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
After patch 23a779bf3d
"Cygwin: pinfo: stop remember doing reattach",
PROC_ADDCHILD actually just sets up a new child, mirroring
PROC_DETACHED_CHILD. The actual attaching of the child is
performed by action PROC_REATTACH_CHILD or pinfo::reattach
respectively.
To better reflect what's going on, rename PROC_REATTACH_CHILD
to PROC_ATTACH_CHILD and rename pinfo::reattach to pinfo::attach.
For better readability change PROC_ADDCHILD to PROC_ADD_CHILD.
Fix comments accordingly.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
change all kinds of setup references to "the Cygwin Setup program";
emphasize 64 bit and deemphasize 32 bit;
update options list;
explain why installing everything is now extremely inadvisable, with stats
- In this implementation, pseudo console is created for each native
console app. Advantages and disadvantages of this implementation
over the previous implementation are as follows.
Advantages:
1) No performance degradation in pty output for cygwin process.
https://cygwin.com/pipermail/cygwin/2020-February/243858.html
2) Free from the problem caused by difference of behaviour of control
sequences between real terminal and pseudo console.
https://cygwin.com/pipermail/cygwin/2019-December/243281.htmlhttps://cygwin.com/pipermail/cygwin/2020-February/243855.html
3) Free from the problem in cgdb and emacs gud.
https://cygwin.com/pipermail/cygwin/2020-January/243601.htmlhttps://cygwin.com/pipermail/cygwin/2020-March/244146.html
4) Redrawing screen on executing native console apps is not necessary.
5) cygwin-console-helper is not necessary for the pseudo console
support.
6) The codes for pseudo console support are much simpler than that
of the previous one.
Disadvantages:
1) The cygwin program which calls console API directly does not work.
2) The apps which use console API cannot be debugged with gdb. This
is because pseudo console is not activated since gdb uses
CreateProcess() rather than exec(). Even with this limitation,
attaching gdb to native apps, in which pseudo console is already
activated, works.
3) Typeahead key inputs are discarded while native console app is
executed. Simirally, typeahead key inputs while cygwin app is
executed are not inherited to native console app.
4) Code page cannot be changed by chcp.com. Acctually, chcp works
itself and changes code page of its own pseudo console. However,
since pseudo console is recreated for another process, it cannot
inherit the code page.
5) system_printf() does not work after stderr is closed. (Same with
cygwin 3.0.7)
6) Startup time of native console apps is about 3 times slower than
previous implemenation.
7) Pseudo console cannot be activated if it is already activated for
another process on same pty.
Any C++ app that calls 'throw' on 64-bit Cygwin results in an
exception of type STATUS_GCC_THROW (0x20474343) generated by the C++
runtime. Don't pollute the strace output by printing information
about this and other GCC exceptions.
This is necessary in order to be consistent with the following comment
in the definition of _Unwind_RaiseException() in the GCC source file
libgcc/unwind-seh.c:
The exception handler installed in crt0 will continue any GCC
exception that reaches there (and isn't marked non-continuable).
Previously we failed to do this and, as a consequence, the C++ runtime
didn't call std::terminate after an unhandled exception.
This fixes the problem reported here:
https://cygwin.com/pipermail/cygwin/2019-October/242795.htmlhttps://sourceware.org/pipermail/cygwin/2020-August/245897.html
- After commit 095972ce5b, charset
conversion in mintty is broken if charset is set to other than
UTF-8. This seems to be caused because mintty does not set locale
yet at fork() call. This patch changes the timing of set_locale()
call again to avoid this issue.
- If native app is exec()'ed in a new pty, setup_locale() loses the
chance to be called. For example, with "mintty -e cmd", charset
conversion does not work as expected. This patch fixes the issue.
- If a lot of mintty are started in a short time from a mintty, some
of them hang with empty screen, crash immediately or hang on exiting
mintty. The following report seems to be related to this issue.
https://cygwin.com/pipermail/cygwin/2020-August/245751.html
The cause is not clear at all, but this patch seems to solve the
issue.
- update path to Unicode windowsZones.xml file
- drop Windows XP considerations
- regenerate tzmap.h
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
This patch has been inspired by the Linux kernel patch
294f69e662d1 compiler_attributes.h: Add 'fallthrough' pseudo keyword for switch/case use
written by Joe Perches <joe AT perches DOT com> based on an idea from
Dan Carpenter <dan DOT carpenter AT oracle DOT com>. The following text
is from the original log message:
Reserve the pseudo keyword 'fallthrough' for the ability to convert the
various case block /* fallthrough */ style comments to appear to be an
actual reserved word with the same gcc case block missing fallthrough
warning capability.
All switch/case blocks now should end in one of:
break;
fallthrough;
goto <label>;
return [expression];
continue;
In C mode, GCC supports the __fallthrough__ attribute since 7.1,
the same time the warning and the comment parsing were introduced.
Cygwin-only: add an explicit -Wimplicit-fallthrough=5 to the build
flags.
Currently raw_read makes two passes through the list of clients. On
the first pass it tries to read from the client from which it last
read successfully. On the second pass it tries to read from all
connected clients.
Add a new pass in between these two, in which raw_read tries to read
from all clients that are in the fc_input_avail case. This should be
more efficient in case select was previously called and detected input
available.
Slightly tweak the first pass. If a client is marked as having the
last successful read but reading from it now finds no input, don't
unmark it unless we successfully read from a different client on one
of the later passes.
The fifo_reader thread function and the function select.cc:peek_fifo()
can both change the state of a fifo_client_handler. These changes are
made under fifo_client_lock, so there is no race, but the changes can
still be incompatible.
Add code to make sure that only one of these functions can change the
state from its initial fc_listening state. Whichever function does
this calls the fhandler_fifo::record_connection method, which is now
public so that peek_fifo can call it.
Slightly modify that method to make it suitable for being called by
peek_fifo.
Make a few other small changes to the fifo_reader thread function to
change how it deals with the STATUS_PIPE_CLOSING value that can
(rarely) be returned by NtFsControlFile.
Add commentary to fhandler_fifo.cc to explain fifo_client connect
states and where they can be changed.
Don't try to read from fifo_client_handlers that are in the fc_closing
state. Experiments have shown that this always yields
STATUS_PIPE_BROKEN, so it just wastes a Windows system call.
Re-order the values in enum fifo_client_connect_state to reflect the
new status of fc_closing.
Rename the existing set_state() to query_and_set_state() to reflect
what it really does. (It queries the O/S for the pipe state.) Add a
new set_state() method, which is a standard setter, and a
corresponding getter get_state().
fhandler_fifo::take_ownership() is called from select.cc::peek_fifo
and fhandler_fifo::raw_read and could potentially block indefinitely
if something goes wrong. This is always undesirable in peek_fifo, and
it is undesirable in a nonblocking read. Fix this by adding a timeout
parameter to take_ownership.
Arbitrarily use a 1 ms timeout in peek_fifo and a 10 ms timeout in
raw_read. These numbers may have to be tweaked based on experience.
Replace the call to cygwait in take_ownership by a call to WFSO.
There's no need to allow interruption now that we have a timeout.
fhandler_fifo::take_ownership() tacitly assumes that the current
owner's fifo_reader_thread will be woken up from WFMO when
update_needed_evt is signaled. But it's possible that the the current
owner's fifo_reader_thread is at the beginning of its main loop rather
than in its WFMO call when that event is signaled.
In this case the owner will never see that the event has been
signaled, and it will never update the shared fifo_client_handlers.
The reader that wants to take ownership will then spin its wheels
forever.
Fix this by having the current owner call update_shared_handlers at
the beginning of its loop, if necessary.
microcode is unsigned long long, printed by _small_sprintf using %x;
Cygwin32 used last 4 bytes of microcode for next field MHz, printing 0;
use correct _small_sprintf format %X to print microcode, producing
correct MHz value under Cygwin32
CPUID 7:0 EDX[14] serialize added in linux-next 5.8 by Ricardo Neri-Calderon:
The Intel architecture defines a set of Serializing Instructions (a
detailed definition can be found in Vol.3 Section 8.3 of the Intel "main"
manual, SDM). However, these instructions do more than what is required,
have side effects and/or may be rather invasive. Furthermore, some of
these instructions are only available in kernel mode or may cause VMExits.
Thus, software using these instructions only to serialize execution (as
defined in the manual) must handle the undesired side effects.
As indicated in the name, SERIALIZE is a new Intel architecture
Serializing Instruction. Crucially, it does not have any of the mentioned
side effects. Also, it does not cause VMExit and can be used in user mode.
This new instruction is currently documented in the latest "extensions"
manual (ISE). It will appear in the "main" manual in the future.
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/arch/x86/include/asm/cpufeatures.h?id=85b23fbc7d88f8c6e3951721802d7845bc39663d
Newlib's posix_spawn has been taken from FreeBSD. The code relies on
BSD-specific behaviour of vfork, namely the fact that vfork blocks
the parent until the child exits or calls execve as well as the fact
that the child shares parent memory in non-COW mode.
This behaviour can't be emulated by Cygwin. Cygwin's vfork is
equivalent to fork. This is POSIX-compliant, but it's lacking BSD's
vfork ingrained synchronization of the parent to wait for the child
calling execve, or the chance to just write a variable and the parent
will see the result.
So this requires a Cygwin-specific solution. The core function of
posix_spawn, called do_posix_spawn is now implemented twice, once using
the BSD method, and once for Cygwin using Windows synchronization under
the hood waiting for the child to call execve and signalling errors
upstream. The Windows specifics are hidden inside Cygwin, so newlib
only calls internal Cygwin functions.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Stop after we've written the dump in response to the initial breakpoint
EXCEPTION_DEBUG_EVENT we recieve for attaching to the process.
(rather than bogusly sitting there for 20 seconds waiting for more debug
events from a stopped process after we've already written the dump).
It's working on 32 bit OSes only anyway. It even fails on WOW64.
Drop unsupported NtMapViewOfSection flags.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Use the (undocumented) MEMORY_WORKING_SET_EX_INFORMATION in dumper to
determine if a MEM_IMAGE region is unsharable, and hence has been
modified.
After this, we will end up dumping memory regions where:
- state is MEM_COMMIT (i.e. is not MEM_RESERVE or MEM_FREE), and
-- type is MEM_PRIVATE and protection allows reads (i.e. not a guardpage), or
-- type is MEM_IMAGE and attribute is non-sharable (i.e. it was WC, got
written to, and is now a RW copy)
After this, we will end up dumping memory regions where:
- state is MEM_COMMIT (i.e. is not MEM_RESERVE or MEM_FREE), and
-- type is MEM_PRIVATE and protection allows reads (i.e. not a guardpage), or
-- type is MEM_IMAGE and protection allows writes
Making this decision based on the current protection isn't 100% correct,
because it may have been changed using VirtualProtect(). But we don't
know how to determine if a region is shareable.
(As a practical matter, anything which gets us the stack (MEM_PRIVATE)
and .data/.bss (RW MEM_IMAGE) is going to be enough for 99% of cases)
Commit 605bdcd410 enabled mapping beyond
EOF in 64 bit environments. But the variable 'orig_len' did not get
rounded up to a multiple of 64K. This rounding was done on 32 bit
only. Fix this by rounding up orig_len on 64 bit, in the same place
where 'len' is rounded up.
Rounding up is needed to make sigbus_page_len a multiple of the
allocation granularity.
In addition, failing to round up could cause orig_len to be smaller
than len. Since these are both unsigned values, the statement
'orig_len -= len' could then cause orig_len to be huge, and mmap would
fail with errno EFBIG.
I observed this failure while debugging the problem reported in
https://sourceware.org/pipermail/cygwin/2020-July/245557.html.
The failure can be seen by running the test case in that report under
gdb or strace.
- After commit 0365031ce1, key input
becomes not working by following steps.
1) Start cmd.exe in mintty.
2) Open another mintty.
3) Execute "echo AAA > /dev/pty*" (pty* is the pty opened in 1.)
This patch fixes the issue.
Use cygwait in take_ownership to allow interruption while waiting to
become owner. Return the cygwait return value or a suitable value to
indicate an error.
raw_read now checks the return value and acts accordingly.
If update_my_handlers fails to duplicate one or more handles, just
mark the corresponding handlers as being in an error state.
But if update_my_handlers is unable to open the process of the
previous owner, it's likely that something serious has gone wrong, so
we continue to make that a fatal error.
When a reader takes ownership in fifo_reader_thread, it now goes
directly to the part of the main loop that listens for a connection.
Previously it went back to the beginning of the loop.
Also, if the reader has to delay taking ownership because the previous
owner has not finished updating the shared fifo_client handlers, it
now checks to see if cancel_evt has been set. Previously it might
have had to spin its wheels unnecessarily only to eventually find that
its thread had been canceled.
Add a bool member 'last_read' to the fifo_client_handler structure,
which is set to true on a successful read. This is used by raw_read
as follows.
When raw_read is called, it first locates the writer (if any) for
which last_read is true. raw_read tries to read from that writer and
returns if there is input available. Otherwise, it proceeds to poll
all the writers, as before.
The effect of this is that if a writer writes some data that is only
partially read, the next attempt to read will continue to read from
the same writer. This should reduce the interleaving of output from
different writers.
When a reader opens, it needs to block if there are no writers open
(unless is is opened with O_NONBLOCK). This is easy for the first
reader to test, since it can just wait for a writer to signal that it
is open (via the write_ready event). But when a second reader wants
to open, all writers might have closed.
To check this, use a new '_nwriters' member of struct fifo_shmem_t,
which keeps track of the number of open writers. This should be more
reliable than the previous method.
Add nwriters_lock to control access to shmem->_nwriters, and remove
reader_opening_lock, which is no longer needed.
Previously only readers had access to the shared memory, but now
writers access it too so that they can increment _nwriters during
open/dup/fork/exec and decrement it during close.
Add an optional 'only_open' argument to create_shmem for use by
writers, which only open the shared memory rather than first trying to
create it. Since writers don't need to access the shared memory until
they have successfully connected to a pipe instance, they can safely
assume that a reader has already created the shared memory.
For debugging purposes, change create_shmem to return 1 instead of 0
when a reader successfully opens the shared memory after finding that
it had already been created.
Remove check_write_ready_evt, write_ready_ok_evt, and
check_write_ready(), which are no longer needed.
When opening a writer and looping to try to get a connection, recheck
read_ready at the top of the loop since the number of readers might
have changed.
To slightly speed up the process of opening the first reader, take
ownership immediately rather than waiting for the fifo_reader_thread
to handle it.
When the owning reader closes and there are still readers open, the
owner needs to wait for a new owner to be found before closing its
fifo_client handlers. This involves a loop in which dec_nreaders is
called at the beginning and inc_nreaders is called at the end. Any
other reader that tries to access shmem->_nreaders during this loop
will therefore get an inaccurate answer.
Fix this by adding an nreaders method and using it instead of
dec_nreaders and inc_nreaders. Also add nreaders_lock to control
access to the shmem->_nreaders.
Make various other changes to improve the reliability of finding a new
owner.
Since FD_CONNECT is only given once, we manually need to set
FD_WRITE for connection failed sockets to have consistent
behaviour in programs calling poll/select multiple times.
Example test to non-listening port: curl -v 127.0.0.1:47
If the acl_t struct was at or above 0x80000000 then the pointer was
sign-extended to 0xffff_ffff_8000_0000 and so the index was lost.
Signed-off-by: David Allsopp <david.allsopp@metastack.com>
This should slightly speed up especially path conversions,
given there's one less function call rearranging all function
arguments in registers/stack (and less stack pressure).
For clarity, rename overloaded sys_wcstombs to _sys_wcstombs
and sys_cp_mbstowcs to _sys_mbstowcs.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Corinna Vinschen
David McFarland via Cygwin-patches
Jon Turney
Ken Brown
Martin Storsjö
Liu Hao
Brian Inglis
Takashi Yano
Marc Hoersken
David Allsopp