while debugging a problem introduced in commit
63b503916d ("Cygwin: tls_pathbuf: Use Windows heap")
a hang in fork was encountered using the original implementation
of tls_pathbuf:
Using tmp_pathbuf inside the code block guarded by __malloc_trylock
may call malloc from tmp_pathbuf::w_get and thus trying to lock an
exclusive SRW lock recursively, which results in a deadlock.
Allocate a small SECURITY_ATTRIBUTES block on the stack rather than
allocating a 64K tmp_pathbuf. This avoids the potential malloc call.
Drop the __malloc_trylock call entirely. There must not be a malloc
call inside the frok::parent block guarded by __malloc_lock, and
just trying to lock is too dangerous inside fork while other threads
might actually chage the content of the heap. Additionally, add a
comment frowning on malloc usage inside tis code block.
Fixes: 44a79a6eca ("Cygwin: convert malloc lock to SRWLOCK")
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Commit e1ce752a1d, "Cygwin: remove miscellaneous 32-bit code", removed
most occurrences of '#ifdef __x86_64__'. Restore those occurrences
that guarded code specific to the AMD64 processor, and #error out if
the processor is different. This will make it easier to find
AMD64-specific code if we ever want to add support for a different
64-bit processor (e.g., ARM64).
These have no effect on x86_64. Retain a few occurrences of __cdecl
in files imported from other sources.
Also retain all occurrences of WINAPI, even though the latter is
simply a macro that expands to __stdcall. Most of these occurrences
are associated with Windows API functions, and removing them might
make the code confusing instead of simpler.
Per https://cygwin.com/pipermail/cygwin-developers/2021-October/012429.html,
we may encounter a crash when starting multiple threads during process
startup (here: fhandler_fifo::fixup_after_{fork,exec}) which in turn
allocate memory via malloc.
The problem is concurrent usage of malloc before the malloc muto has
been initialized.
To fix this issue, convert the muto to a SRWLOCK and make sure it is
statically initalized. Thus, malloc can be called as early as necessary
and malloc_init is only required to check for user space provided malloc.
Note that this requires to implement a __malloc_trylock macro to be
called from fork.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
pinfo::remember with the detach parameter set to true is
the only way to call proc_subproc with PROC_DETACHED_CHILD.
This call is exclusively used in spawn to set up a pinfo for
a detached child, and that pinfo goes out of scope right
afterwards without any further action.
Drop the flag and drop the detach parameter from pinfo::remember.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
After patch 23a779bf3d
"Cygwin: pinfo: stop remember doing reattach",
PROC_ADDCHILD actually just sets up a new child, mirroring
PROC_DETACHED_CHILD. The actual attaching of the child is
performed by action PROC_REATTACH_CHILD or pinfo::reattach
respectively.
To better reflect what's going on, rename PROC_REATTACH_CHILD
to PROC_ATTACH_CHILD and rename pinfo::reattach to pinfo::attach.
For better readability change PROC_ADDCHILD to PROC_ADD_CHILD.
Fix comments accordingly.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
- In this implementation, pseudo console is created for each native
console app. Advantages and disadvantages of this implementation
over the previous implementation are as follows.
Advantages:
1) No performance degradation in pty output for cygwin process.
https://cygwin.com/pipermail/cygwin/2020-February/243858.html
2) Free from the problem caused by difference of behaviour of control
sequences between real terminal and pseudo console.
https://cygwin.com/pipermail/cygwin/2019-December/243281.htmlhttps://cygwin.com/pipermail/cygwin/2020-February/243855.html
3) Free from the problem in cgdb and emacs gud.
https://cygwin.com/pipermail/cygwin/2020-January/243601.htmlhttps://cygwin.com/pipermail/cygwin/2020-March/244146.html
4) Redrawing screen on executing native console apps is not necessary.
5) cygwin-console-helper is not necessary for the pseudo console
support.
6) The codes for pseudo console support are much simpler than that
of the previous one.
Disadvantages:
1) The cygwin program which calls console API directly does not work.
2) The apps which use console API cannot be debugged with gdb. This
is because pseudo console is not activated since gdb uses
CreateProcess() rather than exec(). Even with this limitation,
attaching gdb to native apps, in which pseudo console is already
activated, works.
3) Typeahead key inputs are discarded while native console app is
executed. Simirally, typeahead key inputs while cygwin app is
executed are not inherited to native console app.
4) Code page cannot be changed by chcp.com. Acctually, chcp works
itself and changes code page of its own pseudo console. However,
since pseudo console is recreated for another process, it cannot
inherit the code page.
5) system_printf() does not work after stderr is closed. (Same with
cygwin 3.0.7)
6) Startup time of native console apps is about 3 times slower than
previous implemenation.
7) Pseudo console cannot be activated if it is already activated for
another process on same pty.
Newlib's posix_spawn has been taken from FreeBSD. The code relies on
BSD-specific behaviour of vfork, namely the fact that vfork blocks
the parent until the child exits or calls execve as well as the fact
that the child shares parent memory in non-COW mode.
This behaviour can't be emulated by Cygwin. Cygwin's vfork is
equivalent to fork. This is POSIX-compliant, but it's lacking BSD's
vfork ingrained synchronization of the parent to wait for the child
calling execve, or the chance to just write a variable and the parent
will see the result.
So this requires a Cygwin-specific solution. The core function of
posix_spawn, called do_posix_spawn is now implemented twice, once using
the BSD method, and once for Cygwin using Windows synchronization under
the hood waiting for the child to call execve and signalling errors
upstream. The Windows specifics are hidden inside Cygwin, so newlib
only calls internal Cygwin functions.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
- PTY has a bug reported in:
https://cygwin.com/ml/cygwin/2020-02/msg00067.html.
This is the result of state mismatch between real pseudo console
attaching state and state variable. This patch fixes the issue.
- Pseudo console support introduced by commit
169d65a577 sometimes cause random
crash or freeze by pressing ^C while cygwin and non-cygwin
processes are executed simultaneously in the same pty. This
patch is a workaround for this issue.
- Pseudo console support introduced by commit
169d65a577 has some bugs which
cause mismatch between state variables and real pseudo console
state regarding console attaching and r/w pipe switching. This
patch fixes this issue by redesigning the state management.
- Support pseudo console in PTY. Pseudo console is a new feature
in Windows 10 1809, which provides console APIs on virtual
terminal. With this patch, native console applications can work
in PTYs such as mintty, ssh, gnu screen or tmux.
Do not attach to the child before it was successfully initialized, or we
would need more sophisticated cleanup on child initialization failure,
like suppressing SIGCHILD delivery with multiple threads ("waitproc")
involved.
Improves "Cygwin: fork: Remember child not before success.",
commit f03ea8e1c5, which leads to fork
problems if cygserver is running:
https://cygwin.com/ml/cygwin-patches/2019-q2/msg00155.html
During fork, the child process requires the process table to be
initialized for fixup_shms_after_fork, while still allowing subsequent
dlls.load_after_fork to fail silently (for when the "forkable" hardlinks
are not created yet).
pinfo::remember not performing reattach anymore requires explicit
pinfo::reattach now where appropriate.
Prepares to improve "Cygwin: fork: Remember child not before success."
commit f03ea8e1c5, which leads to fork
problems if cygserver is running:
https://cygwin.com/ml/cygwin-patches/2019-q2/msg00155.html
Do not remember the child before it was successfully initialized, or we
would need more sophisticated cleanup on child initialization failure,
like cleaning up the process table and suppressing SIGCHILD delivery
with multiple threads ("waitproc") involved. Compared to that, the
potential slowdown due to an extra yield () call should be negligible.
Pause the child process after performing fork fixups even if there were
no dynamically loaded dlls with extra data/bss transfers to wait for.
This allows the parent process to cancel the current fork call even if
the child process was successfully initialized already.
This is a preparation for when the parent does remember the child no
earlier than after successful child initialization.
In dll_crt0_0, both threadinterface->Init and sigproc_init allocate
windows object handles using unpredictable memory regions, which may
collide with dynamically loaded dlls when they were relocated.
Create process with standard rights, plus
PROCESS_QUERY_LIMITED_INFORMATION for authenticated users. This
allows to fetch basic process information and thus /proc/<PID>/stat
to succeed on foreign processes.
While at it, fix formatting in CreateProcess calls.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
To support in-cygwin package managers, the fork() implementation must
not rely on .exe and .dll files to stay in their original location, as
the package manager's job is to replace these files. Instead, when the
first fork try fails, and we have NTFS, we use hardlinks to the original
binaries in /var/run/cygfork/ to create the child process during the
second fork try, along the main.exe.local file to enable the "DotLocal
Dll Redirection" feature for the dlls.
The (probably few) users that need an update-safe fork manually have to
create the /var/run/cygfork/ directory for now, using:
mkdir --mode=a=rwxt /var/run/cygfork
* child_info.h: Bump CURR_CHILD_INFO_MAGIC.
(enum child_status): Add _CI_SILENTFAIL flag.
(struct child_info): Add silentfail setter and getter.
* winsup.h (child_copy): Add bool silentfail parameter.
* cygheap.cc: Pass silentfail parameter to child_copy.
* dcrt0.cc: Ditto.
* dll_init.h (struct dll): Define public inline method forkedntname.
(struct dll_list): Declare private method find_by_forkedntname.
* dll_init.cc (struct dll_list): Implement find_by_forkedntname.
(dll_list::alloc): Use find_by_forkedntname when in load after fork.
(dll_list::load_after_fork_impl): Load dlls using dll::forkedntname.
* fork.cc (frok::parent): Set silentfail child info flag. Pass
silentfail parameter to child_copy. Use forkedntname of
dlls.main_executable.
(fork): When first dofork run failed and did not use forkables,
run dofork again with_forkables set to true.
(child_copy): Use debug_printf if silentfail is true,
system_printf otherwise.
In preparation to protect fork() against dll- and exe-updates, create
hardlinks to the main executable and each loaded dll in subdirectories
of /var/run/cygfork/, if that one exists on the NTFS file system.
The directory names consist of the user sid, the main executable's NTFS
IndexNumber, and the most recent LastWriteTime of all involved binaries
(dlls and main executable). Next to the main.exe hardlink we create the
empty file main.exe.local to enable dll redirection.
The name of the mutex to synchronize hardlink creation/cleanup also is
assembled from these directory names, to allow for synchronized cleanup
of even orphaned hardlink directories.
The hardlink to each dynamically loaded dll goes into another directory,
named using the NTFS IndexNumber of the dll's original directory.
* Makefile.in (DLL_OFILES): Add forkable.o.
* dll_init.h (struct dll): Declare member variables fbi, fii,
forkable_ntname. Declare methods nominate_forkable,
create_forkable.
(struct dll_list): Declare enum forkables_needs. Declare member
variables forkables_dirx_size, forkables_dirx_ntname,
forkables_mutex_name, forkables_mutex. Declare private methods
forkable_ntnamesize, prepare_forkables_nomination,
update_forkables_needs, update_forkables, create_forkables,
denominate_forkables, close_mutex, try_remove_forkables,
set_forkables_inheritance, request_forkables. Declare public
static methods ntopenfile, read_fii, read_fbi. Declare public
methods release_forkables, cleanup_forkables. Define public
inline method setup_forkables.
* dll_init.cc (dll_list::alloc): Allocate memory to hold the
name of the hardlink in struct dll member forkable_ntname.
Initialize struct dll members fbi, fii.
(dll_list::load_after_fork): Call release_forkables method.
* fork.cc: Rename public fork function to static dofork, add
with_forkables as bool pointer parameter. Add new fork function
calling dofork. (struct frok): Add bool pointer member
with_forkables, add as constructor parameter.
(frok::parent): Call dlls.setup_forkables before CreateProcessW,
dlls.release_forkables afterwards.
* pinfo.cc (pinfo::exit): Call dlls.cleanup_forkables.
* syscalls.cc (_unlink_nt): Rename public unlink_nt function to
static _unlink_nt, with 'shareable' as additional argument.
(unlink_nt): New, wrap _unlink_nt for original behaviour.
(unlink_nt_shareable): New, wrap _unlink_nt to keep a binary
file still loadable while removing one of its hardlinks.
* forkable.cc: New file.
Implement static functions mkdirs, rmdirs, rmdirs_synchronized,
stat_real_file_once, format_IndexNumber, rootname, sidname,
exename, lwtimename. Define static array forkable_nameparts.
(struct dll): Implement nominate_forkable, create_forkable.
(struct dll_list): Implement static methods ntopenfile,
read_fii, read_fbi. Implement forkable_ntnamesize,
Even for the main executable and cygwin1.dll store the file name as full
NT path. Create the child process using the main executable's file name
converted from the full NT path stored before.
* dll_init.cc (dll_list::alloc): Search for DLL_SELF type entry
with module name like for DLL_LINK, use full NT path to search
for DLL_LOAD type only. For DLL_SELF type do not indicate
having a destructor to be called.
(dll_list::find): Ignore DLL_SELF type entries.
(dll_list::init): Ditto. Call track_self method.
(dll_list::track_self): New.
(dll_list::load_after_fork): Call track_self method.
* dll_init.h (enum dll_type): Add DLL_SELF, for the main
executable and cygwin1.dll.
(struct dll_list): Declare private method track_self. Declare
member variable main_executable.
* fork.cc (frok::parent): Use ntname from dlls.main_executable
to create child process, converted to short path using
dll_list::buffered_shortname.
Store loaded dll's file name as full NT path.
* dll_init.h (struct dll): Rename member variable name to ntname.
(struct dll_list): Declare private static member variable
nt_max_path_buffer. Declare public static methods form_ntname,
form_shortname. Define public static methods nt_max_path_buf,
buffered_shortname.
(dll_list::operator []): Use PCWCHAR rather than const PWCHAR.
(dll_list::find_by_modname): Ditto.
* dll_init.cc (in_load_after_fork): Define earlier in file.
(struct dll_list): Rename member variable name to ntname.
Define nt_max_path_buffer variable.
Implement static methods form_ntname, form_shortname.
(dll_list::operator []): Use PCWCHAR rather than const PWCHAR.
(dll_list::find_by_modname): Ditto.
(reserve_at): Ditto.
(release_at): Ditto.
(dll_list::alloc): Use nt_max_path_buf method instead of local
buffer. Store module file name as full NT path, convert using
the form_ntname static method.
(dll_list::load_after_fork): Call load_after_fork_impl only when
reload_on_fork is set.
* fork.cc (frok::child): Call dlls.load_after_fork even without
need to dynamically load dlls.
(frok::parent): Move syscall_printf into the retry loop.
Move TerminateProcess call into cleanup code to make sure child
doesn't linger in some border cases.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
When fork finally fails although both CreateProcess and creating the
"cygpid.N" shared memory section succeeded, we have to release that
shared memory section as well - before releasing the process handle.
Otherways we leave an orphan "cygpid.N" shared memory section, and any
subsequent cygwin process receiving the same PID fails to initialize.
* fork.cc (frok::parent): Call child.allow_remove in cleanup code.
Using the Windows PID as Cygwin PID has a few drawbacks:
- the PIDs on Windows get reused quickly. Some POSIX applications choke
on that, so we need extra code to avoid too quick PID reuse.
- The code to avoid PID reuse keeps parent process handles and
(depending on a build option) child processes open unnecessarily.
- After an execve, the process has a split personality: Its Windows PID
is a new PID, while its Cygwin PID is the PID of the execve caller
process. This requires to keep two procinfo shared sections open, the
second just to redirect process info requests to the first, correct
one.
This patch changes the way Cygwin PIDs are generated:
- Cygwin PIDs are generated independently of the Windows PID, in a way
expected by POSIX processes. The PIDs are created incrementally in
the range between 2 and 65535, round-robin.
- On startup of the first Cygwin process, choose a semi-random start PID
for the first process in the lower PID range to make the PIDs slightly
unpredictable. This may not be necessary but it seems kind of inviting
to know that the first Cygwin process always starts with PID 2.
- Every process not only creates the shared procinfo section, but also a
symlink in the NT namespace, symlinking the Windows PID to the Cygwin
PID. This drops the need for the extra procinfo section after execve.
- Don't keep other process handles around unnecessarily.
- Simplify the code creating/opening the shared procinfo section and
make a clear distinction between interfaces getting a Cygwin PID and
interfaces getting a Windows PID.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
- Rename files timer.* to posix_timer.*.
- Reimplement using an OS timer rather than a handcrafted wait loop.
- Use a Slim R/W Lock for synchronization.
- Drop timer chaining. It doesn't server a purpose since all timers
are local only.
- Rename ttstart to itimer_tracker to better reflect its purpose.
It's not the anchor for a timer chain anymore anyway.
- Drop fixup_timers_after_fork. Everything is process-local, nothing
gets inherited.
- Rename timer_tracker::disarm_event to disarm_overrun_event for
better readability.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
When fork fails, we can use "%s" now with system_sprintf for the errmsg
rather than a (potentially too small) buffer for the format string.
* fork.cc (fork): Use "%s" with system_printf now.
So far we copy *_impure_ptr into _main_tls->local_clib if the child
process has been forked from a pthread. But that's not required.
The local_clib area of the new thread is on the stack and the stack
gets copied from the parent anyway (in frok::parent). So we only
have to make sure _main_tls is pointing to the right address and
do the simple post-fork thread init.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Bump GPLv2+ to GPLv3+ for some files, clarify BSD 2-clause.
Everything else stays under GPLv3+.
New Linking Exception exempts resulting executables from LGPLv3 section 4.
Add CONTRIBUTORS file to keep track of licensing.
Remove 'Copyright Red Hat Inc' comments.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
* child_info.h (struct child_info): Add member from_main.
* fork.cc (frok::child): Check from_main rather than stackaddr.
(frok::parent): Set ch.from_main if running in the main thread.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
* child_info.h (CURR_CHILD_INFO_MAGIC): Align to below change.
(class child_info_fork): Rename stacktop to stacklimit. Rename
stackbottom to stackbase. Accommodate name change throughout Cygwin.
Rephrase comments to be clearer.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
* cygtls.h (_tlsbase): Remove. Replace throughout with
NtCurrentTeb()->Tib.StackBase.
(_tlstop): Remove. Replace throughout with
NtCurrentTeb()->Tib.StackLimit.
* dcrt0.cc (child_info_fork::alloc_stack): Move definition of local
teb variable up to be used throughout.
* include/cygwin/config.h (__getreent): Use inline function on both
architectures.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
* fork.cc (frok::parent): Set stacktop value based on requested stack
pointer value in child. Explain why.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
* fork.cc (frok::parent): When initializing pinfo for child new PID_NEW flag +
actual defined constant rather than raw number. Don't set start_time here.
* pinfo.cc (pinfo::thisproc): Use PID_NEW when initializing pinfo. Avoid
checking h for NULL multiple times. Don't set start_time here.
(pinfo_init): Aways set ppid last. Tweak strace output.
(pinfo::init): Handle new PID_NEW flag. Wait for shared memory to contain
useful information. Set start_time if PID_NEW.
(_onreturn:h): Define as HANDLE rather than HANDLE *.
(_onreturn::~onreturn): Accommodate h definition change.
(_onreturn::no_close_handle): Rename from no_close_p_handle. Take a pinfo arg
and set hProcess to h before zeroing.
(winpids::add): Don't open a handle to our own process. Change logic
associated with when a handle gets closed. Accommodate no_close_handle
changes.
(winpids::enum_processes): Simplify process enumeration loop.
(winpids::set): Eliminate ill-considered malloc locking.
* sigproc.cc (proc_subproc): Always set ppid last.
stacksize to SIZE_T. Cast to SIZE_T in pointer arithmetic. Slightly
enhance output in case of a fatal error.
* fork.cc (frok::parent): Always set ch.stackaddr to DeallocationStack
value of current thread to help stack reservation in
child_info_fork::alloc_stack_hard_way along. Simplify subsequent code
storing stack values in ch. Print guardsize in hex, too.
Corinna Vinschen
Ken Brown
Takashi Yano
Michael Haubenwallner
Sergejs Lukanihins
Christopher Faylor