- Previously, input and output pipes were switched together between
the traditional pty and the pseudo console. However, for example,
if stdin is redirected to another device, it is better to leave
input pipe traditional pty side even for non-cygwin program. This
patch realizes such behaviour.
- When the I/O pipe is switched to the pseudo console side, the
behaviour of Ctrl-C was unstable. This rarely happens, however,
for example, shell sometimes crashes by Ctrl-C in that situation.
Furthermore, Ctrl-C was ignored if output of non-cygwin program
is redirected to pipe. This patch fixes these issues.
- select() did not work correctly when both read and except are
polled simultaneously for the same fd and the r/w pipe is switched
to pseudo console side. This patch fixes this isseu.
- Pseudo console support introduced by commit
169d65a577 has some bugs which
cause mismatch between state variables and real pseudo console
state regarding console attaching and r/w pipe switching. This
patch fixes this issue by redesigning the state management.
- Support pseudo console in PTY. Pseudo console is a new feature
in Windows 10 1809, which provides console APIs on virtual
terminal. With this patch, native console applications can work
in PTYs such as mintty, ssh, gnu screen or tmux.
So far negative values were denoting files, positive values
denoting directories. We should prefer a less error prone
method. Redefine virtual_ftype_t to contain only positive
values and replace checks for negativ or positive values with
inline functions virt_ftype_isfile() and virt_ftype_isdir().
Drop outdcated comments referring to numerical virtual_ftype_t
values.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Add a method fifo_client_handler::pipe_state that queries Windows for
the state of a pipe instance. Use this to help terminate the
listen_client thread cleanly.
If the last client handler is useless, delete it instead of declaring
it invalid.
Add a method fhandler_fifo::check_listen_client_thread that checks
whether the thread is running. Use it in raw_read instead of just
testing the handle listen_client_thr.
On exit from the listen_client thread, make sure there's no pending
FSCTL_PIPE_LISTEN request. Otherwise we might get a client connection
after restarting the thread, and we won't have a handle for
communicating with that client.
Remove the retry loop in the case of STATUS_PIPE_LISTENING; that case
shouldn't occur.
Remove the now-unused fc_connecting value from
fifo_client_connect_state.
It's now up to the caller to pass a handle to open_pipe and, if
desired, to call set_handle on return.
This will be useful for a future commit, in which we will open a
client connection without setting an io_handle.
When opening a duplexer, open a client connection to the first client
handler. Previously we gave the duplexer a bogus write handle, which
was just a duplicate of the first client handler's handle. This meant
that we had a pipe server with no clients connected, and all I/O
attempts failed with STATUS_PIPE_LISTENING.
Extend the last fcntl change to duplexers.
Remove a now unused fifo_client_handler constructor, as well as the
long unusued method fifo_client_handler::connect.
Don't create the pipe in duplex mode; the server handle will only be
used for reading.
The blocking mode of the Windows pipe underlying a writer is set to
match that of the writer itself when the latter is opened. Define
fhandler_fifo::fcntl to keep the pipe and the writer in sync if the
blocking mode is changed via fcntl.
Add methods need_fixup_before, init_fixup_before, and
fixup_before_fork_exec to accomplish this. Stopping the thread makes
sure that the client handler lists of the parent and child remain in
sync while the forking/execing is in progress.
Define fhandler:fifo::fixup_after_exec, which sets listen_client_thr
and lct_termination_evt to NULL. This forces the listen_client thread
to restart on the first attempt to read after an exec. Previously the
exec'd process could hang in fhandler_fifo::raw_read.
Remove fifo_client_handler::connect and move its code into
listen_client_thread. That way we can check the return status when a
client handler's connect_evt is signaled. Previously we incorrectly
assumed there was a successful connection.
Also simplify listen_client_thread in the following ways:
- Replace fhandler_fifo::disconnect_and_reconnect by a new
delete_client_handler method. Now we just delete invalid client
handlers rather than trying to re-use them.
- Try to maintain a client handler list that consists of connected
client handlers and exactly one that is listening for a connection.
This allows us to call WaitForMultipleObjects with only two wait
objects.
- Remove 'dummy_evt' from the fifo_client_handler struct; it is no
longer needed.
- On exit from listen_client_thread, delete the "extra" (listening)
client handler. Otherwise there could be a connection that doesn't
get recorded in the client handler list. This could happen when a
file descriptor is being duplicated.
Don't set the write end of the pipe to non-blocking mode if the FIFO
is opened in blocking mode.
In fhandler_fifo::raw_write in blocking mode, wait for the write to
complete rather than returning -1 with EAGAIN.
If the amount to write is large, write in smaller chunks (of size
determined by a new data member max_atomic_write), as in
fhandler_base_overlapped.
For convenience, add two new NTSTATUS codes, STATUS_THREAD_SIGNALED
and STATUS_THREAD_CANCELED, to ntdll.h.
After copyto is called, make the new fhandler's pipe_name point to the
new fhandler's pipe_name_buf, which is a *copy* of the old fhandler's
pipe_name_buf. Previously, get_pipe_name would return the wrong
result after a clone/dup, causing create_pipe_instance and open_pipe
to fail.
Also, stop the listen_client thread when cloning. Otherwise the
thread can keep accepting connections that the cloned fhandler won't
know about.
Do this via a new method fhandler_fifo::stop_listen_client, extracted
from fhandler_fifo::close.
Add data members 'reader', 'writer', and 'duplexer' to the
fhandler_fifo class. Set them in fhandler_fifo::open. ('duplexer'
replaces the previous '_duplexer'.)
This will be useful in later commits.
The word "client" suggests something that holds a handle to the client
side of the pipe (in Windows terminology). But our
fifo_client_handlers hold a handle the server side of the pipe, and
they *connect* to clients.
- Add 24 bit color support using xterm compatibility mode in
Windows 10 1703 or later.
- Add fake 24 bit color support for legacy console, which uses
the nearest color from 16 system colors.
Make fhandler_base::clear_readahead virtual, and implement
fhandler_fifo::clear_readahead. This is called by
dtable::fixup_after_exec; it clears the readahead in each client.
If a FIFO is opened with O_RDWR access, create the pipe with
read/write access, and make the first client have the handle of that
pipe as its I/O handle.
Adjust fhandler_fifo::raw_read to account for the result of trying to
read from that client if there's no data.
fhandler_fifo::clone called fhandler_base::clone on each client
fhandler. But those fhandlers are actually fhandler_fifo objects, so
when fhandler_base::clone calls copyto, it's actually
fhandler_fifo::copyto that gets called. This can lead to mysterious
crashes.
Fix this by simply calling clone (which translates to
fhandler_fifo::clone) on each client fhandler.
Add static functions peek_fifo, thread_fifo, start_thread_fifo, and
fifo_cleanup to select.cc. These are based on the corresponding pipe
functions, the main difference being that peek_fifo loops through the
connected clients to see if any of them have data available for
reading.
Add the fhandler_fifo methods select_read, select_write, and
select_except.
Add accessor methods get_nclients, get_handle, and is_connected that
are needed by peek_fifo.
Add a hit_eof method that tries to detect whether any clients are
connected. Before concluding that there are none, it gives the
listen_client thread time to update the client data.
Introduce a 'fifo_client_handler' structure that can be used by a
reader to communicate with a writer using an instance of the named
pipe. An fhandler_fifo opened for reading creates a thread that does
the following:
- maintains a list of fifo_client_handlers
- listens for_clients trying to connect
- creates new pipe instances as needed so that there's always at
least one available for connecting.
The pipe instances are initially created in blocking mode, but they
are set to be non-blocking after a connection is made.
fhandler_fifo::raw_read now loops through the connected clients and
reads from the first one that has data available.
New fhandler_fifo methods: add_client, listen_client,
listen_client_thread, check_listen_client_thread.
Replace the create_pipe method by create_pipe_instance, which allows
unlimited pipe instances.
New helper functions: create_event, set_pipe_non_blocking.
Make fhandler_fifo a derived class of fhandler_base instead of
fhandler_base_overlapped.
Replace the create_pipe macro, which is based on
fhandler_pipe::create, by new create_pipe and open_pipe methods.
These use NT functions instead of Win32 functions. Replace fifo_name
by get_pipe_name, which returns a pointer to a UNICODE_STRING.
Remove the fnevent macro, which would now be needed only once.
Add a raw_write method, adapted from fhandler_base::raw_write.
Adapt all functions to the changes above.
timerfd_tracker and timerfd_shared classes:
- Just because handles are shared, we don't have to store them in
shared memory. Move share handles into timerfd_tracker class.
- Drop shared instance counter since it's not required anymore.
timerfd_shared only stores the actual timer data.
- Drop timerfd_shared::create, just set clock id.
- Drop timerfd_shared::dtor, it's not required anymore.
- Drop timerfd_tracker::close, just call dtor where required.
- Rename timerfd_tracker::increment_instances to timerfd_tracker::dup.
It's the only reason it exists...
- timerfd_tracker::dtor now checks the non-shared pointers for NULL
before attempting to close them.
- timerfd_tracker::dtor handles decrementing the local instance count
by itself.
- Add a method timerfd_tracker::init_fixup_after_fork_exec to set
non-shared pointers to NULL. Together with the dtor patches it
fixes a problem with close_on_exec timerfd descriptors.
- Fix a bug in handling the thread synchronization event. It's
actually nice to create it before using it...
- Drop using sec_none{_nih} in InitializeObjectAttributes. It's
an unnecessary roundabout route just to get a NULL pointer.
- Slightly rework timechange window handling.
- Add more comments to explain what happens.
fhandler_timerfd:
- Drop cnew macro, it just hides what happens.
- fhandler_timerfd::fixup_after_exec now calls
timerfd_tracker::init_fixup_after_fork_exec first, so a subsequent
call to timerfd_tracker::dtor only works on valid handles.
- fhandler_timerfd::close directly calls timerfd_tracker::dtor now.
- Drop dtor call in fhandler_timerfd destructor.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
The reopen code neglected to pass along the requested open
mode correctly. This may end up reopening the file with
incorrect access mask, or duplicating the wrong pipe handle.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Using posix timers "timer_tracker" as base class for timerfd was flawed.
Posix timers are not inherited by child processes and don't survive
execve. The method used by posix timers didn't allow to share timers
between processes. The timers were still per-process timers and worked
entirely separate from each other. Reading from these timers via
different descriptors was only synchronized within the same process.
This does not reflect the timerfd semantics in Linux: The per-file
timers can be dup'ed and survive fork and execve. They are still just
descriptors pointing to the same timer object originally created by
timerfd_create. Synchronization is performed between all descriptor
instances of the same timer, system-wide.
Thus, reimplement timerfd using a timer instance in shared memory,
a kernel timer, and a handful of sync objects.
Every process maintains a per-process timerfd struct on the cygheap
maintaining a per-process thread. Every process sharing the same
timerfd will run this thread checking the state of the timer, similar
to the posix timer thread, just working on the shared objects and
synchronizing its job with each other thread.
Drop the timerfd implementation in the posix timer code and move the
public API to fhandler_timerfd.c. Keep the ttstart timer_tracker
anchor out of "NO_COPY" since the fixup_after_fork code should run to
avoid memory leakage.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Linux returns EINVAL, "fd is attached to an object which is unsuitable
for writing". If we don't handle write locally, write returns EBADF.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
- Puzzeling: Commit ec98d19a08
changed ttstart to NO_COPY but kept all the code to handle
fixup after fork. Revert to not-NO_COPY and make timerfd
fork work.
- On fixup_after_fork, keep timerfd timers and restart thread
if they were armed in the parent.
- Move timerfd timer_trackers to cygheap. Overload timer_tracker
new and delete methods to handle timers accordingly. This is not
exactly required for fork, but exec will be grateful.
- Give up on TFD_TIMER_CANCEL_ON_SET for now. There's no easy way
to recognize a discontinuous change in a clock.
- Be paranoid when cleaning out ttstart.
- Fix some minor issues.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
First cut of a timerfd implementation.
Still TODO:
- fork/exec semantics
- timerfd_settime TFD_TIMER_CANCEL_ON_SET flag
- ioctl(TFD_IOC_SET_TICKS)
- bug fixes
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Allow the signal thread to recognize we're called in consequence of
select on a signalfd. If the signal is part of the wait mask, don't
call any signal handler and don't remove the signal from the queue,
so a subsequent read (or sigwaitinfo/sigtimedwait) still gets the
signal. Instead, just signal the event object at
_cygtls::signalfd_select_wait for the thread running select.
The addition of signalfd_select_wait to _cygtls unearthed the alignment
problem of the context member again. To make sure this doesn't get lost,
improve the related comment in the header file so that this (hopefully)
doesn't get lost (again).
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
First cut of a signalfd implementation.
Still TODO: Non-polling select.
This should mostly work as on Linux except for missing support
for some members of struct signalfd_siginfo, namely ssi_fd,
ssi_band (both SIGIO/SIGPOLL, not fully implemented) and ssi_trapno
(HW exception, required HW support).
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
The new proc fd code accidentally allowed to linkat an O_TMPFILE
even if the file has been opened with O_EXCL. This patch fixes it.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
move special fd symlink code into own fhandler_process_fd class
to simplify further additions to /proc/PID/fd/DESCRIPTOR symlink
handling.
Add a method to handle stat(2) on such a proc fd symlink by handle.
This allows correct reply from stat(2) if the target file has been
deleted. This eventually fixes `awk -f /dev/fd/3 3<<eof'.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Along the same lines as the previous patch: By reopening an
O_TMPFILE by handle, we can now move the file to the bin at
open time and thus free'ing up the parent dir and *still*
open the file as /proc/PID/fd/DESCRIPTOR by linkat(2).
Allows expressions along the lines of `cat /proc/self/fd/0 <<EOF'.
The problem here is that the temporary file used for the here script
has already been deleted by the shell. Opening by filename, as
implemented so far, doesn't work because the file has been moved
to the bin.
Allow reopening files by handle the same way from another process
as long as we have sufficient permissions on the foreign process.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
The fhandler_base_overlapped::copyto clears atomic_write_buf on the
clone, but none of the derived classes were doing this. This allowed
the destructor to double-free the buffer and corrupt cygheap.
Clear atomic_write_buf in copyto of all derived classes.
This code is where the AIO implementation is wired into existing Cygwin
mechanisms for file and device I/O: the fhandler* functions. It makes
use of an existing internal routine prw_open to supply a "shadow fd"
that permits asynchronous operations on a file the user app accesses
via its own fd. This allows AIO to read or write at arbitrary locations
within a file without disturbing the app's file pointer. (This was
already the case with normal pread|pwrite; we're just adding "async"
to the mix.)
* fhandler.h (class fhandler_socket_inet): Add variable bool oobinline.
* fhandler_socket_inet.cc (fhandler_socket_inet::fhandler_socket_inet):
Initialize variable oobinline.
(fhandler_socket_inet::recv_internal): Make the handling of OOB data
as consistent with POSIX as possible. Add simulation of inline mode
for OOB data as a workaround for broken winsock behavior.
(fhandler_socket_inet::setsockopt): Ditto.
(fhandler_socket_inet::getsockopt): Ditto.
(fhandler_socket_wsock::ioctl): Fix return value of SIOCATMARK command.
The return value of SIOCATMARK of winsock is almost opposite to
expectation.
* fhandler_socket_local.cc (fhandler_socket_local::recv_internal):
Remove the handling of OOB data from AF_LOCAL domain socket. Operation
related to OOB data will result in an error like Linux does.
(fhandler_socket_local::sendto): Ditto.
(fhandler_socket_local::sendmsg): Ditto.
This fixes the issue reported in following post.
https://cygwin.com/ml/cygwin/2018-06/msg00143.html
* Change set_socket_type/get_socket_type to virtual methods
* Move various variables into af_unix_shmem_t
* Change sun_name_t to match new usage pattern
* Move shut_state definition and add a name for the 0 value
* Allow marking packet as administrative packet. This allows
filtering out info packets exchange between peers and tweak
data accordingly.
* Rename send_my_name to send_sock_info and send credentials
if not called from bind (so the socket was already connected)
* Handle SO_PASSCRED in setsockopt/getsockopt
* Add input size checking to setsockopt/getsockopt
* Use NT functions where appropriate
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
We need to share socket info between threads *and* processes.
SRWLOCKs are single-process only, unfortunately. Provide a
sharable low-profile spinlock instead.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
E.g. arm ABI requires -fshort-enums for bare-metal toolchains.
Given there are only 29 category enums, the compiler chooses an
8 bit enum type, so a size of 11 bits for the bitfield leads to
a compile time error:
error: width of 'cat' exceeds its type
enum category cat: 11;
^~~
Fix this by aligning the size of the category members to byte
borders.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Per-socket info in fhandler isn't correctly shared between multiple
instances of th same descriptor. Implement a basic shared info which
is shared between all instances of a socket.
This also requires to move the fhandler_socket status bits into
fhandler_socket_wsock since the data is moved to the shared region
for AF_UNIX sockets.
Also, drop backing file requirement for socketpair server socket.
This will be handled differently in recvmsg/sendmsg.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Using TerminateThread potentially leaks resources. In our case,
the connect wait thread may be forcefully terminated after
having successfully opened a client side pipe handle. If this
occurs, we have a stale pipe server instance, so the pipe will
never be closed as long as the process lives.
Avoid this by changing the npfs handle to non-blocking, so we can
wait on a termination event object from inside the thread itself
and cleanly exit from the thread instead of terminating.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
The handle to the device is never needed. As the name impies,
FSCTL_PIPE_WAIT works on the file system, not on the device level.
Drop opening the device and make sure to open only one handle to NPFS.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
They are only used there anyway and it allows to use the AF_UNIX
macro without tweaking header files. While at it, improve
both constructors. The default constructor now creates the name
of an unnamed socket, the constructor taking parameters carefully
checks its input.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
...plus lots of helper functions. Add comment to explain how everything
works. This comment will be improved while implementing the yet missing
parts.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Add constructors and new/delete operators to make sure sun_name_t
objects are allocated on the cygheap.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
fhandler_cygdrive has a size of 696 bytes on x86_64, while the next
biggest fhandler type, fhandler_pty_master, is 584 bytes. The members
responsible for the size are private to opendir/readdir/closedir usage.
fhandler_disk_file stores private readdir data in DIR->__d_internal
instead. Use equivalent method with fhandler_cygdrive. This drops
the size to 464 bytes.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
* Make distinct from AF_LOCAL for testing purposes. This will have
to be reverted as soon as fhandler_socket_unix goes life.
* Move saw_reuseaddr flag back to fhandler_socket status
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Insert another class fhandler_socket_wsock between fhandler_socket
and fhandler_socket_inet/fhandler_socket_local.
Also, add a new method fhandler::is_wsock_socket to allow asking
for sockets in general (is_socket) vs. Winsock-based sockets
(is_wsock_socket).
This allows to develop a new handler_socket_unix class as derived
class from fhandler_socket without any trace of wsock code left
in fhandler_socket.
While this is basically a temporary measure at this time, it may
prove useful for later interoperability with the upcoming Windows 10
AF_UNIX implementation at one point.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
First cut, still incomplete
* fhandler_socket is now base class for other socket classes
* fhandler_socket_inet handles AF_INET and AF_INET6 sockets
* fhandler_socket_local handles AF_LOCAL/AF_UNIX sockets
* finally get rid of fdsock by using set_socket_handle in accept4
* align file-related calls (fstat, fstatvfs, fchown, fchmod, facl)
to Linux.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Add fhandler_socket::socket method
Add fhandler_socket::set_socket_handle method, basically duplicating
what fdsock is doing. This is the first step in getting rid of fdsock.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Windows does not remove FILE_ATTRIBUTE_TEMPORARY by itself after a
file has been closed. It's just some attribute which can be set or
removed at will, despite its purpose.
Apparently there are tools out there which use FILE_ATTRIBUTE_TEMPORARY
accidentally or wrongly, even Microsoft's own tools are affected. In
the end, the filesystem is potentially full of files with this attribute
set.
Implement O_TMPFILE files with FILE_ATTRIBUTE_TEMPORARY and
FILE_ATTRIBUTE_HIDDEN set. This combination is pretty unlikely.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Takashi Yano
Corinna Vinschen
Ken Brown
David McFarland
Mark Geisert