Analyzing the fhandler::copyto logic shows that the fhandler_base::reset
method was only called from copyto anyway.
Trying to convert reset to a protected method uncovered that the copyto
method is actually thought upside down from an object oriented POV.
Rather than calling copyto, manipulating the object given as parameter,
rename the method to copy_from, which manipulates the calling object
itself with data from the object given as parameter.
Eventually make reset a protected method and rename it to
_copy_from_reset_helper to clarify it's only called from copy_from.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
- After commit bb428520, there has been the disadvantage:
7) Pseudo console cannot be activated if it is already activated for
another process on same pty.
This patch clears this disadvantage.
- After commit bb428520, there has been the disadvantage:
2) The apps which use console API cannot be debugged with gdb. This
is because pseudo console is not activated since gdb uses
CreateProcess() rather than exec(). Even with this limitation,
attaching gdb to native app, in which pseudo console is already
activated, works.
This patch clears this disadvantage.
- PTY has a problem that the key input, which is typed during windows
native app is running, disappears when it returns to shell. This is
beacuse pty has two input pipes, one is for cygwin apps and the other
one is for native windows apps. The key input during windows native
program is running is sent to the second input pipe while cygwin
shell reads input from the first input pipe. This issue had been
fixed once by commit 29431fcb, however, the new implementation of
pseudo console support by commit bb428520 could not inherit this
feature. This patch realize transfering input data between these
two pipes bidirectionally by utilizing cygwin-console-helper process.
The helper process is launched prior to starting the non-cygwin app,
however, exits immediately unlike previous implementation.
The isdev_dev check in rmdir is unclean. Create a virtual method
fhandler_dev::rmdir to handle this transparently.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
- The functions pty_master_thread() and pty_master_fwd_thread()
should be static (i.e. should not access class member) because
the instance is deleted if the master is dup()'ed and the first
master is closed. In this case, because the dup()'ed instance
still exists, these master threads are also still alive even
though the instance has been deleted. As a result, accesing
class members in these functions causes accessi violation.
Addresses:
https://cygwin.com/pipermail/cygwin-developers/2021-January/012030.html
- After commit 232fde0e, pty changes console code page when the first
non-cygwin app is executed. If pty is started in real console device,
pty changes the code page of root console. This causes very annoying
result because changing code page changes the font of command prompt
if console is in legacy mode. This patch avoids this by creating a
new invisible console for the first pty started in console device.
- The function close_pseudoconsole() should be static so that it
can be safely called in spawn.cc even after the fhandler_pty_slave
instance has been deleted. That is, there is a problem with the
current code. This patch fixes the issue.
- If application changes the console mode, mode management introduced
by commit 10d8c278 will be corrupted. For example, stdout of jansi
v2.0.1 or later is piped to less, jansi resets the xterm mode flag
ENABLE_VIRTUAL_TERMINA_PROCESSING when jansi is terminated. This
causes garbled output in less because less needs this flag enabled.
This patch fixes the issue.
Replace the 'WCHAR pipe_name_buf[48]' class member by 'PWCHAR
pipe_name_buf', and allocate space for the latter as needed.
Change the default constructor to accommodate this change, and add a
destructor that frees the allocated space.
Also change get_pipe_name and clone to accommodate this change.
Both flags are outdated and collide with official flags in
sys/_default_fcntl.h, which may result in weird misbehaviour
of file functions.
O_NOSYMLINK is not used anyway.
O_DIROPEN is used in fhandler_virtual and derived classes.
The collision with O_NOFOLLOW results in spurious EISDIR
errors when, e. g., reading files in the registry.
fhandler_base::open_fs uses O_DIROPEN in the call to
fhandler_base::open, but it's not used in this context
further down the road.
Drop both flags and create an alternative "diropen" bool
flag in fhandler_virtual.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
- Pseudo console internally sends escape sequence CSI6n (query cursor
position) on startup of non-cygwin apps. If the terminal does not
support CSI6n, CreateProcess() hangs waiting for response. To prevent
hang, this patch disables pseudo console if the terminal does not
have CSI6n. This is checked on the first execution of non-cygwin
app using the following steps.
1) Check if the terminal support ANSI escape sequences by looking
into terminfo database. If terminfo has cursor_home (ESC [H),
the terminal is supposed to support ANSI escape sequences.
2) If the terminal supports ANSI escape sequneces, send CSI6n for
a test and wait for a responce for 40ms.
3) If there is a responce within 40ms, CSI6n is supposed to be
supported.
Also set-title capability is checked, and removes escape sequence
for setting window title if the terminal does not have the set-
title capability.
- In this implementation, pseudo console is created for each native
console app. Advantages and disadvantages of this implementation
over the previous implementation are as follows.
Advantages:
1) No performance degradation in pty output for cygwin process.
https://cygwin.com/pipermail/cygwin/2020-February/243858.html
2) Free from the problem caused by difference of behaviour of control
sequences between real terminal and pseudo console.
https://cygwin.com/pipermail/cygwin/2019-December/243281.htmlhttps://cygwin.com/pipermail/cygwin/2020-February/243855.html
3) Free from the problem in cgdb and emacs gud.
https://cygwin.com/pipermail/cygwin/2020-January/243601.htmlhttps://cygwin.com/pipermail/cygwin/2020-March/244146.html
4) Redrawing screen on executing native console apps is not necessary.
5) cygwin-console-helper is not necessary for the pseudo console
support.
6) The codes for pseudo console support are much simpler than that
of the previous one.
Disadvantages:
1) The cygwin program which calls console API directly does not work.
2) The apps which use console API cannot be debugged with gdb. This
is because pseudo console is not activated since gdb uses
CreateProcess() rather than exec(). Even with this limitation,
attaching gdb to native apps, in which pseudo console is already
activated, works.
3) Typeahead key inputs are discarded while native console app is
executed. Simirally, typeahead key inputs while cygwin app is
executed are not inherited to native console app.
4) Code page cannot be changed by chcp.com. Acctually, chcp works
itself and changes code page of its own pseudo console. However,
since pseudo console is recreated for another process, it cannot
inherit the code page.
5) system_printf() does not work after stderr is closed. (Same with
cygwin 3.0.7)
6) Startup time of native console apps is about 3 times slower than
previous implemenation.
7) Pseudo console cannot be activated if it is already activated for
another process on same pty.
The fifo_reader thread function and the function select.cc:peek_fifo()
can both change the state of a fifo_client_handler. These changes are
made under fifo_client_lock, so there is no race, but the changes can
still be incompatible.
Add code to make sure that only one of these functions can change the
state from its initial fc_listening state. Whichever function does
this calls the fhandler_fifo::record_connection method, which is now
public so that peek_fifo can call it.
Slightly modify that method to make it suitable for being called by
peek_fifo.
Make a few other small changes to the fifo_reader thread function to
change how it deals with the STATUS_PIPE_CLOSING value that can
(rarely) be returned by NtFsControlFile.
Add commentary to fhandler_fifo.cc to explain fifo_client connect
states and where they can be changed.
Don't try to read from fifo_client_handlers that are in the fc_closing
state. Experiments have shown that this always yields
STATUS_PIPE_BROKEN, so it just wastes a Windows system call.
Re-order the values in enum fifo_client_connect_state to reflect the
new status of fc_closing.
Rename the existing set_state() to query_and_set_state() to reflect
what it really does. (It queries the O/S for the pipe state.) Add a
new set_state() method, which is a standard setter, and a
corresponding getter get_state().
fhandler_fifo::take_ownership() is called from select.cc::peek_fifo
and fhandler_fifo::raw_read and could potentially block indefinitely
if something goes wrong. This is always undesirable in peek_fifo, and
it is undesirable in a nonblocking read. Fix this by adding a timeout
parameter to take_ownership.
Arbitrarily use a 1 ms timeout in peek_fifo and a 10 ms timeout in
raw_read. These numbers may have to be tweaked based on experience.
Replace the call to cygwait in take_ownership by a call to WFSO.
There's no need to allow interruption now that we have a timeout.
Use cygwait in take_ownership to allow interruption while waiting to
become owner. Return the cygwait return value or a suitable value to
indicate an error.
raw_read now checks the return value and acts accordingly.
Add a bool member 'last_read' to the fifo_client_handler structure,
which is set to true on a successful read. This is used by raw_read
as follows.
When raw_read is called, it first locates the writer (if any) for
which last_read is true. raw_read tries to read from that writer and
returns if there is input available. Otherwise, it proceeds to poll
all the writers, as before.
The effect of this is that if a writer writes some data that is only
partially read, the next attempt to read will continue to read from
the same writer. This should reduce the interleaving of output from
different writers.
When a reader opens, it needs to block if there are no writers open
(unless is is opened with O_NONBLOCK). This is easy for the first
reader to test, since it can just wait for a writer to signal that it
is open (via the write_ready event). But when a second reader wants
to open, all writers might have closed.
To check this, use a new '_nwriters' member of struct fifo_shmem_t,
which keeps track of the number of open writers. This should be more
reliable than the previous method.
Add nwriters_lock to control access to shmem->_nwriters, and remove
reader_opening_lock, which is no longer needed.
Previously only readers had access to the shared memory, but now
writers access it too so that they can increment _nwriters during
open/dup/fork/exec and decrement it during close.
Add an optional 'only_open' argument to create_shmem for use by
writers, which only open the shared memory rather than first trying to
create it. Since writers don't need to access the shared memory until
they have successfully connected to a pipe instance, they can safely
assume that a reader has already created the shared memory.
For debugging purposes, change create_shmem to return 1 instead of 0
when a reader successfully opens the shared memory after finding that
it had already been created.
Remove check_write_ready_evt, write_ready_ok_evt, and
check_write_ready(), which are no longer needed.
When opening a writer and looping to try to get a connection, recheck
read_ready at the top of the loop since the number of readers might
have changed.
To slightly speed up the process of opening the first reader, take
ownership immediately rather than waiting for the fifo_reader_thread
to handle it.
When the owning reader closes and there are still readers open, the
owner needs to wait for a new owner to be found before closing its
fifo_client handlers. This involves a loop in which dec_nreaders is
called at the beginning and inc_nreaders is called at the end. Any
other reader that tries to access shmem->_nreaders during this loop
will therefore get an inaccurate answer.
Fix this by adding an nreaders method and using it instead of
dec_nreaders and inc_nreaders. Also add nreaders_lock to control
access to the shmem->_nreaders.
Make various other changes to improve the reliability of finding a new
owner.
Use WSAIoctl(SIO_KEEPALIVE_VALS) on older systems.
Make sure that keep-alive timeout is equivalent to
TCP_KEEPIDLE + TCP_KEEPCNT * TCP_KEEPINTVL on older systems,
even with TCP_KEEPCNT being a fixed value on those systems.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
- Commit c4b060e3fe seems to be not
enough. Moreover, it does not work as expected at all in Win10
1809. This patch essentially reverts that commit and add another
fix. After all, the cause of the problem was a race issue in
switch_to_pcon_out flag. That is, this flag is set when native
app starts, however, it is delayed by wait_pcon_fwd(). Since the
flag is not set yet when less starts, the data which should go
into the output_handle accidentally goes into output_handle_cyg.
This patch fixes the problem more essentially for the cause of
the problem than previous one.
- After commit 29431fcb5b, the issue
reported in https://cygwin.com/pipermail/cygwin/2020-May/245057.html
occurs. This is caused by the following mechanism. Cygwin less
called from non-cygwin git is executed under /dev/cons* rather
than /dev/pty* because parent git process only inherits pseudo
console handle. Therefore, less sets ICANON flag for /dev/cons*
rather than original /dev/pty*. When pty is switched to non-cygwin
git process, line_edit() is used in fhandler_pty_master::write()
only to set input_available_event and read ahead buffer is supposed
to be flushed in accept_input(). However, ICANON flag is not set
for /dev/pty*, so accept_input() is not called unless newline
is entered. As a result, the input data remains in the read ahead
buffer. This patch fixes the issue.
This reverts commit 39a9cd9465.
There is no need to explicitly take ownership in fixup_after_exec; if
ownership transfer is needed, it will be taken care of by
fhandler_fifo::close when the parent closes. Moreover, closing the
parent's fifo_reader_thread can cause problems, such as the one
reported here:
https://cygwin.com/pipermail/cygwin-patches/2020q2/010235.html
- After commit 071b8e0cbd, the problem
reported in https://cygwin.com/pipermail/cygwin/2020-May/244873.html
occurs. This is due to freeing console device accidentally rather
than pseudo console. This patch makes sure to call FreeConsole()
only if the process is attached to the pseudo console of the current
pty.
There are currently three functions that call NtQueryInformationFile
to determine the state of a pipe instance. Do this only once, in a
new fifo_client_handler::set_state () function, and call that when
state information is needed.
Remove the fifo_client_handler methods pipe_state and get_state, which
are no longer needed.
Make fhandler_fifo::get_fc_handler return a reference, for use in
select.cc:peek_fifo.
Make other small changes to ensure that this commit doesn't change any
decisions based on the state of a fifo_client_handler.
The tricky part is interpreting FILE_PIPE_CLOSING_STATE, which we
translate to fc_closing. Our current interpretation, which is not
changing as a result of this commit, is that the writer at the other
end of the pipe instance is viewed as still connected from the point
of view of raw_read and determining EOF.
But it is not viewed as still connected if we are deciding whether to
unblock a new reader that is trying to open.
fifo_reader_id_t::operator == and != have been defined without type
accidentally. For some weird reason, only x86 gcc complains about
this problem, not x86_64 gcc.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Although we can have multiple readers open because of dup/fork/exec,
the current code does not support multiple readers opening a FIFO by
explicitly calling 'open'.
The main complication in supporting this is that when a blocking
reader tries to open and there's already one open, it has to check
whether there any writers open. It can't rely on the write_ready
event, whose state hasn't changed since the first writer opened.
To fix this, add two new named events, check_write_ready_evt and
write_ready_ok_evt, and a new method, check_write_ready().
The first event signals the owner's reader thread to call
check_write_ready(), which polls the fc_handler list to check for
connected writers. If it finds none, it checks to see if there's a
writer in the process and then sets/resets write_ready appropriately.
When check_write_ready() finishes it sets write_ready_ok_evt to signal
the reader that write_ready has been updated.
The polling is done via fifo_client_handler::pipe_state(). As long as
it's calling that function anyway, check_write_ready() updates the
state of each handler.
Also add a new lock to prevent a race if two readers are trying to
open simultaneously.
Add a take_ownership method, used by raw_read and select.cc:peek_fifo.
It wakes up all fifo_reader_threads and allows the caller to become
owner. The work is done by the fifo_reader_threads.
For synchronization we introduce several new fhandler_fifo data
members and methods:
- update_needed_evt signals the current owner to stop listening for
writer connections and update its fc_handler list.
- shared_fc_handler() gets and sets the status of the fc_handler
update process.
- get_pending_owner() and set_pending_owner() get and set the reader
that is requesting ownership.
Finally, a new 'reading_lock' prevents two readers from trying to take
ownership simultaneously.
If the owning reader is closing, wait for another reader (if there is
one) to take ownership before closing the owner's pipe handles.
To synchronize the ownership transfer, add events owner_needed_evt and
owner_found_evt, and add methods owner_needed and owner_found to
set/reset them.
Modify the fifo_reader_thread function to wake up all non-owners when
a new owner is needed.
Make a cosmetic change in close so that fhandler_base::close is called
only if we have a write handle. This prevents strace output from
being littered with statements that the null handle is being closed.
If fixup_after_exec is called on a non-close-on-exec reader whose
parent is the owner, transfer ownership to the child. Otherwise the
parent's pipe handles will be closed before any other reader can
duplicate them.
To help with this, make the cancel_evt and thr_sync_evt handles
inheritable, so that the child can terminate the parent's
fifo_reader_thread (and the parent will update the shared fc_handler
list).
Add an optional argument 'from_exec' to update_my_handlers to simplify
its use in this case; no handle duplication is required.
This is in a new shared memory section. We will use it for temporary
storage of the owner's fc_handler list when we need to change owner.
The new owner can then duplicate the pipe handles from that list
before taking ownership.
Add several shared data members and methods that are needed for the
duplication process
Add methods update_my_handlers and update_shared_handlers that carry
out the duplication.
Allow the shared list to grow dynamically, up to a point. Do this by
initially reserving a block of memory (currently 100 pages) and only
committing pages as needed.
Add methods create_shared_fc_handler, reopen_shared_fc_handler, and
remap_shared_fc_handler to create the new shared memory section,
reopen it, and commit new pages. The first is called in open, the
second is called in dup/fork/exec, and the third is called in
update_shared_handlers if more shared memory is needed.
Modify the fifo_reader_thread function to call update_my_handlers when
it finds that there is no owner. Also make it call
update_shared_handlers when the owner's thread terminates, so that the
new owner will have an accurate shared fc_handler list from which to
duplicate.
For convenience, add new methods cleanup_handlers and
close_all_handlers. And add an optional arg to add_client_handler
that allows it to create a new fifo_client_handler without creating a
new pipe instance.
Make fc_handler a pointer to malloc'd memory instead of a fixed-size
array. The size is now a new data member 'shandlers'. Call realloc
in add_client_handler if we need to grow the array.
free fc_handler in close. As long as we're touching that code, also
remove an unneeded lock.
Among all the open readers of a FIFO, one is declared to be the owner.
This is the only reader that listens for client connections, and it is
the only one that has an accurate fc_handler list.
Add shared data and methods for getting and setting the owner, as well
as a lock to prevent more than one reader from accessing these data
simultaneously.
Modify the fifo_reader_thread so that it checks the owner at the
beginning of its loop. If there is no owner, it takes ownership. If
there is an owner but it is a different reader, the thread just waits
to be canceled. Otherwise, it listens for client connections as
before.
Remove the 'first' argument from create_pipe_instance. It is not
needed, and it may be confusing in the future since only the owner
knows whether a pipe instance is the first.
When opening a reader, don't return until the fifo_reader_thread has
time to set an owner.
If the owner closes, indicate that there is no longer an owner.
Clear the child's fc_handler list in dup, and don't bother duplicating
the handles. The child never starts out as owner, so it can't use
those handles.
Do the same thing in fixup_after_fork in the close-on-exec case. In
the non-close-on-exec case, the child inherits an fc_handler list that
it can't use, but we can just leave it alone; the handles will be
closed when the child is closed.
This uniquely identifies an fhandler_fifo open for reading in any
process.
Add a new data member 'me' of this type, which is set in open, dup,
fork, and exec.
Add data and methods to the shared memory that keep track of the
number of open readers.
Increment this number in open, dup, fork, and exec. Decrement it in
close. Reset read_ready if there are no readers left.
Even though we currently allow a FIFO to be opened for reading only
once, we can still have more than one reader open because of dup and
fork. Add a named shared memory section accessible to all readers of
a given FIFO. In future commits we will add information needed by all
readers to this section
Add a class fifo_shmem_t that lets us access this information.
Add a method create_shmem that is called when a reader opens, and add
a method reopen_shmem that is called by dup, fork, and exec. (Each
new reader needs its own view of the shared memory.)
This will simplify future work.
Rename the thread from "listen_client_thread" to "fifo_reader_thread"
because it will be used for more than just listening.
Remove the fixup_before stuff, which won't be needed after future
changes to fixup_after_fork and fixup_after_exec.
- Make read_ready a manual-reset event.
- Signal read_ready in open instead of in the listen_client_thread.
- Don't reset read_ready when the listen_client thread terminates;
instead do it in close().
- Rearrange open and change its error handling.
- Add a wait_open_pipe method that waits for a pipe instance to be
available and then calls open_pipe. Use it when opening a writer if
we can't connect immediately. This can happen if the system is
heavily loaded and/or if many writers are trying to open
simultaneously.
According to Posix, a FIFO open for reading is at EOF if it is empty
and there are no writers open.
The only way to test this is to poll the fifo_client_handlers as in
raw_read and select.cc:peek_fifo. The current hit_eof instead relies
on the value of nconnected, which can be out of date. On the one
hand, it doesn't take into account writers that were connected but
have since closed. On the other hand, it doesn't take into account
writers that are in the process of opening but haven't yet connected.
Fix this by introducing a maybe_eof method that tentatively assumes
EOF if there are no connected writers after polling. Then check for
writers currently opening (via a new 'writer_opening' event), and wait
for the fifo_reader_thread to record any new connection that was made
while we were polling.
To handle the needs of peek_fifo, replace the get_fc_handle method
by a get_fc_handler method, and add a fifo_client_handler::get_state
method.
Remove the is_connected method, which was used only in peek_fifo and
is no longer needed.
Remove the nconnected data member, which was used only for the flawed
hit_eof.
Add some comments about events to fhandler.h.
Always return 0; no one is doing anything with the return value
anyway.
Remove the return value from stop_listen_client.
Make the connection event auto-reset, so that we don't have to reset
it later.
Simplify the process of connecting a bogus client when thread
termination is signaled.
Make some failures fatal.
Remove the unnecessary extra check for thread termination near the end
of listen_client_thread.
Make the values correspond to the possible return values of
fifo_client_handler::pipe_state().
When cleaning up the fc_handler list in listen_client_thread(), don't
delete handlers in the fc_closing state. I think the pipe might still
have input to be read in that case.
Set the state to fc_closing later in the same function if a connection
is made and the status returned by NtFsControlFile is
STATUS_PIPE_CLOSING.
In raw_read, don't error out if NtReadFile returns an unexpected
status; just set the state of that handler to fc_error. One writer in
a bad state doesn't justify giving up on reading.
Corinna Vinschen
Takashi Yano via Cygwin-patches
Ken Brown
Takashi Yano