Per https://cygwin.com/pipermail/cygwin-developers/2021-October/012429.html,
we may encounter a crash when starting multiple threads during process
startup (here: fhandler_fifo::fixup_after_{fork,exec}) which in turn
allocate memory via malloc.
The problem is concurrent usage of malloc before the malloc muto has
been initialized.
To fix this issue, convert the muto to a SRWLOCK and make sure it is
statically initalized. Thus, malloc can be called as early as necessary
and malloc_init is only required to check for user space provided malloc.
Note that this requires to implement a __malloc_trylock macro to be
called from fork.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
This patch unifies the layout of the clipboard descriptor cygcb_t for
32- and 64-bit Cygwin. It allows correct copy/paste between the two
environments without corruption of user's copied data and without access
violations due to interpreting that data as a size field.
The definitions of CYGWIN_NATIVE and cygcb_t are moved to a new include
file, sys/clipboard.h. The include file is used by fhandler_clipboard.cc
as well as getclip.c and putclip.c in the Cygwin cygutils package.
When copy/pasting between 32- and 64-bit Cygwin environments, both must
be running version 3.3.0 or later for successful operation.
- The commit b531d6b0 introduced temporary_query_hdl() which uses
SystemHandleInformation. With this patch, ProcessHandleInformation
rather than SystemHandleInformation is used if it is available.
This request is faster, however, is only available since Windows 8,
therefore, SystemHandleInformation is used for Windows Vista and 7
as before.
- The commit f79a4611 introduced query_hdl, which is the read pipe
handle kept in the write pipe instance in order to determine if
the pipe is ready to write in select(). This implementation has
a potential risk that the write side fails to detect the closure
of the read side if more than one writer exists and one of them
is a non-cygwin process.
With this patch, the strategy of commit f79a4611 is used only if
the process is running as a service. For a normal process,
instead of keeping query_hdl in the write pipe instance, it is
retrieved temporarily when select() is called. Actually, we
want to use tenporary query_hdl for all processes, however, it
does not work for service processes due to OpenProcess()
failure.
open_setup is called by dtable::init_std_file_from_handle and
fhandler_base::open_with_arch. In both cases, failure of open_setup
is now a fatal error.
Currently this can only happen in the following situation: A Cygwin
process is started by a non-Cygwin process, one of the standard IO
handles is a pipe handle, and Cygwin is unable to create a required
mutex (read_mtx or hdl_cnt_mtx).
- query_hdl and hdl_cnt_mtx are moved from fhandler_pipe_fifo to
fhandler_pipe. Then reader_closed() is changed to virtual and
overridden in fhandler_pipe.
- This patch fixes the race issue in the handle counting to detect
closure of read pipe, which is introduced by commit f79a4611.
A mutex hdl_cnt_mtx is introduced for this issue.
- Currently, the stderr handle is duplicated in close_all_files().
This interferes the handle counting for detecting closure of read
pipe, which is introduced by commit f79a4611. This patch stops
duplicating stderr handle if it is write pipe.
NtQueryInformationFile hangs if it's called on the read side handle of
a pipe while another thread or process is performing a blocking read.
Avoid select potentially hanging by calling NtQueryInformationFile
only on the write side of the pipe and using PeekNamedPipe otherwise.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
- In pipe_data_available() in select.cc, PeekNamedPipe() call is
not needed if WriteQuotaAvailable is non-zero because we already
know the write pipe has a space. Therefore, with this patch,
PeekNamedPipe() is called only when WriteQuotaAvailable is zero.
This makes select() on pipe faster a bit.
- Currently, raw_read(), raw_write() and close() release select_sem
unconditionally even if no waiter for select_sem exists. With this
patch, only the minimum number of semaphores required is released.
- Usually WriteQuotaAvailable retrieved by NtQueryInformationFile()
on the write side reflects the space available in the inbound buffer
on the read side. However, if a pipe read is currently pending,
WriteQuotaAvailable on the write side is decremented by the number
of bytes the read side is requesting. So it's possible (even likely)
that WriteQuotaAvailable is 0, even if the inbound buffer on the
read side is not full. This can lead to a deadlock situation:
The reader is waiting for data, but select on the writer side
assumes that no space is available in the read side inbound buffer.
Currently, to avoid this stuation, read() does not request larger
block than pipe size - 1. However, this mechanism does not take
effect if the reader side is non-cygwin app.
The only reliable information is available on the read side, so
fetch info from the read side via the pipe-specific query handle
(query_hdl) introduced.
If the query_hdl (read handle) is kept in write side, writer can
not detect closure of read pipe. Therefore, raw_write() counts
write handle and query_hdl. If they are equal, only the pairs of
write handle and query_hdl are alive. In this case, raw_write()
returns EPIPE and raises SIGPIPE.
- Nonblocking pipes (PIPE_NOWAIT) are not well handled by non-Cygwin
tools, so convert pipe handles to PIPE_WAIT handles when spawning
a non-Cygwin process.
We already fetched the correct SECURITY_ATTRIBUTES at the start of
fhandler_pipe::create, so using another SECURITY_ATTRIBUTES object for
the mutex and semaphore objects doesn't make much sense.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
select_sem gets created on the read side with inheritence settings
depending on the O_CLOEXEC flag. Then it gets duplicated to the write
side with unconditional inheritence. Fix that.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Fold all code branches potentially having read or written data into
a single if branch, so signalling select_sem catches all cases.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Rename fhandler_pipe_and_fifo::max_atomic_write to pipe_buf_size.
This reflect its actual meaning better. The fhandler_pipe_and_fifo
constructor initializes it to DEFAULT_PIPEBUFSIZE (== 64K), which is
the buffer size for the windows pipes created by fhandler_pipe and
fhandler_fifo. But if we inherit a stdio pipe handle from a
non-Cygwin process, the buffer size could be different.
To remedy this, add a method fhandler_pipe::set_pipe_buf_size that
queries the OS for the pipe buffer size, and use it in
dtable::init_std_file_from_handle.
Given we return 1 already if WriteQuotaAvailable is > 0, the condition
for tiny pipes is never true. Fix the comments.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
In blocking mode, the underlying IO must always be terminated,
one way or the other, to make sure the application knows the exact
state after returning from the IO function. Therefore, always call
CancelIo in blocking mode.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Just cancelling a thread doesn't cancel async IO started by this thread.
Fix this by returning from cygwait and calling CancelIo before canceling
self.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
- By guarding read with read_mtx, no more than one ReadFile can
be called simultaneously. So couting read handles is no longer
necessary.
- Make raw_read code as similar as possible to raw_write code.
This is a parent of fhandler_pipe and fhandler_fifo for code that is
common between the two classes. Currently it just contains
max_atomic_write and raw_write(). The latter is identical to what
used to be fhandler_pipe::raw_write().
FILE_PIPE_LOCAL_INFORMATION::WriteQuotaAvailable is unreliable.
Usually WriteQuotaAvailable on the write side reflects the space
available in the inbound buffer on the read side. However, if a
pipe read is currently pending, WriteQuotaAvailable on the write side
is decremented by the number of bytes the read side is requesting.
So it's possible (even likely) that WriteQuotaAvailable is 0, even
if the inbound buffer on the read side is not full. This can lead to
a deadlock situation: The reader is waiting for data, but select
on the writer side assumes that no space is available in the read
side inbound buffer.
This patch implements a workaround by never trying to read more than
half the buffer size blocking if the read buffer is empty. This first
cut tries to take the number of open readers into account by reducing
the amount of requested bytes accordingly.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
POSIX requires atomicity for non-blocking writes <= PIPE_BUF bytes
and writing of at least 1 byte if any buffer space is left.
Windows NtWriteFile returns STATUS_SUCCESS and "0 bytes written"
if the write doesn't match buffer space. Fix this discrepancy.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
The buffer pointer is incremented by "chunk", which is what we
typically try to write, but this isn't what actually got written.
Increment the buffer pointer by what we actually wrote, as returned
by NtWriteFile.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
get_obj_handle_count() is used in flock only so far, but pipe
handling might have a usage, too, soon. Given that this function
might be generally useful and isn't restricted to flock usage,
move it to miscfuncs.cc and make it non-static. Add a prototype
in miscfuncs.h.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
I wasted valuable minutes of my life just to find out why we export
this weird version of pipe. In the pre-2000 era the idea was Cygwin
could be used as drop-in replacement for msvcrt.dll, apparently.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
The read handles of pipes created by CreateNamedPipe don't have
FILE_WRITE_ATTRIBUTES access unless the pipe is created with
PIPE_ACCESS_DUPLEX. This causes set_pipe_non_blocking to fail on such
handles. To fix this, add a helper function nt_create, which uses
NtCreateNamedPipeFile instead of CreateNamedPipe and gives us more
flexibility in setting access rights.
Use this helper function in fhandler_pipe::create (fhandler_pipe *[2],
unsigned, int), which is the version of fhandler_pipe::create used by
the pipe and pipe2 system calls.
For convenience, also add a static member function
fhandler_pipe::npfs_handle similar to those used by fhandler_fifo and
fhandler_socket_unix.
Add methods 'set_pipe_non_blocking' and 'fcntl' to keep the blocking
mode of the Windows pipe in sync with that of the fhandler_pipe
object. This applies to pipes created with the 'pipe' and 'pipe2'
system calls.
Previously fhandler_pipe was derived from fhandler_base_overlapped,
which we are going to remove in a future commit. Make minimal changes
so that the build still succeeds.
In https://cygwin.com/pipermail/cygwin/2021-September/249361.html
Brian pointed out that initializing the structure would be more
future proof, should the developers at Microsoft ever decide to
split the Reserved field and use some bits of the struct for
other purposes.
Fixes: 3d322ac930 ("Cygwin: fix initializing MEM_EXTENDED_PARAMETER")
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
MEM_EXTENDED_PARAMETER consists of a 64 bit bitfield which contains
the Type and the Reserved members. The former usage of designated
initializer lists initialized Type, but not Reserved. Since that's
not possible anymore due to a g++ 11.2 bug, Cygwin initializes the
MEM_EXTENDED_PARAMETER structs explicitely. This results in a
random value in Reserved, which at least VirtualAlloc2 chokes on
(ERROR_INVALID_PARAMETER).
Set Reserved explicitely to 0 for a fix.
Fixes: bdb7991db3 ("Cygwin: workaround a g++ 11.2 initialization bug")
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
The handle is created non-inheritable but gets inheritable when
dup'ing the file descriptor. Fix that.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Modern gcc's generate additional DWARF 5 debug sections, which were
still missing in our Cygwin loader script. With ld from binutils 2.37,
this results in diagnostic output when linking the Cygwin DLL...
ld: cygwin0.dll:/4: section below image base
ld: cygwin0.dll:/20: section below image base
ld: cygwin0.dll:/36: section below image base
...and the section addresses given to these sections (.debug_loclists,
.debug_rnglists, debug_line_str) will be wrong.
Fix this by adding the missing DWARF 5 sections to our linker script
template cygwin.sc.in. Add a comment in terms of the deprecated
DWARF 4 section .debug_types.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Signed-off-by: Jon Turney <jon.turney@dronecode.org.uk>
This was used before switching to automake to allow easy tweaking
of optimization and debugging settings from the command line during
testing. Reenable.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
trying to use aggregate initialization syntax on a member of a
nameless union member failes in g++ 11.2.
Workaround this by using explicit initialization.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
The register keyword was already deprecated with C++11, but
with C++17 it has been entirely removed.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Revert mx parameter and mutex lock while operating the list.
Mutex was removed with 94d24160 informing that:
'Use InterlockedCompareExchangePointer to ensure race safeness
without using a mutex.'
But it does not.
Calling pthread_mutex_init and pthread_mutex_destroy from two or
more threads occasionally leads to hang in pthread_mutex_destroy.
To not change the behaviour of other cases where List_insert was called,
List_insert_nolock is added.
As outlined in the previous patch, the non-atomicity of iterating
over a directory in the NT namespace via NtQueryDirectoryObject
one entry each, results in potential duplication of directory entries.
Fix this for fhandler_procsys::readdir as well by fetching the entire
dir inside fhandler_procsys::opendir, storing it in a buffer, and just
return buffer content from fhandler_procsys::readdir.
Fixes: 43f65cdd7d ("fhandler_procsys.cc: New file.")
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Due to reports on the Cygwin mailing list[1][2], it was uncovered
that a NtOpenDirectoryObject/NtQueryDirectoryObject/NtClose sequence
with NtQueryDirectoryObject iterating over the directory entries,
one entry per invocation, is not running atomically. If new entries
are inserted into the queried directory, other entries may be moved
around and then accidentally show up twice while iterating.
Change (almost) all NtQueryDirectoryObject invocations so that it gets
a really big buffer (64K) and ideally fetches all entries at once.
This appears to work atomically.
"Almost" all, because fhandler_procsys::readdir can't be easily changed.
[1] https://cygwin.com/pipermail/cygwin/2021-July/248998.html
[2] https://cygwin.com/pipermail/cygwin/2021-August/249124.html
Fixes: e9c8cb3193 ("(format_proc_partitions): Revamp loop over existing harddisks by scanning the NT native \Device object directory and looking for Harddisk entries.")
Fixes: a998dd7055 ("Implement advisory file locking.")
Fixes: 3b7cd74bfd ("(winpids::enum_processes): Fetch Cygwin processes from listing of shared cygwin object dir in the native NT namespace.")
Fixes: 0d6f2b0117 ("syscalls.cc (sync_worker): Rewrite using native NT functions.")
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Commit 3434d35a64 fixed a problem when
accessing block devices via their /proc/sys/Device entries. This
changed the way stat info is generated for these devices, resulting
in identical inode numbers for all block devices under /proc/sys/Device.
This patch fixes that by faking a device number for these devices, just as
before.
Fixes: 3434d35a64 ("Cygwin: Fix access to block devices below /proc/sys.")
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
If a service is supported as TCP and UDP service, GetAddrInfo does not
return two entries, one for TCP, one for UDP, as on Linux. Rather, it
just returns a single entry with ai_socktype and ai_protocol set to 0.
If the service only exists as TCP or UDP service, then ai->ai_socktype
is set, but ai_protocol isn't.
Fortunately we copy over the result from Windows into local storage
anyway, so this patch adds code to fix up the fields neglected by
Windows. In case ai_socktype as well as ai_protocol are 0, duplicate
the entry with valid values for ai_socktype and ai_protocol.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
If an interface is disconnected, net.cc:get_ifs tries to fetch IPv4
addresses from the registry. If it fails, it currently returns
pointers to sockaddr structs with zero address. Return a NULL pointer
instead, to signal the caller of getifaddrs that we do not have a
valid struct sockaddr.
Partially addresses: https://cygwin.com/pipermail/cygwin/2021-July/248970.html
It appears to be the case that NtQueryTimer can return a negative time
remaining for an unsignalled timer. The value appears to be less than
the timer resolution.
Signed-off-by: David Allsopp <david.allsopp@metastack.com>
These are updates to wire into the build tree the new tools profiler and
gmondump, and to supply documentation for the tools.
The documentation for profiler and ssp now mention each other but do not
discuss their similarities or differences. That will be handled in a
future update to the "Profiling Cygwin Programs" section of the Cygwin
User's Guide, to be supplied.
The Linux man page for cfsetspeed(3) specifies that the speed argument
must be one of the constants Bnnn (e.g., B9600) defined in termios.h.
But Linux in fact allows the speed to be the numerical baud rate
(e.g., 9600). For consistency with Linux, we now do the same.
Addresses: https://cygwin.com/pipermail/cygwin/2021-July/248887.html
The default PSAPI_VERSION is controlled by WIN32_WINNT, which we set to
0x0a00 when building utils since 48a76190 (and is the default in w32api
>= 9.0.0)
In order for the built executables to run on Windows Vista, we must also
define PSAPI_VERSION as 1 (otherwise '#define GetModuleFileNameExA
K32GetModuleFileNameExA' causes a 'The procedure entry point
K32GetModuleFilenameExA could not be located in the dynamic link library
kernel32.dll' error at run time).
Also drop uneeded psapi.h from dlfcn.cc (31ddf45d), resource.cc
(34a6eeab) and ps.cc (1def2148).
xterm 368 and mintty 3.5.1 implement a new feature to support
notification of terminal scaling via font zooming also if the terminal
text dimensions (rows/columns) stay unchanged, using
ioctl(TIOCSWINSZ), raising SIGWINCH;
this patches cygwin to support that scenario
The new GetFinalPathNameW handling for native symlinks in inner path
components is disabled if caller doesn't want to follow symlinks, or
doesn't want to follow reparse points.
The check for a file or dir within /dev/mqueue is accidentally using
the incoming path, which could be a relative path. Make sure to
restore the absolute POSIX path in path_copy and only then test the
path.
Also, move the actual check for a valid path below /dev/mqueue into
the fhandler_mqueue class.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
/dev has been handled as virtual dir in cwdstuff, thus not allowing
to start native apps from /dev as CWD, even if /dev actually exists
on disk. Unfortunately this also affects Cygwin executables started
from a debugger.
When chdir'ing to /dev, check if /dev exists on disk. If so, treat
it as any other existing path.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
reuse fhandler_virtual implementation to implement read and lseek.
The output from read(2) is modelled after the output when reading
from an mq file on Linux.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
The POSIX entry points are just wrappers now, calling into
fhandler_mqueue. While at it, eliminate mqi_flags, replace with
standard fhandler nonblocking flag.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
The mq_open call is just a framework now. This patch moves the
entire functionality into fhandler_mqueue. To support standard
OS calls (as on Linux), make fhandler_mqueue a derived class from
fhandler_disk_file and keep the base handle the handle to the
default stream, to allow implementing O_PATH functionlaity as well
as reading from the file and NOT reading binary message queue data.
Implement a standard fhandler_mqueue::open method, allowing, for
instance, to touch a file under /dev/mqueue and if it doesn't exist,
to create a message queue file.
FIXME: This introduces a BAD HACK into path_conv::check, which needs
reviewing.
Keep the posix path intact in the fhandler, and change get_proc_fd_name
accordingly to return only the basename plus leading slash for
/proc/<PID>/fd.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
POSIX message queues will be moved into NTFS streams.
Extend get_nt_native_path to provide a filename suffix which is not
subject to special character transposition, to allow specifying
a colon.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
This encapsulated creation, duplication, and closing of all
Windows objects connected to the message queue in the fhandler.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
The function evaluating correctness of ipc object names was a
bit half-hearted. Fix the tests to follow more closely the
desriptions in the Linux man pages.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
build_fh_dev can take the POSIX object name as parameter anyway,
so use that and drop from mqinfo call.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Drop creating a unique ID, the queue name itself is already
unique. This allows to move ipc object generation into the
fhandler in the next step.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
So far, the mqd_t type returned a pointer to an allocated
area under the hood. The mutex and event objects attached
to the message queue were implemented as inheritable types.
As unfortunate side effect the HANDLEs to these objects
were inherited by exec'd child processes, even though all
other message queue properties are not inherted, per POSIX.
Fix this by converting an mqd_t to a descriptor, and create a
matching fhandler_mqueue object to handle various aspects of
the message queues inside the fhandler. Especially, create the
IPC objects as non-inheritable and duplicate the HANDLEs as
part of the fixup_after_fork mechanism.
Drop using mmap and create the memory map with NT functions.
This allows to control duplication of handle and mapping in the
forked child process, without the requirement to regenerate the
map in the same spot. It also allows to dup() the descriptor,
as on Linux, albeit this isn't implemented yet.
This patch is the first cut. There's a bit more to do, like
moving more functionality from the POSIX functions into the
fhandler and making sure the mqd_t type can't be used in other
descriptor-related functions willy-nilly.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
The previous patch is accidentally missing the declaration of
IsWow64Process2. Add it belatedly.
Fixes: 1865a41cb383 ("Cygwin: suppress FAST_CWD warnings on ARM64")
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
The old check was insufficient: new insider preview builds of Windows
allow running x86_64 process on ARM64. The IsWow64Process2 function
seems to be the intended way to figure this situation out.
This avoids MAX_PATH-related problems in native tools in case the
virtual drive points to a deep directory
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Rather than fetching the system Windows directory at dll init time
only on 32 bit, fetch it on all platforms. Store as WCHAR and
UNICODE_STRING. Use where appropriate to simplify code.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
commit 456c3a4638 was only going half-way. It handled symlinks and
junction points as inner path components and made realpath return the
correct path, but it ignored drive letter substitution, i. e., virtual
drives created with, e. g.
subst X: C:\foo\bar
It was also too simple. Just returning an error code from
symlink_info::check puts an unnecessary onus on the symlink evaluation
loop in path_conv::check.
Rework the code to use GetFinalPathNameByHandle, and only do this after
checking the current file for being a symlink failed.
If the final path returned by GetFinalPathNameByHandle is not the same
as the incoming path, replace the incoming path with the POSIXified
final path. This also short-circuits path evaluation, because
path_conv::check doesn't have to recurse over the inner path components
multiple times if all symlinks are of a native type, while still getting
the final path as end result.
Virtual drives are now handled like symlinks. This is a necessary change
from before to make sure virtual drives are handled identically across
different access methods. An example is realpath(1) from coreutils. It
doesn't call readlink(2), but iterates over all path components using
lstat/readlink calls. Both methods should result in the same real path.
Fixes: 456c3a4638 ("path_conv: Try to handle native symlinks more sanely")
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Converting to automake dropped the former, handwritten tags/ctags
target. This leads to a couple of problems:
- For no good reason the tags file gets written to the builddir
instead of to the srcdir where it's needed.
- `make tags' requires etags to exist, rather than checking if it
exists and skipping it.
- Adding the extra ctags arguments to AM_CTAGSFLAGS still results
in a shortened tags file.
(Temporary?) solution: Revert the old tags/ctags rules and silence
the automake warnings.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
page/swap space name >= 40 or size/used >= 8 leaves no space between fields;
ensure a space after name and add extra tabs after size and used fields;
output appears like Linux 5.8 after changes to mm/swapfile(swap_show);
proc-swaps-space-before.log:
==> /proc/swaps <==
Filename Type Size Used Priority
/mnt/c/pagefile.sys file 11567748292920 0
/mnt/d/pagefile.sys file 12582912205960 0
proc-swaps-space-after.log:
==> /proc/swaps <==
Filename Type Size Used Priority
/mnt/c/pagefile.sys file 11567748 241024 0
/mnt/d/pagefile.sys file 12582912 182928 0
_mq_send as well as _mq_receive call ipc_mutex_unlock twice in case
of success, after having introduced __try/__except blocks.
Fixes: 3f3bd10104 ("* Throughout, use __try/__except/__endtry blocks, rather than myfault handler.")
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
POSIX requires that message priorities range from 0 (low) to
sysconf(_SC_MQ_PRIO_MAX) - 1 (high). Cygwin's mq_send erroneously
allowed a message priority of sysconf(_SC_MQ_PRIO_MAX). Fix it.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Specify toollibdir and toolincludedir in terms of tooldir, so the
install location is correct if tooldir is the default
($(exec_prefix)/$(target_alias)), or explicitly specified on the 'make'
command line.
Linux 5.12 Frozen Wasteland added features and changes:
add AMD 0x8000000a EDX:20 v_spec_ctrl virtual speculation control support
add Intel 0x00000007 ECX:24 bus_lock_detect bus lock detect debug exception
v2:
* Include tzmap.h in BUILT_SOURCES
* Make per-file flags appear after user-supplied CXXFLAGS, so they can
override optimization level.
* Correct .o files used to define symbols exported by libm.a
* Drop gcrt0.o mistakenly included in libgmon.a
* Add missing line continuations in GMON_FILES value
v3:
* use per-file flags for .c compilation
* override C{XX,}FLAGS, as they are set on the command line by top-level make
v4:
* Drop -Wno-error=write-strings from path_testsuite CXXFLAGS
v5:
* Update for changes in master
- Add -fno-threadsafe-statics to CXX flags
- Add hypotl.cc
- Remove fenv.cc (in favour of newlib), add fenv.c stub
- Add proc.5 manpage rules
Following POSIX and Linux, allow a connected DGRAM socket's connection
to be reset (so that the socket becomes unconnected). This is done by
calling connect and specifing an address whose family is AF_UNSPEC.
When connect is called on a DGRAM socket, the call to Winsock's
connect can immediately return successfully rather than failing with
WSAEWOULDBLOCK. Set the connect state to "connected" in this case.
Previously the connect state remained "connect_pending" after the
successful connection.
- In commit bb93c6d7, the race issue was not completely fixed. In
the pseudo console inheritance, if the destination process to
which the ownership of pseudo console switches, is found but exits
before switching, the inheritance fails. Currently, this extremely
rarely happens. This patch fixes the issue.
Commit 456c3a4638 added a workaround when handling paths with native
symlinks as inner path components. This patch introduced a problem for
paths handled by the WOW64 File System Redirector (FSR).
Fix this problem by not performing the new code from commit 456c3a4638
for paths under the Windows directory. Only do this in WOW64.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
For local paths, add a check if the inner path components contain native
symlinks or junctions. Compare the incoming path with the path returned
by NtQueryInformationFile(FileNameInformation). If they differ, there
must be at least one native symlink or junction in the path. If so,
treat the currently evaluated file as non-existant. This forces
path_conv::check to backtrack inner path components until we eliminated
all native symlinks or junctions and have a normalized path.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
The simple newlib hypotl for real long double architectures is too
simple at this point. It's implemented as a real call to sqrtl(x^2+y^2).
This has a fatal tendency to overflow for big input numbers. Hypotl
isn't supposed to do that if the result would still be valid in range of
long double.
Given the complexity of implementing hypotl for various architectures,
we just take the hypotl function from Mingw-w64, which is in the public
domain.
Even though this hypotl is an architecture-independent implementation,
we can't use it for newlib yet, unfortunately, because it requires logbl
under the hood. Logbl is yet another function missing in newlib for
real long double architectures.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Use the more official fesetenv(FE_DFL_ENV) from _dll_crt0, thus
allowing to drop the _feinitialise declaration from fenv.h.
Provide a no-op _feinitialise in Cygwin as exportable symbol for really
old applications when _feinitialise was called from mainCRTStartup in
crt0.o.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Drop the Cygwin-specific fenv.cc and fenv.h file and use the equivalent
newlib functionality now, so we have at least one example of a user for
this new mechanism.
fenv.c: allow _feinitialise to be called from Cygwin startup code
fenv.h: add declarations for fegetprec and fesetprec for Cygwin only.
Fix a comment.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
An FD_CLOSE event sets a socket descriptor ready for writing.
This is incorrect if the FD_CLOSE is a result of shutdown(SHUT_RD).
Only set the socket descriptor ready for writing if the FD_CLOSE
is indicating an connection abort or reset error condition.
This requires to tweak fhandler_socket_wsock::evaluate_events.
FD_CLOSE in conjunction with FD_ACCEPT/FD_CONNECT special cases
a shutdown condition by setting an error code. This is correct
for accept/connect, but not for select. In this case, make sure
to return with an error code only if FD_CLOSE indicates a
connection error.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
So far select wrongly sets the descriptor as ready for exception
when a shutdown occurs. This is entirely non-standard. Only set
this bit on an out-of-band event.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
- Currently, functions/variables regarding the handles for cygwin
apps are with "_cyg", and those of handles for non-cygwin apps
are without "_cyg", such as get_handle_cyg() and get_handle().
This patch renames these to the names without "_nat" and with
"_nat" respectively, such as get_handle() and get_handle_nat().
When the Windows Store version of Python is installed, so-called "app
execution aliases" are put into the `PATH`. These are reparse points
under the hood, with an undocumented format.
We do know a bit about this format, though, as per the excellent analysis:
https://www.tiraniddo.dev/2019/09/overview-of-windows-execution-aliases.html
The first 4 bytes is the reparse tag, in this case it's
0x8000001B which is documented in the Windows SDK as
IO_REPARSE_TAG_APPEXECLINK. Unfortunately there doesn't seem to
be a corresponding structure, but with a bit of reverse
engineering we can work out the format is as follows:
Version: <4 byte integer>
Package ID: <NUL Terminated Unicode String>
Entry Point: <NUL Terminated Unicode String>
Executable: <NUL Terminated Unicode String>
Application Type: <NUL Terminated Unicode String>
Let's treat them as symbolic links. For example, in this developer's
setup, this will result in the following nice output:
$ cd $LOCALAPPDATA/Microsoft/WindowsApps/
$ ls -l python3.exe
lrwxrwxrwx 1 me 4096 105 Aug 23 2020 python3.exe -> '/c/Program Files/WindowsApps/PythonSoftwareFoundation.Python.3.7_3.7.2544.0_x64__qbz5n2kfra8p0/python.exe'
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The Windows Store version of Python (and apparently other Windows Store
applications) install a special reparse point called "app execution
alias" into the user's `PATH`.
These applications can be executed without any problem, but they cannot
be read as if they were files. This trips up Cygwin's beautiful logic that
tries to determine whether we're about to execute a Cygwin executable or
not: instead of executing the application, it will fail, saying
"Permission denied".
Let's detect this situation (`NtOpenFile()` helpfully says that this
operation is not supported on this reparse point type), and simply skip
the logic: Windows Store apps are not Cygwin executables (and even if
they were, it is unlikely that they would come with a compatible
`cygwin1.dll` or `msys-2.0.dll`).
This fixes https://github.com/msys2/MSYS2-packages/issues/1943
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
- Currently, names of output pipes are "pty%d-to-master" and "pty%d-
to-master-cyg" and names of input pipes are "pty%d-to-slave" and
"pty%d-from-master". With this patch, these pipes are renamed to
"pty%d-to-master-nat", "pty%d-to-master", "pty%d-from-master-nat"
and "pty%d-from-master" respectively.
- Currently, transfer input is triggered even if the stdin of native
app is not a pseudo console. With this patch it is triggered only
if the stdin is a pseudo console.
- If two non-cygwin apps are started simultaneously, attaching to
pseudo console sometimes fails. This is because the second app
trys to attach to the process not started yet. This patch avoids
the issue by attaching to the stub process rather than the other
non-cygwin app.
- Perhaps current code misunderstand meaning of the IGNBRK. As far
as I investigated, IGNBRK is concerned with break signal in serial
port but there is no evidence that it has effect to ignore Ctrl-C.
This patch stops ignoring Ctrl-C by IGNBRK for non-cygwin apps.
This reverts commit 532b91d24e.
It turned out that this patch has undesired side effects. To wit, if a
newer, post-uname_x executable was linked against or loading an older,
pre-uname_x DLL, and this DLL called uname. This call would jump into
the old uname with the old struct utsname as parameter, but given the
newer executable it would get redirected to uname_x. uname_x in turn
would overwrite stack memory it should leave well alone, given it
expects the newer, larger struct utsname.
For the entire discussion see the thread starting at
https://cygwin.com/pipermail/cygwin/2021-February/247870.html
and continuing in March at
https://cygwin.com/pipermail/cygwin/2021-March/247930.html
For a description where we're coming from, see
https://cygwin.com/pipermail/cygwin/2021-March/247959.html
While we *could* make the scenario in question work by patching dlsym,
the problem would actually be the same, just for dynamic loading. In
the end, we're missing the information, which Cygwin version has been
used when building DLLs.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
- If two non-cygwin apps are started simultaneously and this is the
first execution of non-cygwin apps in the pty, these occasionally
hang up. The cause is the race issue between term_has_pcon_cap(),
reset_switch_to_pcon() and setup_pseudoconsole(). This patch fixes
the issue.
linkat(olddirfd, oldpath, oldname, newdirfd, newname, AT_EMPTY_PATH)
is supposed to create a link to the file referenced by olddirfd if
oldname is the empty string. Currently this is done via the /proc
filesystem by converting the call to
linkat(AT_FDCWD, "/proc/self/fd/<olddirfd>", newdirfd, newname,
AT_SYMLINK_FOLLOW),
which ultimately leads to a call to the appropriate fhandler's link
method. Simplify this by using cygheap_fdget to obtain the fhandler
directly.
If linkat(2) is called with AT_EMPTY_PATH on an AF_LOCAL or
AF_UNIX socket that is not a socket file, the current code calls
fhandler_disk_file::link in most cases. The latter expects to be
operating on a disk file and uses the socket's io_handle, which
is not a file handle.
Fix this by calling fhandler_disk_file::link only if the
fhandler_socket object is a file (determined by testing
dev().isfs()).
Also fix the case of a socket file opened with O_PATH by setting
the fhandler_disk_file's io_handle.
If facl(2) is called on an AF_LOCAL or AF_UNIX socket that is not a
socket file, the current code calls fhandler_disk_file::facl in most
cases. The latter expects to be operating on a disk file and uses the
socket's io_handle, which is not a file handle.
Fix this by calling fhandler_disk_file::facl only if the
fhandler_socket object is a file (determined by testing dev().isfs()).
If fchown(2) is called on an AF_LOCAL or AF_UNIX socket that is not a
socket file, the current code calls fhandler_disk_file::fchown in most
cases. The latter expects to be operating on a disk file and uses the
socket's io_handle, which is not a file handle.
Fix this by calling fhandler_disk_file::fchown only if the
fhandler_socket object is a file (determined by testing dev().isfs()).
If fchmod(2) is called on an AF_LOCAL or AF_UNIX socket that is not a
socket file, the current code calls fhandler_disk_file::fchmod in most
cases. The latter expects to be operating on a disk file and uses the
socket's io_handle, which is not a file handle.
Fix this by calling fhandler_disk_file::fchmod only if the
fhandler_socket object is a file (determined by testing dev().isfs()).
If fstatvfs(2) is called on an AF_LOCAL or AF_UNIX socket that is not
a socket file, the current code calls fhandler_disk_file::fstatvfs in
most cases. The latter expects to be operating on a disk file and
uses the socket's io_handle, which is not a file handle.
Fix this by calling fhandler_disk_file::fstatvfs only if the
fhandler_socket object is a socket file (determined by testing
dev().isfs()).
If fstat(2) is called on an AF_LOCAL or AF_UNIX socket that is not a
socket file, the current code calls fstat_fs. The latter expects to
be operating on a disk file and uses the socket's io_handle, which is
not a file handle.
Fix this by calling fstat_fs only if the fhandler_socket object is a
file (determined by testing dev().isfs()).
This is in the spirit of the Linux requirement that file operations
like fchmod(2), fchown(2), and fgetxattr(2) fail with EBADF on files
opened with O_PATH.
This was done for the fhandler_socket_local class in commits
3a2191653a, 141437d374, and 477121317d, but the fhandler_socket_unix
class was overlooked.
When a FIFO is opened, syscalls.cc:open always calls fstat on the
newly-created fhandler_fifo. This results from a call to
device_access_denied.
To speed-up this fstat call, and therefore the open(2) call, use
PC_KEEP_HANDLE when the fhandler is created. The resulting
conv_handle is retained until after the fstat call if the fhandler is
a FIFO; otherwise, it is closed immediately.
Previously, the call to get_file_attribute for FIFOs set the first
argument to NULL instead of the handle h returned by get_stat_handle,
thereby forcing the file to be opened for fetching the security
descriptor in get_file_sd(). This was done because h might have been
a pipe handle rather than a file handle, and its permissions would not
necessarily reflect those of the file.
That situation can no longer occur with the new fhandler_fifo::fstat
introduced in the previous commit.
Previously fstat on a FIFO would call fhandler_base::fstat.
The latter is not appropriate if fhandler_fifo::open has already been
called (and O_PATH is not set), for the following reason. If a FIFO
has been opened as a writer or duplexer, then it has an io_handle that
is a pipe handle rather than a file handle. fhandler_base::fstat will
use this handle and potentially return incorrect results. If the FIFO
has been opened as a reader, then it has no io_handle, and a call to
fhandler_base::fstat will lead to a call to fhandler_base::open.
Opening the fhandler a second time can change it in undesired ways;
for example, it can modify the flags and status_flags.
The new fhandler_fifo::fstat avoids these problems by creating an
fhandler_disk_file and calling its fstat method in case
fhandler_fifo::open has already been called and O_PATH is not set.
Consider this case:
- Cygwin installed in C:\cygwin64
- mklink /j D:\cygwin64 C:\cygwin64
- create testcase calling
realpath("/", result);
printf ("%s\n", result);
- start cmd
>C:\cygwin64\bin\bash -lc <path-to-testcase>
/
>D\cygwin64\bin\bash -lc <path-to-testcase>
/cygdrive/c/cygwin64
This scenario circumventing the mount point handling which is automated
in terms of /, depending on the path returned from GetModuleFileNameW
for the Cygwin DLL. When calling D:\cygwin64\bin\bash the dir returned
from GetModuleFileNameW is D:\cygwin64\bin, thus root is D:\cygwin64.
However, junctions are treated as symlinks in Cygwin which explains why
the path gets converted to a cygdrive path.
Fix this by calling GetFinalPathNameByHandleW on the result from
GetModuleFileNameW to get the correct root path, even if accessed via
a junction point.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
This only affects the very seldom bordercase of apps calling setmode(fd,
0) on fhandlers not calling fhandler_base::set_open_status(). All
fhandlers not calling set_open_status() are binary mode only, but the
way reset_to_open_binmode worked, calling setmode(fd, 0) would have
"reset" their open flags to O_TEXT accidentally.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
- If ENABLE_LINE_INPUT is set, Ctrl-S is handled by Windows if the
OS is Windows 7. This conflicts with Ctrl-S handling in cygwin
console code. This patch unsets ENABLE_LINE_INPUT flag in cygwin
and set it when native app is executed.
- If ENABLE_VIRTUAL_TERMINAL_INPUT is not set, changing window height
does not generate WINDOW_BUFFER_SIZE_EVENT. This happens if console
is in the legacy mode. Therefore, with this patch, the windows size
is checked every time in cons_master_thread() if the cosole is in
the legacy mode.
Linux 5.11 💕 Valentine's Day Edition 💕 added features and changes:
add Intel 0x00000007 EDX:23 avx512_fp16 and 0x00000007:1 EAX:4 avx_vnni;
group scattered AMD 0x8000001f EAX Secure Mem/Encrypted Virt features at end:
0 sme, 1 sev, 3 sev_es (more to come not yet displayed)
- Currently, Ctrl-Z, Ctrl-\ and SIGWINCH does not work in console
if the process does not call read() or select(). This is because
these are processed in process_input_message() which is called
from read() or select(). This is a long standing issue of console.
Addresses:
https://cygwin.com/pipermail/cygwin/2020-May/244898.htmlhttps://cygwin.com/pipermail/cygwin/2021-February/247779.html
With this patch, new thread which handles only input signals is
introduced so that Crtl-Z, etc. work without calling read() or
select(). Ctrl-S and Ctrl-Q are also handled in this thread.
- Currently, console read() keeps reading after SIGWINCH is sent
even if SA_RESTART flag is not set. With this patch, read()
returns EINTR on SIGWINCH if SA_RESTART flag is not set.
The same problem for SIGQUIT and SIGTSTP has also been fixed.
- Currently, input transfer is performed every time one line is read(),
if the non-cygwin app is running in the background. With this patch,
transfer is triggered by setpgid() rather than read() so that the
unnecessary input transfer can be reduced much in that situation.
Per discussion on cygwin-developers, a Cygwin tmpfile(3) implementation
has been added to syscalls.cc. This overrides the one supplied by
newlib. Then the open(2) flag O_TMPFILE was added to the open call that
tmpfile internally makes.
This v2 patch removes O_CREAT from open() call as O_TMPFILE obviates it.
Note that open() takes a directory's path but returns an fd to a file.
Analyzing the fhandler::copyto logic shows that the fhandler_base::reset
method was only called from copyto anyway.
Trying to convert reset to a protected method uncovered that the copyto
method is actually thought upside down from an object oriented POV.
Rather than calling copyto, manipulating the object given as parameter,
rename the method to copy_from, which manipulates the calling object
itself with data from the object given as parameter.
Eventually make reset a protected method and rename it to
_copy_from_reset_helper to clarify it's only called from copy_from.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
path_conv_handle::dup calls DuplicateHandle unconditionally,
but we only have a handle in some cases. Check handle for being
non-NULL before calling DuplicateHandle.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
path_conv::reset_conv_handle is only called after fhandler::copyto
has been called. This duplicated the path_conv_handle if there was
one, so just setting the conv handle to NULL potentially produces a
handle leak. Replace reset_conv_handle calls with calls to
close_conv_handle and drop the reset_conv_handle method.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
There's a slim chance that duplicating fhandlers may end up duplicating
path_conv_handle handles twice ending up with a handle leak, due to
fhandler_base::reset calling path_conv::operator<< after the only
caller, fhandler::copyto, already called path_conv::operator=.
Just drop the call which basically duplicates what path_conv::operator=
already did.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
This reverts commit 76dca77f04. That
commit was based on the incorrect assumption that get_stat_handle,
when called on a FIFO in fstat_helper, would always return a handle
that is safe to use for getting the file information.
That assumption is true in many cases but not all. For example, if
the call to fstat_helper arises from a call to fstat(2) on a FIFO that
has been opened for writing, then get_stat_handle will return a pipe
handle instead of a file handle.
Allow check_reparse_point_target to recognize reparse points with
reparse tag IO_REPARSE_TAG_AF_UNIX. These are used in recent versions
of Windows 10 to represent AF_UNIX sockets.
check_reparse_point_target now returns PATH_REP on files of this type,
so that they are treated as known reparse points (but not as sockets).
This allows tools like 'rm', 'ls', etc. to operate on these files.
Addresses: https://cygwin.com/pipermail/cygwin/2020-September/246362.htmlhttps://cygwin.com/pipermail/cygwin/2021-January/247666.html
The new header defines some Cygwin-specific limits, using private
names. It is included by include/limits.h.
For example, we now have
#define __OPEN_MAX 3200
in include/cygwin/limits.h and
#define OPEN_MAX __OPEN_MAX
in include/limits.h. The purpose is to hide implementation details
from users who view <limits.h>.
Replace all occurrences of OPEN_MAX_MAX by OPEN_MAX, and define the
latter to be 3200, which was the value of the former. In view of the
recent change to getdtablesize, there is no longer a need to
distinguish between these two macros.
Now that getdtablesize always returns OPEN_MAX_MAX, we can simplify
sysconf(_SC_OPEN_MAX) and getrlimit(RLIMIT_NOFILE) to just use that
same constant instead of calling getdtablesize.
According to the Linux man page for getdtablesize(3), the latter is
supposed to return "the maximum number of files a process can have
open, one more than the largest possible value for a file descriptor."
The constant OPEN_MAX_MAX is the only limit enforced by Cygwin, so we
now return that.
Previously getdtablesize returned the current size of cygheap->fdtab,
Cygwin's internal file descriptor table. But this is a dynamically
growing table, and its current size does not reflect an actual limit
on the number of open files.
With this change, gnulib now reports that getdtablesize and
fcntl(F_DUPFD) work on Cygwin. Packages like GNU tar that use the
corresponding gnulib modules will no longer use gnulib replacements on
Cygwin.
- Currently, thread created by pthread_create() is not suspended by
the signal SIGTSTP. For example, even if a process with a thread
is suspended by Ctrl-Z, the thread continues running. This patch
fixes the issue.
- Currently, read() returns -1 with EINTR if the process is suspended
by Ctrl-Z and resumed by fg command, while pty continues to read.
For example, xxd command stops with error "Interrupted system call"
after Ctrl-Z and fg. This patch aligns the behaviour with pty (and
Linux).
Allow fchmodat with the AT_SYMLINK_NOFOLLOW flag to succeed on
non-symlinks. Previously it always failed, as it does on Linux. But
POSIX permits it to succeed on non-symlinks even if it fails on
symlinks.
The reason for following POSIX rather than Linux is to make gnulib
report that fchmodat works on Cygwin. This improves the efficiency of
packages like GNU tar that use gnulib's fchmodat module. Previously
such packages would use a gnulib replacement for fchmodat on Cygwin.
- After commit bb428520, there has been the disadvantage:
7) Pseudo console cannot be activated if it is already activated for
another process on same pty.
This patch clears this disadvantage.
- After commit bb428520, there has been the disadvantage:
2) The apps which use console API cannot be debugged with gdb. This
is because pseudo console is not activated since gdb uses
CreateProcess() rather than exec(). Even with this limitation,
attaching gdb to native app, in which pseudo console is already
activated, works.
This patch clears this disadvantage.
- After commit bb428520, there has been the disadvantage:
4) Code page cannot be changed by chcp.com. Acctually, chcp works
itself and changes code page of its own pseudo console. However,
since pseudo console is recreated for another process, it cannot
inherit the code page.
This patch clears this disadvantage.
- PTY has a problem that the key input, which is typed during windows
native app is running, disappears when it returns to shell. This is
beacuse pty has two input pipes, one is for cygwin apps and the other
one is for native windows apps. The key input during windows native
program is running is sent to the second input pipe while cygwin
shell reads input from the first input pipe. This issue had been
fixed once by commit 29431fcb, however, the new implementation of
pseudo console support by commit bb428520 could not inherit this
feature. This patch realize transfering input data between these
two pipes bidirectionally by utilizing cygwin-console-helper process.
The helper process is launched prior to starting the non-cygwin app,
however, exits immediately unlike previous implementation.
Corinna Vinschen
Mark Geisert
Ken Brown
Takashi Yano
Aleksand Malikov
Jon Turney
David Allsopp
Thomas Wolff
Jeremy Drake
Brian Inglis
Johannes Schindelin
Takashi Yano via Cygwin-patches
Marek Smetana