Windows NUL device returns only the lower 32 bit of the number of
bytes written. Implement a fake write function to ignore the underlying
NUL device.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
So far we copy *_impure_ptr into _main_tls->local_clib if the child
process has been forked from a pthread. But that's not required.
The local_clib area of the new thread is on the stack and the stack
gets copied from the parent anyway (in frok::parent). So we only
have to make sure _main_tls is pointing to the right address and
do the simple post-fork thread init.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
This test was broken from the start. It leads to creating a completely
new stack for the main thread of the child process when started from
the main thread of the parent. However, the main thread of a process
can easily running on a completely different stack, if the parent's main
thread was created by calling fork() from a pthread. For an example,
see https://cygwin.com/ml/cygwin/2017-03/msg00113.html
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
We use errno AKA _REENT->_errno since the last century and only set
_impure_ptr->_errno for backward compat. Stop that. Also, remove
the last check for _impure_ptr->_errno in Cygwin code.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Note that this always returns with dli_sname and dli_saddr set to NULL,
indicating no symbol matching addr could be found.
Signed-off-by: Jon Turney <jon.turney@dronecode.org.uk>
POSIX states as follows about pthread_cond_wait:
If a signal is delivered to a thread waiting for a condition variable,
upon return from the signal handler the thread resumes waiting for the
condition variable as if it was not interrupted, or it returns zero
due to spurious wakeup.
Cygwin so far employs the latter behaviour, while Linux and BSD employ
the former one.
Align Cygwin behaviour to Linux and BSD.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
There are certain, very obscure scenarios, which render the Windows
CWD handle inaccessible for reopening. An easy one is, the handle can
be NULL if the permissions of the CWD changed under the parent processes
feet.
Originally we just set errno and returned, but in case of init at
process startup that left the "posix" member NULL and subsequent
calls to getcwd failed with EFAULT.
We now check for a NULL handle and change the reopen approach
accordingly. If that doesn't work, try to duplicate the handle instead.
If duplicating fails, too, we set the dir handle to NULL and carry on.
This will at least set posix to some valid path and subsequent getcwd
calls won't fail. A NULL dir handle is ok, because we already do this
for virtual paths.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
This patch alters the behaviour of dll_list::topsort to preserve the
order of dlopen'd units.
The load order of unrelated DLLs is reversed every time fork is called,
since dll_list::topsort finds the tail of the list and then unwinds to
reinsert items. My change takes advantage of what should be undefined
behaviour in dll_list::populate_deps (ndeps non-zero and ndeps and deps
not initialised) to allow the deps field to be initialised prior to the
call and appended to, rather than overwritten.
All DLLs which have been dlopen'd have their deps list initialised with
the list of all previously dlopen'd units. These extra dependencies mean
that the unwind preserves the order of dlopen'd units.
The motivation for this is the FlexDLL linker used in OCaml. The FlexDLL
linker allows a dlopen'd unit to refer to symbols in previously dlopen'd
units and it resolves these symbols in DllMain before anything else has
initialised (including the Cygwin DLL). This means that dependencies may
exist between dlopen'd units (which the OCaml runtime system
understands) but which Windows is unaware of. During fork, the
process-level table which FlexDLL uses to get the symbol table of each
DLL is copied over but because the load order of dlopen'd DLLs is
reversed, it is possible for FlexDLL to attempt to access memory in the
DLL before it has been loaded and hence it fails with an access
violation. Because the list is reversed on each call to fork, it means
that a subsequent call to fork puts the DLLs back into the correct
order, hence "even" invocations of fork work!
An interesting side-effect is that this only occurs if the DLLs load at
their preferred base address - if they have to be rebased, then FlexDLL
works because at the time that the dependent unit is loaded out of
order, there is still in memory the "dummy" DONT_RESOLVE_DLL_REFERENCES
version of the dependency which, as it happens, will contain the correct
symbol table in the data section. For my tests, this initially appeared
to be an x86-only problem, but that was only because the two DLLs on x64
should have been rebased.
Signed-off-by: David Allsopp <david.allsopp@metastack.com>
Match glibc behaviour to expose the public bswap_* macros only with an
explicity #include <byteswap.h>; #include'ing <endian.h> should not expose
them.
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
Don't set SO_RCVBUF/SO_SNDBUF to fixed values, thus disabling autotuning.
Patch modeled after a patch suggestion from Daniel Havey <dhavey@gmail.com>
in https://cygwin.com/ml/cygwin-patches/2017-q1/msg00010.html:
At Windows we love what you are doing with Cygwin. However, we have
been getting reports from our hardware vendors that iperf is slow on
Windows. Iperf is of course compiled against the cygwin1.dll and we
believe we have traced the problem down to the function fdsock in
net.cc. SO_RCVBUF and SO_SNDBUF are being manually set. The comments
indicate that the idea was to increase the buffer size, but, this code
must have been written long ago because Windows has used autotuning
for a very long time now. Please do not manually set SO_RCVBUF or
SO_SNDBUF as this will limit your internet speed.
I am providing a patch, an STC and my cygcheck -svr output. Hope we
can fix this. Please let me know if I can help further.
Simple Test Case:
I have a script that pings 4 times and then iperfs for 10 seconds to
debit.k-net.fr
With patch
$ bash buffer_test.sh 178.250.209.22
usage: bash buffer_test.sh <iperf server name>
Pinging 178.250.209.22 with 32 bytes of data:
Reply from 178.250.209.22: bytes=32 time=167ms TTL=34
Reply from 178.250.209.22: bytes=32 time=173ms TTL=34
Reply from 178.250.209.22: bytes=32 time=173ms TTL=34
Reply from 178.250.209.22: bytes=32 time=169ms TTL=34
Ping statistics for 178.250.209.22:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 167ms, Maximum = 173ms, Average = 170ms
------------------------------------------------------------
Client connecting to 178.250.209.22, TCP port 5001
TCP window size: 64.0 KByte (default)
------------------------------------------------------------
[ 3] local 10.137.196.108 port 58512 connected with 178.250.209.22 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0- 1.0 sec 768 KBytes 6.29 Mbits/sec
[ 3] 1.0- 2.0 sec 9.25 MBytes 77.6 Mbits/sec
[ 3] 2.0- 3.0 sec 18.0 MBytes 151 Mbits/sec
[ 3] 3.0- 4.0 sec 18.0 MBytes 151 Mbits/sec
[ 3] 4.0- 5.0 sec 18.0 MBytes 151 Mbits/sec
[ 3] 5.0- 6.0 sec 18.0 MBytes 151 Mbits/sec
[ 3] 6.0- 7.0 sec 18.0 MBytes 151 Mbits/sec
[ 3] 7.0- 8.0 sec 18.0 MBytes 151 Mbits/sec
[ 3] 8.0- 9.0 sec 18.0 MBytes 151 Mbits/sec
[ 3] 9.0-10.0 sec 18.0 MBytes 151 Mbits/sec
[ 3] 0.0-10.0 sec 154 MBytes 129 Mbits/sec
Without patch:
dahavey@DMH-DESKTOP ~
$ bash buffer_test.sh 178.250.209.22
Pinging 178.250.209.22 with 32 bytes of data:
Reply from 178.250.209.22: bytes=32 time=168ms TTL=34
Reply from 178.250.209.22: bytes=32 time=167ms TTL=34
Reply from 178.250.209.22: bytes=32 time=170ms TTL=34
Reply from 178.250.209.22: bytes=32 time=169ms TTL=34
Ping statistics for 178.250.209.22:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 167ms, Maximum = 170ms, Average = 168ms
------------------------------------------------------------
Client connecting to 178.250.209.22, TCP port 5001
TCP window size: 208 KByte (default)
------------------------------------------------------------
[ 3] local 10.137.196.108 port 58443 connected with 178.250.209.22 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0- 1.0 sec 512 KBytes 4.19 Mbits/sec
[ 3] 1.0- 2.0 sec 1.50 MBytes 12.6 Mbits/sec
[ 3] 2.0- 3.0 sec 1.50 MBytes 12.6 Mbits/sec
[ 3] 3.0- 4.0 sec 1.50 MBytes 12.6 Mbits/sec
[ 3] 4.0- 5.0 sec 1.50 MBytes 12.6 Mbits/sec
[ 3] 5.0- 6.0 sec 1.50 MBytes 12.6 Mbits/sec
[ 3] 6.0- 7.0 sec 1.50 MBytes 12.6 Mbits/sec
[ 3] 7.0- 8.0 sec 1.50 MBytes 12.6 Mbits/sec
[ 3] 8.0- 9.0 sec 1.50 MBytes 12.6 Mbits/sec
[ 3] 9.0-10.0 sec 1.50 MBytes 12.6 Mbits/sec
[ 3] 0.0-10.1 sec 14.1 MBytes 11.7 Mbits/sec
The output shows that the RTT from my machine to the iperf server is
similar in both cases (about 170ms) however with the patch the
throughput averages 129 Mbps while without the patch the throughput
only averages 11.7 Mbps. If we calculate the maximum throughput using
Bandwidth = Queue/RTT we get (212992 * 8)/0.170 = 10.0231 Mbps. This
is just about what iperf is showing us without the patch since the
buffer size is set to 212992 I believe that the buffer size is
limiting the throughput. With the patch we have no buffer limitation
(autotuning) and can develop the full potential bandwidth on the link.
If you want to duplicate the STC you will have to find an iperf server
(I found an extreme case) that has a large enough RTT distance from
you and try a few times. I get varying results depending on Internet
traffic but without the patch never exceed the limit caused by the
buffering.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
The termios code doesn't handle erasing of multibyte characters
in canonical mode, it always erases a single byte. When entering
a multibyte character and then pressing VERASE, the input ends up
with an invalid character.
Following Linux we introduce the IUTF8 input flag now, set by
default. When this flag is set, VERASE or VWERASE will check
if the just erased input byte is a UTF-8 continuation byte. If
so, it erases another byte and checks again until the entire
UTF-8 character has been removed from the input buffer.
Note that this (just as on Linux) does NOT work with arbitrary
multibyte codesets. This only works with UTF-8.
For a discussion what happens, see
https://cygwin.com/ml/cygwin/2017-01/msg00299.html
Sidenote: The eat_readahead function is now member of fhandler_termios,
not fhandler_base. That's necessary to get access to the terminal's
termios flags.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
- Drop virtual_key_code (only used once)
- Convert macros wch and control_key_state to const vars
unicode_char and ctrl_key_state.
- Fix formatting
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Create two new inline functions is_alt_numpad_key(PINPUT_RECORD) and
is_alt_numpad_event(PINPUT_RECORD) which contain the actual checks.
Call these functions from fhandler_console::read and peek_console for
better readability.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
{p}select/{p}poll completely ignored Alt+Numpad key sequences in console
input which results in newer readline using pselect to fail handling such
sequences correctly. See https://cygwin.com/ml/cygwin/2017-01/msg00135.html
During debugging and testing it turned out that while reading console
input, single key presses during an Alt+Numpad sequences where not
ignored, so ultimately a sequence like
Alt-down Numpad-1 Numpad-2 Numpad-3
whihc is supposed to result in a single character in the input stream
will actually result in 4 chars in the input stream, three control
sequences and the actual character.
Both problems should be fixed by this patch.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
This patch fixes the following problem:
Commit 9636c426 refactored the pipe code especially to make sure
to call WriteFile only with chunks matching the maximum atomic write
count. This accidentally introduced a small change in behaviour
on blocking pipes due to the success case falling through into the
error case. Rather then writing atomic chunks until all bytes are
written, the code immediately broke from the loop after writing
the first chunk, basically the same as in case of non-blocking
writes. This behaviour is not compliant to POSIX which requires
"Write requests to a pipe or FIFO [...]
* If the O_NONBLOCK flag is clear, a write request may cause the
thread to block, but on normal completion it shall return nbyte."
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
If newfile already exists and is in use, trying to overwrite it with
NtSetInformationFile(FileRenameInformation) fails exactly as if we
don't have the permissions to delete it. Unfortunately the return code
is the same STATUS_ACCESS_DENIED, so we have no way to distinguish
these cases. What we do here so far is to start a transaction to delete
newfile. If this open fails with a transactional error we stop the
transaction and retry opening the file without transaction.
But, here's the problem: If newfile is in use, NtOpenFile(oldfile)
naturally does NOT fail with a transactional error. Rather, the
subsequent call to unlink_nt(newfile) does, because there's another
handle open to newfile outside a transaction. However, the code does
not check if unlink_nt fails with a transactional error and so fails
to retry without transaction.
This patch recifies the problem and checks unlink_nt's status as well.
Refactor code to get rid of goto into another code block.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
A *very* recent Windows build adds more code to the preamble of
RtlGetCurrentDirectory_U() so that the previous heuristic failed to find
the call to the locking routine.
This only affects the 64-bit version of ntdll, where the 0xe8 byte is
now found at offset 40, not the 32-bit version. However, let's just
double the area we search for said byte for good measure.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Returns the process's environment concatenated into a single block of
null-terminated strings, along with the length of the environment block.
Adds an associated PICOM_ENVIRON commune_process handler.
win32env_to_cygwenv handles converting wchar to char and some other
minor taks. Optionally it handles converting any paths in variables to
posix paths.
This will be useful for implementing /proc/<pid>/environ
commit 67fd2101 introduced a bad bug. Changing sys_privs to a static
area and just returning a pointer is nice... *if* the calling code doesn't
call free() on it. Make sure callers check pointer for sys_privs and
refrain from calling free, if so.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Include ntsecapi.h where required and just redefine RtlGenRandom
correctly in the ntsecapi.h wrapper.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
In preparation of exporting getentropy/getrandom to userspace, rearrange
code a bit:
- Define RtlGenRandom in ntdll.h.
- Drop calls to getentropy in favor of RtlGenRandom (fhandler_socket,
fhandler_dev_random).
- Add try/except blocks in fhandler_dev_random to return EFAULT rather
than crashing if buffer pointer is invalid.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
NtOpenFile/NtCreateFile on non-existent paths on network drives has a
bug. Assuming a path Z:\dir\file. Further assuming that Z:\dir does
not exist.
The first NtOpenFile("Z:\dir\file") correctly returns
STATUS_OBJECT_PATH_NOT_FOUND. Subsequent calls incorrectly
return STATUS_OBJECT_NAME_NOT_FOUND.
This appears to be some kind of caching behaviour. Waiting a while
before repeating the call correctly returns STATUS_OBJECT_PATH_NOT_FOUND
again.
This patch works around the observed misbehaviour.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
So far, when converting from POSIX to Windows notation, cygwin_conv_path
fails to check for .exe suffix, so /path/foo did not return /path/foo.exe
even if this file exists.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
The original code only skipped the "./", but missed to test if more
trailing slashes are present. This in turn leads to invalid conversion.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Don't allow signal 0 in signal(2), sigaction(2), siginterrupt(3).
Don't allow any signal in sigqueue(3) but explicitely handle
signal 0 as in kill(2).
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Provide <memory.h> for all standard Newlib targets and remove
Cygwin-specific header. Most POSIX like systems provide this historic
header.
Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
This makes it possible provide operating system specific types for
<pthread.h>. It is in line with the FreeBSD header file structure and
allows a future cleanup of <pthread.h> to not expose unrelated things
via <sys/types.h> and <unistd.h>. Glibc uses the similar
<bits/pthreadtypes.h> for this purpose.
Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
Update the getconf utility to support the new flag as well as
_PC_POSIX_PERMISSIONS and _PC_POSIX_SECURITY. These were previously
unsupported, probably as an oversight.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Additionally to eccefd97, we need to ensure the exception handler is
installed for the _ljfault used to implement _try/_except to get called.
Also use the correct macro for x86 conditional compilation.
Addresses https://cygwin.com/ml/cygwin/2016-09/msg00143.html
Signed-off-by: Jon Turney <jon.turney@dronecode.org.uk>
Long-standing problem in one of the corner cases of rename(2):
If we rename a directory a check is performed to see if newpath is
identical to oldpath or a subdir of oldpath. This check is
(accidentally? no hints anywhere in ChangeLogs or code) performed
case-insensitive for as long as we use Unicode paths and NT functions.
This leads to the problems described in
https://cygwin.com/ml/cygwin/2016-09/msg00264.html
Change this to be conditional case-sensitive as all other checks but
let's take this with a grain of salt. There may be corner-cases in
this corner-case which require to chek parts of the path always
case-insensitive. Off the top of my head I can't construct such a
case but that's no proof they don't exist :}
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Commit d16a5630 dropped usage of cygheap's locale functions
in favor of local on-the-fly usage of UTF-8 instead of ASCII.
This allowed to use the current local rather than a fixed
system-wide locale set at startup time. d16a5630 just missed
to add the ASCII->UTF-8 conversion in the console as well.
Fixes https://cygwin.com/ml/cygwin/2016-10/msg00000.html
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
On 09/02/2016 11:03 AM, Corinna Vinschen wrote:
> On Sep 2 10:46, Michael Haubenwallner wrote:
>> On 09/01/2016 03:32 PM, Corinna Vinschen wrote:
>>> You could just use the global variable program_invocation_name. If in
>>> doubt, use the Windows path global_progname and convert it to full POSIX
>>> via cygwin_conv_path.
>>
>> Patch updated, using global_progname now.
>
> Looks good and you're right to do it this way since I just noticed
> that program_invocation_name may return a relative pathname.
Yep.
> Btw., in other calls which require the full POSIX path we use
> mount_table->conv_to_posix_path instead of cygwin_conv_path (see
> e. g. fillout_pinfo()). It's a bit faster. Maybe something for a
> followup patch.
No problem - attached.
This renders the original patch 4/4 valid again.
> Note for some later improvement: I really wonder why we don't store
> the absolute POSIX path of the current executable globally yet...
Same here.
Thanks!
/haubi/
>From f7255edd33cb4abe34f27188aab8dccdfa5dd2a0 Mon Sep 17 00:00:00 2001
From: Michael Haubenwallner <michael.haubenwallner@ssi-schaefer.com>
Date: Wed, 31 Aug 2016 18:05:11 +0200
Subject: [PATCH 3/4] dlopen: on x/lib search x/bin if exe is in x/bin
citing https://cygwin.com/ml/cygwin-developers/2016-08/msg00020.html
> Consider the file /usr/bin/cygz.dll:
> - dlopen (libz.so) success
> - dlopen (/usr/bin/libz.so) success
> - dlopen (/usr/lib/libz.so) fails
* dlfcn.c (dlopen): For dlopen("x/lib/N"), when the application
executable is in "x/bin/", search for "x/bin/N" before "x/lib/N".
Rather than searching all search dirs per one basename,
search for all basenames within per one search dir.
pathfinder.h (check_path_access): Interchange dir- and basename-loops.
Instead of find_exec, without changing behaviour use new pathfinder
class with new allocator_interface around tmp_pathbuf and new vstrlist
class.
* pathfinder.h (pathfinder): New file.
* vstrlist.h (allocator_interface, allocated_type, vstrlist): New file.
* dlfcn.cc (dlopen): Avoid redundant GetModuleHandleExW with RTLD_NOLOAD
and RTLD_NODELETE. Switch to new pathfinder class, using
(tmp_pathbuf_allocator): New class.
(get_full_path_of_dll): Drop.
On Windows 8.1 and later, the NetUserChangePassword call apparently
doesn't accept the usual "\\server" string anymore, but requires to
use the "domain" instead, otherwise it emits en error code 1265,
ERROR_DOWNGRADE_DETECTED. Since this is accepted by pre-8.1 as well,
use the domain indiscriminately when calling NetUserChangePassword
from passwd(1).
While at it, do some minor cleanup in passwd.c.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Keep __ctype_ptr__ available on Cygwin only, for backward compatibility
with existing apps referencing it via the ctype macros.
Otherwise initialize __global_locale.ctype_ptr and __C_locale.ctype_ptr
and use them throughout.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Wrap SetThreadName()'s call to RaiseException() in __try/__except/__endtry,
so that if the attached debugger doesn't know about MS_VC_EXCEPTION (e.g.
current gdb and probably strace as well) and continues exception processing,
we ignore it, rather than dying due an unhandled exception.
Also remove an unnecessary cast in the RaiseException() invocation.
Signed-off-by: Jon Turney <jon.turney@dronecode.org.uk>
GDB since commit 24cdb46e [1] can report and use these names.
Add utility function SetThreadName(), which sends a thread name to the
debugger.
Use that:
- to set the default thread name for main thread and newly created pthreads.
- in pthread_setname_np() for user thread names.
- for helper thread names in cygthread::create()
- for helper threads which are created directly with CreateThread.
Note that there can still be anonymous threads, created by system or
injected DLLs.
[1] https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=24cdb46e9f0a694b4fbc11085e094857f08c0419
This patch adds pthread_getname_np and pthread_setname_np.
These were added to glibc in 2.12[1] and are also present in some form on
NetBSD and several UNIXes.
The code is based on NetBSD's implementation with changes to better match
Linux behaviour.
Implementation quirks:
* pthread_setname_np with a NULL pointer segfaults (as linux)
* pthread_setname_np returns ERANGE for names longer than 16 characters (as
linux)
* pthread_getname_np with a NULL pointer returns EFAULT (as linux)
* pthread_getname_np with a buffer length of less than 16 returns ERANGE (as
linux)
* pthread_getname_np truncates the thread name to fit the buffer length.
This guarantees success even when the default thread name is longer than 16
characters, but means there is no way to discover the actual length of the
thread name. (Linux always truncates the thread name to 16 characters)
* Changing program_invocation_short_name changes the default thread name (on
linux, it has no effect on the default thread name)
I'll leave it up to you to decide if any of these matter.
This is implemented via class pthread_attr to make it easier to add
pthread_attr_[gs]etname_np (present in NetBSD and some UNIXes) should it
ever be added to Linux (or we decide we want it anyway).
[1] https://sourceware.org/git/?p=glibc.git;a=blob;f=NEWS
This routine makes a call to fabs instead of fabsl(), causing truncation.
Clang complains (warning: absolute value function 'fabs' given an argument of type 'long double' but has parameter of type 'double' which may cause truncation of value).
Signed-off-by: David Wohlferd <dw@LimeGreenSocks.com>
Corinna Vinschen
Yaakov Selkowitz
Jon Turney
David Allsopp
Michael Haubenwallner
Johannes Schindelin
Erik M. Bray
Sebastian Huber
Ken Brown
David Wohlferd