The locale_t type is provided by <xlocale.h> on Linux, FreeBSD, and Darwin.
While, like on some of those systems, it is automatically included by
<locale.h> with the proper feature test macros, its presence under this
particular name is still presumed in real-world software.
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
The original dll_init code was living under the wrong assumption that
dll_dllcrt0_1 and in turn dll_list::alloc will be called for each
LoadLibrary call. The same wrong assumption was made for
cygwin_detach_dll/dll_list::detach called via FreeLibrary.
In reality, dll_dllcrt0_1 gets only called once at first LoadLibrary
and cygwin_detach_dll once at last FreeLibrary.
In effect, reference counting for DLLs was completely broken after fork:
parent:
l1 = dlopen ("lib1"); // LoadLibrary, LoadCount = 1
l2 = dlopen ("lib1"); // LoadLibrary, LoadCount = 2
fork (); // LoadLibrary in the child, LoadCount = 1!
child:
dlclose (l1); // FreeLibrary actually frees the lib
x = dlsym (l2); // SEGV
* Move reference counting to dlopen/dlclose since only those functions
have to keep track of loading/unloading DLLs in the application context.
* Remove broken accounting code from dll_list::alloc and dll_list::detach.
* Fix error handling in dlclose.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Problem:
After passing locales created by 'duplocale' to 'uselocale',
referencing 'MB_CUR_MAX', which is actually expanded to
'__locale_mb_cur_max()' by preprocessors, causes segmentation faults.
Direct use of locales from 'newlocale' does not cause the problem.
This is the problem of 'duplocale'.
$ echo $LANG
ja_JP.UTF-8
$ cat test.c
#include <stdlib.h>
#include <locale.h>
volatile int var;
int main(void) {
locale_t const loc = newlocale(LC_ALL_MASK, "", NULL);
locale_t const dup = duplocale(loc);
locale_t const old = uselocale(dup);
var = MB_CUR_MAX; /* <-- crashes here */
uselocale(old);
freelocale(dup);
freelocale(loc);
return 0;
}
$ gcc test.c
$ ./a
Segmentation fault (core dumped)
# Note: "core dumped" in the above message was actually written in
# Japanese, but I translated the part to post a mail in English.
Bug:
In the beginning of '__loadlocale' (newlib/libc/locale/locale.c:501),
there is a code which checks if the operations can be skipped:
> /* Avoid doing everything twice if nothing has changed. */
> if (!strcmp (new_locale, loc->categories[category]))
> return loc->categories[category];
While, in the function '_duplocale_r' (newlib/libc/locale/
duplocale.c), '__loadlocale' is called as in the quoted codes:
> /* If the object is not a "C" locale category, copy it. Just call
> __loadlocale. It knows what to do to replicate the category. */
> tmp_locale.lc_cat[i].ptr = NULL;
> tmp_locale.lc_cat[i].buf = NULL;
> if (!__loadlocale (&tmp_locale, i, tmp_locale.categories[i]))
> goto error;
This call of '__loadlocale' results in the skip check being
!strcmp(tmp_locale.categories[i], tmp_locale.categories[i]),
which is always true. This means that the actual operations of
'__loadLocale' will never be performed for 'duplocale'.
Fix:
The call of '__loadlocale' in '_duplocale_r' is modified.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Windows NUL device returns only the lower 32 bit of the number of
bytes written. Implement a fake write function to ignore the underlying
NUL device.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
So far we copy *_impure_ptr into _main_tls->local_clib if the child
process has been forked from a pthread. But that's not required.
The local_clib area of the new thread is on the stack and the stack
gets copied from the parent anyway (in frok::parent). So we only
have to make sure _main_tls is pointing to the right address and
do the simple post-fork thread init.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
This test was broken from the start. It leads to creating a completely
new stack for the main thread of the child process when started from
the main thread of the parent. However, the main thread of a process
can easily running on a completely different stack, if the parent's main
thread was created by calling fork() from a pthread. For an example,
see https://cygwin.com/ml/cygwin/2017-03/msg00113.html
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
We use errno AKA _REENT->_errno since the last century and only set
_impure_ptr->_errno for backward compat. Stop that. Also, remove
the last check for _impure_ptr->_errno in Cygwin code.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Note that this always returns with dli_sname and dli_saddr set to NULL,
indicating no symbol matching addr could be found.
Signed-off-by: Jon Turney <jon.turney@dronecode.org.uk>
POSIX states as follows about pthread_cond_wait:
If a signal is delivered to a thread waiting for a condition variable,
upon return from the signal handler the thread resumes waiting for the
condition variable as if it was not interrupted, or it returns zero
due to spurious wakeup.
Cygwin so far employs the latter behaviour, while Linux and BSD employ
the former one.
Align Cygwin behaviour to Linux and BSD.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
There are certain, very obscure scenarios, which render the Windows
CWD handle inaccessible for reopening. An easy one is, the handle can
be NULL if the permissions of the CWD changed under the parent processes
feet.
Originally we just set errno and returned, but in case of init at
process startup that left the "posix" member NULL and subsequent
calls to getcwd failed with EFAULT.
We now check for a NULL handle and change the reopen approach
accordingly. If that doesn't work, try to duplicate the handle instead.
If duplicating fails, too, we set the dir handle to NULL and carry on.
This will at least set posix to some valid path and subsequent getcwd
calls won't fail. A NULL dir handle is ok, because we already do this
for virtual paths.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
This patch alters the behaviour of dll_list::topsort to preserve the
order of dlopen'd units.
The load order of unrelated DLLs is reversed every time fork is called,
since dll_list::topsort finds the tail of the list and then unwinds to
reinsert items. My change takes advantage of what should be undefined
behaviour in dll_list::populate_deps (ndeps non-zero and ndeps and deps
not initialised) to allow the deps field to be initialised prior to the
call and appended to, rather than overwritten.
All DLLs which have been dlopen'd have their deps list initialised with
the list of all previously dlopen'd units. These extra dependencies mean
that the unwind preserves the order of dlopen'd units.
The motivation for this is the FlexDLL linker used in OCaml. The FlexDLL
linker allows a dlopen'd unit to refer to symbols in previously dlopen'd
units and it resolves these symbols in DllMain before anything else has
initialised (including the Cygwin DLL). This means that dependencies may
exist between dlopen'd units (which the OCaml runtime system
understands) but which Windows is unaware of. During fork, the
process-level table which FlexDLL uses to get the symbol table of each
DLL is copied over but because the load order of dlopen'd DLLs is
reversed, it is possible for FlexDLL to attempt to access memory in the
DLL before it has been loaded and hence it fails with an access
violation. Because the list is reversed on each call to fork, it means
that a subsequent call to fork puts the DLLs back into the correct
order, hence "even" invocations of fork work!
An interesting side-effect is that this only occurs if the DLLs load at
their preferred base address - if they have to be rebased, then FlexDLL
works because at the time that the dependent unit is loaded out of
order, there is still in memory the "dummy" DONT_RESOLVE_DLL_REFERENCES
version of the dependency which, as it happens, will contain the correct
symbol table in the data section. For my tests, this initially appeared
to be an x86-only problem, but that was only because the two DLLs on x64
should have been rebased.
Signed-off-by: David Allsopp <david.allsopp@metastack.com>
Teach makedocbook how to handle some new things seen in the makedoc markup
since bd547490:
- struct lines appearing in the synopsis
- use of @strong{} texinfo markup
The changes in af272aca59 only works when
using gcc/g++ with -E or -save-temps, otherwise newlib's newlib.h gets
used even if -specs=nano.specs is specified. This is because the driver
only use cpp_options spec for the external cpp tool, not for the
integrated one.
This patch uses instead cpp_unique_options which is used in all cases:
it is used directly when the integrated preprocessor is used, and
indirectly by expansion of cpp_options otherwise.
At the moment when targeting bare-metal targets or systems without
definition for the locking primitives newlib, uses dummy empty macros.
This has the advantage of reduced size and faster implementation but
does not allow the application to retarget the locking routines.
Retargeting is useful for a single toolchain to support multiple systems
since then it's only at link time that you know which system you are
targeting.
This patch adds a new configure option
--enable-newlib-retargetable-locking to use dummy empty functions
instead of dummy empty macros. The default is to keep the current
behavior to not have any size or speed impact on targets not interested
in this feature. To allow for any size of lock, the _LOCK_T type is
changed into pointer to struct _lock and the _init function are tasked
with allocating the locks. The platform being targeted must provide the
static locks. A dummy implementation of the locking routines and static
lock is provided for single-threaded applications to link successfully
out of the box.
To ensure that the behavior is consistent (either no locking whatsoever
or working locking), the dummy implementation is strongly defined such
that a partial retargeting will cause a doubly defined link error.
Indeed, the linker will only pull in the file providing the dummy
implementation if it cannot find an implementation for one of the
routine or lock.
Newlib build system defines __SINGLE_THREAD__ to allow concurrency code
to be only compiled when newlib is configured for multithread. One such
example are locks which become useless in single thread mode. Although
most static locks are indeed guarded by !defined(__SINGLE_THREAD__),
some are not.
This commit adds these missing guards to __dd_hash_mutex,
__atexit_recursive_mutex, __at_quick_exit_mutex and __arc4random_mutex.
It also makes sure locking macros in lock.h are noop in single thread
mode.
Hi,
The changes in c028685518 to use
newlib-nano's include directory work for cc1 but not cc1plus. cc1plus
comes with its own cpp spec which does not have a name attached to it.
This patch uses the renaming trick on cpp_options instead of cpp, as
cpp_options is used both by cc1 and cc1plus.
Environ is defined in libgloss and libc:
- libgloss/or1k/syscalls.c
- libc/stdlib/environ.c
When linking we sometimes get errors:
or1k-elf-g++ test.o -mnewlib -mboard=or1ksim -lm -o test
/opt/shorne/software/or1k/lib/gcc/or1k-elf/5.3.0/../../../../or1k-elf/lib/libor1k.a(syscalls.o):(.data+0x0):
multiple definition of `environ'
/opt/shorne/software/or1k/lib/gcc/or1k-elf/5.3.0/../../../../or1k-elf/lib/libc.a(lib_a-environ.o):(.data+0x0):
first defined here
collect2: error: ld returned 1 exit status
This doesnt happen after the fix. Basic things build fine too.
There was an issue revealed in gdb testing where C++ virtual tables
were not getting properly initialized. This seems to be due to the
c++ global constructors moving from ctors to init_array.
This fix makes sure we call the proper method for initializing the
constructors in all places.
or1k uses reentrant calls by default, but there was no open_r defined
which caused failure in C++/C code such as:
int main() { std::cout << "test\n"; return 0; }
or
int main() {open(".", 0);}
Match glibc behaviour to expose the public bswap_* macros only with an
explicity #include <byteswap.h>; #include'ing <endian.h> should not expose
them.
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
In preparation for the patch that would allow retargeting of locking
routines, rename all lock objects to follow this pattern:
"__<name>_[recursive_]mutex".
Following locks were renamed:
__dd_hash_lock -> __dd_hash_mutex
__sfp_lock -> __sfp_recursive_mutex
__sinit_lock -> __sinit_recursive_mutex
__atexit_lock -> __atexit_recursive_mutex
_arc4random_mutex -> __arc4random_mutex
__env_lock_object -> __env_recursive_mutex
__malloc_lock_object -> __malloc_recursive_mutex
__atexit_mutex -> __at_quick_exit_mutex
__tz_lock_object -> __tz_mutex
Give more elements ids, so random ids aren't assigned to them, so anchors
are stable between builds.
Signed-off-by: Jon Turney <jon.turney@dronecode.org.uk>
Don't set SO_RCVBUF/SO_SNDBUF to fixed values, thus disabling autotuning.
Patch modeled after a patch suggestion from Daniel Havey <dhavey@gmail.com>
in https://cygwin.com/ml/cygwin-patches/2017-q1/msg00010.html:
At Windows we love what you are doing with Cygwin. However, we have
been getting reports from our hardware vendors that iperf is slow on
Windows. Iperf is of course compiled against the cygwin1.dll and we
believe we have traced the problem down to the function fdsock in
net.cc. SO_RCVBUF and SO_SNDBUF are being manually set. The comments
indicate that the idea was to increase the buffer size, but, this code
must have been written long ago because Windows has used autotuning
for a very long time now. Please do not manually set SO_RCVBUF or
SO_SNDBUF as this will limit your internet speed.
I am providing a patch, an STC and my cygcheck -svr output. Hope we
can fix this. Please let me know if I can help further.
Simple Test Case:
I have a script that pings 4 times and then iperfs for 10 seconds to
debit.k-net.fr
With patch
$ bash buffer_test.sh 178.250.209.22
usage: bash buffer_test.sh <iperf server name>
Pinging 178.250.209.22 with 32 bytes of data:
Reply from 178.250.209.22: bytes=32 time=167ms TTL=34
Reply from 178.250.209.22: bytes=32 time=173ms TTL=34
Reply from 178.250.209.22: bytes=32 time=173ms TTL=34
Reply from 178.250.209.22: bytes=32 time=169ms TTL=34
Ping statistics for 178.250.209.22:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 167ms, Maximum = 173ms, Average = 170ms
------------------------------------------------------------
Client connecting to 178.250.209.22, TCP port 5001
TCP window size: 64.0 KByte (default)
------------------------------------------------------------
[ 3] local 10.137.196.108 port 58512 connected with 178.250.209.22 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0- 1.0 sec 768 KBytes 6.29 Mbits/sec
[ 3] 1.0- 2.0 sec 9.25 MBytes 77.6 Mbits/sec
[ 3] 2.0- 3.0 sec 18.0 MBytes 151 Mbits/sec
[ 3] 3.0- 4.0 sec 18.0 MBytes 151 Mbits/sec
[ 3] 4.0- 5.0 sec 18.0 MBytes 151 Mbits/sec
[ 3] 5.0- 6.0 sec 18.0 MBytes 151 Mbits/sec
[ 3] 6.0- 7.0 sec 18.0 MBytes 151 Mbits/sec
[ 3] 7.0- 8.0 sec 18.0 MBytes 151 Mbits/sec
[ 3] 8.0- 9.0 sec 18.0 MBytes 151 Mbits/sec
[ 3] 9.0-10.0 sec 18.0 MBytes 151 Mbits/sec
[ 3] 0.0-10.0 sec 154 MBytes 129 Mbits/sec
Without patch:
dahavey@DMH-DESKTOP ~
$ bash buffer_test.sh 178.250.209.22
Pinging 178.250.209.22 with 32 bytes of data:
Reply from 178.250.209.22: bytes=32 time=168ms TTL=34
Reply from 178.250.209.22: bytes=32 time=167ms TTL=34
Reply from 178.250.209.22: bytes=32 time=170ms TTL=34
Reply from 178.250.209.22: bytes=32 time=169ms TTL=34
Ping statistics for 178.250.209.22:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 167ms, Maximum = 170ms, Average = 168ms
------------------------------------------------------------
Client connecting to 178.250.209.22, TCP port 5001
TCP window size: 208 KByte (default)
------------------------------------------------------------
[ 3] local 10.137.196.108 port 58443 connected with 178.250.209.22 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0- 1.0 sec 512 KBytes 4.19 Mbits/sec
[ 3] 1.0- 2.0 sec 1.50 MBytes 12.6 Mbits/sec
[ 3] 2.0- 3.0 sec 1.50 MBytes 12.6 Mbits/sec
[ 3] 3.0- 4.0 sec 1.50 MBytes 12.6 Mbits/sec
[ 3] 4.0- 5.0 sec 1.50 MBytes 12.6 Mbits/sec
[ 3] 5.0- 6.0 sec 1.50 MBytes 12.6 Mbits/sec
[ 3] 6.0- 7.0 sec 1.50 MBytes 12.6 Mbits/sec
[ 3] 7.0- 8.0 sec 1.50 MBytes 12.6 Mbits/sec
[ 3] 8.0- 9.0 sec 1.50 MBytes 12.6 Mbits/sec
[ 3] 9.0-10.0 sec 1.50 MBytes 12.6 Mbits/sec
[ 3] 0.0-10.1 sec 14.1 MBytes 11.7 Mbits/sec
The output shows that the RTT from my machine to the iperf server is
similar in both cases (about 170ms) however with the patch the
throughput averages 129 Mbps while without the patch the throughput
only averages 11.7 Mbps. If we calculate the maximum throughput using
Bandwidth = Queue/RTT we get (212992 * 8)/0.170 = 10.0231 Mbps. This
is just about what iperf is showing us without the patch since the
buffer size is set to 212992 I believe that the buffer size is
limiting the throughput. With the patch we have no buffer limitation
(autotuning) and can develop the full potential bandwidth on the link.
If you want to duplicate the STC you will have to find an iperf server
(I found an extreme case) that has a large enough RTT distance from
you and try a few times. I get varying results depending on Internet
traffic but without the patch never exceed the limit caused by the
buffering.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
The form data sent to the server should be application/x-www-form-urlencoded
This replaces spaces with '+' before being RFC 1738 encoded, so a literal
'+' must be %-encoded also.
See https://cygwin.com/ml/cygwin/2014-01/msg00287.html et seq.
Signed-off-by: Jon Turney <jon.turney@dronecode.org.uk>
Corinna Vinschen
Yaakov Selkowitz
Sebastian Huber
Joel Sherrill
Koichi Murase
Jon Turney
David Allsopp
Michael Haubenwallner
Thomas Preud'homme
Kenneth Nellis
Thomas Preudhomme
Stafford Horne
Olof Kindgren
Freddie Chopin