Replace the simple byte-wise compare in the misaligned case with a
dword compare with page boundary checks in place. For simplicity I've
chosen a 4K page boundary so that we don't have to query the actual
page size on the system.
This results in up to 3x improvement in performance in the unaligned
case on falkor and about 2.5x improvement on mustang as measured using
bench-strcmp in glibc.
This improved memcmp provides a fast path for compares up to 16 bytes
and then compares 16 bytes at a time, thus optimizing loads from both
sources. The glibc memcmp microbenchmark retains performance (with an
error of ~1ns) for smaller compare sizes and reduces up to 31% of
execution time for compares up to 4K on the APM Mustang. On Qualcomm
Falkor this improves to almost 48%, i.e. it is almost 2x improvement
for sizes of 2K and above.
The mutually misaligned inputs on aarch64 are compared with a simple
byte copy, which is not very efficient. Enhance the comparison
similar to strcmp by loading a double-word at a time. The peak
performance improvement (i.e. 4k maxlen comparisons) due to this on
the strncmp microbenchmark in glibc is as follows:
falkor: 3.5x (up to 72% time reduction)
cortex-a73: 3.5x (up to 71% time reduction)
cortex-a53: 3.5x (up to 71% time reduction)
All mutually misaligned inputs from 16 bytes maxlen onwards show
upwards of 15% improvement and there is no measurable effect on the
performance of aligned/mutually aligned inputs.
Bug in current ARM64 WOW64: GetNativeSystemInfo returns
PROCESSOR_ARCHITECTURE_INTEL rather than PROCESSOR_ARCHITECTURE_ARM64.
Provide for this.
Make code better readable.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
The current SYS_EXIT has a bug that when making the call it always uses
the v2 calling convention. This is undefined behavior according to the
semihosting specification:
https://developer.arm.com/docs/100863/latest/semihosting-operations/sys_exit-0x18
This patch fixes it by making sure v1 passes the argument directly in the register instead
of in a block. And for v2 it does the same if the v2 extension isn't supported.
The sequence generated now is
12424: ebfffecd bl 11f60 <_has_ext_exit_extended>
12428: e3500000 cmp r0, #0
1242c: 11a0500d movne r5, sp
12430: 059d5000 ldreq r5, [sp]
12434: e1a00004 mov r0, r4
12438: e1a01005 mov r1, r5
1243c: ef00f000 svc 0x0000f000
Signed-off-by: Tamar Christina <tamar.christina@arm.com>
PREFER_FLOAT_COMPARISON setting was not correct as it could raise
spurious exceptions. Fixing it is easy: just use ISLESS(x, y) instead
of abstop12(x) < abstop12(y) with appropriate non-signaling definition
for ISLESS. However it seems this setting is not very useful (there is
only minor performance difference on various architectures), so remove
this option for now.
Guard the entire operation with the FastPebLock critical section
used by RtlSetCurrentDirectory_U as well, thus eliminating the
race between concurrent chdir/fchdir/SetCurrentDirectory calls.
Streamline comment explaining the fallback method.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Update to the latest versions of config.sub (2018-07-03) and
config.guess (2018-06-26).
Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
* fhandler_socket_local.cc (get_inet_addr_local): Change type from
'static int' to 'int' to be callable from syslog.cc.
* syslog.cc (connect_syslogd): Use get_inet_addr_local() instead of
getsockname() to retrieve name information of the syslogd socket.
The !HAVE_FAST_FMA code path split r = z/c - 1 into r = rhi + rlo such
that when z = 1-tiny and c = 1 then rlo and rhi could have much larger
magnitude than r which later caused large rounding errors.
So do a nearest rounding instead of truncation at the split.
In newlib with default settings this was observable on some arm targets
that enable the new math code but has no fma.
The roundtoint and converttoint internal functions are only called with small
values, so 32 bit result is enough for converttoint and it is a signed int
conversion so the natural return type is int32_t.
The original idea was to help the compiler keeping the result in uint64_t,
then it's clear that no sign extension is needed and there is no accidental
undefined or implementation defined signed int arithmetics.
But it turns out gcc does a good job with inlining so changing the type has
no overhead and the semantics of the conversion is less surprising this way.
Since we want to allow the asuint64 (x + 0x1.8p52) style conversion, the top
bits were never usable and the existing code ensures that only the bottom
32 bits of the conversion result are used.
In newlib with default settings only aarch64 is affected and there is no
significant code generation change with gcc after the patch.
Synchronize code style and comments with Arm Optimized Routines, there
are no code changes in this patch. This ensures different projects using
the same code have consistent code style so bug fix patches can be applied
more easily.
This fix is for some platforms which do not have writev().
*perror.c: Use _write_r() instead of writev().
*psignal.c: Use write() insetad of writev().
Revise commit: d4f4e7ae1b
Our local ntsecapi.h wrapper corrects a bug in the definition of
SystemFunction036 which otherwise leads to crashes on 32 bit when
using RtlGenRandom. The fhandler_socket_local.cc file accidentally
included the incorrect w32api version of that file, rather than the
local wrapper. Fix it.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
This patch turns dependencies of non-bootstrap targets on bootstrap
targets for bootstrap builds into dependencies on stage_last. This
arrangement gets stage1-bubble to run from stage_last if we haven't
started a bootstrap yet, and to use the current stage otherwise. This
was already the case of target libs, just not of non-bootstrapped host
modules.
In order to retain preexisting dependencies in non-bootstrap builds,
or in gcc-less builds, this introduces support for @unless/@endunless
pairs in Makefile.in.
There is a remaining possibility of problem if activating, in a tree
configured for bootstrap, a parallel build of two or more modules, at
least one bootstrapped and one not. In this case, make might decide
to build stage_current and stage_last in parallel, the latter will
start a submake to build stage1 while the initial make, having
satisfied stage_current, proceeds to build the bootstrapped module in
non-bootstrapped configurations. The two builds will overlap and will
likely conflict. This situation does NOT arise in normal settings,
however: a post-bootstrap build of all-host all-target will indeed
activate such targets concurrently, but only after building all
bootstrapped modules successfully, and it will have both stage_last
and stage_current targets already satisfied, so the potential race
between builds will not arise.
Another remaining problem, that is slightly expanded with this patch,
is that of an interrupted build in a tree configured for bootstrap,
continued with a non-bootstrapped target. Target modules that were
not bootstrapped would already fail to complete the current stage when
activated explicitly in the command line for a retry; host modules,
however, would attempt to build their bootstrapped dependencies, which
is what led to the problem of concurrent builds addressed with this
patch. An interrupted or failed build might still recover correctly,
if the non-bootstrapped target is activated in both builds, because
then make will remove stage_last when its build command is
interrupted, so that it will attempt to recreate it with stage1-bubble
in the second try. A bootstrap build, however, will not be attempting
to build stage_last, so the file will remain and the retry won't go
through stage1-bubble. We have lived with that for target modules, so
we can probably live with that for host modules too.
Another undesirable consequence of this change is that non-boostrapped
host modules, in a tree configured for bootstrap, when activated as
make all-<module>, will build all of stage1 instead of only the
module's usual dependencies. This is intentional and necessary to fix
the parallel-build problem. If it's not desirable, disabling the
unnecessary bootstrap configuration will suffice to restore the
original set of dependencies.
for ChangeLog
* configure.ac: Introduce support for @unless/@endunless.
* Makefile.tpl (dep-kind): Rewrite with cond; return
postbootstrap in some cases.
(make-postboot-dep, postboot-targets): New.
(dependencies): Do not output postbootstrap dependencies at
first. Output non-target ones changed for configure to depend
on stage_last @if gcc-bootstrap, and the original deps @unless
gcc-bootstrap.
* configure.in, Makefile.in: Rebuilt.
Certain tape drives (known example: QUANTUM_ULTRIUM-HH6) return
the non-standard ERROR_NOT_READY rather than ERROR_NO_MEDIA_IN_DRIVE
if no media is present. ERROR_NOT_READY is not documented as valid
return code from GetTapeStatus. Without handling this error code
Cygwin's tape code can't report an offline state to user space.
Fix this by converting ERROR_NOT_READY to ERROR_NO_MEDIA_IN_DRIVE
where appropriate.
Add a debug_printf to mtinfo_drive::get_status to allow requesting
user info without having to rebuild the DLL.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
pthread_timedjoin_np returns ETIMEDOUT if a thread is still running,
not EBUSY as pthread_tryjoin_np.
Also, clean up initializing timeout in pthread_tryjoin_np.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
- Move pthread_join to thread.cc to have all `join' calls in
the same file (pthread_timedjoin_np needs pthread_convert_abstime
which is static inline in thread.cc)
- Bump API version
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
The new implementation is provided under !__OBSOLETE_MATH, it uses
ISO C99 code. With default settings the worst case error in nearest
rounding mode is 0.54 ULP with inlined fma and fma contraction. It uses
a 4 KB lookup table in addition to the table in exp_data.c, on aarch64
.text+.rodata size of libm.a is increased by 2295 bytes.
Improvements on Cortex-A72:
latency: 3.3x
thruput: 4.9x
The new implementation is provided under !__OBSOLETE_MATH, it uses
ISO C99 code. With default settings the worst case error in nearest
rounding mode is 0.547 ULP with inlined fma and fma contraction. It uses
a 1 KB lookup table, on aarch64 .text+.rodata size of libm.a is increased
by 1584 bytes.
Note that the math.h header defines log2(x) to be log(x)/Ln2, this is
not changed, so the new code is only used if that macro is suppressed.
Improvements on Cortex-A72:
latency: 2.0x
thruput: 2.2x
The new implementations are provided under !__OBSOLETE_MATH, it uses
ISO C99 code. With default settings the worst case error in nearest
rounding mode is 0.519 ULP with inlined fma and fma contraction. It uses
a 2 KB lookup table, on aarch64 .text+.rodata size of libm.a is increased
by 1703 bytes. The w_log.c wrapper is disabled since error handling is
inline in the new code.
New __HAVE_FAST_FMA and __HAVE_FAST_FMA_DEFAULT feature macros were
added to enable selecting between the code path that uses fma and the
one that does not. Targets supposed to set __HAVE_FAST_FMA_DEFAULT
if they have single instruction fma and the compiler can actually
inline it (gcc has __FP_FAST_FMA macro but that does not guarantee
inlining with -fno-builtin-fma).
Improvements on Cortex-A72:
latency: 1.9x
thruput: 2.3x
The new implementations are provided under !__OBSOLETE_MATH, they use
ISO C99 code. There are several settings, with the default one the
worst case error in nearest rounding mode is 0.509 ULP for exp and
0.507 ULP for exp2 when a multiply and add is contracted into an fma.
They use a shared 2 KB lookup table, on aarch64 .text+.rodata size
of libm.a is increased by 1868 bytes. The w_*.c wrappers are disabled
for the new code as it takes care of error handling inline.
The old exp2(x) code used to be just pow(2,x) so the speedup there
is more significant.
The file name has no special prefix to avoid any name collision with
existing files.
Improvements on Cortex-A72:
exp latency: 3.2x
exp thruput: 4.1x
exp2 latency: 7.8x
exp2 thruput: 18.8x
This change is equivalent to the commit
c65db17340
and only affects code that is from the Arm optimized-routines project.
It does not affect the observable behaviour, but the code generation
can be different on 64bit targets. The intention is to make the
portable semantics of the code obvious by using a fixed size type.
* (mkcategories): Fix a bug that outputs incorrect Unicode category
table for code point ranges.
* (categories.t): Rebuild it using the bug-fixed mkcategories.
This fixes the problem reported in the following post.
https://cygwin.com/ml/cygwin/2018-06/msg00248.html
* fhandler.h (class fhandler_socket_inet): Add variable bool oobinline.
* fhandler_socket_inet.cc (fhandler_socket_inet::fhandler_socket_inet):
Initialize variable oobinline.
(fhandler_socket_inet::recv_internal): Make the handling of OOB data
as consistent with POSIX as possible. Add simulation of inline mode
for OOB data as a workaround for broken winsock behavior.
(fhandler_socket_inet::setsockopt): Ditto.
(fhandler_socket_inet::getsockopt): Ditto.
(fhandler_socket_wsock::ioctl): Fix return value of SIOCATMARK command.
The return value of SIOCATMARK of winsock is almost opposite to
expectation.
* fhandler_socket_local.cc (fhandler_socket_local::recv_internal):
Remove the handling of OOB data from AF_LOCAL domain socket. Operation
related to OOB data will result in an error like Linux does.
(fhandler_socket_local::sendto): Ditto.
(fhandler_socket_local::sendmsg): Ditto.
This fixes the issue reported in following post.
https://cygwin.com/ml/cygwin/2018-06/msg00143.html
Here is the correct patch with both filenames and int cast fixed:
This patch is a complete rewrite of sinf, cosf and sincosf. The new version
is significantly faster, as well as simple and accurate.
The worst-case ULP is 0.56072, maximum relative error is 0.5303p-23 over all
4 billion inputs. In non-nearest rounding modes the error is 1ULP.
The algorithm uses 3 main cases: small inputs which don't need argument
reduction, small inputs which need a simple range reduction and large inputs
requiring complex range reduction. The code uses approximate integer
comparisons to quickly decide between these cases - on some targets this may
be slow, so this can be configured to use floating point comparisons.
The small range reducer uses a single reduction step to handle values up to
120.0. It is fastest on targets which support inlined round instructions.
The large range reducer uses integer arithmetic for simplicity. It does a
32x96 bit multiply to compute a 64-bit modulo result. This is more than
accurate enough to handle the worst-case cancellation for values close to
an integer multiple of PI/4. It could be further optimized, however it is
already much faster than necessary.
Simple benchmark showing speedup factor on AArch64 for various ranges:
range 0.7853982 sinf 1.7 cosf 2.2 sincosf 2.8
range 1.570796 sinf 1.9 cosf 1.9 sincosf 2.7
range 3.141593 sinf 2.0 cosf 2.0 sincosf 3.5
range 6.283185 sinf 2.3 cosf 2.3 sincosf 4.2
range 125.6637 sinf 2.9 cosf 3.0 sincosf 5.1
range 1.1259e15 sinf 26.8 cosf 26.8 sincosf 45.2
ChangeLog:
2018-05-18 Wilco Dijkstra <wdijkstr@arm.com>
* newlib/libm/common/Makefile.in: Regenerated.
* newlib/libm/common/Makefile.am: Add sinf.c, cosf.c, sincosf.c
sincosf.h, sincosf_data.c. Add -fbuiltin -fno-math-errno to CFLAGS.
* newlib/libm/common/math_config.h: Add HAVE_FAST_ROUND, HAVE_FAST_LROUND,
roundtoint, converttoint, force_eval_float, force_eval_double, eval_as_float,
eval_as_double, likely, unlikely.
* newlib/libm/common/cosf.c: New file.
* newlib/libm/common/sinf.c: Likewise.
* newlib/libm/common/sincosf.h: Likewise.
* newlib/libm/common/sincosf.c: Likewise.
* newlib/libm/common/sincosf_data.c: Likewise.
* newlib/libm/math/sf_cos.c: Add #if to build conditionally.
* newlib/libm/math/sf_sin.c: Likewise.
* newlib/libm/math/wf_sincos.c: Likewise.
--
This patch is a complete rewrite of sinf, cosf and sincosf. The new version
is significantly faster, as well as simple and accurate.
The worst-case ULP is 0.56072, maximum relative error is 0.5303p-23 over all
4 billion inputs. In non-nearest rounding modes the error is 1ULP.
The algorithm uses 3 main cases: small inputs which don't need argument
reduction, small inputs which need a simple range reduction and large inputs
requiring complex range reduction. The code uses approximate integer
comparisons to quickly decide between these cases - on some targets this may
be slow, so this can be configured to use floating point comparisons.
The small range reducer uses a single reduction step to handle values up to
120.0. It is fastest on targets which support inlined round instructions.
The large range reducer uses integer arithmetic for simplicity. It does a
32x96 bit multiply to compute a 64-bit modulo result. This is more than
accurate enough to handle the worst-case cancellation for values close to
an integer multiple of PI/4. It could be further optimized, however it is
already much faster than necessary.
Simple benchmark showing speedup factor on AArch64 for various ranges:
range 0.7853982 sinf 1.7 cosf 2.2 sincosf 2.8
range 1.570796 sinf 1.9 cosf 1.9 sincosf 2.7
range 3.141593 sinf 2.0 cosf 2.0 sincosf 3.5
range 6.283185 sinf 2.3 cosf 2.3 sincosf 4.2
range 125.6637 sinf 2.9 cosf 3.0 sincosf 5.1
range 1.1259e15 sinf 26.8 cosf 26.8 sincosf 45.2
ChangeLog:
2018-06-18 Wilco Dijkstra <wdijkstr@arm.com>
* newlib/libm/common/Makefile.in: Regenerated.
* newlib/libm/common/Makefile.am: Add sinf.c, cosf.c, sincosf.c
sincosf.h, sincosf_data.c. Add -fbuiltin -fno-math-errno to CFLAGS.
* newlib/libm/common/math_config.h: Add HAVE_FAST_ROUND, HAVE_FAST_LROUND,
roundtoint, converttoint, force_eval_float, force_eval_double, eval_as_float,
eval_as_double, likely, unlikely.
* newlib/libm/common/cosf.c: New file.
* newlib/libm/common/sinf.c: Likewise.
* newlib/libm/common/sincosf.h: Likewise.
* newlib/libm/common/sincosf.c: Likewise.
* newlib/libm/common/sincosf_data.c: Likewise.
* newlib/libm/math/sf_cos.c: Add #if to build conditionally.
* newlib/libm/math/sf_sin.c: Likewise.
* newlib/libm/math/wf_sincos.c: Likewise.
--
Previously, "test 1 2 3 -a -b -c" was permuted to "test -a -b -c 1 2 3",
but "test 1 2 3 -abc" was left as "test 1 2 3 -abc".
Signed-off-by: Thomas Kindler <mail+newlib@t-kindler.de>
Siddhesh Poyarekar
Corinna Vinschen
Tamar Christina
Szabolcs Nagy
Sebastian Huber
Takashi Yano
Alexandre Oliva
Jon Beniston
Wilco Dijkstra
Thomas Kindler
Ken Brown