Newlib build system defines __SINGLE_THREAD__ to allow concurrency code
to be only compiled when newlib is configured for multithread. One such
example are locks which become useless in single thread mode. Although
most static locks are indeed guarded by !defined(__SINGLE_THREAD__),
some are not.
This commit adds these missing guards to __dd_hash_mutex,
__atexit_recursive_mutex, __at_quick_exit_mutex and __arc4random_mutex.
It also makes sure locking macros in lock.h are noop in single thread
mode.
In preparation for the patch that would allow retargeting of locking
routines, rename all lock objects to follow this pattern:
"__<name>_[recursive_]mutex".
Following locks were renamed:
__dd_hash_lock -> __dd_hash_mutex
__sfp_lock -> __sfp_recursive_mutex
__sinit_lock -> __sinit_recursive_mutex
__atexit_lock -> __atexit_recursive_mutex
_arc4random_mutex -> __arc4random_mutex
__env_lock_object -> __env_recursive_mutex
__malloc_lock_object -> __malloc_recursive_mutex
__atexit_mutex -> __at_quick_exit_mutex
__tz_lock_object -> __tz_mutex
Hi,
With the patch to allow newlib's locking routine to be retargeted currently
under discussion, we need to start thinking of locks as part of newlib's ABI
since newlib depends on specific names being provided by the OS. This patch
renames 2 locks so that they follow the same naming convention as other locks.
It needs to be applied before the retargeting patch, while locks are still an
internal consideration.
Newlib builds successfully with this change.
Ok for master branch?
Best regards,
Thomas
This patch adds further comments to nano-mallocr.c, to more comprehensively
explain how padding works in the malloc_chunk structure.
It was originally discussed in the following thread:
https://sourceware.org/ml/newlib/2017/msg00031.html
2017-01-13 Joe Seymour <joe.s@somniumtech.com>
newlib/
* libc/stdlib/nano-mallocr.c (malloc_chunk, get_chunk_from_ptr)
(nano_malloc): Add comments.
As described in nano-mallocr.c, chunks of heap are represented in memory
as a size (of type long), followed by some optional padding containing a
negative offset to size, followed by the data area.
get_chunk_from_ptr is responsible for taking a pointer to the data area
(as returned by malloc) and finding the start of the chunk. It does this
by assuming there is no padding and trying to read the size, if the size
is negative then it uses that as an offset to find the true size.
Crucially, it reads the padding area as a long.
nano_malloc is responsible for populating the optional padding area. It
does so by casting a pointer to an (int *) and writing the negative
offset into it.
This means that padding is being written as an int but read as a long.
On msp430 an int is 2 bytes, while a long is 4 bytes. This means that 2
bytes are written to the padding, but 4 bytes are read from it: it has
only been partially initialised.
nano_malloc is the default malloc implementation for msp430.
This patch changes the cast from (int *) to (long *). The change to
nano_malloc has has been observed to fix a TI Energia project that
had been malfunctioning because malloc was returning invalid addresses.
The change to nano_memalign is based entirely on code inspection.
I've built and tested as follows:
Configured (gcc+newlib) with: --target=msp430-elf --enable-languages=c
gcc testsuite variations:
msp430-sim/-mcpu=msp430
msp430-sim/-mcpu=msp430x
msp430-sim/-mcpu=msp430x/-mlarge/-mdata-region=either/-mcode-region=either
msp430-sim/-mhwmult=none
msp430-sim/-mhwmult=f5series
My testing has shown no regressions, however I don't know if the gcc
testsuite provides sufficient coverage for this patch?
I don't have write access, so if this patch is acceptable after review,
I would appreciate it if someone would commit it for me.
Thanks,
2017-01-XX Joe Seymour <joe.s@somniumtech.com>
newlib/
* libc/stdlib/nano-mallocr.c (nano_malloc): Fix incorrect cast.
(nano_memalign): Likewise.
* libc/stdlib/strtod.c (strtof_l): Set errno to ERANGE when double to
float conversion results in infinity.
(strtof): Likewise.
* libc/stdlib/wcstod.c (wcstof_l): Likewise.
(wcstof): Likewise.
make pdf on arm-none-eabi targets fails to build after the reorganization in
baf0c9fcb5 to fold is*_l documentation in their
is* counterpart. This is due two issues:
1) newlib/libc/ctype/ctype.tex still including the def file for the long versions
2) missing angle brackets in .c files for some of is*_l functions
This patch fixes the issues and allows make pdf to succeeds.
- Remove charset parameter from low level __foo_wctomb/__foo_mbtowc calls.
- Instead, create array of function for ISO and Windows codepages to point
to function which does not require to evaluate the charset string on
each call. Create matching helper functions. I.e., __iso_wctomb,
__iso_mbtowc, __cp_wctomb and __cp_mbtowc are functions returning the
right function pointer now.
- Create __WCTOMB/__MBTOWC macros utilizing per-reent locale and replace
calls to __wctomb/__mbtowc with calls to __WCTOMB/__MBTOWC.
- Drop global __wctomb/__mbtowc vars.
- Utilize aforementioned changes in Cygwin to get rid of charset in other,
calling functions and simplify the code.
- In Cygwin restrict global cygheap locale info to the job performed
by internal_setlocale. Use UTF-8 instead of ASCII on the fly in
internal conversion functions.
- In Cygwin dll_entry, make sure to initialize a TLS area with a NULL
_REENT->_locale pointer. Add comment to explain why.
Signed-off by: Corinna Vinschen <corinna@vinschen.de>
Newlib defines defaults for internal types via <sys/_types.h> and uses
<machine/_types.h> to let targets define their own type if necessary.
Previously for example
#ifndef __dev_t_defined
typedef short __dev_t;
#endif
However, the __*_t_defined pattern conflicts with the glibc type guard
pattern for user types, e.g. dev_t in this example. Introduce a
__machine_*_t_defined pattern for internal types (defined by
<machine/_types.h>, used by <sys/_types.h>). For example
#ifndef __machine_dev_t_defined
typedef short __dev_t;
#endif
Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
Intel MCU System V ABI are incompartible with i386 System V ABI:
o Minimum instruction set is Intel Pentium ISA minus x87 instructions
o No x87 or vector registers
o First three args are passed in %eax, %edx and %ecx
o Full specification available here:
https://github.com/hjl-tools/x86-psABI/wiki/iamcu-psABI-0.7.pdf
newlib/
* configure.host: Add new ix86-*-elfiamcu target
newlib/libc/include/
* setjmp.h: Change _JBLEN for Intel MCU target
newlib/libc/machine/i386/
* memchr.S: (memchr) Target-specific size-optimized version
* memcmp.S: (memcmp) Likewise
* memcpy.S: (memcpy) Likewise
* memmove.S: (memmove) Likewise
* memset.S: (memset) Likewise
* setjmp.S: (setjmp) Likewise
* strchr.S: (strchr) Likewise
* strlen.S: (strlen) Likewise
newlib/libc/stdlib/
* srtold.c: (__flt_rounds) Disable for Intel MCU
Prototypes also added for initstate() and setstate() but they
were not implemented in the shared newlib code.
* newlib/libc/include/cygwin/stdlib.h: Prototypes added.
* winsup/cygwin/include/cygwin/stdlib.h: Prototypes removed.
* newlib/libc/stdlib/random.c: New file.
* newlib/libc/machine/epiphany/machine/stdlib.h: Removed
* newlib/libc/stdlib/Makefile.am: Added random.c.
* newlib/libc/stdlib/stdlib.tex: Added random.def.
* newlib/libc/stdlib/Makefile.in: Regenerated.
Infinity returned from strtold is recognized as NaN by GCC builtin
functions. The reason is that ULtox is missing to set a bit.
* libc/stdlib/strtorx.c (ULtox): Set high bit in second word
to create valid Infinity value.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
newlib:
* libc/stdlib/arc4random.h: Remove Cygwin-specific locking code.
Conditionalize arc4 locking. Check for _ARC4_LOCK_INIT being
undefined to fall back to default implementation.
cygwin:
* include/machine/_arc4random.h: New file.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
According to the OpenBSD man page, "A Replacement Call for Random". It
offers high quality random numbers derived from input data obtained by
the OpenBSD specific getentropy() system call which is declared in
<unistd.h> and must be implemented for each Newlib port externally. The
arc4random() functions are used for example in LibreSSL and OpenSSH.
Cygwin provides currently its own implementation of the arc4random
family. Maybe it makes sense to use this getentropy() implementation:
http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/lib/libcrypto/crypto/getentropy_win.c?rev=1.4&content-type=text/x-cvsweb-markup
* libc/include/stdlib.h (arc4random): Declare if __BSD_VISIBLE.
(arc4random_buf): Likewise.
(arc4random_uniform): Likewise.
* libc/include/sys/unistd.h (getentropy): Likewise.
* libc/include/machine/_arc4random.h: New file.
* libc/stdlib/arc4random.c: Likewise.
* libc/stdlib/arc4random.h: Likewise.
* libc/stdlib/arc4random_uniform.c: Likewise.
* libc/stdlib/chacha_private.h: Likewise.
* libc/sys/rtems/include/machine/_arc4random.h: Likewise.
* libc/stdlib/Makefile.am (EXTENDED_SOURCES): Add arc4random.c
and arc4random_uniform.c.
* libc/stdlib/Makefile.in: Regenerate.
Throughout, simplify the C99/C11 conditionals, and replace
__STRICT_ANSI__ with the proper internal POSIX macros. The _*_r
reentrant functions need not be guarded (and most haven't been) because
such names in the global scope are reserved to the implementation.
atoff is unique to newlib.
dtoa is not actually exported (_dtoa_r is used internally), is
nonstandard, and the declaration conflicts with the code included in
MySQL, NSPR, and SpiderMonkey.
mktemp was removed in POSIX.1-2001.
The qsort_r declarations are reordered so that the GNU version retains
precedence.
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
Non-reentrant system calls version implies both MISSING_SYSCALL_NAMES
and REENTRANT_SYSCALL_PROVIDED macros to be defined.
Being coupled with --enable-newlib-nano-malloc knob it breaks the build:
bash-4.3$ ../newlib-2.3.0.20160104/configure CC_FOR_TARGET=gcc
AR_FOR_TARGET=ar RANLIB_FOR_TARGET=ranlib CFLAGS_FOR_TARGET="-m32
-DMISSING_SYSCALL_NAMES -DREENTRANT_SYSCALLS_PROVIDED"
--target=i386-elf --enable-newlib-nano-malloc && make
<...omitted output...>
../../../../../../newlib-2.3.0.20160104/newlib/libc/stdlib/nano-mallocr.c:
In function ‘_mallinfo_r’:
../../../../../../newlib-2.3.0.20160104/newlib/libc/stdlib/nano-mallocr.c:489:35:
error: macro "_sbrk_r" requires 2 arguments, but only 1 given
sbrk_now = _sbrk_r(RCALL 0);
^
../../../../../../newlib-2.3.0.20160104/newlib/libc/stdlib/nano-mallocr.c:489:20:
error: ‘_sbrk_r’ undeclared (first use in this function)
sbrk_now = _sbrk_r(RCALL 0);
^
../../../../../../newlib-2.3.0.20160104/newlib/libc/stdlib/nano-mallocr.c:489:20:
note: each undeclared identifier is reported only once for each
function it appears in
Makefile:1512: recipe for target 'lib_a-nano-mallinfor.o' failed
make[8]: *** [lib_a-nano-mallinfor.o] Error 1
In case of non-reentrant system calls _sbrk_r became a macro with TWO
args (defined in reent.h):
#define _sbrk_r(__reent, __incr) sbrk(__incr)
But in our case only one argument is present. (RCALL 0) is considered
as a single argument despite RCALL itself is a macro:)
So intermediate one-arg macro will be enough to expand args before
final _sbrk_r expansion:
#define _SBRK_R(X) _sbrk_r(X)
Here is a patch:
If small reent is enabled (_REENT_SMALL is defined) then malloc() was
used in __register_exitproc() even if user requested it to be disabled
(_ATEXIT_DYNAMIC_ALLOC is defined). With this fix, function fails when
_ATEXIT_DYNAMIC_ALLOC is defined and whole static storage is already
used.
2015-12-21 Freddie Chopin <freddie.chopin@gmail.com>
* libc/stdlib/__atexit.c (__register_exitproc): Fix for
_ATEXIT_DYNAMIC_ALLOC.
2015-12-21 Freddie Chopin <freddie.chopin@gmail.com>
* libc/stdlib/on_exit_args.{c,h}: New files.
* libc/stdlib/Makefile.am: Add new source file.
* libc/stdlib/Makefile.in: Regenerate.
* libc/stdlib/__atexit.c (__register_exitproc): Initialize
_on_exit_args_ptr field of _GLOBAL_ATEXIT on first run.
* libc/stdlib/on_exit.c: Force linking of static instance of
_on_exit_args.
* libc/stdlib/cxa_atexit.c: Likewise.
In some code paths the __atexit_lock held by this function was not
released when returning with an error.
* libc/stdlib/__atexit.c (__register_exitproc): Always release
lock before return.
aligned_alloc() is implemented in terms of posix_memalign() which is
only declared in <stdlib.h> but not defined in Newlib in general. At
least Linux and RTEMS implement this function.
newlib/ChangeLog
2015-10-14 Sebastian Huber <sebastian.huber@embedded-brains.de>
* libc/stdlib/Makefile.am (GENERAL_SOURCES): Add
alloc_aligned.c.c.
* libc/stdlib/Makefile.in: Regenerate.
* libc/stdlib/aligned_alloc.c: New.
Import some <stdlib.h> function declarations from latest FreeBSD and
implement them. I am not sure if we should call the global reent
cleanup in quick_exit() similar to exit().
newlib/ChangeLog
2015-10-14 Sebastian Huber <sebastian.huber@embedded-brains.de>
* libc/include/stdlib.h (at_quick_exit): Declare.
(quick_exit): Likewise.
* libc/stdlib/Makefile.am (GENERAL_SOURCES): Add
quick_exit.c.
* libc/stdlib/Makefile.in: Regenerate.
* libc/stdlib/quick_exit.c: New.
I think these are accidental omissions, as these source files are listed to be
chewed by makedoc, but the result is not included by any texinfo source file.
Future work: Nothing in libc/reent/ which is processed by makedoc is included by
reent.tex
2015-06-23 Jon Turney <jon.turney@dronecode.org.uk>
* libc/stdlib/stdlib.tex: Include itoa and utoa, and add to menu.
* libc/string/strings.tex: Include memrchr and rawmemchr, and add
to menu.
* libm/math/math.tex: Include exp10 and pow10, and add to menu.
* libm/common/s_exp10.c: Improve one-line description.
* libm/common/s_exp10.c: Ditto.
Signed-off-by: Jon TURNEY <jon.turney@dronecode.org.uk>
Don't chew files which contain no documentation markup.
Neither of the alternatives for MALLOCR (mallocr.c or nano-mallocr.c) contain
any documentation markup.
2015-06-23 Jon Turney <jon.turney@dronecode.org.uk>
* libc/stdlib/Makefile.am (CHEWOUT_FILES): Remove $(MALLOCR).def.
* libc/stdio/Makefile.am (CHEWOUT_FILES): Remove getwc.def and
putwc.def.
Signed-off-by: Jon TURNEY <jon.turney@dronecode.org.uk>
Fix typo-ed function names in efgcvt.c documentation, neither gvcvt nor gdvtf
exists.
2015-06-23 Jon Turney <jon.turney@dronecode.org.uk>
* libc/stdlib/efgcvt.c: Fix typo-ed function names in
documentation.
* libc/stdlib/stdlib.tex: Fix function name in menu to match.
Signed-off-by: Jon TURNEY <jon.turney@dronecode.org.uk>
2015-06-23 Jon Turney <jon.turney@dronecode.org.uk>
* libc/stdlib/llabs.c: Mark up parameter name in ANSI_SYNOPSIS.
* libc/time/tzset.c: Add and mark up parameter in SYNOPSIS.
* libm/common/s_nan.c: Ditto.
Signed-off-by: Jon TURNEY <jon.turney@dronecode.org.uk>
"const char *".
* newlib/libc/stdlib/getopt.c (getopt_internal): Use fputs()/fputc()
instead of fprintf() to save code space. Fix signed/unsigned
comparison.
* libc/stdlib/nano-mallocr.c (RONECALL): New parameter macro.
(sbrk_start): New variable.
(current_mallinfo, free_list): Forward declaration of data.
(chunk): Define size as long type.
(mallinfo): Synchronize with latest malloc.h.
(nano_*): Forward declaration of functions.
(get_chunk_from_ptr): Defined to static inline.
(nano_mallinfo): Implement this function.
(nano_malloc_stats): Implement this function.
Lite exit support.
* README: Add information about lite-exit.
* acconfig.h (_LITE_EXIT): New macro.
* configure.in (enable-lite-exit): New option.
(_LITE_EXIT): Define new macro.
* configure: Regenerated.
* newlib.hin (_LITE_EXIT): New macro.
* libc/stdlib/__atexit.c [_LITE_EXIT]: Add dummy explicit
reference to __call_exitprocs.
* libc/stdlib/cxa_atexit.c [_LITE_EXIT]: Make __register_exitproc a
weak reference.
* libc/stdlib/exit.c (exit)[_LITE_EXIT]: Remove TWS and weakly reference
__call_exitprocs.
* libc/include/sys/config.h (_REENT_GLOBAL_ATEXIT): Define for
RTEMS.
* libc/include/sys/reent.h (_reent): Use _REENT_GLOBAL_ATEXIT.
(_global_atexit): Declare if _REENT_GLOBAL_ATEXIT is defined.
* libc/reent/reent.c (_reclaim_reent): Remove atexit cleanup if
_REENT_GLOBAL_ATEXIT is defined.
(_wrapup_reent): Remove atexit handling if _REENT_GLOBAL_ATEXIT
is defined.
* libc/stdlib/__atexit.c (_global_atexit0): Define if
_REENT_GLOBAL_ATEXIT is defined.
* libc/stdlib/__call_atexit.c (_global_atexit): Define if
_REENT_GLOBAL_ATEXIT is defined.
compiler warnings found this way.
* libc/stdio/freopen.c (_freopen_r): Fix bug setting _flags.
* libc/include/stdio.h (_rename): Define when building newlib.
* libc/include/sys/signal.h (_kill): Ditto.
* libc/include/sys/stat.h (_mkdir): Ditto.
* libc/include/sys/time.h (_gettimeofday): Ditto.
* libc/include/sys/times.h (_times): Ditto.
* libc/include/sys/wait.h (_wait): Ditto.
* libc/locale/lmessages.c (empty): Don't define for Cygwin.
* libc/locale/lmonetary.c (cnv): Ditto.
* libc/locale/nl_langinfo.c (nl_langinfo): Ditto for variable s.
* libc/posix/collate.c: Throughout cast to avoid compiler warning.
* libc/posix/engine.c (matcher): Initialize dp to avoid compiler
warning.
* libc/posix/glob.c: Disable on Cygwin. Explain why.
* libc/posix/regcomp.c: Fix "uninitialized" compiler warnings.
(dissect): Deliberately silence gcc compiler warning. Add comment to
explain why.
* libc/posix/wordexp.c (wordexp): Remove num_bytes variable since result
is never used.
* libc/posix/popen.c (popen): Ditto for variable last.
* libc/reent/mkdirr.c: Include sys/stat.h.
* libc/reent/renamer.c: Include stdio.h.
* libc/search/hash.c: Throughout use underscored variants of the stat
function family.
(init_hash): Add missing definition for the __USE_INTERNAL_STAT64 case.
* libc/search/hash_bigkey.c (__big_insert): Add parenthesis to avoid
compiler warning.
* libc/search/hash_page.c (overflow_page): Initalize freep to NULL to
avoid compiler warning.
* libc/stdio/asiprintf.c (_asiprintf_r): Cast unsigned char * to char *
to avoid compiler warning.
(asiprintf): Ditto.
* libc/stdio/asprintf.c (_asprintf_r): Ditto.
(asprintf): Ditto.
* libc/stdio/vasiprintf.c (_vasiprintf_r): Ditto.
* libc/stdio/vasprintf.c (_vasprintf_r): Ditto.
* libc/stdio/mktemp.c (_gettemp): Cast to unsigned char in call to
isdigit to avoid compiler warning.
* libc/stdio/vfprintf.c (_VFPRINTF_R): Initialize variables used for
grouping to avoid compiler warning. Only define and set nseps and
nrepeats if they are really used.
* libc/stdio/vfwprintf.c (_VFWPRINTF_R): Ditto. Only define state if
it is really used.
* libc/stdio/vfscanf.c (u_char): Revert to be defined as unsigned char.
(__SVFSCANF_R): Cast fmt in call to __mbtowc.
* libc/stdlib/mbtowc_r.c (JIS_state_table): Disable when building
Cygwin.
(JIS_action_table): Ditto.
* libc/stdlib/wctomb_r.c (__utf8_wctomb): Add parenthesis to avoid
compiler warning.
* libc/string/strcasestr.c: Deliberately silence gcc compiler warning.
Add comment to explain why.
* libc/time/strptime.c (strptime): Cast to unsigned char in calls to
isspace to avoid compiler warning.
* libm/math/e_atan2.c (__ieee754_atan2): Add parenthesis to avoid
compiler warning.
* libm/math/e_exp.c (__ieee754_exp): Initialize k to 0 to avoid
compiler warning. Drop setting it to 0 later.
* libm/math/ef_exp.c (__ieee754_expf): Ditto.
* libm/math/e_pow.c (__ieee754_pow): Add braces to avoid compiler
warning.
* libm/math/ef_pow.c (__ieee754_powf): Ditto.
* libm/math/er_lgamma.c (__ieee754_lgamma_r): Initialize nadj to 0 to
avoid compiler warning.
* libm/math/erf_lgamma.c (__ieee754_lgammaf_r): Ditto.
* libm/math/e_rem_pio2.c (__ieee754_rem_pio2): Ditto for variable z.
* libm/common/sf_round.c (roundf): Remove signbit variable since result
is never used.
* libc/stdlib/strtol.c: Make sure signed characters are not
sign-extended when converted to int and passed to ctype macros.
* libc/stdlib/strtoul.c: Ditto.
* libc/stdlib/strtoll_r.c: Ditto.
* libc/stdlib/strtoull_r.c: Ditto.
* configure.in: Add test for enable-newlib-register-fini
to set the _WANT_REGISTER_FINI flag.
* configure: Regenerated.
* Makefile.in: Ditto.
* newlib.hin: Add _WANT_REGISTER_FINI flag.
* libc/stdlib/__call_atexit.c: Add test for
_WANT_REGISTER_FINI to enable the register_fini function.
pwc to pmb and convert to array to avoid buffer overflow. Rename c to
wc. Check wc for WEOF instead of for EOF. Return first byte of pmb if
__wctomb conversion returned exactly one byte, EOF otherwise.
* libc/stdlib/__atexit.c (__atexit_lock): Initialise as recursive
rather than non-recursive lock type.
(__register_exitproc): Use recursive locking APIs on it.
* libc/stdlib/__call_atexit.c (__call_exitprocs): Likewise.
* libc/stdlib/getenv.c: Delete "char *_findenv_r ();", as is not a
proper prototype, and is properly prototyped in stdlib.h, anyway.
* libc/stdlib/getenv_r.c: Ditto.
* libc/search/hash.c: Add _DEFUN to __hash_open() declaration; add
#define __DBINTERFACE_PRIVATE to activate prototypes from db_local.h.
* libc/search/db_local.h: Correct __hash_open() prototype.
* libc/sys/linux/cmath/math_private.h: Eliminate compiler warnings:
Remove #define INFINITY (redefines from math.h); remove #define __isnanf
and #define __isinff isinff.
Thomas Preud'homme
Freddie Chopin
Thomas Preudhomme
Joe Seymour
Corinna Vinschen
Jeff Johnston
Jon Turney
Andre Vieira (lists)
Sebastian Huber
Igor Venevtsev
Joel Sherrill
Howland, Craig D
Yaakov Selkowitz
Steve Ellcey
Nick Clifton
DJ Delorie
Joel Sherrill
Ralf Corsepius