Commit Graph

2850 Commits

Author SHA1 Message Date
Corinna Vinschen b93022a82d Cygwin: open: support Linux-specific O_PATH flag
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
2019-01-07 19:35:00 +01:00
Wilco Dijkstra 353ebae304 Improve performance of memmem
This patch significantly improves performance of memmem using a novel
modified Horspool algorithm.  Needles up to size 256 use a bad-character
table indexed by hashed pairs of characters to quickly skip past mismatches.
Long needles use a self-adapting filtering step to avoid comparing the whole
needle repeatedly.

By limiting the needle length to 256, the shift table only requires 8 bits
per entry, lowering preprocessing overhead and minimizing cache effects.
This limit also implies worst-case performance is linear.

Small needles up to size 2 use a dedicated linear search.  Very long needles
use the Two-Way algorithm (to avoid increasing stack size inlining is now disabled).

The performance gain is 6.6 times on English text on AArch64 using random
needles with average size 8 (this is even faster than the recently improved strstr
algorithm, so I'll update that in the near future).

The size-optimized memmem has also been rewritten from scratch to get a
2.7x performance gain.

Tested against GLIBC testsuite and randomized tests.

Message-Id: <DB5PR08MB1030649D051FA8532A4512C883B20@DB5PR08MB1030.eurprd08.prod.outlook.com>
2019-01-01 09:44:59 -06:00
Jeff Johnston 5726873100 Bump release to 3.1.0 for yearly snapshot 2018-12-31 23:40:11 -05:00
Sebastian Huber dc6e94551f RTEMS: Use __uint64_t for __ino_t
FreeBSD uses a 64-bit ino_t since 2017-05-23.  We need this for the
pipe() support in libbsd.

Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2018-12-20 12:12:38 +01:00
markj 44756a36ab Plug routing sysctl leaks.
Various structures exported by sysctl_rtsock() contain padding fields
which were not being zeroed.

Reported by:	Thomas Barabosch, Fraunhofer FKIE
Reviewed by:	ae
MFC after:	3 days
Security:	kernel memory disclosure
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D18333
2018-12-20 12:12:37 +01:00
Jon Beniston b3692aed5e nano-vfprintf_float.c: Fix check if negative for nans. 2018-12-13 13:15:32 +01:00
Sebastian Huber 55db4a8e3a sys/time.h: Remove KASSERT
The KASSERT is only used by the FreeBSD kernel.

Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2018-12-04 07:39:20 +01:00
imp be517bd298 Ensure that all values of ns, us and ms work
for {n,u,m}stosbt

Integer overflows and wrong constants limited the accuracy of these
functions and created situatiosn where sbttoXs(Xstosbt(Y)) != Y. This
was especailly true in the ns case where we had millions of values
that were wrong.

Instead, used fixed constants because there's no way to say ceil(X)
for integer math. Document what these crazy constants are.

Also, use a shift one fewer left to avoid integer overflow causing
incorrect results, and adjust the equasion accordingly. Document this.

Allow times >= 1s to be well defined for these conversion functions
(at least the Xstosbt). There's too many users in the tree that they
work for >= 1s.

This fixes a failure on boot to program firmware on the mlx4
NIC. There was a msleep(1000) in the code. Prior to my recent rounding
changes, msleep(1000) worked, but msleep(1001) did not because the old
code rounded to just below 2^64 and the new code rounds to just above
it (overflowing, causing the msleep(1000) to really sleep 1ms).

A test program to test all cases will be committed shortly. The test
exaustively tries every value (thanks to bde for the test).

Sponsored by: Netflix, Inc
Differential Revision: https://reviews.freebsd.org/D18051
2018-12-04 07:39:15 +01:00
imp 7bf8fc0987 When converting ns,us,ms to sbt, return the ceil()
of the result rather than the floor(). Returning the floor means that
sbttoX(Xtosbt(y)) != y for almost all values of y. In practice, this
results in a difference of at most 1 in the lsb of the sbintime_t. This
difference is meaningless for all current users of these functions, but
is important for the newly introduced sysctl conversion routines which
implicitly rely on the transformation being idempotent.

Sponsored by: Netflix, Inc
2018-12-04 07:30:18 +01:00
ian 68b1d72e1d Correct a misplaced closing paren.
Does not affect the result, but does clarify (at least for me) that the
multiplication happens before the shift.
2018-12-04 07:30:09 +01:00
pfg 3266b2dd5e sys: further adoption of SPDX licensing ID tags.
Mainly focus on files that use BSD 3-Clause license.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.
2018-12-04 07:30:05 +01:00
ian 3c3c17500c Add inline functions to convert between sbintime_t
and decimal time units. Use them in some existing code that is
vulnerable to roundoff errors.

The existing constant SBT_1NS is a honeypot, luring unsuspecting folks into
writing code such as long_timeout_ns*SBT_1NS to generate the argument for a
sleep call.  The actual value of 1ns in sbt units is ~4.3, leading to a
large roundoff error giving a shorter sleep than expected when multiplying
by the trucated value of 4 in SBT_1NS.  (The evil honeypot aspect becomes
clear after you waste a whole day figuring out why your sleeps return early.)
2018-12-04 07:29:51 +01:00
imp 8f4149ea93 Renumber copyright clause 4
Renumber cluase 4 to 3, per what everybody else did when BSD granted
them permission to remove clause 3. My insistance on keeping the same
numbering for legal reasons is too pedantic, so give up on that point.

Submitted by:	Jan Schaumann <jschauma@stevens.edu>
Pull Request:	https://github.com/freebsd/freebsd/pull/96
2018-12-04 07:29:50 +01:00
Corinna Vinschen 09870c6e95 stdio.h: Expose cuserid with __GNU_VISIBLE
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
2018-11-29 11:22:42 +01:00
Corinna Vinschen f4d6ef2d41 time.h: Introduce Linux-specific CLOCK id values
- Add CLOCK_REALTIME_COARSE, CLOCK_MONOTONIC_RAW,
  CLOCK_MONOTONIC_COARSE and CLOCK_BOOTTIME

- Guard new values with __GNU_VISIBLE

- Add CLOCK_REALTIME_COARSE as (clockid_t) 0 for simplicity
  (It allows to have all values < 8 and so be used as array
   index into an array of clocks)

- Fix macro bracketing

Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
2018-11-29 10:01:57 +01:00
Matthew Malcomson 2d6b71ee6d Builtin enable return code with SYS_EXIT_EXTENDED
A previous commit introduced the ability to use the semi-hosting
SYS_EXIT_EXTENDED operation to libgloss, this commit adds the same
ability to the sys/arm/ backend so that building newlib only will
provide the same capabilities.
2018-11-26 10:07:55 -05:00
Wilco Dijkstra df7824d1a4 Fix issue with dst bias in memset
This patch fixes an issue in the previous memset loop change. If the
zva size is >= 256 and there are more than 64 bytes left in the
tail, we could enter the loop and thus need to rebias dst by 32 as
well.

Since no known CPUs use this size this can't be tested natively, so I've
tested it on a simulator initialized with a large zva size.

--
2018-11-08 16:45:19 +00:00
Sebastian Huber 2ab57ad59b Fix v850 target for RTEMS
Do not define __ATTRIBUTE_IMPURE_PTR__ for RTMES on the v850 target.
The previous definition lead to the following linker error in
combination with -fdata-sections:

relocation truncated to fit: R_V810_GPWLO_1 against symbol
`_global_impure_ptr' defined in .rodata._global_impure_ptr section in
libc.a(lib_a-impure.o)

relocation truncated to fit: R_V810_GPWLO_1 against symbol `_impure_ptr'
defined in .data._impure_ptr section in libc.a(lib_a-impure.o)

Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2018-11-08 09:38:13 +01:00
Sebastian Huber 1471e7cd74 RTEMS: Avoid <machine/param.h> in <sys/_cpuset.h>
The <machine/param.h> header file exposes some unrelated stuff not
covered by C or POSIX.  Avoid its use in <sys/_cpuset.h> since it is
included in <rtems.h>.

Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2018-11-08 09:38:12 +01:00
Wilco Dijkstra d80db60066 Adjust writeback in non-zero memset
This fixes an ineffiency in the non-zero memset.  Delaying the writeback
until the end of the loop is slightly faster on some cores - this shows
~5% performance gain on Cortex-A53 when doing large non-zero memsets.

Tested against the GLIBC testsuite.
2018-11-06 14:59:51 +00:00
Sebastian Huber 08eab6396f Fix posix_memalign() attributes
The malloc, alloc_size and alloc_aligned attributes must be only used in
case the function returns the pointer to the allocated memory.

See also:

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=87683

Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2018-10-22 11:26:39 +02:00
Joel Sherrill 037428fae3 newlib/libc/sys/rtems/include/machine/param.h: Add _KERNEL to stop method leakage
The following FreeBSD kernel methods are not in any standard and
prototypes/definitions were leaking into application space:

  + round_page()
  + trunc_page()
  + atop()
  + ptoa()
  + pgtok()
2018-10-18 17:19:50 -05:00
Wilco Dijkstra 473f1a3a5d Improve performance of strstr
v3: Add support for read ahead using strnlen, giving an additional 25% speedup
on large inputs (both short and long needles).

This patch significantly improves performance of strstr by using Sunday's
Quick-Search algorithm.  Due to its simplicity it has the best average
performance of string matching algorithms on almost all inputs.  It uses a
bad-character shift table to skip past mismatches.

The needle length is limited to 254 - this reduces the shift table memory
4 to 8 times, lowering preprocessing overhead and minimizing cache effects.
The limit also implies its worst-case performance is linear.

Larger needles are processed by the Two-Way algorithm.  The macro AVAILABLE
has been improved to use strnlen to read the input in chunks.  This results
in a 2.5 times speedup for large needles, reducing the performance drop when
the Quick-Search algorithm can't be used.

The code for 1-4 byte needles has been simplified and now uses unsigned
char.  Since the optimized code relies on 8-bit chars, we defer to the
size-optimized implementation if CHAR_BIT > 8.

The performance gain of finding a set of randomly chosen words of size 8 in
256 bytes of English text is 14 times on AArch64. For longer haystacks the
gain is well over 20 times.

The size-optimized strstr has also been rewritten from scratch to improve
performance.  On the same test the performance gain is 69%.

Tested against GLIBC testsuite, randomized tests and the GNULIB strstr test
(https://git.savannah.gnu.org/cgit/gnulib.git/tree/tests/test-strstr.c).

--
2018-10-18 19:51:39 +02:00
Christophe Lyon 4f7a6c326a newlib/libc/ctype/jp2uc.c: Declare "cs" variable as "const char *"
Instead of "char *" to avoid compiler warnings.
This is OK because "cs" is only used as input of strcmp.
2018-10-11 16:32:14 +02:00
Sebastian Huber 103b055035 Add generic implementation of fdopendir()
Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2018-10-11 08:29:17 +02:00
Sebastian Huber ab4fdab5d5 Add generic implementation of dirfd()
Use existing HAVE_OPENDIR define to determine if a generic
implementation should be provided.  Cygwin for example has its own
implementation of opendir() and dirfd().

Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2018-10-11 08:29:16 +02:00
Sebastian Huber 738fdc6a42 RTEMS: Add struct dirent::d_type member
This is used by the file system support of libstdc++ for example.  Use
content from latest FreeBSD <sys/dirent.h>

Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2018-10-11 08:29:16 +02:00
Sebastian Huber da418955f5 Move common <sys/dirent.h> content to <dirent.h>
Move common content of the various <sys/dirent.h> and the latest FreeBSD
<dirent.h> to <dirent.h>.

Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2018-10-11 08:29:16 +02:00
Sebastian Huber 61fc64ed97 Open a directory with the usual flags
Use O_RDONLY since you are not supposed to write to a directory.

Use O_DIRECTORY as mandated by POSIX (The Open Group Base Specifications
Issue 7, 2018 edition IEEE Std 1003.1-2017):

"If the type DIR is implemented using a file descriptor, the descriptor
shall be obtained as if the O_DIRECTORY flag was passed to open()."

Use O_CLOEXEC as mandated by POSIX:

"When a file descriptor is used to implement the directory stream, it
behaves as if the FD_CLOEXEC had been set for the file descriptor."

Drop the fcntl() call in favour of O_CLOEXEC.

Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2018-10-11 08:29:16 +02:00
Sebastian Huber d3d838cc26 Make some standard open() flags visible
Make the POSIX O_CLOEXEC, O_NOFOLLOW, O_DIRECTORY, O_EXEC, and O_SEARCH
open() flags available also to non-Cygwin systems.

Make the BSD/glibc O_DIRECT open() flag  available also to non-Cygwin
systems.

Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2018-10-11 08:29:16 +02:00
Corinna Vinschen 256f1171ac newlib: Build internal strtold code only if HAVE_LONG_DOUBLE defined
Commit fbace81684
("Import correctly working strtold from David M. Gay.")
introduced two new files, strtorx.c and strtodg.c.  The functions
are only called from strtold.c.  However, while strtold.c is only
built if HAVE_LONG_DOUBLE is defined, the patch erroneously added
the two new files to GENERAL_SOURCES unconditionally.

Fix this by building both files only if HAVE_LONG_DOUBLE has been
defined.

Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
2018-10-10 18:01:22 +02:00
Corinna Vinschen 35555851d7 newlib: strtold: use __builtin_nanl to avoid libm dependency
Commit 6c212a8b78
("Fix strtod ("nan") and strtold ("nan") returns wrong negative NaN")
introduced an unconditional dependency to nanl and, in turn, to libm.

Rather than including nanl in libc as well, just call __builtin_nanl
from here.  Requires GCC 3.3 or later.

Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
2018-10-10 17:53:55 +02:00
Corinna Vinschen 9479563e48 newlib: Drop incorrect const qualifier from __loadlocale parameter
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
2018-10-10 11:18:20 +02:00
Sebastian Huber 201bbec6e4 Add attributes to malloc-like functions
These attributes help static analysis tools to produce less false
positives, e.g. double free warnings.

Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2018-10-10 07:40:06 +02:00
Christophe Lyon 8a7536e91d [ARM] Make _kill() a noreturn function.
AngelSWI_Reason_ReportException does not return accoring to the ARM
documentation, so it is valid to mark _kill() as noreturn.  This way,
the compiler does not warn about _exit() returning a value despite
being noreturn.

2018-10-01  Christophe Lyon  <christophe.lyon@linaro.org>

	* libgloss/arm/_exit.c (_exit): Declare _kill() as noreturn.
	* libgloss/arm/_exit.c (_kill): Likewise. Remove the return
	statements.
	* newlib/libc/sys/arm/syscalls.c (_kill): Likewise..
2018-10-08 14:35:43 +01:00
Christophe Lyon 3878d82a2b [ARM] Cast string pointers to int to avoid compiler warnings.
2018-10-01  Christophe Lyon  <christophe.lyon@linaro.org>

    	* newlib/libc/sys/arm/syscalls.c (_unlink): Cast 'path' to int.
    	(_system): Cast 's' to int.
    	(_rename): Cast 'newpath' and 'oldpath' to int.
2018-10-05 12:00:44 +01:00
Jon Beniston bd993df0e6 search: Fix Berkeley DB hash code for 16-bit targets.
hash.h: Use 32-bit type for data stored on disk, so code
works for 16 and 64-bit targets. Reduce maximum bucket size on 16-bit
targets, so it fits in available memory.
hash.c: Check bucket size isn't too big for target.
hash_buf.c: Fix overflow warning on 16-bit targets.
2018-09-06 17:29:49 +02:00
Keith Packard 77f8a6dfab Use !__HAVE_LOCALE_INFO__ define to use _ctype_ directly [v2]
When __HAVE_LOCALE_INFO__ is not selected, directly access the
existing _ctype_ variable from __locale_ctype_ptr() and
__locale_ctype_ptr_l(), eliminating the need for any locale or reent
structure

Signed-off-by: Keith Packard <keithp@keithp.com>

v2:
	locale: fix conflict with __locale_ctype_ptr macro

	If we are building without __HAVE_LOCALE_INFO__, there is a
	macro providing __locale_ctype_ptr which in turn fouls up this
	declaration.

	Signed-off-by: Michael Lyle <mlyle@lyle.org>
2018-09-06 14:19:53 +02:00
Keith Packard 3b6994ec5f stdlib: Use __get_numeric_locale instead of __localeconv_l for decimal_point
The string/float conversion functions need to get the locale decimal
point. Instead of calling __localeconv_l (which copies locale data
into lconv form from __get_numeric_locale), use __get_numeric_locale
directly.

Signed-off-by: Keith Packard <keithp@keithp.com>
2018-09-06 14:14:05 +02:00
Keith Packard 28ecec475f Include sys/syslimits.h in limits.h
This makes sure any system-defined limits are specified before the
defaults are checked. Without this, ARG_MAX and PATH_MAX end up
getting the default definitions from limits.h rather than the defines
from syslimits.h. This could potentially cause problems when
different files used different values for the same name.

Signed-off-by: Keith Packard <keithp@keithp.com>
2018-09-06 14:11:45 +02:00
Keith Packard 81e0841dbc xdr_private.h needs sys/types.h for u_char
u_char is defined in sys/types.h, which doesn't appear to get
included, at least in my environments.

Signed-off-by: Keith Packard <keithp@keithp.com>
2018-09-06 14:10:42 +02:00
Jon Beniston 7283d2513c stdlib/arc4random.c: Fix reseed count for 16-bit targets. 2018-09-06 13:26:25 +02:00
Wilco Dijkstra 6dbb20dfc7 Improve strstr performance of short needles
Improve strstr performance for the common case of short needles.  For a single
character strchr is best, for 2-4 characters a small loop is fastest.  For these
the speedup over the Two-Way algorithm is ~10 times on large strings.

Newlib builds, the new code passes GLIBC testsuite. OK for commit?
2018-09-05 10:09:31 +02:00
Andy Koppe 3017f23f1c Avoid ARM SWI Seek when querying file position
Issuing an ARM semi-hosting Seek command when just querying file
position with SEEK_CUR and offset zero is unnecessary, because unlike
the lseek() Unix system call the Seek command does not actually return
the file position. For that reason, syscalls.c for ARM keeps track of
file position in the 'poslog', so we can just return that.

Moreover, since the Seek command only accepts an absolute file position,
SEEK_CUR operations are implemented by adding the relative offset to the
position in the poslog. If the host implements non-binary files with
implicit carriage return characters but doesn't discount those implicit
CRs when implementing Seek (by just mapping straight to Windows file
operations), this actually ended up wrongly changing file position when
using SEEK_CUR with offset zero or functions like ftell() or fgetpos()
that are based on that.

Also, use off_t rather than int for the poslog.
2018-09-03 09:40:39 +02:00
Jon Beniston a9cfb33b6c Add --disable-newlib-fno-builtin to allow compilation without -fno-builtin for smaller and faster code. 2018-08-31 15:40:42 -04:00
Andy Koppe d8ccbcdacc Drop non-reserved parameter names from sys/signal.h
Standard headers shouldn't use non-reserved identifiers as parameter
names in function declarations, because programs could in theory
define macros with such names before including a header.
2018-08-29 17:46:27 +02:00
Keith Packard 82dfae9ab0 Use __inhibit_loop_to_libcall in all memset/memcpy implementations
This macro selects a compiler option that disables recognition of
common memset/memcpy patterns and converting those to direct
memset/memcpy calls.

Signed-off-by: Keith Packard <keithp@keithp.com>
2018-08-29 16:05:37 +02:00
Keith Packard 2c245028af Use nanf("") instead of nanf(NULL)
Newer GCC versions require a non-NULL argument to this function for
some reason.

Signed-off-by: Keith Packard <keithp@keithp.com>
2018-08-29 15:57:27 +02:00
Sebastian Huber d13c84eb07 RTEMS: Add kvaddr_t and ksize_t
These types were introduced by FreeBSD commit:

"Make struct xinpcb and friends word-size independent.

Replace size_t members with ksize_t (uint64_t) and pointer members
(never used as pointers in userspace, but instead as unique
idenitifiers) with kvaddr_t (uint64_t). This makes the structs
identical between 32-bit and 64-bit ABIs.

On 64-bit bit systems, the ABI is maintained. On 32-bit systems,
this is an ABI breaking change. The ABI of most of these structs
was previously broken in r315662.  This also imposes a small API
change on userspace consumers who must handle kernel pointers
becoming virtual addresses.

PR:		228301 (exp-run by antoine)
Reviewed by:	jtl, kib, rwatson (various versions)
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D15386"

In RTEMS, there is no user/kernel space separation.  So, use the types
size_t and uintptr_t.

Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2018-08-24 15:07:29 +02:00
Sebastian Huber d35971f392 RTEMS: Introduce <machine/_kernel_mman.h>
This helps to avoid Newlib updates due to FreeBSD kernel space changes.

Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2018-08-24 15:04:43 +02:00
Sebastian Huber a2a8600f7d RTEMS: Introduce <machine/_kernel_socket.h>
This helps to avoid Newlib updates due to FreeBSD kernel space changes.

Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2018-08-24 15:04:43 +02:00
Sebastian Huber 764d748c9c RTEMS: Introduce <machine/_kernel_if.h>
This helps to avoid Newlib updates due to FreeBSD kernel space changes.

Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2018-08-24 15:04:43 +02:00
Sebastian Huber 0c0dd28596 RTEMS: Introduce <machine/_kernel_in.h>
This helps to avoid Newlib updates due to FreeBSD kernel space changes.

Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2018-08-24 15:04:43 +02:00
Sebastian Huber 9ce55ee716 RTEMS: Introduce <machine/_kernel_in6.h>
This helps to avoid Newlib updates due to FreeBSD kernel space changes.

Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2018-08-24 15:04:43 +02:00
Sebastian Huber c07fa084e0 RTEMS: Introduce <machine/_kernel_uio.h>
This helps to avoid Newlib updates due to FreeBSD kernel space changes.

Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2018-08-24 15:04:43 +02:00
Sebastian Huber 9bbf89dd11 RTEMS: Add __BSD_VISIBLE in <sys/_termios.h>
The __XSI_VISIBLE is not enabled by default in Newlib.  This is an
incompatiblity between FreeBSD and glibc.

Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2018-08-24 15:04:42 +02:00
Sebastian Huber 890c86d633 RTEMS: Update FreeBSD version tags
Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2018-08-24 15:04:39 +02:00
tuexen fe3e8b90dc Add SOL_SOCKET level socket option
with name SO_DOMAIN to get the domain of a socket.

This is helpful when testing and Solaris and Linux have the same
socket option using the same name.

Reviewed by:		bcr@, rrs@
Sponsored by:		Netflix, Inc.
Differential Revision:	https://reviews.freebsd.org/D16791
2018-08-24 15:00:04 +02:00
jtl 823b096471 Implement a limit on on the number of IPv6 reassembly
queues per bucket.

There is a hashing algorithm which should distribute IPv6 reassembly
queues across the available buckets in a relatively even way. However,
if there is a flaw in the hashing algorithm which allows a large number
of IPv6 fragment reassembly queues to end up in a single bucket, a per-
bucket limit could help mitigate the performance impact of this flaw.

Implement such a limit, with a default of twice the maximum number of
reassembly queues divided by the number of buckets. Recalculate the
limit any time the maximum number of reassembly queues changes.
However, allow the user to override the value using a sysctl
(net.inet6.ip6.maxfragbucketsize).

Reviewed by:	jhb
Security:	FreeBSD-SA-18:10.ip
Security:	CVE-2018-6923
2018-08-24 15:00:04 +02:00
jtl 0e5c59050d Add a limit of the number of fragments per IPv6 packet.
The IPv4 fragment reassembly code supports a limit on the number of
fragments per packet. The default limit is currently 17 fragments.
Among other things, this limit serves to limit the number of fragments
the code must parse when trying to reassembly a packet.

Add a limit to the IPv6 reassembly code. By default, limit a packet
to 65 fragments (64 on the queue, plus one final fragment to complete
the packet). This allows an average fragment size of 1,008 bytes, which
should be sufficient to hold a fragment. (Recall that the IPv6 minimum
MTU is 1280 bytes. Therefore, this configuration allows a full-size
IPv6 packet to be fragmented on a link with the minimum MTU and still
carry approximately 272 bytes of headers before the fragmented portion
of the packet.)

Users can adjust this limit using the net.inet6.ip6.maxfragsperpacket
sysctl.

Reviewed by:	jhb
Security:	FreeBSD-SA-18:10.ip
Security:	CVE-2018-6923
2018-08-24 15:00:04 +02:00
rrs 215e33310b This commit brings in a new refactored TCP stack called Rack.
Rack includes the following features: - A different SACK processing
scheme (the old sack structures are not used). - RACK (Recent
acknowledgment) where counting dup-acks is no longer done instead time
is used to knwo when to retransmit. (see the I-D) - TLP (Tail Loss
Probe) where we will probe for tail-losses to attempt to try not to take
a retransmit time-out. (see the I-D) - Burst mitigation using TCPHTPS -
PRR (partial rate reduction) see the RFC.

Once built into your kernel, you can select this stack by either
socket option with the name of the stack is "rack" or by setting
the global sysctl so the default is rack.

Note that any connection that does not support SACK will be kicked
back to the "default" base  FreeBSD stack (currently known as "default").

To build this into your kernel you will need to enable in your
kernel:
   makeoptions WITH_EXTRA_TCP_STACKS=1
   options TCPHPTS

Sponsored by:	Netflix Inc.
Differential Revision:		https://reviews.freebsd.org/D15525
2018-08-24 15:00:04 +02:00
sbruno b40c48e057 Load balance sockets with new SO_REUSEPORT_LB option.
This patch adds a new socket option, SO_REUSEPORT_LB, which allow multiple
programs or threads to bind to the same port and incoming connections will be
load balanced using a hash function.

Most of the code was copied from a similar patch for DragonflyBSD.

However, in DragonflyBSD, load balancing is a global on/off setting and can not
be set per socket. This patch allows for simultaneous use of both the current
SO_REUSEPORT and the new SO_REUSEPORT_LB options on the same system.

Required changes to structures:
Globally change so_options from 16 to 32 bit value to allow for more options.
Add hashtable in pcbinfo to hold all SO_REUSEPORT_LB sockets.

Limitations:
As DragonflyBSD, a load balance group is limited to 256 pcbs (256 programs or
threads sharing the same socket).

This is a substantially different contribution as compared to its original
incarnation at svn r332894 and reverted at svn r332967.  Thanks to rwatson@
for the substantive feedback that is included in this commit.

Submitted by:	Johannes Lundberg <johalun0@gmail.com>
Obtained from:	DragonflyBSD
Relnotes:	Yes
Sponsored by:	Limelight Networks
Differential Revision:	https://reviews.freebsd.org/D11003
2018-08-24 15:00:04 +02:00
mmacy 44e0190a8c iflib(9): Add support for cloning pseudo interfaces
Part 3 of many ...
The VPC framework relies heavily on cloning pseudo interfaces
(vmnics, vpc switch, vcpswitch port, hostif, vxlan if, etc).

This pulls in that piece. Some ancillary changes get pulled
in as a side effect.

Reviewed by:	shurd@
Approved by:	sbruno@
Sponsored by:	Joyent, Inc.
Differential Revision:	https://reviews.freebsd.org/D15347
2018-08-24 15:00:04 +02:00
sbruno 6a98562b52 Revert r332894 at the request of the submitter.
Submitted by:	Johannes Lundberg <johalun0_gmail.com>
Sponsored by:	Limelight Networks
2018-08-24 15:00:04 +02:00
sbruno 5c636abe89 Load balance sockets with new SO_REUSEPORT_LB option
This patch adds a new socket option, SO_REUSEPORT_LB, which allow multiple
programs or threads to bind to the same port and incoming connections will be
load balanced using a hash function.

Most of the code was copied from a similar patch for DragonflyBSD.

However, in DragonflyBSD, load balancing is a global on/off setting and can not
be set per socket. This patch allows for simultaneous use of both the current
SO_REUSEPORT and the new SO_REUSEPORT_LB options on the same system.

Required changes to structures
Globally change so_options from 16 to 32 bit value to allow for more options.
Add hashtable in pcbinfo to hold all SO_REUSEPORT_LB sockets.

Limitations
As DragonflyBSD, a load balance group is limited to 256 pcbs
(256 programs or threads sharing the same socket).

Submitted by:	Johannes Lundberg <johanlun0@gmail.com>
Sponsored by:	Limelight Networks
Differential Revision:	https://reviews.freebsd.org/D11003
2018-08-24 15:00:04 +02:00
brooks 341e131f7f Add 32-bit compat for ioctls that take struct ifgroupreq.
Use an accessor to access ifgr_group and ifgr_groups.

Use an macro CASE_IOC_IFGROUPREQ(cmd) in place of case statements such
as "case SIOCAIFGROUP:". This avoids poluting the switch statements
with large numbers of #ifdefs.

Reviewed by:	kib
Obtained from:	CheriBSD
MFC after:	1 week
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D14960
2018-08-24 15:00:03 +02:00
brooks 79291d6123 Use an accessor function to access ifr_data.
This fixes 32-bit compat (no ioctl command defintions are required
as struct ifreq is the same size).  This is believed to be sufficent to
fully support ifconfig on 32-bit systems.

Reviewed by:	kib
Obtained from:	CheriBSD
MFC after:	1 week
Relnotes:	yes
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D14900
2018-08-24 15:00:03 +02:00
jeff c0f64943e7 Implement several enhancements to NUMA policies.
Add a new "interleave" allocation policy which stripes pages across
domains with a stride or width keeping contiguity within a multi-page
region.

Move the kernel to the dedicated numbered cpuset #2 making it possible
to assign kernel threads and memory policy separately from user.  This
also eliminates the need for the complicated interrupt binding code.

Add a sysctl API for viewing and manipulating domainsets.  Refactor some
of the cpuset_t manipulation code using the generic bitset type so that
it can be used for both.  This probably belongs in a dedicated subr file.

Attempt to improve the include situation.

Reviewed by:	kib
Discussed with:	jhb (cpuset parts)
Tested by:	pho (before review feedback)
Sponsored by:	Netflix, Dell/EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D14839
2018-08-24 15:00:03 +02:00
brooks f967e60cab Fix access to ifru_buffer on freebsd32.
Make all kernel accesses to ifru_buffer go via access functions
which take the process ABI into account and use an appropriate union
to access members in the correct place in struct ifreq.

Reviewed by:	kib
Obtained from:	CheriBSD
MFC after:	1 week
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D14846
2018-08-24 15:00:03 +02:00
kib b0250c7356 Allow to specify PCP on packets not belonging to any VLAN.
According to 802.1Q-2014, VLAN tagged packets with VLAN id 0 should be
considered as untagged, and only PCP and DEI values from the VLAN tag
are meaningful.  See for instance
https://www.cisco.com/c/en/us/td/docs/switches/connectedgrid/cg-switch-sw-master/software/configuration/guide/vlan0/b_vlan_0.html.

Make it possible to specify PCP value for outgoing packets on an
ethernet interface.  When PCP is supplied, the tag is appended, VLAN
id set to 0, and PCP is filled by the supplied value.  The code to do
VLAN tag encapsulation is refactored from the if_vlan.c and moved into
if_ethersubr.c.

Drivers might have issues with filtering VID 0 packets on
receive.  This bug should be fixed for each driver.

Reviewed by:	ae (previous version), hselasky, melifaro
Sponsored by:	Mellanox Technologies
MFC after:	2 weeks
Differential revision:	https://reviews.freebsd.org/D14702
2018-08-24 15:00:03 +02:00
brooks 9cea1c4489 Move uio enums to sys/_uio.h.
Include _uio.h instead of uio.h in several headers to reduce header
polution.

Fix a few places that relied on header polution to get the uio.h header.

I have not moved struct uio as many more things that use it rely on
header polution to get other definitions from uio.h.

Reviewed by:	cem, kib, markj
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D14811
2018-08-24 15:00:03 +02:00
jtl 61d5f8adfa Add the "TCP Blackbox Recorder"
which we discussed at the developer summits at BSDCan and BSDCam in 2017.

The TCP Blackbox Recorder allows you to capture events on a TCP connection
in a ring buffer. It stores metadata with the event. It optionally stores
the TCP header associated with an event (if the event is associated with a
packet) and also optionally stores information on the sockets.

It supports setting a log ID on a TCP connection and using this to correlate
multiple connections that share a common log ID.

You can log connections in different modes. If you are doing a coordinated
test with a particular connection, you may tell the system to put it in
mode 4 (continuous dump). Or, if you just want to monitor for errors, you
can put it in mode 1 (ring buffer) and dump all the ring buffers associated
with the connection ID when we receive an error signal for that connection
ID. You can set a default mode that will be applied to a particular ratio
of incoming connections. You can also manually set a mode using a socket
option.

This commit includes only basic probes. rrs@ has added quite an abundance
of probes in his TCP development work. He plans to commit those soon.

There are user-space programs which we plan to commit as ports. These read
the data from the log device and output pcapng files, and then let you
analyze the data (and metadata) in the pcapng files.

Reviewed by:	gnn (previous version)
Obtained from:	Netflix, Inc.
Relnotes:	yes
Differential Revision:	https://reviews.freebsd.org/D11085
2018-08-24 15:00:03 +02:00
brooks 4d144963ea Add _IOC_NEWLEN() and _IOC_NEWTYPE() macros.
These macros take an existing ioctl(2) command and replace the length
with the specified length or length of the specified type respectively.
These can be used to define commands for 32-bit compatibility with fewer
opportunities for cut-and-paste errors then a whole new definition.

Reviewed by:	cem, kib
Obtained from:	CheriBSD
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D14706
2018-08-24 15:00:03 +02:00
pkelsey b4d6660d85 This is an implementation of the client side of TCP Fast Open (TFO)
[RFC7413]. It also includes a pre-shared key mode of operation in which
the server requires the client to be in possession of a shared secret in
order to successfully open TFO connections with that server.

The names of some existing fastopen sysctls have changed (e.g.,
net.inet.tcp.fastopen.enabled -> net.inet.tcp.fastopen.server_enable).

Reviewed by:	tuexen
MFC after:	1 month
Sponsored by:	Limelight Networks
Differential Revision:	https://reviews.freebsd.org/D14047
2018-08-24 15:00:03 +02:00
ae@FreeBSD.org b43341334e Follow the RFC6980 and silently ignore following IPv6 NDP messages
that had the IPv6 fragmentation header:
o Neighbor Solicitation
o Neighbor Advertisement
o Router Solicitation
o Router Advertisement
o Redirect

Introduce M_FRAGMENTED mbuf flag, and set it after IPv6 fragment reassembly
is completed. Then check the presence of this flag in correspondig ND6
handling routines.

PR:		224247
MFC after:	2 weeks
2018-08-24 15:00:03 +02:00
pfg ba2eaf10ad SPDX: license IDs for some ISC-related files. 2018-08-24 15:00:03 +02:00
glebius d937538075 Garbage collect IFCAP_POLLING_NOCOUNT.
It wasn't used since very beginning of polling(4). The module always
ignored return value from driver polling handler.
2018-08-24 15:00:03 +02:00
pfg fba31eac2e sys/sys: further adoption of SPDX licensing ID tags.
Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.
2018-08-24 15:00:03 +02:00
pfg 1329e846c7 include: further adoption of SPDX licensing ID tags.
Mainly focus on files that use BSD 3-Clause license.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.
2018-08-24 15:00:03 +02:00
pfg 9f0f4785e8 sys: further adoption of SPDX licensing ID tags.
Mainly focus on files that use BSD 3-Clause license.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.
2018-08-24 15:00:03 +02:00
kib 91e828be4f Use hardware timestamps to report packet timestamps
for SO_TIMESTAMP and other similar socket options.

Provide new control message SCM_TIME_INFO to supply information about
timestamp.  Currently it indicates that the timestamp was
hardware-assisted and high-precision, for software timestamps the
message is not returned.  Reserved fields are added to ABI to report
additional info about it, it is expected that raw hardware clock value
might be useful for some applications.

Reviewed by:	gallatin (previous version), hselasky
Sponsored by:	Mellanox Technologies
MFC after:	2 weeks
X-Differential revision:	https://reviews.freebsd.org/D12638
2018-08-24 15:00:02 +02:00
kib 7ff81234c4 Add a place for a driver to report rx timestamps
in nanoseconds from boot for the received packets.

The rcv_tstmp field overlaps the place of Ln header length indicators,
not used by received packets.  The basic pkthdr rearrangement change
in sys/mbuf.h was provided by gallatin.

There are two accompanying M_ flags: M_TSTMP means that there is the
timestamp (and it was generated by hardware).

Another flag M_TSTMP_HPREC indicates that the timestamp is
high-precision.  Practically M_TSTMP_HPREC means that hardware
provided additional precision comparing with the stamps when the flag
is not set.  E.g., for ConnectX all packets are stamped by hardware
when PCIe transaction to write out the completion descriptor is
performed, but PTP packet are stamped on port.  For Intel cards, when
PTP assist is enabled, only PTP packets are stamped in the limited
number of registers, so if Intel cards ever start support this
mechanism, they would always set M_TSTMP | M_TSTMP_HPREC if hardware
timestamp is present for the given packet.

Add IFCAP_HWRXTSTMP interface capability to indicate the support for
hardware rx timestamping, and ifconfig(8) command to toggle it.

Based on the patch by:	gallatin
Reviewed by:	gallatin (previous version), hselasky
Sponsored by:	Mellanox Technologies
MFC after:	2 weeks (? mbuf KBI issue)
X-Differential revision:	https://reviews.freebsd.org/D12638
2018-08-24 15:00:02 +02:00
sephe 1182b9fe17 if: Add ioctls to get RSS key and hash type/function.
It will be needed by hn(4) to configure its RSS key and hash
type/function in the transparent VF mode in order to match VF's
RSS settings. The description of the transparent VF mode and
the RSS hash value issue are here:
https://svnweb.freebsd.org/base?view=revision&revision=322299
https://svnweb.freebsd.org/base?view=revision&revision=322485

These are generic enough to promise two independent IOCs instead
of abusing SIOCGDRVSPEC.

Setting RSS key and hash type/function is a different story,
which probably requires more discussion.

Comment about UDP_{IPV4,IPV6,IPV6_EX} were only in the patch
in the review request; these hash types are standardized now.

Reviewed by:	gallatin
MFC after:	1 week
Sponsored by:	Microsoft
Differential Revision:	https://reviews.freebsd.org/D12174
2018-08-24 15:00:02 +02:00
des b329ee9386 Correct sysctl names. 2018-08-24 15:00:02 +02:00
kib 471f29861e Relax visibility for some termios symbols.
They are defined by XSI or newer SUS.
This is a follow-up to r318780.

Reported by:	jbeich
Obtained from:	DragonflyBSD commit e08b3836c962
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2018-08-24 15:00:02 +02:00
kib eb82d7086c Implement address space guards.
Guard, requested by the MAP_GUARD mmap(2) flag, prevents the reuse of
the allocated address space, but does not allow instantiation of the
pages in the range.  It is useful for more explicit support for usual
two-stage reserve then commit allocators, since it prevents accidental
instantiation of the mapping, e.g. by mprotect(2).

Use guards to reimplement stack grow code.  Explicitely track stack
grow area with the guard, including the stack guard page.  On stack
grow, trivial shift of the guard map entry and stack map entry limits
makes the stack expansion.  Move the code to detect stack grow and
call vm_map_growstack(), from vm_fault() into vm_map_lookup().

As result, it is impossible to get random mapping to occur in the
stack grow area, or to overlap the stack guard page.

Enable stack guard page by default.

Reviewed by:	alc, markj
Man page update reviewed by:	alc, bjk, emaste, markj, pho
Tested by:	pho, Qualys
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D11306 (man pages)
2018-08-24 15:00:02 +02:00
glebius 99b9b925fe Listening sockets improvements.
o Separate fields of struct socket that belong to listening from
  fields that belong to normal dataflow, and unionize them.  This
  shrinks the structure a bit.
  - Take out selinfo's from the socket buffers into the socket. The
    first reason is to support braindamaged scenario when a socket is
    added to kevent(2) and then listen(2) is cast on it. The second
    reason is that there is future plan to make socket buffers pluggable,
    so that for a dataflow socket a socket buffer can be changed, and
    in this case we also want to keep same selinfos through the lifetime
    of a socket.
  - Remove struct struct so_accf. Since now listening stuff no longer
    affects struct socket size, just move its fields into listening part
    of the union.
  - Provide sol_upcall field and enforce that so_upcall_set() may be called
    only on a dataflow socket, which has buffers, and for listening sockets
    provide solisten_upcall_set().

o Remove ACCEPT_LOCK() global.
  - Add a mutex to socket, to be used instead of socket buffer lock to lock
    fields of struct socket that don't belong to a socket buffer.
  - Allow to acquire two socket locks, but the first one must belong to a
    listening socket.
  - Make soref()/sorele() to use atomic(9).  This allows in some situations
    to do soref() without owning socket lock.  There is place for improvement
    here, it is possible to make sorele() also to lock optionally.
  - Most protocols aren't touched by this change, except UNIX local sockets.
    See below for more information.

o Reduce copy-and-paste in kernel modules that accept connections from
  listening sockets: provide function solisten_dequeue(), and use it in
  the following modules: ctl(4), iscsi(4), ng_btsocket(4), ng_ksocket(4),
  infiniband, rpc.

o UNIX local sockets.
  - Removal of ACCEPT_LOCK() global uncovered several races in the UNIX
    local sockets.  Most races exist around spawning a new socket, when we
    are connecting to a local listening socket.  To cover them, we need to
    hold locks on both PCBs when spawning a third one.  This means holding
    them across sonewconn().  This creates a LOR between pcb locks and
    unp_list_lock.
  - To fix the new LOR, abandon the global unp_list_lock in favor of global
    unp_link_lock.  Indeed, separating these two locks didn't provide us any
    extra parralelism in the UNIX sockets.
  - Now call into uipc_attach() may happen with unp_link_lock hold if, we
    are accepting, or without unp_link_lock in case if we are just creating
    a socket.
  - Another problem in UNIX sockets is that uipc_close() basicly did nothing
    for a listening socket.  The vnode remained opened for connections.  This
    is fixed by removing vnode in uipc_close().  Maybe the right way would be
    to do it for all sockets (not only listening), simply move the vnode
    teardown from uipc_detach() to uipc_close()?

Sponsored by:		Netflix
Differential Revision:	https://reviews.freebsd.org/D9770
2018-08-24 15:00:02 +02:00
delphij ca3b7a988a Implement INHERIT_ZERO for minherit(2).
INHERIT_ZERO is an OpenBSD feature.

When a page is marked as such, it would be zeroed
upon fork().

This would be used in new arc4random(3) functions.

PR:	182610
Reviewed by:	kib (earlier version)
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D427
2018-08-24 15:00:02 +02:00
imp 16636ede3c Renumber copyright clause 4
Renumber cluase 4 to 3, per what everybody else did when BSD granted
them permission to remove clause 3. My insistance on keeping the same
numbering for legal reasons is too pedantic, so give up on that point.

Submitted by:	Jan Schaumann <jschauma@stevens.edu>
Pull Request:	https://github.com/freebsd/freebsd/pull/96
2018-08-24 15:00:02 +02:00
ed@FreeBSD.org 08139e557b mprotect(): Change prototype to comply to POSIX.
Our mprotect() function seems to take a "const void *" address to the
pages whose permissions need to be adjusted. POSIX uses "void *". Simply
stick to the POSIX one to prevent us from writing unportable code.

PR:		211423 (exp-run)
Tested by:	antoine@ (Thanks!)
2018-08-24 15:00:02 +02:00
kib c3df6d5155 Implement process-shared locks support
for libthr.so.3, without breaking the ABI. Special value is stored in
the lock pointer to indicate shared lock, and offline page in the shared
memory is allocated to store the actual lock.

Reviewed by:	vangyzen (previous version)
Discussed with:	deischen, emaste, jhb, rwatson,
	Martin Simmons <martin@lispworks.com>
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
2018-08-24 15:00:02 +02:00
jhb 7cfc736e89 Add a new file operations hook for mmap
operations. File type-specific logic is now placed in the mmap hook
implementation rather than requiring it to be placed in
sys/vm/vm_mmap.c. This hook allows new file types to support mmap() as
well as potentially allowing mmap() for existing file types that do not
currently support any mapping.

The vm_mmap() function is now split up into two functions.  A new
vm_mmap_object() function handles the "back half" of vm_mmap() and accepts
a referenced VM object to map rather than a (handle, handle_type) tuple.
vm_mmap() is now reduced to converting a (handle, handle_type) tuple to a
a VM object and then calling vm_mmap_object() to handle the actual mapping.
The vm_mmap() function remains for use by other parts of the kernel
(e.g. device drivers and exec) but now only supports mapping vnodes,
character devices, and anonymous memory.

The mmap() system call invokes vm_mmap_object() directly with a NULL object
for anonymous mappings.  For mappings using a file descriptor, the
descriptors fo_mmap() hook is invoked instead.  The fo_mmap() hook is
responsible for performing type-specific checks and adjustments to
arguments as well as possibly modifying mapping parameters such as flags
or the object offset.  The fo_mmap() hook routines then call
vm_mmap_object() to handle the actual mapping.

The fo_mmap() hook is optional.  If it is not set, then fo_mmap() will
fail with ENODEV.  A fo_mmap() hook is implemented for regular files,
character devices, and shared memory objects (created via shm_open()).

While here, consistently use the VM_PROT_* constants for the vm_prot_t
type for the 'prot' variable passed to vm_mmap() and vm_mmap_object()
as well as the vm_mmap_vnode() and vm_mmap_cdev() helper routines.
Previously some places were using the mmap()-specific PROT_* constants
instead.  While this happens to work because PROT_xx == VM_PROT_xx,
using VM_PROT_* is more correct.

Differential Revision:	https://reviews.freebsd.org/D2658
Reviewed by:	alc (glanced over), kib
MFC after:	1 month
Sponsored by:	Chelsio
2018-08-24 15:00:02 +02:00
jhb 60b466fbc2 Retire the unimplemented MAP_RENAME
and MAP_NORESERVE flags to mmap(2). Older binaries are still permitted
to use these flags.

PR:		193961 (exp-run in ports)
Differential Revision:	https://reviews.freebsd.org/D848
Reviewed by:	kib
2018-08-24 15:00:02 +02:00
jhb 3d5043e2cb Add a new fo_fill_kinfo fileops method
to add type-specific information to struct kinfo_file. - Move the
various fill_*_info() methods out of kern_descrip.c and into the various
file type implementations. - Rework the support for kinfo_ofile to
generate a suitable kinfo_file object for each file and then convert
that to a kinfo_ofile structure rather than keeping a second, different
set of code that directly manipulates type-specific file information. -
Remove the shm_path() and ksem_info() layering violations.

Differential Revision:	https://reviews.freebsd.org/D775
Reviewed by:	kib, glebius (earlier version)
2018-08-24 15:00:02 +02:00
kib de24ef326d Add MAP_EXCL flag for mmap(2).
It should be combined with MAP_FIXED, and prevents the request from
deleting existing mappings in the region, failing instead.

Reviewed by:	alc
Discussed with:	jhb
Tested by:	markj, pho (previous version, as part of the bigger patch)
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2018-08-24 15:00:02 +02:00
jhb 472476a5a7 Add a mmap flag (MAP_32BIT) on 64-bit platforms
to request that a mapping use an address in the first 2GB of the
process's address space. This flag should have the same semantics as the
same flag on Linux.

To facilitate this, add a new parameter to vm_map_find() that specifies an
optional maximum virtual address.  While here, fix several callers of
vm_map_find() to use a VMFS_* constant for the findspace argument instead of
TRUE and FALSE.

Reviewed by:	alc
Approved by:	re (kib)
2018-08-24 15:00:02 +02:00
kib e6a85661ce Implement read(2)/write(2) and neccessary lseek(2)
for posix shmfd. Add MAC framework entries for posix shm read and write.

Do not allow implicit extension of the underlying memory segment past
the limit set by ftruncate(2) by either of the syscalls.  Read and
write returns short i/o, lseek(2) fails with EINVAL when resulting
offset does not fit into the limit.

Discussed with:	alc
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
2018-08-24 15:00:02 +02:00
Sebastian Huber 5d29023c11 Add __nl_item to <sys/_types.h> and use it
Add __nl_item to <sys/_types.h> for FreeBSD compatibility.  Use it in
<langinfo.h> and the Cygwin <nl_types.h>.  Make the enum __nl_item in
<langinfo.h> anonymous.

Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2018-08-24 15:00:01 +02:00
Sebastian Huber 3bc64b88c3 RTEMS: Add __tls_get_addr() to crt0
Add __tls_get_addr() for all targets to crt0.  This is not only used on
ARM.  In particular, it is used on RISC-V.  This helps to adequately
support the GCC libgomp.

Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de
2018-08-20 13:54:22 +02:00
Masamichi Hosoda c8d4c99ecd Fix strtof ("-nan") returns positive NaN
strtof ("-nan") returned positive NaN instead of negative NaN.
strtod ("-nan") and strtold ("-nan") return negative NaN.

Linux glibc has been fixed
that strto{f|d|ld} ("-nan") returns negative NaN.
https://sourceware.org/bugzilla/show_bug.cgi?id=23007

This commit makes strtof preserves the negative sign bit
when parsing "-nan" like glibc.
2018-08-16 13:17:44 +02:00
Masamichi Hosoda 4c8fa88e4d Remove unused NaN's integer representation definitions
By previous commit, strto{d|ld} ("nan")
does not use the definition of NaN.
There is no other function that uses the definitions.

This commit remove the definitions.
2018-08-16 13:17:44 +02:00
Masamichi Hosoda 6c212a8b78 Fix strtod ("nan") and strtold ("nan") returns wrong negative NaN
The definition of qNaN for x86_64 and i386 was wrong.
strto{d|ld} ("nan") returned wrong negative NaN
instead of correct positive NaN
since it used the wrong definition.

On the other hand, strtof ("nan") returns correct positive NaN
since it uses nanf ("") instead of the wrong definition.

This commit makes strto{d|ld} ("nan") uses {nan|nanl} ("")
like strtof ("nan") using.
So strto{d|ld} ("nan") returns positive NaN.
2018-08-16 13:17:44 +02:00
Aditya Upadhyay a9a4554723 Added Restriction on base value 2018-08-13 09:42:21 +02:00
Corinna Vinschen b9dab9f4c6 newlib: strftime: fix over-enthusiastic fix from 0283642f35
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
2018-08-08 23:59:34 +02:00
Corinna Vinschen 6f485ba026 newlib: don't check malloc/free pointer
use preprocessor check for MALLOC_PROVIDED instead

Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
2018-08-08 10:50:19 +02:00
Corinna Vinschen 2d87d95f12 newlib: fix various gcc warnings
* unused variables
* potentially used uninitialized
* suggested bracketing
* misleading indentation

Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
2018-08-08 10:50:19 +02:00
Corinna Vinschen 5ace9004d9 newlib: wordexp: drop dangerous fprintf
wordexp uses fprintf in a dangerous way.  It uses an unchecked
input string as format string, rather than as parameter to a %s.
Replace fprintf with fputs.

Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
2018-08-08 10:50:19 +02:00
Corinna Vinschen ccea3b45b0 newlib: newlocale: fix typo rendering ctype_ptr invalid
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
2018-08-08 09:26:32 +02:00
Corinna Vinschen 0283642f35 newlib: strftime: fix gcc warning on __ctloc
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
2018-08-08 09:26:32 +02:00
Sebastian Huber dedeea9965 RTEMS: Add aligned_alloc() to crt0
This is necessary to build the latest GCC 7 branch.

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=85904

Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2018-08-08 08:42:28 +02:00
Sebastian Huber c233d42264 Declare GNU-specific sched_getcpu()
This is a glibc extension.

Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2018-08-07 14:34:11 +02:00
Sebastian Huber 6158b30e3e RISC-V: Do not use _init/_fini
Introduce new host configuration variable "have_init_fini" which is set
to "yes" by default.  Override it for RISC-V to "no".

Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2018-07-30 10:47:57 +02:00
Sebastian Huber 62a5c6b02c Add attributes to allocator functions
Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2018-07-30 10:47:57 +02:00
Sebastian Huber a31a7bad6a FreeBSD compat. __alloc_size(), __alloc_align()
Restore FreeBSD compatibility for __alloc_size() and __alloc_align().

This is a follow-up to commit e494b56035.

Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2018-07-30 10:47:57 +02:00
pfg 271e856a49 Define a new __alloc_size2 attribute to complement the exiting support.
At least on GCC7 calling __alloc_size(x) twice is not equivalent to
calling using the attribute once with two arguments. The later is the
documented use in GCC documentation so add a new alloc_size(n, x)
alternative to cover for the few places where it is used: basically:
calloc(3), reallocarray(3) and  mallocarray(9).

Submitted by:	Mark Millard
MFC after:	3 days
Reference:
http://docs.freebsd.org/cgi/mid.cgi?F227842D-6BE2-4680-82E7-07906AF61CD7
2018-07-30 10:47:57 +02:00
kib 1736bd3003 Remove lint support from system headers and MD x86 headers.
Reviewed by:	dim, jhb
Discussed with:	imp
Sponsored by:	The FreeBSD Foundation
Differential revision:	https://reviews.freebsd.org/D13156
2018-07-30 10:47:57 +02:00
pfg 00a7ef9163 sys: further adoption of SPDX licensing ID tags.
Mainly focus on files that use BSD 3-Clause license.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.
2018-07-30 10:47:57 +02:00
ed@FreeBSD.org 84e294dc69 Make _Static_assert() work with GCC in older C++ standards.
GCC only activates C11 keywords in C mode, not C++ mode. This means
that when targeting an older C++ standard, we cannot fall back to using
_Static_assert(). In this case, do define _Static_assert() as a macro
that uses a typedef'ed array.

Discussed in:	r322875 commit thread
Reported by:	Mark MIllard
MFC after:	1 month
2018-07-30 10:47:57 +02:00
Sebastian Huber 0e5f252bc7 ctype: Avoid GCC note in towctrans_l.c
The previous version genenerated the following GCC note:

towctrans_l.c:44:1: note: offset of packed bit-field 'diff' has changed in GCC 4.4
 caseconv_table [] = {
 ^~~~~~~~~~~~~~

Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2018-07-30 10:47:57 +02:00
Sebastian Huber d8d18c3e80 ctype: Fix integer type for caseconv_entry::delta
The commit 46ba1675c4 accidently changed a
bit-field from signed to unsigned.  The caseconv_entry::delta must be a
signed integer, see also "newlib/libc/ctype/caseconv.t".

Unfortunately, a standard GCC/Newlib build is done without
-Wsign-conversion.  Using this warning option would have helped to avoid
this bug:

caseconv.t:2:22: warning: unsigned conversion from 'int' to 'unsigned int:17' changes value from '-32' to '131040' [-Wsign-conversion]
   {0x0061, 25, TOUP, -32},

Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2018-07-30 10:47:57 +02:00
Jordi Sanfeliu 613a7f9036 Fix comparison between two character arrays
Hello,

The member 'id' in the 'utmp' struct is not a numeric but a character array,
hence the strncmp() function is needed to compare two members.
2018-07-30 09:36:49 +02:00
Yaakov Selkowitz f6417be2aa Cygwin: define _POSIX_ASYNCHRONOUS_IO
This feature is now available as of the recent AIO commits.

Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2018-07-25 10:48:55 -05:00
Sebastian Huber 46ba1675c4 ctype: Fix bitfield types on 16-bit targets
This prevents errors like this:

newlib/libc/ctype/categories.c:6:3: error: width of 'first' exceeds its type
   unsigned int first: 24;
   ^

Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2018-07-20 14:25:03 +02:00
Sebastian Huber 916ef5fb88 RTEMS: Unconditionally define _off_t to int64_t
Exotic RTEMS targets can define this back to int32_t as an exception if
there are good reasons.

Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2018-07-20 06:53:21 +02:00
Kumar Gala 6d7e0b337c Print sign of NaN values to nano-vfprintf.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2018-07-17 14:41:59 +02:00
Jordi Sanfeliu ab640f4cd5 Fix to stop a fall-through in a switch statement
The following fixes a fall-through that prevented from reading the
next entry in the UTMP file and terminated the program with an abort().
2018-07-13 13:33:33 +02:00
Siddhesh Poyarekar d02cc7a09d strcmp.S: Improve performance for misaligned strings
Replace the simple byte-wise compare in the misaligned case with a
dword compare with page boundary checks in place.  For simplicity I've
chosen a 4K page boundary so that we don't have to query the actual
page size on the system.

This results in up to 3x improvement in performance in the unaligned
case on falkor and about 2.5x improvement on mustang as measured using
bench-strcmp in glibc.
2018-07-13 13:27:54 +02:00
Siddhesh Poyarekar 2d9f35c2cc memcmp.S: optimize for medium to large sizes
This improved memcmp provides a fast path for compares up to 16 bytes
and then compares 16 bytes at a time, thus optimizing loads from both
sources.  The glibc memcmp microbenchmark retains performance (with an
error of ~1ns) for smaller compare sizes and reduces up to 31% of
execution time for compares up to 4K on the APM Mustang.  On Qualcomm
Falkor this improves to almost 48%, i.e. it is almost 2x improvement
for sizes of 2K and above.
2018-07-13 13:27:54 +02:00
Siddhesh Poyarekar f44eee8f1b Improve strncmp for mutually misaligned inputs
The mutually misaligned inputs on aarch64 are compared with a simple
byte copy, which is not very efficient.  Enhance the comparison
similar to strcmp by loading a double-word at a time.  The peak
performance improvement (i.e. 4k maxlen comparisons) due to this on
the strncmp microbenchmark in glibc is as follows:

falkor: 3.5x (up to 72% time reduction)
cortex-a73: 3.5x (up to 71% time reduction)
cortex-a53: 3.5x (up to 71% time reduction)

All mutually misaligned inputs from 16 bytes maxlen onwards show
upwards of 15% improvement and there is no measurable effect on the
performance of aligned/mutually aligned inputs.
2018-07-13 13:27:54 +02:00
Szabolcs Nagy 393a1cb4ea Move __HAVE_FAST_FMA to math_config.h
Define it consistently with other HAVE_* macros that only affect code
using math_config.h.  This is also closer to the Arm Optimized Routines
code.
2018-07-06 10:29:01 +02:00
Takashi Yano 6a3e08a53e Fix newlib functions perror()/psignal() not to use writev().
This fix is for some platforms which do not have writev().
*perror.c: Use _write_r() instead of writev().
*psignal.c: Use write() insetad of writev().

Revise commit: d4f4e7ae1b
2018-07-05 15:33:49 -04:00
Takashi Yano d4f4e7ae1b Fix a bug of perror()/psignal() that changes the orientation of stderr.
* perror.c: Fix the problem that perror() changes the orientation
  of stderr to byte-oriented mode if stderr is not oriented yet.
* psignal.c: Ditto.
2018-07-04 14:17:28 +02:00
Corinna Vinschen 006520ca2b newlib: enable new math functions on Cygwin
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
2018-06-27 15:53:51 +02:00
Szabolcs Nagy e5791079c6 New log implementation
The new implementations are provided under !__OBSOLETE_MATH, it uses
ISO C99 code.  With default settings the worst case error in nearest
rounding mode is 0.519 ULP with inlined fma and fma contraction.  It uses
a 2 KB lookup table, on aarch64 .text+.rodata size of libm.a is increased
by 1703 bytes.  The w_log.c wrapper is disabled since error handling is
inline in the new code.

New __HAVE_FAST_FMA and __HAVE_FAST_FMA_DEFAULT feature macros were
added to enable selecting between the code path that uses fma and the
one that does not.  Targets supposed to set __HAVE_FAST_FMA_DEFAULT
if they have single instruction fma and the compiler can actually
inline it (gcc has __FP_FAST_FMA macro but that does not guarantee
inlining with -fno-builtin-fma).

Improvements on Cortex-A72:
latency: 1.9x
thruput: 2.3x
2018-06-27 15:40:49 +02:00
Takashi Yano 048490485a Fix Unicode table.
* (mkcategories): Fix a bug that outputs incorrect Unicode category
  table for code point ranges.
* (categories.t): Rebuild it using the bug-fixed mkcategories.

This fixes the problem reported in the following post.
https://cygwin.com/ml/cygwin/2018-06/msg00248.html
2018-06-26 10:19:12 +02:00
Thomas Kindler 9dd3c3b0ad newlib: getopt now permutes multi-flag options correctly
Previously, "test 1 2 3 -a -b -c"  was permuted to "test -a -b -c 1 2 3",
but "test 1 2 3 -abc" was left as "test 1 2 3 -abc".

Signed-off-by: Thomas Kindler <mail+newlib@t-kindler.de>
2018-06-18 18:45:44 +02:00
Jeff Johnston 4a3d0a5a5d Fix issue with malloc_extend_top
- when calculating a correction to align next brk to page boundary,
  ensure that the correction is less than a page size
- if allocating the correction fails, ensure that the top size is
  set to brk + sbrk_size (minus any front alignment made)

Signed-off-by: Jeff Johnston <jjohnstn@redhat.com>
2018-05-29 10:16:48 -04:00
Freddie Chopin 3305f35570 Fix 32-bit overflow in mktime() when time_t is 64-bits long
When converting number of days since epoch (32-bits) to seconds,
calculations using 32-bit `long` overflow for years above 2038. Solve
this by casting number of days to `time_t` just before final
multiplication.

Signed-off-by: Freddie Chopin <freddie.chopin@gmail.com>
2018-05-29 15:27:03 +02:00
Jeff Johnston cd31fbb2ae Add nvptx port.
- From: Cesar Philippidis <cesar@codesourcery.com>
  Date: Tue, 10 Apr 2018 14:43:42 -0700
  Subject: [PATCH] nvptx port

  This port adds support for Nvidia GPU's, which are primarily used as
  offload accelerators in OpenACC and OpenMP.
2018-04-13 15:42:37 -04:00
Corinna Vinschen 27652b608d strtod: Convert 64 bit double to 64 bit int during computation
The gdtoa implementation uses the type long, defined as Long, in lots
of code.  For historical reason newlib defines Long as int32_t instead.

This works fine, as long as floating point exceptions are not enabled.
The conversion to 32 bit int can lead to a FE_INVALID situation.

Example:

  const char *str = "121645100408832000.0";
  char *ptr;

  feenableexcept (FE_INVALID);
  strtod (str, &ptr);

This leads to the following situation in strtod

  double aadj;
  Long L;

  [...]
  L = (Long)aadj;

For instance, on x86_64 the code here is

  cvttsd2si %xmm0,%eax

At this point, aadj is 2529648000.0 in our example.  The conversion to
32 bit %eax results in a negative int value, thus the conversion is
invalid.  With feenableexcept (FE_INVALID), a SIGFPE is raised.

Fix this by always using 64 bit ints here if double is not a 32 bit type
to avoid this type of FP exceptions.

Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
2018-04-09 11:31:04 +02:00
Corinna Vinschen 1ee6654e50 newlib: fix iswupper_l in !_MB_CAPABLE case
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
2018-03-27 12:35:27 +02:00
Thomas Wolff fc59da00c8 comments to document struct caseconv_entry
explain design of compact (packed) struct caseconv_entry,
in case it needs to be modified for future Unicode versions
2018-03-26 12:01:50 +02:00
Thomas Wolff b49ce5af1b newlib: fix indentation in toulower
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
2018-03-26 10:00:16 +02:00
Hakan Lindqvist 3ce38df8d1 Reduce qsort stack consumption
Classical function call recursion wastes a lot of stack space.
Each recursion level requires a full stack frame comprising all
local variables and additional space as dictated by the
processor calling convention.

This implementation instead stores the variables that are unique
for each recursion level in a parameter stack array, and uses
iteration to emulate recursion. Function call recursion is not
used until the array is full.

To ensure the stack consumption isn't worsened by this design, the
size of the parameter stack array is chosen to be similar to the
stack frame excluding the array. Each function call recursion level
can handle 8 iterative recursion levels.

Stack consumption will worsen when sorting tiny arrays that do not
need recursion (of 6 elements or less). It will be about equal for
up to 15 elements, and be an improvement for larger arrays. The best
case improvement is a stack size reduction down to about one quarter
of the stack consumption before the change.

A design where the parameter stack array is large enough for the
worst case recursion level was rejected because it would worsen
the stack consumption when sorting arrays smaller than about 1500
elements. The worst case is 31 levels on a 32-bit system.

A design with a dynamic parameter array size was rejected because
of limitations in some compilers.
2018-03-16 10:21:23 +01:00
Hakan Lindqvist 0045445ad6 Ensure qsort recursion depth is bounded
The qsort algorithm splits the input array in three parts. The
left and right parts may need further sorting. One of them is
sorted by recursion, the other by iteration. This update ensures
that it is the smaller part that is chosen for recursion.

By choosing the smaller part, each recursion level will handle
less than half the array of the previous recursion level. Hence
the recursion depth is bounded to be less than log2(n) i.e. 1
level per significant bit in the array size n.

The update also includes code comments explaining the algorithm.
2018-03-16 10:21:23 +01:00
Joel Sherrill 948db3e4b7 Correct prototypes of pthread_mutex_getprioceiling() and pthread_setschedparam() 2018-03-15 09:25:45 -05:00
Richard Earnshaw 0bb8697333 [arm] Fix syscalls.c for newlib embedded syscalls builds
Newlib has a build configuration where syscalls can be directly
embedded in the newlib library rather than relying on libgloss.

This configuration was broken recently by an update to the libgloss
support for Arm that was not propagated to the syscalls interface in
newlib itself.  This patch restores the build.  It's essentially a
copy of https://sourceware.org/ml/newlib/2018/msg00128.html but there
are some other minor cleanups and changes that I've made at the same
time.  None of those cleanups affect functionality.

The prototypes of the following functions have been updated: _link,
_sbrk, _getpid, _write, _swiwrite, _lseek, _swilseek, _read and
_swiread.

Signed-off-by: Richard Earnshaw <Richard.Earnshaw@arm.com>
2018-03-15 09:55:11 +00:00
Yaakov Selkowitz 829820af6e ssp: fix wchar.h with -std=c99
https://sourceware.org/ml/newlib/2018/msg00261.html

Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2018-03-14 10:46:32 -05:00
Yaakov Selkowitz e494b56035 Fix alloc_align and alloc_size macros for multiple arguments
https://sourceware.org/ml/newlib/2018/msg00263.html

This is a follow-up to commit 4564b30f33.

Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2018-03-14 10:17:51 -05:00
Corinna Vinschen 134f93f313 ctype: align size of category bit fields to small targets needs
E.g. arm ABI requires -fshort-enums for bare-metal toolchains.
Given there are only 29 category enums, the compiler chooses an
8 bit enum type, so a size of 11 bits for the bitfield leads to
a compile time error:

  error: width of 'cat' exceeds its type
    enum category cat: 11;
                  ^~~

Fix this by aligning the size of the category members to byte
borders.

Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
2018-03-14 11:38:24 +01:00
Corinna Vinschen edcf783dc2 Revert "ctype: align size of category bit fields to small targets needs"
This reverts commit e98d3eb3eb.

It has accidentally included some work in progress.
2018-03-14 11:36:06 +01:00
Thomas Wolff 44d90834fb fix/enhance Unicode table generation scripts
Scripts do not try to acquire Unicode data by best-effort magic anymore.
Options supported:
-h for help
-i to copy Unicode data from /usr/share/unicode/ucd first
-u to download Unicode data from unicode.org first
If (despite of -i or -u if given) the necessary Unicode files are not
available locally, table generation is skipped, but no error code is
returned, so not to obstruct the build process if called from a Makefile.
2018-03-14 10:44:32 +01:00
Corinna Vinschen e98d3eb3eb ctype: align size of category bit fields to small targets needs
E.g. arm ABI requires -fshort-enums for bare-metal toolchains.
Given there are only 29 category enums, the compiler chooses an
8 bit enum type, so a size of 11 bits for the bitfield leads to
a compile time error:

  error: width of 'cat' exceeds its type
    enum category cat: 11;
                  ^~~

Fix this by aligning the size of the category members to byte
borders.

Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
2018-03-14 10:36:38 +01:00
Corinna Vinschen e186dc8661 towctrans_l: Always return a value from helper functions
touupper and toulower didn't return a value in all cases.  Worse,
this only broke Cygwin when building without optimization for debug
purposes.

Why GCC neglects to notice this is a mystery.

While at it, fix formatting.

Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
2018-03-13 22:09:30 +01:00
Joel Sherrill 5b97e36239 rtems/.../dirent.h: Add alphasort() prototype 2018-03-13 09:11:47 -05:00
Jon Turney 4564b30f33 Correct alloc_size annotation on reallocarray()
Signed-off-by: Jon Turney <jon.turney@dronecode.org.uk>
2018-03-13 09:04:56 -05:00
Thomas Wolff a352730004 character data generation 2018-03-12 11:39:50 +01:00
Thomas Wolff 41f72ab4d7 use generated character data
The tow* functions use an included case conversion table which can be
generated from Unicode data.
The isw* functions use a character categories table (provided by
categories.c) which can be generated from Unicode data.
Delegation between current-locale and specific-locale-dependent functions
was reverted towards the generic locale-dependent functions (*_l.c);
this is however only relevant on systems with non-Unicode wide character
locales, thus not on Cygwin.
2018-03-12 11:39:42 +01:00
Thomas Wolff 3ccfb407af generated character category data, Unicode 10.0
Table categories.t and tag enumeration categories.cat provide
character class data for most of the isw* functions.
These data are generated from Unicode data.
2018-03-12 11:09:31 +01:00
Thomas Wolff 402daa2f80 generated case conversion data, Unicode 10.0
Table caseconv.t provides case conversion data for the tow* functions,
especially towupper and towlower.
These data are generated from Unicode data.
2018-03-12 11:09:31 +01:00
Thomas Wolff 37132125bc width data generation 2018-03-12 10:17:20 +01:00
Thomas Wolff 8e8fd6c849 use generated width data 2018-03-12 10:17:20 +01:00
Thomas Wolff 71291047e2 generated width data, Unicode 10.0
These tables provide character width properties for use by the
wcwidth/wcswidth functions. They are generated from Unicode.
2018-03-12 10:17:20 +01:00
Sebastian Huber f641474cb2 RTEMS: Use int for _CLOCKID_T_
Linux and FreeBSD use int as well.  In addition, this fixes an Ada
incompatiblity problem on 64-bit targets.  See also GCC:

  gcc/ada/libgnarl/s-osinte__rtems.ads

Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2018-03-06 11:40:16 +01:00
Sebastian Huber a9c8434527 Make _CLOCKID_T_ system configurable
Let systems optionally provide the _CLOCKID_T_ type via
<machine/_types.h>.

Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2018-03-06 11:40:16 +01:00
Thomas Wolff f92f048528 Locale modifier @cjkwide to adjust ambiguous-width in non-CJK locales
Locale modifier @cjkwide makes Unicode "ambiguous width" characters
wide.  So ambiguous width characters can be enforced to have width 2
even in non-CJK locales. This gives e.g. users of "Powerline symbols"
the opportunity to adjust their width to the desired behaviour (and the
behaviour apparently expected by some tools) without having to set a CJK
locale and without losing consistence of terminal character width with
wcwidth/wcswidth locale width.
2018-03-05 17:15:12 +01:00
Our Air Quality b7520b14d5 Add global stdio streams support for reent small. 2018-03-01 18:05:31 -05:00
Jaap de Wolff 8329f4867b add forward declaration to __cxa_atexit to aeabi_atexit, to prevent warnings 2018-02-16 12:16:07 +01:00
Jaap de Wolff 337cee51ca Add prototype to _malloc_lock() and *unlock() to malloc.h, and inlude this from nano-mallocr.c 2018-02-16 12:16:07 +01:00
Corinna Vinschen 4c73ad6b20 newlib: drop Cygwin license from sys/select.h
This license was accidentally retained when moving the file from Cygwin to newlib.

Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
2018-02-02 11:27:58 +01:00
Jon Beniston b8272e3b8d Fix vprintf and vfscanf for GCC PR 14577 2018-02-01 13:28:28 +01:00
Sebastian Huber 1658a57715 epiphany: Additional setjmp() and longjmp() syms
At least with Binutils 2.30 and GCC 7.3 we need symbol definitions
without the leading underscore.

Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2018-01-31 08:17:19 +01:00
Yaakov Selkowitz b920561fe3 ssp: do not use __ssp_protected_ symbol prefixes
This is a NetBSD-specific detail which does not apply to Newlib, causing
linking issues in certain scenarios:

https://cygwin.com/ml/cygwin/2018-01/msg00189.html

Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2018-01-26 11:51:15 -06:00
Thomas Preudhomme 7d09d0e261 Disable powf/log2?f/exp2?f optimization for single-precision Arm FPU
New optimized powf, logf, log2f, expf and exp2f yield worse performance
on Arm targets with only single precision instructions because the
double precision arithmetic is then implemented via softfloat routines.
This patch uses the old implementation when double precision
instructions are not available on Arm targets.

Testing: Built newlib with GCC's rmprofile Arm multilibs and compared
before/after -> only the above functions are changed and calls to them
(name change from logf to __ieee754_logf and similar). Testing the
changed function on a panel of values yields the same result before the
original patches to improve them and after this one. Double checking the
performance by looping the same panel of values being tested on Arm
Cortex-M4 does show the performance regression is fixed.
2018-01-25 16:08:35 +01:00
Yaakov Selkowitz bd62f539de Guard langinfo.h nl_item from multiple typedefs
This is a prerequisite of adding nl_types.h support to Cygwin.

Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2018-01-19 13:19:21 -06:00
Orlando Arias b7e0f286a2 Fix syntax error in exit.c
This patch fixes a syntax error in exit.c that was introduced during the
ANSI-fication of newlib. The patch fixes a compile-time issue that arises when
newlib is configured with the --enable-lite-exit feature.
2018-01-19 19:58:10 +01:00
Jeff Johnston fffd2770db Bump release to 3.0.0 for yearly snapshot
- major release required due to removal of K&R support
2018-01-18 13:07:45 -05:00
Kito Cheng cba678ba02 RISC-V: Fix alignment issue in sigjmp_buf 2018-01-18 09:21:10 +01:00
Yaakov Selkowitz 7192f84096 ansification: remove _HAVE_STDC
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2018-01-17 11:47:30 -06:00
Yaakov Selkowitz 70ee6b17df ansification: remove _EXFUN, _EXFUN_NOTHROW
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2018-01-17 11:47:29 -06:00
Yaakov Selkowitz 77f16db546 ansification: remove _EXFNPTR, _EXPARM
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2018-01-17 11:47:27 -06:00
Yaakov Selkowitz 9087163804 ansification: remove _DEFUN
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2018-01-17 11:47:26 -06:00
Yaakov Selkowitz 44276afe2a ansification: remove _VOLATILE, _SIGNED
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2018-01-17 11:47:21 -06:00
Yaakov Selkowitz 67ee0cac4c ansification: remove _VOID
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2018-01-17 11:47:20 -06:00
Yaakov Selkowitz fff27f8429 ansification: remove _DEFUN_VOID
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2018-01-17 11:47:19 -06:00
Yaakov Selkowitz 670b01da7f ansification: remove _CAST_VOID
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2018-01-17 11:47:17 -06:00
Yaakov Selkowitz e6321aa6a6 ansification: remove _PTR
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2018-01-17 11:47:16 -06:00
Yaakov Selkowitz 0403b9c8c4 ansification: remove _VOID_PTR
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2018-01-17 11:47:14 -06:00
Yaakov Selkowitz eea249da3b ansification: remove _PARAMS
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2018-01-17 11:47:13 -06:00
Yaakov Selkowitz e13e191b60 ansification: remove _NOARGS
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2018-01-17 11:47:11 -06:00
Yaakov Selkowitz 2310096fbc ansification: remove _DOTS
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2018-01-17 11:47:10 -06:00
Yaakov Selkowitz 0bda30e1ff ansification: remove _CONST
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2018-01-17 11:47:08 -06:00
Yaakov Selkowitz 6783860a2e ansification: remove _AND
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2018-01-17 11:47:05 -06:00
Yaakov Selkowitz 82339fcd71 Make __always_inline macro compatible with glibc
For example, this is used when cross-compiling the Linux kernel on Cygwin.

Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2018-01-16 13:24:42 -06:00
Yaakov Selkowitz 2cb24159fb cygwin: add LFS_CFLAGS etc. to confstr/getconf
These are used, for instance, when cross-compiling the Linux kernel.

Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2018-01-16 12:12:45 -06:00
Ivan Grokhotkov 1d01586b62 newlib: fvprintf: fix get_arg for !_MB_CAPABLE
Code path for _MB_CAPABLE scans for the '%' character and advances
'fmt' pointer past '%'. Code path for !_MB_CAPABLE leaved fmt pointing
to '%', which caused the state machine to go from START to DONE state
immediately.
2018-01-09 09:52:18 +01:00
Alexander Fedotov fcd33916ac fix incompatible pointer type for va_list in nano versions of printf and scanf for target like PowerPC 2018-01-08 11:05:05 +01:00
Corinna Vinschen 6e5b39940a newlib: ftello{64}: Handle appending stream without fflushing
Neither upstream FreeBSD nor glibc ever call fflush from ftell
and friends.  In border cases it has the tendency to return
wrong or unexpected values, for instance on block devices.

Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
2017-12-18 20:17:51 +01:00
Corinna Vinschen dc2d175721 newlib: ftello{64}: Fix type of returned value
Especially don't just use -1L since _off_t/_off64_t are not
guaranteed to be of type long.

Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
2017-12-18 20:15:27 +01:00
Martin Aberg 1251555311 newlib: Availability of _kill() in sys/signal.h
Make prototype of _kill() always visible when _COMPILING_NEWLIB is
defined. This makes <sys/signal.h> consistent with the use of
_COMPILING_NEWLIB in <sys/unistd.h>, <sys/times.h>, etc.
2017-12-18 19:58:05 +01:00
Jon Turney c006fd459f makedoc: make errors visible
Discard QUICKREF sections, rather than writing them to stderr
Discard MATHREF sections, rather than discarding as an error
Pass NOTES sections through to texinfo, rather than discarding as an error
Don't redirect makedoc stderr to .ref file
Remove makedoc output on error
Remove .ref files from CLEANFILES
Regenerate Makefile.ins

Signed-off-by: Jon Turney <jon.turney@dronecode.org.uk>
2017-12-07 11:54:11 +00:00
Corinna Vinschen 67e628fa33 newlib: vfwscanf: fix negation bug in %[ conversion
Old BSD bug:  While ^ is recognized and the set of matching characters
is negated, the code neglects to increment the pointer pointing to the
matching characters.  Thus, on a negation expression like %[^xyz], the
matching doesn't only stop at x, y, or z, but incorrectly also on ^.

Fix this by setting the start pointer after recognizing the ^.

Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
2017-12-04 17:05:11 +01:00
Yaakov Selkowitz 8f7c712bb8 ssp: add Object Size Checking for wchar.h, part 1
The following functions are also guarded in glibc:
fwprintf, swprintf, wprintf, vfwprintf, vswprintf, vwprintf.

Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2017-12-04 04:09:46 -06:00
Corinna Vinschen b7b6b42cd4 newlib: vfscanf: Implement %l[
Just as %lc and %ls, this is only enabled on ELIX_LEVEL >= 2.

Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
2017-12-01 17:53:59 +01:00
Corinna Vinschen a49209d2bc newlib: vf[w]scanf: Fix conversion multibyte <-> wchar_t
* vfscanf: per POSIX, if the target type is wchar_t, the width is
  counted in (multibyte) characters, not in bytes.

* vfscanf: Handle UTF-8 multibyte sequences converted to surrogate
  pairs on UTF-16 systems.

* vfwscanf: Don't count high surrogates in input against field width
  counting.  Per POSIX, input is

Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
2017-12-01 17:18:26 +01:00
Corinna Vinschen 9638c07527 newlib: vf[w]scanf: Drop width computation mixup
The width value keeps the maximum field width.  This is the maximum
field width of the *input*.  It's *never* to be used in conjunction
with the number of bytes or characters written to the output argument.

However, especially in vfwscanf, the code is partially taken from
NetBSD which erroneously subtracts the number of multibyte chars
written to the argument from the width variable, thus potentially
subtracting up to MB_CUR_MAX from width for a single character in
the input stream.

To make matters worse, the previous patch adding %m added basically
the same mistake for 'c' type input.

Fix it.

Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
2017-12-01 13:47:26 +01:00
Yaakov Selkowitz 571ff42ae4 unix: remove TRAD_SYNOPSIS
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2017-12-01 03:41:53 -06:00
Yaakov Selkowitz 5aa2434de0 time: remove TRAD_SYNOPSIS
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2017-12-01 03:41:53 -06:00
Yaakov Selkowitz adfde9d773 sys: remove TRAD_SYNOPSIS
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2017-12-01 03:41:52 -06:00
Yaakov Selkowitz 352c8f2f0d string: remove TRAD_SYNOPSIS
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2017-12-01 03:41:52 -06:00
Yaakov Selkowitz a38fc79ee9 stdlib: remove TRAD_SYNOPSIS
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2017-12-01 03:41:52 -06:00
Yaakov Selkowitz a60026253d stdio64: remove TRAD_SYNOPSIS
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2017-12-01 03:41:51 -06:00
Yaakov Selkowitz c7ef9668cf stdio: remove TRAD_SYNOPSIS
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2017-12-01 03:41:51 -06:00
Yaakov Selkowitz b88cfbc1e5 signal: remove TRAD_SYNOPSIS
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2017-12-01 03:41:51 -06:00
Yaakov Selkowitz bf3a554bc6 search: remove TRAD_SYNOPSIS
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2017-12-01 03:41:51 -06:00
Yaakov Selkowitz 35d2d2fe2e reent: remove TRAD_SYNOPSIS
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2017-12-01 03:41:51 -06:00
Yaakov Selkowitz aa06fa01dc posix: remove TRAD_SYNOPSIS 2017-12-01 03:41:50 -06:00
Yaakov Selkowitz 191b4f35bc misc: remove TRAD_SYNOPSIS
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2017-12-01 03:41:50 -06:00
Yaakov Selkowitz 1f1e477554 powerpc: remove TRAD_SYNOPSIS
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2017-12-01 03:41:50 -06:00
Yaakov Selkowitz ddd22ee069 nds32: remove TRAD_SYNOPSIS
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2017-12-01 03:41:50 -06:00
Yaakov Selkowitz 4e8c64b928 microblaze: remove TRAD_SYNOPSIS
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2017-12-01 03:41:50 -06:00
Yaakov Selkowitz 59235deeec locale: remove TRAD_SYNOPSIS
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2017-12-01 03:41:50 -06:00
Yaakov Selkowitz 03973c19e9 iconv: remove TRAD_SYNOPSIS
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2017-12-01 03:41:50 -06:00
Yaakov Selkowitz 9db7609592 ctype: remove TRAD_SYNOPSIS
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2017-12-01 03:41:50 -06:00
Yaakov Selkowitz 8ba0bbb913 ssp: add Object Size Checking for unistd.h, part 2
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2017-12-01 03:41:49 -06:00
Corinna Vinschen d43863f569 newlib: vf[w]scanf: Implement POSIX %m modifier
* The new code is guarded with _WANT_IO_POSIX_EXTENSIONS, but
  this is automatically enabled with _WANT_IO_C99_FORMATS for now.

* vfscanf neglects to implement %l[, so %ml[ is not implemented yet
  either.

* Sidenote: vfwscanf doesn't allow ranges in %[ yet.  Strictly this
  is allowed per POSIX, but it differes from vfscanf as well as from
  glibc.

Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
2017-11-30 21:50:23 +01:00
Corinna Vinschen 7161622514 newlib: vfwscanf: fix miscomputation of max field width in %[] case
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
2017-11-30 21:02:38 +01:00
Yaakov Selkowitz b9a662bb4c Feature test macros overhaul: unistd.h, part 3
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2017-11-30 12:09:05 -06:00
Yaakov Selkowitz 1e43e181c2 ssp: add Object Size Checking for stdlib.h
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2017-11-30 12:09:03 -06:00
Corinna Vinschen 0fd2c9bd12 newlib: vf[w]scanf: add validity checks
POSIX requires that directive characters appear in a certain sequence:

1. '%' or '%<n>$'
2. optional '*'
3. optional field width digits
4. optional 'm' (not yet implemented)
5. optional length modifier ('l', 'L', 'll', 'h', 'hh', 'j', 't', 'z')
6. conversion specifier ('d', 's', etc)

Add a few basic validity checks to that effect, otherwise reject
directive as match failure.

Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
2017-11-30 11:55:27 +01:00
Corinna Vinschen 31f11d0572 newlib: vf[w]scanf: Use SIZE_MAX rather than ~0
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
2017-11-30 11:41:36 +01:00
Corinna Vinschen 5e4a1c9c97 newlib: vfscanf: fix formatting
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
2017-11-30 11:41:36 +01:00
Corinna Vinschen 2e328edee4 newlib: vf[w]scanf: Only return from a single point to simplify cleanup
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
2017-11-30 11:41:36 +01:00
Yaakov Selkowitz 8a94bca694 string: add wmempcpy
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2017-11-30 04:06:49 -06:00
Sebastian Huber dadc9e7e4a RTEMS: Add semaphore <sys/lock.h> functions
Declare semaphore try wait and post binary functions.

Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2017-11-30 07:00:45 +01:00
Sebastian Huber 5a2ab9d55e RTEMS: Timed wait by ticks <sys/lock.h> functions
Declare timed wait by ticks functions.

Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2017-11-30 07:00:45 +01:00
Sebastian Huber 186166f67a RTEMS: Add set/get name <sys/lock.h> functions
Add inline functions to set/get the name.

Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2017-11-30 07:00:45 +01:00
Yaakov Selkowitz 6f84ee8105 ssp: add build infrastructure
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2017-11-29 11:25:46 -06:00
Yaakov Selkowitz 192de5a349 ssp: add documentation
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2017-11-29 11:25:45 -06:00
Yaakov Selkowitz 6b02865d80 ssp: add Object Size Checking for unistd.h, part 1
The implementation is from NetBSD, with the addition of feature test macros
for readlink.  glibc also wraps the following functions:
confstr, getdomainname, getgroups, gethostname, getlogin_r, getwd, pread,
readlinkat, ttyname_r.

Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2017-11-29 11:25:44 -06:00
Yaakov Selkowitz 576093d46b ssp: add Object Size Checking for stdio.h, part 1
The implementation is mostly from NetBSD, except for switching fgets to
pure inline, and the addition of fgets_unlocked, fread, and fread_unlocked
for parity with glibc.  The following functions are also guarded in glibc:
asprintf, dprintf, fprintf, printf, vasprintf, vdprintf, vfprintf, vprintf.

Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2017-11-29 11:25:42 -06:00
Yaakov Selkowitz a997f98b2a ssp: add Object Size Checking for strings.h
The implementation is from NetBSD, with the addition of explicit_bzero
for parity with glibc.

Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2017-11-29 11:25:41 -06:00
Yaakov Selkowitz e4fc4d7bc4 ssp: add Object Size Checking for string.h
The implementation is from NetBSD, with the addition of mempcpy (a GNU
extension) for parity with glibc and libssp.

Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2017-11-29 11:25:40 -06:00
Yaakov Selkowitz 3e8fc7d9f2 ssp: add Object Size Checking common code
The Object Size Checking (-D_FORTIFY_SOURCE=*) functionality provides
wrappers around functions suspectible to buffer overflows.  While
independent from Stack Smashing Protection (-fstack-protector*), they
are often used and implemented together.

While GCC also provides an implementation in libssp, it is completely
broken (CVE-2016-4973, RHBZ#1324759) and seemingly unfixable, as there
is no reliable way for a preprocessor macro to trigger a link flag.
Therefore, adding this here is necessary to make it work.

Note that this does require building gcc with --disable-libssp and
gcc_cv_libc_provides_ssp=yes.

Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2017-11-29 11:25:39 -06:00
Yaakov Selkowitz 0a5dfdbd1b ssp: add APIs for Stack Smashing Protection
Compiling with any of the -fstack-protector* flags requires the
__stack_chk_guard data import (which needs to be initialized) and the
__stack_chk_fail{,_local} functions.  While GCC's own libssp can provide
these, it is better that we provide these ourselves.  The implementation
is custom due to being OS-specific.

Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2017-11-29 11:25:37 -06:00
Corinna Vinschen 1bbdb3c953 newlib: [w]scanf: Fix behaviour on matching failure
The special handling of %\0 in [w]scanf is flawed.  It's just a
matching failure and should be handled as such.  scanf also
fakes an int input value on %X with X being an invalid conversion
char.  This is also just a matching failure and should be handled
the same way as %\0.

There's no indication of the reason for this "disgusting
backwards compatibility hacks" in the logs, given this
code made it into newlib before setting up the CVS repo.

Just handle these cases identically as matching failures.

Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
2017-11-29 15:01:30 +01:00
Yaakov Selkowitz 36a0a675b4 Make __nonnull macro compatible with glibc
This form allows for multiple arguments, e.g. __nonnull((1,2)).

Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2017-11-28 10:26:29 -06:00
Yaakov Selkowitz f2b27ce620 cygwin: define _POSIX_TIMEOUTS
Since commit 8128f5482f, we have all the
non-tracing functions listed in posixoptions(7).  The tracing functions
are gated by their own option, and are obsolecent anyway.

Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2017-11-28 10:26:00 -06:00
Jeff Johnston 569d048c89 Add the Aarch64 SVE specific HWCAP_SVE define
Checking in change from Alan Hayward <Alan.Hayward@arm.com>

Signed-off-by: Jeff Johnston <jjohnstn@redhat.com>
2017-11-22 14:10:37 -05:00
Sebastian Huber 6480987f3a RTEMS: Add missing __getreent() to crt0
Default implementation was removed for RTEMS by
b1a388799d.

Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2017-11-20 10:12:18 +01:00
Sebastian Huber 0e3e375377 RTEMS: Add missing functions to crt0
This helps to get some more features from libstdc++.

Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2017-11-20 08:53:30 +01:00
Corinna Vinschen baaceb8f37 Cygwin: fcntl.h: Use test macros to guard non-standard open flags
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
2017-11-14 21:45:31 +01:00
Corinna Vinschen f9d071aaf8 Cygwin: fcntl.h: Define O_NOATIME
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
2017-11-14 21:45:31 +01:00
Corinna Vinschen 0aa99373c1 Cygwin: fcntl.h: Define O_TMPFILE and implement it
Difference to Linux: We can't create files which don't show up
in the filesystem due to OS restrictions.  As a kludge, make a
(half-hearted) attempt to hide the file in the filesystem.

Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
2017-11-14 21:45:25 +01:00
Florian Schmidt 9cf0c4a012 newlib/libc/stdlib/realloc.c: fix variable name
The variable doesn't follow the convention of having the same name as
the function it's bundled with. Furthermore, it clashes with the
variable of the same name in newlib/libc/stdlib/calloc.c.

Signed-off-by: Florian Schmidt <florian.schmidt@neclab.eu>
2017-11-14 10:18:30 +01:00
Joel Sherrill b1a388799d newlib/.../getreent.c: Allow to be provided by host and do so for RTEMS
RTEMS provides the option to have a global or per-thread reentrancy
as part of application configuration. As part of this, RTEMS provides
the implementation of __getreent() as appropriate. Allow the target
to determine if this method is present in libc.a.
2017-11-13 10:19:23 +01:00
Sebastian Huber 1f42dc2bcf Make ffsl() and ffsll() BSD-visible
Since glibc 2.27, they are visible via _DEFAULT_SOURCE (__USE_MISC):

https://git.kernel.org/pub/scm/docs/man-pages/man-pages.git/commit/man3/ffs.3?id=247bbcf00c9a425ab0ad6e303ec8718e4ba844a6

In FreeBSD, they are guarded by __BSD_VISIBLE.

Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2017-11-03 10:46:07 +01:00
Sebastian Huber ce189d8afe RTEMS: Remove internal timecounter API
Change copyright.  Original BSD content moved to
<machine/_kernel_time.h>.

Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2017-10-26 08:47:21 +02:00
Szabolcs Nagy c156098271 New expf, exp2f, logf, log2f and powf implementations
Based on code from https://github.com/ARM-software/optimized-routines/

This patch adds a highly optimized generic implementation of expf,
exp2f, logf, log2f and powf.  The new functions are not only
faster (6x for powf!), but are also smaller and more accurate.
In order to achieve this, the algorithm uses double precision
arithmetic for accuracy, avoids divisions and uses small table
lookups to minimize the polynomials.  Special cases are handled
inline to avoid the unnecessary overhead of wrapper functions and
set errno to POSIX requirements.

The new functions are added under newlib/libm/common, but the old
implementations are kept (in newlib/libm/math) for non-IEEE or
pre-C99 systems.  Targets can enable the new math code by defining
__OBSOLETE_MATH_DEFAULT to 0 in newlib/libc/include/machine/ieeefp.h,
users can override the default by defining __OBSOLETE_MATH.
Currently the new code is enabled for AArch64 and AArch32 with VFP.
Targets with a single precision FPU may still prefer the old
implementation.

libm.a size changes:
arm: -1692
arm/thumb/v7-a/nofp: -878
arm/thumb/v7-a+fp/hard: -864
arm/thumb/v7-a+fp/softfp: -908
aarch64: -1476
2017-10-13 10:58:00 +02:00
Sebastian Huber c165a27c01 RTEMS: Fix _PTHREAD_MUTEX_INITIALIZER
Add missing braces around initializer.

Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2017-10-13 08:07:13 +02:00
Sebastian Huber 3a79700c2d RTEMS: Make pthread_mutex_t self-contained
Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2017-10-05 14:56:13 +02:00
Sebastian Huber 55c5dda9b5 RTEMS: Make pthread_cond_t self-contained
Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2017-10-05 14:56:13 +02:00
Sebastian Huber d902eef093 RTEMS: Make pthread_rwlock_t self-contained
Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2017-10-05 14:56:13 +02:00
Sebastian Huber 9187bb23a0 RTEMS: Make pthread_barrier_t self-contained
Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2017-10-05 14:56:13 +02:00
Sebastian Huber 8253c240cb RTEMS: Make sem_t self-contained
Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2017-10-05 14:56:12 +02:00
Sebastian Huber 4fef7312b3 RTEMS: Optimize pthread_once_t
Reduce size of pthread_once_t and make it zero-initialized.

Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2017-10-05 14:56:12 +02:00
Sebastian Huber 1592a0be0c Fix warnings and documentation in strnstr.c
Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2017-09-19 15:35:09 -05:00
Sebastian Huber 4de8754bac Change time_t to 64-bit by default
In order to avoid the year 2038 problem, define time_t to a signed
integer with at least 64-bits.  The type for time_t can be forced to
long with the --enable-newlib-long-time_t configure option or with the
_USE_LONG_TIME_T system configuration define.

Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2017-09-07 15:39:34 +02:00
Sebastian Huber ad45b86533 Remove harmful casts in gmtime_r()
In case time_t is long, then the cast to long is a nop.  In case time_t
is __int_least64_t, then the cast to long may truncate the value before
the division.

Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2017-09-07 15:39:34 +02:00
Corinna Vinschen e18b7ffa48 stdio.h: Don't define unlocked macros using static inline on C++
In C++, the usage of static inline functions for getchar_unlocked and
putchar_unlocked may result in error messages like

  error: ‘_putchar_unlocked’ was not declared in this scope

Fix this by not using the _getchar_unlocked and _putchar_unlocked
macros in C++.

Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
2017-09-04 10:52:33 +02:00
Yaakov Selkowitz 51d1fb715d include: fix ffs, fls guards
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2017-08-30 12:29:47 -05:00
Corinna Vinschen 192986ab03 newlib: string/Makefile.am (CHEWOUT_FILES): Add strnstr.def
Regenerate strings/Makefile.in

Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
2017-08-30 16:48:55 +02:00
Corinna Vinschen 5fc315b597 newlib: strnstr: drop traditional synopisis
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
2017-08-30 16:45:36 +02:00
Sichen Zhao 42885ea4b8 Add man page entry for strnstr.c. 2017-08-30 15:10:07 +02:00
Sichen Zhao f22054c94d Modify strnstr.c. 2017-08-30 15:08:58 +02:00
Sebastian Huber f1863582ed Fix compile error due to new strnstr()
Remove local strnstr() implementation to fix compile error:

newlib/libc/iconv/lib/aliasesi.c:53:8: error: conflicting types for 'strnstr'
 _DEFUN(strnstr, (haystack, needle, length),
        ^
In file included from newlib/libc/iconv/lib/aliasesi.c:29:0:
newlib/libc/include/string.h:125:10:
note: previous declaration of 'strnstr' was here
 char    *strnstr(const char *, const char *, size_t) __pure;
          ^~~~~~~

Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2017-08-28 17:31:52 +02:00
Corinna Vinschen c070326d31 newlib: rebuild string/Makefile.in
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
2017-08-25 18:00:46 +02:00
Sichen Zhao c206d04422 Port strnstr.c to newlib. 2017-08-25 18:00:46 +02:00
Sichen Zhao 3437665ac8 Import strnstr.c from FreeBSD. 2017-08-25 18:00:46 +02:00
Eric Blake a4961ccd3f Revert "headers: avoid bareword attributes" for clang
This reverts most of commit 979d467ff6.

We cannot avoid some bareword attributes until clang is fixed to
properly support __-decorated attributes; see this bug:
https://bugs.llvm.org/show_bug.cgi?id=34319

The macros in question expand to the empty string under gcc, so
only compilation under clang is affected, and since clang has the
bug, the obvious solution is to roll back the changes, and document
the issue.

Signed-off-by: Eric Blake <eblake@redhat.com>
2017-08-25 09:23:10 -05:00
Sebastian Huber 524eb4dc29 RTEMS: Use __uint64_t for _CLOCK_T_
This addresses:

https://devel.rtems.org/ticket/2135

Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2017-08-25 14:25:42 +02:00
Sebastian Huber 7b2c362190 Make _CLOCK_T_ system configurable
Let systems optionally provide the _CLOCK_T_ type via
<machine/_types.h>.

Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2017-08-25 14:25:42 +02:00
Brian Inglis 406bd10fb4 newlib/libc/time/strptime.c(strptime_l) add %F %s support for strptime 2017-08-25 14:04:22 +02:00
Kito Cheng 6864c08b94 Change license to FreeBSD License for RISC-V
- For prevent confuse about what BSD license variant we used, 2- or
   3-clause license, we change the license to FreeBSD license to make
   it unambiguously refers to the 2-clause license.
2017-08-21 11:08:54 +02:00
Ken Brown f665b1cef3 cygwin: Implement renameat2
Define the RENAME_NOREPLACE flag in <cygwin/fs.h> as defined on Linux
in <linux/fs.h>.  The other RENAME_* flags defined on Linux are not
supported.
2017-08-19 18:06:49 +02:00
Eric Blake 979d467ff6 headers: avoid bareword attributes
Always use the __-decorated form of an attribute name in public
headers, as the bareword form is in the user's namespace, and we
don't want compilation to break just because the user defines the
bareword to mean something else.

Signed-off-by: Eric Blake <eblake@redhat.com>
2017-08-17 07:10:03 -05:00
Kito Cheng 363dbb9e44 Add RISC-V port for newlib
Contributor list:
    - Andrew Waterman  <andrew@sifive.com>
    - Palmer Dabbelt  <palmer@dabbelt.com>
    - Kito Cheng  <kito.cheng@gmail.com>
    - Scott Beamer  <sbeamer@eecs.berkeley.edu>
2017-08-16 18:00:58 -04:00
Corinna Vinschen 9d602b98f8 newlib: regenerate libc/stdlib/Makefile.am
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
2017-08-11 10:17:16 +02:00
Ken Brown d7821c045e Define sigsetjmp/siglongjmp only if __POSIX_VISIBLE 2017-08-07 16:07:46 +02:00
Sebastian Huber b0f271d1db Proper locking for getchar() and putchar()
Add internal inline functions _getchar_unlocked() and
_putchar_unlocked() if __CUSTOM_FILE_IO__ is not defined.  These
functions get _REENT only once.  Use them for getchar_unlocked() and
putchar_unlocked().  Define getchar() and putchar() to these unlocked
internal functions if __SINGLE_THREAD__ is defined, otherwise use the
external functions to use proper locking of the FILE object.

Assumes that __SINGLE_THREAD__ is not defined if __CYGWIN__ is defined.

Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2017-08-07 07:54:58 +02:00
Aditya Upadhyay a1c2491f70 Importing wcstoumax inttypes method from FreeBSD. 2017-08-02 13:02:26 +02:00
Aditya Upadhyay 88abc0958b Importing wcstoimax inttypes method from FreeBSD. 2017-08-02 13:02:26 +02:00
Aditya Upadhyay fd1981a7df Importing strtoumax inttypes method from FreeBSD. 2017-08-02 13:02:26 +02:00
Aditya Upadhyay 910cc30c10 Importing strtoimax inttypes method from FreeBSD. 2017-08-02 13:02:26 +02:00
Yaakov Selkowitz 4aaec3cb88 Add elf.h to newlib
This is copied from musl (MIT license).  This is newer and more thorough
than that of FreeBSD currently shipped only on Cygwin.

Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2017-08-02 01:18:55 -05:00
Sebastian Huber 7dad0e441a stdio: Fix make rule override
The Makefile.am contained two rules for the vfwscanf object.

Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
2017-07-28 15:05:54 +02:00
Corinna Vinschen a7a7980f7b newlib: regenerate stdlib/Makefile.in
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
2017-07-28 12:44:45 +02:00
Aditya Upadhyay 4b2fc8c55e Importing imaxdiv inttypes method from FreeBSD. 2017-07-28 12:23:10 +02:00
Aditya Upadhyay a785f0f69a Importing imaxabs inttypes method from FreeBSD. 2017-07-28 12:23:08 +02:00
Corinna Vinschen 8322a87c16 cygwin: Set __STDC_ISO_10646__ to Unicode 5.2 value
Now that XP is not supported anymore we can freely do that.

Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
2017-07-26 11:49:58 +02:00