These types were introduced by FreeBSD commit:
"Make struct xinpcb and friends word-size independent.
Replace size_t members with ksize_t (uint64_t) and pointer members
(never used as pointers in userspace, but instead as unique
idenitifiers) with kvaddr_t (uint64_t). This makes the structs
identical between 32-bit and 64-bit ABIs.
On 64-bit bit systems, the ABI is maintained. On 32-bit systems,
this is an ABI breaking change. The ABI of most of these structs
was previously broken in r315662. This also imposes a small API
change on userspace consumers who must handle kernel pointers
becoming virtual addresses.
PR: 228301 (exp-run by antoine)
Reviewed by: jtl, kib, rwatson (various versions)
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D15386"
In RTEMS, there is no user/kernel space separation. So, use the types
size_t and uintptr_t.
Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
The __XSI_VISIBLE is not enabled by default in Newlib. This is an
incompatiblity between FreeBSD and glibc.
Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
with name SO_DOMAIN to get the domain of a socket.
This is helpful when testing and Solaris and Linux have the same
socket option using the same name.
Reviewed by: bcr@, rrs@
Sponsored by: Netflix, Inc.
Differential Revision: https://reviews.freebsd.org/D16791
queues per bucket.
There is a hashing algorithm which should distribute IPv6 reassembly
queues across the available buckets in a relatively even way. However,
if there is a flaw in the hashing algorithm which allows a large number
of IPv6 fragment reassembly queues to end up in a single bucket, a per-
bucket limit could help mitigate the performance impact of this flaw.
Implement such a limit, with a default of twice the maximum number of
reassembly queues divided by the number of buckets. Recalculate the
limit any time the maximum number of reassembly queues changes.
However, allow the user to override the value using a sysctl
(net.inet6.ip6.maxfragbucketsize).
Reviewed by: jhb
Security: FreeBSD-SA-18:10.ip
Security: CVE-2018-6923
The IPv4 fragment reassembly code supports a limit on the number of
fragments per packet. The default limit is currently 17 fragments.
Among other things, this limit serves to limit the number of fragments
the code must parse when trying to reassembly a packet.
Add a limit to the IPv6 reassembly code. By default, limit a packet
to 65 fragments (64 on the queue, plus one final fragment to complete
the packet). This allows an average fragment size of 1,008 bytes, which
should be sufficient to hold a fragment. (Recall that the IPv6 minimum
MTU is 1280 bytes. Therefore, this configuration allows a full-size
IPv6 packet to be fragmented on a link with the minimum MTU and still
carry approximately 272 bytes of headers before the fragmented portion
of the packet.)
Users can adjust this limit using the net.inet6.ip6.maxfragsperpacket
sysctl.
Reviewed by: jhb
Security: FreeBSD-SA-18:10.ip
Security: CVE-2018-6923
Rack includes the following features: - A different SACK processing
scheme (the old sack structures are not used). - RACK (Recent
acknowledgment) where counting dup-acks is no longer done instead time
is used to knwo when to retransmit. (see the I-D) - TLP (Tail Loss
Probe) where we will probe for tail-losses to attempt to try not to take
a retransmit time-out. (see the I-D) - Burst mitigation using TCPHTPS -
PRR (partial rate reduction) see the RFC.
Once built into your kernel, you can select this stack by either
socket option with the name of the stack is "rack" or by setting
the global sysctl so the default is rack.
Note that any connection that does not support SACK will be kicked
back to the "default" base FreeBSD stack (currently known as "default").
To build this into your kernel you will need to enable in your
kernel:
makeoptions WITH_EXTRA_TCP_STACKS=1
options TCPHPTS
Sponsored by: Netflix Inc.
Differential Revision: https://reviews.freebsd.org/D15525
This patch adds a new socket option, SO_REUSEPORT_LB, which allow multiple
programs or threads to bind to the same port and incoming connections will be
load balanced using a hash function.
Most of the code was copied from a similar patch for DragonflyBSD.
However, in DragonflyBSD, load balancing is a global on/off setting and can not
be set per socket. This patch allows for simultaneous use of both the current
SO_REUSEPORT and the new SO_REUSEPORT_LB options on the same system.
Required changes to structures:
Globally change so_options from 16 to 32 bit value to allow for more options.
Add hashtable in pcbinfo to hold all SO_REUSEPORT_LB sockets.
Limitations:
As DragonflyBSD, a load balance group is limited to 256 pcbs (256 programs or
threads sharing the same socket).
This is a substantially different contribution as compared to its original
incarnation at svn r332894 and reverted at svn r332967. Thanks to rwatson@
for the substantive feedback that is included in this commit.
Submitted by: Johannes Lundberg <johalun0@gmail.com>
Obtained from: DragonflyBSD
Relnotes: Yes
Sponsored by: Limelight Networks
Differential Revision: https://reviews.freebsd.org/D11003
Part 3 of many ...
The VPC framework relies heavily on cloning pseudo interfaces
(vmnics, vpc switch, vcpswitch port, hostif, vxlan if, etc).
This pulls in that piece. Some ancillary changes get pulled
in as a side effect.
Reviewed by: shurd@
Approved by: sbruno@
Sponsored by: Joyent, Inc.
Differential Revision: https://reviews.freebsd.org/D15347
This patch adds a new socket option, SO_REUSEPORT_LB, which allow multiple
programs or threads to bind to the same port and incoming connections will be
load balanced using a hash function.
Most of the code was copied from a similar patch for DragonflyBSD.
However, in DragonflyBSD, load balancing is a global on/off setting and can not
be set per socket. This patch allows for simultaneous use of both the current
SO_REUSEPORT and the new SO_REUSEPORT_LB options on the same system.
Required changes to structures
Globally change so_options from 16 to 32 bit value to allow for more options.
Add hashtable in pcbinfo to hold all SO_REUSEPORT_LB sockets.
Limitations
As DragonflyBSD, a load balance group is limited to 256 pcbs
(256 programs or threads sharing the same socket).
Submitted by: Johannes Lundberg <johanlun0@gmail.com>
Sponsored by: Limelight Networks
Differential Revision: https://reviews.freebsd.org/D11003
Use an accessor to access ifgr_group and ifgr_groups.
Use an macro CASE_IOC_IFGROUPREQ(cmd) in place of case statements such
as "case SIOCAIFGROUP:". This avoids poluting the switch statements
with large numbers of #ifdefs.
Reviewed by: kib
Obtained from: CheriBSD
MFC after: 1 week
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D14960
This fixes 32-bit compat (no ioctl command defintions are required
as struct ifreq is the same size). This is believed to be sufficent to
fully support ifconfig on 32-bit systems.
Reviewed by: kib
Obtained from: CheriBSD
MFC after: 1 week
Relnotes: yes
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D14900
Add a new "interleave" allocation policy which stripes pages across
domains with a stride or width keeping contiguity within a multi-page
region.
Move the kernel to the dedicated numbered cpuset #2 making it possible
to assign kernel threads and memory policy separately from user. This
also eliminates the need for the complicated interrupt binding code.
Add a sysctl API for viewing and manipulating domainsets. Refactor some
of the cpuset_t manipulation code using the generic bitset type so that
it can be used for both. This probably belongs in a dedicated subr file.
Attempt to improve the include situation.
Reviewed by: kib
Discussed with: jhb (cpuset parts)
Tested by: pho (before review feedback)
Sponsored by: Netflix, Dell/EMC Isilon
Differential Revision: https://reviews.freebsd.org/D14839
Make all kernel accesses to ifru_buffer go via access functions
which take the process ABI into account and use an appropriate union
to access members in the correct place in struct ifreq.
Reviewed by: kib
Obtained from: CheriBSD
MFC after: 1 week
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D14846
According to 802.1Q-2014, VLAN tagged packets with VLAN id 0 should be
considered as untagged, and only PCP and DEI values from the VLAN tag
are meaningful. See for instance
https://www.cisco.com/c/en/us/td/docs/switches/connectedgrid/cg-switch-sw-master/software/configuration/guide/vlan0/b_vlan_0.html.
Make it possible to specify PCP value for outgoing packets on an
ethernet interface. When PCP is supplied, the tag is appended, VLAN
id set to 0, and PCP is filled by the supplied value. The code to do
VLAN tag encapsulation is refactored from the if_vlan.c and moved into
if_ethersubr.c.
Drivers might have issues with filtering VID 0 packets on
receive. This bug should be fixed for each driver.
Reviewed by: ae (previous version), hselasky, melifaro
Sponsored by: Mellanox Technologies
MFC after: 2 weeks
Differential revision: https://reviews.freebsd.org/D14702
Include _uio.h instead of uio.h in several headers to reduce header
polution.
Fix a few places that relied on header polution to get the uio.h header.
I have not moved struct uio as many more things that use it rely on
header polution to get other definitions from uio.h.
Reviewed by: cem, kib, markj
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D14811
which we discussed at the developer summits at BSDCan and BSDCam in 2017.
The TCP Blackbox Recorder allows you to capture events on a TCP connection
in a ring buffer. It stores metadata with the event. It optionally stores
the TCP header associated with an event (if the event is associated with a
packet) and also optionally stores information on the sockets.
It supports setting a log ID on a TCP connection and using this to correlate
multiple connections that share a common log ID.
You can log connections in different modes. If you are doing a coordinated
test with a particular connection, you may tell the system to put it in
mode 4 (continuous dump). Or, if you just want to monitor for errors, you
can put it in mode 1 (ring buffer) and dump all the ring buffers associated
with the connection ID when we receive an error signal for that connection
ID. You can set a default mode that will be applied to a particular ratio
of incoming connections. You can also manually set a mode using a socket
option.
This commit includes only basic probes. rrs@ has added quite an abundance
of probes in his TCP development work. He plans to commit those soon.
There are user-space programs which we plan to commit as ports. These read
the data from the log device and output pcapng files, and then let you
analyze the data (and metadata) in the pcapng files.
Reviewed by: gnn (previous version)
Obtained from: Netflix, Inc.
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D11085
These macros take an existing ioctl(2) command and replace the length
with the specified length or length of the specified type respectively.
These can be used to define commands for 32-bit compatibility with fewer
opportunities for cut-and-paste errors then a whole new definition.
Reviewed by: cem, kib
Obtained from: CheriBSD
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D14706
[RFC7413]. It also includes a pre-shared key mode of operation in which
the server requires the client to be in possession of a shared secret in
order to successfully open TFO connections with that server.
The names of some existing fastopen sysctls have changed (e.g.,
net.inet.tcp.fastopen.enabled -> net.inet.tcp.fastopen.server_enable).
Reviewed by: tuexen
MFC after: 1 month
Sponsored by: Limelight Networks
Differential Revision: https://reviews.freebsd.org/D14047
that had the IPv6 fragmentation header:
o Neighbor Solicitation
o Neighbor Advertisement
o Router Solicitation
o Router Advertisement
o Redirect
Introduce M_FRAGMENTED mbuf flag, and set it after IPv6 fragment reassembly
is completed. Then check the presence of this flag in correspondig ND6
handling routines.
PR: 224247
MFC after: 2 weeks
Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.
The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.
Mainly focus on files that use BSD 3-Clause license.
The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.
Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.
Mainly focus on files that use BSD 3-Clause license.
The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.
Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.
for SO_TIMESTAMP and other similar socket options.
Provide new control message SCM_TIME_INFO to supply information about
timestamp. Currently it indicates that the timestamp was
hardware-assisted and high-precision, for software timestamps the
message is not returned. Reserved fields are added to ABI to report
additional info about it, it is expected that raw hardware clock value
might be useful for some applications.
Reviewed by: gallatin (previous version), hselasky
Sponsored by: Mellanox Technologies
MFC after: 2 weeks
X-Differential revision: https://reviews.freebsd.org/D12638
in nanoseconds from boot for the received packets.
The rcv_tstmp field overlaps the place of Ln header length indicators,
not used by received packets. The basic pkthdr rearrangement change
in sys/mbuf.h was provided by gallatin.
There are two accompanying M_ flags: M_TSTMP means that there is the
timestamp (and it was generated by hardware).
Another flag M_TSTMP_HPREC indicates that the timestamp is
high-precision. Practically M_TSTMP_HPREC means that hardware
provided additional precision comparing with the stamps when the flag
is not set. E.g., for ConnectX all packets are stamped by hardware
when PCIe transaction to write out the completion descriptor is
performed, but PTP packet are stamped on port. For Intel cards, when
PTP assist is enabled, only PTP packets are stamped in the limited
number of registers, so if Intel cards ever start support this
mechanism, they would always set M_TSTMP | M_TSTMP_HPREC if hardware
timestamp is present for the given packet.
Add IFCAP_HWRXTSTMP interface capability to indicate the support for
hardware rx timestamping, and ifconfig(8) command to toggle it.
Based on the patch by: gallatin
Reviewed by: gallatin (previous version), hselasky
Sponsored by: Mellanox Technologies
MFC after: 2 weeks (? mbuf KBI issue)
X-Differential revision: https://reviews.freebsd.org/D12638
It will be needed by hn(4) to configure its RSS key and hash
type/function in the transparent VF mode in order to match VF's
RSS settings. The description of the transparent VF mode and
the RSS hash value issue are here:
https://svnweb.freebsd.org/base?view=revision&revision=322299https://svnweb.freebsd.org/base?view=revision&revision=322485
These are generic enough to promise two independent IOCs instead
of abusing SIOCGDRVSPEC.
Setting RSS key and hash type/function is a different story,
which probably requires more discussion.
Comment about UDP_{IPV4,IPV6,IPV6_EX} were only in the patch
in the review request; these hash types are standardized now.
Reviewed by: gallatin
MFC after: 1 week
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D12174
They are defined by XSI or newer SUS.
This is a follow-up to r318780.
Reported by: jbeich
Obtained from: DragonflyBSD commit e08b3836c962
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Guard, requested by the MAP_GUARD mmap(2) flag, prevents the reuse of
the allocated address space, but does not allow instantiation of the
pages in the range. It is useful for more explicit support for usual
two-stage reserve then commit allocators, since it prevents accidental
instantiation of the mapping, e.g. by mprotect(2).
Use guards to reimplement stack grow code. Explicitely track stack
grow area with the guard, including the stack guard page. On stack
grow, trivial shift of the guard map entry and stack map entry limits
makes the stack expansion. Move the code to detect stack grow and
call vm_map_growstack(), from vm_fault() into vm_map_lookup().
As result, it is impossible to get random mapping to occur in the
stack grow area, or to overlap the stack guard page.
Enable stack guard page by default.
Reviewed by: alc, markj
Man page update reviewed by: alc, bjk, emaste, markj, pho
Tested by: pho, Qualys
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D11306 (man pages)
o Separate fields of struct socket that belong to listening from
fields that belong to normal dataflow, and unionize them. This
shrinks the structure a bit.
- Take out selinfo's from the socket buffers into the socket. The
first reason is to support braindamaged scenario when a socket is
added to kevent(2) and then listen(2) is cast on it. The second
reason is that there is future plan to make socket buffers pluggable,
so that for a dataflow socket a socket buffer can be changed, and
in this case we also want to keep same selinfos through the lifetime
of a socket.
- Remove struct struct so_accf. Since now listening stuff no longer
affects struct socket size, just move its fields into listening part
of the union.
- Provide sol_upcall field and enforce that so_upcall_set() may be called
only on a dataflow socket, which has buffers, and for listening sockets
provide solisten_upcall_set().
o Remove ACCEPT_LOCK() global.
- Add a mutex to socket, to be used instead of socket buffer lock to lock
fields of struct socket that don't belong to a socket buffer.
- Allow to acquire two socket locks, but the first one must belong to a
listening socket.
- Make soref()/sorele() to use atomic(9). This allows in some situations
to do soref() without owning socket lock. There is place for improvement
here, it is possible to make sorele() also to lock optionally.
- Most protocols aren't touched by this change, except UNIX local sockets.
See below for more information.
o Reduce copy-and-paste in kernel modules that accept connections from
listening sockets: provide function solisten_dequeue(), and use it in
the following modules: ctl(4), iscsi(4), ng_btsocket(4), ng_ksocket(4),
infiniband, rpc.
o UNIX local sockets.
- Removal of ACCEPT_LOCK() global uncovered several races in the UNIX
local sockets. Most races exist around spawning a new socket, when we
are connecting to a local listening socket. To cover them, we need to
hold locks on both PCBs when spawning a third one. This means holding
them across sonewconn(). This creates a LOR between pcb locks and
unp_list_lock.
- To fix the new LOR, abandon the global unp_list_lock in favor of global
unp_link_lock. Indeed, separating these two locks didn't provide us any
extra parralelism in the UNIX sockets.
- Now call into uipc_attach() may happen with unp_link_lock hold if, we
are accepting, or without unp_link_lock in case if we are just creating
a socket.
- Another problem in UNIX sockets is that uipc_close() basicly did nothing
for a listening socket. The vnode remained opened for connections. This
is fixed by removing vnode in uipc_close(). Maybe the right way would be
to do it for all sockets (not only listening), simply move the vnode
teardown from uipc_detach() to uipc_close()?
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D9770
INHERIT_ZERO is an OpenBSD feature.
When a page is marked as such, it would be zeroed
upon fork().
This would be used in new arc4random(3) functions.
PR: 182610
Reviewed by: kib (earlier version)
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D427
Renumber cluase 4 to 3, per what everybody else did when BSD granted
them permission to remove clause 3. My insistance on keeping the same
numbering for legal reasons is too pedantic, so give up on that point.
Submitted by: Jan Schaumann <jschauma@stevens.edu>
Pull Request: https://github.com/freebsd/freebsd/pull/96
Our mprotect() function seems to take a "const void *" address to the
pages whose permissions need to be adjusted. POSIX uses "void *". Simply
stick to the POSIX one to prevent us from writing unportable code.
PR: 211423 (exp-run)
Tested by: antoine@ (Thanks!)
for libthr.so.3, without breaking the ABI. Special value is stored in
the lock pointer to indicate shared lock, and offline page in the shared
memory is allocated to store the actual lock.
Reviewed by: vangyzen (previous version)
Discussed with: deischen, emaste, jhb, rwatson,
Martin Simmons <martin@lispworks.com>
Tested by: pho
Sponsored by: The FreeBSD Foundation
operations. File type-specific logic is now placed in the mmap hook
implementation rather than requiring it to be placed in
sys/vm/vm_mmap.c. This hook allows new file types to support mmap() as
well as potentially allowing mmap() for existing file types that do not
currently support any mapping.
The vm_mmap() function is now split up into two functions. A new
vm_mmap_object() function handles the "back half" of vm_mmap() and accepts
a referenced VM object to map rather than a (handle, handle_type) tuple.
vm_mmap() is now reduced to converting a (handle, handle_type) tuple to a
a VM object and then calling vm_mmap_object() to handle the actual mapping.
The vm_mmap() function remains for use by other parts of the kernel
(e.g. device drivers and exec) but now only supports mapping vnodes,
character devices, and anonymous memory.
The mmap() system call invokes vm_mmap_object() directly with a NULL object
for anonymous mappings. For mappings using a file descriptor, the
descriptors fo_mmap() hook is invoked instead. The fo_mmap() hook is
responsible for performing type-specific checks and adjustments to
arguments as well as possibly modifying mapping parameters such as flags
or the object offset. The fo_mmap() hook routines then call
vm_mmap_object() to handle the actual mapping.
The fo_mmap() hook is optional. If it is not set, then fo_mmap() will
fail with ENODEV. A fo_mmap() hook is implemented for regular files,
character devices, and shared memory objects (created via shm_open()).
While here, consistently use the VM_PROT_* constants for the vm_prot_t
type for the 'prot' variable passed to vm_mmap() and vm_mmap_object()
as well as the vm_mmap_vnode() and vm_mmap_cdev() helper routines.
Previously some places were using the mmap()-specific PROT_* constants
instead. While this happens to work because PROT_xx == VM_PROT_xx,
using VM_PROT_* is more correct.
Differential Revision: https://reviews.freebsd.org/D2658
Reviewed by: alc (glanced over), kib
MFC after: 1 month
Sponsored by: Chelsio
and MAP_NORESERVE flags to mmap(2). Older binaries are still permitted
to use these flags.
PR: 193961 (exp-run in ports)
Differential Revision: https://reviews.freebsd.org/D848
Reviewed by: kib
to add type-specific information to struct kinfo_file. - Move the
various fill_*_info() methods out of kern_descrip.c and into the various
file type implementations. - Rework the support for kinfo_ofile to
generate a suitable kinfo_file object for each file and then convert
that to a kinfo_ofile structure rather than keeping a second, different
set of code that directly manipulates type-specific file information. -
Remove the shm_path() and ksem_info() layering violations.
Differential Revision: https://reviews.freebsd.org/D775
Reviewed by: kib, glebius (earlier version)
It should be combined with MAP_FIXED, and prevents the request from
deleting existing mappings in the region, failing instead.
Reviewed by: alc
Discussed with: jhb
Tested by: markj, pho (previous version, as part of the bigger patch)
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
to request that a mapping use an address in the first 2GB of the
process's address space. This flag should have the same semantics as the
same flag on Linux.
To facilitate this, add a new parameter to vm_map_find() that specifies an
optional maximum virtual address. While here, fix several callers of
vm_map_find() to use a VMFS_* constant for the findspace argument instead of
TRUE and FALSE.
Reviewed by: alc
Approved by: re (kib)
for posix shmfd. Add MAC framework entries for posix shm read and write.
Do not allow implicit extension of the underlying memory segment past
the limit set by ftruncate(2) by either of the syscalls. Read and
write returns short i/o, lseek(2) fails with EINVAL when resulting
offset does not fit into the limit.
Discussed with: alc
Tested by: pho
Sponsored by: The FreeBSD Foundation
Add __nl_item to <sys/_types.h> for FreeBSD compatibility. Use it in
<langinfo.h> and the Cygwin <nl_types.h>. Make the enum __nl_item in
<langinfo.h> anonymous.
Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
Add __tls_get_addr() for all targets to crt0. This is not only used on
ARM. In particular, it is used on RISC-V. This helps to adequately
support the GCC libgomp.
Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de
strtof ("-nan") returned positive NaN instead of negative NaN.
strtod ("-nan") and strtold ("-nan") return negative NaN.
Linux glibc has been fixed
that strto{f|d|ld} ("-nan") returns negative NaN.
https://sourceware.org/bugzilla/show_bug.cgi?id=23007
This commit makes strtof preserves the negative sign bit
when parsing "-nan" like glibc.
By previous commit, strto{d|ld} ("nan")
does not use the definition of NaN.
There is no other function that uses the definitions.
This commit remove the definitions.
The definition of qNaN for x86_64 and i386 was wrong.
strto{d|ld} ("nan") returned wrong negative NaN
instead of correct positive NaN
since it used the wrong definition.
On the other hand, strtof ("nan") returns correct positive NaN
since it uses nanf ("") instead of the wrong definition.
This commit makes strto{d|ld} ("nan") uses {nan|nanl} ("")
like strtof ("nan") using.
So strto{d|ld} ("nan") returns positive NaN.
Improve comments in sincosf implementation to make the code easier
to understand. Rename the constant pi64 to pi63 since it's actually
PI * 2^-63. Add comments for fields of sincos_t structure. Add comments
describing implementation details to reduce_fast.
wordexp uses fprintf in a dangerous way. It uses an unchecked
input string as format string, rather than as parameter to a %s.
Replace fprintf with fputs.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Introduce new host configuration variable "have_init_fini" which is set
to "yes" by default. Override it for RISC-V to "no".
Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
Restore FreeBSD compatibility for __alloc_size() and __alloc_align().
This is a follow-up to commit e494b56035.
Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
At least on GCC7 calling __alloc_size(x) twice is not equivalent to
calling using the attribute once with two arguments. The later is the
documented use in GCC documentation so add a new alloc_size(n, x)
alternative to cover for the few places where it is used: basically:
calloc(3), reallocarray(3) and mallocarray(9).
Submitted by: Mark Millard
MFC after: 3 days
Reference:
http://docs.freebsd.org/cgi/mid.cgi?F227842D-6BE2-4680-82E7-07906AF61CD7
Mainly focus on files that use BSD 3-Clause license.
The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.
Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.
GCC only activates C11 keywords in C mode, not C++ mode. This means
that when targeting an older C++ standard, we cannot fall back to using
_Static_assert(). In this case, do define _Static_assert() as a macro
that uses a typedef'ed array.
Discussed in: r322875 commit thread
Reported by: Mark MIllard
MFC after: 1 month
The previous version genenerated the following GCC note:
towctrans_l.c:44:1: note: offset of packed bit-field 'diff' has changed in GCC 4.4
caseconv_table [] = {
^~~~~~~~~~~~~~
Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
The commit 46ba1675c4 accidently changed a
bit-field from signed to unsigned. The caseconv_entry::delta must be a
signed integer, see also "newlib/libc/ctype/caseconv.t".
Unfortunately, a standard GCC/Newlib build is done without
-Wsign-conversion. Using this warning option would have helped to avoid
this bug:
caseconv.t:2:22: warning: unsigned conversion from 'int' to 'unsigned int:17' changes value from '-32' to '131040' [-Wsign-conversion]
{0x0061, 25, TOUP, -32},
Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
This prevents errors like this:
newlib/libc/ctype/categories.c:6:3: error: width of 'first' exceeds its type
unsigned int first: 24;
^
Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
Exotic RTEMS targets can define this back to int32_t as an exception if
there are good reasons.
Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
Replace the simple byte-wise compare in the misaligned case with a
dword compare with page boundary checks in place. For simplicity I've
chosen a 4K page boundary so that we don't have to query the actual
page size on the system.
This results in up to 3x improvement in performance in the unaligned
case on falkor and about 2.5x improvement on mustang as measured using
bench-strcmp in glibc.
This improved memcmp provides a fast path for compares up to 16 bytes
and then compares 16 bytes at a time, thus optimizing loads from both
sources. The glibc memcmp microbenchmark retains performance (with an
error of ~1ns) for smaller compare sizes and reduces up to 31% of
execution time for compares up to 4K on the APM Mustang. On Qualcomm
Falkor this improves to almost 48%, i.e. it is almost 2x improvement
for sizes of 2K and above.
The mutually misaligned inputs on aarch64 are compared with a simple
byte copy, which is not very efficient. Enhance the comparison
similar to strcmp by loading a double-word at a time. The peak
performance improvement (i.e. 4k maxlen comparisons) due to this on
the strncmp microbenchmark in glibc is as follows:
falkor: 3.5x (up to 72% time reduction)
cortex-a73: 3.5x (up to 71% time reduction)
cortex-a53: 3.5x (up to 71% time reduction)
All mutually misaligned inputs from 16 bytes maxlen onwards show
upwards of 15% improvement and there is no measurable effect on the
performance of aligned/mutually aligned inputs.
PREFER_FLOAT_COMPARISON setting was not correct as it could raise
spurious exceptions. Fixing it is easy: just use ISLESS(x, y) instead
of abstop12(x) < abstop12(y) with appropriate non-signaling definition
for ISLESS. However it seems this setting is not very useful (there is
only minor performance difference on various architectures), so remove
this option for now.
The !HAVE_FAST_FMA code path split r = z/c - 1 into r = rhi + rlo such
that when z = 1-tiny and c = 1 then rlo and rhi could have much larger
magnitude than r which later caused large rounding errors.
So do a nearest rounding instead of truncation at the split.
In newlib with default settings this was observable on some arm targets
that enable the new math code but has no fma.
The roundtoint and converttoint internal functions are only called with small
values, so 32 bit result is enough for converttoint and it is a signed int
conversion so the natural return type is int32_t.
The original idea was to help the compiler keeping the result in uint64_t,
then it's clear that no sign extension is needed and there is no accidental
undefined or implementation defined signed int arithmetics.
But it turns out gcc does a good job with inlining so changing the type has
no overhead and the semantics of the conversion is less surprising this way.
Since we want to allow the asuint64 (x + 0x1.8p52) style conversion, the top
bits were never usable and the existing code ensures that only the bottom
32 bits of the conversion result are used.
In newlib with default settings only aarch64 is affected and there is no
significant code generation change with gcc after the patch.
Synchronize code style and comments with Arm Optimized Routines, there
are no code changes in this patch. This ensures different projects using
the same code have consistent code style so bug fix patches can be applied
more easily.
This fix is for some platforms which do not have writev().
*perror.c: Use _write_r() instead of writev().
*psignal.c: Use write() insetad of writev().
Revise commit: d4f4e7ae1b
The new implementation is provided under !__OBSOLETE_MATH, it uses
ISO C99 code. With default settings the worst case error in nearest
rounding mode is 0.54 ULP with inlined fma and fma contraction. It uses
a 4 KB lookup table in addition to the table in exp_data.c, on aarch64
.text+.rodata size of libm.a is increased by 2295 bytes.
Improvements on Cortex-A72:
latency: 3.3x
thruput: 4.9x
The new implementation is provided under !__OBSOLETE_MATH, it uses
ISO C99 code. With default settings the worst case error in nearest
rounding mode is 0.547 ULP with inlined fma and fma contraction. It uses
a 1 KB lookup table, on aarch64 .text+.rodata size of libm.a is increased
by 1584 bytes.
Note that the math.h header defines log2(x) to be log(x)/Ln2, this is
not changed, so the new code is only used if that macro is suppressed.
Improvements on Cortex-A72:
latency: 2.0x
thruput: 2.2x
The new implementations are provided under !__OBSOLETE_MATH, it uses
ISO C99 code. With default settings the worst case error in nearest
rounding mode is 0.519 ULP with inlined fma and fma contraction. It uses
a 2 KB lookup table, on aarch64 .text+.rodata size of libm.a is increased
by 1703 bytes. The w_log.c wrapper is disabled since error handling is
inline in the new code.
New __HAVE_FAST_FMA and __HAVE_FAST_FMA_DEFAULT feature macros were
added to enable selecting between the code path that uses fma and the
one that does not. Targets supposed to set __HAVE_FAST_FMA_DEFAULT
if they have single instruction fma and the compiler can actually
inline it (gcc has __FP_FAST_FMA macro but that does not guarantee
inlining with -fno-builtin-fma).
Improvements on Cortex-A72:
latency: 1.9x
thruput: 2.3x
The new implementations are provided under !__OBSOLETE_MATH, they use
ISO C99 code. There are several settings, with the default one the
worst case error in nearest rounding mode is 0.509 ULP for exp and
0.507 ULP for exp2 when a multiply and add is contracted into an fma.
They use a shared 2 KB lookup table, on aarch64 .text+.rodata size
of libm.a is increased by 1868 bytes. The w_*.c wrappers are disabled
for the new code as it takes care of error handling inline.
The old exp2(x) code used to be just pow(2,x) so the speedup there
is more significant.
The file name has no special prefix to avoid any name collision with
existing files.
Improvements on Cortex-A72:
exp latency: 3.2x
exp thruput: 4.1x
exp2 latency: 7.8x
exp2 thruput: 18.8x
This change is equivalent to the commit
c65db17340
and only affects code that is from the Arm optimized-routines project.
It does not affect the observable behaviour, but the code generation
can be different on 64bit targets. The intention is to make the
portable semantics of the code obvious by using a fixed size type.
* (mkcategories): Fix a bug that outputs incorrect Unicode category
table for code point ranges.
* (categories.t): Rebuild it using the bug-fixed mkcategories.
This fixes the problem reported in the following post.
https://cygwin.com/ml/cygwin/2018-06/msg00248.html
Here is the correct patch with both filenames and int cast fixed:
This patch is a complete rewrite of sinf, cosf and sincosf. The new version
is significantly faster, as well as simple and accurate.
The worst-case ULP is 0.56072, maximum relative error is 0.5303p-23 over all
4 billion inputs. In non-nearest rounding modes the error is 1ULP.
The algorithm uses 3 main cases: small inputs which don't need argument
reduction, small inputs which need a simple range reduction and large inputs
requiring complex range reduction. The code uses approximate integer
comparisons to quickly decide between these cases - on some targets this may
be slow, so this can be configured to use floating point comparisons.
The small range reducer uses a single reduction step to handle values up to
120.0. It is fastest on targets which support inlined round instructions.
The large range reducer uses integer arithmetic for simplicity. It does a
32x96 bit multiply to compute a 64-bit modulo result. This is more than
accurate enough to handle the worst-case cancellation for values close to
an integer multiple of PI/4. It could be further optimized, however it is
already much faster than necessary.
Simple benchmark showing speedup factor on AArch64 for various ranges:
range 0.7853982 sinf 1.7 cosf 2.2 sincosf 2.8
range 1.570796 sinf 1.9 cosf 1.9 sincosf 2.7
range 3.141593 sinf 2.0 cosf 2.0 sincosf 3.5
range 6.283185 sinf 2.3 cosf 2.3 sincosf 4.2
range 125.6637 sinf 2.9 cosf 3.0 sincosf 5.1
range 1.1259e15 sinf 26.8 cosf 26.8 sincosf 45.2
ChangeLog:
2018-05-18 Wilco Dijkstra <wdijkstr@arm.com>
* newlib/libm/common/Makefile.in: Regenerated.
* newlib/libm/common/Makefile.am: Add sinf.c, cosf.c, sincosf.c
sincosf.h, sincosf_data.c. Add -fbuiltin -fno-math-errno to CFLAGS.
* newlib/libm/common/math_config.h: Add HAVE_FAST_ROUND, HAVE_FAST_LROUND,
roundtoint, converttoint, force_eval_float, force_eval_double, eval_as_float,
eval_as_double, likely, unlikely.
* newlib/libm/common/cosf.c: New file.
* newlib/libm/common/sinf.c: Likewise.
* newlib/libm/common/sincosf.h: Likewise.
* newlib/libm/common/sincosf.c: Likewise.
* newlib/libm/common/sincosf_data.c: Likewise.
* newlib/libm/math/sf_cos.c: Add #if to build conditionally.
* newlib/libm/math/sf_sin.c: Likewise.
* newlib/libm/math/wf_sincos.c: Likewise.
--
This patch is a complete rewrite of sinf, cosf and sincosf. The new version
is significantly faster, as well as simple and accurate.
The worst-case ULP is 0.56072, maximum relative error is 0.5303p-23 over all
4 billion inputs. In non-nearest rounding modes the error is 1ULP.
The algorithm uses 3 main cases: small inputs which don't need argument
reduction, small inputs which need a simple range reduction and large inputs
requiring complex range reduction. The code uses approximate integer
comparisons to quickly decide between these cases - on some targets this may
be slow, so this can be configured to use floating point comparisons.
The small range reducer uses a single reduction step to handle values up to
120.0. It is fastest on targets which support inlined round instructions.
The large range reducer uses integer arithmetic for simplicity. It does a
32x96 bit multiply to compute a 64-bit modulo result. This is more than
accurate enough to handle the worst-case cancellation for values close to
an integer multiple of PI/4. It could be further optimized, however it is
already much faster than necessary.
Simple benchmark showing speedup factor on AArch64 for various ranges:
range 0.7853982 sinf 1.7 cosf 2.2 sincosf 2.8
range 1.570796 sinf 1.9 cosf 1.9 sincosf 2.7
range 3.141593 sinf 2.0 cosf 2.0 sincosf 3.5
range 6.283185 sinf 2.3 cosf 2.3 sincosf 4.2
range 125.6637 sinf 2.9 cosf 3.0 sincosf 5.1
range 1.1259e15 sinf 26.8 cosf 26.8 sincosf 45.2
ChangeLog:
2018-06-18 Wilco Dijkstra <wdijkstr@arm.com>
* newlib/libm/common/Makefile.in: Regenerated.
* newlib/libm/common/Makefile.am: Add sinf.c, cosf.c, sincosf.c
sincosf.h, sincosf_data.c. Add -fbuiltin -fno-math-errno to CFLAGS.
* newlib/libm/common/math_config.h: Add HAVE_FAST_ROUND, HAVE_FAST_LROUND,
roundtoint, converttoint, force_eval_float, force_eval_double, eval_as_float,
eval_as_double, likely, unlikely.
* newlib/libm/common/cosf.c: New file.
* newlib/libm/common/sinf.c: Likewise.
* newlib/libm/common/sincosf.h: Likewise.
* newlib/libm/common/sincosf.c: Likewise.
* newlib/libm/common/sincosf_data.c: Likewise.
* newlib/libm/math/sf_cos.c: Add #if to build conditionally.
* newlib/libm/math/sf_sin.c: Likewise.
* newlib/libm/math/wf_sincos.c: Likewise.
--
Previously, "test 1 2 3 -a -b -c" was permuted to "test -a -b -c 1 2 3",
but "test 1 2 3 -abc" was left as "test 1 2 3 -abc".
Signed-off-by: Thomas Kindler <mail+newlib@t-kindler.de>
- when calculating a correction to align next brk to page boundary,
ensure that the correction is less than a page size
- if allocating the correction fails, ensure that the top size is
set to brk + sbrk_size (minus any front alignment made)
Signed-off-by: Jeff Johnston <jjohnstn@redhat.com>
When converting number of days since epoch (32-bits) to seconds,
calculations using 32-bit `long` overflow for years above 2038. Solve
this by casting number of days to `time_t` just before final
multiplication.
Signed-off-by: Freddie Chopin <freddie.chopin@gmail.com>
- From: Cesar Philippidis <cesar@codesourcery.com>
Date: Tue, 10 Apr 2018 14:43:42 -0700
Subject: [PATCH] nvptx port
This port adds support for Nvidia GPU's, which are primarily used as
offload accelerators in OpenACC and OpenMP.
The gdtoa implementation uses the type long, defined as Long, in lots
of code. For historical reason newlib defines Long as int32_t instead.
This works fine, as long as floating point exceptions are not enabled.
The conversion to 32 bit int can lead to a FE_INVALID situation.
Example:
const char *str = "121645100408832000.0";
char *ptr;
feenableexcept (FE_INVALID);
strtod (str, &ptr);
This leads to the following situation in strtod
double aadj;
Long L;
[...]
L = (Long)aadj;
For instance, on x86_64 the code here is
cvttsd2si %xmm0,%eax
At this point, aadj is 2529648000.0 in our example. The conversion to
32 bit %eax results in a negative int value, thus the conversion is
invalid. With feenableexcept (FE_INVALID), a SIGFPE is raised.
Fix this by always using 64 bit ints here if double is not a 32 bit type
to avoid this type of FP exceptions.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Classical function call recursion wastes a lot of stack space.
Each recursion level requires a full stack frame comprising all
local variables and additional space as dictated by the
processor calling convention.
This implementation instead stores the variables that are unique
for each recursion level in a parameter stack array, and uses
iteration to emulate recursion. Function call recursion is not
used until the array is full.
To ensure the stack consumption isn't worsened by this design, the
size of the parameter stack array is chosen to be similar to the
stack frame excluding the array. Each function call recursion level
can handle 8 iterative recursion levels.
Stack consumption will worsen when sorting tiny arrays that do not
need recursion (of 6 elements or less). It will be about equal for
up to 15 elements, and be an improvement for larger arrays. The best
case improvement is a stack size reduction down to about one quarter
of the stack consumption before the change.
A design where the parameter stack array is large enough for the
worst case recursion level was rejected because it would worsen
the stack consumption when sorting arrays smaller than about 1500
elements. The worst case is 31 levels on a 32-bit system.
A design with a dynamic parameter array size was rejected because
of limitations in some compilers.
The qsort algorithm splits the input array in three parts. The
left and right parts may need further sorting. One of them is
sorted by recursion, the other by iteration. This update ensures
that it is the smaller part that is chosen for recursion.
By choosing the smaller part, each recursion level will handle
less than half the array of the previous recursion level. Hence
the recursion depth is bounded to be less than log2(n) i.e. 1
level per significant bit in the array size n.
The update also includes code comments explaining the algorithm.
Newlib has a build configuration where syscalls can be directly
embedded in the newlib library rather than relying on libgloss.
This configuration was broken recently by an update to the libgloss
support for Arm that was not propagated to the syscalls interface in
newlib itself. This patch restores the build. It's essentially a
copy of https://sourceware.org/ml/newlib/2018/msg00128.html but there
are some other minor cleanups and changes that I've made at the same
time. None of those cleanups affect functionality.
The prototypes of the following functions have been updated: _link,
_sbrk, _getpid, _write, _swiwrite, _lseek, _swilseek, _read and
_swiread.
Signed-off-by: Richard Earnshaw <Richard.Earnshaw@arm.com>
E.g. arm ABI requires -fshort-enums for bare-metal toolchains.
Given there are only 29 category enums, the compiler chooses an
8 bit enum type, so a size of 11 bits for the bitfield leads to
a compile time error:
error: width of 'cat' exceeds its type
enum category cat: 11;
^~~
Fix this by aligning the size of the category members to byte
borders.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Scripts do not try to acquire Unicode data by best-effort magic anymore.
Options supported:
-h for help
-i to copy Unicode data from /usr/share/unicode/ucd first
-u to download Unicode data from unicode.org first
If (despite of -i or -u if given) the necessary Unicode files are not
available locally, table generation is skipped, but no error code is
returned, so not to obstruct the build process if called from a Makefile.
E.g. arm ABI requires -fshort-enums for bare-metal toolchains.
Given there are only 29 category enums, the compiler chooses an
8 bit enum type, so a size of 11 bits for the bitfield leads to
a compile time error:
error: width of 'cat' exceeds its type
enum category cat: 11;
^~~
Fix this by aligning the size of the category members to byte
borders.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
touupper and toulower didn't return a value in all cases. Worse,
this only broke Cygwin when building without optimization for debug
purposes.
Why GCC neglects to notice this is a mystery.
While at it, fix formatting.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
The tow* functions use an included case conversion table which can be
generated from Unicode data.
The isw* functions use a character categories table (provided by
categories.c) which can be generated from Unicode data.
Delegation between current-locale and specific-locale-dependent functions
was reverted towards the generic locale-dependent functions (*_l.c);
this is however only relevant on systems with non-Unicode wide character
locales, thus not on Cygwin.
Table categories.t and tag enumeration categories.cat provide
character class data for most of the isw* functions.
These data are generated from Unicode data.
Linux and FreeBSD use int as well. In addition, this fixes an Ada
incompatiblity problem on 64-bit targets. See also GCC:
gcc/ada/libgnarl/s-osinte__rtems.ads
Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
Locale modifier @cjkwide makes Unicode "ambiguous width" characters
wide. So ambiguous width characters can be enforced to have width 2
even in non-CJK locales. This gives e.g. users of "Powerline symbols"
the opportunity to adjust their width to the desired behaviour (and the
behaviour apparently expected by some tools) without having to set a CJK
locale and without losing consistence of terminal character width with
wcwidth/wcswidth locale width.
At least with Binutils 2.30 and GCC 7.3 we need symbol definitions
without the leading underscore.
Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
This is a NetBSD-specific detail which does not apply to Newlib, causing
linking issues in certain scenarios:
https://cygwin.com/ml/cygwin/2018-01/msg00189.html
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
New optimized powf, logf, log2f, expf and exp2f yield worse performance
on Arm targets with only single precision instructions because the
double precision arithmetic is then implemented via softfloat routines.
This patch uses the old implementation when double precision
instructions are not available on Arm targets.
Testing: Built newlib with GCC's rmprofile Arm multilibs and compared
before/after -> only the above functions are changed and calls to them
(name change from logf to __ieee754_logf and similar). Testing the
changed function on a panel of values yields the same result before the
original patches to improve them and after this one. Double checking the
performance by looping the same panel of values being tested on Arm
Cortex-M4 does show the performance regression is fixed.
This patch fixes a syntax error in exit.c that was introduced during the
ANSI-fication of newlib. The patch fixes a compile-time issue that arises when
newlib is configured with the --enable-lite-exit feature.
Code path for _MB_CAPABLE scans for the '%' character and advances
'fmt' pointer past '%'. Code path for !_MB_CAPABLE leaved fmt pointing
to '%', which caused the state machine to go from START to DONE state
immediately.
Neither upstream FreeBSD nor glibc ever call fflush from ftell
and friends. In border cases it has the tendency to return
wrong or unexpected values, for instance on block devices.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Make prototype of _kill() always visible when _COMPILING_NEWLIB is
defined. This makes <sys/signal.h> consistent with the use of
_COMPILING_NEWLIB in <sys/unistd.h>, <sys/times.h>, etc.
Updated patch to use 0.0f in addition to calling rintf.
Tested same way as before, with a testcase that triggers the code and
make check.
OK?
newlib/
* libm/math/wf_pow.c (powf): Call rintf instead of rint. Use 0.0f
for compare.
To update my email address to my current employer. Specifix died quite a while
ago, and I've had two jobs in the interim.
newlib/
* MAINTAINERS: Update my email address.
To follow up the thread starting at [1], since all uses of TRAD_SYNOPSIS
have been removed, and all uses of ANSI_SYNOPSIS have been renamed to
SYNOPSIS, we can now warn about the use of these.
[1] https://sourceware.org/ml/newlib/2017/msg01182.html
Signed-off-by: Jon Turney <jon.turney@dronecode.org.uk>
Discard QUICKREF sections, rather than writing them to stderr
Discard MATHREF sections, rather than discarding as an error
Pass NOTES sections through to texinfo, rather than discarding as an error
Don't redirect makedoc stderr to .ref file
Remove makedoc output on error
Remove .ref files from CLEANFILES
Regenerate Makefile.ins
Signed-off-by: Jon Turney <jon.turney@dronecode.org.uk>
Old BSD bug: While ^ is recognized and the set of matching characters
is negated, the code neglects to increment the pointer pointing to the
matching characters. Thus, on a negation expression like %[^xyz], the
matching doesn't only stop at x, y, or z, but incorrectly also on ^.
Fix this by setting the start pointer after recognizing the ^.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
The following functions are also guarded in glibc:
fwprintf, swprintf, wprintf, vfwprintf, vswprintf, vwprintf.
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
* vfscanf: per POSIX, if the target type is wchar_t, the width is
counted in (multibyte) characters, not in bytes.
* vfscanf: Handle UTF-8 multibyte sequences converted to surrogate
pairs on UTF-16 systems.
* vfwscanf: Don't count high surrogates in input against field width
counting. Per POSIX, input is
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
The width value keeps the maximum field width. This is the maximum
field width of the *input*. It's *never* to be used in conjunction
with the number of bytes or characters written to the output argument.
However, especially in vfwscanf, the code is partially taken from
NetBSD which erroneously subtracts the number of multibyte chars
written to the argument from the width variable, thus potentially
subtracting up to MB_CUR_MAX from width for a single character in
the input stream.
To make matters worse, the previous patch adding %m added basically
the same mistake for 'c' type input.
Fix it.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
* The new code is guarded with _WANT_IO_POSIX_EXTENSIONS, but
this is automatically enabled with _WANT_IO_C99_FORMATS for now.
* vfscanf neglects to implement %l[, so %ml[ is not implemented yet
either.
* Sidenote: vfwscanf doesn't allow ranges in %[ yet. Strictly this
is allowed per POSIX, but it differes from vfscanf as well as from
glibc.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
The implementation is from NetBSD, with the addition of feature test macros
for readlink. glibc also wraps the following functions:
confstr, getdomainname, getgroups, gethostname, getlogin_r, getwd, pread,
readlinkat, ttyname_r.
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
The implementation is mostly from NetBSD, except for switching fgets to
pure inline, and the addition of fgets_unlocked, fread, and fread_unlocked
for parity with glibc. The following functions are also guarded in glibc:
asprintf, dprintf, fprintf, printf, vasprintf, vdprintf, vfprintf, vprintf.
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
The implementation is from NetBSD, with the addition of mempcpy (a GNU
extension) for parity with glibc and libssp.
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
The Object Size Checking (-D_FORTIFY_SOURCE=*) functionality provides
wrappers around functions suspectible to buffer overflows. While
independent from Stack Smashing Protection (-fstack-protector*), they
are often used and implemented together.
While GCC also provides an implementation in libssp, it is completely
broken (CVE-2016-4973, RHBZ#1324759) and seemingly unfixable, as there
is no reliable way for a preprocessor macro to trigger a link flag.
Therefore, adding this here is necessary to make it work.
Note that this does require building gcc with --disable-libssp and
gcc_cv_libc_provides_ssp=yes.
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
Compiling with any of the -fstack-protector* flags requires the
__stack_chk_guard data import (which needs to be initialized) and the
__stack_chk_fail{,_local} functions. While GCC's own libssp can provide
these, it is better that we provide these ourselves. The implementation
is custom due to being OS-specific.
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
The special handling of %\0 in [w]scanf is flawed. It's just a
matching failure and should be handled as such. scanf also
fakes an int input value on %X with X being an invalid conversion
char. This is also just a matching failure and should be handled
the same way as %\0.
There's no indication of the reason for this "disgusting
backwards compatibility hacks" in the logs, given this
code made it into newlib before setting up the CVS repo.
Just handle these cases identically as matching failures.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Since commit 8128f5482f, we have all the
non-tracing functions listed in posixoptions(7). The tracing functions
are gated by their own option, and are obsolecent anyway.
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
Difference to Linux: We can't create files which don't show up
in the filesystem due to OS restrictions. As a kludge, make a
(half-hearted) attempt to hide the file in the filesystem.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
The variable doesn't follow the convention of having the same name as
the function it's bundled with. Furthermore, it clashes with the
variable of the same name in newlib/libc/stdlib/calloc.c.
Signed-off-by: Florian Schmidt <florian.schmidt@neclab.eu>
RTEMS provides the option to have a global or per-thread reentrancy
as part of application configuration. As part of this, RTEMS provides
the implementation of __getreent() as appropriate. Allow the target
to determine if this method is present in libc.a.
The recently added new math code inlines error handling instead of using
error handling wrappers around __ieee754* internal symbols, and thus the
__ieee754* symbols are no longer provided.
However __ieee754_expf and __ieee754_logf are used in the implementation
of a number of other math functions. These symbols are safe to redirect
to the external expf and logf symbols, because those names are always
reserved when single precision math functions are reserved and the
additional error handling code is either not reached or there will be
an error in the final result that will override an internal spurious
errno setting.
For consistency all of __ieee754_expf, __ieee754_logf and __ieee754_powf
are redirected using a macro.
Based on code from https://github.com/ARM-software/optimized-routines/
This patch adds a highly optimized generic implementation of expf,
exp2f, logf, log2f and powf. The new functions are not only
faster (6x for powf!), but are also smaller and more accurate.
In order to achieve this, the algorithm uses double precision
arithmetic for accuracy, avoids divisions and uses small table
lookups to minimize the polynomials. Special cases are handled
inline to avoid the unnecessary overhead of wrapper functions and
set errno to POSIX requirements.
The new functions are added under newlib/libm/common, but the old
implementations are kept (in newlib/libm/math) for non-IEEE or
pre-C99 systems. Targets can enable the new math code by defining
__OBSOLETE_MATH_DEFAULT to 0 in newlib/libc/include/machine/ieeefp.h,
users can override the default by defining __OBSOLETE_MATH.
Currently the new code is enabled for AArch64 and AArch32 with VFP.
Targets with a single precision FPU may still prefer the old
implementation.
libm.a size changes:
arm: -1692
arm/thumb/v7-a/nofp: -878
arm/thumb/v7-a+fp/hard: -864
arm/thumb/v7-a+fp/softfp: -908
aarch64: -1476
In order to avoid the year 2038 problem, define time_t to a signed
integer with at least 64-bits. The type for time_t can be forced to
long with the --enable-newlib-long-time_t configure option or with the
_USE_LONG_TIME_T system configuration define.
Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
In case time_t is long, then the cast to long is a nop. In case time_t
is __int_least64_t, then the cast to long may truncate the value before
the division.
Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
Newlib uses _times_r() in clock(). The problem is that the _times_r()
clock frequency is defined by sysconf(_SC_CLK_TCK). The clock frequency
of clock() is the constant CLOCKS_PER_SEC.
FreeBSD uses getrusage() for clock(). Since RTEMS has only one process,
the implementation can be simplified.
Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
In C++, the usage of static inline functions for getchar_unlocked and
putchar_unlocked may result in error messages like
error: ‘_putchar_unlocked’ was not declared in this scope
Fix this by not using the _getchar_unlocked and _putchar_unlocked
macros in C++.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Remove local strnstr() implementation to fix compile error:
newlib/libc/iconv/lib/aliasesi.c:53:8: error: conflicting types for 'strnstr'
_DEFUN(strnstr, (haystack, needle, length),
^
In file included from newlib/libc/iconv/lib/aliasesi.c:29:0:
newlib/libc/include/string.h:125:10:
note: previous declaration of 'strnstr' was here
char *strnstr(const char *, const char *, size_t) __pure;
^~~~~~~
Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
This reverts most of commit 979d467ff6.
We cannot avoid some bareword attributes until clang is fixed to
properly support __-decorated attributes; see this bug:
https://bugs.llvm.org/show_bug.cgi?id=34319
The macros in question expand to the empty string under gcc, so
only compilation under clang is affected, and since clang has the
bug, the obvious solution is to roll back the changes, and document
the issue.
Signed-off-by: Eric Blake <eblake@redhat.com>
- For prevent confuse about what BSD license variant we used, 2- or
3-clause license, we change the license to FreeBSD license to make
it unambiguously refers to the 2-clause license.
Always use the __-decorated form of an attribute name in public
headers, as the bareword form is in the user's namespace, and we
don't want compilation to break just because the user defines the
bareword to mean something else.
Signed-off-by: Eric Blake <eblake@redhat.com>
Add internal inline functions _getchar_unlocked() and
_putchar_unlocked() if __CUSTOM_FILE_IO__ is not defined. These
functions get _REENT only once. Use them for getchar_unlocked() and
putchar_unlocked(). Define getchar() and putchar() to these unlocked
internal functions if __SINGLE_THREAD__ is defined, otherwise use the
external functions to use proper locking of the FILE object.
Assumes that __SINGLE_THREAD__ is not defined if __CYGWIN__ is defined.
Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
This is copied from musl (MIT license). This is newer and more thorough
than that of FreeBSD currently shipped only on Cygwin.
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
With this change the arm platform can now be fully compiled with Clang.
Tested by comparing the output with GCC 4.8.2, and Clang 4.0, using a
variety of arches, big/little endianness, and arm/thumb mode to verify
the generated assembly output matches between GCC vs Clang with UAL, and
also GCC with UAL vs GCC with non-UAL, for all preprocessor code blocks.
The only difference found is an extra nop at the end of the function
when compiled with GCC using armv7-a/thumb/little-endian/-O2 compared to
Clang. The nop is not emitted when compiled in big-endian mode.
Include <strings.h> in <string.h> if __BSD_VISIBLE like on FreeBSD.
Remove redundant declarations from <string.h>. Make ffsl(), ffsll(),
strncasecmp(), strcasecmp_l(), and strncasecmp_l() visible via
__BSD_VISIBLE instead of __GNU_VISIBLE. Add fls(), flsl(), and flsll()
to <strings.h> if __BSD_VISIBLE.
Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
Use memset() to implement bzero() to profit from machine-specific
memset() optimizations.
Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
In Newlib, the stdio streams are defined to thread-specific pointers
_reent::_stdin, _reent::_stdout and _reent::_stderr. In case
_REENT_SMALL is not defined, then these pointers are initialized via
_REENT_INIT_PTR() or _REENT_INIT_PTR_ZEROED() to thread-specific FILE
objects provided via _reent::__sf[3]. There are two problems with this
(at least in case of RTEMS).
(1) The thread-specific FILE objects are closed by _reclaim_reent().
This leads to problems with language run-time libraries that provide
wrappers to the C/POSIX stdio streams (e.g. C++ and Ada), since they
use the thread-specific FILE objects of the initialization thread. In
case the initialization thread is deleted, then they use freed memory.
(2) Since thread-specific FILE objects are used with a common output
device via file descriptors 0, 1 and 2, the locking at FILE object level
cannot ensure atomicity of the output, e.g. a call to printf().
Introduce a new Newlib configuration option _REENT_GLOBAL_STDIO_STREAMS
to enable the use of global stdio FILE objects.
As a side-effect this reduces the size of struct _reent by more than
50%.
The _REENT_GLOBAL_STDIO_STREAMS should not be used without
_STDIO_CLOSE_PER_REENT_STD_STREAMS.
Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
Commit 8a3b3bb4d7 changed the guard on
some functions from _POSIX_THREADS to __POSIX_VISIBLE. As a consequence,
some use of siginfo_t and pthread_t became visible under configurations
where _POSIX_THREADS is unset but __POSIX_VISIBLE is. Build then fails
because the definition of those types are still unavailable.
This commit make those type definition visible for __POSIX_VISIBLE
configurations. This requires moving the siginfo_t definition out of the
RTEMS specific definitions in sys/signal.h while still guarding it
against cygwin case.
There are two common sigpause variants, both of which take an int argument.
If you request _XOPEN_SOURCE or _GNU_SOURCE, you get the System V version,
which removes the given signal from the process's signal mask; otherwise
you get the BSD version, which sets the process's signal mask to the given
value.
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
While POSIX allows these functions to also be defined as macros in C, in
C++ this is not allowed, and prevents these names (particularly feof) from
being used in a custom namespace.
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
For whatever reason FreeBSD renames several functions provided by
<arpa/inet.h> and uses weak references to provide the standard function
names. This causes problems on targets lacking proper support for weak
references. We do not need this function renaming on RTEMS.
Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
Don't over-read memory returned by _DTOA_R, and never write to it
since the result might be a string literal.
For example, when doing:
swprintf(tt, 20, L"%.*f", 6, 0.0);
we will get back "0".
Instead, write the result returned by _DTOA_R to the output buffer.
After this, write the 0 chars directly to the the output buffer
(if there are any). This also has the (marginal) advantage that
we read/write less memory overall.
According to the FreeBSD man page BIT_CMP() returns true in case the two
sets are NOT equal.
Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
Add the POSIX header files
* arpa/inet.h
* net/if.h
* netdb.h
* netinet/in.h
* netinet/tcp.h
* sys/socket.h
* sys/syslog.h
* sys/uio.h
* sys/un.h
* syslog.h
* termios.h
and their dependencies for RTEMS. The origin of these files is the
latest FreeBSD.
Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
Make the RTEMS <sys/cpuset.h> compatible with the latest FreeBSD
version.
Fix the CPU_COPY() parameter order, see also:
https://devel.rtems.org/ticket/3023
Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
Use a dedicated header file <machine/_bitcount.h> to avoid cyclic header
dependencies in future changes.
Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
The implementation of the POSIX access() function is nothing machine
specific like memcpy(), etc. Move it back to the system domain. This
avoids problems due to the include search order of the Newlib/GCC build
which picks up machine includes before system includes.
Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
Increase the MSIZE for RTEMS to be in line with the latest FreeBSD
version. The legacy network stack of RTEMS will provides its own
definition.
Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
Update the RTEMS <machine/param.h> and <sys/param.h> to be compatible
with the latest FreeBSD version.
Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
previous commit 4c90db7bc8 introduced
a compile time error because libm/common/s_infconst.c used the remove
__fmath, __dmath, and __ldmath union types.
Since this is very old, and unused for a very long time, just drop the
file and thus the __infinity constants entirely.
Exception: Cygwin exports __infinity from the beginning. There's a very,
VERY low probability that any existing executable or lib still uses this
constant, but we just keep it in for backward compat, nevertheless.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Renumber cluase 4 to 3, per what everybody else did when BSD granted
them permission to remove clause 3. My insistance on keeping the same
numbering for legal reasons is too pedantic, so give up on that point.
Submitted by: Jan Schaumann <jschauma@stevens.edu>
Pull Request: https://github.com/freebsd/freebsd/pull/96
Split the QUEUE_MACRO_DEBUG into QUEUE_MACRO_DEBUG_TRACE and
QUEUE_MACRO_DEBUG_TRASH.
Add the debug macrso QMD_IS_TRASHED() and QMD_SLIST_CHECK_PREVPTR().
Document these in queue.3.
Reviewed by: emaste
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D3984
Add two new macros, SLIST_CONCAT and LIST_CONCAT. Note in both the
queue.h header file and in the queue.3 manual page that they are O(n) so
should be used only in low-usage paths with short lists (otherwise an
STAILQ or TAILQ should be used).
Reviewed by: kib
Make the system queue header file fully usable within C++ programs by
adding macros to define class lists.
This change is backwards compatible for all use within C and C++
programs. Only C++ programs will have added support to use the queue
macros within classes. Previously the queue macros could only be used
within structures.
The queue.3 manual page has been updated to describe the new
functionality and some alphabetic sorting has been done while
at it.
Differential Revision: https://reviews.freebsd.org/D2745
PR: 200827 (exp-run)
MFC after: 2 weeks
Add new FOREACH_FROM variants of the queue(3) FOREACH macros which can
optionally start the traversal from a previously found element by
passing the element in as "var". Passing a NULL "var" retains the same
semantics as the regular FOREACH macros.
Kudos to phk for suggesting the "FROM" suffix instead of my original
proposal.
Reviewed by: jhb (previous version), rpaulo
MFC after: 1 week
Renumber cluase 4 to 3, per what everybody else did when BSD granted
them permission to remove clause 3. My insistance on keeping the same
numbering for legal reasons is too pedantic, so give up on that point.
Submitted by: Jan Schaumann <jschauma@stevens.edu>
Pull Request: https://github.com/freebsd/freebsd/pull/96
For example, the FreeBSD GCC (4.2.1) has a spotty support for that
feature. If the static keyword is used with an unnamed array parameter
in a function declaration, then the compilation fails with:
error: static or type qualifiers in abstract declarator
The feature does work if the parameter is named.
So, the restriction introduced in this commit can be removed when all
affected function prototypes have the workaround.
MFC after: 1 week
Sponsored by: Panzura
For consistency with the qualifiers added in r310977, define a new
qualifier _Null_unspecified which is also defined in clang 3.7+.
Add two new macros:
__NULLABILITY_PRAGMA_PUSH
__NULLABILITY_PRAGMA_POP
These are for use in headers when we want avoid noisy warnings if
some pointers are left without nullability annotations.
These are added with way ahead of their first use to teach the GCC
ports headers of their existance before their first use.
This was meant to be used by a future FORTIFY_SOURCE implementation.
Probably for good, FORTIFY_SOURCE and this particular GCCism were never
well supported by clang or other compilers. Furthermore, the technology
has long since been replaced by either static checkers, sanitizers, or
even just the strong stack protector that was enabled by default.
Drop __gnu_inline to avoid cluttering the headers.
MFC after: 5 days
Add two new qualifiers for use by the static checkers:
_Nonnull
The _Nonnull nullability qualifier indicates that null is not a meaningful
value for a value of the _Nonnull pointer type.
_Nullable
The _Nullable nullability qualifier indicates that a value of the
_Nullable pointer type can be null.
These were introduced in Clang 3.7. For more information, see:
http://clang.llvm.org/docs/AttributeReference.html#nonnull
We add these now without using them so that the GCC ports have time to
pick up the header change.
Hinted by: Android Bionic libc [1]
Also seen in: Apple's Libc-1158.20.4
[1]
baa2a973bd
C99 allows array function parameters to use the static keyword for their
sizes. This tells the compiler that the parameter will have at least the
specified size, and calling code will fail to compile if that guarantee is
not met. However, this syntax is not legal in C++.
This commit reverts r300824, which worked around the problem for
sys/md5.h only, and introduces a new macro: min_size(). min_size(x) can
be used in headers as a static array size, but will still compile in C++
mode.
Reviewed by: cem, ed
MFC after: 4 weeks
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D8277
fix a typo in __STDC_VERSION__ in __min_size requirements
MFC after: 1 week
Sponsored by: Panzura
It clashes with the one in libc++'s <atomic> header.
(Previously, the _Atomic() macro was defined in <stdatomic.h>, which is
only for use with C11, but for various reasons it was moved to its
current location in r251804.)
Discussed with: bdrewery, ed
MFC after: 2 weeks
Some architectures like ARM encode the short enum option state in the
object file and the linker checks that this option is consistent for all
objects of an executable. In case applications use -fno-short-enums,
then this leads to linker warnings. Use the enum __packed attribute for
the relevent enums to avoid the -fshort-enums compiler option. This
attribute is at least available on GCC, LLVM/clang and the Intel
compiler.
Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
The locale_t type is provided by <xlocale.h> on Linux, FreeBSD, and Darwin.
While, like on some of those systems, it is automatically included by
<locale.h> with the proper feature test macros, its presence under this
particular name is still presumed in real-world software.
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
Problem:
After passing locales created by 'duplocale' to 'uselocale',
referencing 'MB_CUR_MAX', which is actually expanded to
'__locale_mb_cur_max()' by preprocessors, causes segmentation faults.
Direct use of locales from 'newlocale' does not cause the problem.
This is the problem of 'duplocale'.
$ echo $LANG
ja_JP.UTF-8
$ cat test.c
#include <stdlib.h>
#include <locale.h>
volatile int var;
int main(void) {
locale_t const loc = newlocale(LC_ALL_MASK, "", NULL);
locale_t const dup = duplocale(loc);
locale_t const old = uselocale(dup);
var = MB_CUR_MAX; /* <-- crashes here */
uselocale(old);
freelocale(dup);
freelocale(loc);
return 0;
}
$ gcc test.c
$ ./a
Segmentation fault (core dumped)
# Note: "core dumped" in the above message was actually written in
# Japanese, but I translated the part to post a mail in English.
Bug:
In the beginning of '__loadlocale' (newlib/libc/locale/locale.c:501),
there is a code which checks if the operations can be skipped:
> /* Avoid doing everything twice if nothing has changed. */
> if (!strcmp (new_locale, loc->categories[category]))
> return loc->categories[category];
While, in the function '_duplocale_r' (newlib/libc/locale/
duplocale.c), '__loadlocale' is called as in the quoted codes:
> /* If the object is not a "C" locale category, copy it. Just call
> __loadlocale. It knows what to do to replicate the category. */
> tmp_locale.lc_cat[i].ptr = NULL;
> tmp_locale.lc_cat[i].buf = NULL;
> if (!__loadlocale (&tmp_locale, i, tmp_locale.categories[i]))
> goto error;
This call of '__loadlocale' results in the skip check being
!strcmp(tmp_locale.categories[i], tmp_locale.categories[i]),
which is always true. This means that the actual operations of
'__loadLocale' will never be performed for 'duplocale'.
Fix:
The call of '__loadlocale' in '_duplocale_r' is modified.
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>