Commit Graph

20698 Commits

Author SHA1 Message Date
Gleb Smirnoff 301991542a Introduce flag IFF_NEEDSEPOCH
that marks Ethernet interfaces that supposedly may call into ether_input()
without network epoch.

They all need to be reviewed before 13.0-RELEASE.  Some may need
be fixed.  The flag is not planned to be used in the kernel for
a long time.
2022-07-11 11:52:46 +02:00
Michael Tuexen ebbb6536b7 Add flags for upcoming patches related to improved ECN handling.
No functional change.

Submitted by: Richard Scheffenegger
Reviewed by: rgrimes@, tuexen@
Differential Revision: https://reviews.freebsd.org/D22429
2022-07-11 11:52:46 +02:00
Edward Tomasz Napierala 0c4d87ca5f Make use of the stats(3) framework in the TCP stack.
This makes it possible to retrieve per-connection statistical
information such as the receive window size, RTT, or goodput,
using a newly added TCP_STATS getsockopt(3) option, and extract
them using the stats_voistat_fetch(3) API.

See the net/tcprtt port for an example consumer of this API.

Compared to the existing TCP_INFO system, the main differences
are that this mechanism is easy to extend without breaking ABI,
and provides statistical information instead of raw "snapshots"
of values at a given point in time.  stats(3) is more generic
and can be used in both userland and the kernel.

Reviewed by:	thj
Tested by:	thj
Obtained from:	Netflix
Relnotes:	yes
Sponsored by:	Klara Inc, Netflix
Differential Revision:	https://reviews.freebsd.org/D20655
2022-07-11 11:52:46 +02:00
David Bright 0c854dd6d1 Jail and capability mode for shm_rename;
add audit support for shm_rename

Co-mingling two things here:

  * Addressing some feedback from Konstantin and Kyle re: jail,
    capability mode, and a few other things
  * Adding audit support as promised.

The audit support change includes a partial refresh of OpenBSM from
upstream, where the change to add shm_rename has already been
accepted. Matthew doesn't plan to work on refreshing anything else to
support audit for those new event types.

Submitted by:	Matthew Bryan <matthew.bryan@isilon.com>
Reviewed by:	kib
Relnotes:	Yes
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D22083
2022-07-11 11:52:46 +02:00
John Baldwin 12fb531a70 Add a TOE KTLS mode and a TOE hook for allocating TLS sessions.
This adds the glue to allocate TLS sessions and invokes it from
the TLS enable socket option handler.  This also adds some counters
for active TOE sessions.

The TOE KTLS mode is returned by getsockopt(TLSTX_TLS_MODE) when
TOE KTLS is in use on a socket, but cannot be set via setsockopt().

To simplify various checks, a TLS session now includes an explicit
'mode' member set to the value returned by TLSTX_TLS_MODE.  Various
places that used to check 'sw_encrypt' against NULL to determine
software vs ifnet (NIC) TLS now check 'mode' instead.

Reviewed by:	np, gallatin
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D21891
2022-07-11 11:52:46 +02:00
Kyle Evans 1ef7e3904d MFD_*: swap ordering
This API is still young enough that I would expect no one to be dependant on
this yet... Swap the ordering while it's young to match Linux values to
potentially ease implementation of linuxolator syscall, being able to reuse
existing constants.
2022-07-11 11:52:46 +02:00
David Bright 53648039c4 Add an shm_rename syscall
Add an atomic shm rename operation, similar in spirit to a file
rename. Atomically unlink an shm from a source path and link it to a
destination path. If an existing shm is linked at the destination
path, unlink it as part of the same atomic operation. The caller needs
the same permissions as shm_unlink to the shm being renamed, and the
same permissions for the shm at the destination which is being
unlinked, if it exists. If those fail, EACCES is returned, as with the
other shm_* syscalls.

truss support is included; audit support will come later.

This commit includes only the implementation; the sysent-generated
bits will come in a follow-on commit.

Submitted by:	Matthew Bryan <matthew.bryan@isilon.com>
Reviewed by:	jilles (earlier revision)
Reviewed by:	brueffer (manpages, earlier revision)
Relnotes:	yes
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D21423
2022-07-11 11:52:46 +02:00
Kyle Evans 9243caa8d3 Add linux-compatible memfd_create
memfd_create is effectively a SHM_ANON shm_open(2) mapping with optional
CLOEXEC and file sealing support. This is used by some mesa parts, some
linux libs, and qemu can also take advantage of it and uses the sealing to
prevent resizing the region.

This reimplements shm_open in terms of shm_open2(2) at the same time.

shm_open(2) will be moved to COMPAT12 shortly.

Reviewed by:	markj, kib
Differential Revision:	https://reviews.freebsd.org/D21393
2022-07-11 11:52:46 +02:00
Kyle Evans 99b66f5315 Add a shm_open2 syscall to support upcoming memfd_create
shm_open2 allows a little more flexibility than the original shm_open.
shm_open2 doesn't enforce CLOEXEC on its callers, and it has a separate
shmflag argument that can be expanded later. Currently the only shmflag is
to allow file sealing on the returned fd.

shm_open and memfd_create will both be implemented in libc to use this new
syscall.

__FreeBSD_version is bumped to indicate the presence.

Reviewed by:	kib, markj
Differential Revision:	https://reviews.freebsd.org/D21393
2022-07-11 11:52:46 +02:00
Randall Stewart 878b65b3b6 This commit adds BBR (Bottleneck Bandwidth and RTT) congestion control.
This is a completely separate TCP stack (tcp_bbr.ko) that will be built only if
you add the make options WITH_EXTRA_TCP_STACKS=1 and also include the option
TCPHPTS.  You can also include the RATELIMIT option if you have a NIC interface
that supports hardware pacing, BBR understands how to use such a feature.

Note that this commit also adds in a general purpose time-filter which
allows you to have a min-filter or max-filter. A filter allows you to
have a low (or high) value for some period of time and degrade slowly
to another value has time passes. You can find out the details of
BBR by looking at the original paper at:

https://queue.acm.org/detail.cfm?id=3022184

or consult many other web resources you can find on the web
referenced by "BBR congestion control". It should be noted that
BBRv1 (which this is) does tend to unfairness in cases of small
buffered paths, and it will usually get less bandwidth in the case
of large BDP paths(when competing with new-reno or cubic flows). BBR
is still an active research area and we do plan on  implementing V2
of BBR to see if it is an improvement over V1.

Sponsored by:	Netflix Inc.
Differential Revision:	https://reviews.freebsd.org/D21582
2022-07-11 11:52:46 +02:00
Alan Somers ce921ffca8 Reduce namespace pollution from r349233
Define __daddr_t in _types.h and use it in filio.h

Reported by:	ian, bde
Reviewed by:	ian, imp, cem
MFC after:	2 weeks
MFC-With:	349233
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D20715
2022-07-11 11:52:46 +02:00
Alan Somers 5a6ad7c5bc #include <sys/types.h> from sys/filio.h
This fixes world build after r349231

Reported by:	Jenkins
MFC after:	2 weeks
MFC-With:	349231
Sponsored by:	The FreeBSD Foundation
2022-07-11 11:52:46 +02:00
Alan Somers 8fe49db783 Add FIOBMAP2 ioctl
This ioctl exposes VOP_BMAP information to userland. It can be used by
programs like fragmentation analyzers and optimized cp implementations. But
I'm using it to test fusefs's VOP_BMAP implementation. The "2" in the name
distinguishes it from the similar but incompatible FIBMAP ioctls in NetBSD
and Linux.  FIOBMAP2 differs from FIBMAP in that it uses a 64-bit block
number instead of 32-bit, and it also returns runp and runb.

Reviewed by:	mckusick
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D20705
2022-07-11 11:52:46 +02:00
Brooks Davis c42aaaea4f Move 32-bit compat support for FIODGNAME to the right place.
ioctl(2) commands only have meaning in the context of a file descriptor
so translating them in the syscall layer is incorrect.

The new handler users an accessor to retrieve/construct a pointer from
the last member of the passed structure and relies on type punning to
access the other member which requires no translation.

Unlike r339174 this change supports both places FIODGNAME is handled.

Reviewed by:	kib
Obtained from:	CheriBSD
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D17475
2022-07-11 11:52:46 +02:00
Pedro F. Giffuni eb4cbf4fd3 sys: further adoption of SPDX licensing ID tags.
Mainly focus on files that use BSD 3-Clause license.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.
2022-07-11 11:52:46 +02:00
Sebastian Huber 5c0c0e5c77 RTEMS: Remove FreeBSD version tags 2022-07-11 11:52:46 +02:00
Warner Losh 9331071f02 cdefs.h: Remove redundant #ifdefs
Remove redunant #ifdef __GNUC__ inside an #if defined(__GNUC__)
block. They are nops.

Sponsored by:		Netflix
2022-07-11 11:52:46 +02:00
Mark Johnston f537ff8ee5 cdefs: Add a default definition for __nosanitizememory
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2022-07-11 11:52:46 +02:00
Mark Johnston 8801440e4f cdefs: Make __nosanitizeaddress work for KASAN as well
Add __nosanitizememory while I'm here.

Reviewed by:	andrew, kib
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D30126
2022-07-11 11:52:46 +02:00
Warner Losh 65338d7299 cdefs.h: Remove __GNUCLIKE___OFFSETOF, it's unused
__GNUCLIKE___OFFSETOF is unreferenced in the tree, remove it as long
obsolete.

Sponsored by:		Netflix
2022-07-11 11:52:46 +02:00
Alex Richardson 68109f904b Expose clang's alignment builtins and use them for roundup2/rounddown2
This makes roundup2/rounddown2 type- and const-preserving and allows
using it on pointer types without casting to uintptr_t first. Not
performing pointer-to-integer conversions also helps the compiler's
optimization passes and can therefore result in better code generation.
When using it with integer values there should be no change other than
the compiler checking that the alignment value is a valid power-of-two.

I originally implemented these builtins for CHERI a few years ago and
they have been very useful for CheriBSD. However, they are also useful
for non-CHERI code so I was able to upstream them for Clang 10.0.

Rationale from the clang documentation:
Clang provides builtins to support checking and adjusting alignment
of pointers and integers. These builtins can be used to avoid relying
on implementation-defined behavior of arithmetic on integers derived
from pointers. Additionally, these builtins retain type information
and, unlike bitwise arithmetic, they can perform semantic checking on
the alignment value.

There is also a feature request for GCC, so GCC may also support it in
the future: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98641

Reviewed By:	brooks, jhb, imp
Differential Revision: https://reviews.freebsd.org/D28332
2022-07-11 11:52:46 +02:00
Warner Losh c382ecde55 cdefs.h: remove intel_compiler support
The  age  of   the  intel  compiler  support  is  so   old  as  to  be
uninteresting. No recent recports of  intel compiler support have been
received.  Remove  all the  special  case  workarounds for  the  Intel
compiler. Should there be interest in supporting the compiler, contact
me and I'll work with people to make it happen, though I suspect these
instances are more likely to be in the way than to be helpful.

Reviewed by: cem, emaste, vangyzen, dim
Differential Revision: https://reviews.freebsd.org/D26817
2022-07-11 11:52:46 +02:00
Takashi Yano 421dcf72fb Cygwin: console: Fix an issue which causes when realloc() fails. 2022-07-09 14:43:24 +09:00
Sebastian Huber 282d57d2a8 Reduce namespace pollution from <sys/_types.h>
Provide only __daddr_t through <sys/_types.h>.
2022-07-08 06:57:52 +02:00
Sebastian Huber 4ab39e0a85 RTEMS: Declare ioctl() also if _KERNEL is defined
This fixes the following warning in libbsd:

rtems/blkdev.h:200:10: warning: implicit declaration of function 'ioctl'; did
  you mean 'ifioctl'? [-Wimplicit-function-declaration]

Remove unnecessary includes.
2022-07-08 06:57:52 +02:00
Ken Brown 5d4f405d3b Cygwin: redefine some macros for Linux compatibility
Define FD_SETSIZE (<sys/select.h>) to be 1024 by default, and define
NOFILE (<sys/param.h>) to be OPEN_MAX (== 3200) by default.

Remove the comment in <sys/select.h> that FD_SETSIZE should be >=
NOFILE.

Bump API minor.

Addresses: https://cygwin.com/pipermail/cygwin/2022-July/251839.html
2022-07-07 08:22:40 -04:00
Christian Franke d908a30fdc winsup/doc/faq-setup.xml: align help text with setup 2.919 2022-07-07 13:10:46 +02:00
Takashi Yano fc74dbf22f Cygwin: spawn: Treat empty path as the current directory.
- With this patch, the empty path (empty element in PATH or PATH is
  absent) is treated as the current directory as Linux does. This
  feature is added for Linux compatibility, but it is deprecated.
  POSIX notes that a conforming application shall use an explicit
  pathname to specify the current working directory.
Addresses: https://cygwin.com/pipermail/cygwin/2022-June/251730.html
2022-07-05 13:44:56 +09:00
Takashi Yano b3e25f0bc1 Cygwin: console: Fix issue of pasting very long text input again.
- The recent commit "Cygwin: console: Allow pasting very long text
  input." did not fix the issue enough. This patch adds fixes for
  that.
2022-07-05 11:48:40 +09:00
Ken Brown 72f855f32b Cygwin: implement getfacl(1) for socket files
Do this by defining the acl_get method for the fhandler_socket_local
and fhandler_socket_unix classes.  Also define acl_set for these
classes.

Partially addresses: https://cygwin.com/pipermail/cygwin/2022-July/251768.html
2022-07-04 18:55:19 -04:00
Ken Brown 1503d14af1 Cygwin: stdio: don't try again to read after EOF
This reverts commit 1f8f7e2d54, "* libc/stdio/refill.c (__srefill):
Try again after EOF on Cygwin."  If EOF is set on a file, the stdio
input functions will now immediately return EOF rather than trying
again to read.  This aligns Cygwin's behavior to that of Linux.

Addresses: https://cygwin.com/pipermail/cygwin/2022-June/251672.html
2022-07-04 18:55:08 -04:00
Takashi Yano 2c2c2b631b Cygwin: console: Fix new bugs in cons_master_thread().
- The previous commit for console introduced new bugs in error
  handling in cons_master_thread(). This patch fixes that.
2022-07-03 11:02:28 +09:00
Takashi Yano 746c8116dd Cygwin: console: Allow pasting very long text input.
- Currently, if the text longer than 1024 byte is pasted in console,
  some of the text is discarded. This patch fixes the issue.
Addresses: https://cygwin.com/pipermail/cygwin/2022-June/251764.html
2022-07-02 11:24:11 +09:00
Sebastian Huber 27fd806cd7 RTEMS: _KERNEL tweak for <sys/cpuset.h>
If _KERNEL is defined, then do not delcare CPU_ALLOC() and CPU_FREE() since
__cpuset_alloc() and __cpuset_free() are not declared as well.
2022-07-01 07:25:32 +02:00
Takashi Yano 54930ae7d7 Cygwin: poll: Fix a bug on inquiring same fd with different events.
- poll() has a bug that it returns event which is not inquired if
  events are inquired in multiple pollfd entries on the same fd at
  the same time. This patch fixes the issue.
Addresses: https://cygwin.com/pipermail/cygwin/2022-June/251732.html
2022-06-27 10:54:24 +09:00
Stefan Eßer e7ffbdb018 newlib/libc/sys/rtems/include/sys/cpuset.h
Fix typo in source file.

Reported by:	pluknet at gmail.com (Sergey Kandaurov)
2022-06-22 10:28:42 +02:00
Stefan Eßer e927f541f7 Make CPU_SET macros compliant with other implementations
The introduction of <sched.h> improved compatibility with some 3rd
party software, but caused the configure scripts of some ports to
assume that they were run in a GLIBC compatible environment.

Parts of sched.h were made conditional on -D_WITH_CPU_SET_T being
added to ports, but there still were compatibility issues due to
invalid assumptions made in autoconfigure scripts.

The differences between the FreeBSD version of macros like CPU_AND,
CPU_OR, etc. and the GLIBC versions was in the number of arguments:
FreeBSD used a 2-address scheme (one source argument is also used as
the destination of the operation), while GLIBC uses a 3-adderess
scheme (2 source operands and a separately passed destination).

The GLIBC scheme provides a super-set of the functionality of the
FreeBSD macros, since it does not prevent passing the same variable
as source and destination arguments. In code that wanted to preserve
both source arguments, the FreeBSD macros required a temporary copy of
one of the source arguments.

This patch set allows to unconditionally provide functions and macros
expected by 3rd party software written for GLIBC based systems, but
breaks builds of externally maintained sources that use any of the
following macros: CPU_AND, CPU_ANDNOT, CPU_OR, CPU_XOR.

One contributed driver (contrib/ofed/libmlx5) has been patched to
support both the old and the new CPU_OR signatures. If this commit
is merged to -STABLE, the version test will have to be extended to
cover more ranges.

Ports that have added -D_WITH_CPU_SET_T to build on -CURRENT do
no longer require that option.

The FreeBSD version has been bumped to 1400046 to reflect this
incompatible change.

Reviewed by:	kib
MFC after:	2 weeks
Relnotes:	yes
Differential Revision:	https://reviews.freebsd.org/D33451
2022-06-22 10:28:42 +02:00
Stefan Eßer 6af6e29552 sys/_bitset.h: Fix fall-out from commit 5e04571cf3c
The changes to the bitset macros allowed sched.h to be included
into userland programs without name space pollution due to BIT_*
and BITSET_* macros.

The definition of a "struct bitset" had been overlooked. This name
space pollution caused the build of port print/miktex to fail.

This commit makes the definition of struct bitset depend on the
same condition as the visibility of the BIT_* and BITSET_* macros,
i.e. needs _KERNEL or _WANT_FREEBSD_BITSET to be defined before
including sys/_bitset.h.

It has been tested with "make universe" since a prior attempt to
fix the issue broke the PowerPC64 kernel build.

This commit shall be MFCed together with commit 5e04571cf3c.

Reported by:    arrowd
MFC after:      1 month
2022-06-22 10:15:26 +02:00
Stefan Eßer 3af17aef2b sys/_bitset.h: revert commit 74e014dbfab
It caused kernel build for PowerPC64 to fail.

A different patch is being tested with make universe to make sure it
works on all architectures.

MFC after:	1 month<N [day[s]|week[s]|month[s]].  Request a reminder email>
2022-06-22 10:15:26 +02:00
Stefan Eßer c78c56c06d sys/_bitset.h: Fix fall-out from commit 5e04571cf3c
There is a reference to malloc() in #define __BITSET_ALLOC. Even
though this macro is only defined but not used, it causes the lang/gcc
ports to fail. The gcc ports "poison" a number of functions including
malloc() and prevent their use (including in macro definitions).

This commit moved the declaration of __BITSET_ALLOC into the
conditional block that depends on _KERNEL or _WANT_FREEBSD_BITSET
being defined.

There is no use of __BITSET_ALLOC in the FreeBSD sources, and userland
programs that want to use BITSEC_ALLOC will define _WANT_FREEBSD_BITSET
anyway.

This patch has been tested by building lang/gcc11 and a successful
make buildworld.

This commit shall be MFCed together with commit 5e04571cf3c.

MFC after:	1 month
2022-06-22 10:15:26 +02:00
Konstantin Belousov 2f6651097e sys/_bitset.h: Fix fall-out from commit 5e04571cf3c
The changes to the bitset macros allowed sched.h to be included into
userland programs without name space pollution due to BIT_* and
BITSET_* macros.

The definition of a global variable "bitset" had been overlooked.
This name space pollution caused a compile failure in print/miktex.

This commit renames the bitset variable to __bitset with the same
mapping back to the bitset if _KERNEL or _WANT_FREEBSD_BITSET is
defined.

This fix has been suggested by kib. It has been tested to let the
build of the print/miktex port succeed and to not break buildworld.

This commit shall be MFCed together with commit 5e04571cf3c.

Reported by:	arrowd
MFC after:	1 month
2022-06-22 10:15:26 +02:00
Stefan Eßer a1071cb178 sys/bitset.h: reduce visibility of BIT_* macros
Add two underscore characters "__" to names of BIT_* and BITSET_*
macros to move them to the implementation name space and to prevent
a name space pollution due to BIT_* macros in 3rd party programs with
conflicting parameter signatures.

These prefixed macro names are used in kernel header files to define
macros in e.g. sched.h, sys/cpuset.h and sys/domainset.h.

If C programs are built with either -D_KERNEL (automatically passed
when building a kernel or kernel modules) or -D_WANT_FREENBSD_BITSET
(or this macros is defined in the source code before including the
bitset macros), then all macros are made visible with their previous
names, too. E.g., both __BIT_SET() and BIT_SET() are visible with
either of _KERNEL or _WANT_FREEBSD_BITSET defined.

The main reason for this change is that some 3rd party sources
including sched.h have been found to contain conflicting BIT_*
macros.

As a work-around, parts of shed.h have been made conditional and
depend on _WITH_CPU_SET_T being set when sched.h is included.
Ports that expect the full functionality provided by sched.h need
to be built with -D_WITH_CPU_SET_T. But this leads to conflicts if
BIT_* macros are defined in that program, too.

This patch set makes all of sched.h visible again without this
parameter being passed and without any name space pollution due
to BIT_* macros becoming visible when sched.h is included.

This patch set will be backported to the STABLE branches, but ports
will need to use -D_WITH_CPU_SET_T as long as there are supported
releases that do not contain these patches.

Reviewed by:	kib, markj
MFC after:	1 month
Relnotes:	yes
Differential Revision:	https://reviews.freebsd.org/D33235
2022-06-22 10:15:26 +02:00
Konstantin Belousov 4ac3ee88c7 sched.h: add CPU_EQUAL() for better compatibility with Linux
Reviewed by:	jhb
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D32901
2022-06-22 10:15:26 +02:00
Mark Johnston 6070714e0f cpuset(9): Add CPU_FOREACH_IS(SET|CLR) and modify consumers to use it
This implementation is faster and doesn't modify the cpuset, so it lets
us avoid some unnecessary copying as well.  No functional change
intended.

This is a re-application of commit
9068f6ea697b1b28ad1326a4c7a9ba86f08b985e.

Reviewed by:	cem, kib, jhb
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D32029
2022-06-22 10:15:26 +02:00
Mark Johnston 27e8401c46 bitset: Reimplement BIT_FOREACH_IS(SET|CLR)
Eliminate the nested loops and re-implement following a suggestion from
rlibby.

Add some simple regression tests.

Reviewed by:	rlibby, kib
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D32472
2022-06-22 10:15:26 +02:00
Mark Johnston 96ddb4055e Revert "cpuset(9): Add CPU_FOREACH_IS(SET|CLR) and modify consumers to use it"
This reverts commit 9068f6ea697b1b28ad1326a4c7a9ba86f08b985e.

The underlying macro needs to be reworked to avoid problems with control
flow statements.

Reported by:	rlibby
2022-06-22 10:15:26 +02:00
Mark Johnston 112245b78f cpuset(9): Add CPU_FOREACH_IS(SET|CLR) and modify consumers to use it
This implementation is faster and doesn't modify the cpuset, so it lets
us avoid some unnecessary copying as well.  No functional change
intended.

Reviewed by:	cem, kib, jhb
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D32029
2022-06-22 10:15:26 +02:00
Mark Johnston 37a3e59636 bitset(9): Introduce BIT_FOREACH_ISSET and BIT_FOREACH_ISCLR
These allow one to non-destructively iterate over the set or clear bits
in a bitset.  The motivation is that we have several code fragments
which iterate over a CPU set like this:

while ((cpu = CPU_FFS(&cpus)) != 0) {
	cpu--;
	CPU_CLR(cpu, &cpus);
	<do something>;
}

This is slow since CPU_FFS begins the search at the beginning of the
bitset each time.  On amd64 and arm64, CPU sets have size 256, so there
are four limbs in the bitset and we do a lot of unnecessary scanning.

A second problem is that this is destructive, so code which needs to
preserve the original set has to make a copy.  In particular, we have
quite a few functions which take a cpuset_t parameter by value, meaning
that each call has to copy the 32 byte cpuset_t.

The new macros address both problems.

Reviewed by:	cem, kib
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D32028
2022-06-22 10:15:26 +02:00
Patrick Kelsey 7c03cdf47e iflib: Improve mapping of TX/RX queues to CPUs
iflib now supports mapping each (TX,RX) queue pair to the same CPU
(default), to separate CPUs, or to a pair of physical and logical CPUs
that share the same L2 cache.  The mapping mechanism supports unequal
numbers of TX and RX queues, with the excess queues always being
mapped to consecutive physical CPUs.  When the platform cannot
distinguish between physical and logical CPUs, all are treated as
physical CPUs.  See the comment on get_cpuid_for_queue() for the
entire matrix.

The following device-specific tunables influence the mapping process:
dev.<device>.<unit>.iflib.core_offset       (existing)
dev.<device>.<unit>.iflib.separate_txrx     (existing)
dev.<device>.<unit>.iflib.use_logical_cores (new)

The following new, read-only sysctls provide visibility of the mapping
results:
dev.<device>.<unit>.iflib.{t,r}xq<n>.cpu

When an iflib driver allocates TX softirqs without providing reference
RX IRQs, iflib now binds those TX softirqs to CPUs using the above
mapping mechanism (that is, treats them as if they were TX IRQs).
Previously, such bindings were left up to the grouptaskqueue code and
thus fell outside of the iflib CPU mapping strategy.

Reviewed by:	kbowling
Tested by:	olivier, pkelsey
MFC after:	3 weeks
Differential Revision:	https://reviews.freebsd.org/D24094
2022-06-22 10:15:26 +02:00
Ryan Libby fb0a5865e4 bitset: implement BIT_TEST_CLR_ATOMIC & BIT_TEST_SET_ATOMIC
That is, provide wrappers around the atomic_testandclear and
atomic_testandset primitives.

Submitted by:	jeff
Reviewed by:	cem, kib, markj
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D22702
2022-06-22 10:15:26 +02:00