This HOWTO file contains notes for maintainers and contributors to Newlib. For information on using Newlib (building, installing), see README. (In particular, the "Regenerating Configuration Files" section in README is of interest to both users and contributors.) (This file could probably use some other "sections" in addition to the initially-provided sections. Please help by adding as appropriate.) DOCUMENTATION ============= All the documentation for Newlib comes as part of the machine-readable distribution. Functions are documented in the source files, although not every file contains documentation, as many functions share manual page information. For example, lround(), lroundf(), llround(), and llroundf() share a single man page, which is in the source for lround(). The documentation is written partially in a custom format and partially in Texinfo format. The custom part comprises makedoc.c and doc.str, both of which are in the doc directory. doc.str is a macro file that can be used to define things to be done by makedoc, using building blocks that are built into makedoc.c. The basic function of makedoc is two-fold. First, it recognizes comments in the proper format to pull out of source files. Second, it adds some Texinfo commands as well as translating certain sequences into the appropriate Texinfo commands. Refer to makedoc.c and doc.str for what these are. (makedoc.c is not particularly-well commented.) Another way to see how they work is to simply examine the source files with documentation comments. (A couple of examples that use some of the fancier options: libm/common/s_isnan.c ("o+" variable-"bullet" list), libc/stdio/sprintf.c ("O+" bullet list; "." for example code).) In addition to the makedoc.c stuff, Texinfo commands can be directly used. Texinfo is a documentation system that uses a single source file to produce both on-line information and a printed manual. You can use one of the Info formatting commands to create the on-line version of the documentation and TeX (or `texi2roff') to typeset the printed version. While Newlib contains a copy of the texinfo package (texinfo.tex), the manual for it is not included. The latest one may be found at https://www.gnu.org/software/texinfo/manual/texinfo/texinfo.html (which could be for a newer version of texinfo.tex than is included in Newlib). In addition to Texinfo commands, straight TeX commands can also be used, but these, however, should be carefully limited and be given alternates for when a non-printed manual is produced--such as when info pages are produced. For an example of this kind of usage, see libm/common/s_logb.c. If writing a new function that requires documentation, the required sections are FUNCTION, INDEX, SYNOPSIS, DESCRIPTION, RETURNS, and PORTABILITY. BUGS, NOTES, SEEALSO and WARNINGS should be added as appropriate. Source files which contain documentation are processed into ".def" files with the extracted information. These .def files are noted in the makefiles as CHEWOUT_FILES. These .def files need to be included into an appropriate .tex file for inclusion in the manuals (one each for libc and libm). Pay special attention under libc, as the manual is arranged by header file name, but not all header files are represented by directories (e.g. wcsftime.c is found under libc/time, but goes under wchar.h in the manual.) In summary, to add new documentation: 1. Add properly-formatted comments to source file (e.g. src.c); 2. add "chewout" file to CHEWOUT_FILES list in Makefile.am (e.g. src.def), re-generate Makefile.in; 3. @include that .def file in the appropriate .tex file, add texinfo menu entries that reference the @node(s) in that .def file; 4. make ChangeLog entry and generate patch. EL/IX (ELIX_LEVEL_n, ELIX_n_SOURCES) ==================================== Some of the Makefiles contain definitions of ELIX_LEVEL_1 ... ELIX_LEVEL_4, and corresponding definitions for ELIX_1_SOURCES ... ELIX_4_SOURCES. These are referring to the EL/IX standards created by Red Hat for the purpose of Linux-based open standards for embedded development. In brief, the intent is to define multiple levels for API functions that allows the user to size the library appropriately for their application--at least in terms of the predefined lists. For full details, refer to the EL/IX home page at https://sourceware.org/elix/. The likely best way to tell how to classify any new functions in terms of needing an ELIX level qualification is to ask Jeff Johnston. To see how it works and try classification on your own, refer to the API specification on the web site, https://sourceware.org/elix/api/current/api.pdf (Unfortunately, it is not complete with respect to either the C99 or POSIX standards, so particular C and POSIX functions may or may not be found.) The following definitions of the levels are from the (draft) standard. Level 1 RTOS compatible layer. Functions available in both Linux and a typical deeply embedded operating system (eCos, RTEMS, VxWorks, pSOS, VRTX32, etc.). Some functions may have reduced or modified semantics. Level 2 Linux single process only. Includes level 1 plus any functions from Linux that are not easily implemented on an RTOS. Also full implementations of reduced functions in Level 1. Level 3 Linux multiprocess for embedded applications. This is basically POSIX.1 with some of the functions that are obviously not for embedded applications (such as job control) removed. Level 4 Full POSIX or Linux compliance. Essentially these are functions that are present in a standard Linux kernel but are irrelevant to an embedded system. These functions do not form part of the EL/IX API. BUILD SYSTEM ============ Newlib utilizes GNU Autotools (GNU Autoconf & GNU Automake) for its build system. It can be broken down into two phases: configure & compile (make). NB: GNU Libtool is not used. Newlib only produces static archives, and GNU Automake is capable of handling that. As noted in the README file, you do not need to worry about which tools to run at any particular point. Simply invoke `autoreconf` and it will do it all. Tool versions are tightly controlled. GNU Autoconf 2.69 exactly must be used. Attempts to use any other version, newer or older, will be rejected. If the current system does not include that version, it can be downloaded from the GNU Autoconf website and installed into a local directory tree (like ~/.local). GNU Automake 1.15.1 is required as a minimum. Newer versions are allowed when running locally, but developers submitting patches must use 1.15.1 exactly. The configure phase naturally starts with the configure script. It merges multiple inputs to produce the final script. * aclocal.m4: Generated by aclocal; do not edit. Copies all the various m4 macros used by configure from the system install into the local copy. This allows regenerating configure without needing all the same system packages. * autom4te.cache/: Generated by aclocal to cache system search results. Safe to delete at any time, but never edit anything in this directory. * iconv.m4: Generated by libc/iconv/ces/mkdeps.pl; do not edit. Keeps the set of dynamic iconv synced between the source files and configure. * acinclude.m4: Historically, a way of sharing configure logic with subdirs. At some point, it should all be merged into configure.ac directly. * configure.ac: The main configure script for supporting all Newlib options. * configure.host: An index of every supported Newlib host and the settings to use when compiling for it. * **/acinclude.m4: Subdirectories may include custom configure tests without having to clutter the common configure.ac or acinclude.m4 file. When adding a new file, it must be manually included -- there is no mechanism to search and automatically include every such file in the tree. Look for m4_include lines in configure.ac or existing acinclude.m4 files as examples. Care must be taken to not run excessive tests for incompatible targets, or to declare generic names or symbols that can collide with other targets. Any such file should try to match $host (or similar) settings before running. Settings from configure.host (e.g. machine_dir or sys_dir) are guaranteed to be available. When creating names (Makefile variables, defines, etc...), try to include the respective library and machine or sys name. Do not confuse these with the historical top-level acinclude.m4 file! The configure script has to take special consideration to the fact that it usually runs with a toolchain that lacks a C library (since Newlib will provide it once it's compiled & installed). Newlib uses the uncommon AC_NO_EXECUTABLES macro to tell GNU Autoconf to not automatically require a toolchain that can link executables. As a result, Newlib itself cannot perform any link tests. It shouldn't really need to, but configure authors must keep this in mind and stick to the appropriate source-level test -- AC_PREPROC for testing behavior of the preprocessor and AC_COMPILE for testing compiler behavior. The newlib.h header is created from newlib.hin during the configure phase. The newlib.hin file is managed by autoheader, so do not edit it directly. Any symbol declared in configure via AC_DEFINE is automatically included. The file declares all the Newlib-specific configure settings and will be installed as part of the standard set of headers. Critically, all symbols are namespaced with a leading _ prefix to avoid collisions with user code, and any symbols not namespaced are automatically removed. This is necessary due to some system m4 macros automatically adding their own defines that we cannot disable. It also means that, when using AC_DEFINE, the _ prefix must be manually added. A single Makefile is created during the configure phase. This will build all of Newlib using non-recursive make (i.e., there are no subdirectories with their own Makefiles). The single Makefile.am file includes Makefile.inc files in its immediate subdirectories, and those further include Makefile.inc files in their subdirectories. This means all variables and targets exist in a single global flat namespace, so care must be taken to not use generic names like "SOURCES" in any Makefile.inc file. Instead, use %C%_ on all variables to get a unique prefix, and GNU Automake will expand it. In order to use one of the standard GNU Automake variables (e.g., CLEANFILES), first declare it at the top of the Makefile.am by assigning it (e.g., CLEANFILES =), and then always append to it in Makefile.inc (e.g. CLEANFILES += ...). Again, GNU Automake will take care of combining all of this logic together to produce a portable Makefile. Developers used to changing to a subdirectory and running make commands to iterate quickly on a smaller section of Newlib might be surprised that this does not work -- there is no Makefile in any subdirectory. Instead, make has to be run from the top-level, and the full path to the target is specified. So instead of running `cd subdir && make foo.o`, run `make subdir/foo.o`. This applies to any target, not just objects. Some common targets: make libm.a make libc.a Conditionals are allowed in Makefile.inc files, but using them to elide an entire subdir Makefile.inc should be avoided. Instead, GNU Automake conditionals should be used only within a single Makefile.inc file to control settings specific to that directory. See the documentation section below for more information. Per-directory compiler settings are allowed. Similar to the syntax used above, the _%C% suffix can be used. See the Makefile.am file for the current set of flags (e.g. CFLAGS, CPPFLAGS, etc...) that are supported. AM_CFLAGS_%C% = ... libm_a_CFLAGS_%C% = ... libc_a_CFLAGS_%C% = ... Per-file compiler settings are allowed. Similar to the syntax used above, the _%C% suffix can be used. See the Makefile.am file for the current set of flags (e.g. CFLAGS, CPPFLAGS, etc...) that are supported. AM_CFLAGS_%C%_moo.c = ... libm_a_CFLAGS_%C%_foo.c = ... libc_a_CFLAGS_%C%_bar.c = ... An important aspect of the build system is allowing machine-specific directories to override common implementations. For example, Newlib provides generic C implementations of string & memory functions in libc/string/ that perform adequately for any target, but many architectures include hand written assembly files for smaller & faster execution. Those files live under the corresponding libc/machine// directory. Both versions are compiled when building Newlib, but when generating the libc.a archive, the common object is added first, followed by the machine-specific one. This works because the object names follow a consistent convention: libm_a-.o for the math library and libc_a-.o for the C library. If the common code stores a function foo in foo.c, then it can be overridden by another foo.c or foo.S or foo.s file since they all produce foo.o. For this reason, all machine Makefile.inc files must be included last after all common Makefile.inc files. Note that the Makefiles do not use VPATH to search source files across multiple source directories. VPATH is used to match a single build directory to the corresponding source directory, but that is it. For more details on how machine directories override functions in common directories, see the previous section. Keep in mind that machine overrides only work at the file level, not at the symbol level. If a common file defines multiple symbols, then the machine override must define all the same symbols. This is why most common code stores only one function per source file. The documentation, both the manual and man pages and header references, should be consistent regardless of what configure options are used (including the target system). This makes it easy to get a complete documentation build at all times. If inclusion of chapters depended on specific configure options, it would be difficult to find the set of options necessary to produce the complete manual, if it were even possible in the first place (as some configure options are incompatible). Since documentation chapters & APIs are declared in the respective directories and not in a centralized location, it is important to not use any GNU Automake conditional to control whether Makefile.inc files are included. Documentation that covers optional APIs should note that it is not guaranteed to be included in every build of Newlib for every target. The level of detail is left up to the author's discretion. Newlib automatically builds itself for all multilib configurations that the active toolchain supports. This logic is provided by common code from the combined toolchain source tree, but bolts itself into the GNU Autoconf and Automake files. The critical parts to keep in mind: * ../config/multi.m4: AM_ENABLE_MULTILIB adds the --{en,dis}able-multilib options to configure for users, and takes care of calling ../config-ml.in for the Makefile in the current directory. While not applicable to Newlib anymore (since Newlib only produces a single Makefile now), this does not automatically run for any subdir Makefiles. Instead, it must be run manually in AC_CONFIG_FILES's commands argument. * ../config-ml.in: Rewrites the specified Makefile to include the various multilib variables and the multi-do command. Notably, this only makes the multi-do rule available, it does not cause it to be used anywhere. * ../multilib.am: Manually included in Newlib's Makefile.am. This adds calls to multi-do for the common important rules: all (i.e. compiling), install, clean, etc... * FLAGS_TO_PASS: Newlib's Makefile.am declares all the variables to pass to each multilib configuration via the multi-do rule. While the multi-do rule includes many common flags automatically, Newlib passes along a few extra that are unique to it. The testsuite is not integrated at all currently. It is not really well tested in general. The README file has more information. Hopefully someone someday will step up to integrate it into the existing framework.