Red Hat at the ISO C++ Standards Meetings (November 2016, Issaquah, and February 2017, Kona): Library

I attended the recent Issaquah and Kona ISO C++ standards meetings, representing Red Hat and the GCC project, and helping to complete the C++17 standard. As usual, I spent the majority of my time in the Library Working Group (LWG) sessions, but also took part in a subgroup focusing on the Filesystem library, more on that below.

Following the National Body (NB) ballot on the C++17 draft, the LWG had a huge number of comments to process. Some issues were resolved in Issaquah, but many were only triaged in Issaquah and put aside to be dealt with at the Kona meeting. Some of the more interesting changes that came out of those meetings were:

• API changes to the new std::any, std::optional and std::variant types, making them more consistent with each other.

Changes to std::tuple_size<const T> and node handles to work better with the new structured bindings feature in the core language.

• A new multiline syntax option for std:: regex (equivalent to the MultiLine property in ECMAScript).
• Making more functions in std::char_traits and std::string_view usable in constant expressions.
• A new std::is_aggregate type trait.
• A new std::byte type, which is a non-integral type with the same representation as unsigned char, but not supporting arithmetic operations or implicit conversions to integers.
• Forbidding calling std::addressof with rvalue arguments.
• Allowing nullptr to be written to ostreams.

Some features that were new in C++11 were deprecated for C++17:

• std::result_of, which is superseded by the less error-prone std::invoke_result. The std::is_callable trait (new for C++17) was renamed to std::is_invocable, and the functionality of also checking for convertibility to another type was split out into std::is_invocable_r (with corresponding is_nothrow_invocable and is_nothrow_invocable_r traits for also checking for noexcept guarantees).

The reason for deprecating std::result_of is that its syntax was always a trap for non-experts, and it gave errors in some cases, or the wrong answer in some odd corner cases. Instead of using result_of<CallableType(ArgTypes...)> you can now use invoke_result<CallableType, ArgTypes...>, which works correctly in all cases. See P0604R0 for the full rationale for this change.

• The code conversion facets std::codecvt_utf8, std::codecvt_utf16, and std::codecvt_utf8_utf16 have been deprecated, due to several problems in their interfaces and unclear specification of their expected behavior. The utilities std::wstring_convert and std::wbuffer_convert have also been deprecated because their only real use cases are with those codecvt facets. This deprecates the only facilities in the C++ Standard Library for converting between different Unicode encodings, but the intention is to add new features ASAP that are easier to use, support all encodings, and have better APIs. See P0618R0 for more rationale.

Two new C++17 features were taken back out of the draft based on NB comments:

• The std::default_order trait (added by P0181R1) was backed out, and the changes to make std::lock_guard a variadic template were reverted and a new std::scoped_lock added instead (see P0156R2). In both cases, this was done because the features introduced ABI incompatibilities, and the benefits were not deemed important enough to warrant forcing an ABI change on users if they use C++17.

There were several changes to the new parallel versions of STL algorithms that were added to C++17, originating in the Parallelism TS. Compared to the TS, the post-Kona C++17 draft:

• Removes the parallel form of inner_product, adding new overloads of transform_reduce, and reordering the parameters of transform_inclusive_scan and transform_exclusive_scan.
• Relaxes requirements to allow additional copies to be made by parallel algorithms, and relaxes complexity requirements to allow more optimal implementations.
• Changes all parallel algorithms accepting Input Iterators to require at least Forward Iterators, which give the multi-pass guarantee and so allow operating on different subsequences of the range in parallel.

Finally, there were a large number of changes to the new Filesystem library, which originated in the Filesystem TS. A few people from LWG split off from the main group to focus on the Filesystem issues, and that's where I spent most of my time. The subgroup included people who had worked on the implementations from GCC, Boost, Windows, and IBM, as well as the submitters of many of the more gnarly issues, so most of the relevant people on the committee were involved in the subgroup.

During the week, the subgroup presented some ideas to the Library Evolution Working Group (LEWG) to get their input and approval for the design changes we were proposing. At the end of the Issaquah meeting, we made some recommendations to LWG for how to resolve some of the issues. Between the meetings, work was done on the more complicated problems, and a paper was written with our suggested resolutions, to be reviewed by LWG during the Kona meeting and approved by the full committee.

Many of the changes made to the Filesystem library were not very user-visible and only altered the specification, or cleaned up some error handling or corner cases. One important set of specification changes removed some implicit assumptions so that as well as conventional directories on POSIX and Windows filesystem, the specification can also apply to "data sets" on mainframes.

One of the more user-visible changes was to how filenames are decomposed into a stem and an extension. In the original design a filename like ".bashrc" would be decomposed into an empty stem and an extension of ".bashrc", which may be surprising to people used to POSIX-like environments. That was changed so that ".bashrc" is the stem, with no extension (but e.g. ".bashrc.bak" would have an extension of ".bak", as you'd expect).

The most significant change was a redesign of how path decomposition determines the filename part of a path. Originally, the expression path("/foo/").filename() would return "." (which is the same result as the path("/foo/.").filename() returns). This was done to allow distinguishing paths like "/foo/" from "/foo" because the trailing slash meant it must refer to a directory, just as "/foo/." does. However, this led to some surprising properties, such as path("/foo/").has_filename() being true, because it has a filename of ".", and path("/").has_filename() being true. In fact, p.has_filename() is true for any non-empty path!

This definition of the "filename" part of a path meant that the remove_filename() member function gave some surprising results. Because every non-empty path has a filename, calling remove_filename() was more like a "pop back" operation, which would keep removing one component of the path until there were none left.

The changes accepted in Kona mean that the "filename" part of a path is the portion of a path from the last slash to the end of the path. So:

path("/foo/").filename() == "" (i.e. the empty path)

path("/foo/.").filename() == "."

path("/foo").filename() == "foo"

With these new semantics, the meaning of remove_filename() makes more sense, as it simply removes the filename portion. So path("/foo/").remove_filename() has no effect, but path("/foo").remove_filename() still yields "/" as before.

Adjusting the grammar for describing paths and updating the specifications of the member functions to meet the new semantics (without introducing any new inconsistencies or surprises) was quite a complex task. It took careful redesign and a lot of checking to be sure the new wording actually said the right thing, and that we all agreed that "the right thing" was the same thing we'd decided on a few months earlier at the Issaquah meeting!

Most of the changes to the Filesystem design can be found in P0492R2, which was approved in Kona. As is often the case with standardization work, it might take a few years to know if we made the right choices. Hopefully diverging from the tried & tested Boost.Filesystem semantics will make the C++17 filesystem library easier to learn and use.

Download this Kubernetes cheat sheet for automating deployment, scaling and operations of application containers across clusters of hosts, providing container-centric infrastructure.