Comparing TR1 Implementations
A full implementation of TR1 roughly doubles the size of the current
Standard C++ library. (See our description of TR1.) This
very large chunk can be divided into three large components:
- C++ templates, based largely on Boost technology
(roughly 25 per cent)
- Special math functions (roughly 50 per cent)
- C99 library additions (roughly 25 per cent)
Each component presents a different set of implementation problems, and
all are nontrivial to implement.
Components 2 and 3 — roughly 3/4 of the total code required to
implement TR1 — are available only from Dinkumware,
as part of the Dinkum Compleat Libraries. But there are currently two other
sources for (most of) component 1, both of which are available for free:
- Boost provides many of the templates that served as the basis for
TR1 proposals. These templates can be downloaded and built for use on a
variety of compilers. You have to download the utility bjam,
along with the source, to stage the libraries for a given compiler.
Then you have to determine, for each TR1 header, the Boost headers
to include and the namespaces where each of the pieces resides.
With a few hours' work, you can contrive a thin layer of properly
named new headers and rewritten standard headers — with aliases to rename
the namespaces — to deal with most of the superficial differences between
the Boost components and those required by TR1. (But see below.)
- Gcc V4 provides many of the same templates, some based on Boost
code. These, of course, work only with V4 and later versions of Gcc,
not with any other compilers. To include the new headers, you have to
include
<tr1/array>, for example, instead of
<array>, but they otherwise work out of the box.
The only comprehensive test suite for TR1 currently available comes
from our Quick Proofer (which you get as part of our source license).
Component 1 is subject to about 18,000 Quick Proofer tests,
distributed unevenly over the seven new headers —
array, random, regex, tuple, type_traits, unordered_map,
unordered_set — and the TR1 additions to three existing headers —
functional, memory, utility.
Some of these additions are way more complex than others, and
the number of tests for each addition does not always reflect how
hard it is to implement. The headers functional and type_traits, for
example, have many test cases to cover various exotic combinations
even though they together comprise less than 1/4 of the total code
added for TR1 component 1.
We feel it best to show the percentage of test
cases passed for each header, because that tells you how well
each header conforms to TR1 today and Standard C++ tomorrow,
and hence how usable it might be in its own right.
But presenting the headers on an equal footing — as equal height
bars in a common graph — obscures the fact that some headers are
roughly 50 times harder to implement than others. To convey the
overall degree of conformance, we include a weighted sum of the
results of the individual headers. The weights we assigned, based
on our own implementation experience, are:
2.0% array
12.5% functional
10.0% memory
25.0% random
25.0% regex
5.0% tuple
10.0% type_traits
5.0% unordered_map
5.0% unordered_set
0.5% utility
Testing Boost
We've tested V1_33_1 atop Mingw V3.4.5 here.
For Boost, we altered our tests to include the appropriate
Boost headers and to look in the appropriate Boost namespaces. This
is equivalent to the process we described above, of adding a thin layer
of headers and aliases to rename namespaces. We also repartitioned our
tests so that if one subtest failed to compile it did not take out other
good tests packaged with it. That process got almost all the tests for
type_traits to pass, and about 1/6 of the tests for random, but nothing
else. So, with this criterion for "out of the box", Boost has a
weighted pass rate of around 14 per cent for component 1.
But we then went further. Some tests failed because the parameters
in a function signature changed. Others failed because
Boost provided for, say, up to nine arguments instead of up to ten,
as required by TR1. Others failed because of similar specification
changes. We altered our tests to avoid penalizing the Boost code for
such shortcomings that a reasonable programmer might possibly tolerate
(even if that means eventually making changes to match the upcoming revision
of the C++ Standard).
That process raised the weighted pass rate to 59 per cent
for component 1, and
yielded the following actual pass rates on a header-by-header basis:
% PASS/TOTAL HEADER
59 35/59 array
39 1467/3767 functional
96 260/271 memory
68 316/466 random
48 1137/2349 regex
96 305/318 tuple
98 10154/10330 type_traits
0 0/143 unordered_map
0 0/137 unordered_set
14 1/7 utility
Here are the same results in bar-graph form:
Note that the bars for the existing headers functional, memory, and
utility just describe the contributions from TR1 additions — they are
not diluted by tests of existing Standard C++ features.
Testing Gcc
We've tested V4.1.1 on PC Linux here.
For Gcc, we altered our tests to look in the tr1 include subdirectory
for the new headers. We also repeated the process of isolating tests that
failed to compile. The result is a weighted pass rate
of 47 per cent for component 1,
and the following actual pass rates on a header-by-header basis:
% PASS/TOTAL HEADER
76 45/59 array
96 3617/3767 functional
100 270/271 memory
0 0/466 random
0 0/2349 regex
100 318/318 tuple
100 10321/10330 type_traits
76 109/143 unordered_map
74 102/137 unordered_set
100 7/7 utility
Here are the same results in bar-graph form:
Once again, the bars for the existing headers functional, memory, and
utility just describe the contributions from TR1 additions.
Testing Dinkumware
We've tested V5.01 atop Mingw V3.4.5 here.
For the record, here is the bar graph for the Dinkum Compleat Libraries:
We get a perfect score (no surprise) with Gcc V3 or later, Microsoft
VC++ V7.1 or later, or any compiler based on the Edison Design Group
front end.
We also get a perfect score on components 2 and 3 of TR1,
which, we remind you, comprise roughly 75 per cent of the total code
and are not available from any other source. With that in mind,
here are the weighted pass rates for all of TR1:
15% Boost
12% Gcc
100% Dinkumware