I first conducted these tests in October 2014 in response to a 3DPro thread discussing the likely effects
of using ACES primaries (specifically AP0) as a rendering color space. Since then the ACEScg specification
was released (AP1), which is roughly equivalent to Rec.2020, so if you like you can read "Rec.2020" as
"ACEScg" below.
This page compares images rendered in Arnold using spectral rendering and different sets of colourspace primaries: Rec.709,
Rec.2020, ACES and DCIP3. The SPD data for the GretagMacbeth Color Checker are the measurements of Noburu
Ohta, taken from Mansencal, Mauderer and Parsons (2014) colourscience.org.
Each box is lit by a small area light with a D65 SPD. All spectral calculations are performed over the
interval [380nm, 780nm] at 5nm increments. For the RGB renders, the colour swatches are preconverted into
the appropriate primaries before rendering, and the lightsource is pure white (1,1,1). To make comparison
easier, all RGB colour spaces are assumed to have a D65 whitepoint. All materials are pure lambertian and
the scene was rendered with 10 bounces.
You can compare the images with the slider thingy I nicked from Wojciech
Jarosz's website or you can download all the jpegs and exrs from google drive here:
color_checker_comparisons_v0002.zip
TL;DR: There's no clear winner, but Rec. 2020 performs the best from the options so far.
Contents
1. Rec. 709 Display
First, we compare images rendered in spectral, Rec.709, Rec.2020 and ACES and converted for a Rec.709 display using a display
gamma of 2.2. In other words, it asks the question "what are the best set of primaries for rendering when
targeting a computer monitor?"
Spectral
RGB Rec. 709
RGB Rec. 2020
RGB ACES
Spectral
RGB Rec. 709
RGB Rec. 2020
RGB ACES
Spectral
RGB Rec. 709
RGB Rec. 2020
RGB ACES
Spectral
RGB Rec. 709
RGB Rec. 2020
RGB ACES
Spectral
RGB Rec. 709
RGB Rec. 2020
RGB ACES
Spectral
RGB Rec. 709
RGB Rec. 2020
RGB ACES
Spectral
RGB Rec. 709
RGB Rec. 2020
RGB ACES
Spectral
RGB Rec. 709
RGB Rec. 2020
RGB ACES
Spectral
RGB Rec. 709
RGB Rec. 2020
RGB ACES
Spectral
RGB Rec. 709
RGB Rec. 2020
RGB ACES
Spectral
RGB Rec. 709
RGB Rec. 2020
RGB ACES
Spectral
RGB Rec. 709
RGB Rec. 2020
RGB ACES
Spectral
RGB Rec. 709
RGB Rec. 2020
RGB ACES
Spectral
RGB Rec. 709
RGB Rec. 2020
RGB ACES
Spectral
RGB Rec. 709
RGB Rec. 2020
RGB ACES
Rec. 709 (gamma 2.2) display comparison
Click the links below to open the fullres (3k) jpegs.
RGB Rec. 709  RMSE 0.0138161
RGB Rec. 2020  RMSE 0.00704586
RGB ACES  RMSE 0.0153788
RGB Rec. 709  RMSE 0.0138161
RGB Rec. 2020  RMSE 0.00704586
RGB ACES  RMSE 0.0153788
RGB Rec. 709  RMSE 0.0138161
RGB Rec. 2020  RMSE 0.00704586
RGB ACES  RMSE 0.0153788
RGB Rec. 709  RMSE 0.0138161
RGB Rec. 2020  RMSE 0.00704586
RGB ACES  RMSE 0.0153788
RGB Rec. 709  RMSE 0.0138161
RGB Rec. 2020  RMSE 0.00704586
RGB ACES  RMSE 0.0153788
RGB Rec. 709  RMSE 0.0138161
RGB Rec. 2020  RMSE 0.00704586
RGB ACES  RMSE 0.0153788
RGB Rec. 709  RMSE 0.0138161
RGB Rec. 2020  RMSE 0.00704586
RGB ACES  RMSE 0.0153788
RGB Rec. 709  RMSE 0.0138161
RGB Rec. 2020  RMSE 0.00704586
RGB ACES  RMSE 0.0153788
RGB Rec. 709  RMSE 0.0138161
RGB Rec. 2020  RMSE 0.00704586
RGB ACES  RMSE 0.0153788
RGB Rec. 709  RMSE 0.0138161
RGB Rec. 2020  RMSE 0.00704586
RGB ACES  RMSE 0.0153788
RGB Rec. 709  RMSE 0.0138161
RGB Rec. 2020  RMSE 0.00704586
RGB ACES  RMSE 0.0153788
RGB Rec. 709  RMSE 0.0138161
RGB Rec. 2020  RMSE 0.00704586
RGB ACES  RMSE 0.0153788
RGB Rec. 709  RMSE 0.0138161
RGB Rec. 2020  RMSE 0.00704586
RGB ACES  RMSE 0.0153788
RGB Rec. 709  RMSE 0.0138161
RGB Rec. 2020  RMSE 0.00704586
RGB ACES  RMSE 0.0153788
RGB Rec. 709  RMSE 0.0138161
RGB Rec. 2020  RMSE 0.00704586
RGB ACES  RMSE 0.0153788
Rec. 709 2x diff comparison
Primaries

RMS error

Rec. 709

0.0138161

Rec. 2020

0.00704586

ACES

0.0153788

2. DCIP3 Display
Next, we target a DCIP3 display, rendering spectrally and with Rec.709, Rec.2020, ACES and DCIP3 primaries (but with a
D65 white point). In this case we are asking, "what is the best set of primaries for rendering when targeting
a digital cinema projector?"
Spectral
RGB Rec. 709
RGB Rec. 2020
RGB ACES
Spectral
RGB Rec. 709
RGB Rec. 2020
RGB ACES
Spectral
RGB Rec. 709
RGB Rec. 2020
RGB ACES
Spectral
RGB Rec. 709
RGB Rec. 2020
RGB ACES
Spectral
RGB Rec. 709
RGB Rec. 2020
RGB ACES
Spectral
RGB Rec. 709
RGB Rec. 2020
RGB ACES
Spectral
RGB Rec. 709
RGB Rec. 2020
RGB ACES
Spectral
RGB Rec. 709
RGB Rec. 2020
RGB ACES
Spectral
RGB Rec. 709
RGB Rec. 2020
RGB ACES
Spectral
RGB Rec. 709
RGB Rec. 2020
RGB ACES
Spectral
RGB Rec. 709
RGB Rec. 2020
RGB ACES
Spectral
RGB Rec. 709
RGB Rec. 2020
RGB ACES
Spectral
RGB Rec. 709
RGB Rec. 2020
RGB ACES
Spectral
RGB Rec. 709
RGB Rec. 2020
RGB ACES
Spectral
RGB Rec. 709
RGB Rec. 2020
RGB ACES
DCIP3 (gamma 2.2) display comparison
Click the links below to open the fullres (3k) jpegs.
RGB DCIP3  RMSE 0.00740128
RGB Rec. 709  RMSE 0.011618
RGB Rec. 2020  RMSE 0.00581693
RGB ACES  RMSE 0.0130438
RGB DCIP3  RMSE 0.00740128
RGB Rec. 709  RMSE 0.011618
RGB Rec. 2020  RMSE 0.00581693
RGB ACES  RMSE 0.0130438
RGB DCIP3  RMSE 0.00740128
RGB Rec. 709  RMSE 0.011618
RGB Rec. 2020  RMSE 0.00581693
RGB ACES  RMSE 0.0130438
RGB DCIP3  RMSE 0.00740128
RGB Rec. 709  RMSE 0.011618
RGB Rec. 2020  RMSE 0.00581693
RGB ACES  RMSE 0.0130438
RGB DCIP3  RMSE 0.00740128
RGB Rec. 709  RMSE 0.011618
RGB Rec. 2020  RMSE 0.00581693
RGB ACES  RMSE 0.0130438
RGB DCIP3  RMSE 0.00740128
RGB Rec. 709  RMSE 0.011618
RGB Rec. 2020  RMSE 0.00581693
RGB ACES  RMSE 0.0130438
RGB DCIP3  RMSE 0.00740128
RGB Rec. 709  RMSE 0.011618
RGB Rec. 2020  RMSE 0.00581693
RGB ACES  RMSE 0.0130438
RGB DCIP3  RMSE 0.00740128
RGB Rec. 709  RMSE 0.011618
RGB Rec. 2020  RMSE 0.00581693
RGB ACES  RMSE 0.0130438
RGB DCIP3  RMSE 0.00740128
RGB Rec. 709  RMSE 0.011618
RGB Rec. 2020  RMSE 0.00581693
RGB ACES  RMSE 0.0130438
RGB DCIP3  RMSE 0.00740128
RGB Rec. 709  RMSE 0.011618
RGB Rec. 2020  RMSE 0.00581693
RGB ACES  RMSE 0.0130438
RGB DCIP3  RMSE 0.00740128
RGB Rec. 709  RMSE 0.011618
RGB Rec. 2020  RMSE 0.00581693
RGB ACES  RMSE 0.0130438
RGB DCIP3  RMSE 0.00740128
RGB Rec. 709  RMSE 0.011618
RGB Rec. 2020  RMSE 0.00581693
RGB ACES  RMSE 0.0130438
RGB DCIP3  RMSE 0.00740128
RGB Rec. 709  RMSE 0.011618
RGB Rec. 2020  RMSE 0.00581693
RGB ACES  RMSE 0.0130438
RGB DCIP3  RMSE 0.00740128
RGB Rec. 709  RMSE 0.011618
RGB Rec. 2020  RMSE 0.00581693
RGB ACES  RMSE 0.0130438
RGB DCIP3  RMSE 0.00740128
RGB Rec. 709  RMSE 0.011618
RGB Rec. 2020  RMSE 0.00581693
RGB ACES  RMSE 0.0130438
DCIP3 (gamma 2.2) 2x diff comparison
Primaries

RMS error

DCIP3

0.00740128

Rec. 709

0.011618

Rec. 2020

0.00581693

ACES

0.0130438

3. Colour Volume Anaysis
Finally, we examine the range of colours generated relative to different colour volumes. Since we're targeting digital cinema,
we'll just consider the P3 gamut and Pointer's gamut (please see colourscience.org
for more volumes).
The images here are a small subset of the full analysis that Thomas Mansencal did on the renders. For the full results, including
chroma diagrams, please see his
notebook
.
First, DCIP3. The P3 volume is large enough that even rendering with ACES primaries doesn't generate any outofgamut colours,
so we're safe!
Rec.709
Rec.2020
ACES
DCIP3
Rec.709
Rec.2020
ACES
DCIP3
Rec.709
Rec.2020
ACES
DCIP3
Rec.709
Rec.2020
ACES
DCIP3
Rec.709
Rec.2020
ACES
DCIP3
Rec.709
Rec.2020
ACES
DCIP3
Rec.709
Rec.2020
ACES
DCIP3
Rec.709
Rec.2020
ACES
DCIP3
Rec.709
Rec.2020
ACES
DCIP3
Rec.709
Rec.2020
ACES
DCIP3
Rec.709
Rec.2020
ACES
DCIP3
Rec.709
Rec.2020
ACES
DCIP3
Rec.709
Rec.2020
ACES
DCIP3
Rec.709
Rec.2020
ACES
DCIP3
Rec.709
Rec.2020
ACES
DCIP3
Next, Pointer's gamut, a real object colour gamut, representing reflectance factors of approximately 4000 different objects.
All the primaries generate nonphysical colours, with ACES (unsurprisingly) being the worst, but not by
much.
Rec.709
Rec.2020
ACES
DCIP3
Rec.709
Rec.2020
ACES
DCIP3
Rec.709
Rec.2020
ACES
DCIP3
Rec.709
Rec.2020
ACES
DCIP3
Rec.709
Rec.2020
ACES
DCIP3
Rec.709
Rec.2020
ACES
DCIP3
Rec.709
Rec.2020
ACES
DCIP3
Rec.709
Rec.2020
ACES
DCIP3
Rec.709
Rec.2020
ACES
DCIP3
Rec.709
Rec.2020
ACES
DCIP3
Rec.709
Rec.2020
ACES
DCIP3
Rec.709
Rec.2020
ACES
DCIP3
Rec.709
Rec.2020
ACES
DCIP3
Rec.709
Rec.2020
ACES
DCIP3
Rec.709
Rec.2020
ACES
DCIP3
4. Conclusions
The main takeaway is that, as expected, rendering in any set of 3 primaries is inadequate compared to spectral rendering.
Subjectively speaking, the differences aren't huge (at least in this limited test) and all the spaces suffer
from some noticeable colour shifts relative to the reference. Rec. 709 seems to suffer the worst from this,
generating the most saturated colours. In terms of what makes a good colour space for rendering, Rec. 2020
gives both the lowest RMS error and the smallest amount of colours that lie outside Pointer's gamut, the
latter being theoretically important for generating physically plausible renders.
More tests need to be done. In particular the next round of tests will consider glossy objects. It would be good to try Greg
Ward's SharpRGB colour space and see how that compares. The tests should be fixed to use native white points
and proper chromatic adaptation, to see if that affects the results. It would also be good to try some
different scenes with different spectral data  a physical sun/sky system perhaps.