Introduction
Chroma is a simulation package for optical photon raytracing that aims to be physically accurate and very fast.
In fact, an instance of Chroma running on a single GPU can propagate photons up to 200x faster than similar methods on a single CPU.
It’s built on PyCUDA and contains CUDA kernels for propagating photons in triangular mesh geometries.
This allows photons to be propagated in parallel on GPUs, which are increasingly available at high performance compute sites, with their applications to machine learning.
The overall idea here is that Chroma can be used to simulate the optical response of neutrino detectors, which ultimately detect flashes of light produced by the high energy particles resulting from neutrino interactions within a detector.
Over the last year I have been adding to and optimizing the Chroma package (originally written by Stan Seibert and Tony LaTorre) to include additional optical effects necesary for simulating R&D projects being developed at UPenn using dichroic filters.
I’ve also been working on making Chroma more accessible, and have packaged it and all of its dependencies in a Docker container that can be easily deployed at high performance compute sites, or local compute clusters with GPUs.
See the recent SNOWMASS 2021 LOI on Chroma for details, or the description below for a high-level overview of Chroma.
Geometries
Each triangle in the mesh can specify optical properties, which allows one to simulate the complex optical environment present in neutrino detectors.
Triangular meshes are ubiquitous in computer graphics and computer assisted design, which allows Chroma to leverage existing modeling tools for building its geometries.
In fact, Chroma can directly import STL files and define optical properties for the triangles specified within the file.
Further, raytracing with triangular meshes is a well explored method in computer graphics, and Chroma can utilize highly optimized algorithms for propagating photons in these geometries.
Photon generation
Chroma can either a set of specific photons for debugging (i.e. photons generated from a point, or in a beam) or can use the simulation package Geant4 to generate photons from fundamental physics interactions, such as high energy electrons, or products of radioactive decay. In the latter mode, Chroma can be used as a complete detector simulation as long as a sufficiently accurate Geant4 geometry is also produced. There is ongoing work for developing a bridge using ZeroMQ and/or shared memory which can pass photons from an arbitrary source to Chroma for simulation, and return the results. A photon source could be a different physics simulation package such as RAT-PAC, allowing Chroma to take the place of the traditional CPU photon propagation algorithm, with minimal changes to the rest of the simulation and analysis framework.
Photon propagation
All physical processes relevant to propagating photons on the 0-meter to 100-meter scale are implemented in Chroma CUDA kernels:
- Wavelength-dependent index of refraction
- Absorption in the bulk material
- Reemission from a wavelength shifter
- Rayleigh scattering
- Diffuse reflection at surfaces
- Specular reflection at surfaces
- Arbitrary angle, wavelength dependent transmission and reflection (dichroic)
- Standard Fresnel reflection and refraction
This happens entirely on the GPU once photons are transferred to GPU memory, allowing Chroma to propagate photons up to 200 times faster than CPU implementations found in packages like Geant4.
Photon detection
Chroma allows triangles to specify a detection probability as a function of wavelength.
This can be used to model very sensitive photon detectors called photomultiplier tubes (PMTs), which are used in neutrino experiments and many other branches of science.
These PMTs work by having a thin metal film plated inside a glass enclosure with a very small work function, meaning it is easy for visible photons to kick electrons out of the metal.
These electrons are amplified up to detectable voltage pulses, which can be used to record precisely when a photon struck the device.
Chroma can return only the photons detected by light detectors on the GPU, significantly reducing the processing load on the CPU and the memory bandwidth required for obtaining results.