Faster JPEGli Decode with Lower Attributed Energy on M4 Max

JPEG reconstruction is parallel enough to encourage GPU use. I wanted to find out whether Apple Metal could make JPEGli faster for the one image at a time use case typical of browser image loading --- not for a server benchmark, but for the image somebody is waiting to see. JPEGli's encoder can make smaller JPEG files at the same subjective quality [1]. Its CPU decoder is slower than the common alternatives. This project moves JPEGli reconstruction, and selected baseline entropy work, to Metal while keeping its output byte-identical.

Thrice in my career, for three different employers, I've found myself behind the keyboard crafting an image pipeline. In each case the target group of viewers was large enough that I not only had to consider the obvious effects—image size and download time—but second-order follow-on effects such as encoder CPU consumption, decode time, and battery consumption. Solutions varied depending on the employer's specific image characteristics, performance and image quality tradeoffs, and budgets, but the theme was consistent: rather than tackling the JPEG encoding pipeline as a whole, a lot could be obtained by classifying images by superficial, easy-to-compute characteristics and choosing encoding parameters from there.

In each case, one of the secondary follow-on characteristics I modeled was the compute time and estimated power consumption of the client device. This became especially important as networks grew in capacity, screens grew in resolution and fidelity, computers shrank in size, and the world became increasingly decoupled from AC power, with growing expectations for device battery life. No longer was the smallest encoding for a specific quality level automatically the fastest to display or the one that gave the longest-running user experience.

This weekend I decided to revisit this from the other side: could a client preserve JPEGli's output while spending less time and energy decoding it? The GPU should not invent different pixels; it should make the intended pixels cheaper to display.

I began by measuring the sum of per-image median baseline and progressive decode times for 30 images from the CLIC 2025 test set [2] for single-threaded CPU JPEGli, TurboJPEG, and Apple ImageIO.

I found TurboJPEG was about twice as fast as CPU JPEGli on the baseline sweep.

The difference comes from reconstruction choice. TurboJPEG's 8-bit path uses optimized integer SIMD for inverse DCT, chroma upsampling, and color conversion [10]. JPEGli instead dequantizes each nonzero DCT coefficient to the expected value of its quantization interval using a Laplacian model, then carries dequantization, inverse DCT, upsampling, and color conversion in floating point until the final output conversion [11]. This postpones integer rounding until the end, supports 16-bit and floating-point output, and preserves the precision of JPEGli's expectation-based reconstruction. It does require more work per output pixel on the CPU.

My question is whether an M4 GPU could close that gap for one image at a time while maintaining JPEGli's floating point and Laplacian semantics.

The implementation

In my implementation, the CPU always handles JPEG markers and format state. Progressive scans, restart intervals, and unsupported inputs keep the existing CPU entropy path. For an eligible single-scan baseline image, the CPU finds the scan, removes byte stuffing, and prepares the Huffman tables. Metal self-synchronizes fixed-size chunks, decodes the quantized coefficients, repairs the DC dependencies with prefix passes, and continues through dequantization, 8×8 IDCT, chroma upsampling, and conversion to RGBA.

The backend uses direct Objective-C++ and Metal. It initializes lazily, reuses scratch memory, and automatically releases retained GPU resources. The existing scanline API still works. A separate Apple endpoint returns a Metal texture without a CPU readback.

Single-image latency

My Direct Metal implementation was fastest from quality level 25 through Q90 for baseline files; TurboJPEG led at Q95 and Q100. For progressive files, TurboJPEG stayed fastest, while Direct Metal beat CPU JPEGli and ImageIO at every quality.

The Direct Metal timing ends with the decoded RGBA8 image resident in a Metal texture; it does not include CPU readback.

Progressive gains are smaller because entropy and scan processing currently remain on the CPU. A cold direct-texture baseline decode still beat CPU JPEGli: 7.928 ms versus 9.639 ms. Metal initialization took 0.28–0.40 ms.

These are serial, single-image results. The benchmark did not overlap images or use worker threads.

That quality sweep mixes image sizes. A separate size sweep exposes the crossover more directly.

Where Metal cuts latency and attributed energy

The energy campaign covered ten images and six JPEG profiles: baseline and progressive scans, qualities 50, 90, and 95, and 4:2:0, 4:2:2, and 4:4:4 sampling. The chart plots process-attributed energy per decoded megapixel across the full image-size range.

For images at or above 1 MP, current direct Metal beat CPU JPEGli on both latency and attributed energy in all 24 of 24 measured cases. Baseline latency fell 63.9% and attributed energy fell 82.4%. Progressive latency fell 26.0% and attributed energy fell 33.4%.

Against TurboJPEG, the win region was baseline images at or above 1 MP. Current direct Metal was 12.5% faster and used 53.2% less attributed energy, winning both measures in 13 of 16 cases. Combined CPU+GPU rail energy was 3.9% lower in the same region. Large progressive files remained 3.6% slower than TurboJPEG and used 2.3% more attributed energy. Below 1 MP, TurboJPEG remained the better choice.

Direct output is part of that result. Adding CPU readback made Metal 5.5% slower than TurboJPEG in the large-baseline region. The energy values in the chart come from macOS process accounting; they are not battery or wall-plug measurements.

Crossover and runtime policy

The JPEGli CPU-versus-Metal crossover sweep used 13 sizes from 4,096 to 12,582,912 pixels.

There are two top-level runtime policies, with a narrower entropy selector inside the exact-JPEGli path:

Need exact JPEGli output:
    below 480k pixels       -> CPU JPEGli
    480k pixels and above   -> Metal JPEGli

Within an eligible baseline Metal decode:
    first image or below 1.5 MP        -> CPU entropy
    fine quantization + useful density -> GPU entropy
    otherwise                           -> CPU entropy

Output may differ and the next consumer is on the GPU:
    baseline, at least 1 MP -> direct Metal JPEGli
    everything else         -> TurboJPEG

The first rule chooses between two byte-identical JPEGli reconstruction paths. The entropy selector is deliberately narrower: it requires a fine luma quantization table and 1.75–5.5 coded bits per pixel; direct 4:2:0 uses a 3.0-bit lower bound. Progressive, restart, and multiscan files remain on CPU entropy. The final rule is a broader application policy fitted to this M4 Max corpus. Applied to the 60 measured energy cases, it cut attributed energy by 18.3% and latency by 3.5% compared with always using TurboJPEG. It should be retested on other Apple chips and image sets.

Quality is not the decoder argument

Bruse et al. [1] found that JPEGli needed about 1.5 bits per pixel to match the estimated perceptual quality of a 2.1-bit-per-pixel libjpeg-turbo image, a 28% bitrate reduction. That result supports JPEGli's encoder.

I also compared CPU JPEGli, TurboJPEG, and ImageIO decodes of JPEGli files using SSIMULACRA2 [3], Butteraugli [4], ColorVideoVDP [5], and DISTS [6]. The sweep covered 30 images, qualities 25–100, and 4:4:4, 4:2:2, and 4:2:0. It did not find a consistent decoder winner or support a quality-based decoder classifier.

So the case for this backend is simple: when an application needs JPEGli's reconstruction, Metal can produce the same pixels with less latency and attributed energy in the measured region. It is not a claim that JPEGli is always the best JPEG decoder.

Coverage and prior work

Metal reconstruction covers baseline and progressive JPEG, grayscale, RGB/YCbCr, 4:4:4, 4:2:2, 4:2:0, odd dimensions, and restart intervals. The GPU entropy path is narrower: single-scan baseline input without restart markers. Progressive, restart, incremental, malformed, and unsupported cases transparently use CPU entropy. Malformed-input behavior, abort and destroy cleanup, Metal-disabled builds, installed headers, exported symbols, and the existing libjpeg-compatible API are covered by tests.

Sodsong et al.'s 2016 decoder [7] is the closest ancestor of the reconstruction path: it kept entropy decode on the CPU and moved IDCT, upsampling, and color conversion to OpenCL, using image size and entropy in its runtime model. The baseline path is closer to JParEnt [8], but it establishes chunk state and DC prefixes on the GPU instead of using GPU entropy for every image. Weißenberger and Schmidt [9] push more of JPEG onto the GPU for a throughput-oriented target. This project uses those ideas only where they reduce the latency of one consumer image.

References

1. Martin Bruse, Luca Versari, Zoltan Szabadka, and Jyrki Alakuijala. “Users Prefer Jpegli over Same-sized libjpeg-turbo or MozJPEG.” arXiv:2403.18589, 2024.
2. Challenge on Learned Image Compression. “CLIC 2025 Tasks and Data.” 7th Challenge on Learned Image Compression, 2025.
3. Jon Sneyers. “SSIMULACRA2: A Perceptual Image Quality Metric.” Project repository, 2023.
4. Google. “Butteraugli: A Tool for Measuring Perceived Differences Between Images.” Project repository, 2023.
5. Rafał K. Mantiuk, Param Hanji, Maliha Ashraf, Yuta Asano, and Alexandre Chapiro. “ColorVideoVDP: A Visual Difference Predictor for Image, Video, and Display Distortions.” ACM Transactions on Graphics 43(4), article 129, 2024.
6. Keyan Ding, Kede Ma, Shiqi Wang, and Eero P. Simoncelli. “Image Quality Assessment: Unifying Structure and Texture Similarity.” IEEE Transactions on Pattern Analysis and Machine Intelligence 44(5), 2567–2581, 2022.
7. Wasuwee Sodsong, Jingun Hong, Seongwook Chung, Yeong-Kyu Lim, Shin-Dug Kim, and Bernd Burgstaller. “Dynamic Partitioning-based JPEG Decompression on Heterogeneous Multicore Architectures.” Concurrency and Computation: Practice and Experience 28(2), 517–536, 2016. DOI: 10.1002/cpe.3620.
8. Wasuwee Sodsong, Minyoung Jung, Jinwoo Park, and Bernd Burgstaller. “JParEnt: Parallel Entropy Decoding for JPEG Decompression on Heterogeneous Multicore Architectures.” Concurrency and Computation: Practice and Experience 29, e4111, 2017.
9. André Weißenberger and Bertil Schmidt. “Accelerating JPEG Decompression on GPUs.” 2021 IEEE 28th International Conference on High Performance Computing, Data, and Analytics (HiPC), 121–130, 2021. DOI: 10.1109/HiPC53243.2021.00026.
10. libjpeg-turbo Project. “SIMD Coverage of the libjpeg Algorithms.” Project documentation, 2026.
11. Google. “Jpegli: an improved JPEG encoder and decoder implementation.” Project repository, 2026. The Metal decoder source and benchmarks, latency results, four-metric quality corpus, and energy report are on the experimental branch.