Handling images in embedded systems is always a battle against constraints.

On microcontrollers like the RP2350, the situation is even more extreme. For DVI output on this chip, you get about 6 cycles per pixel. Within that budget, you must:

1. Fetch pixel data (memory read)
2. Palette lookup (color conversion)
3. Store to line buffer (memory write)

Total: ~6 cycles — almost no margin for error.

nibnib, an image format created by Raphael Finkel, was born from these extreme constraints. And it offers more than simple compression — it presents an algorithmically interesting approach to a hard problem.

nibnib's Core Idea: Delta Palettes

Standard palette-based image formats store an index for each pixel and look up the absolute color from a palette during decoding.

Standard palette image:
  Data: [3, 7, 12, 5, ...] (indices)
  Palette: [0xFF0000, 0x00FF00, 0x0000FF, ...]
  Decode: index → absolute color from palette

Problem: In smooth regions (gradients), large color differences
         require many palette entries, and quantization errors are visible.

nibnib stores deltas (differences) in the palette, not absolute colors.

nibnib delta palette:
  Data: [3, 7, 12, 5, ...] (indices)
  Palette: [0xFF0000, +0x010101, -0x020100, ...] (deltas)
  Decode: previous pixel × 2 + delta → two RGB 565 pixels

The key insight: adjust colors relative to the previous pixel. In smooth regions, adjacent pixels have similar colors, so small deltas can represent them accurately.

Decoding Algorithm: Two Pixels Per Byte

nibnib's decoding is simple and deterministic.

Decode loop (pixel pair unit):

1. Double the last decoded pixel (left = right = prev)
2. Read one byte from input stream → palette[index]
3. Add palette[index] to the doubled pixel
4. Lower 16 bits = left pixel (RGB 565)
   Upper 16 bits = right pixel (RGB 565)
5. Continue to next pair

One byte → two pixels = 4bpp compression

This algorithm can be implemented very efficiently on microcontrollers.

Estimated cycles on RP2350:

  prev_pixel << 1           → 1 cycle
  load_byte(input)          → 1 cycle
  load_palette(index)       → 2 cycles (L1 cache hit)
  add + split               → 1 cycle
  store_to_line_buffer      → 1 cycle

  Total: ~6 cycles per pixel pair

File Format Structure

┌──────────────┐
│ Width (2B)   │  Image width (16-bit)
├──────────────┤
│ Height (2B)  │  Image height (16-bit)
├──────────────┤
│ Palette      │  256 × 32-bit delta entries
│ (8192 bytes) │
├──────────────┤
│ Row 0        │  compressed data (stride = 4-byte aligned)
├──────────────┤
│ Row 1        │
├──────────────┤
│ ...          │
└──────────────┘

Row stride is rounded up to a multiple of 4 bytes, designed for compatibility with RP2350 DMA transfers.

Encoding: Floyd-Steinberg Dithering

Decoding is simple, but encoding is complex. nibnib adopts an asymmetric design.

Encoding: slow (high-quality palette search + Floyd-Steinberg dithering)
Decoding: fast (simple delta lookup)

In embedded environments, decoding speed is far more important.
Encoding happens once; decoding repeats every frame.

Floyd-Steinberg dithering distributes quantization error to adjacent pixels, producing smoother gradients.

Floyd-Steinberg error diffusion:

  Current pixel [X] → distribute error to next pixels:

    [X] 7/16
     ↓
   ┌───┬───┬───┐
   │   │ 3/16│ 5/16│
   └───┴───┴───┘
 1/16   3/16

Error spreads to neighboring pixels, making quantization less visible.

The Author: Raphael Finkel

nibnib was created by Raphael Finkel, a well-known figure in computer science.

  • Creator of TeX's picture environment
  • Former Microsoft Research researcher
  • Author of multiple algorithms and compiler papers

This background gives nibnib technical depth. It's not a casual experiment — it's a format carefully designed from an algorithmic perspective.

Limitations and Considerations

  1. Not for archival: nibnib is an embedded format designed for tight integration with a decoder. Not suitable for general-purpose archival.
  2. RGB 565 only: Supports 16-bit color only. Does not handle HDR or wide color gamuts.
  3. Encoding speed: Floyd-Steinberg dithering and palette search make encoding slow. Not suitable for real-time encoding.
  4. Limited ecosystem: Still an obscure format with no wide adoption.

Developer Takeaway

nibnib offers three lessons.

1. Constraints drive innovation. The extreme 6-cycle constraint produced the delta palette — a creative algorithm born from necessity.

2. Asymmetric design is embedded systems' default. Encoding can be slow; decoding must be fast. This principle applies almost universally in embedded systems.

3. Delta palettes excel at gradients. Storing differences instead of absolute colors achieves higher quality with far fewer palette entries in smooth regions.

Even beyond RP2350, nibnib's approach is worth studying for anyone handling images in resource-constrained environments.