Add pixel formats and transparency

[madsmtm]

Add:

impl<D, W> Surface<D, W> {
    pub fn alpha_mode(&self) -> AlphaMode { ... }
    pub fn pixel_mode(&self) -> PixelMode { ... }
    pub fn configure(
        &mut self,
        width: NonZeroU32,
        height: NonZeroU32,
        alpha_mode: AlphaMode,
        pixel_mode: PixelMode,
    ) -> Result<(), SoftBufferError> { ... }
    
    // `resize` now calls `configure`:
    pub fn resize(&mut self, width: NonZeroU32, height: NonZeroU32) -> Result<(), SoftBufferError> {
        self.configure(width, height, self.alpha_mode(), self.pixel_mode())
    }

    // Earlier formats are preferred.
    //
    // TODO: We could possibly instead return:
    // `impl IntoIterator<Item = &[PixelFormat]>`
    //
    // To support "tranches" like Wayland does, i.e. groups of pixel formats with the same preference.
    pub fn supported_pixel_formats(&self, alpha_mode: AlphaMode) -> &[PixelFormat] { ... }
}

pub enum AlphaMode {
    #[default]
    Opaque,
    PreMultiplied,
    PostMultiplied,
}

// From https://github.com/rust-windowing/softbuffer/pull/289
pub const DEFAULT_PIXEL_FORMAT: PixelFormat = ...;

// Notice the lack of `Bgrx8` and similar!
// If `AlphaMode::Opaque`, we will treat `A` as `X` (i.e. ignore the alpha channel).
pub enum PixelFormat {
    // Normal RGB formats
    Bgr8,
    Rgb8,
    Bgra8,
    Rgba8,
    Abgr8,
    Argb8,
    Bgr16,
    Rgb16,
    Bgra16,
    Rgba16,
    Abgr16,
    Argb16,

    // Grayscale formats
    R1,
    R2,
    R4,
    R8,
    R16,

    // Packed formats
    B2g3r3,
    R3g3b2,
    B5g6r5,
    R5g6b5,
    Bgra4,
    Rgba4,
    Abgr4,
    Argb4,
    Bgr5a1,
    Rgb5a1,
    A1bgr5,
    A1rgb5,
    Bgr10a2,
    Rgb10a2,
    A2bgr10,
    A2rgb10,

    // Floating point formats
    // Uses the `half` crate for the `f16` type.
    #[cfg(feature = "f16")]
    Bgr16f,
    #[cfg(feature = "f16")]
    Rgb16f,
    #[cfg(feature = "f16")]
    Bgra16f,
    #[cfg(feature = "f16")]
    Rgba16f,
    #[cfg(feature = "f16")]
    Abgr16f,
    #[cfg(feature = "f16")]
    Argb16f,
    Bgr32f,
    Rgb32f,
    Bgra32f,
    Rgba32f,
    Abgr32f,
    Argb32f,
}

impl PixelFormat {
    pub const fn bits_per_pixel(self) -> u8 { ... }
    pub const fn components(self) -> u8 { ... }
}

// Which one of these methods is desirable depends on the format:
// - Normal formats will want to use the format with the same size as their component.
//   - Bgr8 -> `data`, `Bgr16` -> `data_u16`, `Bgr16f` -> `data_f16`, `Bgr32f` -> `data_f32`.
//   - Users will want to use `as_chunks_mut::<{ format.components() }>()` on each row.
// - Packed formats will want to use the format with the same size as _the entire format_:
//   - `B2g3r3` -> `data`, `Bgra4` -> `data_u16`, `Bgr10a2` -> `data_u32`.
impl Buffer<'_> {
    pub fn data(&mut self) -> &mut [u8] { ... }
    pub fn rows(&mut self) -> impl DoubleEndedIterator<Item = &mut [u8]> + ExactSizeIterator { ... }

    pub fn data_u16(&mut self) -> &mut [u16] { ... }
    pub fn rows_u16(&mut self) -> impl DoubleEndedIterator<Item = &mut [u16]> + ExactSizeIterator { ... }

    pub fn data_u32(&mut self) -> &mut [u32] { ... }
    pub fn rows_u32(&mut self) -> impl DoubleEndedIterator<Item = &mut [u32]> + ExactSizeIterator { ... }

    #[cfg(feature = "f16")]
    pub fn data_f16(&mut self) -> &mut [f16] { ... }
    #[cfg(feature = "f16")]
    pub fn rows_f16(&mut self) -> impl DoubleEndedIterator<Item = &mut [f16]> + ExactSizeIterator { ... }
    
    pub fn data_f32(&mut self) -> &mut [f32] { ... }
    pub fn rows_f32(&mut self) -> impl DoubleEndedIterator<Item = &mut [f32]> + ExactSizeIterator { ... }
}

// Existing are only valid when `pixel_format == DEFAULT_PIXEL_FORMAT`.
// If that is not the case, the `Pixel` values are non-sensical (but still sound).
// (Or perhaps we make these methods panic then?).
impl Buffer<'_> {
    fn pixels(&mut self) -> &mut [Pixel] { ... }
    fn pixel_rows(&mut self) -> impl Iterator<Item = &mut [Pixel]> { ... }
    fn pixels_iter(&mut self) -> impl Iterator<Item = (u32, u32, &mut Pixel)> { ... }
}

When the pixel format or alpha mode is not supported by the backend, we allocate an intermediate buffer which is what the user is given to render into, and then we copy from that buffer to the actual buffer in present. The actual buffer in this case is chosen to be DEFAULT_PIXEL_FORMAT to reduce the different permutations of pixel format conversions.

Fixes #17 by adding AlphaMode.
Fixes #98 by adding PixelFormat and conversions between those.
Replaces #186.
Replaces #241.

Draft because this relies on a bunch of other PRs to land first, and because I haven't actually implemented it fully yet. I'll probably split this PR out further too.

[madsmtm]

So, I'm fairly certain that we want this in some form, but the question is how?

Asked another way, what should happen when a requested pixel format is not supported by the backend?

1. We allocate an additional buffer, give that to the user as Buffer::pixels, and convert in Buffer::present.
2. The above, but as an optimization, when the format only differs in order (PixelFormat::Rgba vs PixelFormat::Bgra), we convert it in place in Surface::buffer_mut and Buffer::present (and Buffer::drop).
3. We expose a conversion function publicly, and error out on unsupported pixel formats. The user is expected to check the supported formats, and then use the conversion function to convert the buffer themselves.

See also the discussion in #98.

I haven't yet explored option 3, but it's likely it would be the most desirable, it will be surprising for the user when extra allocations and copying happens, and it seems more in line with Rust's philosophy to allow them to control this.

Especially relevant is that cargo bloat --example winit --crates --release reveals:

• Std: 279.1KiB
• Winit: 107.8KiB
• Softbuffer before this PR: 4.3KiB
• Softbuffer with convert_fallback symbol (current, alpha monomorphized): 90.7KiB
• Softbuffer with convert_fallback symbol (alpha dynamic): 37.7KiB

Which would impact option 1 and 2 unless we change convert_fallback to be much more dynamic (and thus much less performant). Option 3 would be able to strip it out when unused, and probably also allow only emitting the relevant conversion code needed for a given application if they specify the pixel formats to be converted in line.

[nicoburns]

The match seems problematic here. It forces every format to be included in the final binary even if they are never used.

If each individual conversion function (one per format) were exposed as a separate function rather than being wrapped into one function that dynamically dispatches (and it was left to the user to implement (static-or-dynamic) dispatch for the format(s) that they actually support) then the unused functions could likely be optimised out by the compiling at higher opt modes / LTO.

And if that isn't sufficient then they could be feature flagged.

[madsmtm]

Yeah, I'm leaning towards not doing format conversion internally, and instead leave that up to the user.

If each individual conversion function (one per format) were exposed as a separate function rather than being wrapped into one function that dynamically dispatches (and it was left to the user to implement (static-or-dynamic) dispatch for the format(s) that they actually support) then the unused functions could likely be optimised out by the compiling at higher opt modes / LTO.

I think something like the following would still be able to be dead-code stripped fairly easily?

impl Buffer<'_> {
    /// Convert to `PixelFormat::default()`.
    #[inline]
    pub fn convert_to_pixels(&mut self, source: &[u8], source_stride: u32, source_format: PixelFormat) {
        match format {
            PixelFormat::Rgba8 => ...,
            PixelFormat::Bgra8 => ...,
            // ...
        }
    }
}

// Usage

let width = buffer.width().get();
let height = buffer.height().get();
let mut temporary = vec![0u8, width as usize * height as usize];
draw(&mut temporary);
buffer.convert_to_pixels(&temporary, width, PixelFormat::Rgba8); // Specify pixel format inline

A similar pattern is done with std::sync::atomic::Ordering.

[nicoburns]

I don't think it's going to be able to strip it out very easily while it's matching on an enum. It would need to be certain that nothing in the entire binary calls it with one of the other variants (and it is possible to dynamically construct a variant via casting, etc so I don't see how it could know that).

I think you'd need buffer.convert_to_rgba8, buffer.convert_to_bgra8`, etc for dead code to be eliminated.

[madsmtm]

I don't think it's going to be able to strip it out very easily while it's matching on an enum. It would need to be certain that nothing in the entire binary calls it with one of the other variants (and it is possible to dynamically construct a variant via casting, etc so I don't see how it could know that).

As long as the enum value is known statically to the compiler, and the function is inlined, I'm pretty sure it can.

See for example this godbolt. The string from the other variant is not present in the final binary.

Maybe we're talking past each other? Let's establish a baseline: if there is no code that calls Buffer::convert_to_pixels, it can be stripped, either by the compiler or the linker. There is no dynamic language feature that allows you to call a function that isn't mentioned anywhere.

[nicoburns]

Maybe we're talking past each other?

Yeah, maybe.

As long as the enum value is known statically to the compiler, and the function is inlined

I wonder if it depends on it getting inlined though. In any case, I suppose it's an empirical question that we can measure and check.