One way is to stack each channel on height axis and use that big pixel buffer as a backing of MLMultiArray. So:

// size = (width, height * 4)
// format = kCVPixelFormatType_OneComponent16Half
let backingPixelBuffer = ...
let multiArray = MLMultiArray(pixelBuffer: backingPixelBuffer, 
                              shape: [4, height, width])

This won’t work if your image representation is so called “packed” format, where each channel is interleaved in the frame buffer. We appreciate your feedback assistant report with a use case if that’s what you are looking for.