GHSA-w5p8-4jcx-2j6r: imageproc: integer overflow in kernel size check leads to out-of-bounds read

A bounds verification of a slice storage of a 2-dimensional matrix’s coefficients (a kernel) would compare the total size against the product of individual dimensions. This would erroneously cast after the multiplication and consequently fail to detect possible violations when overflow occurs.

Afterwards, the individual sizes were trusted to properly constrain coordinates within the matrix to indices valid for the underlying storage. With a crafted Kernel object, certain combinations of coordinates could then cause an out-of-bounds access in an unsafe function while fulfilling its documented preconditions. The kernel value could be passed to library functions that trusted the preconditions and then performed such reads.