patching

Portable image patch extraction and masked patch embeddings.

PatchLayer2Dclass
MaskedPatchEncoder2Dclass

PatchLayer2D(height: int, width: int, ch: int, patch_height: int, patch_width: int, **kwargs={})

Extract flattened patches from images shaped (batch, height, width, ch).

height = height

width = width

ch_size = ch

patch_height = patch_height

patch_width = patch_width

resize = keras.layers.Reshape((-1, patch_height * patch_width * ch))

call(images: Tensor) -> Tensor

get_config() -> dict[str, Any]

show_patched_image(images: keras.KerasTensor, patches: keras.KerasTensor) -> int

Plot one image and its reconstructed patches; return its batch index.

reconstruct_from_patch(patch: keras.KerasTensor) -> keras.KerasTensor

Reconstruct one image from non-overlapping patches in row-major order.

MaskedPatchEncoder2D(
    patch_height: int,
    patch_width: int,
    ch_size: int,
    projection_dim: int,
    mask_proportion: float,
    downstream: bool = False,
    seed: int | None = None,
    **kwargs={},
)

Project patches with position embeddings and sample masks from a seeded stream.

patch_height = patch_height

patch_width = patch_width

ch_size = ch_size

projection_dim = projection_dim

mask_proportion = mask_proportion

downstream = downstream

seed = seed

seed_generator = keras.random.SeedGenerator(seed)

projection = keras.layers.Dense(units=self.projection_dim)

position_embedding = None

build(input_shape)

call(patches: Tensor) -> Tensor | tuple[Tensor, Tensor, Tensor, Tensor, Tensor]

get_random_indices(batch_size: int) -> tuple[Tensor, Tensor]

get_config() -> dict[str, Any]

generate_masked_image(patches: keras.KerasTensor, unmask_indices: keras.KerasTensor)