# microallocator

Arena allocation and memory-planner integration.

## tflite::MicroAllocator

`class` · `cpp`

```cpp
class MicroAllocator
```

Allocator responsible for allocating memory for all intermediate tensors necessary to invoke a model.

The lifetime of the model, tensor arena and error reporter must be at least as long as that of the allocator object, since the allocator needs them to be accessible during its entire lifetime.

The `MicroAllocator` simply plans out additional allocations that are required to standup a model for inference in TF Micro. This class currently relies on an additional allocator - SingleArenaBufferAllocator - for all allocations from an arena. These allocations are divided into head (non-persistent) and tail (persistent) regions:

Memory layout to help understand how it works This information could change in the future version. ************** .memory_allocator->GetBuffer() Tensors/Scratch buffers (head) ************** .head_watermark unused memory ************** .memory_allocator->GetBuffer() + ->GetMaxBufferSize()

- ->GetDataSize() persistent area (tail) ************** .memory_allocator->GetBuffer() + ->GetMaxBufferSize()

Source: `tensorflow/lite/micro/micro_allocator.h:128`

### tflite::MicroAllocator::Create--classtflite_1_1MicroAllocator_1a421bb109a0a7e5132d8489e435abf97f

`method` · `cpp`

```cpp
static MicroAllocator * tflite::MicroAllocator::Create(
    uint8_t *tensor_arena,
    size_t arena_size,
    MemoryPlannerType memory_planner_type = MemoryPlannerType::kGreedy
)
```

Creates a `MicroAllocator` instance from a given tensor arena. This arena will be managed by the created instance. The `GreedyMemoryPlanner` will by default be used and created on the arena. Note: Please use alignas(16) to make sure tensor_arena is 16 bytes aligned, otherwise some head room will be wasted. TODO(b/157615197): Cleanup constructor + factory usage.

**Parameters**

| Name | Type | Default | Description |
| --- | --- | --- | --- |
| tensor_arena | uint8_t * | Required |  |
| arena_size | size_t | Required |  |
| memory_planner_type | MemoryPlannerType | MemoryPlannerType::kGreedy |  |

Source: `tensorflow/lite/micro/micro_allocator.h:136`

### tflite::MicroAllocator::Create--classtflite_1_1MicroAllocator_1a018b9d20e8d9ebd17360948e622237b7

`method` · `cpp`

```cpp
static MicroAllocator * tflite::MicroAllocator::Create(
    uint8_t *tensor_arena,
    size_t arena_size,
    MicroMemoryPlanner *memory_planner
)
```

Creates a `MicroAllocator` instance from a given tensor arena and a given MemoryPlanner. This arena will be managed by the created instance. Note: Please use alignas(16) to make sure tensor_arena is 16 bytes aligned, otherwise some head room will be wasted.

**Parameters**

| Name | Type | Default | Description |
| --- | --- | --- | --- |
| tensor_arena | uint8_t * | Required |  |
| arena_size | size_t | Required |  |
| memory_planner | MicroMemoryPlanner * | Required |  |

Source: `tensorflow/lite/micro/micro_allocator.h:144`

### tflite::MicroAllocator::Create--classtflite_1_1MicroAllocator_1a35bcf5002a662e06502afefed8f643c6

`method` · `cpp`

```cpp
static MicroAllocator * tflite::MicroAllocator::Create(
    SingleArenaBufferAllocator *memory_allocator,
    MicroMemoryPlanner *memory_planner
)
```

Creates a `MicroAllocator` instance using the provided SingleArenaBufferAllocator instance and the MemoryPlanner. This allocator instance will use the SingleArenaBufferAllocator instance to manage allocations internally.

**Parameters**

| Name | Type | Default | Description |
| --- | --- | --- | --- |
| memory_allocator | SingleArenaBufferAllocator * | Required |  |
| memory_planner | MicroMemoryPlanner * | Required |  |

Source: `tensorflow/lite/micro/micro_allocator.h:151`

### tflite::MicroAllocator::Create--classtflite_1_1MicroAllocator_1a91f02616f35cb777c2c9f2607f18588c

`method` · `cpp`

```cpp
static MicroAllocator * tflite::MicroAllocator::Create(
    uint8_t *persistent_tensor_arena,
    size_t persistent_arena_size,
    uint8_t *non_persistent_tensor_arena,
    size_t non_persistent_arena_size,
    MemoryPlannerType memory_planner_type = MemoryPlannerType::kGreedy
)
```

Creates a `MicroAllocator` instance using the provided SingleArenaBufferAllocator instance and the MemoryPlanner. This allocator instance will use the SingleArenaBufferAllocator instance to manage allocations internally.

**Parameters**

| Name | Type | Default | Description |
| --- | --- | --- | --- |
| persistent_tensor_arena | uint8_t * | Required |  |
| persistent_arena_size | size_t | Required |  |
| non_persistent_tensor_arena | uint8_t * | Required |  |
| non_persistent_arena_size | size_t | Required |  |
| memory_planner_type | MemoryPlannerType | MemoryPlannerType::kGreedy |  |

Source: `tensorflow/lite/micro/micro_allocator.h:158`

### tflite::MicroAllocator::GetDefaultTailUsage

`method` · `cpp`

```cpp
static size_t tflite::MicroAllocator::GetDefaultTailUsage(bool is_memory_planner_given)
```

Returns the fixed amount of memory overhead of `MicroAllocator`.

**Parameters**

| Name | Type | Default | Description |
| --- | --- | --- | --- |
| is_memory_planner_given | bool | Required |  |

Source: `tensorflow/lite/micro/micro_allocator.h:164`

### tflite::MicroAllocator::preserves_all_tensor

`method` · `cpp`

```cpp
bool tflite::MicroAllocator::preserves_all_tensor() const
```

Returns True if the `MicroAllocator` uses a `LinearMemoryPlanner`(is compatible with the PerserveAllTensors flag / feature ) and False otherwise.

Source: `tensorflow/lite/micro/micro_allocator.h:168`

### tflite::MicroAllocator::StartModelAllocation

`method` · `cpp`

```cpp
SubgraphAllocations * tflite::MicroAllocator::StartModelAllocation(const Model *model)
```

Allocates internal resources required for model inference for each subgraph from the arena.

This method will run through the flatbuffer data supplied in the model to properly allocate tensor, node, and op registration data. This method is expected to be followed with a call to `FinishModelAllocation()` Returns a pointer to an array of `SubgraphAllocations` (also stored in the tail of the arena) where each index corresponds to a different subgraph in the model. Return value is nullptr if the allocations failed.

**Parameters**

| Name | Type | Default | Description |
| --- | --- | --- | --- |
| model | const Model * | Required |  |

Source: `tensorflow/lite/micro/micro_allocator.h:181`

### tflite::MicroAllocator::FinishModelAllocation

`method` · `cpp`

```cpp
TfLiteStatus tflite::MicroAllocator::FinishModelAllocation(
    const Model *model,
    SubgraphAllocations *subgraph_allocations,
    ScratchBufferHandle **scratch_buffer_handles
)
```

Finish allocating internal resources required for model inference.

-Plan the memory for activation tensors and scratch buffers. -Update eval tensors for each subgraph based on planned offsets. -Allocate scratch buffer handles array and update based on planned offsets.

This method should be called after assigning model resources in `StartModelAllocation()`. The subgraph_allocations pointer should be the value passed into this class during `StartModelAllocation()`. Scratch buffer handles are stored in the out-param `scratch_buffer_handles` array which is allocated in this method. This value will be used in `GetScratchBuffer` call to retrieve scratch buffers.

**Parameters**

| Name | Type | Default | Description |
| --- | --- | --- | --- |
| model | const Model * | Required |  |
| subgraph_allocations | SubgraphAllocations * | Required |  |
| scratch_buffer_handles | ScratchBufferHandle ** | Required |  |

Source: `tensorflow/lite/micro/micro_allocator.h:195`

### tflite::MicroAllocator::AllocatePersistentTfLiteTensor

`method` · `cpp`

```cpp
virtual TfLiteTensor * tflite::MicroAllocator::AllocatePersistentTfLiteTensor(
    const Model *model,
    const SubgraphAllocations *subgraph_allocations,
    int tensor_index,
    int subgraph_index
)
```

Allocates a `TfLiteTensor` struct and populates the returned value with properties from the model flatbuffer. This struct is allocated from persistent arena memory is only guaranteed for the lifetime of the application. The eval_tensors pointer should be the value passed into this class during `StartModelAllocation()` and contains the source-of-truth for buffers.

**Parameters**

| Name | Type | Default | Description |
| --- | --- | --- | --- |
| model | const Model * | Required |  |
| subgraph_allocations | const SubgraphAllocations * | Required |  |
| tensor_index | int | Required |  |
| subgraph_index | int | Required |  |

Source: `tensorflow/lite/micro/micro_allocator.h:205`

### tflite::MicroAllocator::AllocateTempTfLiteTensor

`method` · `cpp`

```cpp
virtual TfLiteTensor * tflite::MicroAllocator::AllocateTempTfLiteTensor(
    const Model *model,
    const SubgraphAllocations *subgraph_allocations,
    int tensor_index,
    int subgraph_index
)
```

Allocates a `TfLiteTensor` struct and populates the returned value with properties from the model flatbuffer. This struct is allocated from temporary arena memory is only guaranteed until a call is made to `ResetTempAllocations()`. Subgraph_allocations contains the array of TfLiteEvalTensors. If the newly allocated temp at the specified subgraph and tensor index is already present int the `TfLiteEvalTensor` array, its data buffer will be reused.

**Parameters**

| Name | Type | Default | Description |
| --- | --- | --- | --- |
| model | const Model * | Required |  |
| subgraph_allocations | const SubgraphAllocations * | Required |  |
| tensor_index | int | Required |  |
| subgraph_index | int | Required |  |

Source: `tensorflow/lite/micro/micro_allocator.h:216`

### tflite::MicroAllocator::DeallocateTempTfLiteTensor

`method` · `cpp`

```cpp
virtual void tflite::MicroAllocator::DeallocateTempTfLiteTensor(TfLiteTensor *)
```

**Parameters**

| Name | Type | Default | Description |
| --- | --- | --- | --- |
|  | TfLiteTensor * | Required |  |

Source: `tensorflow/lite/micro/micro_allocator.h:220`

### tflite::MicroAllocator::AllocateTempBuffer

`method` · `cpp`

```cpp
virtual uint8_t * tflite::MicroAllocator::AllocateTempBuffer(size_t size, size_t alignment)
```

Returns a pointer to a buffer from the temporary arena memory and is only guaranteed until a call is made to `ResetTempAllocations()`.

**Parameters**

| Name | Type | Default | Description |
| --- | --- | --- | --- |
| size | size_t | Required |  |
| alignment | size_t | Required |  |

Source: `tensorflow/lite/micro/micro_allocator.h:224`

### tflite::MicroAllocator::DeallocateTempBuffer

`method` · `cpp`

```cpp
virtual void tflite::MicroAllocator::DeallocateTempBuffer(uint8_t *buffer)
```

Signals that the temporary buffer no longer needed.

**Parameters**

| Name | Type | Default | Description |
| --- | --- | --- | --- |
| buffer | uint8_t * | Required |  |

Source: `tensorflow/lite/micro/micro_allocator.h:227`

### tflite::MicroAllocator::ResetTempAllocations

`method` · `cpp`

```cpp
virtual TfLiteStatus tflite::MicroAllocator::ResetTempAllocations()
```

Resets all temporary allocations. This method should be called after a chain of temp allocations (e.g. chain of `TfLiteTensor` objects via AllocateTfLiteTensor()).

Source: `tensorflow/lite/micro/micro_allocator.h:232`

### tflite::MicroAllocator::IsAllTempDeallocated

`method` · `cpp`

```cpp
virtual bool tflite::MicroAllocator::IsAllTempDeallocated()
```

Returns true if all temporary buffers including temp `TfLiteTensor` are already deallocated.

Source: `tensorflow/lite/micro/micro_allocator.h:236`

### tflite::MicroAllocator::AllocatePersistentBuffer

`method` · `cpp`

```cpp
virtual void * tflite::MicroAllocator::AllocatePersistentBuffer(size_t bytes)
```

Allocates persistent buffer which has the same life time as the allocator. The memory is immediately available and is allocated from the tail of the arena.

**Parameters**

| Name | Type | Default | Description |
| --- | --- | --- | --- |
| bytes | size_t | Required |  |

Source: `tensorflow/lite/micro/micro_allocator.h:241`

### tflite::MicroAllocator::RequestScratchBufferInArena

`method` · `cpp`

```cpp
TfLiteStatus tflite::MicroAllocator::RequestScratchBufferInArena(size_t bytes, int subgraph_idx, int *buffer_idx)
```

Register a scratch buffer of size `bytes` for Node with `node_id`. This method only requests a buffer with a given size to be used after a model has finished allocation via `FinishModelAllocation()`. All requested buffers will be accessible by the out-param in that method.

**Parameters**

| Name | Type | Default | Description |
| --- | --- | --- | --- |
| bytes | size_t | Required |  |
| subgraph_idx | int | Required |  |
| buffer_idx | int * | Required |  |

Source: `tensorflow/lite/micro/micro_allocator.h:247`

### tflite::MicroAllocator::FinishPrepareNodeAllocations

`method` · `cpp`

```cpp
TfLiteStatus tflite::MicroAllocator::FinishPrepareNodeAllocations(int node_id)
```

Finish allocating a specific `NodeAndRegistration` prepare block (kernel entry for a model) with a given node ID. This call ensures that any scratch buffer requests and temporary allocations are handled and ready for the next node prepare block.

**Parameters**

| Name | Type | Default | Description |
| --- | --- | --- | --- |
| node_id | int | Required |  |

Source: `tensorflow/lite/micro/micro_allocator.h:254`

### tflite::MicroAllocator::used_bytes

`method` · `cpp`

```cpp
size_t tflite::MicroAllocator::used_bytes() const
```

Returns the arena usage in bytes, only available after `FinishModelAllocation`. Otherwise, it will return 0.

Source: `tensorflow/lite/micro/micro_allocator.h:258`

### tflite::MicroAllocator::GetBuiltinDataAllocator

`method` · `cpp`

```cpp
TfLiteBridgeBuiltinDataAllocator * tflite::MicroAllocator::GetBuiltinDataAllocator()
```

Source: `tensorflow/lite/micro/micro_allocator.h:260`

### tflite::MicroAllocator::MicroAllocator--classtflite_1_1MicroAllocator_1a18615b3ce4daf12e3d715e09600366c3

`method` · `cpp`

```cpp
tflite::MicroAllocator::MicroAllocator(SingleArenaBufferAllocator *memory_allocator, MicroMemoryPlanner *memory_planner)
```

**Parameters**

| Name | Type | Default | Description |
| --- | --- | --- | --- |
| memory_allocator | SingleArenaBufferAllocator * | Required |  |
| memory_planner | MicroMemoryPlanner * | Required |  |

Source: `tensorflow/lite/micro/micro_allocator.h:263`

### tflite::MicroAllocator::MicroAllocator--classtflite_1_1MicroAllocator_1a240a5f8a03e3962648a92cb83cdd9d1e

`method` · `cpp`

```cpp
tflite::MicroAllocator::MicroAllocator(
    IPersistentBufferAllocator *persistent_buffer_allocator,
    INonPersistentBufferAllocator *non_persistent_buffer_allocator,
    MicroMemoryPlanner *memory_planner
)
```

**Parameters**

| Name | Type | Default | Description |
| --- | --- | --- | --- |
| persistent_buffer_allocator | IPersistentBufferAllocator * | Required |  |
| non_persistent_buffer_allocator | INonPersistentBufferAllocator * | Required |  |
| memory_planner | MicroMemoryPlanner * | Required |  |

Source: `tensorflow/lite/micro/micro_allocator.h:265`

### tflite::MicroAllocator::~MicroAllocator

`method` · `cpp`

```cpp
virtual tflite::MicroAllocator::~MicroAllocator()
```

Source: `tensorflow/lite/micro/micro_allocator.h:268`

### tflite::MicroAllocator::AllocateNodeAndRegistrations

`method` · `cpp`

```cpp
virtual TfLiteStatus tflite::MicroAllocator::AllocateNodeAndRegistrations(
    const Model *model,
    SubgraphAllocations *subgraph_allocations
)
```

Allocates an array in the arena to hold pointers to the node and registration pointers required to represent the inference graph of the model.

**Parameters**

| Name | Type | Default | Description |
| --- | --- | --- | --- |
| model | const Model * | Required |  |
| subgraph_allocations | SubgraphAllocations * | Required |  |

Source: `tensorflow/lite/micro/micro_allocator.h:282`

### tflite::MicroAllocator::AllocateTfLiteEvalTensors

`method` · `cpp`

```cpp
virtual TfLiteStatus tflite::MicroAllocator::AllocateTfLiteEvalTensors(
    const Model *model,
    SubgraphAllocations *subgraph_allocations
)
```

Allocates the list of persistent TfLiteEvalTensors that are used for the "eval" phase of model inference. These structs will be the source of truth for all tensor buffers.

**Parameters**

| Name | Type | Default | Description |
| --- | --- | --- | --- |
| model | const Model * | Required |  |
| subgraph_allocations | SubgraphAllocations * | Required |  |

Source: `tensorflow/lite/micro/micro_allocator.h:288`

### tflite::MicroAllocator::AllocateVariables

`method` · `cpp`

```cpp
virtual TfLiteStatus tflite::MicroAllocator::AllocateVariables(
    const SubGraph *subgraph,
    TfLiteEvalTensor *eval_tensors,
    const int32_t *offline_planner_offsets
)
```

Allocates persistent tensor buffers for variable tensors in the subgraph. Online and offline variable tensors are handled differently hence the offline_planner_offsets parameter is needed.

**Parameters**

| Name | Type | Default | Description |
| --- | --- | --- | --- |
| subgraph | const SubGraph * | Required |  |
| eval_tensors | TfLiteEvalTensor * | Required |  |
| offline_planner_offsets | const int32_t * | Required |  |

Source: `tensorflow/lite/micro/micro_allocator.h:294`

### tflite::MicroAllocator::AllocatePersistentTfLiteTensorInternal

`method` · `cpp`

```cpp
virtual TfLiteTensor * tflite::MicroAllocator::AllocatePersistentTfLiteTensorInternal()
```

Allocate and return a persistent `TfLiteTensor`. TODO(b/162311891): Drop this method when the interpreter has an API for accessing `TfLiteEvalTensor` structs.

Source: `tensorflow/lite/micro/micro_allocator.h:301`

### tflite::MicroAllocator::PopulateTfLiteTensorFromFlatbuffer

`method` · `cpp`

```cpp
virtual TfLiteStatus tflite::MicroAllocator::PopulateTfLiteTensorFromFlatbuffer(
    const Model *model,
    TfLiteTensor *tensor,
    int tensor_index,
    int subgraph_idx,
    bool allocate_temp
)
```

Populates a `TfLiteTensor` struct with data from the model flatbuffer. Any quantization data is allocated from either the tail (persistent) or temp sections of the arena based on the allocation flag.

**Parameters**

| Name | Type | Default | Description |
| --- | --- | --- | --- |
| model | const Model * | Required |  |
| tensor | TfLiteTensor * | Required |  |
| tensor_index | int | Required |  |
| subgraph_idx | int | Required |  |
| allocate_temp | bool | Required |  |

Source: `tensorflow/lite/micro/micro_allocator.h:306`