GreedyMemoryPlanner

class GreedyMemoryPlanner : public tflite::MicroMemoryPlanner

A memory planner that uses a greedy algorithm to arrange buffers in memory to minimize the overall arena size needed.

The algorithm works like this:

The client enters the buffer information through AddBuffer().
When a function like GetOffsetForBuffer() is called, the CalculateOffsetsIfNeeded() method is invoked.
If an up to date plan is not already present, one will be calculated.
The buffers are sorted in descending order of size.
The largest buffer is placed at offset zero.
The rest of the buffers are looped through in descending size order.
The other buffers that need to be in memory at the same time are found.
The first gap between simultaneously active buffers that the current buffer fits into will be used.
If no large-enough gap is found, the current buffer is placed after the last buffer that’s simultaneously active.
This continues until all buffers are placed, and the offsets stored.

This is not guaranteed to produce the best placement, since that’s an NP-Complete problem, but in practice it should produce one that’s decent.

tflite::GreedyMemoryPlanner::GreedyMemoryPlanner()

tflite::GreedyMemoryPlanner::~GreedyMemoryPlanner()

TfLiteStatus tflite::GreedyMemoryPlanner::Init(unsigned char *scratch_buffer, int scratch_buffer_size)

You need to pass in an area of memory to be used for planning. The client should ensure the validity of the memory when it needs to use this object. This memory isn’t owned by this object, so management should be handled by the client. This is so it can be stack or globally allocated if necessary on devices without dynamic memory allocation. How many buffers can be planned for will depend on the size of this scratch memory, so you should enlarge it if you see an error when calling AddBuffer(). The memory can be reused once you’re done with the planner, as long as you copy the calculated offsets to another location. Each buffer requires about 36 bytes of scratch.

Parameters

Parameters of Init
Name	Type	Default	Description
`scratch_buffer`	`unsigned char *`	Required
`scratch_buffer_size`	`int`	Required

TfLiteStatus tflite::GreedyMemoryPlanner::AddBuffer(int size, int first_time_used, int last_time_used)

Record details of a buffer we want to place.

Parameters

Parameters of AddBuffer
Name	Type	Default
`size`	`int`	Required
`first_time_used`	`int`	Required
`last_time_used`	`int`	Required

TfLiteStatus tflite::GreedyMemoryPlanner::AddBuffer(
    int size,
    int first_time_used,
    int last_time_used,
    int offline_offset
)

Record details of an offline planned buffer offset we want to place. offline_offset is the buffer offset from the start of the arena.

Parameters

Parameters of AddBuffer
Name	Type	Default
`size`	`int`	Required
`first_time_used`	`int`	Required
`last_time_used`	`int`	Required
`offline_offset`	`int`	Required

size_t tflite::GreedyMemoryPlanner::GetMaximumMemorySize()

Returns the high-water mark of used memory. This is the minimum size of a memory arena you’d need to allocate to hold these buffers.

int tflite::GreedyMemoryPlanner::GetBufferCount()

How many buffers have been recorded.

TfLiteStatus tflite::GreedyMemoryPlanner::GetOffsetForBuffer(int buffer_index, int *offset)

Where a given buffer should be placed in the memory arena. This information is stored in the memory arena itself, so once the arena is used for inference, it will be overwritten.

Parameters

Parameters of GetOffsetForBuffer
Name	Type	Default	Description
`buffer_index`	`int`	Required
`offset`	`int *`	Required

void tflite::GreedyMemoryPlanner::PrintMemoryPlan()

Prints an ascii-art diagram of the buffer layout plan.

bool tflite::GreedyMemoryPlanner::DoAnyBuffersOverlap()

Debug method to check whether any buffer allocations are overlapping. This is an O(N^2) complexity operation, so only use for testing.

static size_t tflite::GreedyMemoryPlanner::per_buffer_size()

Number of bytes required in order to plan a buffer.

bool tflite::GreedyMemoryPlanner::preserves_all_tensors() const

Returns False because the GreedyMemoryPlanner doesn’t preserves all tensors after invocation. Do to the fact that tensors that tensor data for tensors that aren’t being used during a phase of invocation are overwritten.