Cocco: Hardware-Mapping Co-Exploration towards Memory Capacity-Communication Optimization

TL;DR

Memory bandwidth and energy usage dominate performance in data-intensive DNN accelerators, and layer-level optimizations fall short on complex or irregular graphs. The authors introduce a graph-level subgraph execution scheme and a genetic-algorithm–based hardware-mapping co-exploration framework called Cocco, linking network structure features to memory configurations. Across plain, multi-branch, and irregular networks, Cocco achieves lower external memory access, reduced bandwidth, and substantial cost savings compared with greedy, DP, and SA baselines, demonstrating robust graph-partitioning and co-optimization capabilities. The work provides a scalable path to memory- and energy-efficient deployment of diverse DNN topologies on memory-constrained on-chip accelerators, with broad implications for future DNN accelerator design and automated mapping pipelines.

Abstract

Memory is a critical design consideration in current data-intensive DNN accelerators, as it profoundly determines energy consumption, bandwidth requirements, and area costs. As DNN structures become more complex, a larger on-chip memory capacity is required to reduce data movement overhead, but at the expense of silicon costs. Some previous works have proposed memory-oriented optimizations, such as different data reuse and layer fusion schemes. However, these methods are not general and potent enough to cope with various graph structures. In this paper, we explore the intrinsic connection between network structures and memory features to optimize both hardware and mapping. First, we introduce a graph-level execution scheme with a corresponding dataflow and memory management method. This scheme enables the execution of arbitrary graph patterns with high data reuse and low hardware overhead. Subsequently, we propose Cocco, a hardware-mapping co-exploration framework leveraging graph-level features of networks. It aims to minimize communication overhead, such as energy consumption and bandwidth requirements, with a smaller memory capacity. We formulate the graph-partition scheduling and memory configuration search as an optimization problem and employ a genetic-based method to achieve efficient co-exploration for large and irregular networks. Experiments demonstrate that Cocco obtains lower external memory access, lower bandwidth requirements, and more stable optimization for graph partition compared to the greedy algorithm and dynamic programming introduced in prior works. Cocco also reduces the costs by 1.89% to 50.33% using co-exploration compared to other typical methods.

Figures (14)

• Figure 1: The effect of different memory capacities for a computation graph. Intermediate results can be buffered in the on-chip memory if it is large enough. The on-chip memory of small capacity can only buffer two nodes (marked in the red dotted box), and the larger memory can cover a larger subgraph (right side).
• Figure 2: Left: performance v.s. memory capacity of several industrial NPUs. Right: a summary of SRAM area ratio in these accelerators.
• Figure 3: Evaluations on subgraphs fusing different number of layers (denoted as L=1,3,5). Y-axis is in the log domain. The 2TOPS NPU accelerator is configured with a 1MB global buffer and a 1.125MB weight buffer. The bandwidth requirement of weights is from the prefetch of the next subgraph, while that of activations is from the inputs and outputs of each subgraph.
• Figure 4: A conceptual comparison between two manners to process a subgraph. The node marked with a negative number represents the input node. The corresponding subgraph is shown in the upper right, where $F\times F/s$ refers to the convolution kernel size ($F$) and stride ($s$).
• Figure 5: The flow to determine the execution scheme of a subgraph (i.e., the computed tile size of each node, the tile offset, and the processing sequence of nodes). For simplicity, we discuss the 1D-CONV in this example and it is similar in the 2D-CONV case.