Abstract: In this paper, a high-performance heterogeneous multi-core DSP architecture is proposed and implemented based on CostarII core. The multi-core DSP architecture integrates four DSP cores and a RISC core, each core of which has its own local memory. All the cores can access data shared-memory with parallel multi-bank structure, and the four DSP cores can use program shared-memory. Furthermore, all the cores hold several synchronization and communication mechanisms, such as mailbox, semaphore and interruptions. To verify the multi-core DSP architecture, the JPEG parallel decoding is implemented in the system and verified in FPGA validation platform. The results show that the design is easy to program and efficient for parallel tasks.