We report Linpack benchmark results on the TSUBAME supercomputer, a large scale heterogenous system with graphics processing units (GPUs) and ClearSpeed SIMD accelerators. With all of about 10,000 Opteron cores, 640 Xeon cores, 648 ClearSpeed accelerators and 624 NVIDIA Tesla GPUs, we have achieved 87TFlops. This paper describes careful tuning and load balancing method required to achieve this performance. On the other hand, since the peak speed is 163 TFlops, the efficiency is 53%, which is slower than other systems. This paper also discusses the reason of this gap from the aspect of system architecture.