Answers are grounded in this paper and cite their sources.
Current GPUs are massively parallel multicore processors optimised for workloads with a large degree of SIMD parallelism. Good performance requires highly idiomatic programs, whose development is work intensive and requires expert knowledge.
No takes yet. Share an insight, caveat, or question.
Chakravarty et al. (2011) studied this question.