{"id":152,"date":"2010-08-01T12:36:11","date_gmt":"2010-08-01T11:36:11","guid":{"rendered":"http:\/\/numbercrunch.de\/blog\/?p=152"},"modified":"2015-08-27T15:58:06","modified_gmt":"2015-08-27T14:58:06","slug":"book-review-programming-massively-parallel-processors-a-hands-on-approach","status":"publish","type":"post","link":"https:\/\/www.numbercrunch.de\/blog\/2010\/08\/book-review-programming-massively-parallel-processors-a-hands-on-approach\/","title":{"rendered":"Book review: Programming Massively Parallel Processors: A Hands-on Approach"},"content":{"rendered":"<p style=\"text-align: justify;\">Morgan Kaufmann Publishers have published one of the very first books on Nvidia&#8217;s popular Compute Unified Device Architecture (<a href=\"http:\/\/en.wikipedia.org\/wiki\/CUDA\">CUDA<\/a>) parallel computing technology, <em><a href=\"http:\/\/www.elsevierdirect.com\/morgan_kaufmann\/kirk\/index.html\">Programming Massively Parallel Processors: A Hands-on Approach<\/a><\/em>, written by two experts in the field, David B. Kirk and Wen-mei W. Hwu. Kirk was Chief Scientist at Nvidia and he is currently an Nvidia Fellow. Hwu is principle investigator for the first Nvidia <a href=\"http:\/\/research.nvidia.com\/content\/uiuc-ccoe-summary\">CUDA Center of  Excellence<\/a> at the University of Illinois at Urbana-Champaign.<\/p>\n<p style=\"text-align: justify;\">Chapers 1 and 2 give an introduction to graphics processing units (GPUs) as massive parallel computers and present a short history of GPU computing. In the book&#8217;s main part, chapers 3 to 6, the authors present a hands-on introduction to CUDA by developing a parallel matrix multiplication routine. Starting from a naive parallel implementation of the textbook matrix multiplication formula, the authors show how to improve its performance by taking into account the hardware characteristics of modern GPUs. The book describes CUDA thread organization, memory hierarchies, memory bandwidth, data prefetching and other techniques that may be necessary to achieve maximal GPU performance. Chapter 7 is a general chapter on floating point arithmetic and chapters 8 and 9 present two real wold case studies of CUDA applications. The book finishes with a general chapter on parallel programming and a short introduction to <a href=\"http:\/\/en.wikipedia.org\/wiki\/Opencl\">OpenCL<\/a>.<\/p>\n<p style=\"text-align: justify;\">The book is very clearly written and well organized. It should be accessible to everybody who is familiar with the C programming language, previous knowledge in parallel programming is not required but may be useful. In fact, I felt a little bit bored sometimes when reading the book because some very basic things are explained very broadly, for example in the chapter on floating point arithmetic.\u00a0 Nevertheless, I think the book is an valuable introduction to parallel programming in the CUDA environment.<\/p>\n<p style=\"text-align: justify;\">The book&#8217;s main merit is that it gives a good understanding of the Tesla and Fermi (in chapter 12) GPU hardware architecture which is essential for writing high-performance CUDA codes. The book also gives an introduction to the CUDA runtime API, but here it scratches only the surface. Thus, in order to write complex CUDA applications or applications that utilize the CUDA driver API one also has to read the original CUDA API documentation by Nvidia. However, the book&#8217;s introduction to the CUDA runtime API is sufficient to write useful CUDA applications. The book by Kirk and Hwu and the original CUDA documentation by Nvidia are complementary to each other. This distinguishes Kirk&#8217;s and Hwu&#8217;s work from the vast majority of the computing and programming books which merely rephrase API documentations and man-pages.<\/p>\n<p style=\"text-align: justify;\">In summary, <em>Programming Massively Parallel Processors<\/em> is a must-have for every CUDA programmer.<\/p>\n<p style=\"text-align: justify;\">\n","protected":false},"excerpt":{"rendered":"<p>Morgan Kaufmann Publishers have published one of the very first books on Nvidia&#8217;s popular Compute Unified Device Architecture (CUDA) parallel computing technology, Programming Massively Parallel Processors: A Hands-on Approach, written by two experts in the field, David B. Kirk and Wen-mei W. Hwu. Kirk was Chief Scientist at Nvidia and he is currently an Nvidia&hellip; <a href=\"https:\/\/www.numbercrunch.de\/blog\/2010\/08\/book-review-programming-massively-parallel-processors-a-hands-on-approach\/\" class=\"more-link\">Continue reading <span class=\"screen-reader-text\">Book review: Programming Massively Parallel Processors: A Hands-on Approach<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[12],"tags":[],"class_list":["post-152","post","type-post","status-publish","format-standard","hentry","category-book-reviews"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v25.6 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Book review: Programming Massively Parallel Processors: A Hands-on Approach - Number Crunch<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.numbercrunch.de\/blog\/2010\/08\/book-review-programming-massively-parallel-processors-a-hands-on-approach\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Book review: Programming Massively Parallel Processors: A Hands-on Approach - Number Crunch\" \/>\n<meta property=\"og:description\" content=\"Morgan Kaufmann Publishers have published one of the very first books on Nvidia&#8217;s popular Compute Unified Device Architecture (CUDA) parallel computing technology, Programming Massively Parallel Processors: A Hands-on Approach, written by two experts in the field, David B. Kirk and Wen-mei W. Hwu. Kirk was Chief Scientist at Nvidia and he is currently an Nvidia&hellip; Continue reading Book review: Programming Massively Parallel Processors: A Hands-on Approach\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.numbercrunch.de\/blog\/2010\/08\/book-review-programming-massively-parallel-processors-a-hands-on-approach\/\" \/>\n<meta property=\"og:site_name\" content=\"Number Crunch\" \/>\n<meta property=\"article:published_time\" content=\"2010-08-01T11:36:11+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2015-08-27T14:58:06+00:00\" \/>\n<meta name=\"author\" content=\"Heiko Bauke\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Heiko Bauke\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.numbercrunch.de\/blog\/2010\/08\/book-review-programming-massively-parallel-processors-a-hands-on-approach\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.numbercrunch.de\/blog\/2010\/08\/book-review-programming-massively-parallel-processors-a-hands-on-approach\/\"},\"author\":{\"name\":\"Heiko Bauke\",\"@id\":\"https:\/\/www.numbercrunch.de\/blog\/#\/schema\/person\/e73eab65b1721dd0c3d408edb887e413\"},\"headline\":\"Book review: Programming Massively Parallel Processors: A Hands-on Approach\",\"datePublished\":\"2010-08-01T11:36:11+00:00\",\"dateModified\":\"2015-08-27T14:58:06+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.numbercrunch.de\/blog\/2010\/08\/book-review-programming-massively-parallel-processors-a-hands-on-approach\/\"},\"wordCount\":452,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/www.numbercrunch.de\/blog\/#\/schema\/person\/e73eab65b1721dd0c3d408edb887e413\"},\"articleSection\":[\"Book reviews\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/www.numbercrunch.de\/blog\/2010\/08\/book-review-programming-massively-parallel-processors-a-hands-on-approach\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.numbercrunch.de\/blog\/2010\/08\/book-review-programming-massively-parallel-processors-a-hands-on-approach\/\",\"url\":\"https:\/\/www.numbercrunch.de\/blog\/2010\/08\/book-review-programming-massively-parallel-processors-a-hands-on-approach\/\",\"name\":\"Book review: Programming Massively Parallel Processors: A Hands-on Approach - Number Crunch\",\"isPartOf\":{\"@id\":\"https:\/\/www.numbercrunch.de\/blog\/#website\"},\"datePublished\":\"2010-08-01T11:36:11+00:00\",\"dateModified\":\"2015-08-27T14:58:06+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/www.numbercrunch.de\/blog\/2010\/08\/book-review-programming-massively-parallel-processors-a-hands-on-approach\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.numbercrunch.de\/blog\/2010\/08\/book-review-programming-massively-parallel-processors-a-hands-on-approach\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.numbercrunch.de\/blog\/2010\/08\/book-review-programming-massively-parallel-processors-a-hands-on-approach\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.numbercrunch.de\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Book review: Programming Massively Parallel Processors: A Hands-on Approach\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.numbercrunch.de\/blog\/#website\",\"url\":\"https:\/\/www.numbercrunch.de\/blog\/\",\"name\":\"Number Crunch\",\"description\":\"A computational science blog.\",\"publisher\":{\"@id\":\"https:\/\/www.numbercrunch.de\/blog\/#\/schema\/person\/e73eab65b1721dd0c3d408edb887e413\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.numbercrunch.de\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":[\"Person\",\"Organization\"],\"@id\":\"https:\/\/www.numbercrunch.de\/blog\/#\/schema\/person\/e73eab65b1721dd0c3d408edb887e413\",\"name\":\"Heiko Bauke\",\"logo\":{\"@id\":\"https:\/\/www.numbercrunch.de\/blog\/#\/schema\/person\/image\/\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Book review: Programming Massively Parallel Processors: A Hands-on Approach - Number Crunch","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.numbercrunch.de\/blog\/2010\/08\/book-review-programming-massively-parallel-processors-a-hands-on-approach\/","og_locale":"en_US","og_type":"article","og_title":"Book review: Programming Massively Parallel Processors: A Hands-on Approach - Number Crunch","og_description":"Morgan Kaufmann Publishers have published one of the very first books on Nvidia&#8217;s popular Compute Unified Device Architecture (CUDA) parallel computing technology, Programming Massively Parallel Processors: A Hands-on Approach, written by two experts in the field, David B. Kirk and Wen-mei W. Hwu. Kirk was Chief Scientist at Nvidia and he is currently an Nvidia&hellip; Continue reading Book review: Programming Massively Parallel Processors: A Hands-on Approach","og_url":"https:\/\/www.numbercrunch.de\/blog\/2010\/08\/book-review-programming-massively-parallel-processors-a-hands-on-approach\/","og_site_name":"Number Crunch","article_published_time":"2010-08-01T11:36:11+00:00","article_modified_time":"2015-08-27T14:58:06+00:00","author":"Heiko Bauke","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Heiko Bauke","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.numbercrunch.de\/blog\/2010\/08\/book-review-programming-massively-parallel-processors-a-hands-on-approach\/#article","isPartOf":{"@id":"https:\/\/www.numbercrunch.de\/blog\/2010\/08\/book-review-programming-massively-parallel-processors-a-hands-on-approach\/"},"author":{"name":"Heiko Bauke","@id":"https:\/\/www.numbercrunch.de\/blog\/#\/schema\/person\/e73eab65b1721dd0c3d408edb887e413"},"headline":"Book review: Programming Massively Parallel Processors: A Hands-on Approach","datePublished":"2010-08-01T11:36:11+00:00","dateModified":"2015-08-27T14:58:06+00:00","mainEntityOfPage":{"@id":"https:\/\/www.numbercrunch.de\/blog\/2010\/08\/book-review-programming-massively-parallel-processors-a-hands-on-approach\/"},"wordCount":452,"commentCount":0,"publisher":{"@id":"https:\/\/www.numbercrunch.de\/blog\/#\/schema\/person\/e73eab65b1721dd0c3d408edb887e413"},"articleSection":["Book reviews"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.numbercrunch.de\/blog\/2010\/08\/book-review-programming-massively-parallel-processors-a-hands-on-approach\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.numbercrunch.de\/blog\/2010\/08\/book-review-programming-massively-parallel-processors-a-hands-on-approach\/","url":"https:\/\/www.numbercrunch.de\/blog\/2010\/08\/book-review-programming-massively-parallel-processors-a-hands-on-approach\/","name":"Book review: Programming Massively Parallel Processors: A Hands-on Approach - Number Crunch","isPartOf":{"@id":"https:\/\/www.numbercrunch.de\/blog\/#website"},"datePublished":"2010-08-01T11:36:11+00:00","dateModified":"2015-08-27T14:58:06+00:00","breadcrumb":{"@id":"https:\/\/www.numbercrunch.de\/blog\/2010\/08\/book-review-programming-massively-parallel-processors-a-hands-on-approach\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.numbercrunch.de\/blog\/2010\/08\/book-review-programming-massively-parallel-processors-a-hands-on-approach\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.numbercrunch.de\/blog\/2010\/08\/book-review-programming-massively-parallel-processors-a-hands-on-approach\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.numbercrunch.de\/blog\/"},{"@type":"ListItem","position":2,"name":"Book review: Programming Massively Parallel Processors: A Hands-on Approach"}]},{"@type":"WebSite","@id":"https:\/\/www.numbercrunch.de\/blog\/#website","url":"https:\/\/www.numbercrunch.de\/blog\/","name":"Number Crunch","description":"A computational science blog.","publisher":{"@id":"https:\/\/www.numbercrunch.de\/blog\/#\/schema\/person\/e73eab65b1721dd0c3d408edb887e413"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.numbercrunch.de\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":["Person","Organization"],"@id":"https:\/\/www.numbercrunch.de\/blog\/#\/schema\/person\/e73eab65b1721dd0c3d408edb887e413","name":"Heiko Bauke","logo":{"@id":"https:\/\/www.numbercrunch.de\/blog\/#\/schema\/person\/image\/"}}]}},"_links":{"self":[{"href":"https:\/\/www.numbercrunch.de\/blog\/wp-json\/wp\/v2\/posts\/152"}],"collection":[{"href":"https:\/\/www.numbercrunch.de\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.numbercrunch.de\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.numbercrunch.de\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.numbercrunch.de\/blog\/wp-json\/wp\/v2\/comments?post=152"}],"version-history":[{"count":35,"href":"https:\/\/www.numbercrunch.de\/blog\/wp-json\/wp\/v2\/posts\/152\/revisions"}],"predecessor-version":[{"id":683,"href":"https:\/\/www.numbercrunch.de\/blog\/wp-json\/wp\/v2\/posts\/152\/revisions\/683"}],"wp:attachment":[{"href":"https:\/\/www.numbercrunch.de\/blog\/wp-json\/wp\/v2\/media?parent=152"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.numbercrunch.de\/blog\/wp-json\/wp\/v2\/categories?post=152"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.numbercrunch.de\/blog\/wp-json\/wp\/v2\/tags?post=152"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}