CS 264 Blog

2011-09-02T14:56:00.001-04:00

Lecture 1 is online

Click Here to watch!

Uploading your Project Videos

2011-05-09T11:47:00.000-04:00

It appears that the Harvard's iSites Dropbox is full (argnnnn!%#$!) so
you won't be able to upload your video.

Please upload it to youtube and send us your link by email at
staff+video@cs264.org with subject:
"Video: TITLE OF YOUR PROJECT - STUDENT NAMES".

Thank you.

Regards,

Nicolas

Project Week: Final Schedule (May 8th - May 13th)

2011-05-05T00:18:00.000-04:00

Dear cs264 gang,

The project deadlines are approaching really fast, I hope that you
have made good progress on your projects and, most importantly, that
you have learned a lot while working on them. You will find below more
detailed information about next week's schedule.

First, please verify that your project is properly listed at:
http://www.cs264.org/projects/projects/list.php
If not, please contact us at staff@cs264.org asap.

Here is the final schedule:

- Sunday May 8th 11pm:
DUE: Project (Final) Code + Video (2 minutes)
NO extension will be allowed (final grades are due on May 15th)
Instructions at:
http://www.cs264.org/projects/projects.html
http://www.cs264.org/projects/projects/presentations.html

- Tuesday May 10th 7:35-9:35pm (MD 125)
Students: Project Presentations Session #1
Presence is _mandatory_

- Thursday May 12th 7:35-9:35pm (MD 125)
Students: Project Presentations Session #2
Presence is _mandatory_

- Friday May 13th 11pm:
DUE: Project Webpage Report
NO extension will be allowed (final grades are due on May 15th)
Instructions at: http://www.cs264.org/projects/projects.html

Due to logistic constraints, the final project prize (a brand new
Tesla C2070 donated by NVIDIA) will be based on votes from the staff
and will be announced after May 15th.

Good luck to everyone !

Sincerely,

Nicolas

Fwd: mrjob v0.2.5 released

2011-05-02T16:23:00.001-04:00

---------- Forwarded message ----------
From: Jimmy Retzlaff <jimmy@retzlaff.com>
Date: Sat, Apr 30, 2011 at 1:20 PM
Subject: mrjob v0.2.5 released
To: python-announce@python.org, python-list@python.org

What is mrjob?
-----------------------

mrjob is a Python package that helps you write and run Hadoop Streaming jobs.

mrjob fully supports Amazon's Elastic MapReduce (EMR) service, which
allows you to buy time on a Hadoop cluster on an hourly basis. It also
works with your own Hadoop cluster.

Some important features:

* Run jobs on EMR, your own Hadoop cluster, or locally (for testing).
* Write multi-step jobs (one map-reduce step feeds into the next)
* Duplicate your production environment inside Hadoop
* Upload your source tree and put it in your job's $PYTHONPATH
* Run make and other setup scripts
* Set environment variables (e.g. $TZ)
* Easily install python packages from tarballs (EMR only)
* Setup handled transparently by mrjob.conf config file
* Automatically interpret error logs from EMR
* SSH tunnel to hadoop job tracker on EMR
* Minimal setup
* To run on EMR, set $AWS_ACCESS_KEY_ID and $AWS_SECRET_ACCESS_KEY
* To run on your Hadoop cluster, install simplejson and make
sure $HADOOP_HOME is set.

More info:

* Install mrjob: python setup.py install
* Documentation: http://packages.python.org/mrjob/
* PyPI: http://pypi.python.org/pypi/mrjob
* Discussion: http://groups.google.com/group/mrjob
* Development is hosted at github: http://github.com/Yelp/mrjob

What's new?
-------------------

v0.2.5, 2011-04-29 -- Hadoop input and output formats
* Added hadoop_input/output_format options
* You can now specify a custom Hadoop streaming jar (hadoop_streaming_jar)
* extra args to hadoop now come before -mapper/-reducer on EMR, so
that e.g. -libjar will work (worked in hadoop mode since v0.2.2)
* hadoop mode now supports s3n:// URIs (Issue #53)
--
http://mail.python.org/mailman/listinfo/python-announce-list

Support the Python Software Foundation:
http://www.python.org/psf/donations/

--
Nicolas Pinto
http://web.mit.edu/pinto

TALK: CUDA Optimization: A Case Study of High-Performance Sorting (Duane Merrill from University of Virginia, Tuesday April 26th, 7:35 PM, Harvard MD G125)

2011-04-23T18:20:00.000-04:00

Title: CUDA Optimization: A Case Study of High-Performance Sorting
Speaker: Duane Merrill (University of Virginia)

Date: 4-26-2011
Time: 7:35 PM
Location: Harvard Maxwell Dworkin G125 (http://j.mp/eCgV66)

Harvard CS264 2011 Guest Lecture Series
"Massively Parallel Computing" Course
http://www.cs264.org
Host: Nicolas Pinto (Harvard, MIT)

Abstract:

In this presentation, we use our implementation for high performance
radix sorting as a case study for illustrating advanced design
patterns and idioms. These techniques have allowed us to demonstrate
Fermi sorting rates that exceed 1.0 billion 32-bit keys per second
(and over 770 million key-value pairs per second), making it the
fastest fully-programmable micro-architecture for this genre of
sorting problems. Although the CUDA programming model is elegantly
decoupled from any particular hardware configuration, we present
techniques for exploiting knowledge of the NVIDIA GPU machine model in
order to produce more efficient implementations. Our design patterns
enable the compiler to specialize a single program text for a variety
of architectures, resulting in target code that "fits" the underlying
hardware significantly better than more general approaches. In
particular, we discuss strategies for kernel fusion, warp-synchronous
programming, flexible granularity via meta-programming, algorithm
serialization, and data-movement tuning.

Speaker biography:

Duane Merrill is a Ph. D. candidate at the University of Virginia,
Department of Computer Science. His advisor is Professor Andrew
Grimshaw. His current research interests lay in parallel and
high-performance computing, specifically in regard to programming
models and algorithmic primitives for GPU architectures. His
dissertation work investigates efficient strategies and algorithms for
solving classes of fine-grained, parallel problems that are typically
thought to be poorly-suited for the bulk-synchronous GPU machine
model. Much of his prior academic work has involved concurrent
systems in one form or another, including grid and distributed
computing; virtual machines and hypervisor technologies; operating
systems and meta-systems; and security architecture and protocols.

--
Nicolas Pinto, PhD
http://web.mit.edu/pinto

Tomorrow: Project Clinic @ 53 Church St. #104

2011-04-20T15:34:00.000-04:00

Dear cs264 students,

Tomorrow we will have a project clinic at 53 Church St. #104 from
7:35pm to 9:35pm. Please bring any project-related
question/bug/discussion with you!

See you there.

Regards,

Nicolas

Cluster Compute/GPU Spot Instances Now Available!

2011-04-18T16:15:00.001-04:00

Hi Class,

Amazon has just announced support for Cluster Compute/GPU spot
instances which should dramatically lower your costs of using the HPC
instance types on EC2:

http://aws.amazon.com/about-aws/whats-new/2011/04/07/announcing-amazon-ec2-spot-integration-with-hpc-instances/

I've added support for the new Cluster Compute/GPU spot instances to
StarCluster. You will need to re-download and install the development
version of StarCluster in order to request spot HPC instances. At the
time of writing the spot price stats for the HPC instances are:

Cluster Compute Instances:
-------------------------------------
% starcluster spothistory cc1.4xlarge
StarCluster - (http://web.mit.edu/starcluster) (v. 0.9999)
Software Tools for Academics and Researchers (STAR)
Please submit bug reports to starcluster@mit.edu

>>> Current price: $0.53
>>> Max price: $1.01
>>> Average price: $0.57

Cluster GPU Instances:
-------------------------------
% starcluster spothistory cg1.4xlarge
StarCluster - (http://web.mit.edu/starcluster) (v. 0.9999)
Software Tools for Academics and Researchers (STAR)
Please submit bug reports to starcluster@mit.edu

>>> Current price: $0.78
>>> Max price: $0.78
>>> Average price: $0.74

As you can see the average spot price for the HPC instance types are
much lower than the flat-rate price (~$2.10/instance/hr). I hope you
will find this useful in making your credits last even longer during
your final projects.

~Justin

Fwd: CCS Seminar - Friday - April 22, 2011 - 12:00 - prb595 - Ron Dror (D.E. Shaw Research) (fwd)

2011-04-17T23:18:00.001-04:00

---------- Forwarded message ----------
From: Mike Clark <mikec@seas.harvard.edu>
Date: Sat, Apr 16, 2011 at 9:07 AM
Subject: Fwd: CCS Seminar - Friday - April 22, 2011 - 12:00 - prb595 -
Ron Dror (D.E. Shaw Research) (fwd)
To: scigpu@googlegroups.com

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
CCS Seminar
Friday - April 22, 2011
12:00
Physics Research Building, Boston University
Room 595
3 Cummington Street
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Ron O. Dror
D.E. Shaw Research

"Overcoming Communication Latency Barriers to Enable Massively
Parallel Biomolecular Dynamics Simulations on Anton"

Strong scaling of scientific applications on parallel architectures is
increasingly limited by communication latency. This talk will
describe the techniques used to reduce latency and mitigate its
effects on performance in Anton, a massively parallel special-purpose
machine that accelerates molecular dynamics (MD) simulations by orders
of magnitude compared with the previous state of the art. Achieving
this speedup required both specialized hardware mechanisms and a
restructuring of the application software to reduce network latency,
sender and receiver overhead, and synchronization costs. Anton
delivers end-to-end inter-node latency significantly lower than any
other large-scale parallel machine, and the total critical-path
communication time for an Anton MD simulation is less than 3% that of
the next-fastest MD platform. The resulting speedup allows Anton to
capture functionally important biological events that often occur on
timescales ranging from several microseconds to a millisecond,
including the "folding" of proteins into their native
three-dimensional shapes, the structural changes that underlie protein
function, and the interactions between a protein and a drug molecule.

Cheryl Endicott
Center for Computational Science
3 Cummington Street
Boston, MA 02215
tel: 617-358-1470
fax: 617-358-2487
http://ccs.bu.edu

--
Nicolas Pinto
http://web.mit.edu/pinto

TALK: The Onset of Parallelism, Changes in Computer Architecture and Microsoft's Role in the Transition (David Rich from Microsoft Research, Tuesday April 19th, 8:35 PM, Harvard MD G125)

2011-04-16T21:38:00.001-04:00

Title: The Onset of Parallelism, Changes in Computer Architecture and
Microsoft's Role in the Transition
Speaker: David Rich (Microsoft Research)

Date: 4-19-2011
Time: 8:35 PM
Location: Harvard Maxwell Dworkin G125 (http://j.mp/eCgV66)

Harvard CS264 2011 Guest Lecture Series
"Massively Parallel Computing" Course
http://www.cs264.org
Host: Nicolas Pinto (Harvard, MIT)

Abstract:

Computing performance continues to rise, but seemingly only at the
cost of increased complexity and programming challenge. Architectures
are changing and may become more diverse in the near future. This
talk will discuss the progression of computer architecture while
relating that progression to building construction. Microsoft's plans
for parallel programming and supporting frameworks will also be
discussed.

Speaker biography:

David joined Microsoft as part of the acquisition of Interactive
Supercomputing where he was VP of Marketing. At Microsoft his focus is
on developing strategy and business and is now focused on both GPU
computing and the manufacturing vertical. Previous to Microsoft and
ISC, he was director of marketing at AMD where he started and ran
AMD's HPC and high-end embedded businesses. He also served as
president of the Hypertransport Consortium. His experience includes
VP business development and VP marketing assignments at Fujitsu, and
startups API Networks and Dolphin Interconnect. He started and
managed the TotalView Parallel Debugger business and in the very early
days worked on BBN Butterfly parallel computers and Apollo
workstations. His only job writing code was Z80 assembler for an SNA
(3274) protocol converter. He has a degree in computer science from
Brown University.

--
Nicolas Pinto, PhD
http://web.mit.edu/pinto

TALK: Jacket - Visual Computing (James Malcolm from AccelerEyes, Tuesday April 19th, 7:35 PM, Harvard MD G125)

2011-04-16T21:34:00.001-04:00

Title: Jacket -- Visual Computing
Speaker: James Malcolm (AccelerEyes)

Date: 4-19-2011
Time: 7:35 PM
Location: Harvard Maxwell Dworkin G125 (http://j.mp/eCgV66)

Harvard CS264 2011 Guest Lecture Series
"Massively Parallel Computing" Course
http://www.cs264.org
Host: Nicolas Pinto (Harvard, MIT)

Abstract:

Jacket is a platform for data-parallel programming in MATLAB, Python,
and C/C++ using NVIDIA GPUs for acceleration. The platform includes
libraries for matrix and array manipulation, linear algebra, signal
and image processing, and visualization.

The first part of the talk will highlight many of these features
through code examples. The second part of the talk will detail some
of the design decisions in the underlying implementation.

Speaker biography:

James Malcolm is a developer at AccelerEyes. He has a PhD from
Georgia Tech in computer vision with a fellowship at Harvard Medical
in diffusion MRI imaging. More info: http://jgmalcolm.com

--
Nicolas Pinto, PhD
http://web.mit.edu/pinto

TALK: Enabling CPU/GPU Portability via Execution Model Translation Speaker (Gregory Diamos from Georgia Tech, Thursday April 14th, 7:35 PM, Harvard Hall 104)

2011-04-12T18:39:00.001-04:00

Title: Enabling CPU/GPU Portability via Execution Model Translation
Speaker: Gregory Diamos (Georgia Tech)

Date: 4-14-2011
Time: 7:35 PM
Location: Harvard Hall 104 (http://j.mp/icUlNw)

Harvard CS264 2011 Guest Lecture Series
"Massively Parallel Computing" Course
http://www.cs264.org
Host: Nicolas Pinto (Harvard, MIT)

Abstract:

The emergence of heterogeneous and many-core architectures has changed
the hardware/software interface. While architecture innovations have
traditionally been hidden under ISA abstractions that could be
efficiently mapped to hardware, new CPU/GPU processors expose multiple
cores with heterogeneous micro-architectures directly to programmers
and languages.

Bulk-synchronous and data-parallel execution model abstractions have
risen in response. They enable the same program representation to be
efficiently mapped to processors composed of CPU and GPU cores. This
talk focuses on these abstractions, which have the potential to
continue the exponential increase in computing potential made possible
by technology scaling via translation to CPU, GPU, and future core
architectures that have yet to be designed.

These abstractions and the translation processes that map them to
hardware are discussed in the context of Ocelot, an open-source
dynamic compiler targeting multi-core CPUs, NVIDIA GPUs, and AMD GPUs

Speaker biography:

Gregory Diamos is a PhD candidate in the Computer Architecture and
Systems Lab at the Georgia Institute of Technology, under the
direction of Professor Sudhakar Yalamanchili, as well as a Research
Scientist at NVIDIA. He received his B.S. and M.S. in Electrical
Engineering from the Georgia Institute of Technology in 2006 and 2008,
respectively, where he focused on architecture techniques for
controlling PVT variations and runtime scheduling techniques for
heterogeneous processors.
His current research interests follow the industry shift to
heterogeneous many core architectures, where mounting communication
requirements place increasing demands on processor memory systems and
the ability to tightly integrate heterogeneous cores on-chip offers
the potential for dramatic improvements in efficiency at the cost of
increased design complexity; his research is directed toward
developing abstractions that maintain this efficiency while reducing
design complexity.

--
Nicolas Pinto, PhD
http://web.mit.edu/pinto

Fwd: InPar 2011 - Call for Papers

2011-04-12T08:04:00.001-04:00

---------- Forwarded message ----------
From: Schuh, Andrew <aschuh@illinois.edu>
Date: Tue, Apr 12, 2011 at 6:29 AM
Subject: InPar 2011 - Call for Papers
To: "nicolas.pinto@gmail.com" <nicolas.pinto@gmail.com>

Innovative Parallel Computing
Foundations & Applications of GPU, Manycore, and Heterogeneous Systems (INPAR'11)

Call for participation

We are pleased to announce the 2011 Innovative Parallel Computing: Foundations & Applications of GPU, Manycore, and Heterogeneous Systems (InPar'11). This new conference provides a first-tier academic venue for peer-reviewed publications in the emerging fields of parallel computing, encompassing the topics of GPU computing, manycore computing, and heterogeneous computing.

InPar has dual focus on "Foundations"—the fundamental advances in parallel computing itself— and "Applications"—case studies and lessons learned from the application of commodity parallel computing in domains across science and engineering. The goal of InPar is to bring together researchers in the myriad fields being revolutionized by GPUs to share experiences, discover commonalities, and both inform and learn from the computer scientists working on the foundations of parallel computing.

Topics: InPar encourages papers involving current GPU/manycore architectures, new or emerging commodity parallel architectures (such as Intel "MIC" products), and hybrid or heterogeneous systems. Possible topics include, but are not limited to:

Foundations:		Applications:
	Programming systems Parallel programming models Languages, including domain-specific languages (DSLs) Compilers and runtime systems Operating system support for GPU/manycore processors and heterogeneity Approaches and tools for program analysis, profiling and debugging Rethinking algorithms and data structures for parallel computing Core algorithms and data structures: sorting, hash tables, etc. Graph algorithms Numerical algorithms and data structures Heterogeneous computing with a primary focus toward GPUs/manycore processors Compilation for heterogeneous applications Runtime support for heterogeneous applications Applications that effectively use heterogeneous resources Cross-platform solutions Languages and runtime systems (OpenCL, Ocelot, PyOpenCL, etc) Performance evaluation and comparisons Computer architecture for commodity parallel computing Parallel system simulators and predictive models Microarchitecture Memory system architecture System interconnect Fault tolerance and reliable computing		Computational physics and chemistry molecular physics, quantum chemistry condensed-matter physics, fluid dynamics and material science earth sciences, climate and weather modeling astrophysics and cosmology Life sciences & computational biology bioinformatics, protein folding neuroscience biomedical imaging Engineering simulation & design optimization electronic design automation product development Statistical modeling and computational finance random number generators, Monte Carlo data analysis and forecasting Data-intensive applications machine learning artificial intelligence real-time processing Computer vision fast graph algorithms visual saliency, recognition, object detection Video/image/audio and signal processing speech recognition photo-realistic rendering spectral analysis visual effects for film and video

Committee:

Dates:

General co-chairs:

Amitabh Varshney (University of Maryland)
David Luebke (NVIDIA Research)

Paper co-chairs:

Lorena Barba (Boston University)
John Owens (University of California Davis)

Program co-chairs:

John Stone (University of Illinois)
Hanspeter Pfister (Harvard University)

Steering Committee chair:

Wen-Mei Hwu (University of Illinois)

Treasurer:

Andrew Schuh (University of Illinois)

Abstract due:                                       June 10
Papers due:                                         June 17
Authors informed of decisions:           July 26
Camera-ready papers due:                August 16
Conference:                                        October 10-11

Proceedings: InPar is a highly selective, peer-reviewed, and archival publication venue. Due to the diverse, multi-disciplinary nature of this conference, some authors may be unfamiliar with publication traditions that are customary in computer science but uncommon in other fields. We are currently working towards ACM sponsorship of this conference. After approval, the papers will be published via the ACM International Conference Proceedings Series and will appear in the ACM Digital Library, will receive an ISBN number, etc. The program committee and tertiary reviewers will review, rank, and select only the best papers for presentation at the conference and publication in the proceedings. Authors will receive review reports, but the short publication cycle allows only minor revisions.

The conference website is available at www.innovativeparallel.org.

--
Nicolas Pinto
http://web.mit.edu/pinto

TALK: R-Stream - Programming GPUs without Writing a Line of CUDA (Nicolas Vasilache from Reservoir Labs, Tuesday April 12th, 7:35 PM, Harvard MD G125)

2011-04-10T21:46:00.001-04:00

Title: R-Stream - Programming GPUs without Writing a Line of CUDA
Speaker: Nicolas Vasilache (Reservoir Labs)

Date: 4-12-2011
Time: 7:35 PM
Location: Harvard Maxwell Dworkin G125 (http://j.mp/eCgV66)

Harvard CS264 2011 Guest Lecture Series
"Massively Parallel Computing" Course
http://www.cs264.org
Host: Nicolas Pinto (Harvard, MIT)

Abstract:

A new class of multicore processors and accelerators has emerged with
significant parallelism and on-chip distributed memory (e.g., GPGPU,
IBM's Cell, SCC) that requires explicit control and communication
among tiled computation kernels.

R-Stream is a "High Level Compiler" for embedded signal/knowledge
processing and HPC algorithms, and is designed to seamlessly generate
parallelized code to target-specific (low-level) C compilers.
Specifically, R-Stream is a source-to-source compiler that accepts a
sequential C program as input and produces code that has been
parallelized and optimized for these new types of processors.
R-Stream can output optimized code in a variety of formats for
downstream processors, including highly optimized OpenMP and CUDA.

R-Stream performs multiple advanced transformations from the input C
source code in order to achieve high performance. These include
special forms of array expansion (to remove constraints on
parallelism), joint scheduling for parallelism and locality, task
granularity selection, communications/DMA generation, software
pipelining, memory region reshaping, and back end dialect generation.
The resulting mapped program is much more than simply parallelized —
it represents a detailed choreography of computation and data motion
across parallel units and through explicitly managed memory
hierarchies.

Speaker biography:

Dr. Nicolas T. Vasilache, Senior Engineer, holds an Engineering Degree
from the ENST Télécom Paris and M.S. / PhD from the Université Paris
Sud in Orsay. His doctoral studies focused on high performance
compilation in the polyhedral model, and his dissertation addresses
fundamental theoretical problems in polyhedral compilation. Prior to
joining Reservoir, he worked as a technical co-op specialist for the
IBM TJ Watson Research Center where he successfully implemented his
ideas in the XLC compiler. He in an inventor of 4 US patents filed on
behalf of IBM. Since joining Reservoir Labs, Dr. Vasilache has worked
on Alef, Reservoir's distributed satisfiability solver for one year.
He is one of the main developers of Reservoir's R-Stream compiler and
focuses on various aspect of high-level code optimization in a
polyhedral intermediate representation. Dr. Vasilache is also
Reservoir's main contributor to the UHPC effort in the team lead by
Intel. He is in an inventor of 4 US patents filed on behalf of
Reservoir Labs.

--
Nicolas Pinto, PhD
http://web.mit.edu/pinto

TALK: Irregular Parallelism on the GPU: Algorithms and Data Structures (John Owens from UC Davis, Thursday April 7th, 7:35 PM, Harvard Hall 104)

2011-04-04T23:47:00.000-04:00

Title: Irregular Parallelism on the GPU: Algorithms and Data Structures

Speaker: John Owens (UC Davis, Electrical and Computer Engineering)

Date: 4-7-2011
Time: 7:35 PM
Location: Harvard Hall 104 (http://j.mp/icUlNw)

Harvard CS264 2011 Guest Lecture Series
"Massively Parallel Computing" Course
http://www.cs264.org
Host: Nicolas Pinto (Harvard, MIT)

Abstract:

The computational power of GPUs, coupled with advances in its
programmability, is making the GPU an increasingly compelling platform
for high-performance computing. In this talk I'll discuss some of our
recent research results in supporting *irregular parallelism* on the
GPU. While GPUs are particularly good at regular, structured codes,
it's still a large challenge to efficiently support more irregular
data structures and algorithms. I'll talk about our recent work in
task queuing, hash tables, fragment compositing, and multi-GPU
MapReduce, and also touch on some of the research problems I'd like to
address going forward.

Speaker biography:

John Owens is an associate professor of electrical and computer
engineering at the University of California, Davis, where he leads a
research group with a focus on parallel computing, concentrating on
fundamental primitives and applications on the GPU. He is also one of
the lead developers of the popular CUDPP GPU computing library. At UC
Davis, he earned the Department of Energy Early Career Principal
Investigator Award (2004), an NVIDIA Teaching Fellowship (2006), and
his department's Graduate Student Association Award for Graduate
Teaching and Mentorship (2009). He is a PI in both the DOE's SciDAC
Institute for Ultrascale Visualization and in the new Intel Science
and Technology Center for Visual Computing. John earned his Ph.D. in
electrical engineering in 2003 from Stanford University and his B.S.
in electrical engineering and computer sciences in 1995 from the
University of California, Berkeley.

--
Nicolas Pinto, PhD
http://web.mit.edu/pinto

TALK: Analysis-Driven Performance Optimization with CUDA (Cliff Woolley from NVIDIA, Tuesday April 5h, 8:35 PM, Harvard MD G125)

2011-04-03T19:21:00.000-04:00

Title: Analysis-Driven Performance Optimization with CUDA
Speaker: Cliff Woolley (CUDA Developer Technology Engineer, NVIDIA Corporation)

Date: 4-5-2011
Time: 8:35 PM
Location: Harvard Maxwell Dworkin G125 (http://j.mp/eCgV66)

Harvard CS264 2011 Guest Lecture Series
"Massively Parallel Computing" Course
http://www.cs264.org
Host: Nicolas Pinto (Harvard, MIT)

Abstract:

The goal of this talk is to demystify performance optimization by
transforming it into an analysis-driven process. There are three
fundamental limiters to kernel performance: instruction throughput,
memory throughput, and latency. In this talk we will describe how to
use profiling tools and source code instrumentation to assess the
significance of each limiter, what optimizations to apply for each
limiter, and how to determine when hardware limits are reached.

Notes: It is assumed that attendees are already familiar with
fundamental optimization techniques. This talk was originally
prepared and delivered by my colleague, Paulius Micikevicius, for
GTC2010.

Speaker biography:

Cliff Woolley is a CUDA Developer Technology Engineer with NVIDIA
Corporation. He received his Master's degree in Computer Science from
the University of Virginia in 2003. He was among the earliest
academic researchers to investigate the use of graphics processors for
general purpose computation, having applied these early GPGPU ideas
both to non-traditional graphics rendering techniques as well as to
non-graphical algorithms such as a multigrid solver for PDEs.

--
Nicolas Pinto, PhD
http://web.mit.edu/pinto

TALK: Programming the Memory Hierarchy with Sequoia (Mike Bauer from Stanford, Tuesday April 5h, 7:35 PM, Harvard MD G125)

2011-04-03T19:11:00.000-04:00

Title: Programming the Memory Hierarchy with Sequoia
Speaker: Mike Bauer (Stanford)

Date: 4-5-2011
Time: 7:35 PM
Location: Harvard Maxwell Dworkin G125 (http://j.mp/eCgV66)

Harvard CS264 2011 Guest Lecture Series
"Massively Parallel Computing" Course
http://www.cs264.org
Host: Nicolas Pinto (Harvard, MIT)

Abstract:

Due to power concerns modern computer architectures have turned to
parallelism to make use of the additional transistors provided by
Moore's law. Each new computer architecture will contain an order of
magnitude more computational units than the previous generation. With
sufficient on-chip computational resources all but the most
compute-intensive applications will become bound by memory and
communication bandwidth. In order to ensure efficient execution of
memory- and communication-bound programs, parallel programming
languages must be locality-aware to effectively utilize the complex
memory hierarchies of new architectures. This talk will describe
Sequoia, a locality-aware parallel programming language for
programming deep memory hierarchies in a portable manner. An
implementation of the Sequoia compiler and runtime systems will be
covered as well as case studies using Sequoia to program a variety of
memory hierarchies.

Speaker biography:

Michael Bauer is a third-year PhD student in the Department of
Computer Science at Stanford University and an NVIDIA Graduate Student
Fellow.
--
Nicolas Pinto, PhD
http://web.mit.edu/pinto

HW2 Answers

2011-03-31T12:00:00.001-04:00

Please see this post for link to HW2 answer guide: http://forum.cs264.org/index.php?topic=365.0

... Dwight

Office Hours 4/1

2011-03-31T10:14:00.000-04:00

I'll be holding office hours on 4/1 from 6-8 in 53 Church St.

-Jud

TALK: cl.oquence: High-Level Language Abstractions for Low-Level Programming (Cyrus Omar from CMU, Thursday March 31st, 2011, 8:35 PM, Harvard Hall 104)

2011-03-30T23:29:00.000-04:00

Title: cl.oquence: High-Level Language Abstractions for Low-Level Programming
Speaker: Cyrus Omar (CMU)

Date: 3-31-2011
Time: 8:35 PM
Location: Harvard Hall 104 **Room change**

Harvard CS264 2011 Guest Lecture Series
"Massively Parallel Computing" Course (http://www.cs264.org)
Host: Nicolas Pinto (Harvard, MIT)

Abstract:

When performance is important, developers turn to "low-level"
languages like C, FORTRAN, CUDA and OpenCL. These languages handle
issues related to register and stack allocation for their users, but
do not otherwise attempt to manage memory allocation or control data
movement. Developers continue to favor this design, despite the
increased development burden, because small changes to an algorithm's
data movement patterns can dramatically impact overall performance on
modern hardware, including GPUs, but compilers for high-level
languages are not yet able to reliably outperform motivated human
experts. Indeed, the most prominent productivity-oriented languages in
computational science -- MATLAB and Python -- come with significant
performance overhead.

In this talk, I will demonstrate that many of the
productivity-enhancing features characteristic of high-level languages
can be reconciled with a low-level programming model with no
performance penalty, using techniques like type inference, structural
polymorphism and compile-time metaprogramming. The resulting language,
called cl.oquence, compiles directly into OpenCL source code. Because
OpenCL is low-level, supports a variety of architectures and can be
called from several languages, cl.oquence functions can be integrated
into existing projects easily. For Python users, special bindings have
been developed, based on pyopencl, that allow users to write
cl.oquence functions inline and call them directly, without even the
minimal syntactic overhead that a more generic binding would require.

Although the language remains in beta, I have been using it to run
large-scale neurobiological circuit simulations. An informal case
study of this simulation framework will be presented. The language is
free, open-source and well-documented. It will be available at
http://cl.oquence.org/ shortly. Interested people can email me
(cyrus@cmu.edu).

Speaker biography:

Cyrus Omar is a graduate student in the Computer Science Department at
Carnegie Mellon University. His research focuses on building better
programming languages, compilers, development environments and data
sharing infrastructure for computational science and high-performance
computing, with a particular focus on computational neurobiology.
http://www.cs.cmu.edu/~comar

--
Nicolas Pinto, PhD
http://web.mit.edu/pinto

TALK: Easy, Effective, Efficient: GPU Programming in Python with PyOpenCL and PyCUDA (Andreas Kloeckner, Thursday March 31st, 2011, 7:35 PM, Harvard MD G125)

2011-03-30T09:10:00.000-04:00

Title: Easy, Effective, Efficient: GPU Programming in Python with
PyOpenCL and PyCUDA
Speaker: Andreas Kloeckner (NYU)

Date: 3-31-2011
Time: 7:35 PM
Location: Harvard Hall 104 **Room change**

Harvard CS264 2011 Guest Lecture Series
"Massively Parallel Computing" Course (http://www.cs264.org)
Host: Nicolas Pinto (Harvard, MIT)

Abstract:

High-level scripting languages are in many ways polar opposites to
GPUs. GPUs are highly parallel, subject to hardware subtleties, and
designed for maximum throughput, and they offer a tremendous advance
in the performance achievable for a significant number of
computational problems. On the other hand, scripting languages such as
Python favor ease of use over computational speed and do not generally
emphasize parallelism. PyOpenCL and PyCUDA are two packages that
attempt to join the two together. By showing concrete examples, both
at the toy and the whole-application level, this talk aims to
demonstrate that by combining these opposites, a programming
environment is created that is greater than just the sum of its two
parts.

Speaker biography:

Andreas Klöckner obtained his PhD degree working with Jan Hesthaven at
the Department of Applied Mathematics at Brown University. He worked
on a variety of topics all aiming to broaden the utility of
discontinuous Galerkin (DG) methods. This included their use in the
simulation of plasma physics and the demonstration of their particular
suitability for computation on throughput-oriented graphics processors
(GPUs). He also worked on multi-rate time stepping methods and shock
capturing schemes for DG.

In the fall of 2010, he joined the Courant Institute of Mathematical
Sciences at New York University as a Courant Instructor. There, he is
working on problems in computational electromagnetics with Leslie
Greengard.

His research interests include:

- Discontinuous Galerkin and integral equation methods for wave
propagation

- Programming tools for parallel architectures

- High-order unstructured particle-in-cell methods for plasma simulation

--
Nicolas Pinto, PhD
http://web.mit.edu/pinto

Posted: Homework 4 (due Friday April 8th, 2011 11PM EST)

2011-03-25T22:17:00.000-04:00

Hey gang,

Homework 4 has been posted:
http://www.cs264.org/homeworks/homeworks.html

Let us know how it goes.

Good luck

Reminder: Project Proposals & HW3 due tomorrow

2011-03-24T16:27:00.000-04:00

Hey gang,

Please don't forget that your project proposal & HW3 are due tomorrow.
Note that no further extension will be given (the schedule is getting
tight).

Thanks.

Best,

Hadoop/EC2 lab tomorrow

2011-03-23T15:20:00.000-04:00

Hi everyone,

We'll be covering the basics of hadoop and EC2 in the lab tomorrow. In order to simplify getting the tools and environment setup, we'll be using the Cloudera development VM. This is an ubuntu virtual machine image that comes preinstalled with hadoop and supporting tools. It can be found at http://www.cloudera.com/downloads/virtual-machine/. To get the most out of the lab, please download and install this VM image on your laptop before coming to lab.

To run this VM image, you'll need either VMware Player (http://downloads.vmware.com/d/info/desktop_downloads/vmware_player/3_0) or VMware Fusion on a Mac. VMware player is available for free. You can get a 30 day free trial of VMware fusion if you don't already have a license.

Thanks,

Jud

HW3 Relief

2011-03-23T13:20:00.001-04:00

Two items of good news on HW3:

1) The due date for HW3 is now Friday, 3/25 at 11:00 pm.

2) The timing values are being revised due to slow performance observed on resonance. The new timing targets for part D are:

• 2 processes 10,000 ms (unchanged)
• 4 processes 4,200 ms
• 6 processes 3,000 ms

... Dwight

CS264 AWS Free Credits

2011-03-18T17:11:00.000-04:00

Dear gang,

The AWS codes have been sent by email.

Please let us know if you did not receive yours.

Regards,

--
Nicolas Pinto
http://web.mit.edu/pinto

Posted: Homework 3 (due Wednesday March 23rd, 2011 11PM EST)

2011-03-14T09:15:00.000-04:00

Hey gang,

Homework 3 has been posted:
http://www.cs264.org/homeworks/homeworks/hw3.php

Let us know how it goes.

Good luck

Regards,

--
Nicolas Pinto, PhD
http://web.mit.edu/pinto

CS264 Projects

2011-03-05T18:35:00.000-05:00

Hey gang,

Information about the projects has been posted on the website and
project ideas from research groups will be posted on the forum.

For more details, see:
http://www.cs264.org/projects/projects.html
http://forum.cs264.org/index.php?board=6.0

Please submit your proposal as soon as possible so that you can start
working on it sooner.

Best,

--
Nicolas Pinto
http://web.mit.edu/pinto

HW1 Grades

2011-03-01T11:57:00.000-05:00

HW1 grades have been sent out. If you did not receive your grade, or have any questions, please let us know.

HW2 due date update and Lab 4+5 recordings

2011-02-26T17:54:00.000-05:00

Lab 4 recording is available at http://continuinged.na5.acrobat.com/p10438725/

Lab 5 recording is available at http://continuinged.na5.acrobat.com/p87132471/

HW2 is now due on Monday, 3/14 at 11:00pm.

Enjoy!

Homework 0 Solution Posted

2011-02-23T19:42:00.001-05:00

The solution to the C programming questions in Homework 0 have been
posted to the Homework Solutions board:

http://forum.cs264.org/index.php?topic=195.msg467#msg467

NOTE: You must be logged into the forums to access the solution.

Those of you that turned in Homework 0 should have received an email
containing your grade. Please have a look at the solutions and make
sure you understand them.

If you have any questions or concerns please send an email to staff
'at' cs264 'dot' org or post to the homework forum board.

Posted: Homework 2 (due Fri Mar 11th, 2011 11PM EST)

2011-02-20T17:04:00.001-05:00

Hey gang,

Homework 2 has been posted:
http://www.cs264.org/homeworks/homeworks/hw2.php

Let us know how it goes.

Good luck!

Best,

--
Nicolas Pinto
http://web.mit.edu/pinto

Lab 3 Video

2011-02-12T01:27:00.001-05:00

The Lab 3 video has been posted:
http://continuinged.na5.acrobat.com/p48543334/

(see "Lectures" section on cs264.org)

Thanks Dwight for navigating the painful Adobe Connect maze...

Enjoy!

Office Hours for 02/11/2011

2011-02-10T23:05:00.000-05:00

Office hours will be help on Friday, Feb. 11 from 10am-12pm and from 5:30pm-7pm in Maxwell Dworkin 307.

Thanks,

Jud

Fwd: Save the date for GPU Technology Conference 2011

2011-02-10T15:08:00.001-05:00

---------- Forwarded message ----------
From: GPU Technology Conference <gtc@nvidia.com>
Date: Thu, Feb 10, 2011 at 11:33 AM
Subject: Save the date for GPU Technology Conference 2011
To: Pinto Nicolas <pinto@mit.edu>

Save the date for GPU Technology Conference 2011

We're excited to announce another unrivaled week of innovation, learning and networking. Mark your calendar.

GTC 2011 will take place October 11-14, 2011 at the McEnery Convention Center in San Jose, California.

Whether you're a returning alumni or planning to come for the very first time, you'll find the best and the brightest from all over the world demonstrating cutting-edge work across a wide variety of fields. GTC highlights will include:

Practical code development tools and techniques
Exposure to a spectrum of impactful science and engineering driven by GPUs
A unique opportunity to network with peers and establish new connections

We're always looking for talented minds in academia and industry. Please consider submitting a proposal to present. Visit GTC 2010 content archives for highlights of last year's conference sessions. Call for Submissions will open in March. Watch your inbox for submission details.

GPUs are changing the science of computing, as well as mission critical practical applications! Get ready for the GPU event of the year!

Spread the word!

Copyright © 2011 NVIDIA, the NVIDIA logo, Quadro, the Quadro logo, and CUDA, are registered trademarks and/or trademarks of NVIDIA Corporation in the United States and other countries. All other company and product names are trademarks or registered trademarks of the respective owners with which they are associated.

Please note that this message was sent to the following email address: (pinto@mit.edu)
If you would like to no longer receive email from NVIDIA, click here to unsubscribe.

NVIDIA Corporation, 2701 San Tomas Expressway, Santa Clara, CA 95050

--
Nicolas Pinto, PhD
http://web.mit.edu/pinto

Fwd: Exclusive Webinars for CUDA Developers

2011-02-10T12:50:00.001-05:00

---------- Forwarded message ----------
From: NVIDIA <news@nvidia.com>
Date: Thu, Feb 10, 2011 at 12:46 PM
Subject: Exclusive Webinars for Developers
To: Pinto Nicolas <pinto@mit.edu>

Dear Developer,

You are receiving this email since you registered as a GPU Computing Developer.

One of the privileges of this membership is the opportunity to engage with technical experts from NVIDIA and partners. We are also introducing new Coffee Time webinars, where you can get answers to your questions directly from the experts.

You can find about more information about the webinars and the presenters by following the links below.

Register now for these exclusive 2011 webinars (many more coming!).
Feb 11th, 9am PST CUDA 3.2 Library Performance Results Review
Feb 16th,10am PST Overview & Demo of CUDA-gdb
March 2nd,10am PST Overview & Demo of CUDA-memcheck
March 23rd,9am PST Parallel Nsight-A Technical Overview
April 13th,9am PST Coffee with NVIDIAs Satish Salian, CUDA gdb & memcheck

If you have friends who would like to attend please suggest they register as a developer.

If you haven't been to the secure developer portal recently then please login here: https://nvdeveloper.nvidia.com/login.asp
There are links to reset your password – just incase!

Another exclusive benefit of the registered developer program is early access to new revisions of the CUDA Toolkit, SDK and drivers.

We look forward to re-engaging with you as we schedule more webinars over the next few months.

Best regards,
NVIDIA GPU Computing Developer Program Team

Please note that this message was sent to the following email address: (pinto@mit.edu)
If you would like to stop receiving emails from NVIDIA, Click here to unsubscribe

--
Nicolas Pinto, PhD
http://web.mit.edu/pinto

Posted: Lecture 3 slides, Homework 1 (due Fri Feb 18th, 2011 11PM EST)

2011-02-09T12:39:00.000-05:00

Hey gang,

You'll find the slides of yesterday's lecture here:
http://www.cs264.org/lectures/lectures.html

And homework 1 has been posted:
http://www.cs264.org/homeworks/homeworks.html

Good luck!

Best,

--
Nicolas Pinto
http://web.mit.edu/pinto

Updated Office Hours for 02/04/2011

2011-02-04T11:10:00.001-05:00

If anyone needs to meet with me today during office hours I will be at
53 Church Street from 5:00pm-7:00pm in the Mac cluster.

~Justin

Posted: slides for Lecture 1 and 2.

2011-02-03T22:58:00.001-05:00

Hey gang,

The slides for the first two lectures have been posted at
http://www.cs264.org/lectures/lectures.html

Lecture 2 has a lot of material but don't worry, you don't have to
know it all (just need to be aware of it)!

I would advice you to focus primarily on the "Thinking Parallel"
section and the parts that you have never seen before (e.g
"Architecture"). The "Patterns" section is more for reference (it will
become handy very soon...).

Let us know if you have any question regarding the material.

HTH

Cheers,

--
Nicolas Pinto, PhD
http://web.mit.edu/pinto

gpu-login changes

2011-02-02T17:49:00.001-05:00

Folks,

Due to the issues we were seeing with the normal gpu-login, we've
changed the way gpu-login works. Now you need not do agent forwarding
... once you are on the cluster, gpu-login should work without any
credentials.

IRCS will close all tickets related to this, so let us know if
problems persist after 6 pm this evening (Wednesday).

Thanks for your patience,

Rob and the IRCS Team

--
Robert E. Parrott, Ph.D. (Phys. '06)
Director, Instructional and Research Computing Services
Harvard University Sch. of Eng. and App. Sci.
Maxwell-Dworkin 211,
33 Oxford St.
Cambridge, MA 02138
(617)-496-1520

Tonight's lecture has been cancelled due to snow storm (02/01/2011)

2011-02-01T13:41:00.000-05:00

Hi Class,

Due to the snow storm the Harvard Extension School has cancelled all
classes on 02/01 and 02/02.

Please note that even though we don't have class tonight we will still
have lab on Thursday at 53 Church St.

We will send out another announcement once we've posted the lecture
slides for tonight's cancelled class. You should read over them before
coming to lab on Thursday.

~Justin

Resonance GPU cluster downtime on 02/01/2011

2011-01-31T18:43:00.001-05:00

SEAS will be moving the resonance cluster starting tomorrow February
1st. The cluster should be up by early afternoon tomorrow (sometime
between 12pm-2pm). We will post another announcement as soon as the
cluster is back online. Sorry for any inconvenience.

Updated Office Hours Location for 01/28/2011

2011-01-28T13:48:00.001-05:00

I will now be at 53 Church Street Lab from 3pm-5pm for anyone needing
to meet during my Office Hours.

~Justin

HW0 posted (DUE Feb. 4th)

2011-01-28T13:44:00.001-05:00

http://www.cs264.org/homeworks/homeworks.html

Lecture Posted

2011-01-26T16:45:00.001-05:00

The video for yesterday's introductory lecture has been posted.

Fwd: BBN Distributed Computing is Hiring

2010-11-22T09:00:00.001-05:00

Begin forwarded message:

From: Kurt Rohloff <krohloff@bbn.com>
Date: November 21, 2010 11:52:58 PM EST
To: Hanspeter Pfister <pfister@seas.harvard.edu>
Subject: BBN Distributed Computing is Hiring

Hanspeter-

I hope all is well with you. I wanted to send you a quick note to say that our distributed systems group at BBN is hiring R&D software engineers in our Cambridge office. We're looking for strong students at the MS or PHD level with hands-on interests in high-performance computing, real-time systems, QoS management, distributed resource allocation and information assurance. I'm also looking for promising folks with experience in cloud computing, GPUs, FPGAs and other scientific computing paradigms.

We need folks who get their hands dirty with programming and developing working R&D prototypes. Interest or demonstrated ability to publish is a big plus.

*REQUIRED: Ability & willingness to obtain a security clearance for which the DoD requires US citizenship.*

If you know of anyone good, please have them drop me a line and/or fill out the appropriate application below.

http://careers.bbn.com/servlet/av/jd?ai=715&ji=2484696&sn=I
http://careers.bbn.com/servlet/av/jd?ai=715&ji=2484700&sn=I

Thanks!

Kurt

CS 264 Blog

Uploading your Project Videos

Project Week: Final Schedule (May 8th - May 13th)

Fwd: mrjob v0.2.5 released

TALK: CUDA Optimization: A Case Study of High-Performance Sorting (Duane Merrill from University of Virginia, Tuesday April 26th, 7:35 PM, Harvard MD G125)

Tomorrow: Project Clinic @ 53 Church St. #104

Cluster Compute/GPU Spot Instances Now Available!

Fwd: CCS Seminar - Friday - April 22, 2011 - 12:00 - prb595 - Ron Dror (D.E. Shaw Research) (fwd)

TALK: The Onset of Parallelism, Changes in Computer Architecture and Microsoft's Role in the Transition (David Rich from Microsoft Research, Tuesday April 19th, 8:35 PM, Harvard MD G125)

TALK: Jacket - Visual Computing (James Malcolm from AccelerEyes, Tuesday April 19th, 7:35 PM, Harvard MD G125)

TALK: Enabling CPU/GPU Portability via Execution Model Translation Speaker (Gregory Diamos from Georgia Tech, Thursday April 14th, 7:35 PM, Harvard Hall 104)

Fwd: InPar 2011 - Call for Papers

TALK: R-Stream - Programming GPUs without Writing a Line of CUDA (Nicolas Vasilache from Reservoir Labs, Tuesday April 12th, 7:35 PM, Harvard MD G125)

TALK: Irregular Parallelism on the GPU: Algorithms and Data Structures (John Owens from UC Davis, Thursday April 7th, 7:35 PM, Harvard Hall 104)

TALK: Analysis-Driven Performance Optimization with CUDA (Cliff Woolley from NVIDIA, Tuesday April 5h, 8:35 PM, Harvard MD G125)

TALK: Programming the Memory Hierarchy with Sequoia (Mike Bauer from Stanford, Tuesday April 5h, 7:35 PM, Harvard MD G125)

HW2 Answers

Office Hours 4/1

TALK: cl.oquence: High-Level Language Abstractions for Low-Level Programming (Cyrus Omar from CMU, Thursday March 31st, 2011, 8:35 PM, Harvard Hall 104)

TALK: Easy, Effective, Efficient: GPU Programming in Python with PyOpenCL and PyCUDA (Andreas Kloeckner, Thursday March 31st, 2011, 7:35 PM, Harvard MD G125)

Posted: Homework 4 (due Friday April 8th, 2011 11PM EST)

Reminder: Project Proposals & HW3 due **tomorrow**

Hadoop/EC2 lab tomorrow

HW3 Relief

CS264 AWS Free Credits

Posted: Homework 3 (due Wednesday March 23rd, 2011 11PM EST)

CS264 Projects

HW1 Grades

HW2 due date update and Lab 4+5 recordings

Homework 0 Solution Posted

Posted: Homework 2 (due Fri Mar 11th, 2011 11PM EST)

Lab 3 Video

Office Hours for 02/11/2011

Fwd: Save the date for GPU Technology Conference 2011

Fwd: Exclusive Webinars for CUDA Developers

Posted: Lecture 3 slides, Homework 1 (due Fri Feb 18th, 2011 11PM EST)

Updated Office Hours for 02/04/2011

Posted: slides for Lecture 1 and 2.

gpu-login changes

Tonight's lecture has been cancelled due to snow storm (02/01/2011)

Resonance GPU cluster downtime on 02/01/2011

Updated Office Hours Location for 01/28/2011

HW0 posted (DUE Feb. 4th)

Lecture Posted

Fwd: BBN Distributed Computing is Hiring

Reminder: Project Proposals & HW3 due tomorrow