BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:America/New_York
X-LIC-LOCATION:America/New_York
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20260422T143138Z
LOCATION:B302
DTSTART;TZID=America/New_York:20241117T144600
DTEND;TZID=America/New_York:20241117T151100
UID:submissions.supercomputing.org_SC24_sess744_ws_memo102@linklings.com
SUMMARY:PIMnast: Balanced Data Placement for GEMV Acceleration with Proces
 sing-In-Memory
DESCRIPTION:Mohamed Ibrahim, Mahzabeen Islam, and Shaizeen Aga (AMD, Inc.)
 \n\nWith unprecedented demand for GenAI inference, acceleration of primiti
 ves that dominate GenAI, such as GEMV, is receiving considerable attention
 . A challenge with GEMVs is the high memory-bandwidth this primitive deman
 ds. Multiple memory vendors have proposed commercially-viable PIM prototyp
 es that attain bandwidth boost over processor via augmenting memory banks 
 with compute capabilities and broadcasting same command to all banks. Whil
 e proposed PIM designs stand to accelerate GEMV, we observe that a key imp
 ediment to harness PIM acceleration is deducing optimal data-placement to 
 place the matrix in memory banks. To this end, we tease out factors that i
 mpact data-placement and propose PIMnast which, like a gymnast, balances t
 hese factors to identify data-placements that deliver GEMV acceleration. A
 cross a spectrum of GenAI models, PIMnast, along with additional orchestra
 tion knobs we identify, delivers up to 6.86x speedup for GEMVs (of the ava
 ilable 7x roofline speedup) leading to up to 5x speedup for per-token late
 ncies.\n\nTag: Data Movement and Memory, Emerging Technologies\n\nRegistra
 tion Category: Workshop Reg Pass\n\nSession Chairs: Ron Brightwell (Sandia
  National Laboratories), Maya Gokhale (Lawrence Livermore National Laborat
 ory (LLNL)), Kyle Hale (Oregon State University), and Ivy Peng (KTH Royal 
 Institute of Technology)\n\n
END:VEVENT
END:VCALENDAR