BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:America/New_York
X-LIC-LOCATION:America/New_York
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20250626T233527Z
LOCATION:B302-B305
DTSTART;TZID=America/New_York:20241120T100000
DTEND;TZID=America/New_York:20241120T170000
UID:submissions.supercomputing.org_SC24_sess533_post269@linklings.com
SUMMARY:Profiling and Bottleneck Identification for Large Language Model O
 ptimizations
DESCRIPTION:Alvin Hoang and Brian Chen (University of California, Riversid
 e; Pacific Northwest National Laboratory (PNNL))\n\nLarge language models 
 (LLMs) have shown they can perform scientific tasks. They are capable of a
 ssisting researchers in data interpretation, instrument operation, knowled
 ge synthesis, and hypothesis generation. However, LLMs must first be train
 ed on a large dataset of scientific tasks and data. Training these models 
 requires a substantial amount of time, energy, and computational resources
 , as the process of altering a model’s parameters through each iteration i
 s expensive. Researchers have developed optimizations that can speed up th
 e process of training LLMs with new data. In our research, we aim to profi
 le LLMs with optimizations during the steps of fine-tuning to identify bot
 tlenecks or improvements in runtime. Some of the optimizations we utilized
  include Low-Rank Adaptation (LoRA), BitFit, and Adapter. From our visual 
 diagrams and runtime charts, we can gain a better understanding of their p
 erformance and profile breakdown during training and fine-tuning.\n\nRegis
 tration Category: Tech Program Reg Pass, Exhibits Reg Pass\n\nSession Chai
 rs: Ayesha Afzal (Friedrich-Alexander University, Erlangen-Nuremberg; Erla
 ngen National High Performance Computing Center); Sally Ellingson (Univers
 ity of Kentucky); and Alan Sussman (University of Maryland)\n\n
END:VEVENT
END:VCALENDAR