This page contains an archive of the past five years of Directed Research Groups led by Professor Craft. View his currently offered DRGs »
Spring 2024
Interpreting Robustness in Large Language Models
Instructors: Dr. Brock Craft, Adam Hyland, Anna Shang
Meeting Time: 1 - 2:20 p.m. Mondays, Spring Quarter
Meeting Location: Sieg 420
Credits: 2
This course is an intensive two credit small group seminar studying interpretability and adversarial robustness in large language models (LLMs). Interpretability is a line of research trying to understand the cause of a Machine Learning (ML) model’s decision. What do LLMs know and understand? How do LLMs perceive reality? And if we were to study this, how can we probe and hack the models to construct its “cognition”? Adversarial robustness is the ability of a model to return correct or safe responses even in the presence of deliberate attacks. LLMs are susceptible to these attacks–something as simple as suggesting a chatbot act as a helpful grandma can be enough to get it to divulge private data or produce instructions to make a chemical weapon.
These two concepts are often explored as separate research domains, but models must be interpretable while being robust to a sophisticated interpreter trying to extract private information or induce undesirable behavior. Not only that, they are as much a matter for user research and design as they are for machine learning and statistics; learning by looking at a model (interpretability) and learning by poking one (robustness) are both fundamentally human interfaces.
In this DRG we will invite students to engage with these topics through readings, discussions, and play. Programming or machine learning experience is not expected but students should be able to demonstrate evidence of interest in the topic area. While we do not require students to produce a self-led or group project in this course, we invite students to propose projects in the topic area of interpretable AI, robustness in foundation models, or both.
Winter 2024
Prompt Surfing: Riding the Generative Waves of Text and Image Creation
Instructors: Dr. Brock Craft, Adam Hyland, Anna Shang
Meeting Time: 1:00 p.m. - 2:30 p.m. Mondays, Winter Quarter, Sieg 429
Capacity: 10 students
Credits: 2
This DRG focuses on generative AI for text and image creation, covering key topics such as prompt engineering, synthetic images, and text generation. The discussions extend beyond ChatGPT to explore other large language models and various systems for image generation, even exploring the possibility of running generative AI on an mobile phone. The seminar also addresses interpretability methods and includes hands-on activities with fine-tuned models. Attendees will explore how these AI systems integrate into other frameworks, such as feeding generated text into systems like Whisper or DALL-E. Additionally, the seminar investigates the intriguing concept of an LLM introspecting its own output and the impact of chain-of-thought prompting. Throughout, the focus is on understanding how these models shape the ensemble of possibilities available to us in the realm of generative AI.
[This description was generated by ChatGPT.]
Instructors' note: The goal of the DRG is not to be "dazzled" by these technologies but rather to problematise them, to understand how these platforms work, how they can be influenced. The only way to do so is to engage with them actively and discuss them critically from a human-centered perspective, which is what we will be doing.
Student Requirements:
- Strong interest in creative uses of generative AI
- Strong interest in researching how to manipulate LLMs
- Self directed and curious. (We will not hold your hand in this DRG.)
- Programming skills in Python a huge plus, but not a requirement
Questions? Email bcraft@uw.edu
Spring 2023
Human Centered Design in London Preparatory Seminar
This bi-weekly seminar will prepare students for the design work that they will undertake during the Study Abroad Program. We will prepare to explore the cultural and design context of London and discuss readings on design strategies and methods. Practical considerations for the program will also be covered.
This course meets bi-weekly, for 2 hours. This DRG is exclusive to and required for participants in the HCDE in London Study Abroad program.
Spring 2023
Climate Migration Design Workshops
This DRG is a week-long experiment connecting UCD and transition design practices. Inspired by a local project exploring assisted climate migration for the Western Red Cedar, we will explore the connections between creative practice and human and nonhuman worlds. As members of this DRG you will be responsible for the continuity of ideas over the week. In other words, you will be the “designer glue,” shepherding initial ideas to realization.
The DRG will be run by Drs. Tyler Fox, Brock Craft, and John Fass (from the MAUX program at the London College of Communication), and PhD Candidate Michael Beach.