|Resources||Policies||Schedule||Projects||Accomodations||Diversity and Inclusion|
This course explores the vital new domain of Machine Learning (ML) for the arts. Though born out of computer science research, contemporary ML techniques are reimagined through creative application to diverse tasks such as style transfer, generative portraiture, music synthesis, and textual chatbots and agents. Through direct, hands-on experience with state of the art ML tools, students will develop their skills in this nascent area and form critical perspectives on the strengths and limitations of current approaches.
As ML permeates multiple aspects of culture, industry, and scholarship, it is essential both to train the next generation of ML-literate artists and engineers, and to equip them with critical tools to evaluate these new techniques. How do computational tools augment, complicate, or supercede human creative endeavor? What new approaches to artistic production are possible with the advent of affordable graphics hardware and ML software?
This project-based course will be conducted primarily in python using free, open-source machine learning and scientific computing toolkits, running on cloud-based educational computing resources. In addition to hands-on experience with ML techniques, students will become familiar with cloud-based workflows, jupyter notebooks, and kubernetes containers. Architectures and topics covered include Recurrent Neural Networks (RNNs), Convolutional Neural Networks (CNNs), LSTMs, Wavenets, Generative Adversarial Networks (GANs) and others. Students will be responsible both for technical implementation and creative value of course projects.
Prequisites: ECE16, ECE143, or equivalent course on Python.
Discussion: we will use slack for discussion https://join.slack.com/t/ucsd-ml-art/signup (join with your @ucsd.edu or @eng.ucsd.edu account)
Compute Resources: We will use datahub.ucsd.edu for our in class computing environment. If you did not attend the bootcamp you need to contact me to be approved for datahub access. I will base it off of our final enrollment.
Code: code examples are here: https://github.com/roberttwomey/ml-art-code
Canvas: this is where I will post your grades: https://canvas.ucsd.edu/courses/5343
Github Classroom: this is where you will submit your code: https://classroom.github.com/classrooms/49459697-ece188-fa2019
Work will be evaluated on the quality of concept, the degree of experimentation (both aesthetic and technical), and final realization (again, aesthetic and technical). I will share a rubric with the first project assignment.
This course will be a mix of group work and individual work depending on each assignment. I want you each to develop your own personal research interests, but also to pool your resources and talents to produce the best projects possible. Group work is encouraged, but not required.
We will have critique/group discussion for each of the projects this quarter.
Day 1: Course and Syllabus 9/29
git clone https://github.com/roberttwomey/ml-art-code/
Homework: Post something you are interested in (a project, paper, github link) to #shiny.
Day 2: Introduction to ML and the Arts 10/2
Lab 1: Make sure you can log in
Homework: Read Andrej Karpathy’s The Unreasonable Effectiveness of RNNs (2015) http://karpathy.github.io/2015/05/21/rnn-effectiveness/
Day 3: Generative Text 10/7
Day 4: Text part 2 10/9
Lab 2: GPT-2 Examples
Homework: Project proposal
Day 5: Chatbots 10/14
Day 6: Autoencoders, Embeddings, Sketch-RNN 10/16 Autoencoders, Embeddings, Sketch-RNN
Lab: Autoencoders and online Sketch-RNN demos
Homework: Project 1
Project 1 Due 10/20 11:59pm
Day 7: Project 1 discussion 10/21
Day 8: Intro to Generative Audio 10/23
Day 9: Generative Networks for Music 10/28
Day 10: Speech Generation 10/30
Lab: Hands-On with ML Speech
Homework: Project proposal
Day 11: Visual Processing 11/4
Day 12: Style Transfer 11/6
Day 13: VETERANS DAY NO CLASS 11/11
Project 2 Due: 11/12, 11:59pm, through github classroom.
Day 14: Critique: Project 2 11/13
Day 15: Deep Dream and Gradient Ascent 11/18
Assign Project 3: Generative Visual
Day 16: GANs 11/20
Homework: Project 3 proposal
Day 17: Image Captioning and Segmentation 11/25
Day 18: Platforms 11/27
Project 3 due: 12/1, 11:59pm.
Day 19: Project 3 Critique 12/2
Day 20: Creativity Metric Activity, Final Project Check-in 12/4
FINAL TIME: Wednesday December 11, 8-11am. Location TBD.
Generative Text Assignment. Due 10/20/2019, 11:59pm.
Submit online to github classroom: https://classroom.github.com/g/sJIzmAcR
Generative Audio Assignment. Due 11/12/2019, 11:59pm.
Submit online to github classroom: https://classroom.github.com/g/ujfzX5Wp
Generative Visual Assignment. Due 12/1/2019, 11:59pm.
Submit online to github classroom: https://classroom.github.com/g/AMOrRaOj
Refine, enhance, extend one of your earlier projects for the showcase during Finals Week.
PROJECT DUE 12/11, 8-11am. Location TBD.
REPORT DUE 12/13, 11:59pm. Add the pdf to your github, please.
For the final project you will need to submit two things:
We will have a couple of opportunities to interact with a similar class running this Spring at Carnegie Mellon University, as well as making a joint, online, public-facing exhibition for excellent student work (opt-in). More info coming soon!
We do our processing on datahub.ucsd.edu. Here is their instruction manual:
The Office for Students with Disabilities (OSD), an Academic Affairs department, is responsible for the review of medical documentation and the determination of reasonable accommodations based on a disability. Authorization for Accommodation (AFA) letters are issued by the OSD and given to undergraduate, graduate, and Professional School students directly. If you have an AFA letter, meet with the CSE Student Affairs representative, and schedule an appointment with your instructor by the end of Week 2 to ensure that reasonable accommodations for the quarter can be arranged.
We are committed to fostering a learning environment for this course that supports a diversity of thoughts, perspectives and experiences, and respects your identities (including race, ethnicity, heritage, gender, sex, class, sexuality, religion, ability, age, educational background, etc.). Our goal is to create a diverse and inclusive learning environment where all students feel comfortable and can thrive.
Our instructional staff will make a concerted effort to be welcoming and inclusive to the wide diversity of students in this course. If there is a way we can make you feel more included please let one of the course staff know, either in person, via email/discussion board, or even in a note under the door. Our learning about diverse perspectives and identities is an ongoing process, and we welcome your perspectives and input.
We also expect that you, as a student in this course, will honor and respect your classmates, abiding by the UCSD Principles of Community https://ucsd.edu/about/principles.html. Please understand that others’ backgrounds, perspectives and experiences may be different than your own, and help us to build an environment where everyone is respected and feels comfortable.If you experience any sort of harassment or discrimination, please contact the instructor as soon as possible. If you prefer to speak with someone outside of the course, please contact the Office of Prevention of Harassment and Discrimination: https://ophd.ucsd.edu/