If you intend to enroll for Fall 2019, please fill out this questionnare: https://forms.gle/iHiggRiVbPUsWMm46, and enroll for the class through the EASy system.
|Details||Resources||Policies||Code||Schedule||Projects||CMU Collaboration||Accomodations||Diversity and Inclusion|
This course explores the vital new domain of Machine Learning (ML) for the arts. Though born out of computer science research, contemporary ML techniques are reimagined through creative application to diverse tasks such as style transfer, generative portraiture, music synthesis, and textual chatbots and agents. Through direct, hands-on experience with state of the art ML tools, students will develop their skills in this nascent area and form critical perspectives on the strengths and limitations of current approaches.
As ML permeates multiple aspects of culture, industry, and scholarship, it is essential both to train the next generation of ML-literate artists and engineers, and to equip them with critical tools to evaluate these new techniques. How do computational tools augment, complicate, or supercede human creative endeavor? What new approaches to artistic production are possible with the advent of affordable graphics hardware and ML software?
This project-based course will be conducted primarily in python using free, open-source machine learning and scientific computing toolkits, running on cloud-based educational computing resources. In addition to hands-on experience with ML techniques, students will become familiar with cloud-based workflows, jupyter notebooks, and kubernetes containers. Architectures and topics covered include Recurrent Neural Networks (RNNs), Convolutional Neural Networks (CNNs), LSTMs, Wavenets, Generative Adversarial Networks (GANs) and others. Students will be responsible both for technical implementation and creative value of course projects.
Prequisites: ECE16, ECE143, or equivalent course on Python. Students need to attend the bootcamp 3/21 to enroll in the course.
Discussion: we will use slack for discussion https://ucsd-ml-art.slack.com/signup (join with your @ucsd account)
Compute Resources: We will use datahub.ucsd.edu for our in class computing environment. If you did not attend the bootcamp you need to contact me to be approved for datahub access. I will base it off of our final enrollment.
Work will be evaluated on the quality of concept, the degree of experimentation (both aesthetic and technical), and final realization (again, aesthetic and technical). I will share a rubric with the first project assignment.
This course will be a mix of group work and individual work depending on each assignment. I want you each to develop your own personal research interests, but also to pool your resources and talents to produce the best projects possible.
We will have critique/group discussion for each of the projects this quarter.
Code examples are here: https://github.com/roberttwomey/ml-art-code
We do our processing on datahub.ucsd.edu. Here is their instruction manual:
Day 1: Course and Syllabus 4/1/2019
Homework: Sign up for slack and post something you are interested in (a project, paper, github link) to #shiny.
Day 2: Introduction to ML and the Arts 4/3/2019
Lab 1: Get set up in our environment 4/5/2019
Day 3: Generative Text 4/8/2019
Day 4: Text part 2 4/10/2019
Lab 2: Project 1 Work Time
Day 5: Autoencoders, Embeddings, Sketch-RNN 4/15/2019
Day 6: Hands-On with VAE, Sketch-RNN 4/17/2019
git clone https://github.com/roberttwomey/ml-art-class
Project 1 Due 4/18/2019, 11:59pm.
Lab 3: Project 1 discussion 4/19/2019
Day 7: Handwriting 4/22/2019
Day 8: Generative Audio 4/24/2019
Lab 3: Project 2 Work 4/26/2019
Day 9: Generative Networks for Music 4/29/2019
Day 10: class cancelled 5/1/2019
Project 2 Due: 5/2/2019, 11:59pm. Submit online to github classroom: https://classroom.github.com/a/gP_-KCrL
Lab 4: Project 2 Discussion 5/3/2019
Day 11: Speech Generation 5/6/2019
Day 12: Visual Processing 5/8/2019
Day 13: Work Day 5/13/2019
Day 14: Style Transfer, Continued 5/15/2019
Project 3 Due: 5/16/2019 at 11:59pm.
Submit online to github classroom: https://classroom.github.com/a/JPtQMEm9
Day 15: Deep Dream and Gradient Ascent 5/20/2019
Assign Project 4: Generative Visual
Day 16: GANs 5/22/2019
Day 17: MEMORIAL DAY 5/27/2019
Day 18: Recognition 5/29/2019
Project 4 due 5/31/2019 at 12:30pm
Day 19: Platforms 6/3/2019
Day 20: Work time, Q&A 6/5/2019
FINAL TIME: Wednesday June 12, 11:30am-2:30pm. JKW Auditorium in CMRR.
PROJECT DUE: 11:30am, Wednesday June 12.
REPORT DUE: 11:59pm, Friday June 12 (add to your github repository as pdf)
Generative Text Assignment, due 4/18/2019, 11:59pm.
Submit online to github classroom: https://classroom.github.com/a/F_S6X9eN
Sketching with ML Assignment, due 5/2/2019, 11:59pm.
Submit online to github classroom: https://classroom.github.com/a/gP_-KCrL
Generative Audio Assignment, due 5/16/2019, 11:59pm.
Submit online to github classroom: https://classroom.github.com/a/JPtQMEm9
Due 5/31/2019, 12:30pm. Open ended.
Submit online to github classroom: https://classroom.github.com/g/tngoWMjM
Refine, enhance, extend one of your earlier projects for the showcase during Finals Week.
PROJECT DUE 6/12, 11:30am-2:30pm. JKW Auditorium in CMRR.
REPORT DUE 6/14, 11:59pm. Add the pdf to your github, please.
For the final project you will need to submit two things:
We will have a couple of opportunities to interact with a similar class running this Spring at Carnegie Mellon University, as well as making a joint, online, public-facing exhibition for excellent student work (opt-in). More info coming soon!
The Office for Students with Disabilities (OSD), an Academic Affairs department, is responsible for the review of medical documentation and the determination of reasonable accommodations based on a disability. Authorization for Accommodation (AFA) letters are issued by the OSD and given to undergraduate, graduate, and Professional School students directly. If you have an AFA letter, meet with the CSE Student Affairs representative, and schedule an appointment with your instructor by the end of Week 2 to ensure that reasonable accommodations for the quarter can be arranged.
We are committed to fostering a learning environment for this course that supports a diversity of thoughts, perspectives and experiences, and respects your identities (including race, ethnicity, heritage, gender, sex, class, sexuality, religion, ability, age, educational background, etc.). Our goal is to create a diverse and inclusive learning environment where all students feel comfortable and can thrive.
Our instructional staff will make a concerted effort to be welcoming and inclusive to the wide diversity of students in this course. If there is a way we can make you feel more included please let one of the course staff know, either in person, via email/discussion board, or even in a note under the door. Our learning about diverse perspectives and identities is an ongoing process, and we welcome your perspectives and input.
We also expect that you, as a student in this course, will honor and respect your classmates, abiding by the UCSD Principles of Community https://ucsd.edu/about/principles.html. Please understand that others’ backgrounds, perspectives and experiences may be different than your own, and help us to build an environment where everyone is respected and feels comfortable.If you experience any sort of harassment or discrimination, please contact the instructor as soon as possible. If you prefer to speak with someone outside of the course, please contact the Office of Prevention of Harassment and Discrimination: https://ophd.ucsd.edu/