Logo UAB

Advanced Machine Learning

Code: 106575 ECTS Credits: 6
2024/2025
Degree Type Year
2504392 Artificial Intelligence OT 3
2504392 Artificial Intelligence OT 4

Contact

Name:
Andrey Barsky
Email:
Desconegut

Teaching groups languages

You can view this information at the end of this document.


Prerequisites

Recommended "Fundamentals of Machine Learning" and "Neural Networks and Deep Learning"


Objectives and Contextualisation

This subject aims to give a detailed overview of techniques that enhance and optimise neural network models beyond the implementation of task-specific architectures. Building upon the foundation of previous subjects, this course focuses on meta-objectives that transfer, extend, or analyse the operation of existing models. 

Students will push their existing understanding of the mechanics of deep learning and address a range of practical challenges in deploying and scaling neural network models in realistic settings. They will learn to train models in resource-constrained settings; probe the behaviour of models to understand their failure cases; learn to subvert models as an adversary, and design models that are secure against subversion. 

By the end of this subject, students should have developed skills and expertise to think about secondary objectives in the development of deep models, and be able to reason about the behaviour of neural networks at a high level. They will also have practical experience with these advanced deep learning techniques, equipping them with the knowledge to create systems that are more adaptable, efficient, and robust. 


Competences

    Artificial Intelligence
  • Analyse and solve problems effectively, generating innovative and creative proposals to achieve objectives.
  • Develop critical thinking to analyse alternatives and proposals, both one's own and those of others, in a well-founded and argued manner.
  • Develop strategies to formulate and solve different learning problems in a scientific, creative, critical and systematic way, knowing the capabilities and limitations of the different existing methods and tools.
  • Introduce changes to methods and processes in the field of knowledge in order to provide innovative responses to society's needs and demands.
  • Know, understand, use and apply appropriately the mathematical foundations necessary to develop systems for reasoning, learning and data manipulation.
  • Students must be capable of collecting and interpreting relevant data (usually within their area of study) in order to make statements that reflect social, scientific or ethical relevant issues.

Learning Outcomes

  1. Adapt and reuse existing models in different domains.
  2. Analyse and solve problems effectively, generating innovative and creative proposals to achieve objectives.
  3. Critically analyse different time series models and use them to analyse and model sequential data and time series.
  4. Develop critical thinking to analyse alternatives and proposals, both one's own and those of others, in a well-founded and argued manner.
  5. Identify and model statistical distributions.
  6. Propose new methods or informed alternative solutions.
  7. Students must be capable of collecting and interpreting relevant data (usually within their area of study) in order to make statements that reflect social, scientific or ethical relevant issues.
  8. Understand different machine learning paradigms, and decide on the most appropriate learning method according to the characteristics of the data to be analysed.
  9. Weigh up the risks and opportunities of both your own and others' proposals for improvement.

Content

Efficient & adaptive learning 

  • Transfer learning 

  • Distillation 

  • Semi-supervised learning 

  • Meta-learning 

  • Online learning 

Responsible & trustworthy AI 

  • Explainability & interpretability 

  • Adversarial robustness 

  • Security & privacy 

Advanced architectures & paradigms 

  • Graph neural networks 

  • Neural ordinary differential equations 

  • Diffusion models 


Activities and Methodology

Title Hours ECTS Learning Outcomes
Type: Directed      
Laboratoy sessions 21 0.84 1, 2, 3, 4, 6, 9
Theory classes 24 0.96 5, 7, 8
Type: Supervised      
Work in projects/exercises 42 1.68 1, 2, 4, 6, 9
Type: Autonomous      
Individual study 20 0.8 2, 5, 7, 8
Work in projects/exercises 30 1.2 1, 2, 4, 6, 9

Neural network design is guided by the types of problems that it aims to solve. Throughout this subject it will be that typology of problems that will provide the motivation of each section and will direct the organization of the contents. 

There will be two types of sessions: 

Theory classes: The objective of these sessions is for the teacher to explain the theoretical background of the subject. For each one of the topics studied, the theory and mathematical formulation is explained, as well as the corresponding algorithmic solutions. 

Laboratory sessions: Laboratory sessions aim to facilitate interaction, collaborative work and to reinforce the comprehension of the topics seen in the theory classes. During laboratory sessions the students will work through practical cases that require the design of solutions using the methods studied in theory classes. Problem solving will be initiated in the class and will be complemented by a weekly set of problems to work through at home. 

The above activities will be complemented by a system of tutoring and consultations outside class hours. 

Course Project: A course project will be carried out during the semester, where students will have to solve a specific problem of certain complexity. The students will work in small groups of 2-3 students, where each member of the group must equally contribute to the final solution. These working groups will be maintaineduntil the end of the semester and must be self-managed in terms of distribution of roles, work planning, assignment of tasks, management of available resources, conflicts, etc. To develop the project, the groups will work autonomously, while some of the laboratory sessions will be used (1) for the teacher to present the projects’ theme and discuss possible approaches, (2) for monitoring the status of the project and (3) for the teams to present their final results. 

  

All the information of the subject and the related documents that the students need will be available at the virtual campus (cv.uab.cat). 

Annotation: Within the schedule set by the centre or degree programme, 15 minutes of one class will be reserved for students to evaluate their lecturers and their courses or modules through questionnaires.


Assessment

Continous Assessment Activities

Title Weighting Hours ECTS Learning Outcomes
Delivery of exercises 10% 5 0.2 1, 2, 3, 4
Delivery of the project 40% 4 0.16 1, 2, 3, 4, 6, 9
Written exams 50% 4 0.16 5, 7, 8

To assess the level of student learning, a formula is established that combines theoretical and practical knowledge acquisition, and the ability to solve problems. 

Final Grade 

The final grade is calculated weighted in the following way and according to the different activities that are carried out: 

Final grade = 0.5 * Theory Grade + 0.1 * Problems Portfolio Grade + 0.4 * Project Grade 

This formula will be applied as long as the theory and the practical evaluation grades are higher than 5. There is no restriction on the problems portfolio grade. If doing the calculation of the formula yields >= 5 but the theory grade or practical evaluation grade does not reach the minimum required, then a final grade of 4.5 will be given. 

Theory Grade 

The theory grade aims to assess the individual abilities of the student in terms of the theoretical content of the subject, this is done continuously during the course through two partial exams. The overall theory grade is the average of the grades of the two partial exams. 

The mid-term exam (Exam 1) is done in the middle of the semester and serves to eliminate part of the subject if it is passed. The final exam (Exam 2) is done at the end of the semester and serves to eliminate the rest of subject if it is passed. 

These exams aim to assess the abilities of each student in an individualized manner, both in terms of solving problems using the techniques explained in class, as well as evaluating the level of conceptualization that the student has made of the techniques seen. In order to obtain a final pass theory grade, it will be required for the both partial exam grades to be higher than 4. If doing the calculation of the formula yields >= 5 but the grades of any of the two partial exams do not reach the minimum required, then the grade for theory that will be used in the final calculation will be 4.5. 

Recovery exam. In case the theory grade does not reach the adequate level to pass, the students can take a recovery exam, destined to recover the failed part (1, 2 or both) of the continuous evaluation process. 

Problems Portfolio Grade 

The aim of the problems (exercises) is for the student to train with the contents of the subject continuously and become familiar with the application of the theoretical concepts. As evidence of this work, the presentation of a portfolio is requested in which the exercises worked out will be collated. To obtain a problems grade it is required that the student submits a minimum of 70% of the problem sets, in the opposite case, the problems grade will be 0. 

Project Grade 

The course project carries an essential weight in the overall mark of the subject. Developing the project requires that the students work collaboratively and design an integral solution to the defined challenge. In addition, the students must demonstrate their teamwork skills and present the results to the class. 

The project is evaluated through its deliverables, an oral presentation that students will make in class, and a self-evaluation process. The participation of students in all three activities (preparing the deliverable, presentation and auto evaluation) is necessary in order to obtain a projects grade. The grade is calculated as follows: 

Project Grade = 0.6 * Grade Deliverables + 0.3 * Grade Presentation + 0.1 * Grade Self-evaluation 

If performing the above calculation yields >= 5 but the student did not participate in any of the activities (deliverable, presentation, auto evaluation), then a final grade of 4.5 will be given to the course project. 

In case the minimum grade is not achieved, the group will be able to recover the project, restricted to a maximum grade of 7/10. 

In case of not submitting the project deliverables, not presenting the project, or considering the contents copied or synthetically generated, the course project will be considered failed. 

Important notes 

Notwithstanding other disciplinary measures deemed appropriate, and in accordance with the academic regulations in force, evaluation activities will be suspended with zero (0) whenever a student commits any academic irregularities that may alter such evaluation (for example, plagiarizing, copying, letting copy, ...). The evaluation activities qualified in this way and by this procedure will not be recoverable. If you need to pass any of these assessment activities to pass the subject, this subject will be failed directly, without opportunity to recover it in the same course. 

In case there the student does not deliver any exercise solutions, does not participate in any practical evaluation, and does not take any exam, the corresponding grade will be a "non-evaluable". In any other case, the “no shows” count as a 0 for the calculation of the weighted average. 

In order to pass the course with honours, the final grade obtained must be equal or higher than 9 points. Because the number of students with this distinction cannot exceed 5% of the total number of students enrolled in the course, it is given to whoever has the highest final marks. In case of a tie, the results of the partial exams will be taken into account. 

More details about the evaluation process will be given at class during the first weeks of the semester. In case of any discrepancy between this guide and the information given at class, the information provided at class takes precedence. 


Bibliography

Libros / llibres: 

  • Deep Learning, Ian Goodfellow, Yoshua Bengio, and Aaron Courville, MIT Press, 1st Ed. 2016 

  • Deep learning with Python, François Chollet, Manning Publications, 2nd Ed., 2022 

  • Pattern Recognition and Machine Learning, Christopher Bishop, Springer, 2011 

libros online / llibres en línia: 

  • Zhang, Z.C. Lipton, M. Li, A.J. Smola, “Dive into Deep Learning”, 2021 


Software

Per a les activitats pràctiques del curs utilitzarem Python (NumPy, MatPlotLib, SciKit Learn) i PyTorch 


Language list

Name Group Language Semester Turn
(PAUL) Classroom practices 1 English second semester morning-mixed
(TE) Theory 1 English second semester morning-mixed