Neural Networks and Deep Learning
Code: 104407 ECTS Credits: 6| Degree | Type | Year | Semester |
|---|---|---|---|
| 2503740 Computational Mathematics and Data Analytics | OB | 3 | 2 |
Contact
- Name:
- Oriol Ramos Terrades
- Email:
- oriol.ramos@uab.cat
Teaching groups languages
You can check it through this link. To consult the language you will need to enter the CODE of the subject. Please note that this information is provisional until 30 November 2023.
Teachers
- Carlos Boned Riera
Prerequisites
To have completed the subjects of Materia 7: Artificial Intelligence, Machine Learning, and the subjects of Modelling and Inference (2nd), Complex Data Analysis (2nd), and Information Theory (3rd).
Objectives and Contextualisation
This subject aims to give a practical introduction to neural network models and deep learning.
The students will consolidate and extend their theoretical background, by building on top of previous subjects on machine learning and complementing previous knowledge with new concepts on neural network design, deep learning frameworks, and the training process for such models.
The students should finish this subject, having a broad knowledge of different neural network architectures and their typical use scenarios, and a demonstrated capacity to critically choose the right architecture and training mechanisms for each task.
Finally, the students will receive hands-on training and acquire practical experience on using current deep learning frameworks to solve specific tasks.
Competences
- Make effective use of bibliographical resources and electronic resources to obtain information.
- Solve problems related to the analysis of large volumes of data through the design of intelligent systems and computational learning.
- Students must be capable of applying their knowledge to their work or vocation in a professional way and they should have building arguments and problem resolution skills within their area of study.
- Students must be capable of communicating information, ideas, problems and solutions to both specialised and non-specialised audiences.
- Students must have and understand knowledge of an area of study built on the basis of general secondary education, and while it relies on some advanced textbooks it also includes some aspects coming from the forefront of its field of study.
- Using criteria of quality, critically evaluate the work carried out.
- Work cooperatively in a multidisciplinary context assuming and respecting the role of the different members of the team.
Learning Outcomes
- Identify and define computational solutions in multiple domains for decision making based on exploring alternatives, uncertain reasoning and task planning.
- Learn and apply the most appropriate learning techniques for solving computational problems in distinct case studies.
- Make effective use of bibliographical resources and electronic resources to obtain information.
- Students must be capable of applying their knowledge to their work or vocation in a professional way and they should have building arguments and problem resolution skills within their area of study.
- Students must be capable of communicating information, ideas, problems and solutions to both specialised and non-specialised audiences.
- Students must have and understand knowledge of an area of study built on the basis of general secondary education, and while it relies on some advanced textbooks it also includes some aspects coming from the forefront of its field of study.
- Understand and evaluate the results and limitations of the most common learning techniques.
- Understand the deep-learning mechanisms based on neural networks in order to design and apply the most appropriate architecture to a particular problem.
- Using criteria of quality, critically evaluate the work carried out.
- Work cooperatively in a multidisciplinary context, taking on and respecting the role of the distinct members in the team.
Content
- Neural networks
- Perceptron
- Multi-layer perceptron
- Backpropagation
- Training process
- Initialization
- Optimization algorithms
- Regularization techniques
- Tasks and cost functions
- Deep neural network architectures
- Convolutional networks
- Recurrent networks
- Unsupervised learning (autoencoders)
- Generative Adversarial Networks
Methodology
Neural network design is guided by the types of problems that it aims to solve. Throughout this subject it will be that typology of problems that will provide the motivation of each section and will direct the organization of the contents.
There will be three types of sessions:
Theory classes: The objective of these sessions is for the teacher to explain the theoretical background of the subject. For each one of the topics studied, the theory and mathematical formulation is explained, as well as the corresponding algorithmic solutions.
Laboratory sessions: Laboratory sessions aim to facilitate interaction, collaborative work and to reinforce the comprehension of the topics seen in the theory classes. During laboratory sessions the students will work through practical cases that require the design of solutions using the methods studied in theory classes. Problem solving will be initiated in the class and will be complemented by a weekly set of problems to work through at home.
Project sessions: towards the end of the course the last sessions will be devoted to a project follow-up. This project will consist of a task that must be solved with the tools and concepts worked on throughout the course.
All the information of the subject and the related documents that the students need will be available at the virtual campus (cv.uab.cat).
Annotation: Within the schedule set by the centre or degree programme, 15 minutes of one class will be reserved for students to evaluate their lecturers and their courses or modules through questionnaires.
Activities
| Title | Hours | ECTS | Learning Outcomes |
|---|---|---|---|
| Type: Directed | |||
| Practical session | 14 | 0.56 | 8, 2, 7, 1, 6, 5, 4, 10, 3 |
| Project Sessions | 16 | 0.64 | 9, 8, 2, 7, 1, 5, 4, 10, 3 |
| Theory session | 20 | 0.8 | 8, 2, 7, 1, 6 |
| Type: Supervised | |||
| Tutoring | 5 | 0.2 | 9, 8, 2, 7, 1, 3 |
| Type: Autonomous | |||
| Dedication to practical work | 29 | 1.16 | 9, 8, 2, 7, 6, 5, 4, 3 |
| Project development | 16 | 0.64 | 9, 2, 7, 1, 10, 3 |
| Reading and study of material | 45 | 1.8 | 8, 2, 7, 1, 3 |
Assessment
To assess the level of student learning, a formula is established that combines theoretical and practical knowledge acquisition, and the ability to solve problems.
Final Grade
The final grade is calculated weighted in the following way and according to the different activities that are carried out:
Final grade = 0.45 * Theory Grade + 0.1 * Problems Portfolio Grade + 0.45 * Project Evaluation
This formula will be applied whenever the theory grade be greater than 4. There is no restriction on the problems portafolio grade or the projects grade. If the calculation of the formula yields >= 5, but the theory grade do not reach the minimum required, then the final grade that will be the smallest among the results of applying the previous formula and 4.5.
Theory Grade
The theory grade aims to assess the individual abilities of the student in terms of the theoretical content of the subject, this is done continuously during the course through two partial exams. The overall theory grade is the average of the grades of the two partial exams.
Theory Grade = 0.5 * Exam 1 Grade + 0.5 * Exam 2 Grade
These exams aim to assess the abilities of each student in an individualized manner, both in terms of solving problems using the techniques explained in class, as well as evaluating the level of conceptualization that the student has made of the techniques seen. If the calculation of the formula yields a theory grade < 5, it will be required to pass the 2n chance exam.
2n chance exam. In case the theory grade does not reach the adequate level to pass, the students can take a 2n chance exam about all the contents of the subject. In case of passing, the maximum score of the recovery exam will be 5.
Problems Portfolio Grade
The aim of the problems (exercises) is for the student to train with the contents of the subject continuously and become familiar with the application of the theoretical concepts. As evidence of this work, the presentation of a portfolio is requested in which the exercises worked out will be collated. To obtain a problems grade it is required that the student submits a minimum of 70% of the problem sets, in the opposite case, the problems grade will be 0.
Project Evaluation Grade
The most practical training part of this subject is based on the development of a project that is done at the end of the course. It has an essential weight in the overall grade of the subject and it is expected students face the problem of designing a solution to a challenge that is presented in a contextualized way and that, therefore, requires the design of a full solution, from the beginnig, processing data to the end, designing and training a neural network model. In addition, students must demonstrate their skills to work as a team and convincingly present the results.
Projects are evaluated through their deliverable, an oral presentation that the students will make to class, and a self-assessment process by the students. The note is calculated in the following way:
Project Score = 0.5 * Deliverable Score + 0.3 * Presentation Score + 0.2 * Self-Assessment Score
In case of not passing the project, it is allowed a 2n chance to pass the project at the group level or at the individual level with the restriction that the maximum grade of the project will be a 5 (approve). In the 2n chance of the project, the grades obtained from the presentation and self-assessment are not considered.
Important notes
This subject does not foresee the single assessment system.
Notwithstanding other disciplinary measures deemed appropriate, and in accordance with the academic regulations in force, evaluation activities will be suspended with zero (0) whenever a student commits any academic irregularities that may alter such evaluation (for example, plagiarizing, copying, letting copy, ...). The evaluationactivities qualified in this way and by this procedure will not be recoverable. If you need to pass any of these assessment activities to pass the subject, this subject will be failed directly, without opportunity to recover it in the same course.
In case there the student does not deliver any exercise solutions, does not participate in any practical evaluation, and does not take any exam, the corresponding grade will be a "non-evaluable". In any other case, the “no shows” count as a0 forthe calculation of the weighted average.
In order to pass the course with honours, the final grade obtained must be equal or higher than 9 points. Because the number of students with this distinction cannot exceed 5% of the total number of students enrolled in thecourse, it is given to whoever has the highest final marks. In case of a tie, the results of the partial exams will be taken into account.
Assessment Activities
| Title | Weighting | Hours | ECTS | Learning Outcomes |
|---|---|---|---|---|
| Examinations | 45 | 4 | 0.16 | 8, 2, 7, 1, 6, 4, 3 |
| Problems deliverables | 10 | 0 | 0 | 8, 2, 7, 1, 6, 5, 4, 3 |
| Project Evaluation | 45 | 1 | 0.04 | 9, 8, 2, 7, 6, 5, 4, 10, 3 |
Bibliography
Books:
- Deep Learning, Ian Goodfellow, Yoshua Bengio, and Aaron Courville, MIT Press, 1st Ed. 2016
- Deep learning with Python, François Chollet, Manning Publications, 1st Ed., 2017
- Pattern Recognition and Machine Learning, Christopher Bishop, Springer, 2011
- Neural Networks for Pattern Recognition, Christopher Bishop, Oxford University Press, 1st ed., 1996
Books online
- Michael Nielsen’s Neural Networks and Deep Learning
http://neuralnetworksanddeeplearning.com/ - Zhang, Z.C. Lipton, M. Li, A.J. Smola, “Dive into Deep Learning”, 2021
https://d2l.ai/
Software
For the practical activities of the course we will use Python (NumPy, MatPlotLib, SciKit Learn) and PyTorch