Logo UAB
2021/2022

Neural Networks and Deep Learning

Code: 104407 ECTS Credits: 6
Degree Type Year Semester
2503740 Computational Mathematics and Data Analytics OB 3 2
The proposed teaching and assessment methodology that appear in the guide may be subject to changes as a result of the restrictions to face-to-face class attendance imposed by the health authorities.

Contact

Name:
Dimosthenis Karatzas
Email:
Dimosthenis.Karatzas@uab.cat

Use of Languages

Principal working language:
spanish (spa)
Some groups entirely in English:
No
Some groups entirely in Catalan:
No
Some groups entirely in Spanish:
No

Other comments on languages

Written material of the subject will be prepared in English.

Teachers

Pau Riba Fierrez
Lluis Gomez Bigorda

Prerequisites

To have completed the subjects of Materia 7: Artificial Intelligence, Machine Learning, and the subjects of Modelling and Inference (2nd), Complex Data Analysis (2nd), and Information Theory (3rd).

Objectives and Contextualisation

This subject aims to give a practical introduction to neural network models and deep learning.

The students will consolidate and extend their theoretical background, by building on top of previous subjects on machine learning and complementing previous knowledge with new concepts on neural network design, deep learning frameworks, and the training process for such models.

The students should finish this subject, having a broad knowledge of different neural network architectures and their typical use scenarios, and a demonstrated capacity to critically choose the right architecture and training mechanisms for each task.

Finally, the students will receive hands-on training and acquire practical experience on using current deep learning frameworks to solve specific tasks.

Competences

  • Make effective use of bibliographical resources and electronic resources to obtain information.
  • Solve problems related to the analysis of large volumes of data through the design of intelligent systems and computational learning.
  • Students must be capable of applying their knowledge to their work or vocation in a professional way and they should have building arguments and problem resolution skills within their area of study.
  • Students must be capable of communicating information, ideas, problems and solutions to both specialised and non-specialised audiences.
  • Students must have and understand knowledge of an area of study built on the basis of general secondary education, and while it relies on some advanced textbooks it also includes some aspects coming from the forefront of its field of study.
  • Using criteria of quality, critically evaluate the work carried out.
  • Work cooperatively in a multidisciplinary context assuming and respecting the role of the different members of the team.

Learning Outcomes

  1. Identify and define computational solutions in multiple domains for decision making based on exploring alternatives, uncertain reasoning and task planning.
  2. Learn and apply the most appropriate learning techniques for solving computational problems in distinct case studies.
  3. Make effective use of bibliographical resources and electronic resources to obtain information.
  4. Students must be capable of applying their knowledge to their work or vocation in a professional way and they should have building arguments and problem resolution skills within their area of study.
  5. Students must be capable of communicating information, ideas, problems and solutions to both specialised and non-specialised audiences.
  6. Students must have and understand knowledge of an area of study built on the basis of general secondary education, and while it relies on some advanced textbooks it also includes some aspects coming from the forefront of its field of study.
  7. Understand and evaluate the results and limitations of the most common learning techniques.
  8. Understand the deep-learning mechanisms based on neural networks in order to design and apply the most appropriate architecture to a particular problem.
  9. Using criteria of quality, critically evaluate the work carried out.
  10. Work cooperatively in a multidisciplinary context, taking on and respecting the role of the distinct members in the team.

Content

  • Neural networks
    • Perceptron
    • Multi-layer perceptron
    • Backpropagation
  • Training process
    • Initialization
    • Optimization algorithms
    • Regularization techniques
    • Tasks and cost functions
  • Deep neural network architectures
    • Convolutional networks
    • Recurrent networks
    • Unsupervised learning (autoencoders)
    • Metric learning (Siamese and triplet networks)
    • Generative Adversarial Networks

Methodology

Neural network design is guided by the types of problems that it aims to solve. Throughout this subject it will be that typology of problems that will provide the motivation of each section and will direct the organization of the contents.

There will be two types of sessions:

Theory classes: The objective of these sessions is for the teacher to explain the theoretical background of the subject. For each one of the topics studied, the theory and mathematical formulation is explained, as well as the corresponding algorithmic solutions.

Laboratory sessions: Laboratory sessions aim to facilitate interaction, collaborative work and to reinforce the comprehension of the topics seen in the theory classes. During laboratory sessions the students will work through practical cases that require the design of solutions using the methods studied in theory classes. Problem solving will be initiated in the class and will be complemented by a weekly set of problems to work through at home.

The above activities will be complemented by a system of tutoring and consultations outside class hours.

All the information of the subject and the related documents that the students need will be available at the virtual campus (cv.uab.cat).

Annotation: Within the schedule set by the centre or degree programme, 15 minutes of one class will be reserved for students to evaluate their lecturers and their courses or modules through questionnaires.

Activities

Title Hours ECTS Learning Outcomes
Type: Directed      
Practical session 22 0.88 8, 2, 7, 1, 6, 5, 4, 10, 3
Theory session 28 1.12 8, 2, 7, 1, 6
Type: Supervised      
Tutoring 5 0.2 9, 8, 2, 7, 1, 3
Type: Autonomous      
Dedication to practical work 45 1.8 9, 8, 2, 7, 6, 5, 4, 3
Reading and study of material 45 1.8 8, 2, 7, 1, 3

Assessment

To assess the level of student learning, a formula is established that combines theoretical and practical knowledge acquisition, and the ability to solve problems.

Final Grade

The final grade is calculated weighted in the following way and according to the different activities that are carried out:

Final grade = 0.5 * Theory Grade + 0.1 * Problems Portfolio Grade + 0.4 * Practical Evaluation

This formula will be applied as long as the theory and the practical evaluation grades are higher than 5. There is no restriction on the problems portfolio grade. If doing the calculation of the formula yields >= 5 but the theory grade or practical evaluation grade does not reach the minimum required, then a final grade of 4.5 will be given.

Theory Grade

The theory grade aims to assess the individual abilities of the student in terms of the theoretical content of the subject, this is done continuously during the course through two partial exams. The overall theory grade is the average of the grades of the two partial exams.

The partial exam #1 is done in the middle of the semester and serves to eliminate part of the subject if it is passed. The partial exam #2 is done at the end of the semester and serves to eliminate part of the subject if it is passed.

These exams aim to assess the abilities of each student in an individualized manner, both in terms of solving problems using the techniques explained in class, as well as evaluating the level of conceptualization that the student has made of the techniques seen. In order to obtain a final pass theory grade, it will be required for the both partial exam grades to be higher than 4. If doing the calculation of the formula yields >= 5 but the grades of any of the two partial exams do not reach the minimum required, then the grade for theory that will be used in the final calculation will be 4.5.

Recovery exam. In case the theory grade does not reach the adequate level to pass, the students can take a recovery exam, destined to recover the failed part (1, 2 or both) of the continuous evaluation process.

Problems Portfolio Grade

The aim of the problems (exercises) is for the student to train with the contents of the subject continuously and become familiar with the application of the theoretical concepts. As evidence of this work, the presentation of a portfolio is requested in which the exercises worked out will be collated. To obtain a problems grade it is required that the student submits a minimum of 70% of the problem sets, in the opposite case, the problems grade will be 0.

Practical Evaluation Grade

The practical abilities of the students will be evaluated twice during the semester. The evaluation will be based on the students presenting and explaining the practical work they have individually completed at home.

The overall practical evaluation grade is the average of the two practical evaluations.

In case of not passing any of the practical evaluations, the student will be able to recover the failed part, restricted to a maximum grade of 7/10.

Important notes

Notwithstanding other disciplinary measures deemed appropriate, and in accordance with the academic regulations in force, evaluation activities will be suspended with zero (0) whenever a student commits any academic irregularities that may alter such evaluation (for example, plagiarizing, copying, letting copy, ...). The evaluation activities qualified in this way and by this procedure will not be recoverable. If you need to pass any of these assessment activities to pass the subject, this subject will be failed directly, without opportunity to recover it in the same course.

In case there the student does not deliver any exercise solutions, does not participate in any practical evaluation, and does not take any exam, the corresponding grade will be a "non-evaluable". In any other case, the “no shows” count as a 0 for the calculation of the weighted average.

In order to pass the course with honours, the final grade obtained must be equal or higher than 9 points. Because the number of students with this distinction cannot exceed 5% of the total number of students enrolled in the course, it is given to whoever has the highest final marks. In case of a tie, the results of the partial exams will be taken into account.

Assessment Activities

Title Weighting Hours ECTS Learning Outcomes
Examinations 50 4.5 0.18 8, 2, 7, 1, 6, 4, 3
Practicals Evaluation 40 0.5 0.02 9, 8, 2, 7, 6, 5, 4, 10, 3
Problems deliverables 10 0 0 8, 2, 7, 1, 6, 5, 4, 3

Bibliography

Books:

  • Deep Learning, Ian Goodfellow, Yoshua Bengio, and Aaron Courville, MIT Press, 1st Ed. 2016
  • Deep learning with Python, François Chollet, Manning Publications, 1st Ed., 2017
  • Pattern Recognition and Machine Learning, Christopher Bishop, Springer, 2011
  • Neural Networks for Pattern Recognition, Christopher Bishop, Oxford University Press, 1st ed., 1996

Books online

Software

For the practical activities of the course we will use Python (NumPy, MatPlotLib, SciKit Learn) and PyTorch