Neural Networks and Deep Learning
Code: 104407 ECTS Credits: 6| Degree | Type | Year | Semester |
|---|---|---|---|
| 2503740 Computational Mathematics and Data Analytics | OB | 3 | 2 |
Contact
- Name:
- Dimosthenis Karatzas
- Email:
- Dimosthenis.Karatzas@uab.cat
Use of Languages
- Principal working language:
- spanish (spa)
- Some groups entirely in English:
- No
- Some groups entirely in Catalan:
- No
- Some groups entirely in Spanish:
- No
Other comments on languages
Written material of the subject will be prepared in English.Prerequisites
To have completed the subjects of Materia 7: Artificial Intelligence, Machine Learning, and the subjects of Modelling and Inference (2nd), Complex Data Analysis (2nd), and Information Theory (3rd).
Objectives and Contextualisation
This subject aims to give a practical introduction to neural network models and deep learning.
The students will consolidate and extend their theoretical background, by building on top of previous subjects on machine learning and complementing previous knowledge with new concepts on neural network design, deep learning frameworks, and the training process for such models.
The students should finish this subject, having a broad knowledge of different neural network architectures and their typical use scenarios, and a demonstrated capacity to critically choose the right architecture and training mechanisms for each task.
Finally, the students will receive hands-on training and acquire practical experience on using current deep learning frameworks to solve specific tasks.
Competences
- Make effective use of bibliographical resources and electronic resources to obtain information.
- Solve problems related to the analysis of large volumes of data through the design of intelligent systems and computational learning.
- Students must be capable of applying their knowledge to their work or vocation in a professional way and they should have building arguments and problem resolution skills within their area of study.
- Students must be capable of communicating information, ideas, problems and solutions to both specialised and non-specialised audiences.
- Students must have and understand knowledge of an area of study built on the basis of general secondary education, and while it relies on some advanced textbooks it also includes some aspects coming from the forefront of its field of study.
- Using criteria of quality, critically evaluate the work carried out.
- Work cooperatively in a multidisciplinary context assuming and respecting the role of the different members of the team.
Learning Outcomes
- Identify and define computational solutions in multiple domains for decision making based on exploring alternatives, uncertain reasoning and task planning.
- Learn and apply the most appropriate learning techniques for solving computational problems in distinct case studies.
- Make effective use of bibliographical resources and electronic resources to obtain information.
- Students must be capable of applying their knowledge to their work or vocation in a professional way and they should have building arguments and problem resolution skills within their area of study.
- Students must be capable of communicating information, ideas, problems and solutions to both specialised and non-specialised audiences.
- Students must have and understand knowledge of an area of study built on the basis of general secondary education, and while it relies on some advanced textbooks it also includes some aspects coming from the forefront of its field of study.
- Understand and evaluate the results and limitations of the most common learning techniques.
- Understand the deep-learning mechanisms based on neural networks in order to design and apply the most appropriate architecture to a particular problem.
- Using criteria of quality, critically evaluate the work carried out.
- Work cooperatively in a multidisciplinary context, taking on and respecting the role of the distinct members in the team.
Content
- Neural networks
- Perceptron
- Multi-layer perceptron
- Backpropagation
- Training process
- Initialization
- Optimization algorithms
- Regularization techniques
- Tasks and cost functions
- Deep neural network architectures
- Convolutional networks
- Recurrent networks
- Hopfield nets and Boltzmann machines
- Unsupervised learning (autoencoders)
- Metric learning (Siamese and triplet networks)
- Generative Adversarial Networks
Methodology
Neural network design and applications is guided by the types of problems that it aims to solve. Throughout this subject it will be that typology of problems that will provide the motivation of each section and will direct the organization of the contents.
There will be three types of sessions:
Theory classes: The objective of these sessions is for the teacher to explain the theoretical background of the subject. For each one of the topics studied, the theory and mathematical formulation is explained, as well as the corresponding algorithmic solutions.
Practice sessions: Practice sessions aim to facilitate interaction and to reinforce the comprehension of the topics seen in the theory classes by working through practical cases that require the design of a solution in which the methods seen in the theory classes are used. These sessions will be initiated in the class and will be complemented by a weekly set of problems to work through at home.
Project sessions: Project sessions complement the weekly practice sessions. They are sessions in which different types of activities related to the realization of a series of projects are carried out. Students will work collaboratively on these projects in small teams. The projects will have a common base, and an open-end part that teams can decide to approach in different ways. During the project sessions (1) the teacher will present and discuss the projects and possible approaches, and (2) the teams will present their final results to the class. The teams will have to design and implement a solution, manage the distribution and organization of the work to be carried out, and present final results to the teacher and the rest of the students.
The above activities will be complemented by a system of tutoring and consultations outside class hours.
The subject will follow a mixed methodology paradigm, where theoretic explanations (theory classes) will be combined with practical examples (practice sessions), switching between the two modalities as necessary during the sessions. The ratio between theory and practice will be approximately 2 to 1 in the classroom, complemented by self-taught activities (problem sets), and teamwork activities (projects).
All the information of the subject and the related documents that the students need will be available at the virtual campus (cv.uab.cat).
Activities
| Title | Hours | ECTS | Learning Outcomes |
|---|---|---|---|
| Type: Directed | |||
| Practical session (problems) | 13 | 0.52 | 8, 2, 7, 1, 6, 5, 4, 3 |
| Theory session | 28 | 1.12 | 8, 2, 7, 1, 6 |
| Type: Supervised | |||
| Project session | 9 | 0.36 | 9, 8, 2, 7, 1, 6, 5, 4, 10 |
| Tutoring | 5 | 0.2 | 9, 8, 2, 7, 1, 3 |
| Type: Autonomous | |||
| Dedication to projects | 37 | 1.48 | 8, 2, 7, 1, 6, 5, 4, 10, 3 |
| Dedication to resolve problems | 12 | 0.48 | 8, 2, 7, 6, 5, 4, 3 |
| Reading and study of material | 40 | 1.6 | 8, 2, 7, 1, 3 |
Assessment
To assess the level of student learning, a formula is established that combines knowledge acquisition, the ability to solve problems and the ability to work as a team, as well as the presentation of the results obtained.
Final note
The final grade is calculated weighted in the following way and according to the different activities that are carried out:
Final grade = 0.4 * Theory Grade + 0.1 * Problems Grade + 0.5 * Projects Grade
This formula will be applied as long as the theory and the projects grades, are higher than 5. There is no restriction on the problems grade. If doing the calculation of the formula yields >= 5 but the grades of theory or projects do not reach the minimum required, then a final grade of 4.5 will be given.
Theory Grade
The theory grade aims to assess the individual abilities of the student in terms of the theoretical content of the subject, this is done continuously during the course through two partial exams:
Theory Grade = 0.5 * Grade Exam #1 + 0.5 * Grade Exam #2
The partial exam #1 is done in the middle of the semester and serves to eliminate part of the subject if it is passed.
The partial exam #2 is done at the end of the semester and serves to eliminate part of the subject if it is passed.
These exams aim to assess the abilities of each student in an individualized manner, both in terms of solving problems using the techniques explained in class, as well as evaluating the level of conceptualization that the student has made of the techniques seen. In order to obtain a final pass theory grade, it will be required for the both partial exam grades to be higher than 4. If doing the calculation of the formula yields >= 5 but the grades of any of the two partial exams do not reach the minimum required, then the grade for theory that will be used in the final calculation will be 4.5.
Recovery exam. In case the theory grade does not reach the adequate level to pass, the students can takea recovery exam, destined to recover the failed part (1, 2 or both) of the continuous evaluation process.
Problems Grade
The aim of the problems (exercises) is for the student to train with the contents of the subject continuously and become familiar with the application of the theoretical concepts. As evidence of this work, the presentation of a portfolio is requested in which the exercises worked out will be collated. To obtain a problems grade it is required that the student submits a minimum of 70% of the problem sets, in the opposite case, the problems grade will be 0.
Exercises Grade = Portfolio evaluation
Projects Grade
The part of practical training is based in a series of projects that will be developed during the course. It carries an essential weight in the overall mark of the subject and aims for the student to design a solution to a problem that is set out in a contextualized way. Such problems will require the design of an integral solution, from data preparation to the design and modelling of neural models. In addition, the students must demonstrate their teamwork skills and present the results to the class convincingly.
Each of the projects is evaluated through its deliverable, an oral presentation that students will make in class, and an self-evaluation process. The grade is calculated as follows:
Individual Project Grade = 0.5 * Grade Deliverables + 0.3 * Grade Presentation + 0.2 * Grade Self-evaluation
The overall grade for the projects is the average of the grades of the N individual projects.
Projects Grade = SUM(Individual Grade Project)/N
In case of not passing any of the projects, the recovery of the part of the deliverables of the unsuccessful projects will be allowed, restricted to a maximum grade of 7/10. The oral presentation cannot be recovered.
Important notes
Notwithstanding other disciplinary measures deemed appropriate, and in accordance with the academic regulations in force, evaluation activities will be suspended with zero (0) whenever a student commits any academic irregularities that may alter such evaluation (for example, plagiarizing, copying, letting copy, ...). The evaluation activities qualified in this way and by this procedure will not be recoverable. If you need to pass any of these assessment activities to pass the subject, this subject will be failed directly, without opportunity to recover it in the same course.
In case there the student does not deliver any exercise solutions, does not attend any project presentation session during the laboratory sessions and does not take any exam, the corresponding grade will be a "non-evaluable". In another case, the “no shows” count as a 0 for the calculation of the weighted average.
In order to pass the course with honours, the final grade obtained must be equal or higher than 9 points. Because the number of students with this distinction cannot exceed 5% of the total number of students enrolled in the course, it is given to whoever has the highest final marks. In case of a tie, the results of the partial exams will be taken into account.
Assessment Activities
| Title | Weighting | Hours | ECTS | Learning Outcomes |
|---|---|---|---|---|
| Examinations | 40 | 5 | 0.2 | 8, 2, 7, 1, 6, 4, 3 |
| Problems deliverables | 10 | 0 | 0 | 8, 2, 7, 1, 6, 5, 4, 3 |
| Project deliverables | 25 | 0 | 0 | 9, 8, 2, 7, 6, 5, 4, 10, 3 |
| Project presentations | 15 | 1 | 0.04 | 9, 8, 2, 7, 5, 4, 10, 3 |
| Self-evaluation | 10 | 0 | 0 | 9, 10 |
Bibliography
- Deep Learning, Ian Goodfellow, Yoshua Bengio, and Aaron Courville, MIT Press, 1st Ed. 2016
- Deep learning with Python, François Chollet, Manning Publications, 1st Ed., 2017
- Pattern Recognition and Machine Learning, Christopher Bishop, Springer, 2011
- Neural Networks for Pattern Recognition, Christopher Bishop, Oxford University Press, 1st ed., 1996