Deep Learning
Code: 44737 ECTS Credits: 6| Degree | Type | Year | Semester |
|---|---|---|---|
| 4318303 Reseach and Innovation in Computer Based Science and Engineering | OT | 0 | 2 |
Contact
- Name:
- Debora Gil Resina
- Email:
- debora.gil@uab.cat
Teaching groups languages
You can check it through this link. To consult the language you will need to enter the CODE of the subject. Please note that this information is provisional until 30 November 2023.
Prerequisites
It is recommended that students have knowledge and skills of:
- Programming in the Python programming language
- Signal, Image and Video Processing
- Statistical validation
Objectives and Contextualisation
This subject aims to give a practical introduction to neural network models and deep learning.
The students will consolidate and extend their theoretical background, by building on top of previous subjects on machine learning and complementing previous knowledge with new concepts on neural network design, deep learning frameworks, and the training process for such models.
The students should finish this subject, having a broad knowledge of different neural network architectures and their typical use scenarios, and a demonstrated capacity to critically choose the right architecture and training mechanisms for each task.
Finally, the students will receive hands-on training and acquire practical experience on using current deep learning frameworks to solve specific tasks.
Learning Outcomes
- CA18 (Competence) Design the most appropriate neural network architecture in order to solve a given problem.
- CA19 (Competence) Design computational solutions in multiple domains related to decision making based on the exploration of alternatives, uncertain reasoning and task planning.
- KA23 (Knowledge) Describe the structure of convolutional and recurrent neural networks and to which environments they best suited for use.
- KA24 (Knowledge) Describe the different data structuring and representation models.
- KA25 (Knowledge) Describe advanced techniques for handling neural networks such as reinforcement learning, as well as adequately visualise the intermediate results of processing.
- SA31 (Skill) Solve problems related to the analysis of large volumes of data by designing intelligent systems and using computational learning techniques.
- SA32 (Skill) Solve specific problems using deep learning systems based on neural networks.
- SA33 (Skill) Select the most appropriate neural network architecture according to the available data in order to obtain the expected results.
- SA34 (Skill) Use neural network visualisation systems to evaluate possible optimisations that improve the system's performance.
Content
Introduction to neural networks: Perceptron, function of loss, training, back propagation.
Introduction to deep learning convolutional neural networks: classification architectures, segmentation architectures (Unet), fine tunning.
Validation: reliable metrics, detection of biases in models, unbalanced problems, level of generalization of models. Explainability: activation and attention maps.
Processing of temporal data: recurrent networks (LSTMs), transformers (translators, visual transformers), backbone architectures
Unsupervised learning, autoencoders and anomaly detection.
Generative models, adversarial Networks, variational autoencoders.
Metric Learning (triplet loss, Barlow twins), one-shot approaches, Siamese networks.
Methodology
The course will follow a teaching learning methodology called Project Based Learning (PBL). The ABP methodology aims to empower and motivate the student in their learning. Groups of between 5 and 6 students will be formed who will be tasked with carrying out a set of projects (of medium size) throughout the semester. There will be weekly monitoring and both group and individual tutoring of the students.
The projects are set by the teaching staff so that they meet the following conditions: be as real as possible; be treatable using elementary tools; not have a standard solution algorithm associated with it.
On the other hand, it is essential to understand that it is not about finding an algorithm that works in 100 x 100 of the cases - often there is no such thing - but simply about "giving a reasonable solution proposal".
Projects must develop each team with the maximum possible autonomy. Each team will be assigned a tutor who will follow the evolution but in principle will refrain from imposing their ideas. On the other hand, the student must be clear that it is not a question of looking for the 'solution' of the problem in other places, but of making an original contribution. This does not mean that you have to give up the information that may exist in the bibliography or on the Internet; but when it is used, the teacher must be informed and explain it in memory.
The realization of the project must end in a program and a final report. In addition to delivering it in written form, the results of this report will be the subject of an oral presentation. Both, written memory and oral presentation, should be directed mainly to the entity, surely hypothetical, that would have proposed the problem. As a general rule, technicalities will be relegated to specific sections of the written report.
In the oral presentations of the projects, the entire class is expected to attend, and that they intervene through questions and observations.
Annotation: Within the schedule set by the centre or degree programme, 15 minutes of one class will be reserved for students to evaluate their lecturers and their courses or modules through questionnaires.
Activities
| Title | Hours | ECTS | Learning Outcomes |
|---|---|---|---|
| Type: Directed | |||
| Theoretical Explanations | 10 | 0.4 | CA18, KA23, SA32, CA18 |
| Type: Supervised | |||
| Grupal Problem Resolution | 20 | 0.8 | CA18, CA19, KA23, KA24, KA25, SA31, SA32, SA33, SA34, CA18 |
| Type: Autonomous | |||
| Personal work | 115 | 4.6 | CA18, CA19, KA23, KA24, KA25, SA31, SA32, SA33, SA34, CA18 |
Assessment
Given that most of the work revolves around a set of projects that is developed throughout the course, the evaluation has a continuous nature, and its final result is not recoverable.
Evaluation Methodology
The minutes that the group will write after each tutored session will be evaluated, in which they will describe their discussions and agreements and the self-evaluation surveys that each student will take on their classmates and on themselves. At the end of each project, the students will make an oral presentation of the project and will deliver a report of the work carried out. Both will be evaluated by the teachers of the subject, whether or not they are tutors. Students will not take any written exams.
For the evaluation, the following INSTRUMENTS and ACTIVITIES will be used:
-
An evaluation made by the teachers from the presentation of the projects carried out by the group (quality of work, presentation, memory delivered). Group grade (0 to 10) From:
-
STUDENT PORTFOLIO: Document where the development of the work done is explained: project approach, meeting minutes, information sought, explanation of the implemented application with a small user manual and tests and tests performed.
-
PRESENTATION: Oral presentation in 10-15 slides on the project developed and results obtained.
-
APPLICATION: developed program.
-
ACTS AND CONTROLS: Presentation of the documentation delivered.
-
An individual evaluation based on the observations made by the tutors in the tutored sessions, where the attitude, initiative, participation, attendance and punctuality of the student in the group sessions will be taken into account. Individual mark (0 to 10).
-
Co-evaluation and self-evaluation surveys among group members at the end of each project. Peer Coevaluation Note (0 to 10).
-
Oral presentations are made before the students and the groups will assess the work of their classmates as a ranking. The group that is in 1st position will receive 10 points, the 2nd 8 points and so on. Class Coevaluation Note (0 to 10).
Grades
Each project will have a grade that will be calculated as follows:
Project Grade = 0.5 * Group Grade + 0.3 * Individual Grade + 0.1 * Peer Coevaluation Grade + 0.1 * Class Coevaluation Grade
The final grade will come from the weighted average of the projects carried out. The weighting will be the same for all projects
To distinguish between 'failed' and 'no-show', a deadline is set for students to unsubscribe from the evaluation, in which case they will appear as 'no-show'. To unsubscribe, you must notify the teacher, in writing or by email, and obtain an acknowledgment of receipt.
Assessment Activities
| Title | Weighting | Hours | ECTS | Learning Outcomes |
|---|---|---|---|---|
| Class Coevaluation Note | 10% | 0 | 0 | CA19, SA31 |
| Group Mark | 50% | 5 | 0.2 | CA18, CA19, KA23, KA24, KA25, SA31, SA32, SA33, SA34 |
| Individual Mark | 30% | 0 | 0 | CA18, CA19, KA23, KA24, KA25, SA31, SA32, SA33, SA34 |
| Peer Coevaluation Note | 10% | 0 | 0 | SA31, SA32, SA33 |
Bibliography
Books
Deep Learning, Ian Goodfellow, Yoshua Bengio, and Aaron Courville, MIT Press, 1st Ed. 2016
Deep learning with Python, François Chollet, Manning Publications, 1st Ed., 2017
Pattern Recognition and Machine Learning, Christopher Bishop, Springer, 2011
Neural Networks for Pattern Recognition, Christopher Bishop, Oxford University Press, 1st ed., 1996
Books online:
Michael Nielsen's Neural Networks and Deep Learning http://neuralnetworksanddeeplearning.com/
Zhang, Z.C. Lipton, M. Li, A.J. Smola, "Dive into Deep Learning", 2021 https://d2l.ai/
Links (Tutorials and Talks):
https://towardsdatascience.com
https://cs.stanford.edu/~sanmi/talks.html
Software
Python with OpenCV, pytorch, cuda. As far as possible, we facilitate access to clusters of GPUs and CPUs.