Neural Networks and Deep Learning
Code: 104362 ECTS Credits: 6| Degree | Type | Year | Semester |
|---|---|---|---|
| 2503758 Data Engineering | OB | 3 | 2 |
Contact
- Name:
- Ramon Baldrich Caselles
- Email:
- ramon.baldrich@uab.cat
Teaching groups languages
You can check it through this link. To consult the language you will need to enter the CODE of the subject. Please note that this information is provisional until 30 November 2023.
Teachers
- Carlos Boned Riera
Prerequisites
It is essential to have acquired a good mathematical background as well as to have a good level of programming, mainly in Python. It is essential to have taken the subject of Computational Learning in the first semester. Some of the concepts developed in this subject are the basis of the content and development of Neural Networks
Objectives and Contextualisation
Students will consolidate and expand their theoretical training, based on the knowledge acquired in previous subjects related to machine learning, completing their profile in this area. The aim of the course is to end up having a broad knowledge of the concepts, techniques and structures typical of neural networks, as well as being able to understand and apply the particular methodology of these techniques to real case studies, and finally develop the ability. to choose the most appropriate mechanisms and structures for each particular case of application.
Skills
Make effective use of bibliographic and electronic resources to obtain information.
Solve problems related to the analysis of large volumes of data through the design of intelligent systems and computer learning.
Students must be able to apply their knowledge to their job or vocation in a professional manner and must be able to establish arguments and problem-solving skills.
Students must be able to communicate information, ideas, problems, and solutions to both specialized and non-specialized audiences.
Using quality criteria, critically evaluate the work done.
Work cooperatively in a multidisciplinary context assuming and respecting the role of the different members of the team.
Competences
- Analyse data efficiently for the development of smart systems with the capacity for autonomous learning and/or data mining.
- Students must be capable of collecting and interpreting relevant data (usually within their area of study) in order to make statements that reflect social, scientific or ethical relevant issues.
- Work cooperatively in complex and uncertain environments and with limited resources in a multidisciplinary context, assuming and respecting the role of the different members of the group.
Learning Outcomes
- Design and implement an integrated strategy of statistical techniques and artificial intelligence for the development of descriptive and predictive systems.
- Students must be capable of collecting and interpreting relevant data (usually within their area of study) in order to make statements that reflect social, scientific or ethical relevant issues.
- Work cooperatively in complex and uncertain environments and with limited resources in a multidisciplinary context, assuming and respecting the role of the different members of the group.
Content
-logistic regression
-perceptron
-activation function
-gradient descent
-MLP
-backpropagation
2 Practical aspects of Neural Networks
- Overfitting
- Regularization
- Dropout
- Input normalization
- Vanishing / exploding gradients
- Initialization of weights
- Gradient check
3 Convolutional Networks
- Computer Vision
- What is convolution
- Padding, stride convolutions
- Filter algebra
- pooling layers
- softmax regression
- first networks: AlexNet, VGG
4 CNN Case Studies: Classification
- inception
- residual networks
- networ in network: 1x1 convolution
5 Practical aspects of Neural Networks II
- Adjustment of hyperparameters
- Normalization of activations, batch norm
- Data augmentation
- Transfer Learning
6 CNN: object detection
- object detection vs classification
- box prediction
- metric: intersection over union
- Non-max deletion
- Anchor boxes
- Base Networks: Yolo, FasterRCNN
7 Sequential Networks: Recurrent Neural Networks
- Model of recurrent neural networks
- Backpropagation over time
- Type of RNN
- Language model and sequence generation
- GRU & LSTM
- Word2vec
Methodology
All the information of the subject and the related documents that the students need will be found in the page Virtual Campus (http://cv.uab.cat/).
The different activities that will be carried out in the subject are organized as follows:
Lectures
The main concepts and algorithms of each theory topic will be presented. These subjects suppose the starting point in the work of the subject.
Problem seminars
They will be classes with small groups of students, which facilitate the interaction, or of individual character, according to the cases. In these classes, practical cases will be considered that require the design of a solution in which the methods seen in the theory classes are used. It is impossible to follow the kinds of problems if the contents of the theory classes are not followed. The result of these sessions is the resolution of the problems that must be delivered on a weekly basis. The specific mechanism for the delivery, and the evaluation process, will be indicated on the web page of the subject (Charon space).
Laboratory practicum
The working groups will be formed by groups of 3-4 students and should form the second week of the course. These working groups must be maintained until the end of the course and they must self-manage: role distribution, work planning, assignment of tasks, management of available resources, conflicts, etc. Although the teacher will guide the learning process, his intervention in the management of the groups will be minimal.
At the beginning of the course, the problems to be solved will be presented and the students will define their own project. Throughout the semester, students will work in cooperative groups and should analyze the chosen problem, design and implement solutions based on different computational learning algorithms seen in class, analyze the results obtained in eachof themethods and defend their project inpublic.
To developthe project, the groups will work autonomously and the practice sessions will be devoted mainly to answer questions with the teacher who will monitor the status of the project, indicate errors to be corrected, propose improvements, etc.
Some of the sessions will be marked as control sessions in which some part of the project must be delivered. In these sessions the groups must explain the work done and the teacher will ask questions to all group members to assess the work done. Attendance at these sessions is mandatory.
In the last session of each of the internship projects, the groups will make a presentation of the project where they will explain the project developed, the solution adopted and the results obtained. In this presentation each member of the group must make a part of the presentation.
Annotation: Within the schedule set by the centre or degree programme, 15 minutes of one class will be reserved for students to evaluate their lecturers and their courses or modules through questionnaires.
Activities
| Title | Hours | ECTS | Learning Outcomes |
|---|---|---|---|
| Type: Directed | |||
| Theoretical content | 22 | 0.88 | 1, 2 |
| Type: Supervised | |||
| lab practicums | 16 | 0.64 | 1, 3 |
| serminars | 10 | 0.4 | 1, 2 |
| Type: Autonomous | |||
| Setup an ddevelopment of practical projects | 52 | 2.08 | 1, 2, 3 |
| study | 28 | 1.12 | 1, 2 |
Assessment
Activities and evaluation instruments:
To evaluate the achievement of the knowledge and skills associated with the subject, an evaluation mechanism is established that combines the assimilation of knowledge, the ability to solve problems, and significantly, the ability to generate computational solutions to problems complex, both as a group and individually.
With this objective, the evaluation is divided into three parts:
- Evaluation of contents
The final content grade will be calculated from several partial exams:
Note Contents = 1/N * Test_i
The number of tests may vary and will be set at the beginning of the course. In order to be able to have a content grade, the grades for each of the tests must be greater than 4.
The partial tests will be done during the course and will mainly be of conceptual content where you will answer different questions about the content developed in the 'theoretical' sessions.
These tests aim to be an individualized evaluation of the student with his abilities to understand the techniques explained in class as well as to evaluate the level of conceptualization that the student has made of the techniques seen.
Recovery tests. In the event that the content grade does not reach the appropriate level in any of the tests, in order to obtain a final grade sufficient to consider the achievement of the knowledge, students can appear in the examination of the call for the subject and return to take an exam that evaluates the content seen in the subject of the part(s) not passed.
There are no validations if the theoretical part had been passed in previous years.
- Evaluation of the work in the problem seminars
The aim of the problems is to cause the student to enter the contents of the subject continuously and from small problems that make them familiar directly with the application of the theory. As evidence of this work, the presentation of a portfolio is requested in which you will have kept the problems you have been working on. This portfolio will have weekly digital delivery. The student will be able to self-assess continuously since he will have the solutions to each of the sets of problems once the delivery period has ended. Together with the hours of tutoring in case doubts arise, it is enough for each student to identify their weak points.
- Group project evaluation.
In the last weeks of the semester, a larger project will be carried out than the exercises done during the course in the problem sessions. This project will be evaluated both individually and as a group. The evaluation mechanisms will be code, report, presentation and monitoring of the project in the assigned class sessions.
The final mark of the subject is obtained by combining the evaluation of these 3 activities as follows:
Final Grade = (0.3 * Contents) + (0.1 * problem portfolio) + (0.6 * Project)
the project will have a mark for its defense and another for its development and depth of the solution
Conditions to pass the subject:
The final content grade must be greater than or equal to 4 in order to pass the subject.
The grade for the project and its defense must be greater than or equal to 6 in order to pass the subject.
In the event that the grade, applying the formula of the previous section ("final grade of the subject"), was higher than 5 but the minimum required in any of the parts had not been exceeded, the final grade in the record will be a 4.5.
As many honors registrations will be assigned as the current regulations allow as long as the mark is higher than 9.0. The assignment of the registrations will be done following the order of notes. In case there are multiple candidates with the same evaluation likely to receive Md'H, additional activities will be proposed to determine the best candidate (s).
The student will be graded as "Not Evaluable" if he / she has no evaluated part of either the theoretical or practical contents.
Important notices:
- The dates of continuous evaluation and delivery of works, as well as all the teaching material will be published in the virtual campus (http://cv.uab.cat/), in the space of this subject and can be subject to changes of programming for reasons of adaptation to possible incidents. Cerbero.uab.cat will always be informed about these changes as it is understood that CV will become the usual mechanism for the exchange of information between teachers and students.
- For each assessment activity, a place, date and time of review will be indicated in which the student will be able to review the activity with the teacher. In this context, claims may be made on the grade of the activity, which will be evaluated by the teacher responsible for the subject. If the student does not appear for this review, this activity will not be reviewed later. Without prejudice to other disciplinary measures
- Without prejudice to other disciplinary measures deemed appropriate, and in accordance with current academic regulations, irregularities committed by a student that may lead to a variation in the grade in an assessable activity will be graded with a zero (0). Assessment activities qualified in this way and by this procedure will not be recoverable. If it is necessary to pass any of these assessment activities to pass the course, this course will be suspended directly, without the opportunity to retake it in the same course. These irregularities include, but are not limited to: - the total or partial copy of a practice, report, or any other evaluation activity;
- let copy;
- present a group work not entirely by the members of the group (applied to all members, not only those who have not worked);
- present as their own materials prepared by a third party,even if they are translations or adaptations, and in general works with non-original and exclusive elements of the student;
- have digital and / or communication devices (such as mobile phones, smart watches, camera pens, etc.) accessible during individual theoretical-practical assessment tests (exams).
- talk with classmates during individual theoretical-practical assessment tests (exams). - observe / look at the theoretical-practical evaluation tests (exams) of other classmates during the performance of the same, even if the copy has not been carried out.
- observe / look at the table, sheets, wall, etc. writings related to the subject during the realization of the theoretical-practical evaluation tests (exams) even if the copy has been proceeded.
The numerical mark of the transcript will be the lower value between 3.0 and the weighted average of the marks in case the student has committed irregularities in an act of evaluation (and therefore it will not be possible to pass it for compensation). In short: copying, copying or plagiarizing (or attempting to) in any of the assessment activities is equivalent to a SUSPENSION with a grade below 3.5.
No unique evaluation planned
Assessment Activities
| Title | Weighting | Hours | ECTS | Learning Outcomes |
|---|---|---|---|---|
| Concept tests | 30% | 7 | 0.28 | 1, 2 |
| Group Project | 40% | 5 | 0.2 | 1, 2, 3 |
| Problem porfolio | 10 | 5 | 0.2 | 1, 2 |
| Project Defence | 20% | 5 | 0.2 | 1, 2, 3 |
Bibliography
Bibliography
Web links
- Subject web page: http://cv.uab.cat
- Deep Learning. MIT Press book. https://www.deeplearningbook.org
Basic Bibliography
- Deep learning with Python, François Chollet, Manning Publications, 1st Ed., 2017
- Pattern Recognition and Machine Learning, Christopher Bishop, Springer, 2011
- Neural Networks for Pattern Recognition, Christopher Bishop, Oxford University Press, 1st ed., 1996
Software
No special software other than the usual ones will be used in these studies.