Neural Networks and Deep Learning
Code: 104362 ECTS Credits: 6| Degree | Type | Year | Semester |
|---|---|---|---|
| 2503758 Data Engineering | OB | 3 | 2 |
Contact
- Name:
- Ramón Baldrich Caselles
- Email:
- Ramon.Baldrich@uab.cat
Use of Languages
- Principal working language:
- catalan (cat)
- Some groups entirely in English:
- No
- Some groups entirely in Catalan:
- Yes
- Some groups entirely in Spanish:
- No
Prerequisites
It is essential to have acquired a good mathematical background as well as to have a good level of programming, mainly in Python. It is essential to have taken the subject of Computational Learning in the first semester. Some of the concepts developed in this subject are the basis of the content and development of Neural Networks
Objectives and Contextualisation
Students will consolidate and expand their theoretical training, based on the knowledge acquired in previous subjects related to machine learning, completing their profile in this area. The aim of the course is to end up having a broad knowledge of the concepts, techniques and structures typical of neural networks, as well as being able to understand and apply the particular methodology of these techniques to real case studies, and finally develop the ability. to choose the most appropriate mechanisms and structures for each particular case of application.
Skills
Make effective use of bibliographic and electronic resources to obtain information.
Solve problems related to the analysis of large volumes of data through the design of intelligent systems and computer learning.
Students must be able to apply their knowledge to their job or vocation in a professional manner and must be able to establish arguments and problem-solving skills.
Students must be able to communicate information, ideas, problems, and solutions to both specialized and non-specialized audiences.
Using quality criteria, critically evaluate the work done.
Work cooperatively in a multidisciplinary context assuming and respecting the role of the different members of the team.
Competences
- Analyse data efficiently for the development of smart systems with the capacity for autonomous learning and/or data mining.
- Students must be capable of collecting and interpreting relevant data (usually within their area of study) in order to make statements that reflect social, scientific or ethical relevant issues.
- Work cooperatively in complex and uncertain environments and with limited resources in a multidisciplinary context, assuming and respecting the role of the different members of the group.
Learning Outcomes
- Design and implement an integrated strategy of statistical techniques and artificial intelligence for the development of descriptive and predictive systems.
- Students must be capable of collecting and interpreting relevant data (usually within their area of study) in order to make statements that reflect social, scientific or ethical relevant issues.
- Work cooperatively in complex and uncertain environments and with limited resources in a multidisciplinary context, assuming and respecting the role of the different members of the group.
Content
-logistic regression
-perceptron
-activation function
-gradient descent
-MLP
-backpropagation
2 Practical aspects of Neural Networks
- Overfitting
- Regularization
- Dropout
- Input normalization
- Vanishing / exploding gradients
- Initialization of weights
- Gradient check
3 Convolutional Networks
- Computer Vision
- What is convolution
- Padding, stride convolutions
- Filter algebra
- pooling layers
- softmax regression
- first networks: AlexNet, VGG
4 CNN Case Studies: Classification
- inception
- residual networks
- networ in network: 1x1 convolution
5 Practical aspects of Neural Networks II
- Adjustment of hyperparameters
- Normalization of activations, batch norm
- Data augmentation
- Transfer Learning
6 CNN: object detection
- object detection vs classification
- box prediction
- metric: intersection over union
- Non-max deletion
- Anchor boxes
- Base Networks: Yolo, FasterRCNN
7 Sequential Networks: Recurrent Neural Networks
- Model of recurrent neural networks
- Backpropagation over time
- Type of RNN
- Language model and sequence generation
- GRU & LSTM
- Word2vec
Methodology
All the information of the subject and the related documents that the students need will be found in the page Virtual Campus (http://cv.uab.cat/).
The different activities that will be carried out in the subject are organized as follows:
Lectures
The main concepts and algorithms of each theory topic will be presented. These subjects suppose the starting point in the work of the subject.
Problem seminars
They will be classes with small groups of students, which facilitate the interaction, or of individual character, according to the cases. In these classes, practical cases will be considered that require the design of a solution in which the methods seen in the theory classes are used. It is impossible to follow the kinds of problems if the contents of the theory classes are not followed. The result of these sessions is the resolution of the problems that must be delivered on a weekly basis. The specific mechanism for the delivery, and the evaluation process, will be indicated on the web page of the subject (Charon space).
Laboratory practicum
The working groups will be formed by groups of 3-4 students and should form the second week of the course. These working groups must be maintained until the end of the course and they must self-manage: role distribution, work planning, assignment of tasks, management of available resources, conflicts, etc. Although the teacher will guide the learning process, his intervention in the management of the groups will be minimal.
At the beginning of the course, the problems to be solved will be presented and the students will define their own project. Throughout the semester, students will work in cooperative groups and should analyze the chosen problem, design and implement solutions based on different computational learning algorithms seen in class, analyze the results obtained in eachof themethods and defend their project inpublic.
To developthe project, the groups will work autonomously and the practice sessions will be devoted mainly to answer questions with the teacher who will monitor the status of the project, indicate errors to be corrected, propose improvements, etc.
Some of the sessions will be marked as control sessions in which some part of the project must be delivered. In these sessions the groups must explain the work done and the teacher will ask questions to all group members to assess the work done. Attendance at these sessions is mandatory.
In the last session of each of the internship projects, the groups will make a presentation of the project where they will explain the project developed, the solution adopted and the results obtained. In this presentation each member of the group must make a part of the presentation.
Annotation: Within the schedule set by the centre or degree programme, 15 minutes of one class will be reserved for students to evaluate their lecturers and their courses or modules through questionnaires.
Activities
| Title | Hours | ECTS | Learning Outcomes |
|---|---|---|---|
| Type: Directed | |||
| Theoretical content | 22 | 0.88 | 1, 2 |
| Type: Supervised | |||
| lab practicums | 16 | 0.64 | 1, 3 |
| serminars | 10 | 0.4 | 1, 2 |
| Type: Autonomous | |||
| Setup an ddevelopment of practical projects | 52 | 2.08 | 1, 2, 3 |
| study | 28 | 1.12 | 1, 2 |
Assessment
Assessment activities and instruments:
To evaluate the achievement of the knowledge and skills associated with the subject, an evaluation mechanism is established that combines the assimilation of knowledge, the ability to solve problems, and significantly, the ability to generate computational solutions to complex problems, both group and individual.
To this end, the evaluation is divided into three parts:
- Content evaluation
The final grade of contents will be calculated from several partial exams:
Note Contents = 1 / N * Test_i
The number of tests may vary and will be at least 2. In order to have a content grade, the grades of each of the tests must be higher than 4.
The partial tests will be done during the course and will eminently be of conceptual content where to answer different questions about the content developed in the ‘theoretical’ sessions.
These tests are intended to be an individualized assessment of the student with their abilities to understand the techniques explained in class and assess the level of conceptualization that the student has made of the techniques seen.
Recovery tests. In case the content grade does not reach the appropriate level in any of the tests, to obtain a final grade sufficient to consider the achievement of knowledge, students can take the exam of the call of the subject and return to take an exam that evaluates the contents seen in the subject of the part (s) not passed. In case of appearing to raise note, the highest note prevails.
There are no validations in case the theoretical part had been passed in previous years.
- Evaluation of the work in the seminars of problems
The problems aim to cause the student to enter the contents ofthe subject continuously and from small problems that make him become familiar directly in the application of the theory. As evidence of this work, the presentation of a portfoliois requested in which he will have kept the problems he has been carrying out. This portfolio will have weekly digital delivery. The student will be able to self-evaluate continuously since he will have the solutions of each one of the sets of problems once finalized the period of delivery. Along with the tutoring hours in case of doubt, it is enough for each student to identify their weaknesses.
- Evaluation of the defense of solutions to problems
At least twice during the course, each student must defend, either orally or in writing, the solutions he has provided as a solution in the problem seminars. This assessment will be individual and may focus on a subset of problems.
Eventually, more in-depth problems can be posed in a non-targeted way to which a computational solution will have to be presented.
The final grade of the course is obtained by combining the assessment of these 3 activities as follows:
Final Note = (0.3 * Contents) + (0.5 * defense of computational solutions) + (0.2 * Portfolio)
Conditions to pass the subject:
The final grade of contents must be greater than or equal to 4 in order to pass the subject.
The grade of the portfolio and the defense of solutions (separately) must be greater than or equal to 6 to be able to pass the subject.
In the event that the grade, applying the formula of the previous section ("final grade of the subject"), was higher than 5 but the minimum required in any of the parts had not been exceeded, the final grade in the record will be a 4.5.
As many honors registrations will be assigned as the current regulations allow as long as the mark is higher than 9.0. The assignment of the registrations will be done following the order of notes. In case there are multiple candidates with the same evaluation likely to receive Md'H, additional activities will be proposed to determine the best candidate (s).
The student will be graded as "Not Evaluable" if he/ she has no evaluated part of either the theoretical or practical contents.
Important notices:
- The dates of continuous evaluation and delivery of works, as well as all the teaching material will be published in the virtual campus (http://cv.uab.cat/), in the space of this subject and can be subject to changes of programming for reasons of adaptation to possible incidents. Cerbero.uab.cat will always be informed about these changes as it is understood that CV will become the usual mechanism for the exchange of information between teachers and students.
- For each evaluation activity, a place, date and time of revision in which the student can review the activity with the teacher will be indicated. In this context, claims may be made on the activity grade, which will be evaluated by the faculty responsible for the subject. If the student does not appear in this revision, it will not be revised later to this activity.
- Notwithstanding other disciplinary measures deemed appropriate, and in accordance with the academic regulations in force, assessment activities will receive a zero whenever a student commits academic irregularities that may alter such assessment. Assessment activities graded in this way and by this procedure will not be re-assessable. If passing the assessment activity or activities in question is required to pass the subject, the awarding of a zero for disciplinary measures will also entail a direct fail for the subject, with no opportunity to re-assess this in the same academic year. Irregularities contemplated in this procedure include, among others:
- the total or partial copying of a practical exercise, report, or any other evaluation activity;
- allowing others to copy;
- presenting group work that has not been done entirely by the members of the group;
- presenting any materials prepared by a third party as one’s own work, even if these materials are translations or adaptations, including work that is not original or exclusively that of the student;
- having communication devices (such as mobile phones, smart watches, etc.) accessible during theoretical-practical assessment tests (individual exams).
The numerical note of the subject will be the lower value between 3.0 and the weighted average of the marks in case the student has committed irregularities in an evaluation act (and therefore the approved by compensation will not be possible). In summary: copy, leave copy or plagiarize (or attempt to) in any of the evaluation activities is equivalent to a FAIL with a grade lower than 3.5.
Assessment Activities
| Title | Weighting | Hours | ECTS | Learning Outcomes |
|---|---|---|---|---|
| Concept tests | 30% | 7 | 0.28 | 1, 2 |
| Problem porfolio | 20 | 5 | 0.2 | 1, 2 |
| Problem solutions defense (code + presentation + follow-up) | 50% | 10 | 0.4 | 1, 2, 3 |
Bibliography
Bibliography
Web links
- Subject web page: http://cv.uab.cat
- Deep Learning. MIT Press book. https://www.deeplearningbook.org
Basic Bibliography
- Deep learning with Python, François Chollet, Manning Publications, 1st Ed., 2017
- Pattern Recognition and Machine Learning, Christopher Bishop, Springer, 2011
- Neural Networks for Pattern Recognition, Christopher Bishop, Oxford University Press, 1st ed., 1996
Software
No special software other than the usual ones will be used in these studies.