Paradigms of Machine Learning
Code: 106574 ECTS Credits: 6| Degree | Type | Year |
|---|---|---|
| 2504392 Artificial Intelligence | OT | 3 |
| 2504392 Artificial Intelligence | OT | 4 |
Contact
- Name:
- Jordi Casas Roma
- Email:
- jordi.casas.roma@uab.cat
Teaching groups languages
You can view this information at the end of this document.
Prerequisites
It is important to have completed the subjects "Fundamentals of Machine Learning" and "Neural Networks and Deep Learning".
Objectives and Contextualisation
The objective of this subject is to provide comprehensive training in a wide range of methodologies, techniques, and algorithms in reinforcement learning.
Students will learn, implement and use a range of techniques and algorithms covering the most relevant learning methods.
They will derive their own implementations, as well as explore existing libraries, and apply this new knowledge to solve practical problems.
Competences
- Artificial Intelligence
- Act within the field of knowledge by evaluating the social, economic and environmental impact beforehand.
- Analyse and solve problems effectively, generating innovative and creative proposals to achieve objectives.
- Develop strategies to formulate and solve different learning problems in a scientific, creative, critical and systematic way, knowing the capabilities and limitations of the different existing methods and tools.
- Introduce changes to methods and processes in the field of knowledge in order to provide innovative responses to society's needs and demands.
- Students must be capable of collecting and interpreting relevant data (usually within their area of study) in order to make statements that reflect social, scientific or ethical relevant issues.
- Work cooperatively to achieve common objectives, assuming own responsibility and respecting the role of the different members of the team.
Learning Outcomes
- Adapt and reuse existing models in different domains.
- Analyse and solve problems effectively, generating innovative and creative proposals to achieve objectives.
- Identify the social, economic and environmental implications of academic and professional activities for the field of knowledge.
- Propose new ways of measuring the success or failure of implementing innovative proposals or ideas.
- Students must be capable of collecting and interpreting relevant data (usually within their area of study) in order to make statements that reflect social, scientific or ethical relevant issues.
- Understand different machine learning paradigms, and decide on the most appropriate learning method according to the characteristics of the data to be analysed.
- Weigh up the risks and opportunities of both your own and others' proposals for improvement.
- Work cooperatively to achieve common objectives, assuming own responsibility and respecting the role of the different members of the team.
Content
- Introduction and Gymnasium
- Introduction to RL and Gymnasium library.
- Tabular solutions
- Markov Decision Process
- Dynamic Programming
- Monte Carlo
- TD learning
- Approximate solutions
- Deep Q-Networks
- Policy Gradients
- Actor-Critic
- Real-world problems
- Environment design and implementation
Activities and Methodology
| Title | Hours | ECTS | Learning Outcomes |
|---|---|---|---|
| Type: Directed | |||
| Non-evaluable practical activities | 21 | 0.84 | 1, 2, 6 |
| Theory | 21 | 0.84 | 1, 4, 6 |
| Type: Autonomous | |||
| Evaluable practical activities | 42 | 1.68 | 1, 4, 5, 6, 7 |
| Project | 50 | 2 | 1, 2, 4, 5, 6, 8 |
There will be four types of teaching activities:
- Theory sessions
- Evaluable practical exercises
- Theory tests
- Project
Theory sessions: Presentation of the theoretical content of the subject. For each of the topics studied, the main theoretical concepts and mathematical formulation are exposed, as well as the corresponding algorithmic solutions. Additionally, non-evaluable practical exercises could be proposed in order to reinforce the understanding of the topics covered in the theory classes. During these sessions, we will address the resolution of practical exercises (problems) in Python environment.
Evaluable practical exercises: These practical exercises will be carried out during the semester, where students will have to individually solve a set of problems covering the contents of the subject.
Theory test: Theory Moodle test will be carried out individually during the semester.
Project: The project will be carried out during the semester, where students will have to solve a specific problem of certain complexity. The projects will be solved in groups of 2 students. These working groups must be maintained during the development of the whole project and must be self-managed in terms of distribution of roles, work planning, assignment of tasks, management of available resources, conflicts, etc. Each group will work autonomously to develop the project.
The above activities will be complemented by a system of tutoring and consultations outside class hours.
All the information of the subject and the related documents that the students need will be available at the virtual campus (cv.uab.cat).
Annotation: Within the schedule set by the centre or degree programme, 15 minutes of one class will be reserved for students to evaluate their lecturers and their courses or modules through questionnaires.
Assessment
Continous Assessment Activities
| Title | Weighting | Hours | ECTS | Learning Outcomes |
|---|---|---|---|---|
| Individual exams | 40% | 4 | 0.16 | 3, 6 |
| Practical activities | 20% | 4 | 0.16 | 1, 2, 5, 6 |
| Project deliverable | 20% | 4 | 0.16 | 1, 2, 4, 5, 6, 8 |
| Project presentation | 10% | 2 | 0.08 | 2, 4, 6, 7, 8 |
| Theory test | 10% | 2 | 0.08 | 5, 6 |
To assess the level of student learning, a formula is established that combines knowledge acquisition and the ability to solve problems.
Final grade
The final grade is calculated by the following formula:
Final grade = 0.4 * Theory exams + 0.1 * Theory tests + 0.2 * Practical Activities + 0.3 * Project
This formula will be applied as long as the theory and the project grade are >=5.
If the final grade computed through the aforementioned formula is >= 5, but it does not reach the minimum required in any of the evaluation activities, then a final grade will be 4.5.
1. Theory exams
The theory grade aims to assess the individual abilities of the student in terms of the theoretical content of the subject, this is done continuously during the course through two partial exams:
Theory Grade = 0.5 * Grade Exam 1 + 0.5 * Grade Exam 2
The midterm exam (Exam 1) is done in the middle of the semester, and it serves to eliminate part of the subject if it is passed. The final exam (Exam 2) is done at the end of the semester and serves to eliminate the rest of the subject if it is passed.
In order to obtain a final pass theory grade, it will be required for the partial exam grades 1 and 2 to be both >=4.
In case the theory grade does not reach the adequate level to pass, the students can take a recovery exam, destined to recover the failed part (1, 2 or both) of the continuous evaluation process.
2. Theory tests
The aim of the theoretical tests is to deepen the theoretical concepts.
Theory tests = Equal weighting of all course tests
3. Practical Activities
The aim of the practical activities is to become familiar with the practical implementation of the theoretical concepts.
Practical Activities = Equal weighting of all practical exercises in the course
4. Project
The project requires that the students work in groups and design an integral solution to the defined challenge. In addition, the students must demonstrate their teamwork skills and present the results to the class.
Each of the two projects is evaluated through its deliverable and an oral presentation to the class. The participation of students in all activities (preparing the deliverable and presentation) is necessary in order to obtain a project's grade.
The project grade is calculated as follows:
Project = 0.7 * Deliverables + 0.3 * Presentation
If performing the above calculation yields >= 5, but the student did not participate in any of the activities (deliverable, presentation), then a final grade of 4.5 will be given to the project.
There is no recovery of the projects: in case of not presenting a deliverable or a grade does <5, the student will not be able to pass the subject.
Important notes
Notwithstanding other disciplinary measures deemed appropriate, and in accordance with the academic regulations in force, evaluation activities will be suspended with zero (0) whenever a student commits any academic irregularities that may alter such evaluation (for example, plagiarizing, copying, letting copy, etc.). The evaluation activities qualified in this way and by this procedure will not be recoverable. If you need to pass any of these assessment activities to pass the subject, this subject will be failed directly, without opportunity to recover it in the same course.
In case there the student does not deliver any exercise solutions, does not attend any project presentation session during the laboratory sessions and does not take any exam, the corresponding grade will be a "non-evaluable". In another case, the “no shows” count as a 0 for the calculation of the weighted average.
In order to pass the course with honours, the final grade obtained must be equal or higher than 9 points. Because the number of students with this distinction cannot exceed 5% of the total number of students enrolled in the course, it is given to whoever has the highest final marks. In case of a tie, the results of the partial exams will be taken into account.
Bibliography
- Reinforcement Learning: An Introduction (Second edition). R. S. Sutton, A. G. Barto, MIT Press, Cambridge, MA, 2018.
- Deep Reinforcement Learning Hands-On. M. Lapan, Packt Publishing, 2018.
Software
We will use Python, the standard machine learning libraries (such as NumPy, MatPlotLib, SciKit Learn, Pandas, etc) and Gymnasium library for reinforcement learning practical exercises.
Language list
| Name | Group | Language | Semester | Turn |
|---|---|---|---|---|
| (PAUL) Classroom practices | 1 | English | first semester | afternoon |
| (PLAB) Practical laboratories | 1 | English | first semester | afternoon |
| (TE) Theory | 1 | English | first semester | afternoon |