DeepMind Ontol: World's Leading Artificial Intelligence Content
Scientists from DeepMind have compiled an Curated Resource List of educational materials for those who want to connect their lives with AI and machine learning. I call this selection “ontol” - a list of what forms a picture of the world on this issue, ranked by importance and compiled by a living person, a specialist who has a reputation for this list (so that there is no marketing and biased garbage in it).
As planned, if the top ten companies in the field of AI ask their leading specialists (each) to make a selection of the best materials that formed them as specialists, then we will get an array of collections (list of top 10/100 resources + name of the compiler) and based on this, will draw interesting conclusions (a) on the quality of materials, which should be taught first of all b) on the quality of specialists who can highlight the main thing c) something else). So we “mark up” all open texts/videos in the field of AI. Then we will take up other topics: food, trust, the work of life, family, cooperation, cognitive distortions, etc. that form the picture of the world.
Test the prototype beta.ontol.org and subscribe to the channel @Ontol
Table of Contents
Theory and fundamental concepts
Natural Language Processing
Unsupervised Learning and Generative Models
21 Definitions of Fairness and Their Politics (video) - Arvind Narayanan discusses the various definitions of justice and their compromises that they represent for Society.
Fairness and Machine Learning Book (book, video) - An overview of fairness in machine learning topics.
Harvard University's Justice Course (video) - In-depth and exciting lectures on justice and moral philosophy ( Translation ).
NeurIPS 2017 Tutorial on Fairness in Machine Learning (video) - Solon Barokaz and Moritz Hardt discuss in detail the sociotechnical elements of justice in machine learning.
The Trouble with Bias - NeurIPS 2017 (video) - Kate Crawford discusses the ethnic consequences of bias in artificial intelligence systems.
AGI Safety Literature Review (publication) - An excellent review of the literature on general artificial intelligence safety until 2018 with hundreds of links for further study.
AI Alignment newsletter by Rohin Shah - Weekly newsletter summarizing the latest work in the field of artificial intelligence security.
AI safety YouTube channel by Robert MIles (video) - Educational and entertaining videos that introduce the audience to key concepts of common artificial security Intelligence.
Concrete Problems in AI safety (publication) - A useful overview on artificial intelligence security, an initial article that has already become a classic in the field of AI security.
Human Compatible: Artificial Intelligence and the Problem of Control by Stuart Russell ( book) - A must-read book on the safety of artificial intelligence by authorship of the original AI.
Theory and fundamental concepts
3Blue1Brown Youtube channel (video) - A great series of tutorials. Video from scratch on linear algebra and neural networks is especially useful.
A 2020 vision of Linear Algebra (Gilbert Strang, MIT) (video) - Briefly from a new angle summarize the entire course on linear algebra with technical details: how linear algebra is applied in real life, and especially in the field of machine learning.
Andrew Ng's Machine Learning course (online course) - The first very practical and extensive machine learning course. Since the course is on the Coursera platform, your assignments can be evaluated, and assistants and other students can help you with the course material.
Causal Inference in Statistics: A Primer (online preprint) - A great introduction to causal conclusion. This is a preprint of the full version of the latest book.
Causal Inference: What If (online book) - A new book about a causal conclusion.
David MacKay, information theory course videos (video) - Covers a wide range of areas of McKay’s corporate identity.
David MacKay's Course on Information Theory, Pattern Recognition, and Neural Networks (video) Course the legendary David McKay on information theory, data pattern discovery and neural networks.
Decision-theoretic foundations for statistical causality (online article) - An alternative way to formulate causal conclusion operations.
Deep Bayes summer school lectures and lab materials (video) - Lectures and practical exercises on probabilistic modeling and Bayesian training.
Elements of Causal Inference: Foundations and Learning Algorithms (online book) - This book introduces the reader in a simple and accessible way with a causal conclusion.
Essence of Linear Algebra (3blue1brown) (video) - Gives a very good idea of the key ideas in linear algebra without going into into the technical details. Accompanies a traditional linear algebra textbook or college course.
Francis Bach's blog (blog) - Useful tips and tricks, in-depth analysis of various machine learning concepts.
Human intelligence enterprise course (course materials) - History of Human Intelligence.
Is the Abstract Mathematics of Topology Applicable to the Real World? (video) - Introduction is a great introduction to the basics topologies. The workshop convincingly describes specific applications.
KhanAcademy courses (video) - A great introduction for beginners to the statistics, probability theory, and maths needed to understand machine learning.
Learning from Data course - Caltech (video) - A neat introduction to machine learning. A very clear explanation of the topics.
Lecture Notes on Monte Carlo (course notes) - A brief explanation of the Monte Carlo method.
Mathematics for Machine Learning (book) - A great book covering basic math concepts needed for machine learning.
MIT Machine Learning course ( online course) - An excellent 2006 course on the basics (and now history) of machine learning before deep learning and many levels of abstraction became mainstream.
Nando de Freitas Course on Machine Learning (video) - Useful course and presentation on machine learning training.
Princeton Companion to Mathematics (book) - Probably the most striking mathematical source of all. The book provides a detailed overview of the most important concepts in modern mathematics, which does not imply any background in the self-proclaimed format of "bedtime stories" - fascinating, easy to understand and intuitive.
Project Euler (Problem Solving Community) - A series of complex mathematical problems and problems from computer science to activate the brain. They are very interesting to solve, and the acquired knowledge will help you in your career in the field of deep learning.
Statistical Learning Theory course (online course) - A free machine learning course focusing on people with math education Professors Hasti and Tibshirani.
Strang All the Key Ideas of Linear Algebra in 1 Lesson ( video) - Concisely, comprehensively.
The Book of Why (chapters from the book) - An easy introduction to causal conclusion and historical excursion into its development.
Brain Inspired Podcast (podcast) - A podcast that fuses neuroscience and artificial intelligence.
Center for Brains Minds + Machines Summer School Lectures (video) - Lectures from the famous Woods Hole Summer School on Computational * Cognitive * Neurobiology (more about high-level cognition, behavior, and relationships with machine learning).
Computational Cognitive Modeling @ NYU (slides and texts) - An overview of computational approaches to modeling human cognition, closely related to artificial intelligence and machine learning.
Computational models of the neocortex (Class notes) - Interdisciplinary and advanced.
Lectures from Methods in Computational Neuroscience Woods Hole Summer School (video) - Lectures from the famous Woods Hole Summer School on computational * system * neurobiology (more on the cycles and system properties of the brain)
Marr's Levels of Analysis (Vision, 1982, Chapter 1) (chapter from the book) - Perfectly explains by examples on examples of useful algorithms like EM. Serves as a great addition to Bishop's book.
MIT Brains, Minds, and Machines Summer Course (video) - Graduate course at the intersection of cognitive science, neuroscience and artificial intelligence.
Probabilistic Models of Cognition (interactive tutorial) - An interactive tutorial that describes how to use the probabilistic model to create and model human-like behavior.
The challenge of understanding the brain: where we stand in 2015 (publication ) - A good overview of neurobiology in terms of biology.
Theoretical Neuroscience (online book) - A popular introduction to theoretical neurobiology.
Natural Language Processing
A Code-first Introduction to Natural Language Processing (video) - Introduction to Natural Language Processing for people with technical education.
A Primer on Neural Network Models for Natural Language Processing (publication) - A clear overview of what how neural networks are used in natural language processing.
CS224n: Natural Language Processing with Deep Learning (video) - Stanford Course on Modern Natural Language Processing.
NLP Progress (list of datasets and results) - A community-driven website listing a large number of tasks, datasets and modern natural language processing results.
Oxford/DM NLP Course 2017 (lecture course) - Advanced lecture course on natural language text processing, read at Oxford DeepMinder.
Speech and Language Processing (book) - Authoritative reference to natural language processing - now in 3D version and available online.
The Annotated Transformer (blog post) - A great introduction to the dominant natural language processing model.
Amii's Coursera Machine Learning: Algorithms in the Real World Specialization (online course) - Great review on the formation and identification of machine learning problems and their solutions.
Bayesian Reasoning and Machine Learning (online book ) - Fundamentals of probabilistic reasoning and modeling.
David MacKay, Gaussian Process Basics (video) - The most accessible and understandable introduction to the Gaussian process.
David MacKay's book "Information Theory, Inference, and Learning Algorithms" (book) - David McKay presents a unique look on the relationship between information theory, inference and learning. The style of his writing is unique, as is the humor in the book.
Getting into machine learning (blog) - Blog for those who wants to do machine learning.
Lecture notes on Machine Learning (compendium) - Synopsis from Herbert Jager's lectures on machine learning. Describe the many basics and standards of machine learning topics. Very well written (almost like a book).
Machine Learning at UBC 2012 (video) - 2012 Machine Learning Course from the University of British Columbia.
Machine Learning, Probability and Graphical Models (Sam Roweis) (video) - A great explanation of the graphic models by the legendary Sam Roways.
Ranking of ML online courses (list of resources) - A fairly complete overview of top online machine learning courses.
Stanford's Machine Learning Course (video) - Introduction to the Machine Learning Course.
Sunday Classics (resource list) - A collection of classic works on all topics in machine learning, cognitive science, statistics, theory Information, Neurobiology, Artificial Intelligence, Signal Processing, Operations Research, Econometrics, etc.
WEKA: a workbench for machine learning (online resources) - A large set of free software tools for familiarizing with data, data visualization, classification, regression, selection of characteristics and the basis of data science. I constantly use these resources to teach others to see patterns in data and appreciate how much the system can see and use these and more complex patterns.
David MacKay, all videolectures (video) - The name of David McKay is well known in this field, especially in the field of statistics and probabilistic machine learning.
Andrej Karpathy blog/hacker guide (blog entry) - A very accessible introduction to neural networks. He can also find practical tips for life on his blog.
An overview of gradient descent optimization algorithms (blog post) - An exhaustive post reviewing the main options for gradient descent used for optimization neural networks
Chris Olah blog (blog) - Chris Ola's approach can be called very educational for learning key concepts (such as understanding concepts and elements) in machine learning at a deep level. Chris is passionate about education and writes great posts.
Crash Course AI (video) - A useful, well-prepared introductory series. It is likely the best for students and beginners.
CS231: Convolutional Neural Networks for Visual Recognition (Stanford) (video) - Great notes on the link: cs231n.github.io A good continuation of Andrew Un's course, which plunges us much deeper into convolutional neural networks (this was briefly mentioned at the end of the previous course) and Introduces more advanced concepts such as generative models, deep reinforcement.
CS231n: Convolutional Neural Networks for Visual Recognition (Stanford's legendary CNN lectures) (video) - Great overview of both classic and earliest works on convolutional neural networks, which form the basis for most of the work with visual data
Deep Learning at Oxford 2005 (video) - 2015 Oxford Deep Learning Course.
Deep Learning Book (book) - Extensive introduction to the basics of deep learning by some discoverers in this field.
Deep Learning Indaba Practicals (Colabs) - There are tutorials that have been developed and tested on people in for many years, for teaching deep learning from fundamental to advanced topics such as building an automatic differentiation framework or training a generative-competitive network.
Dive into Deep Learning (book) - A great format that turns the study of key machine learning concepts into a fun and interactive experience.
DL + RL course with UCL (video) - This course covered many questions related to deep learning and reinforcement learning. It consisted of two, mostly separate, paths: one in deep learning and one in reinforcement learning, which could be studied separately.
EEML ( first / second edition ) Lab materials (Colabs) - Lectures and practical tasks on probabilistic modeling and Bayesian training.
EEML slides from lectures (slides) - Slides for last year's EEML lectures (unfortunately, there are no entries). They cover a large amount of material from introduction to more complex presentations.
Full Stack Deep Learning (online course) - Deep learning models do not exist in a vacuum. This course covers practical aspects of deep learning, such as implementation model, infrastructure, debugging, and even preparation for deep learning interviews.
Intro to machine learning talk at Lviv workshop ( times , two ) (lectures) - Introduction to machine learning. It introduces a theory on the basis of which a deep learning mechanism can be built.
Khipu videos and practicals + github ( video + slides) - Materials from Khipu - videos and practical exercises for students to complete.
Lilian Weng's blog (blog) - The Lilian blog contains posts on a variety of topics, starting with teaching a curriculum and learning based on self-control, meta-training, etc. The posts themselves are not too detailed, sometimes they go too deep into specialization, but quite often they are updated with new information that appeared after the release of the original post.
MIT 6.S191 Intro to Deep Learning (video and tutorials) - Massachusetts Institute of Technology Introductory Course on Deep Learning and Information Systems.
Online journal (journal) - A peer-reviewed online journal that allows you to create informative visualizations and code, including to facilitate understanding of research papers and increase transparency and reproducibility.
Parallel Distributed Processing (online book) - A classic for everyone who wants to understand the roots of deep learning even in that the moment when it was “connectionism.”
Practical Deep Learning for Coders (online course) - Recommended by friends of other technical specialties (like physics and mathematics) as an excellent introduction to deep learning.
Stanford's NLP with Deep Learning Course - Useful for anyone who wants to start learning about natural language processing.
Sutton and Barto's Reinforcement Learning (tutorial) - This is a tutorial for all reinforcement learning textbooks. It is built from very basic things to advanced topics. Accompanying David Silver's lectures.
Alberta RL 4-course Specialization (online course) - Four consecutive reinforcement training courses, starting with Bandits and ending with Bandits function approximation (NNs), gradient method and average reward.
CS330: Metalearning and Multitask (video) - Provides an overview of recent work in the field of meta-learning and multitasking. An inspirational and very useful video for keeping up with modern ideas in the field.
David Silver, Introduction to Reinforcement Learning (video) - Picks up ideas from the Sutton and Barto tutorial: Why do we should you think about these issues? How do the ideas we have discussed relate to? and so on.
David Silver's RL Course from UCL (video) - Useful for anyone who wants to learn about reinforcement training.
Emma Brunskill RL Course (video) - Video tutorials with reinforcement from Emma Beng's course.
OpenAI blog (blog) - Available presentations of basic and advanced reinforcement learning algorithms.
Reinforcement Learning: an Introduction (2018 edition) (book) - This is the same introductory book on reinforcement learning. Rich perfectly explains the fundamental concepts of reinforcement learning, and also walks with the reader all the way to advanced open research tasks.
UofA/Amii Coursera RL Specilization by White and White (online course) - Project of the University of Alberta - reinforcement learning research center. Adam White is associated with Deep Mind; A holistic and well-designed series of courses that provides the most important reinforcement learning basics.
Spinning Up in Deep RL (code) - An educational resource created by OpenAI facilitates deep learning with reinforcement.
Unsupervised Learning and Generative Models
Ermon's graphical models course at Stanford (compendium) - Covers a large number of probabilistic methods.
How to Use t-SNE Effectively (interactive tutorial) - This is an interactive and in-depth journey to all the pitfalls of using tSNE, has become one of the most used low-dimensional data attachments.
Mathematicalmonk Youtube videos (video) - Awesome explanation using examples of useful algorithms like EM. A great addition to Bishop’s book.
Monte Carlo Gradient Estimation in Machine Learning (publication) - Useful for those with reinforcement or generative modeling training.
Reproducing kernel Hilbert spaces in Machine Learning (course materials) - Suitable for those who interested in generative modeling and more.
Variational inference a feview for statisticians by David Blei (publication) - The best explanation of variational methods in the context of generative modeling.
Chelsea Finn's Multi-Task and Meta-Learning Course (video) - Multitasking video lectures training.
Goodman (1955). The New Riddle of Induction. (chapter from the book) Philosophical premises of inductive bias and why it is difficult to draw conclusions and introduction.
Lex Fridman's AI podcast (video) - Conversations with varied and impressive guest speakers.
Stanford Physics lecture series by Leonard Susskind (video) - A great resource for exploring many important areas of modern physics, including classical, statistical and quantum mechanics. These lectures do not imply great background knowledge; Leonard can introduce and explain complex ideas in an accessible and fascinating manner.
Mike Bostock interactive visualisations - Mike Bostock’s interactive visualizations.
Probability in high dimensions - A clear book about “ideas at the intersection of probability theory, analysis and geometry that arise in a wide range of contemporary issues in various fields. ”
Strogatz nonlinear dynamics course (video) - Video course on nonlinear dynamics.
Thank you Ale Blankmer for the help with the translation.
Learn the details of how to get a sought-after profession from scratch or Level Up in skills and salary by completing SkillFactory paid online courses:
- /Machine Learning Course a>(12 weeks)
- Learning Data Science from scratch (12 months)
- Analyst profession with any starting level (9 months)
- Python for Web Development Course (9 months)
- Trends in the Data Scene 2020
- Data Science has died. Long live Business Science
- Cool Data Scientists do not waste time on statistics
- How to become a Data Scientist without online courses
- Sorting cheat sheet for Data Science
- Data Science for the humanities: what is “data”
- Steroid Data Scenes: Introducing Decision Intelligence