The MachineLearning EngineerIssue #54

This week in Issue #54:

• From system 1 to system 2 by Yoshua Bengio
• The Artificial Intelligence Index 2019 Report
• Microsoft’s NLP Best Practices
• The day that changed Netflix tech
• Attention and Augmented RNNs
• Featured OSS Production ML Libraries
• Awesome AI Guidelines to check out this week
• AI conferences
• + more 🚀

Forward the email, or share the online version on 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

If you would like to suggest articles, ideas, papers, libraries, jobs, events or provide feedback just hit reply or send us an email to a@ethical.institute! We have received a lot of great suggestions in the past, thank you very much for everyone’s support!

Yoshua Bengio; Towards system 2

Past progress in deep learning has concentrated mostly on learning from a static dataset, mostly for perception tasks and other System 1 tasks which are done intuitively and unconsciously by humans. However, in recent years, a shift in research direction and new tools such as soft-attention and progress in deep reinforcement learning are opening the door to the development of novel deep architectures and training frameworks for addressing System 2 tasks (which are done consciously), such as reasoning, planning, capturing causality and obtaining systematic generalization in natural language processing and other applications. Yoshua Bengio shared very interesting insights on this NeurIPS talk which covered the key concepts that will enable expansion from System 1 tasks to System 2 tasks.

AI Index 2019 Report

The AI Index, a Stanford-backed initiative to assess the progress and impact of AI, has launched its 2019 report. The new report contains a vast amount of data relating to AI, covering areas ranging from bibliometrics, to technical progress, to analysis of diversity within the field of AI. Jack Clark from OpenAI, who is part of the steering committee outlined some key statistics that include: 300% growth in volume of peer-reviewed AI papers, 800% growth in NeurIPS attendance since 2012, $70b invested worldwide in AI, and more.

Microsoft’s NLP Best Practices

In recent years, natural language processing (NLP) has seen quick growth in quality and usability, and this has helped to drive business adoption of artificial intelligence (AI) solutions. Microsoft has put together a great repository with examples and best practices for building NLP systems, provided as Jupyter notebooks and utility functions. The focus of the repository is on state-of-the-art methods and common scenarios that are popular among researchers and practitioners working on problems involving text and language.

The day that changed Netflix tech

On Christmas Eve 2012, Netflix streaming service experienced an outage. This particular incident got a lot of media coverage for obvious reasons, and was caused due to an AWS region becoming fully unavailable. To mitigate region-based outages, Netflix invested heavily in Resiliency Engineering and Cloud Platform teams to create a discipline to break things on purpose. This post provides really interesting insight on some of the approaches taken to address these issues.

Attention and Augmented RNNs

Recurrent neural networks are one of the staples of deep learning, allowing neural networks to work with sequences of data like text, audio and video. Such models have been found to be very powerful, achieving remarkable results in many tasks including translation, voice recognition, and image captioning. As a result, recurrent neural networks have become very widespread in the last few years. This post provides a great overview on RNNs, together with intuition on some of the core concepts around these.

OSS: Data Stream Processing

The theme for this week’s featured ML libraries is Data Stream Processing. The four featured libraries this week are:

• Apache Flink - Open source stream processing framework with powerful stream and batch processing capabilities.

• Faust - Streaming library built on top of Python’s Asyncio library using the async kafka client inspired by the kafka streaming library.

• Kafka Streams - Kafka client library for buliding applications and microservices where the input and output are stored in kafka clusters

• Spark Streaming - Micro-batch processing for streams using the apache spark framework as a backend supporting stateful exactly-once semantics

If you know of any libraries that are not in the “Awesome MLOps” list, please do give us a heads up or feel free to add a pull request!

OSS: Awesome AI Guidelines

As AI systems become more prevalent in society, we face bigger and tougher societal challenges. We have seen a large number of resources that aim to takle thiese challenges in the form of AI Guidelines, Principles, Ethics Frameworks, etc, however there are so many resources it is hard to navigate. Because of this we started an Open Source initiative that aims to map the ecosystem to make it simpler to navigate. We will be showcasing three resources from our list so we can check them out every week. This week’s resources are:

• UK Government’s Data Ethics Workbook - A resource put together by the Department for Digital, Culture, Media and Sport (DCMS) which provides a set of questions that can be asked by practitioners in the public sector, which address each of the principles in their Data Ethics Framework Principles.
• World Economic Forum’s Guidelines for Procurement - The WEF has put together a set of guidelines for governments to be able to safely and reliably procure machine learning related systems, which has been trialled with the UK government.
• San Francisco City’s Ethics & Algorithms Toolkit - A risk management framework for government leaders and staff who work with algorithms, providing a two part assessment process including an algorithmic assessment process, and a process to address the risks.

If you know of any libraries that are not in the “Awesome MLOps” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a UK-based research centre that carries out world-class research into responsible machine learning systems.

Check out our website