Abstract:
In recent years, Machine Learning models and architectures have become increasingly complex. This growing complexity makes it more difficult to deliver high quality in terms of model performance, robustness and explainability. The introduction of automated evaluations of the trustworthiness of a model is one solution to guarantee that your clients can rely on your model’s predictions. This talk will guide you through the different AI quality pillars, their importance and how to evaluate them.

What you’ll Learn:
– Which are the three AI quality pillars: performance, robustness & explainability
– Why is it important to evaluate the quality and trustworthiness of your ML model
– How can you integrate this evaluation seamlessly into your MLOps workflow

What is unique about this, which can’t be found online?
It gives exclusive insights into a novel quality assessment of ML models – aidkit is the first tool of its kind.

Technical Level: 4

Why is this takeaway(s) important?
Explainable AI and robustness are topics that have caught the attention of ML researchers in recent years and there are few online resources. Being a core contributor to aidkit, I have the expertise needed to explain the quality pillars and give exclusive insights on how to introduce these essential checks in an MLOps workflow.

María is a Machine Learning Research Engineer focused on AI robustness and explainability. She holds two degrees in Mathematics and Physics and is currently working at neurocat GmbH where she contributes to the core of aidkit, an AI quality and trustworthiness evaluation tool. She is also the founder of “NLP en ES 🤗” – the Hugging Face community of Spanish-speaking NLP professionals – and part of the Core Team of Women in AI & Robotics.

Abstract:
There is generally a gap between data scientists who create machine learning models and existing devops tools and processes to put those models into production. This can lead to models not being properly validated and isolated from appropriate business logic and associated monitoring for their proper use. Many models remain as research outputs and do not make it to production.

In this talk we will show how using Tempo (an open source Python SDK) data scientists can easily prepare and test locally their machine learning models and then deploy them either directly or as part of CI/CD and Gitops processes. Data scientists can be involved in the inference logic needed to call their models correctly and easily add other required inference components for their models such as outlier detectors and model explainers as part of their work.

We will illustrate deploying simple models to more advanced outlier detection, model explanation and multi-model ensembles with custom business logic. We will show models locally tested and then pushed to production on Kubernetes with Seldon or Kubeflow’s KFServing.

Data scientists will learn how they can get their work to production faster while ensuring their model is used correctly when it reaches production.

What you’ll Learn:
They will learn about different types of security vulnerabilities when it comes to machine learning and how to handle such issues.

What is unique about this, which can’t be found online?
Machine learning security is not a topic covered often/enough. So, this talk is unique in nature due to the subject matter alone.

Technical Level: 5

Why is this takeaway(s) important?
The end of a machine learning cycle involves taking a model into production where it is used on real world data. Up until this point, most practitioners have considered accuracy, bias, potential data drift, etc. However, security concerns are most likely not a big factor or may even be overlooked. This talk will cover common attacks and how to be prepare for such attacks, which is important for the following takeaway: A model that has been hacked exposes your organization and customers to many harmful outcomes, which should be avoided if at all possible.

Clive is CTO of Seldon. Seldon helps enterprises put machine learning into production. Clive developed Seldon’s open source Kubernetes based machine learning deployment platform Seldon Core. He is also a core contributor to the Kubeflow and KFServing projects.

Abstract:
As one of Canada’s largest grocery store chains, our team runs several ML systems on production. We wanted to take one such system and integrate it with several MLOps tools including MLflow, Seldon Core, and Feast on Google Cloud Platform. The goal is to reduce overall development time of a model from EDA to deployment, as well as enable end to end tracking of these ML pipelines.

In this talk, we will share our experience on setting up a recommender system using the tools mentioned. We will also talk about our overall platform architecture and how the MLOps tools fit into the end-to-end ML pipeline.

What you’ll Learn:
How to put a ML system on production using some of the most commonly used MLOps tools and how MLOps fit into the end to end ML pipeline.

What is unique about this, which can’t be found online?
This talk focuses on a specific e-commerce machine learning problem that we are solving here at Loblaw Digital using open source MLOps tools

Technical Level: 7

Why is this takeaway(s) important?
There are several MLOps tools out there but it is important to evaluate how they integrate together in production and what could be the challenges that arise when we go from EDA to Model Serving.

Mefta Sadat is a Senior ML Engineer at Loblaw Digital. He has been here for three years building the Data Engineering and Machine Learning platform. He focuses on productionizing ML services, tools and data pipelines. Previously Mefta worked at a Toronto based Video Streaming Company and designed and built the recommendation system for the Zoneify App from scratch. He received his MSc in Computer Science from Ryerson University focusing on research to mitigate risk in Software Engineering using ML.

Abstract:
Data Scientist has become the “sexiest” job title in the 21st century. However, we see significant challenges, especially in traditional industries, regarding model adoption and deploying ML to production. This is due to ML being mostly in academia for the last 70 years and not a lot of thought that went into operationalization of ML. A lot of Data Scientists are traditionally more focused on scientific approach (scripting and testing models) without considering operational aspects of it. We need to realize there must be more engineering rigour in developing ML models vs just calling model.fit. It’s also not as simple as taking everything from software development. When software development initially came from academia there were no proper development frameworks and early adopters borrowed traditional engineering processes (waterfall!). It took a long time to invent agile. I believe we have a similar situation with ML where people think we can just apply software development practices to ML development, whereas not everything is directly applicable. But we can take the best practices and build upon them. In this talk we will take a look at those challenges and explore potential solutions with switching model development into hands of ML Developers/Engineers that are focused on developing machine learning software systems (from scratch – data source) keeping in mind that it will be deployed on production.

What you’ll Learn:
Best practices of ML model development. We need to realize there must be more engineering rigour in developing ML models for them to be used in industry.

Technical Level: 3

Marcin holds a PhD in Software Engineering and Intelligent Systems from the University of Alberta, Canada, and an MSc in Applied Computing Science from AGH University of Science and Technology, Poland. He has published over 30 scientific, peer-reviewed publications and has over a decade of practical academic and industry experience with Software Engineering, Machine Learning, Big Data, and Predictive Analytics. Marcin has a passion for building and growing high performing Data Science teams, and for managing and coordinating Data Science, ML, and Big Data products. Currently, he is leading Data Science initiatives at AltaML as VP of Data Science. Marcin is also highly involved in Edmonton’s Data Science community, has co-founded the Edmonton DS meetup, and is teaching Data Science at North Alberta Institute of Technology.

Abstract:
How can a feature store help with MLOps?
In this talk, we introduce how a Feature Store for Machine Learning (ML) can decompose end–to-end ML pipelines into
(1) a feature pipeline that takes raw data from data warehouses, a data lake, and operational data stores and transforms it into features that are cached in the feature store, and
(2) model training/validation/deployment pipelines. Both pipelines have different requirements, often favor different technologies (Spark for feature pipelines, Python for model training), can run at different cadences, and are even managed by different teams (data engineering, and data scientists or ML engineers). The benefits of the Feature Store for ML architecture will be elucidated throughout the talk.

What you’ll Learn:
The state-of-the-art in structure ML pipelines with a Feature Store

What is unique about this, which can’t be found online?
We have experience running ML Pipelines in production with large companies and can share some of our recent experiences in when to use Spark or Python for feature engineering. When to use Deequ vs Great Expectations. Which open-source feature store is best for you?

Technical Level: 4

Jim Dowling is CEO of Logical Clocks and an Associate Professor at KTH Royal Institute of Technology. He is lead architect of the open-source Hopsworks platform, a horizontally scalable data platform for machine learning that includes the industry’s first Feature Store.

Abstract:
In this talk, I will present two recently developed tools for ML optimization: Skyline and RL-Scope. Skyline: Interactive in-editor computational performance profiling, visualization, and debugging for PyTorch deep neural networks. RL-Scope is a cross-stack profiler for deep reinforcement learning workloads.

What you’ll Learn:
The availability of tools for efficient optimization for DNN and RL models

Technical Level: 5

Gennady Pekhimenko is an Assistant Professor at the University of Toronto, CS department and (by courtesy) ECE department, where he is leading the EcoSystem (Efficient Computing Systems) group. Gennady is also a Faculty Member at Vector Institute and a CIFAR AI chair. Before joining Univ. of Toronto, he spent a year in 2017 at Microsoft Research in Redmond in Systems Research group. He got his PhD from the Computer Science Department at Carnegie Mellon University in 2016. Gennady is a recipient of Amazon Machine Learning Research Award, Facebook Faculty Research Award, Connaught New Researcher Award, NVIDIA Graduate, Microsoft Research, Qualcomm Innovation, and NSERC CGS-D Fellowships. His research interests are in the areas of systems, computer architecture, and applied machine learning.

Abstract:
In this talk we’ll share Ecolab’s journey with MLOps, and how this global hygiene technologies provider built a cutting-edge, cloud data science architecture which enables them to deploy new AI services every 30-90 days, to address risks before they occur.
Building scalable AI applications that generate value in real business environments require not just advanced technologies, but also better processes for data science, engineering and DevOps teams to collaborate effectively.

What you’ll Learn:
In this session we’ll touch upon the technology – and also the processes that need to be put in place to break the silos and efficiently implement MLOps across the enterprise

Yaron Haviv is a serial entrepreneur who has been applying his deep technological experience in data, cloud, AI and networking to leading startups and enterprise companies since the late 1990s. As the co-founder and CTO of Iguazio, Yaron drives the strategy for the company’s data science platform and leads the shift towards real-time AI. He also initiated and built Nuclio, a leading open source serverless platform with over 3,400 Github stars and MLRun, Iguazio’s open source MLOps orchestration framework.

Prior to co-founding Iguazio in 2014, Yaron was the Vice President of Datacenter Solutions at Mellanox (now NVIDIA), where he led technology innovation, software development and solution integrations. He was also the CTO and Vice President of R&D at Voltaire, a high-performance computing, IO and networking company which floated on the NYSE in 2007. Yaron is an active contributor to the CNCF Working Group and was one of the foundation’s first members. He presents at major industry events and writes tech content for leading publications including TheNewStack, Hackernoon, DZone, Towards Data Science and more.

Abstract:
The number of components and level of sophistication in end-to-end ML pipelines can vary from problem to problem but there’s one common element that is the key to make the whole system great and useful: your training data. The more time you spend developing the training dataset in your ML pipeline, the more positive results you’ll get. In this talk we’ll present the use case of a text classification pipeline we developed from scratch to integrate with one of our products. We’ll show details of how we designed an appropriate classification taxonomy, a consistent annotated training dataset, and how the end-to-end pipeline was pieced together to deploy a BERT-based model in a low latency real-time text classification system.

What you’ll Learn:
Audiences will learn strategies to develop a training dataset, particularly a nlp ready dataset, from scratch and how to connect it with the rest of the ML pipeline to train, monitor and deploy models for production use.

What is unique about this, which can’t be found online?
Most resources found online provide generic guides to develop and deploy language models using custom dataset. We intend fo give a very specific example that can be used in other contexts and domains.

Technical Level: 4

Why is this takeaway(s) important?
Recently, the availability of great pre-trained ML models ready to be fine-tuned with custom datasets has moved the focus of model selection to data development and curation. This makes end-to-end ML pipelines center around making sure your data is in its best possible shape and less on making sure you’re using the most optimal version of a chosen model. The use case I present in the context of NLP will highlight this data centricity as the key to achieving great model performance.

Diego Castaneda is a Data Scientist with a background in computational astrophysics. I currently support the needs of the messaging data team at Shopify.

Jennifer Bader is a Content Strategist. She works at Shopify, an e-commerce platform that makes it easier for people to start, run, and grow a business.

She joined Shopify to lead content strategy for Kit, Shopify’s AI-powered virtual employee. She now works on the Messaging team, which gives entrepreneurs tools to connect with their customers and collaborate with their team.

Jennifer has worked in UX content design and communications for the last 20 years. Before Shopify, she worked for OpenTable, IDEO, and a technology commercialization institute.

She’s passionate about the intersection of data science and content design. And using conversational interfaces to humanize e-commerce.

Abstract:
Hyperparameter optimization is a ubiquitous task, mostly treated as an expensive black-box optimization problem with high resource consumption. This talk will introduce MSR technologies to perform low-cost and effective hyperparameter optimization. We build FLAML: a fast and lightweight AutoML library to automatically find accurate machine learning models at low cost. It significantly outperforms top-ranked AutoML libraries on a large open source AutoML benchmark under equal, or sometimes orders of magnitude smaller budget constraints.

What you’ll Learn:
FLAML is a lightweight Python library that finds accurate machine learning models automatically, efficiently and economically. It frees users from selecting learners and hyperparameters for each learner. It is fast and economical. The simple and lightweight design makes it easy to extend, such as adding customized learners or metrics. FLAML is powered by a new, cost-effective hyperparameter optimization and learner selection method invented by Microsoft Research. It significantly outperforms top-ranked AutoML libraries on a large open source AutoML benchmark under equal, or sometimes orders of magnitude smaller budget constraints.

Technical Level: 2

Chi Wang is a principal researcher in Microsoft Research @ Redmond. His research is centered around the interplay between theories and systems related to data science and data platforms, including AutoML, ML for systems, and scalable data science. He received AMiner Most Influential Scholar Award in 2018 and 2016, and SIGKDD Data Science/Data Mining PhD Dissertation Award in 2015.

Dr. Qingyun Wu is currently a postdoc researcher at Microsoft Research, the New York City lab. Her research interests include AutoML and reinforcement learning. Qingyun will be joining Penn State University as an assistant professor in the fall of 2021.

Abstract:
Building and operating ML applications requires different infrastructure from traditional software, which has led to the development of “ML platforms” specifically designed to build and manage ML applications. In this talk, I’ll discuss some of the common challenges in productionizing ML based on experience building MLflow, an open source ML platform started at Databricks. MLflow is now the most widely used open source project in this area, with over 4 million downloads per month and integrations with dozens of other products. I’ll also highlight some interesting problems users face at scale, such as the need for versioning and reproducibility on petabyte-scale datasets, the impact of privacy and interpretability regulations on ML, and “hands-free” ML use cases that can train thousands of models without direct tuning from the ML developer. I’ll show how ongoing work in MLflow and the open source ecosystem around it (Delta Lake, PyTorch, PyCaret, Apache Spark, etc) is helping to tackle these problems.

What you’ll Learn:
Challenges running ML in production, based on our experience at thousands of enterprise customers, and features available in MLflow and its open source integrations.

Technical Level: 5

Matei Zaharia is a Cofounder and Chief Technologist at Databricks and an Assistant Professor of Computer Science at Stanford, where he does research on machine learning and distributed systems. Matei started the widely used Apache Spark and MLflow open source projects, and has contributed to other data and AI infrastructure projects such as Delta Lake and Koalas. Matei’s work was recognized through the 2014 ACM Doctoral Dissertation Award for the best PhD dissertation in computer science and the US Presidential Early Career Award for Scientists and Engineers (PECASE).

Abstract:
Released last year, the O’Reilly book Machine Learning Design Patterns captures best practices and solutions to recurring problems in machine learning. The authors, three Google engineers, catalog proven methods to help data scientists tackle common problems throughout the ML process. These design patterns codify the experience of hundreds of experts into straightforward, approachable advice. In this talk, Sara will dive into three patterns from the book focused on MLOps.

What you’ll Learn:
After this talk, attendees will have a better understanding of how to approach MLOps for their own ML tasks.

Technical Level: 5

Sara is a Senior Developer Advocate for Google Cloud, focusing on machine learning. She inspires developers and data scientists to integrate ML into their applications through demos, online content, and events. Sara also co-authored the O’Reilly book Machine Learning Design Patterns. Before Google, she was a Developer Advocate on the Firebase team. When she’s not writing code, she can be found on a spin bike or eating frosting.

Abstract:
For any organization whose core product or business depends on ML models (think Slack search, Twitter feed ranking, or Tesla Autopilot), ensuring that production ML models are performing with high efficacy is crucial. In fact, according to the McKinsey report on model risk, defective models have led to revenue losses of hundreds of millions of dollars in the financial sector alone. However, in spite of the significant harms of defective models, tools to detect and remedy model performance issues for production ML models are missing.

In this talk, we discuss why model monitoring matters, what do we mean by model monitoring, and considerations when setting up a model monitoring system.

What you’ll Learn:
Why model monitoring matters, what do we mean by model monitoring, and considerations when setting up a model monitoring system.

What is unique about this, which can’t be found online?
Model Monitoring is an emerging field with lots of confusion around what it entails. This talk will simplify the concepts and make them accessible to the audience.

Technical Level: 4

Why is this takeaway(s) important?
AI & ML models are being used in a variety of applications but there is inadequate monitoring in place to ensure that models work as expected. This session will raise awareness for model monitoring, what it entails and what the audience can do today to get started.

Manasi Vartak is the founder and CEO of Verta, an MIT spinoff building an open-core MLOps platform for the full ML lifecycle. Verta grew out of Manasi’s Ph.D. work at MIT on ModelDB, the first open-source model management system deployed at Fortune 500 companies. The Verta MLOps platform enables data scientists and ML engineers to robustly take trained ML models through the end-to-end MLOps cycle including versioning, packaging, release, operations, and monitoring. Previously, Manasi worked on feed ranking at Twitter and dynamic ad-targeting at Google. Manasi has spoken at several top research as well as industrial conferences such as the Strata O’Reilly Conference, SIGMOD, VLDB, Spark Summit, and AnacondaCON, and has authored a course on model management.

Abstract:
What is it like to build an ML platform during the pandemic, with a new team, at a new company?

DoorDash’s mission is to grow and empower local economies. As DoorDash’s business grows, it is essential to establish a centralized ML platform to accelerate the ML development process and to power the numerous ML use cases.

This presentation will detail the DoorDash ML platform journey during the pandemic, which includes the way we establish a close collaboration and relationship with the Data Science community, how we intentionally set the guardrails in the early days to enable us to make progress, the principled approach of building out the ML platform while meeting the needs of the Data Science community, and finally the technology stack and architecture that powers billions of predictions per day and supports a diverse set of ML use cases.

What you’ll Learn:
Building an ML platform during the pandemic is possible, but it requires a systematic approach and creative ideas.

Technical Level: 3

Hien Luu is a Sr. Engineering Manager at DoorDash, leading the Machine Learning Platform team. He is particularly passionate about the intersection between Big Data and Artificial Intelligence. He is the author of the Beginning Apache Spark 2 book. Teaching is one his passions and he is currently teaching Apache Spark course at UCSC Silicon Valley Extension school. He has given presentations at various conferences like QCon SF, QCon London, Hadoop Summit, JavaOne, ArchSummit and Lucene/Solr Revolution.

Dawn Lu is a senior data scientist at DoorDash, a technology company that empowers merchants to grow their businesses by offering on-demand delivery. She focuses primarily on developing and deploying machine learning models that power DoorDash’s logistics engine, including food ready time predictions, supply & demand forecasting, as well as pay algorithms. Before joining the technology industry, she worked as a strategy consultant — and during this time she discovered her passion for using quantitative approaches to better understand and predict consumer behavior. She holds a Bachelor’s degree in Economics from Yale University.

Abstract:
The recent paradigm shift of using data and algorithms to produce code (in the shape of models), called “Software 2.0”, has seen an acceleration of investment into various tools and platforms across the entire industry. Startups, cloud providers, internal platform at large companies: everyone needs an MLOps platform.

What you’ll Learn:
What it means to build an MLOps platform for AV.

Nico is a Sr Director of AI Infrastructure at NVIDIA. He is currently managing the team delivering the MLOps platform that enables the development of Autonomous Vehicles software for NVIDIA partners. Previously, Nico was managing the Cortex team at Twitter, responsible for delivering the deep learning platform for Twitter.

Abstract:
Graph neural networks (GNNs) research has surged to become one of the hottest topics in machine learning in the last years. GNNs have seen a series of recent successes in problems from the fields of biology, chemistry, social science, physics, and many others. So far, GNN models have been primarily developed for static graphs that do not change over time. However, many interesting real-world graphs are dynamic and evolving in time, with prominent examples including social networks, financial transactions, and recommender systems. In many cases, it is the dynamic behavior of such systems that conveys important insights, otherwise lost if one considers only a static graph. This talk will discuss Temporal Graph Networks, a recent and general approach for machine learning over dynamic graphs.

What you’ll Learn:
What dynamic graphs are, that they are ubiquitous in applications, and how to learn on dynamic graphs

Technical Level: 5

Emanuele is currently a Machine Learning Researcher at Twitter, as well as a PhD student at Imperial College. He obtained his Bachelor degree from Imperial College and his Masters Degree from Cambridge University. His research focuses on Graph Neural Networks, in particular developing new models that can handle dynamic graphs, i.e. graphs that change over time.

Abstract:
In this talk we will examine the Mayflower project — its goals; its breakthroughs; its technology — and highlight it as an example of how AI is being brought to the edge.

What you’ll Learn:
Edge Computing

Technical Level: 5

Rob High is the Vice President and Chief Technology Officer for the IBM Edge Computing portfolio. High has architecture and technical strategy responsibility for offerings that will enable a new generation of edge computing — where work is placed close to where data is produced and actions are performed. Prior to this assignment he was the CTO for IBM Watson, responsible for setting the technical strategy and vision for IBM’s leadership in AI computing. High is recognized as a global technical leader, author and evangelist, and was named an IBM Fellow in 2008 for his pioneering work in the field of SOA and for his leadership on WebSphere.

Abstract:
As more businesses adopt AI, upcoming AI regulations are forcing companies to build responsible and ethical AI. Previously, many practitioners thought validating a model with Explainable AI before deployment was enough. Not anymore. With more AI solutions in place, firms are putting governance at the forefront by enforcing it throughout the AI lifecycle to ensure responsible and ethical AI development. In this session, we introduce a Model Performance Management framework, its advantages in the machine learning life cycle and providing a continuous feedback loop to establish a responsible AI practice.

What you’ll Learn:
Using Model Performance Management to improve models more efficiently and how AI explainability enables MPM at the core.

Krishna is the co-founder and CEO of Fiddler, a Machine Learning Monitoring company that empowers companies to efficiently monitor and troubleshoot ML models with Explainable AI. Prior to founding Fiddler, Gade led the team that built Facebook’s explainability feature ‘Why am I seeing this?’. He’s an entrepreneur with a technical background with experience creating scalable platforms and expertise in converting data into intelligence. Having held senior engineering leadership roles at Facebook, Pinterest, Twitter and Microsoft, he’s seen the effects that bias has on AI and machine learning decision making processes, and with Fiddler, his goal is to enable enterprises across the globe solve this problem.

Abstract:
Data owners often desire to publish a synthetic data instance that ensures the privacy of individual records, but is similarly useful as the original sensitive data. Existing differentially private data synthesis methods usually fail in preserving one of the most fundamental data properties: the underlying correlations and dependencies among tuples and attributes (i.e., the structure of the data), compromising the utility of the output to downstream tasks.

With probabilistic database models and show how to: (i) privately learn a probabilistic model based on the original private data (how data was generated); and (ii) sample a new synthetic private instance based on the learned model. We leverage the HoloClean framework to learn the underlying data distribution, and we show that learning this model privately is technically challenging.

In this talk I briefly describe the HoloClean framework for structured data prediction, and Kamino a system that learns HoloClean models privately and use them to synthesize “useful” private data instances.

What you’ll Learn:
They will learn about different types of security vulnerabilities when it comes to machine learning and how to handle such issues.

What is unique about this, which can’t be found online?
Machine learning security is not a topic covered often/enough. So, this talk is unique in nature due to the subject matter alone.

Technical Level: 5

Why is this takeaway(s) important?
The end of a machine learning cycle involves taking a model into production where it is used on real world data. Up until this point, most practitioners have considered accuracy, bias, potential data drift, etc. However, security concerns are most likely not a big factor or may even be overlooked.

This talk will cover common attacks and how to be prepare for such attacks, which is important for the following takeaway: A model that has been hacked exposes your organization and customers to many harmful outcomes, which should be avoided if at all possible.

Ihab Ilyas is a professor in the Cheriton School of Computer Science and the NSERC-Thomson Reuters Research Chair on data quality at the University of Waterloo. His main research focuses on the areas of big data and database systems, with special interest in data quality and integration, managing uncertain data, machine learning for data curation, and information extraction.

Ihab is a co-founder of Tamr, a startup focusing on large-scale data integration, and he is also the co-founder of inductiv (acquired by Apple), a Waterloo-based startup on using AI for structured data cleaning. He is a recipient of the Ontario Early Researcher Award, a Cheriton Faculty Fellowship, an NSERC Discovery Accelerator Award, and a Google Faculty Award, and he is an ACM Fellow.

Abstract:
At Avast we complete over 17 million phishing detections a day, providing crucial online protection for this type of attacks. There are a lot of challenges that data scientists frequently face, such as inconsistent environments, transitioning from research to production, access management, model deployment.

In this talk Joao Da Silva will present Avast MLOps maturity journey, tooling and cultural shift that has increased velocity, improved collaboration and brought structure for productising machine learning whilst integrating model tracking, storage, cross-system orchestration and E2E model deployments for complete and modern machine learning pipelines

Joao Da Silva will deliver a presentation that anyone in any organisation can relate to and take notes on a real journey with regards to MLOps tooling, mindset and adoption in a large organisation.

What you’ll Learn:
Audience will learn about a real journey in improving MLOps mindset and maturity levels in a organisation whilst proving information on the tooling and strategies created for it to become a reality.

What is unique about this, which can’t be found online?
It’s the true MLOps strategy, Architecture and platform of Avast which came to fruition after initial adopting of principles and MATS stack as the first step into productising anti-virus and phishing protection research

Technical Level: 4

Why is this takeaway(s) important?
As part of our journey we identified that Avast needed to shift it’s mindset and culture around ML delivery, it’s also true from my experience from our local that many organisations are still reluctant to adopt a MLOps strategy. This talk will present that with a MLOps maturity level based strategy, minor adjustments and small investment MLOps mindset can increase their output and ML quality greatly

Public speaker in spark-summit, data-summit and local ML Meetups as well as Meetup co-organizer in Prague. I have been involved in the Data engineering field for a while and I am a FP lover. I am currently Architecting MLOps platform and strategy for Avast so we can provide a safe online environment to the wider population.

Abstract:
Advances in machine learning and big data are disrupting every industry. However, even when companies deploy to production, they face significant challenges in staying there, with performance degrading significantly from offline benchmarks over time, a phenomenon known as performance drift. Models deployed over an extended period of time often experience performance drift due to changing data distribution.

In this talk, we discuss approaches to mitigate the effects of performance drift, illustrating our methods on a sample prediction task. Leveraging my experience at a startup deploying and monitoring production-grade ML pipelines for predictive maintenance, we also address several aspects of machine learning often overlooked in academia, such as the incorporation of non-technical collaborators and the integration of machine learning in an agile framework.

What you’ll Learn:
This talk will consist of:
– Using Python and open-source datasets to demonstrate an example of training and validating a model in an offline setting that subsequently experiences performance degradation when it is deployed.
– Using Prometheus, Grafana, and mltrace to show how one can build tools to monitor production pipelines and enable teams of different stakeholders to quickly identify performance degradation using the right metrics.

This talk will be a slideshow presentation accompanied by a Python notebook demo. It is aimed towards engineers that deploy and debug models in production, but may be of broader interest for people building machine learning-based products, and requires a familiarity with machine learning basics (train/test sets, decision trees).

Technical Level: 5

Shreya is a computer scientist living in San Francisco interested in making machine learning work in the “real world.” She is an incoming PhD student at UC Berkeley. Previously, she was the first ML engineer at Viaduct, did ML research at Google Brain, and completed her BS and MS in computer science at Stanford.

Abstract:
There are several known attacks against ML models that can lead to altered, harmful model outcomes or exposure of sensitive training data. Unfortunately, traditional model assessment measures don’t tell us much about whether a model is secure. In addition to other debugging steps, it may be prudent to add some or all of the known ML attacks into any white-hat hacking exercises or red-team audits your organization is already conducting. This talk will go over common machine learning security attacks and the remediation steps an organization can take to deter these pitfalls.

What you’ll Learn:
They will learn about different types of security vulnerabilities when it comes to machine learning and how to handle such issues.

What is unique about this, which can’t be found online?
Machine learning security is not a topic covered often/enough. So, this talk is unique in nature due to the subject matter alone.

Technical Level: 5

Why is this takeaway(s) important?
The end of a machine learning cycle involves taking a model into production where it is used on real world data. Up until this point, most practitioners have considered accuracy, bias, potential data drift, etc. However, security concerns are most likely not a big factor or may even be overlooked. This talk will cover common attacks and how to be prepare for such attacks, which is important for the following takeaway: A model that has been hacked exposes your organization and customers to many harmful outcomes, which should be avoided if at all possible.

Navdeep Gill is a Lead Data Scientist, Team Lead at H2O.ai where he leads efforts around Responsible AI and previously focused on GPU accelerated machine learning, automated machine learning, and the core H2O-3 platform.

Prior to joining H2O.ai, Navdeep worked at Cisco focusing on data science and software development. Before that Navdeep was a researcher/analyst in several neuroscience labs at the following institutions: California State University, East Bay, University of California, San Francisco, and Smith Kettlewell Eye Research Institute.

Navdeep graduated from California State University, East Bay with a M.S. in Computational Statistics, a B.S. in Statistics, and a B.A. in Psychology with a minor in Mathematics.

Michelle is a Data Scientist and Product Manager of the AI App Store at H2O.ai. Her background is in pure math and computer science and she is passionate about applying these skills to answer real world questions. When not coding or thinking of analytics, Michelle can be found hanging out with her dog or playing rocking out with the band.

Abstract:
The day an ML application is deployed to production and begins facing the real world invariably begins in triumph and ends in frustration for the model builder. The joy of seeing accurate predictions is quickly overshadowed by a myriad of operational challenges that arise, from debugging to troubleshooting to monitoring. In DevOps, analogous software operations have long been refined into an artform. Sophisticated tools enable engineers to quickly identify and resolve issues, continuously improving software stability and robustness. By contrast, in the ML world, operations are still done with Jupyter notebooks and shell scripts. One of the cornerstones of the DevOps toolchain is logging. Traces and metrics are built on top of logs, thus enabling monitoring and feedback loops. What would a good logging tool look like in an ML system? In this talk, we present a powerful new tool—a logging library—that enables data logging for AI applications. We discuss how this solution enables testing, monitoring and debugging of both an AI application and its upstream data pipeline(s). We offer a deep dive into some of the key properties of the logging library that enable it to handle TBs of data, run with a constraint memory footprint, and produce statistically accurate log profiles of structured and unstructured data. Attendees will leave the talk equipped with best practices to supercharge MLOps in their teams.

What you’ll Learn:
How to enable logging in data & ml applications in order to unlock MLOps activities such as testing, debugging, and monitoring.

With a joint background in data science and engineering, Gulsen leads data product and AI/ML efforts globally across Udemy, including the development of Udemy’s catalog search, recommendations, knowledge graph, and personalization technologies as well as ML platform. She has 15+ years of experience in the field of software engineering and management with a particular focus on ML applications and platforms for the last 7+ years. She has 4+ years of experience in the field of data science including hands-on experience in search and recommendations data science. Prior to Udemy, Gulsen worked in various areas including project management, full-stack development, game development, and mobile applications.

Alessya Visnjic is the CEO and co-founder of WhyLabs, the AI Observability company on a mission to build the interface between AI and human operators. Prior to WhyLabs, Alessya was a CTO-in-residence at the Allen Institute for AI (AI2), where she evaluated commercial potential for the latest advancements in AI research. Earlier in her career, Alessya spent 9 years at Amazon leading Machine Learning adoption and tooling efforts. She was a founding member of Amazon’s first ML research center in Berlin, Germany. Alessya is also the founder of Rsqrd AI, a global community of 1,000+ AI practitioners who are committed to making AI technology Robust & Responsible.

Abstract:
This talk describes Marius, a software system that aims to make scaling of modern AI models over billion-edge graphs dramatically easier. Marius focuses on a key bottleneck in the development of machine learning systems over large-scale graph data: data movement during training. Marius addresses this bottleneck with a novel data flow architecture that maximizes resource utilization of the entire memory hierarchy (including disk, CPU, and GPU memory). Marius’ no-code paradigm allows users to simply define a model and enjoy resource-optimized training out of the box. This talk will describe how Marius can train deep learning models over graphs with more than a billion edges and 550GB of total parameters 10x faster and 5x cheaper than competing industrial systems.

What you’ll Learn:
1) How we can enable training of large-scale machine learning models over sparse structured data in a resource-optimizer manner
2) Learn about Marius and its functionalities

Technical Level: 5

Theodoros (Theo) Rekatsinas is an Assistant Professor in the Department of Computer Sciences at the University of Wisconsin-Madison. His work focuses on machine learning methods and systems over structured data. Theo is also a co-founder of Inductiv (now part of Apple), which developed technology that uses artificial intelligence to automate processes that involve identifying and correcting errors in data.

Abstract:
At Udemy, multi-faceted growth created different challenges in terms of ML platform and tooling as well as the related processes. One such challenge was to come up with a scalable platform to train and execute different types of ML models in real-time or batch. We overcame this particular challenge by building generic components that increased reuse and led to faster delivery and lower maintenance costs. Another challenge was to find a good way to efficiently serve the needs of the different parts of the organization despite their varying requirements. For instance, distributed applications cannot be avoided for many of our use cases that deal with web-scale data, but there are still a significant number of cases which can be more efficiently handled without the complexities of distributed applications.

To overcome this challenge, we had to unify the frameworks (and tooling whenever possible) for both cases, mainly to decrease the learning curve for data scientists and increase the maintainability.
Another challenge we faced was the need for increased focus on developer and data science ergonomics as the organization grew.

In this talk, we will present an overview of the main ML-related challenges we faced at Udemy as the company experienced explosive growth in users, product complexity, and organization size. We will also present best practices that we developed and outstanding problems in our current state.

What you’ll Learn:
1) Best practices for ML operations in different company sizes
2) Challenges for ML-related areas in real-life settings

What is unique about this, which can’t be found online?
The content is based on years of observations and trial-and-error experiments in a fast-growing company. Given that every stage of growth introduced different challenges, the experience conveyed in this talk will be a unique one.

Technical Level: 3

With a joint background in data science and engineering, Gulsen leads data product and AI/ML efforts globally across Udemy, including the development of Udemy’s catalog search, recommendations, knowledge graph, and personalization technologies as well as ML platform. She has 15+ years of experience in the field of software engineering and management with a particular focus on ML applications and platform for the last 7+ years. She has 4+ years of experience in the field of data science including hands-on experience in search and recommendations data science. Prior to Udemy, Gulsen has worked in various areas including project management, full-stack development, game development, and mobile applications.

Sam Cohan is a Principal Machine Learning Engineer at Udemy. Sam’s current focus at Udemy is enabling efficient workflows for building data products. Prior to joining Udemy, Sam was an early employee and founding team member at multiple data-heavy startups in the Bay Aarea ranging from revenue intelligence to mobile advertising to crypto trading. Sam is a big believer in the power of building teams that can blur the lines between engineering and data science. Sam has a degree in Computer Engineering from the University of Waterloo and a Masters in Financial Engineering from UC Berkeley.

Abstract:
At Yelp, we use ElasticSearch(ES) to power most of our Search. However, the process for updating a model or replacing it was slow and error-prone; engineers needed to spend time implementing any new feature transformations when training and serving, as well as ensuring parity between the two. We improve this situation by building an ES plugin that integrates neatly into Yelp’s Model Platform. Spark Pipelines trained and stored as MLeap bundles to MLFlow using the Model Platform can now be deployed directly to ES as MLeap pipelines. These Spark/MLeap pipelines encapsulate not only the ML model itself but also it’s feature engineering pipeline. Subsequently, this allows the ES plugin to swap one modeling pipeline for another as long as the base features for their pipelines are available through the ES index!

In this talk, we will discuss this ES plugin, as well as lessons learnt in making model pipelines performant.

What you’ll Learn:
The audience will walk through a working framework for easy training and deployment of ML models for online ranking tasks. They will learn about pitfalls we ran into when trying to make our models more performant.

What is unique about this, which can’t be found online?
A framework for quick model deployments when powering search. Experiences in making model pipelines more performant.

Technical Level: 4

Why is this takeaway(s) important?
ML models for online ranking tasks are used widely. However, automated frameworks for easy training and deployment of such models are not.

Software Engineer with the ML Infrastructure team at Yelp. PhD with a focus on Randomized and Distributed Algorithms from the University of Iowa.

Abstract:
MLOps and ModelOps are different. In this session, we will cover how ModelOps not only encompasses the basic model monitoring and tuning capabilities of MLOps, but also includes a continuous feedback loop and 360 view of your models throughout the enterprise, providing reproducibility, compliance and auditability of all your business critical models.

What you’ll Learn:
You will learn about good practices for:
– Continuous model monitoring for early problem detection
– Automated remediation for accelerating time to resolution
– Establishing a continuous feedback loop model improvement

What is unique about this, which can’t be found online?
We will cover model operational practices that you can apply to AI models along with other types of analytical models.

Jim is the CTO at ModelOp, where he leads the technical design of ModelOp Center, the leading ModelOps platform. Jim has been a technical innovator for the past 30 years, leading architectural design for products at a variety of companies, including Teradata, Qualtrics, and Novell. Jim holds two software patents and has taught several ModelOps master classes. For the last year, Jim has been living off the grid in the Rocky Mountains of Colorado.

Abstract:
ML deployment is not the end — it is where ML models start to materialize impact and value to the business and people. We need monitoring and testing to ensure ML models behave as expected.

What you’ll Learn:
A set of tests and metrics that ML practitioners should care about post ML deployment.

What is unique about this, which can’t be found online?
There is currently no industry standard on the types of tests and monitoring methods one should have in place. Resources are disparate and scattered online.

Technical Level: 6

Why is this takeaway(s) important?
ML deployment is not the end — it is where ML models start to materialize impact and value to the business and people. We need monitoring and testing to ensure ML models behave as expected.

Chengyin Eng is a data science consultant at Databricks, where she leads cross-functional customer projects and implements scalable end-to-end machine learning pipeline solutions. She also delivers machine learning workshops, with topics ranging from Deep Learning and Natural Language Processing to Machine Learning in Production. She received her M.S. in Computer Science from the University of Massachusetts, Amherst; and completed her B.A. in Environmental Studies and Statistics at Mount Holyoke College, Massachusetts.

Abstract:
Security has been an afterthought and has been a separate function in many projects but with the advent of DevSecOps, the security professionals could effectively collaborate with development, testing and operations teams. This helps to build-in the required information assurance to continuously integrate and deliver secure solutions. Many machine learning solutions are implemented using DevOps methodology too. However, incorporating security functions and capabilities in the model lifecycle have not been done rigorously, exposing many security risks at a later time.

MLSecOps is a new and upcoming approach proposed by security professionals to inject security in the model lifecycle so as to address threats identified even before the model is into production. This includes but not limited to addressing security vulnerabilities, privacy issues, non-compliant to standards and mandatory regulations. During the session, participants will get to see the vital elements of MLSecOps and how that can be practiced, the concept of “shift-left security” in model building by incorporating security requirements in the early stage of lifecycle and the benefits that could be reaped, and finally the best practices of security from a practioners standpoint for an effective risk management in your machine learning projects.

What you’ll Learn:
Data Scientists and Machine Learning Engineers will learn about the concept of shift-left security, security in the pipeline and security for the pipeline and importantly the concept of codifying security as they build the models and the pipeline, before they are deployed

What is unique about this, which can’t be found online?
The concept of “Shift-left security” & “Codifying Security”

Technical Level: 5

Why is this takeaway(s) important?
The session presents on the importance of security by understanding the threat landscape of ML models at first, then how integrating security in MLOps using MLSecOps helps address many of the risks.

All of these are an effort taken to build the integrity in the Machine Learning Models and make them more reliable.

Arun is a Security Consultant in the DevSecOps practice at Security Compass. He has distinctive and resourceful experience in Secure System Development Life Cycle activities including secure design, threat modeling, vulnerability management and solutioning across different domains and platforms. He has a keen interest in transformative technologies like Machine Learning, Blockchain and Internet of Things. Arun is a Certified practitioner and has experience working as a Product Security Architect for product and consulting firms. His present area of research is on Security Data Science and loves to write about emerging trends in Information Security.

Abstract:
The lifecycle of a machine learning model only begins once it’s in production. In this talk we provide a practical deep dive on best practices, principles, patterns and techniques around production monitoring of machine learning models. We will cover standard microservice monitoring techniques applied into deployed machine learning models, as well as more advanced paradigms to monitor machine learning models through concept drift, outlier detector and explainability.

We’ll dive into a hands on example, where we will train an image classification machine learning model from scratch, deploy it as a microservice in Kubernetes, and introduce advanced monitoring components as architectural patterns with hands on examples. These monitoring techniques will include AI Explainers, Outlier Detectors, Concept Drift detectors and Adversarial Detectors. We will also be understanding high level architectural patterns that abstract these complex and advanced monitoring techniques into infrastructural components that will enable for scale, introducing the standardised interfaces required for us to enable monitoring across hundreds or thousands of heterogeneous machine learning models.

What you’ll Learn:
This talk will benefit the audience by providing cross-functional knowledge, bringing together best practices from data scientists, software engineers and DevOps to tackle the challenge of machine learning monitoring at scale. During this talk we will shed light into cloud native architectural patterns that can be adopted towards machine learning orchestration, and we will provide a conceptual and practical hands on deep dive which will allow the community to both, tackle this issues and help further the discussion.

What is unique about this, which can’t be found online?
There are resources that provide in-depth content related to advanced monitoring components in isolation, such as explainers, outlier detectors, drift detectors and performance metrics, but this talk dives into architectural patterns that abstract these complex / advanced machine learning concepts into standardised infrastructural components that can scale to thousands of models.

Technical Level: 5

Why is this takeaway(s) important?
As more individuals and practitioners deploy machine learning models, more challenges get introduced which require new innovative solutions to address them. This talk provides practical intuition into best practices that could enable practitioners to improve their end to end MLOps capabilities through advanced machine learning monitoring capabilities that can scale to thousands of models.

Alejandro is the Director of Machine Learning Engineering at Seldon Technologies, where he leads large scale projects implementing open source and enterprise infrastructure for Machine Learning Orchestration and Explainability. Alejandro is also the Chief Scientist at the Institute for Ethical AI & Machine Learning, where he leads the development of industry standards on machine learning explainability, adversarial robustness and differential privacy. With over 10 years of software development experience, Alejandro has held technical leadership positions across hyper-growth scale-ups and has a strong track record building cross-functional teams of software engineers.

Abstract:
The objective of this talk is to understand the threats of machine learning models in production after the start of the pandemic. The talk will encompass the ML risk mitigation techniques to deploy, monitor, and recalibrate ML models that are essential for industry domains specially supply chain, retail, travel and hospitality, automobiles – both of which are positively and negatively impacted by Covid. In this context, the talk illustrates a few of the industry use-cases with examples from Traditional Machine Learning as well as Reinforcement learning. As the talk highlights different strategic business actions, it also offers insights on the best ethical practices that need to be imbibed in MLOps. Through understanding of smart intelligent automated AI/ML techniques, the audience will be able to develop ethical automated models by addressing the technical and business challenges of the post covid world.

What you’ll Learn:
1. Build resilient robust ML models by deep diving into advanced techniques and MLOps
2. Addressing business challenges by bringing in automated active learning strategies

What is unique about this, which can’t be found online?
Actual problems incurred and experience of solving the industry use-cases in production

Technical Level: 4

Why is this takeaway(s) important?
Create a long term sustainable business , that can rely on machine learning models in production and are able to suitably adapt , in any adverse situation be it technical or business

Sharmistha Chatterjee is a Data Science Evangelist with 15+ years of professional experience in Machine Learning and Cloud applications. She is currently working as a Senior Manager of Data Sciences at Publicis Sapient where she leads the digital transformation of clients across industry verticals. She is an international speaker, 2X Google Developer Expert, Hackernoon Tech award winner for 2020, selected as 40 under 40 Data Scientist by AIM and listed as 21 tech trailblazers 2021′ by Google.

Abstract:
Advanced customer analytics is one of the key success factors of Scotiabank’s transformation to digital banking leader and insight driven organization. Recently, Scotiabank launched a new Global AI platform which enables the automation of machine learning and the delivery of analytics assets with advanced technology such as airflow and Kubernetes. At International Banking, our team’s (AIML) main objective is focused on leveraging this platform for our analytics experiments, model training, and also for operationalizing them. With this modernized technology stack, the Bank’s operations in Colombia, analytics application deployments increased by 5X in model iterations, 8X increase in processing frequency and were 10X faster due to automated processes. In this talk, we will share our journey of machine learning product development life cycle and MLOps practices at Scotiabank.

What you’ll Learn:
– Machine learning product development life cycle in banking
– Automation of data and ML pipelines
– Job scheduling and operationalization for the delivery of analytics assets
– Advanced customer analytics success stories in international banking

What is unique about this, which can’t be found online?
We will be talking about MLOps practices within Scotiabank, this information is not available anywhere online

Technical Level:

Why is this takeaway(s) important?
These takeaways are very important since they are first hand industry experiences, this can be very helpful for growing companies or the companies who are looking to advance in MLOps

Pooja is a Senior Data Scientist at Artificial intelligence and Machine Learning, International Banking team at Scotiabank. She has been working with Scotiabank since last 3 years. She holds a Masters degree in Computer Science. Pooja’s expertise includes big data analytics, applying deep learning, classification, and clustering to real world problems. She has been delivering several successful projects at Scotiabank over last couple of years. She has also given several talks as part of analytics community at Scotiabank. Last year at TMLS, she conducted a very successful workshop on “Leveraging Pretrained Language Models for Natural Language Understanding”.

Min Li is the Director of AIML for International Banking at Scotiabank. In the past 4 years, he has led the development and deployment of multiple AI-driven applications (such as NLP, Fraud detection and Customer Insights) in both retail and wholesale banking. Currently, the IB AIML team is working towards the adoption of MLOps to build robust and reusable ML pipelines and to deliver analytical assets. Min has a PhD in Electrical and Computer Engineering from McGill. He has more than 20 publications in the area of Artificial Intelligence, Evolutionary Computation and Numerical Optimization.

Abstract:
Acerta Analytics builds machine learning solutions for automotive manufactures that help increase assembly/production line efficiency, provide actionable insights and reduce the operational costs.

With a growing customer base:
– We have to keep up with increasing Data volume
– Ensure ML models in production are robust
– Handle Data Drift

Deploying and maintaining machine learning in production is hard but reproducibility and explainability of results is even harder. ML model development lifecycle (MDLC)/MLOps is critical in creating a production grade system that scales when deploying hundreds of models at customer’s assembly lines.

We designed and built an ML platform from scratch using concepts of MLOps that orchestrated the complete workflow of training/deploying/monitoring/updating machine learning models scalable across different environments. This was critical in achieving traceability and reproducibility of ML models deployed at customer sites. Building this platform involved containerizing each step in the model lifecycle starting from data ingestion, preprocessing, training, validation, deployment, and performance monitoring.

Implementing this system in production facilitated the following deliverables:
1. Reusable infrastructure for different ML projects
2. End-to-end testing and performance checks
3. Robust codebase with versioning to ensure reproducibility
4. Model metadata store for traceability
5. Ability to auto-retrain based on thresholds and feedback loop
6. Reduced development and deployment lifecycle
7. Real-time Alerting/monitoring dashboards
8. Platform scalability and resilience to failure

What you’ll Learn:
Achieving traceability and reproducibility of ML with MLOps

Technical Level: 4

Amit Jain is an accomplished hands-on technology leader and strives to build products that solve business problems with technology. He has 16+ years of industry experience in Software Engineering, Cloud computing, Machine Learning, Product management. He enjoys taking research to production and has pivoted his career into building end-to-end production grade machine learning solutions.

Harika is a Data Scientist with 4+ years of experience currently working at Acerta Analytics. She is a graduate from University of Toronto specialized in Machine Learning. She enjoying hiking and photography during her free time.

Abstract:
The rapidly growing compute demands and complexity of training AI models necessitate the creation of a new computing architecture that can efficiently enable the unprecedented scale-out to 10s and even 100s of thousands of devices. While many vendors focused primarily on layering the system architecture on top of a “fundamentally single-device architecture” to address the scale-out challenges, Tenstorrent has designed every aspect of its microarchitecture from the ground up for scale-out to a virtually unlimited number of devices. This presentation describes our holistic approach to hardware/software architecture.

Director of Software at Tenstorrent, received her PhD in computer engineering at the University of Toronto. She spent time at Intel, leading an ML compiler team targeting FPGAs, and spent the last 3 years leading the compiler team at Tenstorrent.

Abstract:
Today’s AI is too big. Deep neural networks demand extraordinary levels of data and computation, and therefore power, for training and inference. This severely limits the practical deployment of AI in edge devices. We aim to improve the efficiency of neural network design. First, I’ll present MCUNet [1] that brings deep learning to IoT devices. MCUNet is a framework that jointly designs the efficient neural architecture (TinyNAS) and the light-weight inference engine (TinyEngine), enabling ImageNet-scale inference on micro-controllers that have only 1MB of Flash. Next I will introduce Once-for-All Network [2], an efficient neural architecture search approach, that can elastically grow and shrink the model capacity according to the target hardware resource and latency constraints. From inference to training, I’ll present TinyTL [3] that enables tiny transfer learning on-device, reducing the memory footprint by 7-13x. Finally, I will describe data-efficient GAN training techniques [4] that can generate photo-realistic images using only 100 images, which used to require tens of thousands of images. We hope such TinyML techniques can make AI greener, faster, more efficient and more sustainable.

[1] MCUNet: Tiny Deep Learning on IoT Devices, (NeurIPS’20 spotlight)
[2] Once-for-All: Train One Network and Specialize it for Efficient Deployment (ICLR’19)
[3] Tiny Transfer Learning: Reduce Memory, not Parameters for Efficient On-Device Learning (NeurIPS’20)
[4] Differentiable Augmentation for Data-Efficient GAN Training (NeurIPS’20)

What you’ll Learn:
It is now possible to do Imagenet classification on microcontrollers. the era of tiny machine learning on IoT devices has arrived

Song Han is an assistant professor at MIT’s EECS. He received his PhD degree from Stanford University. His research focuses on efficient deep learning computing. He proposed “deep compression” technique that can reduce neural network size by an order of magnitude without losing accuracy, and the hardware implementation “efficient inference engine” that first exploited pruning and weight sparsity in deep learning accelerators. His team’s work on hardware-aware neural architecture search that bring deep learning to IoT devices was highlighted by MIT News, Wired, Qualcomm News, VentureBeat, IEEE Spectrum, integrated in PyTorch and AutoGluon, and received many low-power computer vision contest awards in flagship AI conferences (CVPR’19, ICCV’19 and NeurIPS’19). Song received Best Paper awards at ICLR’16 and FPGA’17, Amazon Machine Learning Research Award, SONY Faculty Award, Facebook Faculty Award, NVIDIA Academic Partnership Award. Song was named “35 Innovators Under 35” by MIT Technology Review for his contribution on “deep compression” technique that “lets powerful artificial intelligence (AI) programs run more efficiently on low-power mobile devices.” Song received the NSF CAREER Award for “efficient algorithms and hardware for accelerated machine learning” and the IEEE “AIs 10 to Watch: The Future of AI” award.

Abstract:
While diversity, inclusion, representation remain areas of focus in the AI community, there is one other way we are uniquely positioned to reduce racism and bias in the world. Vector Institute developed a Bias in AI program to ‘Audit, Investigate & Mitigate’, take AIM at racial and language bias in AI models. We will share how we got 10 small to medium enterprises to rethink their models and product offerings to address bias. And, we will share how you can reduce bias in AI.

What you’ll Learn:
Audit, Investigate, and Mitigate bias in AI models.

Technical Level: 4

Shingai is the Director of Professional Development/ Technical Education at the Vector Institute for Artificial Intelligence in Toronto, where she translates advanced AI research into educational programming to drive AI adoption and innovation in industry. Shingai serves on the advisory council for, “Accelerating the adoption of AI in healthcare,” a program to empower front-line healthcare workers with AI skills by the Michener Institute of Education at UHN and the Vector Institute. Shingai also serves on the board of the Canada Institute on Governance (IOG).
Shingai holds a Master’s degree in Business Analytics from New York University’s Stern School of Business and her children’s book, ‘The Computer and the Cancelled Music Lessons’ teaches data science to kids from ages 5 to 12. Shingai is the 2020 recipient of the Public Policy Forum, ‘Emerging Leader’ award.

Godwin Liu is an Industrial Technology Advisor (ITA) supporting the downtown Toronto region, with the National Research Council of Canada’s Industrial Research Assistance Program (NRC IRAP). At NRC IRAP, a federal innovation support program, Godwin specializes in AI and Big Data companies, leading a national team of expert advisors in these areas. Godwin has a background in medical devices, consumer electronics and software development. Recently, in collaboration with the Vector Institute, NRC IRAP supported a program to help small-to-medium businesses learn about and apply leading edge best practices to address Bias in AI.

Abstract:
We build machine learning products to support discovery and automation within the fitness health and wellness sector. Our products range from building recommender systems to enable our consumers to discover products from our customers within our fitness marketplace to applying natural language techniques to enable our customers to create automated marketing emails to delight their customers.
In this talk, we will present our solution for training and deploying machine learning models into our production environment. We will talk about how our pipeline has evolved with open source tools like DBT, AirFlow, and MLFlow to address various pain points in building and scaling our data pipelines to support our machine learning solutions across the breadth of our wellness and beauty product ecosystem. Through this, we were able to reduce our product release time by 85%.

What you’ll Learn:
The audience will learn how to design build their AIML services and data workflows for re-usability and scalability and what open-source tools they can use to get there.

What is unique about this, which can’t be found online?
We dive into the design of our platform and how we apply it to real-world examples. We also show the implementation of these open-source tools in our systems, rather than simple how-to’s.

Technical Level: 3

Why is this takeaway(s) important?
Because bringing ML applications to production is unnecessarily tedious and time consuming. With correct planning and foundational work, you can drastically reduce the time to delivery for future projects.

Genna Gliner is a machine learning engineer working at Mindbody building products for our customers and consumers in the health and wellness space. Previously, Genna was a data science consultant at Oliver Wyman who specializes in providing machine learning solutions to her clients. Genna was an organizer for WiML 2017 as the student liaison chair.

Brandon Davis is a Machine Learning Engineer at Mindbody. With a background in Software Engineering and API Development, he brings this experience to the MLOps space to lead the ML team’s efforts in data engineering as well as deployment, monitoring and integration of ML Services at Mindbody.

Abstract:
In this forward leaning session we will discuss recent trends in MLOps and our predictions for the next 2-3 years. Hear what our experts think is in store for MLOps and production-grade AI as we move towards a more intelligent tomorrow.

What you’ll Learn:
The future for MLOps

Tristan is a product manager at DataRobot, focused on improving outcomes with ML through automation. Prior to DataRobot, he led product teams at Oracle focused on data discovery and analytics. He lives near Boston with his family and two cats.

Abstract:
Multi-armed bandits have become a popular method for online experimentation which can often out-perform traditional A/B tests. In this talk, Brian Amadio will explain the challenges to scaling multi-armed bandits, and how he solved them for the Stitch Fix experimentation platform. His solution allows Data Scientists to build and integrate sophisticated contextual bandit reward models, and includes an entirely new method for efficient, deterministic Thompson sampling.

What you’ll Learn:
The audience will learn about the main challenges to scaling multi-armed bandits, and the methods I’ve successfully used to solve these challenges at Stitch Fix. This includes an architecture that allows Data Scientists to independently deploy and update bandit reward models, as well as a novel method for Thompson sampling which is deterministic, repeatable, and very fast. They’ll also learn how they can use the library I wrote in order to put multi-armed bandits into production at their own organizations.

What is unique about this, which can’t be found online?
There is plenty of information available online about the theory behind multi-armed bandits, and lots of notebooks with toy examples. But there is very little information about putting multi-armed bandits into production in a real, scalable, cloud-native platform used by hundreds of Data Scientists and Engineers. This talk helps fill that gap and gives the audience new tools and knowledge that they are unlikely to find anywhere else.

Technical Level: 5

Why is this takeaway(s) important?
Multi-armed bandits are a valuable and popular method for online experimentation and optimization. They can be used in situations where the rate of data is limited or when the number of variants to test is very large, in order discover a likely optimal choice more quickly and with lower opportunity cost than traditional A/B testing. Many organizations are looking to use multi-armed bandits in production. The methods I’ll describe were developed for our own in-house experimentation platform, but they can also be used by organizations who use third-party A/B testing platforms.

I’m a Data Platform Engineer at Stitch Fix, helping to build and maintain our innovative experimentation platform. Recently I developed Mab, a production-ready open-source library for multi-armed bandit selection strategies. You can find a live demo of the library at the end of the talk I gave at O’Reilly Superstream: AI & ML in Production. You can also find out more about the Stitch Fix experimentation platform and multi-armed bandits in the blog post I wrote for the Stitch Fix Multithreaded blog.

Previously I worked as a data scientist, delivering high-value Machine Learning projects and solving challenging data problems across a range of complex domains. I have a PhD in experimental particle physics from UC Berkeley, where I analyzed huge datasets from the Large Hadron Collider in search of supersymmetry and microscopic black holes (I never found any).

Abstract:
This session is about how to train models on the cloud without changing a single line of code. From model development to hyperparameter sweeps, Grid automates all the engineering, so you can focus on machine learning instead of getting bogged down in infrastructure bottlenecks. In this session, William will walk you through the general workflow of research, from creating a hypothesis on a model on jupyterlab or on SSH to training a large-scale network using multiple GPUs simultaneously without having to change a single line of code.

What you’ll Learn:
The audience the general workflow of AI research, ideation, hypothesis on a model, training on multiple GPUs without having to change a single line of code.

What is unique about this, which can’t be found online?
William has a perspective unique perspective on AI and ML. From being a Navy Seal trainee to working in Finance to creating PyTorch Lightning, his perspective in the space is unqiue amongst others in the industry due to his untraditional path into data science.

Technical Level: 5

Why is this takeaway(s) important?
The key takeaway from this session is that those doing AI Research should not have to worry about learning infrastructure rather, they should focus on machine learning and going through ideas quickly.

William Falcon is the creator of the popular open-source project PyTorch Lightning, and the recently announced Grid.aI. William created Lightning while doing his PhD at NYU and as a PhD researcher at Facebook AI; Lightning allows users to scale models without the boilerplate and Grid enables large-scale training on the cloud.

Previously he co-founded the now acquired NextGenVest and spent time at Goldman Sachs.

His PhD (currently on leave to focus on Lightning and Grid), is funded by Google Deepmind and NSF Foundation. His research interest is in unsupervised learning and the intersection of AI and neuroscience.
William is a native of Venezuela and holds a BA from Columbia University in Computer Science and Statistics, with a minor in Math.

Abstract:
Fireside chat with Fatima Kardar

What you’ll Learn:
Learn how experts at Microsoft empower AI and machine learning at scale.

Fatima is Vice President and Chief Technical Advisor to Microsoft’s Chief Technology Officer Kevin Scott. She leads a diverse team of technical experts formulating and driving cross company technical strategy and culture, including various incubations, initiatives and programs. She has worked at Microsoft for more than 19 years in various roles within the product and engineering organizations – building platforms and developer APIs.

She has a passion for technology, for making a positive impact on the world and for building innovative products that solve real problems. She is a seasoned technology leader who has experience building and leading cross functional teams. She is also passionate about the role of culture and of diversity and inclusion both at work and outside. She is the D&I executive sponsor for her division and supports various company wide diversity programs such as LEAP.

Fatima earned an undergraduate degree in Computer Science from Lahore University of Management Sciences and an MBA from Duke University.

Sandhya leads Site Reliability Engineering team at LinkedIn focused on operating the big data and AI/ML platforms at scale reliably. “

Abstract:
Data Science usage at Netflix goes much beyond our eponymous recommendation systems. It touches almost all aspects of our business – from optimizing content delivery and informing buying decisions to fighting fraud. Our unique culture affords our data scientists extraordinary freedom of choice in ML tools and libraries, all of which results in an ever-expanding set of interesting problem statements and a diverse set of ML approaches to tackle them. Our data scientists, at the same time, are expected to build, deploy, and operate complex ML workloads autonomously without the need to be significantly experienced with systems or data engineering.
In this talk, I will discuss some of the challenges involved in improving the development and deployment experience for ML workloads. I will focus on Metaflow, our ML platform, which offers useful abstractions for managing the model’s lifecycle end-to-end, and how a focus on human-centric design positively affects our data scientists’ velocity.

What you’ll Learn:
How should people think about building/adopting ML platforms within their organisations

Technical Level: 4

Why is this takeaway(s) important?
How should people think about building/adopting ML platforms within their organisations

Savin is a software engineer responsible for Metaflow, Netflix’s ML platform. He focuses on building generalizable infrastructure to accelerate the impact of data science at Netflix and beyond.

Abstract:
We present PANAMA, a novel in-network aggregation framework for distributed machine learning (ML) training on shared clusters serving a variety of jobs. PANAMA comprises two key components: (i) a custom in-network hardware accelerator that can support floating-point gradient aggregation at line rate without compromising accuracy; and (ii) a lightweight load-balancing and congestion control protocol that exploits the unique communication patterns of ML data-parallel jobs to enable fair sharing of network resources across different jobs while ensuring high throughput for long-running jobs and low latency for short jobs and other latency-sensitive traffic. We evaluate the feasibility of PANAMA using an FPGA-based prototype with 10 Gbps transceivers and via large-scale simulations. Our large-scale simulations demonstrate that PANAMA decreases the average training time of large jobs by up to a factor of 1.34. More importantly, by drastically decreasing the load placed on the network by large data-parallel jobs, PANAMA provides significant benefits to non-aggregation flows, especially latency-sensitive short flows, as it reduces their 99%-tile completion time by up to 4.5x.

What you’ll Learn:
1. Challenges to scaling distributed training in shared ML clusters.
2. Limitations of previously proposed solutions for scaling distributed data-parallel training.
3. Co-designing the in-network aggregation logic, host software, and congestion control protocol not only benefits data-parallel ML jobs, but provides even more benefits to non-aggregation latency sensitive short flows.

Technical Level: 5

Why is this takeaway(s) important?
It highlights how tailoring the cluster to support properties of data-parallel ML applications can benefit the entire cluster and not only data-parallel jobs.

Final year Computer Science PhD student at Imperial College London. Her research focuses on accelerating data center applications with programmable in-network logic.

Abstract:
Over the past decade Yelp has scaled our ML reliance from fringe usage by a handful of enthusiastic developers to a core competency leveraged by many teams of dedicated experts to deliver tens of millions of dollars in incremental revenue. In that time we’ve experimented with several organizational structures and ML processes, and observed several categories of pitfalls. In this talk we’ll discuss how to diagnose several of these pitfalls especially as related to ML project velocity, ML practitioners’ happiness & retention, and scaling ML adoption to broader parts of your business objectives.

What you’ll Learn:
How to recognize and act upon common failure modes in your ML organization and process structure. Failure modes will be illustrated by several case studies from Yelp’s ML journey.

What is unique about this, which can’t be found online?
This talk will focus more on ML as a personnel capability and how to empower, organize, and scale ML practitioners throughout an organization. Most online resources I’ve seen focus on technical capabilities while neglecting the more personal aspects

Technical Level: 2

Why is this takeaway(s) important?
Setting your organization up for success starts with people. A successful ML organization needs to be equipped with the right tools, presented with the right opportunities, and deployed collaboratively.

Dr. Jason Sleight is currently a group tech lead on the Core ML team at Yelp where he focuses on the intersection of machine learning (ML) and systems engineering. He leads several initiatives to create platforms for ad hoc computing, data ETL, and ML as well as to collaborate with stakeholders across all of Yelp to apply these platforms towards Yelp’s business goals. Prior to Yelp, Jason completed his PhD from the University of Michigan studying artificial intelligence and cooperative multiagent systems

Daniel Yao is currently the engineering director for Yelp’s Applied Machine Learning (AML) org. AML includes teams working on the company’s ML platform as well as product-facing ML initiatives, such as personalized recommender systems, revenue optimization models, and content integrity signals. Prior to Yelp, Daniel worked on quantitative trading systems and strategies for hedge funds and proprietary trading firms.