MLOps: Machine Learning in Production New York City

September 14th – 15th, 2021

A Uniquely Interactive Experience

Why MLOps Production & Engineering?

Too few companies have effective AI leaders or an effective AI strategy.

An interactive conference is one in which you meet attendees before the event, message speakers directly, discuss specific areas of your work and establish a stronger network. We’re working to help you with a clearer understanding of best practices, methodologies, and principles of effective ML.

The micro-summit includes:

16 Speakers
4 Workshops

Join this new initiative to help push the AI community forward.

We’re Hosting

BREAKOUT SESSIONS
(All Levels)

DISCUSSION GROUPS

WORKSHOPS

VIRTUAL PLATFORM

Chair

Harish Doddi

Founder & CEO, Datatron

Victor Thu

VP of Customer Success & Ops, Datatron

Bio & Abstract

Talk: Best Practices Deploying MLOps Solutions in Enterprise

Mengdi Huang

Workshop Facilitator

More to be announced

Join Us

Gold Sponsors

Silver SPONSORS

Join Our Community

Our goal is to provide an open, inclusive community of ML practitioners who can share projects, best practices and case studies.

Join our open group, meet our community and share your work with practitioners from around the world.

Join us here and learn more:

Abe Gong

CEO of Core Contributor

Abe Gong a core contributor to the Great Expectations open source library, and founder and CEO of Superconductive, the company supporting the project.

Prior to Superconductive, Abe was Chief Data Officer at Aspire Health, the founding member of the Jawbone data science team, and lead data scientist at Massive Health.

Abe has been leading teams using data and technology to solve problems in health, tech, and public policy for over a decade.

Talk: Fighting pipeline debt with Great Expectations

You have Successfully Subscribed!

Boris Lublinsky

Principal Architect, Lightbend

Boris Lublinsky is a principal architect at Lightbend, where he specializes in big data, stream processing, and services. Boris has over 30 years’ experience in enterprise architecture.

Previously, he was responsible for setting architectural direction, conducting architecture assessments, and creating and executing architectural road maps in fields such as big data (Hadoop-based) solutions, service-oriented architecture (SOA), business process management (BPM), and enterprise application integration (EAI).

Boris is the coauthor of Applied SOA: Service-Oriented Architecture and Design Strategies, Professional Hadoop Solutions, and Serving Machine Learning Models. He’s also cofounder of and frequent speaker at several Chicago user groups.

Talk: Using Model Serving in Streaming Applications

You have Successfully Subscribed!

Brooke Wenig

Machine Learning Practice Lead at Databricks

Brooke Wenig is a Machine Learning Practice Lead at Databricks. She leads a team of data scientists who develop large-scale machine learning pipelines for customers, as well as teaching courses on distributed machine learning best practices.

Previously, she was a Principal Data Science Consultant at Databricks. She received an MS in computer science from UCLA with a focus on distributed machine learning.

Talk: Managing Machine Learning Experiments with MLflow

You have Successfully Subscribed!

Chanchal Chatterjee

Cloud AI/ML of Google Inc.

Chanchal Chatterjee, Ph.D, held several leadership roles in machine learning, deep learning and real-time analytics. He is currently leading Machine Learning and Artificial Intelligence at Google Cloud Platform with a focus on Financial Services.

Previously, he was the Chief Architect of EMC CTO Office where he led end-to-end deep learning and machine learning solutions for data centers, smart buildings and smart manufacturing for leading customers. He was instrumental in the Industrial Internet Consortium, where he published an AI framework for large enterprises.

Chanchal received several awards including Outstanding paper award from IEEE Neural Network Council for adaptive learning algorithms recommended by MIT professor Marvin Minsky. Chanchal founded two tech startups between 2008-2013. Chanchal has 29 granted or pending patents, and over 30 publications. Chanchal received M.S. and Ph.D. degrees in Electrical and Computer Engineering from Purdue University.

Talk: Quickly Deploy ML Workloads on Multi-Cloud using Kubeflow Pipelines

You have Successfully Subscribed!

Ebrahim Safavi

Senior Data Scientist of Mist, a Juniper Company

Ebrahim is a Senior Data Scientist at Juniper, focusing on knowledge discovery from big data using machine learning and large-scale data mining where he has developed, and implemented several key production components including company's chat bot inference engine and anomaly detections.

Ebrahim has won Microsoft research award for his work on Information Retrieval and Recommendation systems in graph-structured networks. Ebrahim holds a Ph.D. degree on Cognitive Learning Networks from Stevens Institute of Technology.

Talk: Automated Pipeline for Large-Scale Neural Network Training and Inference

You have Successfully Subscribed!

Nick Pogrebnyakov

Founder, Leverness, Senior AI Researcher at Thomson Reuters & Associate Professor at Copenhagen Business School

Nick is a Senior AI Researcher at Thomson Reuters, where he works on natural language processing. He is also an Associate Professor of International Business at Copenhagen Business School and founder of Leverness, a platform for machine learning talent.

Nick received a PhD in Information Sciences and Technology from Pennsylvania State University and an MSc in Mathematics and Computer Science from Belarusian State University for Informatics and Electronics.

Talk: Implementing a Machine Learning initiative: Guidance for Leaders

You have Successfully Subscribed!

Sabina Stanescu

Principal Data Scientist, Altair Engineering

7+ years working as a Data Science professional.

Currently manages a team of Data Scientists for consulting engagements with customers across use-cases such as Credit Risk, Marketing and Engineering Process Design.

Previously worked at Points as Lead Data Scientist in Marketing and as Lead Product Manager for Machine Learning, integrating machine learning into Points' products.

MSc in Ecology, University of Guelph, with focus on data analysis and modeling in R

Talk: Built ML deployment end-to-end infrastructure from scratch at Points

You have Successfully Subscribed!

ML Ops that works: How we Built ML Pipelines for Deploying Models for Autonomous Factories, Saranyan Vigraham, Head of Engineering, Petuum

Abstract: ML Ops is currently is as much an art as much a science. Unlike Devops, which can be codified to a set of tools and practices to ensure consistent and efficient software delivery, the trial and error nature of ML projects pose a different set of challenges. At Petuum, we have deployed many ML models, which autonomously operate factory equipments. Because the factory environments are dynamic, it required us to build flexible pipelines that allow easy iteration and deployment, sometimes testing multiple models concurrently. In this talk, I will present one such ML pipeline (powered by an enterprise platform) that has allowed us to deploy hundreds of models. I will discuss some design/architectural patterns and anti-patterns based on our real world experiences.

Technical level? (3/5)

What is unique about this talk?

It is hard to come across a talk that focuses on design patterns in ML from a lessons learned perspective. This is not a generic boilerplate. We have deployed ML models in the real world and iterated with customer feedback. We have not yet shared these learnings anywhere, so it will be beneficial for ML engineers who are trying to ship their software in the real world.

What are some infrastructure and languages discussed?

Spark, Kubernetes, Nifi, Pulsar, Language agnostic. Focus will be on the system level. Jenkins, Kubernetes, Prometheus

2-3 topics for post-discussion?

What's the DL framework behind Spark NLP NER models, Why the NER models are crucial in Clinical domain, some use cases of NLP in Healthcare

What you'll learn?

Challenges to deploy ML in the real world, ML design and anti-patterns

Prerequisite Knowledge

Basic understanding of ML flow

You have Successfully Subscribed!

Quickly Deploy ML Workloads on Multi-Cloud using Kubeflow Pipelines, Chanchal Chatterjee, Cloud AI / ML, Google Inc.

Abstract: Learn how to quickly produce an end-to-end AI pipeline and easily deploy ML workloads onto Google Cloud using the well known open-source platform called Kubeflow Pipelines. All major components of the AI pipeline such as data pre-processing, hyperparameter tuning, model training, model prediction, model explanation, and training orchestration can be easily implemented on the cloud with just a few easy steps.

Technical level? (3/5)

What are some infrastructure and languages discussed?

Google Cloud Platform, Kubeflow Pipelines, Python

What you'll learn?

Challenges to deploy ML in the real world, ML design and anti-patterns

What is unique about this talk?

A unique solution is offered for ML Ops to simplify productionization of ML Workloads

You have Successfully Subscribed!

Saranyan Vigraham

Head of Engineering – Petuum Click for Bio

Saranyan loves building meaningful products. In the past, he has led diverse engineering teams of over one hundred engineers, rallying them around vision and engineering excellence.

Saranyan has shipped both enterprise and consumer products during his tenure at companies like Petuum, Elementum, Meta and PayPal. With a PhD in Computer Science, he is always learning how to push the boundaries of technology to shape society in meaningful ways.

Talk: ML Ops that works: How we built ML pipelines for deploying models for autonomous factories

You have Successfully Subscribed!

Kenny Daniel

Founder, Algorithmia

Kenny Daniel is a founder and CTO of Algorithmia. He came up with the idea for Algorithmia while working on his PhD and seeing the plethora of algorithms that never saw the light of day.

In response, he built the Algorithmia Cloud AI Layer, which has helped more than 80,000 developers share, pipeline, and consume more than 7000 models. Through his work with hundreds of companies implementing ML, he then created the Enterprise AI Layer, which helps the largest organizations in the world deploy, connect, manage, and secure machine learning operations at scale.

Kenny holds degrees from Carnegie Mellon University and the University of Southern California, where he studied artificial intelligence and mechanism design.

Talk: DevOps for Machine Learning and other Half-Truths: Processes and Tools for the ML Lifecycle

You have Successfully Subscribed!

Patrick Hall

Principal Scientist, bnh.ai, Senior Director of Product, H2O.ai

Patrick Hall is a principal scientist at bnh.ai, an advisor to H2O.ai, and an adjunct professor in the Department of Decision Sciences at The George Washington University.

Talk: Real-World Strategies for Model Debugging

You have Successfully Subscribed!

Veysel Kocaman

Senior Data Scientist of John Snow Labs

Veysel is a Lead Data Scientist at John Snow Labs, lecturer at Leiden University and a seasoned ML Engineer with a strong background in every aspect of data science including artificial intelligence and big data with over ten years of experience. He is also working towards his PhD in Computer Science and is a Google Developer Expert in Machine Learning.

Talk: Closed-loop online NLP learning systems for high compliance industries using Spark NLP3

You have Successfully Subscribed!

Vin Vashishta

Data Scientist, Strategist, Speaker, Author, V2 Machine Learning Consulting

25 years ago, I was installing servers and networks. Now I install strategies that monetize machine learning models. I have spent the last eight years pushing this field forward and helping people break into machine learning. My technical expertise is custom model development for pricing and natural language understanding.

Talk: Now What? Machine Learning After COVID-19

You have Successfully Subscribed!

Brandy Freitas

Senior Data Scientist, Pitney Bowes

Brandy Freitas is a principal data scientist at Pitney Bowes, where she works with clients in a wide variety of industries to develop analytical solutions for their business needs. Brandy is a research physicist-turned-data scientist based in Boston, MA. Her academic research focused primarily on protein structure determination, applying machine learning techniques to single-particle cryoelectron microscopy data.

Brandy is a National Science Foundation Graduate Research Fellow and a James Mills Pierce Fellow. She holds an undergraduate degree in physics and chemistry from the Rochester Institute of Technology and did her graduate work in biophysics at Harvard University.

Talk: Team Roles in a Machine Learning Project and Project Flow

You have Successfully Subscribed!

Patricia Thaine

CEO of Private AI

She is the Co-Founder and CEO of Private AI, a Toronto- and Berlin-based startup creating a suite of privacy tools that make it easy to comply with data protection regulations, mitigate cybersecurity threats, and maintain customer trust.

Patricia is a recipient of the NSERC Postgraduate Scholarship, the RBC Graduate Fellowship, the Beatrice “Trixie” Worsley Graduate Scholarship in Computer Science, and the Ontario Graduate Scholarship. She has eight years of research and software development experience, including at the McGill Language Development Lab, the University of Toronto's Computational Linguistics Lab, the University of Toronto's Department of Linguistics, and the Public Health Agency of Canada.

She is also a member of the Board of Directors of Equity Showcase, one of Canada's oldest not-for-profit charitable organizations.

Talk: Privacy-Preserving Machine Learning

You have Successfully Subscribed!

Mark McQuade

Practice Manager, Data Science and Engineering, Onica, a Rackspace Company

My name is Mark McQuade and I am a AWS and Cloud-Based Solution Specialist, Knowledge Addict, Relationship Builder and Practice Manage at Rackspace focusing on Machine Learning and data analytics.

My passion is in all things ML and I take great pride and joy in working through data and proving clients with insights on their business and data through Artificial Intelligence.

Talk: Automating Production Level Machine Learning Operations on AWS

You have Successfully Subscribed!

Hamza Tahir

CTO, maiot GmbH

Hamza is a software engineer cum machine learning engineer based in Munich, Germany. He has a passion for trying to connect the dots between his various learning experiences and to continually learn and grow from new challenges.

Hamza is curently co-founding his ML startup, maiot, with the aim of bringing proper software engineering practices into machine learning workflows.

Click to VIew Linkedin

Talk: Why ML in production is (still) broken

You have Successfully Subscribed!

DevOps for Machine Learning and other Half-Truths: Processes and Tools for the ML Lifecycle, Kenny Daniel, Founder, Algorithmia

Abstract:

Traditional software development has a Software Development Life Cycle, coalesced around a set of tools and processes. In contrast, machine learning is a tangle of tools, languages, and infrastructures, with almost no standardization.To build and deploy enterprise-ready machine learning models that generate real value, organizations need to consider a standard ML focused life cycle that supports IT’s Operations Management and Infrastructure groups.

What you'll learn:
- Key differences between ML and traditional software development
- Where the SDLC works with ML, and where it breaks down
- An overview of the new ML stack, from training to deployment to production
- The five biggest infrastructure and process mistakes ML teams commit
- How successful early movers have succeeded, and real-world lessons you can use today

What is unique about this talk?

Algorithmia is a pioneers in helping enterprises put models in production. This talk wil

You have Successfully Subscribed!

Your First ML model in Production: Considerations and Examples,, Sabina Stanescu, Principal Data Scientist, Altair Engineering

Abstract: As Data Science professionals, we want to do innovative, impactful work. Thus, our work on data munging and building machine learning models cannot happen in isolation from business objectives and the infrastructure of our organizations. In this talk, I will explore ways to identify impactful, executable Data Science work, and how to take this work to production. I will discuss what it means to have a model in production, including ways to score the model in real-time versus batch. I will discuss sample architectures required to make model scores available for your application, such as through an API or database. Finally, I will tie everything together with some of the processes and frameworks that allow for iteration and testing to complete the full life-cycle of model deployment. I will provide a real example of taking an ML project all the way from data capture to real-time scoring in production.

Technical level? (2/5)

What is unique about this talk?

All ML Ops infrastructure and processes were built from scratch by a small team. Early poor performance of models prompted us to start developing better testing infrastructure and build in multi-armed bandit testing in addition to traditional AB testing, which led to receiving a technology innovation award within the company.

What are some infrastructure and languages discussed?

Gitlab pipelines, DockerPython, Ruby, PMMLgitlab CI/CD pipelines

2-3 topics for post-discussion?

What tools would be relevant today to re-do this project? How is it, as Data Scientist, collaborating with software engineers?

What you'll learn?

A real example of all the steps required to get to production, and the decisions and compromises took on that journey.

Prerequisite Knowledge

Basic ML model building knowledge

You have Successfully Subscribed!

Real-World Strategies for Model Debugging, Patrick Hall, Principal Scientist, bnh.ai, Senior Director of Product, H2O.ai

Abstract: You used cross-validation, early stopping, grid search, monotonicity constraints, and regularization to train a generalizable, interpretable, and stable model. Its fit statistics look just fine on out-of-time test data, and better than the linear model it's replacing. You selected your probability cutoff based on business goals and you even containerized your model to create a real-time scoring engine for your pals in information technology (IT). Time to deploy?

Not so fast. Unfortunately, current best practices for machine learning (ML) model training and assessment can be insufficient for high-stakes, real-world ML systems. Much like other complex IT systems, ML models must be debugged for logical or run-time errors and security vulnerabilities. Recent, high-profile failures have made it clear that ML models must also be debugged for disparate impact and other types of discrimination

This presentation introduces model debugging, an emergent discipline focused on finding and fixing errors in the internal mechanisms and outputs of ML models. Model debugging attempts to test ML models like code (because they are usually code). It enhances trust in ML directly by increasing accuracy in new or holdout data, by decreasing or identifying hackable attack surfaces, or by decreasing discrimination. As a side-effect, model debugging should also increase the understanding and interpretability of model mechanisms and predictions. Want a sneak peek of some model debugging strategies? Check out these open resources: https://towardsdatascience.com/strategies-for-model-debugging-aa822f1097ce.

Technical level? (4/5)

What is unique about this talk?

Model debugging is a novel topic with few online resources.

What are some infrastructure and languages discussed?

Core machine learning (ML) infrastructure (e.g., XGBoost, TensorFlow) and automated quality assurance (QA) for ML.PythonH2O scoring engines (i.e., POJOs, MOJOs), containers -- all open

2-3 topics for post-discussion?

Finding bugs in machine learning (ML) models, fixing bugs in ML models, increasing interpretability of ML models.

What you'll learn?

How to train a clinical NER in Spark using custom annotations.

Familiarity with NLP

Working knowledge of common machine learning (ML) techniques, especially linear models, decision trees, and neural networks.
Working knowledge of Python for the resources associated with the talk.

You have Successfully Subscribed!

Closed-loop online NLP learning systems for high compliance industries using Spark NLP3, Veysel Kocaman, Senior Data Scientist, John Snow Labs

Abstract: NLP is an uber domain specific field and NLP models usually need to be tuned for a domain-specific dataset to be useful in real projects. During the talk, Veysel will show a workflow that automates Clinical Named Entity Recognition (NER) model training in Spark NLP based on labelled clinical data in John Snow Labs annotation tool. This will mean building a semi-automated integration with the annotator, and showing the closed-loop integration for starting with a pre-trained model and tuning it based on annotated custom data.

Technical level? (4/5)

What is unique about this talk?

The speaker shares how to deal with clinical NLP problems and delivering solutions in this domain on a daily basis using SOTA frameworks with human-annotated data.

What are some infrastructure and languages discussed?

Spark NLP, Colab, Google SlidesPython, SparkColab

2-3 topics for post-discussion?

What's the DL framework behind Spark NLP NER models, Why the NER models are crucial in Clinical domain, some use cases of NLP in Healthcare

What you'll learn?

How to train a clinical NER in Spark using custom annotations.

Familiarity with NLP

Basic ML model building knowledge

You have Successfully Subscribed!

Now What? Machine Learning After COVID-19?, Vin Vashishta, Data Scientist, Strategist, Speaker, Author, V2 Machine Learning Consulting

Abstract: What can you expect after Covid-19

Why ML in Production is (Still) Broken, Hamza Tahir, CTO, maiot GmbH

Abstract: Around 87% of machine learning projects do not survive to make it to production. There is a disconnect between machine learning being done in Jupyter notebooks on local machines and actually being served to end-users to provide some actual value.
The oft-quoted Hidden Technical Debt paper, by Scully et. al., has been in circulation since 2017, yet still, ML in production has ways to go to catch up to the quality standards attained by more conventional software development.
This talk will aim to break down the key aspects of what sets machine learning apart from traditional software engineering, and how treating data as a first-class citizen is a fundamental shift in our understanding of complex production ML systems.

Technical level? (4/5)

What is unique about this talk?

Speaker has multiple years of experience focusing exclusively on transitioning ML from research to production.

What are some infrastructure and languages discussed?

Lambda architectures, batch paradigms (Apache Beam), pipeline frameworks such as Tensorflow Extended, Gitlab CI, Kubernetes, Kubeflow, Airflow

2-3 topics for post-discussion?

How often to retrain models, how to manage multiple models in production, training-serving drift in productio

What you'll learn?

A precise way to manage and scale machine learning systems in production.

Prerequisite Knowledge

Basic ops paradigms like Docker, Kubernetes, and basic ML knowledge

You have Successfully Subscribed!

Managing Machine Learning Experiments with MLflow, Brooke Wenig, Machine Learning Practice Lead at Databricks

Abstract: Successfully building and deploying a machine learning model is difficult. Enabling other data scientists to reproduce your pipeline, compare the results of different versions, and rollback models is much harder. This talk will introduce MLflow, an open-source project that helps developers reproduce and share ML experiments, manage models, and control the challenges associated with making models "production ready."

You have Successfully Subscribed!

Graphs in Cassandra: Modeling, Querying, and Seeing Graph Data within the PowerGrid,

How you contribute to a dynamic graph with the flight of a light switch, Denise Gosnell, Denise (Koessler) Gosnell, Ph.D, Chief Data Officer at DataStax, Author of The Practitioners Guide to Graph Data

Abstract: Walk-through session:

Self-organizing networks, like the power grid in the United States, rely on communicationbetween sensors and call towers for transmitting the network's status. Thiscommunication network forms a dynamic graph that helps power companies havereal-time visibility into different network failure scenarios.

For instance, what happens if the tower goes down? And, how does a graph data structure get involved in the network's healing process?

In this session, Denise Gosnell will show you how we built a graph database to model a dynamic network within the power grid. Within a notebook environment, she will walk through the code for the database model and graph database queries. Then, we willshow how to apply path information to triage network sensors that are at risk. You will walk away with knowledge of a new use case for graph data in the power industry and assets to play with the data and code on your own. Get ready to go deepinto the world of graph data and graph queries. Technical Assets for Audience Usage Afterwards: Content Repository on GitHubhttps://github.com/datastax/graph-book

You have Successfully Subscribed!

The Do’s and Don’ts of Delivering AI Projects: A Practitioners Guide, Jan Zawadzki, Head of AI, Carmeq GmbH, Volkswagen Group

Abstract: Artificial Intelligence (AI) offers vast opportunities across industries and sectors. While traditional project or software management techniques have been around for decades, AI is new territory. According to an IDC study, 50% of AI Projects are doomed to fail, versus 14% in traditional projects. How could that be and could we prevent our AI Project from failing?

AI Projects differ from traditional software projects. First, brainstorming suitable AI Projects is an art in itself. Next, uncertainty and scoping issues surround almost every AI Project. Can this project really be done? And if it works out, how valuable will it be in the end? And if you come up with a really valuable but risky project, when should you skip or go for it? Finding answers to these questions is vital to securing your first, second, and third steps with AI. This talk presents you with a practitioner’s approach to delivering AI Projects. The presentation will touch upon topics like finding and evaluating AI Projects and introduce tools like the Impact-Risk AI Matrix, AI Maturity Journey, and the AI Project Canvas.

Technical level? (2/5)

What you'll learn?

A framework to ideate, evaluate, and scope AI Projects

Prerequisite Knowledge

Basic knowledge about Machine Learning

2-3 topics for post-discussion?

Applying the AI Project Canvas in the real world, challenges in ideating AI projects, evaluating an AI Project Idea

You have Successfully Subscribed!

GitHub Actions in Action, Jon Peck,Technical Advocate, Executive Briefing Center, GitHub

Abstract: We all love the conventional uses of CI/CD platforms, from automating unit tests to multi-cloud service deployment. But most CI/CD tools are abstract code execution engines, meaning that we can also leverage them to do non-deployment-related tasks. In this session, we'll explore how GitHub Actions can be used to train a machine learning model, then run predictions in response to file commits, enabling an untrained end-user to predict the value of their home by simply editing a text file. As a bonus, we'll leverage Apple's CoreML framework, which normally only runs in an OSX or iOS environment, without ever requiring the developer to lay their hands on an Apple device

What is unique about this talk?

GitHub actions is an interesting new feature from GitHub that software development teams are excited about and starting to adopt in their workflows. Usually, it's only seen leveraged in the context of traditional software development. You will see it applied with respect to ML projects, and end-users predictions.

You have Successfully Subscribed!

Automated Pipeline for Large-Scale Neural Network Training and Inference, Ebrahim Safavi, Jisheng Wang, Senior Data Scientist, Senior Director of Data Science

Abstract: Anomaly detection models are essential to run data-driven businesses intelligently. In order to manage tens of thousands of anomaly detection models at Mist, we have built a cloud-native and scalable ML training pipeline which automates all steps of ML operations including data collection, model training, model validation, model deployment and version control. The inference workflow is decoupled from the training process to increase the agility and minimize the delay of model service.
Motivated by the recent impressive performance of recurrent neural networks (RNNs) on a wide spectrum of tasks, we have developed confident deep bidirectional long-short term memory (BiLSTM) models which leverage a large amount of data across numerous dimensions to capture trends and catch anomalies across thousands of Wifi networks and address issues in real-time. The proposed BiLSTM models are capable of predicting the uncertainty of their detection which is essential for the anomaly detection purpose.
In addition, to address the challenges imposed by the stochastic nature of unsupervised anomaly detection on the workflow pipeline, we have developed novel statistical models for the training workflow to leverage historical data and automate model validation, deployment and version control.
The anomaly detection service happens hourly and the training jobs occur weekly through the pipeline which consists of different steps including managing the training and serving data stream, model versioning for predictions, training and serving for each network’s model. The workflow pipeline utilizes different technologies including Secor service, Amazon S3 service, Apache Spark across Amazon EMR cluster, Apache Kafka and Elasticsearch.
In this talk, we first briefly discuss the details of the unsupervised confident deep multivariate models we have built to automatically detect the WiFi network issues. Then, we dive deeper into the details of our cloud-based pipeline and how we use relative entropy to automate the training workflow. Finally, we share lessons learned and insights specifically, how to productize and monitor thousands of ML models to automate anomaly detection.

Technical level? (4/5)

What is unique about this talk?

Throughout this talk we share lessons learned and insights specifically, how to productize and monitor thousands of ML models to automate anomaly detection.

What are some infrastructure and languages discussed?

Secor service, Amazon S3 service, Apache Spark across Amazon EMR cluster, Apache Kafka and Elasticsearch, Python

2-3 topics for post-discussion?

Anomaly Detection, Automated ML Pipeline, Large-Scale Neural Network Training and Inference

What you'll learn?

Building a fully functional ML pipeline with a focus on the customer experience.

You have Successfully Subscribed!

Automated Pipeline for Large-Scale Neural Network Training and Inference, Ebrahim Safavi, Jisheng Wang, Senior Data Scientist, Senior Director of Data Science

Technical level? (4/5)

What is unique about this talk?

Throughout this talk we share lessons learned and insights specifically, how to productize and monitor thousands of ML models to automate anomaly detection.

What are some infrastructure and languages discussed?

Secor service, Amazon S3 service, Apache Spark across Amazon EMR cluster, Apache Kafka and Elasticsearch, Python

Validate and Monitor Your AI and Machine Learning Models, Olivier Blais, Co-founder and VP Data Science, Moov AI

Abstract: You’ve created a wicked AI or machine learning model that changes the way you do business. Good job. But how do you validate your model and monitor it in the long run?
Advanced machine learning and AI models get more and more powerful. They also tend to become more complicated to validate and monitor.
This has a major impact in the business’ adoption of models. Initial validation and monitoring are not only critical to ensure the model’s sound performance, but they are also mandatory in some industries like banking and insurance.
You will learn the best techniques that can be applied manually or automatically to validate and monitor statistical models. Techniques below will be discussed and demonstrated to perform a full model validation: - Techniques used for initial validation.

2-3 topics for post-discussion?

Model validation, model monitoring, machine learning use cases in general.

What are some infrastructure and languages discussed?

This talk is infrastructure agnostic. Python (mostly tensorflow or pytorch)

What you'll learn?

You'll learn a cutting edge framework which you can't find on Google, yet. We'll show DevOps techniques using open source packages. You will learn the best techniques that can be applied manually or automatically to validate and monitor statistical models.

You have Successfully Subscribed!

An Introduction to Reinforcement Learning, Enzo Vernon, ML Cloud Security Engineer , Choice Hotels International

Abstract: I’m going to define reinforcement learning, cover why it's important, explain how it differs from other branches of machine learning, and then delve into the concepts behind this exciting field of AI.

What you'll learn?

The principles of D3QN with Prioritized Experience Replay

You have Successfully Subscribed!

Abstract:

You have Successfully Subscribed!

Smart Data Products: From prototype to production, Maximo Gurmendez, Founder and Chief Engineer, Montevideo Labs

Abstract: Notebooks are a great tool for Big Data. They have drastically changed the way scientists and engineers develop and share ideas. However, most world-class ML products cannot be easily engineered, tested and deployed just by modifying or combining notebooks. Taking a prototype to production with high quality typically involves proper software engineering and process. At Montevideo Labs we have many years of experience helping our clients to architect large systems capable of processing data at peta-byte scale. We will share our experience on how we productize ML starting from a prototype to production. Data Scientists should be truly free to use any tool and library available. On the other hand engineers need artifacts that are modular, robust, readable, testable, reusable and performant. We'll outline strategies to bridge these two needs and aid the concepts with a live demo.

What you'll learn?

We will share our experience engineering smart data products based on ML prototypes. At Montevideo Labs we've faced many challenges incorporating data science based artifacts into user-facing products. We'll go through a list of right and wrongs when it comes to properly deploying an ML idea to production. We'll not just dive in the technological recommendations but also in the way we think a successful engineering and data science collaboration process looks like.

You have Successfully Subscribed!

Abstract:

You have Successfully Subscribed!

Adapting continuous integration and continuous delivery for ML, Elle O'Brien, Data Scientist, Iterative.ai

Abstract: Machine learning (ML) is maturing as a discipline: now that it’s trivially easy to create and train models, it’s never been more challenging to manage the complexity of experiments, changing datasets, and the demands of a full-stack project. Luckily, ML is not the first field to undergo such a transition: looking to DevOps as an example of how software developers and operations specialists found practices to balance the competing needs for rapid experimentation and stability in production, the new discipline of MLOps is promising. In this talk, we’ll examine why one of the staples of DevOps, continuous integration and continuous delivery (CI/CD), has been so challenging to implement in ML projects so far and how it can be done using open-source tools like Git, GitHub Actions, and DVC (Data Version Control). These tools are both flexible and powerful, enabling teams to automate model training and evaluation, track experiments with existing version control systems, and make the model selection process more like code reviews. We argue that the CI/CD workflow makes sense for heterogeneous teams of ML engineers, data scientists, and software engineers who need to balance frequent experiments and changing datasets with strong engineering practices.

Technical level? (Not Sure)

What is unique about this talk?

- I'm highly experienced at public speaking and really enjoy it. I do ML and data science to get to discuss it, not the other way around.

- I come from a data science/academic background and have slowly, over years of trial and error and horrible encounters with legacy code, learned the importance of engineering best practices for ML projects. So I bring with me the perspective of a) what are the priorities of non-engineers in ML/data science, b) what are the pressures they face that prevent them from incorporating known best practices, and c) what engineering practices have realistic learning curves for scientists and researchers.

What are some infrastructure and languages discussed?

Cloud computing resources (not specific to any provider) for storage and job execution. Nearly everything I'll discuss will be available in free-tier from AWS/GCP/Azure, with the possible exception of GPUs.

2-3 topics for post-discussion?

How can we encourage a cultural shift, similar to DevOps, that gets ML scientists to participate in engineering workflows and practices? And likewise, operations specialists to become more fluent in the ML workflow? Why or why not use Git version control for ML projects? What would an ideal version control system look like for models, datasets, and pipelines?

What you'll learn?

MLOps is so new as a discipline that there's so much yet to be said! In a talk like this, I'm hoping to spark a discussion about whether continuous integration and delivery makes sense for ML engineers and data scientists. Every time I've shared work on this topic, it tends to elicit very compelling discussion- is ML's emphasis on experimentation compatible with systematic engineering practices? Of course, I think yes. I think of this speech as making my best case for it. If there's time for questions, I expect those will be especially illuminating about how CI/CD will fare in the ML community.

Prerequisite Knowledge

Some familiarity with version control (Git) is helpful but not required to understand the main ideas of the talk.

You have Successfully Subscribed!

Abstract:

You have Successfully Subscribed!

MLOps at Scale: Predicting Bus Departure Times using 18,000 ML Models, Alice Gibbons, Technical Specialist, Microsoft, Hubert Duan, Cloud Solution Architect, Microsoft

Abstract: "TransLink, Metro Vancouver’s transit agency, needed more precise and reliable bus departure estimations in order to improve rider satisfaction. For this they created the Bus Departure Prediction System: A large-scale, end-to end Azure solution that uses over 18,000 machine learning models to predict when the next bus will depart from each stop. This system leverages a micro-model service approach, where each model is separately trained and deployed, but runs together in production. The Bus Departure Prediction System has been rolled out to TransLink’s entire fleet of over 200 bus routes, servicing 2.5M residents in the greater Vancouver area, and happens to include the busiest bus route in North America. When it comes to the sheer number of models involved, this is the largest single-solution deployment of ML models on Azure in Canada and one of the largest worldwide. This session provides an in-depth review of the system including the motivation behind the number of models, the underlying modeling technique, Azure architecture and Drift Detection integration, all wrapped in an end-to-end ML Ops process. This solution has been highlighted as a marquee Microsoft Customer Success Story: https://customers.microsoft.com/EN-US/story/768972-translink-travel-and-transportation-azure"

Technical level? (2/5)

What is unique about this talk?

This is a unique Case Study as it involves so many ML models. I don't believe there are many MLOps implementations within Microsoft (or elsewhere) that involve upwards of 18,000 models running in production today.

What are some infrastructure and languages discussed?

(NOT SURE)

2-3 topics for post-discussion?

(NOT SURE)

What you'll learn?

System architecture of a MLOps system at scale that is deployed into a production environment.

Prerequisite Knowledge

Basic knowledge of MLOps paradigm and the concept of CI/CD for ML models.

You have Successfully Subscribed!

Simplify ML Pipeline Automation and Tracking using Kubeflow and serverless functions, Yaron Haviv, Founder and CTO, Iguazio

Abstract: Simplify ML Pipeline Automation and Tracking using Kubeflow and Serverless Functions

What you'll learn?

The process of moving from data-science research to production pipelines is long and resource consuming, new practices like MLOps and tools like Kubeflow (ML toolkit and pipeline management over Kubernetes) are emerging to provide the equivalent of CI/CD for data science projects, but this require dedicated ML engineering teams to translate data-scientists/engineers work to production ready code. Serverless can simplify data science by automating the process of code to container and enables users to add instrumentation and auto-scaling with minimum overhead. However, serverless has many limitations involving performance, lack of concurrency, lack of GPU support, limited application patterns and limited debugging possibilities. Yaron Haviv will introduce Kubeflow, and how it works with Nuclio and MLRun, open source projects enabling serverless data-science and full ML lifecycle automation over Kubeflow. Yaron will show real-world examples and a demo and , explain how it can significantly accelerate projects time to market and save resources.

You have Successfully Subscribed!

Programming knowledge, preferably Scala or Java. General understanding of Machine learning and model serving

You have Successfully Subscribed!

Alice Gibbons

Technical Specialist of Microsoft

Alice joined Microsoft from university as part of the Microsoft Aspire Program where she held roles as a Technical Specialist and Cloud Solution Architect in the Azure application development space.

Currently, as a Cloud Native Global Black Belt, Alice helps customers with application modernization through architecture design sessions, hackathons, and by getting hands-on with proof of concepts.

These days she is enthusiastic about emerging OSS technologies, the Kubernetes ecosystem, and devops tooling. She is also a former front-end developer and holds a degree in Computer Science and Math from the University of Victoria.

Talk: MLOps at Scale: Predicting Bus Departure Times using 18,000 ML Models

You have Successfully Subscribed!

Hubert Duan

Cloud Solution Architect of Microsoft

Hubert Duan is a Data Scientist as part of the National AI team at Microsoft Canada, focused on solving business problems using data science and machine learning on Microsoft’s cloud Azure platform.

He has 10 years of professional experience in data science, and has 7 peer-reviewed paper publications with 1 granted patent. Hubert’s areas of specialization include retail recommendation engines, transportation departure predictions, predictive analytics, big data with Spark, and ML Ops.

Talk: MLOps at Scale: Predicting Bus Departure Times using 18,000 ML Models

You have Successfully Subscribed!

Allal Houssaïni

Director of Data Business Unit

Talk:

You have Successfully Subscribed!

Azam Rabiee

Post-doc researcher at KAIST, Korea 2017-2019

Talk:

You have Successfully Subscribed!

Babak

CEO and ML Mastermind Fascilator, Machine Learning Mastermind

Talk:

You have Successfully Subscribed!

Benedikt Koller

CTO of maiot GmbH

I'm a seasoned SRE/Opsguy with 10+ years experience in data-heavy companies (ecommerce, SaaS, advertising). For over two years now I'm one of two CTOs of maiot, a Munich-based AI startup.Originally focused on predictive maintenance / asset optimization of industrial assets and commercial vehicles, we're now making our internal tech stack available to a broad audience.

Originally focused on predictive maintenance / asset optimization of industrial assets and commercial vehicles, we're now making our internal tech stack available to a broad audience.

Talk: A tale of a thousand pipelines

You have Successfully Subscribed!

Burak Özen

Currently Amsterdam-based and working at eBay. Senior Data Scientist with M.Sc degree in Machine Learning and professional experience in teaching Data Science classes.

Special Interests : Human Behaviour Patterns, Predictive Analytics, Predictive Marketing, Data Mining, Machine Learning and Deep Learning -- Creating predictive models which mainly seek to answer the question: WHO is likely to buy WHICH product and WHEN? Domain Knowledge Expertise: e-Commerce, Telecommunication and Classifieds (Marketplaces)

Talk:

You have Successfully Subscribed!

Talk:

You have Successfully Subscribed!

Dariuš Butkevičius

Talk:

You have Successfully Subscribed!

David Talby

CTO of John Snow Labs

David Talby is a chief technology officer at John Snow Labs, helping fast-growing companies apply NLP and AI to solve real-world problems in healthcare, life science, and related fields.

David has extensive experience in building and operating web-scale data science and business platforms, as well as building world-class, agile, distributed teams.

Previously, he led business operations for Bing Shopping in the US and Europe with Microsoft’s Bing Group and built and ran distributed teams that helped scale Amazon’s financial systems with Amazon in both Seattle and the UK.

David holds a PhD in computer science and master’s degrees in both computer science and business administration.

Talk: Lessons learned building natural language processing systems in healthcare

You have Successfully Subscribed!

Don Ward

Sr Software Engineer, Quicken Loans

Don Ward is a Senior Software Engineer on the Data Science Team at Quicken Loans, the nation's largest mortgage lender, which is based in Detroit, Michigan.

As the previous Director of Mobile Development at Quicken Loans, he has been building mobile apps for the past 10 years. He leads the local chapters of the Google Developers Group in Detroit and Windsor, Canada. Always up for a good challenge, Don entered and won a 24-hour hackathon leveraging the power of wearables such as the Apple Watch.

Recently, Don purchased an oversized boat and has been honing his captain skills on the Detroit River and Great Lakes. As an avid lover of all things Android, he's waiting on the boating industry to release an excellent Android chart plotter so he can write apps for his boat. To stay up-to-date with Don, follow him on Twitter at @donwardpeng.

Talk: Edge AI - The Next Frontier

You have Successfully Subscribed!

Eduardo Piairo

Operations Engineer, Architect - Deeper nsights

Operations Architect @ Deeper Insights that enjoys build software, pipelines and communities.

Always ready to learn the path to production using source control, continuous integration and continuous delivery for applications, databases and infrastructure.

The deployment pipeline is his favourite technical and cultural tool.

Talk:

You have Successfully Subscribed!

Elle O’Brien

Data Scientist of Iterative.ai

Elle O'Brien is a data scientist at Iterative, Inc. (the team behind DVC).

She holds a PhD from the University of Washington and has presented about data science, AI, and statistical methods at the University College of London, Rev Data Science Summit, the American Statistical Association's Symposium on Data Science & Statistics, and more.

Previously, she conducted research in computational neuroscience and speech perception, and worked as the Chief Scientist at Botnik Studios, an AI-comedy writing collective.

Talk: Adapting continuous integration and continuous delivery for ML

You have Successfully Subscribed!

Flavio Clesio

Machine Learning Engineer, MyHammer AG

Flavio Clesio is a Machine Learning Engineer (NLP, CV, Marketplace RecSys) and at the moment works at MyHammer AG, where he helps build Core Machine Learning applications to exploit revenue opportunities and automation in decision making.

Prior to MyHammer, Flavio was a Data Intelligence lead in the mobile industry, and business intelligence analyst in financial markets, specifically in Non-Performing Loans. He holds a master’s degree in computational intelligence applied in financial markets (exotic credit derivatives).

Talk:

You have Successfully Subscribed!

Ivan

Sr. Associate Customer Advisor | CI & Analytics | ModelOps | Decisioning - SAS

As a member of Pre-Sales CI & Analytics Support Team, I'm specialized on ModelOps and Decisioning I've been involved in operationalizing analytics using different Open Source technologies in a variety of industries. My focus is on providing solutions to deploy, monitor and govern Model Life Cycle in Production and optimize business decisioning processes.

To reach this goal, I work with software technologies (SAS Viya platform, Container, CI/CD tools) and Cloud (AWS)

Talk:

You have Successfully Subscribed!

Jan Zawadzki

Head of AI – Car.Software.Org

Jan Zawadzki is the acting Head of AI of Volkswagen’s new company Car.Software.Org, which centralizes automotive software development to develop an automotive operating system. As a former global Management Consultant and Data Scientist, Jan has experience in delivering AI Projects, developing an AI Strategy, and building ML Teams.

Jan is passionate about advancing the automotive industry through Machine Learning and sharing his knowledge in the fields of Project Management and AI. He is a top contributor to the “Towards Data Science” Publication on Medium and enjoys supporting the team around Deep Learning Luminary Andrew Ng.

Talk: The Do’s and Don’ts of Delivering AI Projects: A Practitioners Guide

You have Successfully Subscribed!

Kaushik Roy

Location Insights Specialist, Municipal Property Assessment Corporation

Kaushik Roy is a Location Insights specialist at the Municipal Property Assessment Corporation in Pickering. He holds a Principal Researcher position at the AI Hub in Durham College, Oshawa.

His postgraduate research was done at University of Windsor, where he produced publications on computational biology. He has over 12 years experience in delivering return on investment through analytical and data products for financial, telecom, and government clients.

Talk: Are Your Models Location Smart?

You have Successfully Subscribed!

Lina Palianytsia

Lead Machine Learning Engineer of GlobalLogic

Machine Learning Engineer with a focus on quality and stability of ML based products.

Major in Math and Finance.

Talk: Talk: Metrics: Holistic Health Metrics of ML-Based Product

You have Successfully Subscribed!

Mark Weber

Research of MIT-IBM Watson AI Lab

Mark Weber is (@markrweber) the Strategy & Operations Lead at the MIT-IBM Watson AI Lab, a $250 million partnership funding over 200 scientists making fundamental breakthroughs in AI.

Through the lab’s corporate Membership Program and Advanced Prototyping Team, which he leads, Mark works closely with global industry leaders on the creative challenge of bridging core AI science to real-world impact.

Prior to IBM Research, Mark earned his MBA from MIT and conducted applied research in blockchain technology at the MIT Media Lab’s Digital Currency Initiative, where he led the group’s engagements with member companies and governments.

Mark has published works in top peer-reviewed venues in both business and technology, and speaks to diverse audiences around the world on the topics AI, blockchain, and sustainable development.

Talk: Practical Syllogisms for AI Fairness

You have Successfully Subscribed!

Moez Ali

Data Scientist, PyCaret

Moez Ali is a data scientist, founder and author of PyCaret. He is active community contributor and recently he has open-sourced machine learning library in python.

Talk: Machine Learning with PyCaret

You have Successfully Subscribed!

Per Nyberg

Chief Commercial Officer, Stradigi AI

Per is a global technology and AI executive with over two decades of experience, known for his empowering leadership style and his measured approach to innovation strategies. Prior to joining Stradigi AI, Per held a number of leadership roles at Cray Inc.

Most notably, he was Vice President of Market Development for the company’s Artificial Intelligence and Cloud solutions. In this role, Per brought machine learning and deep learning solutions to market for global enterprise clients across multiple verticals.

Today, he oversees all growth initiatives at Stradigi AI, including marketing, customer success, and business development. He lives in Montreal with his two children and wife, and envisions a world where AI is both ubiquitous and bettering the lives of people everywhere.

Talk:

You have Successfully Subscribed!

Rogelio Cuevas

Data Scientist, Senior Manager, TD

Rogelio Cuevas is a Senior Manager, Senior Data Scientist at TD bank where, as part of a team, offers internal consultancy data science services across different lines of business. Prior to his current tenure with TD, he worked as a Data Scientist at Scotiabank where his main responsibility was developing models for risk management within retail banking.

His experience also includes collaborations with IBM through its Cognitive Class initiative (formerly known as Big Data University) as well as on-going engagement with a mid-size San Francisco based company where he offers remote mentoring and coaching of technical and business-oriented professionals for successful adoption of data science skills and practices.

Rogelio has also been invited speaker and guest lecturer at University of Toronto, Rotman School of Business and RiskNet; a United Kingdom based organization which offers, among other services, machine learning training for practitioners and decision makers.

As part of his former academic professional life, he performed research in research-oriented institutions such as The University of Western Ontario, Duke University, Fermi National Accelerator Laboratory and McMaster University, where he received his PhD.

Talk: Embracing an MLOps mindset in the financial sector: From proof-of-concept notebooks to production-ready solutions.

You have Successfully Subscribed!

Sharat Singh

CEO and Chief Architect, Quadrical.ai

Sharat is a Technology leader who builds and manages the full life cycle of AI products at both large enterprises (Adobe, MakeMyTrip) as well as startups. A graduate of IIT-Delhi and NYU, Sharat has lived and worked in eight major cities in three countries.

Talk: Engineering and Production Techniques for managing feature-drift through Quality aware models

You have Successfully Subscribed!

Sivan Metzger

Managing Director, MLOps, DataRobot - DataRobot

Sivan brings more than 20 years of enterprise software business leadership experience, from Mercury Interactive (acq.HP), Kenshoo Inc., co-founding and leading ParallelM - The MLOps Company [Acq. by DataRobot], and now leading the MLOps business as part of DataRobot.

Talk:

You have Successfully Subscribed!

Subhodeep Moitra

Research Software Engineer of Google

Subhodeep Moitra is a researcher in the Google Brain team at Montreal working on program synthesis. He’s been at Google since 2015 and has previously worked on (1) Dopamine, a library for deep reinforcement learning and (2) Google Vizier, a service for bayesian optimization and AutoML.

Before Google, Subhodeep obtained his PhD in applied machine learning from Carnegie Mellon university on generative models of protein sequences. At present, he lives in the suburbs with his wife and two cats and enjoys the outdoors of the Laurentians.

Talk: Deep Learning for Program Repair

You have Successfully Subscribed!

Timo Mechler

Product Manager & Architect, Smartdeploy AI

Timo is a Product Manager and Architect at SmartDeployAI. He has close to a decade of financial data modeling experience working both as an analyst and strategist in the energy commodities sector.

At SmartDeployAI he now works closely with product development and engineering teams to solve interesting data modeling challenges.

Talk: Simplifying The Creation of Machine Learning Workflow Pipelines For Near Real-Time Inferencing At Scale On Kubernetes Using Kubeflow

You have Successfully Subscribed!

Wei Qu

Senior Machine Learning Engineer, Domaingroup

Talk:

You have Successfully Subscribed!

Yiannis Kanellopoulos

Founder of Code4Thought

Yiannis Kanellopoulos has spent the better part of two decades analyzing and evaluating software systems in order to help organizations address any potential risks and flaws related to them. (In his experience, these risks or flaws are always due to human involvement.)

With his startup, Code4Thought, Yiannis is turning his expertise into democratizing technology by rendering algorithms transparent and helping organizations become accountable.

He's also a founding member of Orange Grove Patras, a business incubator sponsored by the Dutch Embassy in Greece to promote entrepreneurship and counter youth unemployment.

Yiannis holds a PhD in computer science from the University of Manchester.

Talk: How F.A.T. is your ML Model? Quality in the era of Software 2.0.

You have Successfully Subscribed!

Ali Hirji

AI Hub and Centre for Cybersecurity Innovation lead ,Ali Hirji has over 14 years of experience working on a variety of technology implementations in the government, academic and not for profit sectors.

With a specific interest in broadband communications, Ali has held senior roles in projects to enable remote connectivity, implement technical trainings, bolster cyber security frameworks and enhance access to mission critical applications.

He has also taken a lead role in securing funding for over 15 related projects and established numerous public private partnerships. Ali is completing his PhD in communications and holds multiple research and teaching positions.

Talk: AI to AEYE : See the Value of AI as an Investor

You have Successfully Subscribed!

Anthony

Talk:

You have Successfully Subscribed!

Ginger Grant

Talk:

You have Successfully Subscribed!

Ravi Bhanabhai

Talk:

You have Successfully Subscribed!

Luna Feng

Research Scientist, Thomson Reuters Center for AI and Cognitive Computing

Luna is a Research Engineer at Thomson Reuters Cognitive Computing Center. In this role, she takes initiatives to understand various business needs and incorporate AI capabilities to reduce the costs, increase the efficiencies and improve the bottom line performance.

Her current assignment is utilizing the state-of-the-art AI models to solve the real-world problems, especially in the legal domain with Natural Language Processing techniques such as entity extraction, document classification, information retrieval to provide insight of a document to the legal practitioners to improve their work efficiencies.

She is passionate about teaching people how to apply AI into practical use including how to formulate a real-world problem as an AI problem, and how to identify the challenges and overcome them using various techniques. Most recently, she is fine tuning existed language models towards the legal domain to address the challenges due to the nature of the legal documents. Outside of work she enjoys traveling with her families and friends and taking pictures of them.

Talk: Create harmony between ML engineers and researchers

You have Successfully Subscribed!

Lina Weichbrodt

Machine Learning Lead Engineer, DKB Bank

Lina has 8+ years of industry experience in developing scalable machine learning models and bringing them into production. She currently works as the Machine Learning Lead Engineer in the data science group of the German online bank DKB. She previously worked at Zalando, one of Europe’s biggest online fashion retailers, where she developed real-time, deep learning personalization models for more than 32M users.

Talk: How To Monitor Machine Learning Stacks - Why current monitoring is unable to detect serious issues and what to do about it

You have Successfully Subscribed!

Ira Cohen

Co Founder, Chief Data Scientist and VP Fish care of Anodot

Ira is the chief data scientist at Anodot, working on learning algorithms for analyzing time series signals at large scale - from anomaly detection, clustering and forecasting. Prior to Anodot, Ira was Chief Data Scientist at HP Software, defining and developing advanced data analytics & big data initiatives.

Before that Ira was a senior researcher at HP Labs, leading R&D in machine learning and data mining for analyzing large scale event streams. He is the author of numerous patents and publications and holds a PhD in Electrical and Computer Engineering from the University of of Illinois at Urbana Champaign.

Talk: Machine learning monitoring Machine learning: Scalable monitoring of machine learning models in production environments

You have Successfully Subscribed!

John Peach

Principal Data Scientist, Oracle

"A modern polymath, John possesses a unique and diverse set of skills, knowledge and experience. Having earned advanced degrees in Mechanical Engineering, Kinesiology and Data Science, his expertise focuses on machine learning, solutions to novel and ambiguous problems. He has a proven history of taking a problem from ideation to production by using a logical, but creative, data-driven approach. As a highly skilled Data Scientist, he has developed new techniques, lead teams, developing innovative data products and is a trusted advisor to decision-makers.

John is a natural leader, customer-focused, excellent communicator and problem-solver. He loves new challenges and opportunities. His extensive background in software development and modelling serves him well. His curiosity, creativity, focus and attention to detail have resulted in a track record of discovering hidden secrets in data.

As a Sr. Applied Data Scientist at Amazon, John lead the Alexa Skill Store Science team. He worked closely with engineering to build systems that enabled Alexa customers to engage with third-party applications, skills. He built machine learning models to arbitrate between skills, entity resolution, search, and personalization.

Currently, John is a Principal Data Scientist at Oracle. He works on the Data Science service as part of the Oracle Cloud Infrastructure team. Leveraging his extensive hands-on experience building machine learning models, he is now defining the tooling to improve the data science workflow. This interest grew out of the challenges that he and his team members have faced working with data at scale in a logical, rigorous and reproducible way.

John fosters the growth of scientists by starting the Amazon Machine Learning University in Irvine and the Alexa wide Data Science Excellence program. He frequently gives talks at universities and conferences. He is working to improve upon and formalize data science best practices. The focus has been on reproducible research. To that end, he has developed an approach to improve data validation and reliability by using data unit tests. He has also developed the Data Science Design Thinking concept; to formalize and increase the efficiency of the analysis process. He also coordinates the largest R meetup group in Southern California (OCRUG)."

Talk: Literate Statistical Programming

You have Successfully Subscribed!

Xiaoming Zhang

Senior Data Scientist – Loblaw Digital

Xiaoming Zhang is a senior data scientist at Loblaw Digital. She has been focusing on personalization and recommendation for digital shopping experience, as well as ML model deployment infrastructure to productionize the data science services.

Xiaoming came from a physics background, with a focus on geophysics for her phd studies and condensed matter theory for her master studies and undergrad.

Talk: Productionizing ML Models at Online Shopping at Loblaws ; One of Canada’s Largest Grocery Store Chains

You have Successfully Subscribed!

Dean Wampler

Head of Developer Relations – Anyscale

Industry expert in ML engineering, streaming data, and Scala.

O'Reilly author and frequent public speaker.

Works for Anyscale. Lives in Chicago.

Talk: Ray and how it enables easier DevOps

You have Successfully Subscribed!

Muna Khayyat

Vice President, Machine Learning Expert – Morgan Stanley

Muna Khayyat is a Machine Learning Specialist at the Corporate and Funding Technology department at Morgan Stanley. She has a Ph.D. from the Computer Science and Software Engineering department at Concordia University, Montreal, Canada. She is also an affiliated associate professor at Concordia University. While pursuing her Ph.D. Muna has worked as a researcher at Concordia's Centre of Pattern Recognition and Machine Intelligence (CENPARMI). Muna is an author of numerous technical papers on Machine Learning and Pattern Recognition, she is specialized in Language Models, Ensemble Classifiers and Cursive language recognition.

Talk: Machine Learning in Finance, Best Practices and Insights

You have Successfully Subscribed!

Stacey Svetlichnaya

Deep Learning Engineer – Weights & Biases

Stacey Svetlichnaya is a deep learning engineer at Weights & Biases, building developer tools for accessibility, transparency, and collaboration in deep learning. Her research in computer vision and natural language processing includes image aesthetic quality and style classification, object recognition, photo caption generation, and language modeling for emoji. She has worked extensively on image search, data pipelines, productionizing machine learning systems, and automating content discovery and recommendation on Flickr, the first and longest-active photo-sharing website. Prior to Flickr, she developed a visual similarity search engine with LookFlow, a startup of 5 engineers which Yahoo acquired in 2013. Stacey holds a BS ‘11 and MS ’12 in Symbolic Systems from Stanford University.

Talk: Hyperparmeter Tuning With a Focus on Weights & Biases Sweeps

You have Successfully Subscribed!

Denise Gosnell

Chief Data Officer of DataStax

Denise Koessler Gosnell is the chief data officer at DataStax, where she applies her experiences as a machine learning and graph data practitioner to make more informed decisions with data. Her career centers on her passion for examining, applying, and advocating the applications of graph data. She has patented, built, published, and spoken on dozens of topics related to graph theory, graph algorithms, graph databases, and applications of graph data across all industry verticals.

Previously, Denise created and led the global graph practice, a team that builds some of the largest distributed graph applications in the world at DataStax. She also has deep experience working in data engineering and data science roles in the bioinformatics, telecommunications, and healthcare industries. Denise earned her PhD in computer science from the University of Tennessee as an NSF fellow. Her research coined the concept “social fingerprinting” by applying graph algorithms to predict user identity from social media interactions.

Talk: Modeling, Querying, and Seeing Time Series Data within a Self-Organizing Mesh Network

You have Successfully Subscribed!

Jisheng Wang

Senior Director of Data Science of Mist, a Juniper Company

Dr. Jisheng Wang has 10+ years of experience applying state-of-the-art big data and data science technologies to solve challenging enterprise problems including: security, networking and IoT. He is currently the Head of Data Science at Mist Systems, and leads the development of Marivs – the first AI-driven virtual network assistant that automates the visibility, troubleshooting, reporting and maintenance of enterprise networking.

Before joining Mist, Jisheng worked as the Senior Director of Data Science in the CTO office of Aruba, a Hewlett-Packard Enterprise company since its acquisition of Niara in February 2017. As the Chief Scientist at Niara, Jisheng led the overall innovation and development effort in big data infrastructure and data science. He also invented the industry’s first modular and data-agonistic User and Entity Behavior Analytics (UEBA) solution, which is widely deployed today among global enterprises. Before that, Jisheng was a technical lead in Cisco responsible for various security products.

Jisheng received his Ph.D. in Electric Engineering from Penn State University, and is also a frequent speaker at AI/ML conferences, including: O'Reilly Strata AI, Frontier AI, Spark Summit, Hadoop Summit and BlackHat.

Talk: Automated Pipeline for Large-Scale Neural Network Training and Inference

You have Successfully Subscribed!

Jonathan Peck

Technical Advocate, GitHub

A full-stack developer with two decades of industry experience, Jon Peck constantly strives to make technical concepts digestible -- demonstrating the value of new technology at every level, from developers through execs.

Former speaker at DeveloperWeek, OSCON, AI Next, O'Reilly AI, ODSC, API World. Former developer/advocate at Mass General Hospital, Cornell University, Algorithmia. Current technical advocate at GitHub.

Talk: GitHub Actions in Action

You have Successfully Subscribed!

Clement Chung

Applied Science Manager of Amazon Alexa

Clement Chung has over 20 years of developing business solutions and conducting applied research in deep learning, natural language processing and understanding, predictive and advanced analytics, computational biology, recommendation systems. He is passionate about building AI and Data Science teams and has helped a number of successful startups and large corporations.

Clement is currently an Applied Science Manager with Amazon Alexa in Toronto, where he and his team develop and apply state-of-the-art automatic speech recognition, natural language understanding, transfer learning and model compression to bring Alexa on-device. Prior to Amazon, he held leadership positions with Wave Financial, Freckle IoT, Rubicon Project and Chango, and holds a Ph.D. from University of Toronto Machine Learning Group.

Talk:

You have Successfully Subscribed!

Maximo Gurmendez

Founder and Chief Engineer of Montevideo Labs

Maximo holds a master’s degree in computer science/AI from Northeastern University, where he attended as a Fulbright Scholar. As Chief Engineer of Montevideo Labs he leads data science engineering projects for complex systems in large US companies. He is an expert in big data technologies and co-author of the popular book ‘Mastering Machine Learning on AWS.’ Additionally, Maximo is a computer science professor at the University of Montevideo and is director of its data science for business program.

Talk: Smart Data Products: From prototype to production

You have Successfully Subscribed!

Isaac Faber Ph.D.

Chief Data Scientist at U.S. Army AI Task Force

Chief Data Scientist building tools for DoD decision-makers. Currently active military at the United States Army AI Task Force.

Talk: Building an AI Capability in the United States Army

You have Successfully Subscribed!

Yaron Haviv

Founder and CTO, Iguazio

Yaron Haviv is a serial entrepreneur who has deep technological experience in the fields of data, cloud, AI and networking. As the CTO of Iguazio, Yaron defines the company’s vision and strategy for the company’s data science platform and headed the shift towards real-time AI.

He also initiated and built Nuclio, a leading open source serverless platform with over 3,000 Github stars. Prior to Iguazio, Yaron was the Vice President of Datacenter Solutions at Mellanox, where he led technology innovation, software development and solution integrations.

He was also the CTO and Vice President of R&D at Voltaire, a high-performance computing, IO and networking company. Yaron is an active contributor to the CNCF working group and was one of the foundation’s first members. He presents at major industry events and writes tech content for leading publications like TheNewStack, Hackernoon, DZone, Towards Data Science and more.

Talk: Simplify ML Pipeline Automation and Tracking using Kubeflow and serverless functions

You have Successfully Subscribed!

Amir Jafari

Talk:

You have Successfully Subscribed!

Olivier Blais

Co-founder and VP Data Science of Moov AI

Olivier is a data science expert whose leading field of expertise and cutting-edge knowledge of AI and machine learning led him to support many companies’ digital transformations, as well as implementing projects in different industries.

He has led the data team and put in place a data culture in companies like Pratt & Whitney Canada, L’Oréal and GSoft.

Olivier is the laureate of the prestigious “30 under 30” prize (Infopresse – 2019). He is co-author of a patent for an advanced algorithm that evaluates the credit worthiness of a borrower.

Talk: Validate and Monitor Your AI and Machine Learning Models

You have Successfully Subscribed!

Abstract:

You have Successfully Subscribed!

Abstract:

You have Successfully Subscribed!

Abstract:

You have Successfully Subscribed!

Are your models 'Location' smart?, Kaushik Roy, Location Insights Specialistb

Abstract: What was the promised land? A location. What do we protect with all our might? Our Borders? The Royal Bank of Canada considers any account login from Russia a red flag. Starbucks considers hundreds of locations before deciding on one. The use of AI techniques in urban analytics and the earth sciences as such is not new. A combination of multiple techniques aids in integrating autonomous vehicles with intelligent transport systems by incorporating real-time information gathered from traffic cameras and other sensors. These include image classification, object detection, scene segmentation, simulation and interpolation, link prediction, (natural language based) retrieval and question answering, on-the-fly data integration, geo-enrichment, and many others. From agriculture to space travel, geospatial data enables processes like no other kind of data. To acquire, warehouse, analyze, and serve such data and models, one has to be an experienced and cautious polymath with a penchant for big data. This talk aims to cover a gamut of topics which are the lifeblood of the GEOINT (Geospatial Intelligence) lifecycle. Spatial data science (SDS) uses cases are not your average correlation and regression. The data itself produces the first barrier, as it is not tabular information in its raw form. Simply put, the rules are Spatial Data Engineering (SDE) are totally different, as basic SQL was never conducive to geometrical data processing. To that end, the Open Geospatial Consortium (OGC) defines a number of standards, both for data models and for online services, that has been widely adopted in the GEOINT community. This has led to a number of software development efforts, online data archives, and open source suites. Finally, we are going to explore the tooling ecosystem from both commercial and open source perspectives and consider pros and cons. Please feel free to ask any questions, as I would be happy to share my experiences and make it an interactive learning experience.

Technical level? (2/5)

What is unique about this talk?

I have over 12 years of experience handling both conventional and geometrical data. Over time I have turned wounds into wisdom, and produced lot revenue for my clients. The challenges I have faced are relevant to every data professional and business. Also, to be honest, spatial data is one of the most expensive sources of data,and not everyone has the luxury to have had access to such volumes of it as I have over the years. I think this is why my speech is unique.

2-3 topics for post-discussion?

Spatial Data Engineering, Open source geospatial projects, revenue generation through spatial products, Location Intelligence, Spatial regression, Geovizualisation

What you'll learn?

Google is the biggest competitor in this space, and has a high chance to bias your search results to cloud your judgement. My speech explores and imparts from a neutral perspective of things. Also, the available literature in this space is so vast that it needs the human touch and controlled brevity that a google search cannot suffice. People will learn how to incorporate this into their own systems. I will talk abotu the stacks used etc. Commercial and open-source options- he'll walk through the pro's and con's of both.

Prerequisite Knowledge

AI/ML/Computational Geometry/SQL/Cartography

You have Successfully Subscribed!

Machine Learning with PyCaret, Data Scientist, PyCaret

Abstract: PyCaret is an open source, low-code machine learning used for rapid prototyping and developing ML pipelines for production.

You have Successfully Subscribed!

Abstract:

You have Successfully Subscribed!

Abstract:

You have Successfully Subscribed!

Abstract:

You have Successfully Subscribed!

Edge AI - The Next Frontier, Don Ward, Sr Software Engineer – Quicken Loans

Abstract: AI and ML is everywhere in our digital lives. Machine Learning models running in data centers around the globe analyze and make decisions for us based on the data they receive. Imagine if these decisions could be made immediately against real time data right at the source? Imagine having powerful decisions made in real time from your cell phone, or within your home appliances. This new field - called EdgeAI, is experiencing huge growth as the ability to running machine learning models on smaller devices improves yearly and our appetite for more intelligent devices grows along with it. With over 1 billion smartphones shipped in 2019 and 30 billion microcontrollers shipped in 2019 the opportunities to develop and run machine learning models on the 'edge' are immense. Join this talk as we cover the use cases and opportunities for EdgeAI spanning from smartphones to the smallest of microcontrollers. As an extra bonus, hands on demos will be provided of projects I have done with EdgeAI using smartphones and microcontrollers.

Technical level? (4/5)

What is unique about this talk?

As an enthusiast of EdgeAI I have applied it both a work at Quicken Loans as well as developing side projects using it. I will be covering 3 major areas of it - from mobile development, microprocessor development and microcontroller development using EdgeAI - all of which I have hands on experience with. My overall goal with this talk is to have the audience excited to jump in and get started asap.

What are some infrastructure and languages discussed?

Where should you start with EdgeAI? What hardware do I need?

2-3 topics for post-discussion?

Best practice for open source usage, Which AI framework to choose

What you'll learn?

What you'll learn? I am currently doing a 100DaysOfCode challenge on this top to build real world ML Models for EdgeAI. I will be bringing over 3 months of day to day development experience, struggles and solutions to the talk.

Prerequisite Knowledge

Technical level? (3/5)

What is unique about this talk?

This talk specifics and learning from real systems and applied research we implemented in our own systems -- all of techniques shared would include limitations, metrics and resultant improvements. There are a lot of anecdotal ideas and approaches being talked about which sounds like it should work, but doesn't scale or work.

2-3 topics for post-discussion?

Practical and production experiences with: a) Monolithic model vs mini-models for better fitment over domain vector-space or clusters b) Model quality variance over domain vector space c) Ensembling d) Model promotion rules -- stochastic and metrics sufficient? e) multi-model AI pipelines

What you'll learn?

Deep insights on challenges and approaches to implement complex AI systems resilient to feature-drift

Prerequisite Knowledge

Real-world experience building, deploying, and managing AI systems in production

You have Successfully Subscribed!

Abstract:

You have Successfully Subscribed!

Abstract:

You have Successfully Subscribed!

Productionizing ML Models at Online Shopping at Loblaws ; One of Canada's Largest Grocery Store Chains, Xiaoming Zhang, Senior Data Scientist, Loblaw Digital

Abstract: In this talk, I will present our recent work about how we productionize ML models as managed microservices through ci pipelines, achieve continuous improvement through online AB or MAB testing, and the technical implementation by leveraging the cloud platform and open source tools including seldon-core.

Technical level? (4/5)

What is unique about this talk?

At Loblaw Digital, our data science team have the full ownership of the services. We need to stick with engineering best practices in our dev work, to build services that meets the requirements of performance, scalability and security. Our e-commerce platform is in the process of pulling out the core capacities from a very monolithic architecture to microservices, which poses unique challenges to us. I will share the stories of the trade-offs we made along the way when we need to co-exist with a legacy system If your organization is facing similar challenges, our experience may help navigate your journey.

What you'll learn?

How to productionize machine learning models for an e-commerce business in real-world setting, the challenges we faced and how we tackle them, reasonings behind our design choice

Prerequisite Knowledge

Cloud services, containerization and orchestration, ci pipeline

You have Successfully Subscribed!

Ray and how it enables easier DevOps, Dean Wampler, Head of Developer Relations, Anyscale

Abstract: Ray was developed at U. C. Berkeley’s RISELab in response to the performance challenges and compute requirements for reinforcement learning, hyperparameter tuning, model serving, and other scenarios requiring large-scale computation using clusters. Ray is a Python library for distributing Python tasks across a cluster with an easy to use API. It also provides facilities for distributed state management.
In this talk I’ll discuss several ML APIs implemented with Ray, including RLlib for reinforcement learning, Tune for hyperparameter tuning, and a few others. I’ll explain the performance and scalability problems they solve and how Ray facilitates these solutions.

You have Successfully Subscribed!

Machine Learning in Finance, Best Practices and Insights, Muna Khayyat, Vice President, Machine Learning Expert, Morgan Stanley

Abstract: Intelligent systems are gaining great attention due to the rapid involvement of these systems into our day-to-day tasks. This is because of the vast amount of data and more affordable computing power. Consequently, intelligent systems are emerging into financial technologies (FinTech) and reshaping the financial services industry. This will most likely significantly change the financial strategies and customer behaviors. All the aforementioned changes require awareness and changes in the mindset of both service providers and consumers. A better understanding of the machine learning capabilities, job market change, and the policies related to privacy and ethics, is needed. There are already Machine Learning Applications in Finance, we will discuss the life cycle, stages and roles needed to build these Applications (Intelligent Systems).

What you'll learn?

Machine Learning (ML) Applications in Finance (why, how, what) Solution Implementation Live cycle of ML in finance ML stages and Roles ML Design (Training and Testing)

You have Successfully Subscribed!

Hyperparmeter Tuning With a Focus on Weights & Biases Sweeps, Stacey Svetlichnaya, Deep Learning Engineer, Weights & Biases

Abstract: Deep learning models are incredibly powerful but often tricky to adapt to new use cases. Whether you’re finetuning a pretrained net on new data, trying to build an intuition for a complex model, or throwing a variety of architectures at a unique problem, hyperparameter exploration can help. I will share high-level approaches and useful visualizations for hyperparameter search, grounded in concrete examples from semantic segmentation, language understanding, and other domains. Though I will focus on Weights & Biases Sweeps as a comprehensive tool for this task, these practices are framework-agnostic, and I hope they can accelerate your progress regardless of your dev setup.

Technical level? (4/5)

What is unique about this talk?

I have extensive experience productionizing ML models, reproducing other people's code repositories for ML projects, and developing the Weights & Biases Sweeps product to be both easy to use with powerful defaults and flexible/customizable for more advanced cases. I hope to present a variety of examples and some general insights/best practices for figuring out how to get started with a deep learning model, how to improve an existing model, and how we might be more principled and efficient in these tasks.

What you'll learn?

You will learn some ways of approaching hyperparameter tuning in deep learning, with concrete examples from computer vision and natural language processing. You will learn about the potential and the limitations of hyperparameter search and the difference between exploration and optimization. I will present some helpful practices for both cases and some tools to try in common scenarios.

Prerequisite Knowledge

Know what it's like to train a deep learning model / ideally have trained one yourself

You have Successfully Subscribed!

Literate Statistical Programming, John Peach, Principal Data Scientist, Oracle

Abstract: Science is facing a crisis around reproducibility and data science is not immune. Literate Statistical Programming is a workflow that binds the code used in an analysis to the interpretation of the results. While this creates reproducibility it also addresses issues around, auditing, reusability and allows for rapid iteration and experimentation. This talk will describe a workflow that I have successfully used on small-scale datasets in start-ups and on massive-scale problems in my work at Oracle and Amazon, Alexa. The talk will cover the tooling, workflow, and the philosophy you need to master Literate Statistical Programming.

What you'll learn?

You will learn the philosophy of Literate Statistical Programming (LSP). LSP is a fast and flexible workflow for data science projects. This talk will teach you the guiding principals, introduce some of the tools and how you can transition an ad-hoc workflow into one that is reproducible, auditable and allows for rapid iteration.

You have Successfully Subscribed!

How To Monitor Machine Learning Stacks - Why Current Monitoring is Unable to Detect Serious Issues and What to Do About It, Machine Learning Lead Engineer, DKB Bank

Abstract: Monitoring usually focusses on the “four golden signals”: latency, errors, traffic, and saturation. Machine learning services can suffer from special types of problems that are hard to detect with these signals. The talk will introduce these problems with practical examples and suggests additional metrics that can be used to detect them. A case study demonstrates how these new metrics work for the recommendation stacks at Zalando, one of Europe’s largest fashion retailers.

You have Successfully Subscribed!

Create harmony between ML engineers and researchers, Luna Feng, Research Scientist, Thomson Reuters Center for AI and Cognitive Computing

Abstract: I have been an engineer and a researcher in our team for the past years in different time periods. As an engineer, I work with researchers to refactor their code and make it production-ready. As a researcher, I collaborate with engineers to help them understand my algorithm and support their refactoring process. Experienced both of the roles allows me to understand the work on each side and pay attention to the points that can improve the efficiency on both sides.

As a researcher:
we experiment in notebook, so the code is usually not modulized, it’s hard to maintain and reuse it
we are not clear the line to draw between the performance of a model and the scalability of a model
we usually do not profile our code to measure the speed performance
we may not handle exceptions properly
it’s hard for us to write unittest to evaluate the output of a model
it’s challenging for us to reproduce results due to some randomness of a model
we may not be familiar with version control tools, or the best practice of it

As an engineer:
it’s hard to understand the researcher’s algorithms
results can be different if model run on GPU vs CPU
it’s hard to test the results because engineers do not know the expected results
researchers use Python (or even R) and there may be challenges in putting that code into production
if multiple modules are going into the production, we should separate the process for tokenization, text preprocessing, for example, to avoid duplicate work along modules, but researchers usually work independently at first, this brings the challenge to unify these process later on

So I listed a couple of points under each of the roles, and for each of the challenges, I am going to talk about the suggestions or tips, for example, write comments in code, knowledge sharing session etc.

Technical level? (2/5)

What is unique about this talk?

I have been an engineer and a researcher in our team for the past years in different time periods. As an engineer, I work with researchers to refactor their code and make it production-ready. As a researcher, I collaborate with engineers to help them understand my algorithm and support their refactoring process. Experienced both of the roles allows me to understand the work on each side and pay attention to the points that can improve the efficiency on both sides.

What you'll learn?

You'll learn the challenges to bring a ML solution into production regarding the collaborations between ML engineers and researchers and you'll learn the tips on each role to make the whole process more efficient.

Prerequisite Knowledge

Technical level? (3/5)

What is unique about this talk?

Localized examples from regional angel investment groups

2-3 topics for post-discussion?

Understanding the ROI of AI, cost-benefit analysis and investment decks

What you'll learn?

ROI stands for return on imagination - how do you imagine what AI can achieve while also being realistic

Prerequisite Knowledge

None

You have Successfully Subscribed!

5 Key Factors That Can Attract Investors to an AI company, Sheila Beladinejad, CEO/Technology Strategist/O Canada Tech

Abstract: Whether you are a startup and on the path of building up your AI-focused company from the ground up or an established company interested in attracting investors (banks, Private Equity firms, government agencies) in your AI solution or perhaps you are looking into acquiring an AI-based company to improve your value proposition in the market and have the upper hand against your competitors, do you know what the key success factors of an AI solution/company are?

Technical level? (3/5)

What is unique about this talk?

You will gain insight into what makes an AI solution/company a great candidate for investment.

2-3 topics for post-discussion?

Best practice for open source usage, Which AI framework to choose

What you'll learn?

Actionable insight into steps you can take to groom your company for future investments. For investors, key elements to consider when assessing a target with an AI solution/product. I will also share a few use cases of unsuccessful projects and what demotivated the investors to step back from their investment thesis objectives.

You have Successfully Subscribed!

Sheila Beladinejad

CEO/Technology Strategist/O Canada Tech

I am the founder and CEO of O Canada Tech. I have 20+ years of experience in Software Engineering with a special interest and expertise in Artificial Intelligence evaluation. I help Private Equity firms going through the merger and acquisition process by providing technical due diligence assessments of the target companies’ software infrastructure. Additionally, I work with executive management teams of technology companies to define and executing strategic roadmaps for digital transformation and adoption of AI. I am also the Ambassador of Women in AI for the city of Munich, Germany.

Talk: 5 Key Factors That Can Attract Investors to an AI company

You have Successfully Subscribed!

Enzo Vernon

ML Cloud Security Engineer, Choice Hotels International

Experienced machine learning engineer with a history of succeeding in the entertainment & hospitality industries. Building reinforcement learning models that implement deep neural networks and various temporal difference strategies is one of my passions. Skilled in AI, AWS, GraphQL, Hadoop, Java, Javascript, TensorFlow, Python, Spark, and SQL. Strong security professional with an enthusiasm for community development.

Talk: Automating Production Level Machine Learning Operations on AWS2

You have Successfully Subscribed!

Jules Damji

Jules Damji, Developer Advocate at Databricks

Jules S. Damji is a Developer Advocate at Databricks and an MLflow contributor. He is a hands-on developer with over 20 years of experience and has worked as a software engineer at leading companies such as Sun Microsystems, Netscape, @Home, LoudCloud/Opsware, VeriSign, ProQuest, and Hortonworks, building large-scale distributed systems. He holds a BSc and MSc in computer science and an MA in political advocacy and communication from Oregon State University, Cal State, and Johns Hopkins University, respectively.

Talk: Managing Machine Learning Experiments with MLflow

You have Successfully Subscribed!

Managing Machine Learning Experiments with MLflow, Jules Damji, Developer Advocate at Databricks

You have Successfully Subscribed!

Javier Buquet

Senior Data Engineer of Montevideo Labs

Javier holds a degree in Computer Science from ORT University (Montevideo) and since 2015 has been working with Montevideo Labs as a Senior Data Engineer for large big data projects. He has helped top tech companies to architect their Spark applications, leading many successful projects from design and implementation to deployment. He is also an advocate of clean code as a central paradigm for development.

Talk: Smart Data Products: From prototype to production

You have Successfully Subscribed!

Smart Data Products: From prototype to production, Javier Buquet, Senior Data Engineer, Montevideo Labs

What you'll learn?

You have Successfully Subscribed!

Ramzi Abdelmoula

AI Innovation Team Lead, General Motors

Ramzi Abdelmoula leads the AI Innovation Team at General Motors Canada where he focuses on leveraging AI technologies to create operational efficiencies and deliver higher level innovation. He undertakes designing AI POCs and taking them into production while maximizing their business value to customers.

Ramzi has multiple patents, defensive publications and papers in the field of AI & ML. He is also the winner of the “Coup de coeur du jury” award from the Toronto ReleveTO. Previously, he worked and consulted with multiple speech recognition and natural language Canadian start-ups.

Talk: Determining Successful ML Project

You have Successfully Subscribed!

Determining Successful ML Project, Ramzi Abdelmoula, AI Innovation Team Lead, General Motors

Abstract: AI research is exponentially expanding data science toolsets by the day, and will continue to do so as we explore new ways of leveraging and generating data. However, only 20% of AI related POCs make it to production. In this talk, we will explore how to build credibility with decision makers and how to select the right projects to work on to maximize production level success.

You have Successfully Subscribed!

Tanya Vucetic

Data Science, Onica, a Rackspace Company

Tanya Vucetic is a Data Science Architect at Onica, a Rackspace company. She specializes in AWS and cloud-based consulting engagements across a wide range of industries and use-cases.

Tanya holds an undergraduate degree in Economics from UCLA, MBA from The Hong Kong University of Science and Technology and is currently studying Computational Analytics at Georgia Tech.

Talk: Automating Production Level Machine Learning Operations on AWS

You have Successfully Subscribed!

Automating Production Level Machine Learning Operations on AWS, Tanya Vucetic, Data Science, Onica, a Rackspace Company

Technical level? (4/5)

What are some of the infrastructure/tools you plan to discuss

AWS based infrastructure, MLFlow, KubeFlow, Step Functions

What you'll learn?

How to bring CI/CD to your ML models and into production

Prerequisite Knowledge

Machine Learning and CI/CD basics

You have Successfully Subscribed!

Jian Chang

Staff Algorithm Expert, Alibaba Group

Dr. Jian Chang is a Staff Algorithm Expert at the Alibaba Group. Dr. Chang has led many innovation projects and R&D activities to promote data science best practices within large organizations.

He’s a frequent speaker at technology conferences, such as the O’Reilly Strata and AI Conferences, NVIDIA’s GPU Technology Conference, Hadoop Summit, DataWorks Summit, Amazon re:Invent and has published and presented research papers and posters at many top-tier conferences and journals. Dr. Chang holds a PhD from the Department of Computer and Information Science at University of Pennsylvania.

Talk: Building Intelligent Analytics Through Time Series Data

You have Successfully Subscribed!

Abstract:

You have Successfully Subscribed!

Jörg Schad

Head of Engineering and Machine Learning, ArangoDB Inc

Jörg Schad is Head of Machine Learning at ArangoDB. In a previous life, he has worked on machine learning pipelines in healthcare and finance, distributed systems at Mesosphere, and in-memory databases.

He received his Ph.D. for research around distributed databases and data analytics. He’s a frequent speaker at meetups, international conferences, and lecture halls.

Talk: Building and Operating an Open Source Data Science Platform

You have Successfully Subscribed!

Abstract:

You have Successfully Subscribed!

Josh Poduska

Chief Data Scientist, Domino Data Lab

Josh Poduska is the Chief Data Scientist at Domino Data Lab. He has 18 years of experience in the analytics space. As a practitioner, he has designed and implemented data science solutions across a number of domains including manufacturing, public safety, and retail.

Josh has managed teams and led strategic initiatives for multiple analytical software companies. Josh has a Masters in Applied Statistics from Cornell University.

Talk: Turbo-Charging Data Science with AutoML

You have Successfully Subscribed!

Abstract:

You have Successfully Subscribed!

Alexander Wong

Chief Scientist, DarwinAI Corp.

Alexander Wong is the Chief Scientist of DarwinAI, and currently a Canada Research Chair in the area of Artificial Intelligence, co-director of the Vision and Image Processing Research Group, and an associate professor in the Department of Systems Design Engineering at the University of Waterloo.

He has published over 550 refereed journal and conference papers, as well as patents, in various fields such as computational imaging, artificial intelligence, and computer vision.

Talk: How a Human-Machine Collaboration Approach to Deep Learning Development

You have Successfully Subscribed!

Abstract:

You have Successfully Subscribed!

Srikar Kovvali

Senior Data Scientist, DataVector AI

Srikar leads DataVector's machine learning and data science operations in Supply Chain Automation. In his career thus far, he has worked as a Global Resource Developer at KPMG, a Front End Engineer at one of the world's leading mobile payment companies - PayTM and as a Full Stack Engineer in Supply Chain Automation at Tesla.

He brings expertise in scaling automation architecture as well as predictive analytics.

Talk: Post-COVID MLOps Model Re-Training

You have Successfully Subscribed!

Abstract:

You have Successfully Subscribed!

Bahareh Atoufi

Data Science and AI Client Technical Professional, IBM

With a background in computer software engineering, Bahareh started her professional career as a software developer. She then pursued a MSc degree in Artificial Intelligence where she focused on application of machine learning in predicting epileptic seizures. This led to a PhD in Biomedical Engineering where Bahareh studied novel ML algorithms for controlling smart prosthetic hands.

Bahareh collaborated with Mitacs as director for business development. She also co-founded a start-up in the social network domain. Prior to joining IBM, Bahareh was a business development manager at Ontario Centres of Excellence.

Bahareh has been a reviewer, member of organizing committees, and author of research papers in multiple academic journals and international conferences in the field of AI and biomedical engineering.

She is currently with IBM Data Science/AI team helping FSS clients leverage Watson AI technology to better analyse their data and infuse AI into their business.

Talk: Ethical and Trusted AI: A Workshop on IBM Watson OpenScale

You have Successfully Subscribed!

Ethical and Trusted AI: A Workshop on IBM Watson OpenScale, Bahareh Atoufi, Data Science and AI Client Technical Professional, IBM

Abstract:

You have Successfully Subscribed!

Safura Suleymanova

Data Scientist, IBM

Safura Suleymanova is a Data Scientist at the IBM Data Science and AI Elite team, and she develops Machine Learning powered solutions for customers across various industries, including finance, telecom, supply chain, among others. She feels passionate about how data can be used to streamline more routine processes ultimately improving business decisions.

To improve existing operational processes at her previous workplaces she started leveraging the data, which led her to the field of Data Science. Safura has a bachelor’s degree in Mathematics, with a specialization in Risk Management and Statistics from University of Waterloo.

Talk: Ethical and Trusted AI: A workshop on IBM Watson OpenScale

You have Successfully Subscribed!

Ethical and Trusted AI: A Workshop on IBM Watson OpenScale, Safura Suleymanova, Data Scientist, IBM

Abstract:

You have Successfully Subscribed!

Eric Dong

Senior Data Scientist, IBM

Eric Dong is a Senior Data Scientist at the ML Hub Canada part of IBM Data Science and AI Elite team. Leveraging his broad knowledge base in Data Science and industry solutions, as well as in depth expertise in AI/ML, Eric is helping customers solve real-world data science problems in a wide array of industries.

Talk: Ethical and Trusted AI: A workshop on IBM Watson OpenScale

You have Successfully Subscribed!

Ethical and Trusted AI: A Workshop on IBM Watson OpenScale, Eric Dong, Senior Data Scientist, IBM

Abstract:

You have Successfully Subscribed!

Ayesha Hafeez

Machine Learning Engineer, Filament AI

Ayesha (MEng in Computer Engineering, University of Toronto) is our lead Machine Learning Engineer at Filament AI North America. She has 3 years of industry experience having worked at Markdale Financial Management in Toronto, Accenture in Dubai and Etisalat in Abu Dhabi.

She holds a BSc in Computer Engineering from the American University of Sharjah, UAE, and an MEng in Computer Engineering from the University of Toronto.

Talk: Utterance Generator Fast Tracks Chatbot Training

You have Successfully Subscribed!

Utterance Generator Fast Tracks Chatbot Training, Ayesha Hafeez, Machine Learning Engineer, Filament AI

Abstract: The opportunity for conversational AI in business to scale customer and other external communications has been recognized for some time. Unfortunately, chatbots have often been implemented ineffectively, and user satisfaction tends to be low.

At Filament, we’ve worked on over 65 conversational AI projects globally since our founding in 2016. Our founding team worked at IBM on the Watson suite, and we have partnerships with all of the major NLU providers. Given that experience, we know that a successful “chatbot” implementation boils down to two factors: (a) creating a comprehensive set of intents in a detailed but well-structured conversational design, and (b) training the NLU on a large enough set of utterances.

As an agency, we often undertake that work ourselves for our clients; our chatbot management product, Enterprise Bot Manager, also enables our customers to create and manage their conversational AI capability internally. However, the intent creation and utterance training work streams are time-consuming tasks if they are as exhaustive as they need to be for success.

In this presentation, we will explain how we have solved this problem by using Machine Learning to generate the required intents from pre-existing “human-to-human” conversational logs, and to generate automatically the volume of utterances needed to train each of the intents before launching a chatbot. In typical enterprise-grade projects, these solutions save weeks of work, decreasing cost barriers and dramatically shortening time-to-value.

The machine learning solution to fast-track and improve chatbot development comprises two processes.

You have Successfully Subscribed!

You have Successfully Subscribed!

Talk: Metaflow: Supercharging Our Data Scientist Productivity

Abstract:

Netflix's unique culture affords its data scientists an extraordinary amount of freedom. They are expected to build, deploy, and operate large machine learning workflows autonomously without the need to be significantly experienced with systems or data engineering. Metaflow, our ML framework (now open-source at metaflow.org), provides them with delightful abstractions to manage their project's lifecycle end-to-end, leveraging the strengths of the cloud: elastic compute and high-throughput storage.

In this talk, we will have one of our data scientists working in Content Demand Modeling present one of the challenges that they faced earlier this year. We will use that as a backdrop to present the human-centric design principles that govern the design of Metaflow and its internals. Finally, we will tie up the presentation outlining the team's experience using Metaflow and the impact of their work.

You have Successfully Subscribed!

Talk: Recommendation System with Deep Learning and PyTorch

Abstract:

In this session we'll dive into recommendation systems: understand the problem of recommendations, machine learning techniques for building such systems, and will focus on modern neural network architectures for recommendations. We will go over a hands on example of creating and training a recommendation model using PyTorch, and explore model design and deployment tradeoffs.
Attendees will learn how to apply deep learning to the problem of recommendations and ranking, and how they can leverage PyTorch to rapidly implement recommendation systems for various business use cases.

You have Successfully Subscribed!

Talk: Machine Learning for Trust and Safety at Pinterest

Abstract:

Trust and Safety (T&S) team at Pinterest is responsible for proactively preventing misuse of Pinterest’s ecosystem for creation or distribution of any or all forms of policy violating contents, e.g. spam, porn, misinformation, hate-speech, etc. There are many challenges involved in developing solutions for T&S problems like adversarial nature of the problem, scalability, cost-effectiveness, legal challenges arising from country or state specific laws and continuously evolving geo-political scenarios, content, and their corresponding policies.
We have successfully developed and productionized multiple batch and near real-time ML solutions, using state of the art techniques like DNN, CNN, and GCN etc., for effectively and efficiently filtering policy violating users and contents from Pinterest. We use both content and behavioral features for these models. We have established processes and guidelines for the design, development, and productionization of these models. Our design principles, mostly driven by business needs, ensure that these solutions are cost-effective, have performance elasticity, can be easily extended to future needs, and also can be easily iterated for freshness. We have comprehensive offline metrics, which are a mix of both standard ML metrics as well as business metrics, for evaluating these models. These models go through review and live A/B experiments before going live in production.
In this talk, I would give a broad-level overview of some of the ML solutions currently productionized at Pinterest along with its tech-stack. I would also discuss challenges involved in development and productionization of these models like lack of labeled dataset, skewness in the dataset, business requirement of a very low false positive rates etc. I would extend the talk with ML model design principles as well as ML solution development processes.

You have Successfully Subscribed!

Ravi Kiran Chirravuri

Machine Learning Infrastructure, Netflix

Ravi is an individual contributor on the Machine Learning Infrastructure (MLI) team at Netflix. With almost a decade of industry experience, he has been building large scale systems focusing on performance, simplified user journeys and intuitive APIs in MLI and previously Search Indexing and Tensorflow at Google.

Talk: Metaflow: Supercharging our data scientist productivity

You have Successfully Subscribed!

Hagay Lupesko

Engineering Leader, AI and ML, Facebook

Hagay has been busy building software for the past 15 years and still enjoys every bit of it (literally). He engineered and shipped products across various domains: from 3D medical imaging, through global scale web systems, and up to deep learning systems used at scale by engineers and scientists world-wide. He is currently based in the Silicon Valley, in sunny California, and focuses on democratizing AI and Machine Learning.

Talk: Recommendation System with Deep Learning and PyTorch

You have Successfully Subscribed!

Vishwakarma Singh

Machine Learning Researcher, Pinterest

Vishwakarma Singh is presently ML Tech Lead for Trust and Safety at Pinterest where he has led successful development and productionization of state of the art ML solutions for filtering spam, unsafe content, and fake content. Previosuly, he worked as Principal ML scientist and engineer at Apple. He has a PhD in Computer Science from UCSB, Santa Barbara, USA. He has published many papers in peer-reviewed conferences and journals.

Talk: Machine Learning for Trust and Safety at Pinterest

You have Successfully Subscribed!

Jan Florjanczyk

Senior Data Scientist, Netflix, Pinterest

Jan is a Senior Data Scientist on the Content Demand Modeling team at Netflix, where he supports a variety of content acquisition functions by applying machine learning to offer model driven insights and predictions.

Talk: Metaflow: Supercharging our data scientist productivity

You have Successfully Subscribed!

Talk: Metaflow: Supercharging Our Data Scientist Productivity

Abstract:

You have Successfully Subscribed!

Jules Damji

Developer Advocate, Databricks

Jules S. Damji is a Senior Developer Advocate at Databricks, an MLflow contributor, and O’Reilly co-author of Learning Spark 2nd. He is a hands-on developer with over 20 years of experience and has worked at leading companies, such as Sun Microsystems, Netscape, @Home, Opsware/Loudcloud, VeriSign, ProQuest, and Hortonworks, building large-scale distributed systems. He holds a B.Sc and M.Sc in Computer Science (from Oregon State University and Cal State, Chico respectively), and an MA in Political Advocacy and Communication (from Johns Hopkins University).

Workshop: Introduction to MLflow: Machine Learning Life Cycle Management Platform

You have Successfully Subscribed!

Workshop: Introduction to MLflow: Machine Learning Life Cycle Management Platform

Abstract:

Machine Learning (ML) development brings many new complexities beyond the traditional software development lifecycle. Unlike in traditional software development, ML developers want to try multiple algorithms, tools and parameters to get the best results, and they need to track this information to reproduce work. In addition, developers need to use many distinct systems to productionize models.
To solve these challenges, MLflow, an open source project, simplifies the entire ML lifecycle. MLflow introduces simple abstractions to package reproducible projects, track results, encapsulate models that can be used with many existing tools, and central repository to share models, accelerating the ML lifecycle for organizations of any size.
In this introductory workshop, partly lecture and partly hands-on tutorial, we will cover concepts and motivations behind MLFlow, cover its four components, and work through a simple example notebook.

Goal and Objective:

Aimed at beginner or intermediate level, this workshop aims to introduce a data scientist or ML developer in how you leverage MLflow as a platform to track experiments, package projects to reproduce runs, use model flavors to deploy in diverse environments, and manage models in a central repository for sharing.

What you will learn:

Understand the four main components of open source MLflow—MLflow Tracking, MLflow Projects, MLflow Models, and Model Registry—and how each compopnent helps address challenges of the ML lifecycle.
MLflow Tracking to record and query experiments: code, data, config, and results.
MLflow Projects packaging format to reproduce runs
MLflow Models general format to send models to diverse deployment tools.
Model Registry for collaborative model lifecycle management
MLflow UI to visually compare and contrast experimental runs with different tuning parameters and evaluate metrics

Prerequisites:

Python 3, pip, and conda pre installed
Knowledge on how to use conda
Knowledge of Python 3 and programming in general
Preferably a UNIX-based, fully-charged laptop with 8-16 GB, with a Chrome or Firefox browser
Familiarity with GitHub, git, and an account on Github
Some knowledge of Machine Learning concepts, libraries, and frameworks
scikit-Learn
pandas and Numpy
Loads of virtual laughter, curiosity, and a sense of humor ... :-)

You have Successfully Subscribed!

Stan Kirdey

Senior Distributed Systems Engineer - Insights Engineering, Netflix

Stanislav is working at Netflix where he focuses on large scale applied machine learning in the areas of NLP, search and anomaly detection. Outside of Machine Learning and work, Stanislav is obsessed with RuPaul's Drag Race, POSE and Star Trek.

Talk: Using Reinforcement Learning in Large Scale Automation

You have Successfully Subscribed!

Talk: Using Reinforcement Learning in Large Scale Automation

Abstract:

An overview of self-service reinforcement learning tools that helps to optimize schedules of automation tasks and help to reduce time to detect failures in software, hardware and cloud infrastructure. An agent learns the best order of execution of automation tasks to either fail fast or find clusters of failures. Lessons learned from writing and deploying applied RL service at Netflix.

You have Successfully Subscribed!

Ramy Nassar

Ramy is a facilitator, TEDx speaker, teacher and advocate for design thinking and effective digital strategy. He teaches in the FinTech program at the University of Toronto's Rotman MBA Program, Ryerson's Masters of Engineering, Innovation and Entrepreneurship and leads McMaster's post-graduate Design Thinking course.

He's worked with global brands such as Apple, Nike, Air Canada, Blackberry, RBC, SportCheck, Mattel, and Canadian Tire. Ramy is the author of the upcoming AI Product Design Handbook, set for realease in late 2020.

Workshop: Design Thinking for AI + Machine Learning Workshop

You have Successfully Subscribed!

Workshop: Design Thinking for AI + Machine Learning Workshop

Abstract:

This highly interactive and hands-on workshop offers a deep dive into AI, machine learning, and other emerging technologies targeted at leaders responsible for creating disruptive new digital products & services. The program is intended for those with or without a strong background in machine learning, AI, and related technologies - no technical expertise is assumed.
Participants can expect to walk away with a comprehensive understanding of how AI and machine learning work as core technologies and a wide range of applications, including; recommendation engines, personalization, predictive analytics, conversational/voice interfaces, and process automation. An overview of ethicical or responsible AI frameworks will wrap up the workshop, to ensure that participants are equiped with the skills to think about how data bias, training scenarios and other factors can impact outcomes.
Problem reframing methods, confusion matrices, error recovery optimization frameworks, and AI-focused ideation methods will all be introduced throughout the workshop.

You have Successfully Subscribed!

Parin Shah

Data Scientist

Parin is a ML engineer and a data scientist at NuData Security, Mastercard. He works on developing ML components of NuData Security’s award-wining online security software, NuDetect, which protects consumers and institutions from threats such as account takeover and fraud in real-time. Parin has developed systems and models for fraud detection, device-fingerprinting and passive-biometrics-based user identification that are used by leading e-commerce companies, banks and airlines. He is currently working on developing an internal Apache-Spark-based ML Feature Store to enable feature discovery and to provide a consistent interface for feature generation during model training and inference. NuData Security’s innovative ML products have led Parin to be an inventor on more than 10 patents. Apart from his work at NuData Security, Parin is a mentor with SharpestMinds, and a data science instructor at BrainStation, a professional education institute, where he teaches Python programming and ML fundamentals to students and professionals from diverse fields. Previously, Parin has worked on diverse data science consulting projects with KPMG and at an e-commerce company for pet health products, Natural Wellbeing. Academically, Parin is an economics graduate from the University of British Columbia.

Talk: Designing A Real-Time Model to Prevent Fake Account Sign-Up

You have Successfully Subscribed!

Talk: Designing A Real-Time Model to Prevent Fake Account Sign-Up

Abstract:

New account fraud, also known as application fraud, is when a bad actor creates a fake account, often using stolen identities. This fake account can then be used to receive credit or goods, to name a few of the dangers of this hack. This talk explains how NuData Security developed a real-time machine learning (ML) service that institutions can use to prevent fake account sign-ups. It also explains statistical and computational challenges encountered while scaling an extremely low-latency ML inference service.

To develop an application fraud model, we collect fraud and chargeback labels from our clients, and utilize our passive-biometrics technology to engineer model features that can capture latent characteristics to identify anomalous bot-like behaviour and information re-use. We use an internal Python library developed on top of Luigi and common ML frameworks to train our models and a custom domain-specific language to deploy and manage the production model versions. We then explain some of the common model deployment pitfalls one should keep in mind when deploying real-time models. Pitfalls are related to issues such as model compression, concept drift and inconsistencies in feature calculation. We look into tools and techniques that can be used to address these pitfalls such as robust model monitoring metrics, and more importantly, a purpose-built ML feature store that can help resolve inconsistencies in generating feature values during model training as well as inference. We showcase how AWS Glue and Apache Spark can be used to build a ML Feature Store.

Real-time ML model deployments require several different components in place to work together effectively, and this talk gives you an overview of exactly that.

You have Successfully Subscribed!

Benedikt Koller

CTO, maiot GmbH

I'm an Ops-Guy at heart and one of the technical co-founders of maiot - the team behind the Core Engine MLOps platform.

Talk: Talk: ML in Production: We’ve Been Here Before

You have Successfully Subscribed!

Talk: ML in Production: We’ve Been Here Before

Abstract:

Productionizing Machine Learning poses often huge hurdles to organisations that otherwise run high-performing software engineering departments. Why is that, and what learnings from decades of "traditional" software development seem to have been forgotten?

You have Successfully Subscribed!

Sandro Mund

Student, Porsche AG

Sandro Mund is currently a student and works at Porsche AG and has been active in research and development for artificial intelligence at various companies and universities for several years.

He wants to combine knowledge-based approaches with machine learning methods to make models more interpretable and safer and is especially interested in applications of autonomous driving.

In the past, he has helped to establish a research laboratory in Luxembourg, the results of which have been published in a book at Springer and at an IEEE conference. He was then heavily involved in the Krones AG to promote machine learning methods with different applications, such as image processing, data driven development and self-regulating systems.

Talk: Best Practices and Lessons Learned: Machine Learning Model Management

You have Successfully Subscribed!

Talk: Best Practices and Lessons Learned: Machine Learning Model Management

Abstract:

It is easy to lose track of which parameters and data a model was trained with. Tracking back how accurate the pre-processing looked or how much computing time an algorithm had is almost impossible without a system that stores this information. It is advisable to create a preliminary clear definition of what information needs to be stored. If this is not done, it can happen that, for example, predictions can no longer be made because it is impossible to determine which selection of parameters was used, in which order and with what unit and coding.

A rational model of the relationships of this information can prevent this. Assigning different versions of indexes allows for a history of how the data sources have changed during the course of a project, such as name changes or the removal and addition of attributes. This versioning is a necessary step that allows to automate further steps of a pipeline. This metadata can then be used to automatically generate graphical user interfaces with the appropriate input mask and allow reports to be generated.

This talk sensitizes to this topic and shows solutions that have been proven in real industrial projects.

You have Successfully Subscribed!

Talk: Explainable Monitoring for Successful Impact with AI Deployments

Abstract:

AI is a game changer for business decision-making, but deployed AI can be problematic because of its opaque nature. AI deployments are unique given the potential operational challenges like data drift, outliers, data integrity, model decay, and bias/fairness. This makes it difficult to understand the ‘how’ and ‘why’ behind the decisions AI models make.

And while training and deploying ML models is relatively fast and cheap, maintaining, monitoring, and governing them over time is difficult and expensive. An Explainable Monitoring system extends traditional monitoring to provide deep model insights with actionable steps. This session will cover ways to increase transparency and actionability across the entire AI lifecycle using explainable monitoring, allowing for better understanding of problem drivers, root cause issues, and model analysis through AI deployment.

You have Successfully Subscribed!

Léa Genuit

Data Scientist, Fiddler

Léa Genuit is a Data Scientist with a background in machine learning, deep learning and applied statistics. Léa is a Data Scientist at Fiddler, an Explainable Monitoring company that helps address problems regarding bias, fairness and transparency in AI. Prior to Fiddler, she spent time at various companies like Zillow, AXA Rosenberg Investment Management, and Airbus, focusing on using Machine Learning and Deep Learning to solve business challenges. Léa received her Masters Degree in Data Science from the University of San Francisco and Masters Degree in Statistics and Econometrics from Toulouse School of Economics.

Talk: Explainable Monitoring for Successful Impact with AI Deployments

You have Successfully Subscribed!

Meeta Dash

VP Product, Verta

Meeta is a passionate, customer-centric product leader with a track record of launching innovative products that solve real business problems. As VP Product at Verta she is building MLOps tool to help data science teams track, deploy, operate and monitor models and bring order to AI/ML chaos. Prior to Verta, Meeta held several product leadership roles at Appen, Figure Eight, Cisco Systems, Tokbox/Telefonica and Computer Associates building ML data platforms, Voice/Conversational AI products, and IT/Operational Monitoring tools.

Talk: Simplifying AI DevOps with Model Registry

You have Successfully Subscribed!

Talk: Simplifying AI DevOps with Model Registry

Abstract:

To effectively deploy and scale ML models across the development pipeline requires a mix of machine learning, software engineering and operational skills which is rare to find in a single person or even in a single team. Additionally, organizations with hundreds of models today face the unique challenge arising from the heterogeneity in ML workflows and the siloed nature of these teams.

In this talk we will talk about how to fast track time to value and reduce risks in your ML journey from development to production.

Learn more about:

- How to ensure model reproducibility & portability across local, dev and production environments

- How to build transparency and governance with a Model Registry

You have Successfully Subscribed!

Aslı Sabancı

Applied Machine Learning Engineer, Algorithmia

I'm an applied machine learning engineer, carrying the distilled gems of 10+ years of software engineering experience in my toolbox. I find my joy in putting my engineering skills to good use; deploying, managing and scaling machine learning portfolios; while enjoying life in beautiful California.

Talk: Continuous Deployment from ML Repository to Algorithmia

You have Successfully Subscribed!

Talk: Continuous Deployment from ML Repository to Algorithmia

Abstract:

GithubActions integration between a user's machine learning repository at Github and an algorithm at Algorithmia, automating ML model and inference layer deployment through a continuous deployment workflow.

You have Successfully Subscribed!

Maryna Karpusha

Machine Learning Research Team Lead, Borealis AI

Talk: Technical Debts in Machine Learning Projects and How to Mitigate Them

You have Successfully Subscribed!

Talk: Technical Debts in Machine Learning Projects and How to Mitigate Them

Abstract:

Machine learning allows us to build quickly complex systems that improve automatically through experience. However, similar to the software engineering framework of

Technical debt, important designing details can be missed initially. Fixing technical debts can take a lot of effort in the future. In this presentation, we explore several ML-specific risk factors to account for in system design. In this presentation, we discuss:

What is technical debt, and why it is essential to consider it while designing systems;

Why ML systems have additional challenges that are not present in classical software systems;

Different types of technical debts in ML projects;

How we can recognize and mitigate ML-related technical debts.

You have Successfully Subscribed!

Eimear Nolan

Senior Data Scientist, Unbounce

Eimear is a Senior Data Scientist at Unbounce and leads a machine learning prototyping team. She originally comes from a bioengineering background, graduating from Imperial College London. For the last 5 years she has worked on ML projects in multiple industries including wearable tech, insurance and e-commerce.

Talk: Diversify Your Way To Rapid ML Experimentation

You have Successfully Subscribed!

Talk: Diversify Your Way To Rapid ML Experimentation

Abstract:

Machine learning experimentation can be a slow, expensive process especially when it involves customer experience. At Unbounce we’ve bootstrapped this process by creating nimble, multi-faceted teams that can bring ML prototypes to production quicker to gauge their potential. We do this by constructing a diverse team and by using a combination of Sagemaker, Docker and our native ‘Model Broker’. In this talk I’ll go through how we setup this practice and the tools we use to make it successful.

You have Successfully Subscribed!

Andreas Putz

Discipline Lead (Software) & Head of Internal Projects, MistyWest

Andreas Putz is a Scientific Computing expert and Data Analyst at MistyWest focusing on creating intelligent, connected devices. His special interest lies in thoughtful application of machine learning to real-world client problems. MistyWest is an engineering design consultancy that exists to create futuristic technologies that enable a healthier planet and bring prosperity to all humankind, with a focus primarily on projects that advance the UN Sustainable Development Goals.

Talk: Machine Learning on the Edge - From Microcontrollers to Embedded Linux Devices

You have Successfully Subscribed!

Talk: Machine Learning on the Edge - From Microcontrollers to Embedded Linux Devices

Abstract:

Machine learning has been the domain of large scale and high powered infrastructure. A large number of tools exist to orchestrate the development and orchestrated deployment of your models in all major cloud environments or within your own datacenter.

However, a huge push by hardware providers ( Raspberry Pi Foundation, NVIDIA, Intel, Google, NXP, Qualcomm and many more) has created a wide range of small footprint, low powered edge devices capable of running hardware-accelerated without fallback to any cloud backend.

In this talk, we will discuss the general landscape of ML capable edge devices starting from Microprocessor-based teal time systems to fully-fledged embedded Linux devices. We will then move to the challenges of large scale deployment: Adaptation and size reduction of ML models. Over the air updates for microprocessors and Dockerized deployment for embedded Linux devices.

Finally, we will deploy a basic computer vision demo on three small scale embedded Linux devices: Raspberry Pi, Google Coral and NVIDIA Jetson.

You have Successfully Subscribed!

Nathalie Rauschmayr

Machine Learning Scientist, AWS

Workshop:

You have Successfully Subscribed!

Workshop:

Abstract:

You have Successfully Subscribed!

Patricia Thaine

CEO; PhD Candidate, Private AI; University of Toronto

Patricia Thaine is a Computer Science PhD Candidate at the University of Toronto and a Postgraduate Affiliate at the Vector Institute doing research on privacy-preserving natural language processing, with a focus on applied cryptography. She also does research on computational methods for lost language decipherment. Patricia is the Co-Founder and CEO of Private AI, a Toronto- and Berlin-based startup working on making privacy preservation and GDPR compliance simple and easy. She is a recipient of the NSERC Postgraduate Scholarship, the RBC Graduate Fellowship, the Beatrice “Trixie” Worsley Graduate Scholarship in Computer Science, and the Ontario Graduate Scholarship. She has eight years of research and software development experience, including at the McGill Language Development Lab, the University of Toronto's Computational Linguistics Lab, the University of Toronto's Department of Linguistics, and the Public Health Agency of Canada. She is also a member of the Board of Directors of Equity Showcase, one of Canada's oldest not-for-profit charitable organizations.

Talk: Privacy-Preserving Machine Learning

You have Successfully Subscribed!

Talk: Privacy-Preserving Machine Learning

Abstract:

An organization aiming to create a privacy-preserving machine learning pipeline is faced with a plethora of privacy tools to choose from, which can either be used on their own or in combination with one another in order to achieve different privacy goals. These tools include federated learning, homomorphic encryption, differential privacy, anonymization/pseudonimization, secure multiparty computation, and trusted execution environments, among others. This talk will use practical examples to show how to strategically think about privacy problems. We will consider risk, implementation complexity, and available computational resources.

You have Successfully Subscribed!

Elle O’Brien

Data Scientist, Iterative.ai

Elle is a data scientist at Iterative, a startup building open source software tools for machine learning. She completed her PhD at the University of Washington where she conducted research on speech and hearing using mathematical models. Elle is broadly interested in developing methods, standards, and educational resources for anyone who works with data.

Workshop: How to Automate Machine Learning with GitHub Actions

You have Successfully Subscribed!

Workshop: How to Automate Machine Learning with GitHub Actions

Abstract:

- How to write your own GitHub Actions to automatically train, test, and report machine learning models

- Why automation is key to speeding up development cycles and getting models into production faster

You have Successfully Subscribed!

Rustem Feyzkhanov

Machine Learning Engineer, Instrumental

AWS ML Hero. Author of the course and book "Serverless Deep Learning with TensorFlow and AWS Lambda" and "Practical Deep Learning on the Cloud". Main contributor to open source repository for serverless packages https://github.com/ryfeus/lambda-packs and https://github.com/ryfeus/stepfunctions2processing.

Talk: Serverless for Machine Learning Pipelines.

You have Successfully Subscribed!

Talk: Serverless for Machine Learning Pipelines.

Abstract:

One of the main issues with ML and DL deployment is finding the right way to train and operationalize the model within the company. Serverless approach for deep learning provides simple, scalable, affordable yet reliable architecture. The challenge of this approach is to keep in mind certain limitations in CPU, GPU and RAM, and organize training and inference of your model.

My presentation will show how to utilize services like Amazon SageMaker, AWS Batch, AWS Fargate, AWS Lambda and AWS Step Functions to organize deep learning workflows.

My talk will be beneficial for machine learning engineers and data scientists.

You have Successfully Subscribed!

Sheila Beladinejad

CEO, Strategist, Technology Consultant, Germany Ambassador, O Canada Tech

I am the founder and CEO of O Canada Tech. I have 20+ years of experience in Software Engineering with a special interest and expertise in evaluations of Artificial Intelligence. I help Private Equity firms going through the merger and acquisition process by providing technical due diligence assessments of the target companies software. Additionally, I work with executive management teams of technology companies to define and execute strategic roadmaps for Digital Transformation, Adoption of AI and Agile Transformation. I am also an advisor to Greentech Alliance, Member of European AI Alliance and Ambassador for Women in AI, Germany.

Talk: CEO, Strategist, Technology Consultant, Germany Ambassador

Talk: DVC: Data Versioning and ML Experiments on Top of Git

You have Successfully Subscribed!

Harish Doddi

Founder & CEO, Datatron

Over the past decade, Harish Doddi has focused on AI and data science. Before Datatron, he worked on the surge pricing model for Lyft, the backend for Snapchat Stories, the photo storage platform for Twitter, and designing and developing human workflow components for Oracle. Harish graduated with a Masters in Computer Science degree from Stanford. He specialized in the Systems and Databases field."

Abstract:

Dr. James G. Shanahan

Church and Duncan Group and UC Berkeley

Dr. James G. Shanahan has spent the past 30 years developing and researching artificial intelligent systems. He splits his time between consulting and academia (UC Berkeley).

James leads organizations through digital transformation; from strategic planning, defining and high-grading product and service portfolios, to large scale data science and AI pipeline development and deployments for retail, digital advertising, and search.

He manages teams from strategy through execution and drives alignment across lines of business and his recommendations incorporate customer centricity and market evolution while considering the intent of an organization to develop innovative data science and AI solutions that maximize business growth. This often translates down to James' very hands-on approach (leading by example) to problem solving that culminates in custom architectures and loss functions for deep learning pipelines in PyTorch that maximize domain KPIs and throughput, both in the cloud and on the edge. Previously he has founded several startups and has held appointments at AT&T (Executive Director of Research), NativeX (SVP of data science), Xerox Research (staff research scientist), and Mitsubishi. James received his PhD in engineering mathematics and computer vision from the University of Bristol, U.K.

Abstract:

The main focus of object detection, one of the most challenging problems in computer vision (CV), is to predict a set of bounding boxes and category labels for each object of interest in an image or in a point cloud. As such, object detection has a variety of exciting downstream applications such as self-driving cars, checkout-less shopping, smart cities, cancer detection, and more. This field has been revolutionized by deep learning over the past five years, where during this time, two-stage approaches to object detection have given way to simpler, more efficient, one-stage models. Mean average precision (mAP) on benchmark problems such as the COCO Object Detection dataset has improved almost 4X over the course of five years from 15% (Fast RCNN, a two-stage approach) to 55% (EfficientDet7x, a one-stage approach). This talk looks under the hood of state-of-the-art object detection systems, such as two-stage, one-stage, and also more recent approaches based upon transformers. It will provide a cheatsheet on how to jumpstart or upgrade your detection pipelines to state-of-the-art, while also highlighting some of the key challenges that remain. Examples and associated detection pipelines will be provided in Jupyter Notebooks using PyTorch, Python, and OpenCV. While the primary focus is on object detection in digital images from cameras and videos, this talk will also introduce object detection in 3D point clouds.

What You'll Learn:

Learn about the key components in state-of-the-art object detection systems.

Learn the difference between different detection approaches such as two-stage, one-stage, and also more recent approaches based upon transformers.

It will provide a cheatsheet on how to jumpstart or upgrade your detection pipelines to state-of-the-art

Get to experiment with object detection pipelines in Jupyter Notebooks (written in PyTorch, Python, and OpenCV)

Introduce object detection in 3D point clouds and state-of-the-art approaches

Learn about some of the key challenges that remain in object detection.

Talk: State-of-the-Art Deep Learning-Based Object Detection in 2D and 3D

You have Successfully Subscribed!

Dr. Maria Börner

Program Manager, Adap GmbH

Maria has a Ph.D. in physics and an academic background in data analysis and processing. During her Ph.D., she worked at the world's largest Particle Colliders CERN and DESY. She previously managed AI projects at XAIN AG and was responsible for realising them for Porsche, PwC and Daimler.

Abstract:

Today, AI is trained in the cloud on centralized data having regulatory constraints and large latency. New algorithmic advances as federated learning or federated analytics enable AI that runs and trains locally on edge devices, cars, server or different organizations. It allows sharing the learnings without sharing the data with full data privacy. Many applications and use cases benefit from federated learning and analytics that will be presented. In addition, you will learn which requirements your project needs to set it up.

What You'll Learn:

You will learn about the federated learning and federated analytic concepts and requirements for setting it up. You will also get an idea where you can use federated learning and federated analytics.

Talk: Federated Learning in Production

You have Successfully Subscribed!

Josh Lovejoy

Head of Design, Microsoft Ethics & Society

Josh leads Design for Ethics & Society at Microsoft, where his team works at the intersection of product development, user experience, AI, law, and philosophy. Prior to Microsoft, he was UX lead for People+AI Research at Google and founded Amazon’s unified design system for online shopping experiences.

Abstract:

For an industry that prides itself on moving fast, the tech community has been remarkably slow to adapt to the differences of designing with AI. Machine learning is an intrinsically fuzzy science, yet when it inevitably returns unpredictable results, we tend to react like it’s a puzzle to be solved; believing that with enough algorithmic brilliance, we can eventually fit all the pieces into place and render something approaching objective truth. But objectivity and truth are often far afield from the true promise of AI, as we’ll soon discuss.

Senior Director of Data Science, Comcast

Jim Olsen

CTO, ModelOps

Jim is the CTO at ModelOp, where he leads the technical design of ModelOp Center, the leading ModelOps platform. Jim has been a technical innovator for the past 30 years, leading architectural design for products at a variety of companies, including Teradata, Qualtrics, and Novell. Jim holds two software patents. For the last year, Jim has been living off the grid in the Rocky Mountains of Colorado.

Abstract:

MLOps and ModelOps are different. In this session, we will cover how ModelOps not only encompasses the basic model monitoring and tuning capabilities of MLOps, but also includes a continuous feedback loop and 360 view of your models throughout the enterprise, providing reproducibility, compliance and auditability of all your business critical models.

You will learn about good practices for:

What You'll Learn:

The audience will view an overview of the Algorithmia platform, and understand how to put models into production quickly, securely, and learn about new and exciting features. There will be several short product demonstrations outlining our capabilities in real-world scenarios.

Workshop: An Introduction to Algorithmia

You have Successfully Subscribed!

Sonya David

Vice President, Data Scientist Lead, JPMorgan Chase & Co.

Sonya David is a Data Scientist Lead in the US Wealth Management group at JPMorgan Chase. She has experience in data science and analytics. Sonya holds an Honours Bachelor’s degree in Neuroscience from the University of Toronto and is a 2021 candidate for a Master of Computer Science from Wilfrid Laurier University. Before joining JPMorgan Chase, she was a Data Scientist with Clearco (formerly Clearbanc) and a Business Analyst on the Partner Solutions team at Telus.
Sonya is deeply involved in the community, having volunteered for institutions such as Sick Kids and Sunnybrook, among others, holding positions ranging from Research Assistant to Committee Executive.
As Chair, Sonya will bring her committee experience, data science chops, and wide-ranging partnerships to the role.

You have Successfully Subscribed!

Alon Gubkin

VP, R&D, Aporia

Alon Gubkin is the VP R&D of Aporia, a customizable monitoring platform that enables data science teams to build their own monitoring solution for ML models in production.
Writing code from the age of seven, Alon is passionate about programming, machine learning, biology, video games, and software architecture. He created his first successful open source project, a video chat platform for mobile, and Spider, a new programming language, when he was 17.
Before joining Aporia, Alon served as an R&D lead in the elite 81 intelligence unit of the Israel Defense Forces. While in the IDF, he worked on multiple machine learning projects including a CNN-based head pose estimation and video game bots.

Abstract:

In this workshop you’ll learn how to set up an ML platform using open-source tools like Cookiecutter, DVC, MLFlow, KFServing, Pulumi, GitHub Actions and more.
We'll explain each tool in an intuitive way and the problem they solve, in order to build a useful platform that combines all of them. All code will be available on GitHub after the workshop, so you'll be able to easily integrate it into your existing work environment.
There’s no “one size fits all” ML Platform. Each organization has their own needs and requires a customizable and flexible solution. In this workshop you’ll learn how to create the right solution for your own organization.
The workshop is intended for data scientists and ML engineers from all industries – from small startups to large corporations and academic institutions.

What You'll Learn:

Building an ML platform doesn’t have to be time consuming and difficult. And it doesn’t need to be a huge platform in order to be effective. With the right tools, you can easily build your own customizable platform that will increase your productivity and the quality of your models.

Workshop: Building an ML Platform from Scratch

You have Successfully Subscribed!

Harish Doddi

Founder & CEO, Datatron

Harish Doddi has 10+ years experience building AI infrastructure systems early on at places like LYFT, TWTR and SNAP Some notable work of him relates to building LYFT AI pricing engine from scratch which contributes to a significant revenue today. He has finished his graduation from Stanford University and undergrad from International Institute of Information Technology, Hyderabad, India. He got inspired from his previous experiences to start Datatron, an Enterprise grade MLOps and Model Governance platform which helps large enterprises to accelerate the time to production for models. The platform has already been proven in several fortune 100 companies

Amit Paka

CPO & Co-founder, Fiddler AI

Abstract:

What You'll Learn:

Talk: Build High Performance MLOps With ML Monitoring and AI Explainability

You have Successfully Subscribed!

Deepak Pai

Senior Manager, Machine Learning, Adobe

Deepak is a Machine Learning Engineer with 16 years of experience. He has published papers in top peer reviewed conferences and holds multiple US patents. Currently he manages a team of Machine Learning Engineers developing multiple products and services at Adobe, that are part of the Digital Experience business. Deepak holds Masters and Bachelor degree in Computer Science from a leading universities in India.

Vijay Srivastava

Manager, Machine Learning, Adobe

I have 15 years of industry experience across ML & E-Learning products. I have extensive experience in developing scalable Machine Learning & e-Learning cloud services from inception. During my technical journey at Adobe, I worked on multiple key positions as Senior Computer Scientist and Staff Data Scientist. As a part of my current job, I manage team of machine learning engineers developing core ML services which feeds ML insights to Experience Cloud Intelligent Services. I hold bachelor’s degree from Indian Institute of Information Technology - Allahabad, India.

Abstract:

You spend lots of time cleansing data, visualizing it to gain insights, feature engineering, modeling while ensuring that you picked the best algorithm and architecture, best hyper parameters and so on. Finally you deploy the model and claim victory and move on. Well, that is not the end of task for an ML engineer. Model monitoring is much more critical than model building and often neglected area of MLOps. Despite their critical roles, ML models in production are not actively monitored. Ideally one should monitor the production systems proactively, but unfortunately being reactive is the norm. When a problem first arises, it may go unnoticed for some time. Once it is noticed, investigating its underlying cause is a time-consuming, manual process, not to mention the damage that is already done in production. Even if you are manually monitoring the models in production, the approach does not scale when you get to tens of models if not hundreds.
Like the saying goes, “A stitch in time saves nine”, wouldn’t it be great if the model’s output were automatically monitored? If they could be visualized, sliced by different dimensions? If the system could automatically detect performance degradation and trigger alerts? If problems in the model output could be attributed to the characteristics of input data? In this presentation, we describe our experience from building such a core machine-learning services: Model Evaluation and Data Quality. Our service provides automated, continuous evaluation of the performance of a deployed model over commonly-used metrics like the area-under-the-curve (AUC), root-mean-square-error (RMSE) among other

A Uniquely Interactive Experience

Why MLOps Production & Engineering?

We’re Hosting

BREAKOUT SESSIONS (All Levels)

DISCUSSION GROUPS

WORKSHOPS

VIRTUAL PLATFORM

Chair

Sonya David

Bio

Speakers

Mike Berg

Bio & Abstract

Matt Zeiler

Bio & Abstract

Gilad Shaham

Bio & Abstract

Eric Duffy

Bio & Abstract

Jaya Kawale

Bio & Abstract

Harish Doddi

Victor Thu

Bio & Abstract

Mengdi Huang

Shashank Verma

Bio & Abstract

Christopher A. Choquette Choo

Bio & Abstract

Amit Paka

Bio & Abstract

Milecia Mcgregor

Bio & Abstract

Mike Purewal

Bio & Abstract

Daniel Jeffries

Bio & Abstract

Eric Schles

Bio & Abstract

Samantha Zeitlin

Bio & Abstract

Workshop Facilitator

Adam Gugliciello

Bio & Abstract

Alon Gubkin

Bio & Abstract

Mohamed Sabri

Bio & Abstract

More to be announced

Gold Sponsors

Silver SPONSORS

Join Our Community

Thank you for Subscribing!

Abe Gong

You have Successfully Subscribed!

Boris Lublinsky

You have Successfully Subscribed!

Brooke Wenig

You have Successfully Subscribed!

Chanchal Chatterjee

You have Successfully Subscribed!

Ebrahim Safavi

You have Successfully Subscribed!

Nick Pogrebnyakov

You have Successfully Subscribed!

Sabina Stanescu

You have Successfully Subscribed!

You have Successfully Subscribed!

You have Successfully Subscribed!

Saranyan Vigraham

You have Successfully Subscribed!

Kenny Daniel

You have Successfully Subscribed!

Patrick Hall

You have Successfully Subscribed!

Veysel Kocaman

You have Successfully Subscribed!

Vin Vashishta

You have Successfully Subscribed!

Brandy Freitas

BREAKOUT SESSIONS
(All Levels)