AI Superstream: Building with Open Source Generative AI Models and Frameworks

Susan Shu Chang: Introduction (5 minutes) - 8:00am PT | 11:00am ET | 3:00pm UTC/GMT

• Susan Shu Chang welcomes you to the AI Superstream.

Leandro von Werra–Keynote: Open LLMs—The Comeback of Open Models and Current Trends in the LLM Landscape (15 minutes) - 8:05am PT | 11:05am ET | 3:05pm UTC/GMT

• Leandro von Werra gives an overview of the driving forces behind the accelerated progress of LLMs, shares some of the challenges and opportunities of open large language models, and discusses what it means to be truly open. He then dives into recent developments in the LLMs space and discusses what to expect in the coming months.
• Leandro von Werra is a research lead at Hugging Face and co-leads the BigCode open science project that aims at developing large language models for code in an open and responsible way with models such as StarCoder2. He’s the creator of a popular Python library called TRL, which combines transformers with reinforcement learning and other effective fine-tuning methods.

James Spiteri: Using Mistral's Open Source MoE model—Mixtral 8X-7B—to Power Elastic's AI Assistant (30 minutes) - 8:20am PT | 11:20am ET | 3:20pm UTC/GMT

• James Spiteri shares how Elastic has integrated GenAI models—notably the open source Mixtral 8X-7B large language model—to create its Security AI Assistant, helping users respond to threats and generate queries, all with natural language. He outlines the setup process and demonstrates the significant benefits and versatility of these technologies for use in your own customer-facing products.
• James Spiteri is a director of product management at Elastic, leading generative AI and machine learning efforts for Elastic Security. Previously, he served as director of product marketing and as a security specialist on Elastic's solutions architecture team, helping customers architect their Elastic deployments for security analytics. James has also built custom SIEM platforms for security operations centers and is the creator of ohmymalware.com, whichphish.com, and log4shell.threatsearch.io. He’s a regular speaker at conferences around the world, including RSA, SecTor, AWS re:Invent, Google Next, GITEX, and GISEC.
• Break (5 minutes)

Denys Linkov: Optimizing Entity Extraction with DSPY (30 minutes) - 8:55am PT | 11:55am ET | 3:55pm UTC/GMT

• Optimizing prompts is still a dedicated process with many steps. Denys Linkov discusses how Voiceflow optimizes prompts, including defining edge cases and curating training and evaluation data. He takes you through a case study of how Voiceflow used DSPY to build a new entity extraction feature for a complex conversational AI use case.
• Denys Linkov is head of machine learning at Voiceflow, an ML advisor, and instructor whose GenAI courses have helped more than 100,000 learners build key skills. He's worked with more than 50 enterprises across the AI product stack, building key ML systems, managing product delivery teams, and working directly with customers on best practices.

Nicole Königstein: Using Open Source LLMs for Your Applications—Challenges and Practical Insights (30 minutes) - 9:25am PT | 12:25pm ET | 4:25pm UTC/GMT

• Nicole Königstein provides a practical and realistic overview of using open source LLMs, helping you make informed decisions for building AI applications. She explains how to decide which model to choose and how to evaluate model performance, and she addresses the specific challenges that come with using frameworks like LangChain. She also discusses cybersecurity concerns and offers strategies to mitigate them.
• Nicole Königstein is a data scientist and quantitative researcher working as chief AI officer and head of quantitative research at quantmate. She also serves as an AI consultant, leading workshops and guiding companies from AI concept to deployment. She’s the author of Mathematics for Machine Learning with NLP and Python and Transformers in Action with Manning Publications, and the author of the forthcoming book Transformers: The Definitive Guide, with O'Reilly Media. As a guest lecturer, Nicole shares her expertise in Python, machine learning, and deep learning at various universities.
• Break (5 minutes)

Avin Regmi: Optimizing LLM Infrastructure for Training (30 minutes) - 10:00am PT | 1:00pm ET | 5:00pm UTC/GMT

• Optimizing ML infrastructure is crucial in training LLMs at scale. It involves utilizing various distributed training strategies, such as data parallelism and model parallelism, to maximize resource utilization and minimize waste. Avin Regmi explains how to optimize your infrastructure and apply various techniques to fine-tune LLMs.
• Avin Regmi is an engineering manager at Spotify, leading the ML training and compute team for the Hendrix ML Platform. His areas of expertise include training and serving ML models at scale, ML infrastructure, and growing high-performing teams. Previously, he led the ML platform team at Bell, focusing on distributed training and serving LLMs. He’s also the founder of Panini AI, a cloud solution that serves ML models at low latency using adaptive distributed batching. Outside of work, Avin practices yoga and meditation and enjoys high-altitude climbing and hiking.

Mandy Gu: Building LLM Infrastructure Toward Security, Accessibility, and Optionality (30 minutes) - 10:30am PT | 1:30pm ET | 5:30pm UTC/GMT

• At the beginning of 2023, Wealthsimple developed and open-sourced its LLM Gateway to provide security, accessibility, and optionality for its employees to interact with GenAI. It has since extended its LLM platform to include retrieval and search, with answers grounded in the company's knowledge base. Today, LLMs are used across the organization to improve productivity, optimize operations, and provide a more delightful client experience. Mandy Gu shares the lessons learned in developing this infrastructure.
• Mandy Gu is a senior software development manager at Wealthsimple, where she leads the machine learning engineering and data engineering teams which provide a simple and reliable platform to empower the rest of the company to iterate quickly on machine learning applications and GenAI tools and leverage data assets to make better decisions. Previously, Mandy worked in the NLP space as a researcher and data scientist.

Susan Shu Chang: Closing Remarks (5 minutes) - 11:00am PT | 2:00pm ET | 6:00pm

• Susan Shu Chang closes out today’s event.