Generative AI in the Real World: The Future of Programming with Matt Welsh

Don’t miss Tim O’Reilly and Reid Hoffman talking about DeepFakes, AI, and the American Economy  on September 27 at 12:30 PM EDT.

Join us for a conversation between Ben Lorica and Matt Welsh, cofounder of Fixie.ai, former engineer at Apple and Google, and one of Mark Zuckerberg’s professors at Harvard. Learn how AI is changing computing. Whether it’s in C or a human language, programming is telling a computer what you want it to do—but AI opens up new classes of things that we can ask it to do.It’s not just simplifying (or replacing) coding; it’s creating new opportunities and new kinds of applications that we couldn’t imagine two or three years ago.

About the Generative AI in the Real World podcast: In 2023, ChatGPT put AI on everyone’s agenda. In 2024, the challenge will be turning those agendas into reality. In Generative AI in the Real World, Ben Lorica interviews leaders who are building with AI. Learn from their experience to help put AI to work in your enterprise.

Points of Interest

• 0:00: Introduction.
• 2:38: The changing nature of programming. What will replace programming?
• 3:07: Ultimately, the idea of writing a program will be replaced by telling a language model what you want to do. The language model will do what you want directly.
• 5:03: I can do things I couldn’t imagine doing—for example, summarize a transcript or find bios of speakers and relevant papers.
• 7:01: There’s a whole new field of kinds of computation we couldn’t do before.
• 7:48: People in fields like medicine used to have to ask computer scientists to do things for them. Now, you don’t have to get a computer scientist to translate an idea into reality.
• 11:30: What is missing from the current tooling?
• 11:40: It’s way too hard for people without programming ability to integrate language models into their workflows. Ultimately, AI needs to be deeply integrated into products and the OS.
• 13:45: Are people in the UX community inventing new ways to interact?
• 14:40: We are very embedded in a web/mobile-based way of thinking about interacting. AI changes the ways we interact with computers—for example, voice.
• 16:07: There’s a lot of information encoded into voice that you miss when you encode it into text.
• 18:15: What about programming itself?
• 18:30: Programming is changing radically. At Fixie, we mandated that employees have access to ChatGPT and similar tools.
• 20:34: What is the role of testing and QA?
• 21:28: People will struggle to find the right trade-offs. We’re not throwing out all of the processes we’ve developed, like testing and code reviews.
• 25:25: Every company can train AI to scale their best engineers.
• 25:55: We’re being sloppy as an industry. Curation of good code and good documents will be important. We don’t just need more data, we need better data.
• 28:23: What is Aryn doing?
• 29:17: When people wanted to use AI models to ask questions about their data, they started with simple processes: break text into chunks, store in vector database, and at question time, feed them back in to the prompt.
• 30:10: We need the ability to extract data from unstructured documents. The structure is there, but it’s hidden. The first part of Aryn: How do you extract the structure inherent in documents?
• 32:46: The second part of Aryn: A Python framework, Sycamore, lets you build ETL pipelines from these documents. ETL does things like normalize location information.
• 35:45: Another part of the Aryn stack is LLM-powered unstructured analytics (LUNA) that allows you to make queries based on the unstructured data in the documents.
• 37:34: The future of programming is using language models as computers to perform computation that would be difficult to express in a programming language.
• 38:22: People are talking about GraphRAG, which is RAG with knowledge graphs, but how do you get a knowledge graph? Can Aryn help that?
• 39:15: Yes, we’re effectively doing knowledge graph construction. But once you have the right underlying structure, you may not need knowledge graphs at all.
• 40:50: Are tools for evaluating AI lagging behind development tools?
• 41:16: The meaning of “evaluation” is often not well-defined.
• 43:03: Evaluation will come down to establishing trust.
• 43:32: We need tools that will allow people to collaborate early on evaluations. You need to give people that help them understand what’s happening.