This week on The Data Stack Show, Eric and Kostas chat with Nick Schrock, Founder of Dagster Labs. During the episode, Nick discusses his background at Facebook and his involvement in successful open-source technologies like React and GraphQL. Nick explains the mismatch between the complexity and available tools in the data infrastructure space, leading him to start Dagster Labs. The group also talks about the challenges and fragmentation in the data engineering industry, the need for better abstraction layers, the role of orchestrators, comparing it to GraphQL’s role in product engineering, the importance of data orchestration in the future of data infrastructure and engineering, and more.

This week on The Data Stack Show, Eric and Kostas chat with Amr Awadallah, the Founder and CEO of Vectara. During the episode, Amr discusses his extensive experience in the data industry and his new company, Vectara, which focuses on enabling companies to integrate capabilities in their products using large language models (LLMs) with security, reliability, and ease of use. The conversation also covers the advancements in data, computing, and algorithms that have led to emergent behaviors in neural networks, the practical applications of Vectara’s technology, the challenges and considerations in working with large language models, the importance of addressing technology misuse and aligning different value systems in society, and more.

This week on The Data Stack Show, Eric and Kostas chat with Alex Gallego, the Founder & CEO of Redpanda Data. During the episode, the group discusses the importance of Kafka in data infrastructure and the lack of competition in the market. Alex shares his background in streaming systems and the development of Redpanda. The conversation also includes the challenges of operationalizing streaming systems and the evolution of storage systems, using WebAssembly as the interface for writing functions, the key differences between Redpanda and other streaming platforms, where the name Redpanda came from, and more.