Presentation: From Software Development to ML - A Team's Transformation
What You’ll Learn
- Hear experience and challenges leading a traditional software engineering team into a machine learning/AI journey.
- Hear recommendations on how to approach the idea of transitioning expert-level developers into a new field.
- Understand what the priorities are when teaching a team new skill in machine learning.
Abstract
As companies seek to "inject" Machine Learning into their existing operations, they inevitably face a number of challenges. But perhaps the greatest and most-overlooked one is the people challenge. How can a successful software engineer make a transition to become an equally capable Machine Learning or Data Engineer?
How do team dynamics change when ML-specific processes, tools and pipelines are introduced? How can an engineering manager or team lead navigate this transition to retain their talented engineers and build a cohesive, high-performing ML team?
In this talk I will share some "war stories" and lessons learned from my own experience leading a multi-functional team through such a transition. The insights will hopefully be useful to both engineers and managers about to go into this experience.
QCon: What is the focus of your work?
Sally: Almost a year ago, I quit my job to finish my Phd. in AI and BioInformatics. It involves the use of AI techniques in the research of genetics and computational biology. Based on my Phd. research, I am also starting a company in the same space.
QCon: Your talk is called “From Software development to ML: A Team’s transformation”. What is the motivation for the talk?
Sally: The motivation came from my own experience leading a traditional software team into a machine learning. I had previously led a team for a health tech startup which was focused on AI and ML.
Before I joined this company, the team had been working on traditional software development projects in PHP. I was required to transform this team to an AI team to implement a challenging product for predicting health care needs. However, the biggest challenge turned out to be, turning a traditional software team into a high performing AI team.
Since many companies are trying to achieve this transformation, I wanted to share my experience.
QCon: What exactly does this transition to an AI-ML team comprise of?
Sally: The team had to learn Python from scratch. I had to hire some additional Python developers. Going from being a superstar PHP developer to a Python beginner was a challenge for some of the team members.
The mindset of a Python engineer is very different. It requires understanding a different framework and the Machine Learning pipeline. The individual also needs to understand how to deal with high volumes of data coming in at high speeds from different sources.
There are differences from a team perspective as well. For example, the Data Scientist role within a team is new for a traditional development team. AI-ML teams need to interact more with DevOps teams.
The challenge for me was not only to acquire a new skill set but also to hold the team together during this transition and maximize everyone's potential and strengths.
QCon: What will be the focus of your talk? Culture, tools, and/or people?
Sally: Culture and people will be my primary focus while tools and processes will be secondary. The main focus will be on how to deal with individual team members going through this transition. I will share a few examples of people and situations from my experience. I will share the lessons I learned from mistakes as well as the decisions which worked well for my team.
I will also talk about how the others in the organization perceive the AI-ML team and what are their expectations.
QCon: Who is the primary audience this talk is intended for?
Sally: Primarily, it is for team leaders who could be also be engineering managers or project managers. It should also be relevant to senior developers or engineers who are part of a team that is being transformed into an ML team.
QCon: What tips and recommendations would you give to an individual developer who wants to adopt AI and ML in their career?
Sally: The individual should be willing to invest a lot of personal time in learning. Self Learning and Self Motivation are important. I would recommend going through online courses and participating in competitions on Kaggle or looking at what others have done on GitHub.
You also need to make your supervisors aware of your interest, so that they can help you.
QCon: What do you think is the most important trend in software today?
Sally: I'm obviously biased, so I'll talk about AI. I am really excited to see what happens with the combination of AI and voice applications in the next few years. I believe voice will be the new main interface to many personal devices in the near future, and I don't think we've quite cracked it yet both on the voice recognition and generation fronts. In my opinion, voice interfaces combined with NLP techniques, deep learning and IOT will be the next wave.
Similar Talks
Psychologically Safe Process Evolution in a Flat Structure
Director of Software Development @Hunter_Ind
Christopher Lucian
Cultivating High-Performing Teams in Hypergrowth
Chief Scientist @n26
Patrick Kua
Inside Job: How to Build Great Teams Within a Legacy Organization?
Engineering Director @Meetup
Francisco Trindade
Liberating Structures @CapitalOne
Agile Coach, Engineering @CapitalOne
Greg Myers
Self-Selection for Resilience and Better Culture
Agile/DevOps Trainer & Founder of Agile Play Consulting, LLC
Dana Pylayeva
Front End Architecture in a World of AI
Front End Architect @oqtonai
Thijs Bernolet
Machine-Learned Indexes - Research from Google
Senior Research Scientist @Google
Alex Beutel
Breaking Hierarchy - How Spotify Enables Engineer Decision Making
Senior Engineering Manager, Data and Machine Learning Infrastructure @Spotify
Kristian Lindwall
Context Matters: Improving the Performance and Wellbeing of Teams
Director of IT @Etsy