#46 Paris Women in Machine Learning and Data Science: evaluation framework for RAGs, managing international tech teams, and a journey in long-life training
Last Wednesday marked our inaugural event of the year, graciously hosted by Hugging Face in their offices nestled in the heart of Paris.
Attendees were warmly welcomed, treated to an array of Hugging Face goodies, and gifted with exclusive T-shirts from both Hugging Face and WiMLDS!
The event began with the usual presentation of the mission and initiatives of WiMLDS, aimed at empowering women in the fields of Data Science and Artificial Intelligence. Then our volunteer Juliette announced the next special event organised by the Paris Chapter: to celebrate the opening of the Maison Henri Poincaré, a mathematics museum, 30 members of our community were invited to participate in a one-hour guided tour in the museum, for free.
Our first 2 speakers, Nour El Mawass, GenAI Tech Lead, and Maria Knorps, Senior Data Engineer, from Tweag, presented their work on developing an evaluation framework for Retrieval-Augmented Generation (RAG) systems, aiming to improve intelligent search and suggestion across Confluence and Bazel documents.
Nour initiated the presentation by demystifying the concept of RAG systems. She highlighted the challenge of enabling LLMs to respond to queries involving private data. Fine-tuning the LLM with new data is effective, but is also expensive and demands substantial technical expertise, not to mention the necessity for repeated fine-tuning with every new data influx.
In contrast, the team’s chosen strategy leverages the power of RAG by introducing relevant documents or contexts to the prompts before processing them with LLMs. Maria took the stage to delve into the intricacies of Tweag’s RAG evaluation framework, structured around four critical pillars:
- Benchmarking: The team’s approach involved using question-answer pairs or question-document pairs. The challenge lay in curating questions that accurately reflect the dataset, user interactions, and ensure variety. Here, LLMs played a crucial role in generating and refining questions.
- Parameter Space Exploration: Maria provided insights into the parameters that shape the framework, including embedding models, the chunking model, the ‘top_k’ parameter for selecting the most relevant documents, and the preprocessing model.
- Evaluation Metrics: The framework employs a dual approach to metrics, incorporating traditional information retrieval metrics such as recall and precision, alongside LLM-based evaluations focusing on context relevance and context recall.
- Experiment Tracking: An essential component for iterative improvement and validation of their methods.
This presentation highlighted the utility of RAG systems in augmenting LLM capabilities and introduced a methodical approach to their evaluation. Kudos to Nour and Maria for your work!
If you want to know more, check out her slides below or checkout their blog post.
With several years of experience in international work environments, our second speaker, Natasha Dimban, shared her experience in managing international tech teams.
Managing international tech teams faces unique challenges, heightened by the shift towards remote work due to technological progress and the COVID-19 pandemic. This situation has led to benefits such as increased flexibility, access to global talent, and cost savings, but also presents difficulties in coordination across different time zones, language barriers, and cultural differences.
The speaker shared experiences with teams in Paris, Africa, and Asia, emphasising the importance of effective team management, which involves leadership, motivation, and setting clear goals and deadlines. Key challenges include managing time differences, overcoming language and cultural barriers, ensuring effective project management and coordination, maintaining team cohesion, or adapting leadership styles to accommodate cultural diversity.
To address these challenges, a strong foundation based on clear governance is essential. This includes establishing a shared vision and clear goals, using effective communication channels that suit the team’s needs, fostering cultural intelligence and effective communication to bridge gaps, and promoting collaboration and efficient time management to handle different time zones.
In essence, successfully managing international tech teams requires a strategic approach that combines strong governance, transparency, the right technological tools, and a focus on the human factor to ensure project success and team well-being.
Thank you Natasha you sharing your experience with us!
If you want to know more, check out her slides below:
Our final speaker, Morgane Pannegon, delivered an inspiring talk about life-long training — a journey she embarked upon herself. Following a pivotal encounter with Nicole El Karoui five years ago, Morgane was inspired to return to academia at the age of 50, diving into the fields of data, statistics, and psychology.
These new skills validated, Morgane decided to challenge this sentence: ‘Correctly implemented, lifelong training can be a lever in the management of inequalities between men and women.’
Within her organisation, she has been able to analyse the impact of life-long training programs (DIF, then CPF) on gender disparities. Her findings indicate that while lifelong training can enhance productivity and potentially increase salaries for women in senior positions by up to 5%, access to these programs is influenced by social factors such as marital status and parenthood, revealing underlying gender stereotypes.
Morgane’s study, leveraging a decade of HR data, showed that the DIF program was predominantly utilised by women in the early stages of their careers, aiming to boost their educational capital. In contrast, the CPF program presented a more balanced gender distribution but was affected by social determinants, with men having children under six more likely to access training, reflecting societal stereotypes.
Thank you Morgane for this inspiring talk, highlighting the importance of ensuring equitable access to career advancement opportunities!
If you want to know more, check out her slides below:
Our next event is likely to take place at the end of April. Stay tuned for updates!
If you do not want to miss our events, you can:
🔗 follow us on Twitter, Meetup, and LinkedIn
📑 check our Google spreadsheet if you want to speak 📣, host 💙, help 🌠
📍join our Slack channel for information, discussions, and opportunities
📩 send an email to the Paris WiMLDS team to paris@wimlds.org
🎬 subscribe to our WiMLDS Paris medium page and Youtube channel
📸 follow the global WiMLDS on Instagram, LinkedIn, and Facebook
🔥 share your company or lab’s job offers for free on the global WiMLDS’ website.
