A group of machine learning engineers from Xomnia attended the Pydata London 2024 event and enjoyed several talks that were on the agenda. Machine Learning Engineers Ilse van Beelen and Sofia Chatzopoulou were among them, and they summarized below some of the talks they found to be most interesting at the event.
1) RAG for a Medical Company
As GenAI continues to be a hot topic, several tutorials and talks at the conference focused on Retrieval Augmented Generation (RAG). This is an architectural approach that can improve the efficiency of large language model (LLM) applications by retrieving and injecting its user's own data.
However, achieving this effectively remains a challenge.
The talk by Generative AI Lead at Sicara (Theodo Group) Noé Achache focused on the technical and product challenges that arise when implementing RAG as an online knowledge base at medical companies. They aimed to use GenAI to help doctors find technical information on medication more quickly during a consultation with a patient. Some of the challenges facing them are providing only the right information, and highlighting or giving citations about where the information was found.
Building a tailored RAG application to achieve this required a lot of customization and manual work. Experts with medical knowledge are needed to collect and evaluate prompts to help with prompt engineering. They also had to focus on the entire machine learning cycle, aka LLMOps, to make sure that they were building a product that always works as expected. Closely monitoring the inputs and outputs of LLMs is crucial in LLM applications, which is why at Xomnia we recommend using an LLMOps tool from the start of a project.
During the talk, it was mentioned that it usually takes doctors up to 3 minutes to find the right medication during a consultation with a patient. The RAG app, however, can generate an answer in 50 seconds, including citations and highlights from the text where it found the information. It was mentioned, however, that doctors who tested this product did not find the generated answers to be trustworthy. They, nevertheless, found the highlighted text to be very insightful. The number of doctors who tested the model was not specified.
Want to build your own custom chatbot with private data (RAG)? Xomnia can help you get it right!
Overall, this application of RAG seems very promising, as it can potentially help doctors spend time more efficiently with their patients. However, the model still needs to be improved as it has only been tested on one small dataset so far. Another major challenge facing this model and others similar to it is that the target audience did not see the generated answer as trustworthy. This might change over time, however, as reliance on LLMs for different consultations becomes increasingly common across different industries.
Other RAG-talks covered topics like graph-based approaches and LLM function calling for querying structured data sources (SQL databases). In general, RAG continues to be the most practical application of GenAI technology at the moment, though getting it right is a big challenge.
You can watch the full presentation on YouTube by clicking here.
Read further: How to transform any website into a chatbot using LLMs?
2) Test-Driven Data Analysis in Python
Getting data analysis right is tough. We often have to deal with analytical goals that are poorly specified, missing values, data corruption, problems with interpreting input data, etc.
The talk on test-driven data analysis (TDDA) by Nick Radcliffe attempted to help with these problems by applying the ideas of test-driven development to data analysis. It can help with improving quality, correctness and robustness. All of this is captured in the tdda Python module.
The tdda package from the Python Package Index (PyPI) can help validate data by generating constraints for them and validating new data against those constraints. It can also create unit tests for dataset based on performing a set of automated data checks. This can help when exploring new data in a PoC to ensure that it is consistent with our expectations in production pipelines. The tdda package is also useful to create constraints and run tests on a dataset quickly and without writing any code. This can be very insightful when exploring a new dataset for the first time. However, it’s not a complete data quality framework such as great expectations.
The package also has a kind of GenAI component: Gentest that can help you write tests from the command line (only Unix/Linux). The tests it generates don’t really test if the code is correct, but will run it multiple times and test if the behavior is consistent and doesn’t generate errors. Gentest runs code multiple times and tries to provide for variations.
The full recording of the talk can be found on YouTube.
Joining the leading data & AI consultancy in The Netherlands. View our vacancies.
3) Modelling Emotional Nuance of Musical Performance
The talk "Can machines play the piano?" by Wojciech Matejuk from the Warsaw University of Technology explored using deep learning to model the emotional nuances of piano performances. It covered the intersection of music theory and data science, focusing on transforming MIDI recordings into data suitable for machine learning. The session demonstrated using Python tools to analyze musical expression, discussed training large models like BERT on MIDI data, and addressed challenges in data processing.
It was especially interesting how they managed to transform 'tabular' MIDI data into tokens (tokenization) in order to feed it to Large Language Models like GPT-2. The talk also tried to identify the importance of subject specialization when building machine learning algorithms and the challenges associated with interpreting music sheets from a person and a computer.
Overall, this was a very interesting talk, but this is more of a scientific project than a commercial one. We believe AI has the potential to improve human creativity, but should not fully take over.
The full presentation can be found on GitHub and the recorded video can be found on YouTube.
Read further: How to speed up Deep Reinforcement Learning with PPMP and RLHF?