vukadinovic's comments

Inspired by Karpathy's nanochat, I developed a minimal, from scratch implementation of a visual language model for report generation from medical images. AI in healthcare models are rarely fully open-source, causing researchers to struggle adapting these models on their own data, and barriers to entry for newcomers in the field. I've made echovlm fully open-source. I provide a complete pipeline to train the model end-to-end using 120k publicly available imaging reports and matched synthetic videos for just $5 in 2 hours. To demonstrate echovlm in practice, I've included an inference example using a real video of my own heart. I'm looking forward to researchers using this for their experiments, students to learn about vlms for medicine and welcome new pull requests from contributors.

This repo is a full-stack implementation of a VLM for medical report generation from imaging scans. It is designed to serve as a practical example for researchers, demonstrating end-to-end training of VLMs on medical imaging data, and can be adapted to various imaging modalities. echovlm is inspired by Karpathy's nanochat, and is also a fully open source, reproducible codebase which makes it one of the few public codebases for medical machine learning. As a running example, we use an echocardiography dataset with synthethic reports and study embeddings, which allows us to simulate VLM training data despite not having access to raw medical imaging scans. echovlm is very light and it can run on a single gpu via speedrun.sh script, that runs the entire pipeline start to end. This includes dataset preparation, tokenization, training, evaluation, inference and real example on the scan of my heart.

Distances in t-SNE/UMAP don't mean anything. They are clustering algorithms

I was thinking the same!

I used VIM plugin until now, was happy to see this. However, tried installing it on top of neovim 0.9.0-beta and VS code complains that the version nees to be 0.5 or higher... switching between modes worked but moving (jkli) didn't. So gave up trying to install it. Might try later looks interesting

When you're so rich and bored so you start doing regular jobs lol

I have been using Google Colab and AWS for 'big' personal projects while I use my PC to train light models. Colab is neat because it's free and I like the style of their notebooks. However, you cannot always find available GPU and I wouldn't recommend free Colab for anything else than learning and experimenting with ML. I have been using AWS Ubuntu server set up with PyTorch and had OK experience, but you need to be careful about pricing and set up policies, as well as remember to turn off your machines when you're not working on them if you don't want your credit card to blow up. In the future, I might give Google Colab Pro a try, but most of the work now I do on the company's server.

Anecdote: When I was taking the 'Computing For Data Science' class, we had a task to learn to use AWS tools like SageMaker, NLP bot or DeepRacer and present it in the class. The professor was also new to the whole AWS ecosystem. He opened many instances and left them running for a week which ended up taking 1000$ from bank account.. (Moral of the story: don't use aws with the card where all your money is)

Google misinterpreted Karpathy's (joke) tweet, and by searching " what is the best learning rate for adam optimizer " says that " 3e-4 is the best hands down ".

Location: Helena, Montana, USA

Remote: yes

Willing to relocate: yes

Technologies: Julia, Python, C++, Java, C#, Javascript, SQL, Hadoop, PyTorch, AWS, R

Résumé: https://drive.google.com/file/d/18pZ18H5mxIvDTxVNSu41K9gJI30...

Email: [email protected]

Linkedin: https://www.linkedin.com/in/milos-vukadinovic-b7b0171a2/

GitHub: https://github.com/vukadinovic936

++ MACHINE LEARNING INTERN, DATA SCIENCE INTERN, SOFTWARE DEVELOPER ++

My interests include computer vision, machine learning and data science. I am doing a double major in Mathematics & Computer Science with expected graduation in Spring 2022. I have shown excellent performance in university courses and have been a student assistant for both Math and Computer Science department. From May 2020 to December 2020, I have been a part of Computer Vision team in Ablera, where I was researching and developing convolutional neural networks for car damage detection. I am looking for a place where I will be able to use my skills in mathematical modelling and an environment where I will be able to grow and learn new things.