WingNews logo WingNews GitHub [2]

ianand's comments

ianand | 2 years ago | on: Spreadsheets are all you need

It's actually available in beta. It was announced while I working on this project but I kept going with pure Excel functions because I wanted to illustrate the transformer without abstractions getting in the way. It would make many aspects easier but also make it easier to hide a lot.

That being said, Python+Excel makes a ton of sense in general. And in this project, it would help in the tutorials. For example, in the embeddings tutorial I'm working on I wanted use PCA plots and SVD to illustrate the workings of embeddings but neither are natively supported in Excel without paid plug-ins. But both are easy in Python.

ianand | 2 years ago | on: Spreadsheets are all you need

This is so awesome!! I'm going to have to show this in the embeddings video I'm working on when I discuss non-text embeddings and CLIP.

While I created spreadsheet-are-all-you-need.ai as teaching tool, as I've been playing with it I've been having a growing suspicion the spreadsheet interface for AI might be useful beyond teaching, either as a power user control interface or for interpretability. For example, making simple changes to the architecture of GPT and observing how it changes the model behavior can be as simple as cloning a tab and a few spreadsheet functions. Of course, you can do the same in python as well so it remains to be seen.

A HUGE THANKS!

ianand | 2 years ago | on: Spreadsheets are all you need

I try to look at it as motivation to stay committed.

Like spreadsheets, domain names are a bit of an obsession for me. One of the other AI side projects I'm working on is a CustomGPT to help come up with domain names (https://niftynamer.com) so I don't have to come-up-with-another-unwieldy-long-domain-name.again.

ianand | 2 years ago | on: Spreadsheets are all you need

Happy to help and thanks! Do let me know how the material could be better. Always looking to improve it.

To answer your question:

EECS Major in college; 20 years of engineering and product management experience. I have given a few technical talks at conferences and I do enjoy the process of explaining things though it takes a surprising amount of work.

When I went to school ironically neural nets were the one thing they didn't cover in the intro to AI courses. I've basically learned modern AI from just filling my own curiosity over the years through online resources on nights and weekends. Learned a lot from Jeremy Howard's Fast.ai and Andrej Karpathy's stuff just like everyone else. I really wanted to know how every step of GPT worked, kind of like how you learn Computer Architecture in college: you learn how CPUs work in principle starting with circuits. Then I got a crazy idea the whole model could fit in a spreadsheet because well I just really like spreadsheets. Went down a 2-3 month rabbit hole in my non-existent sparetime to make it work.

ianand | 2 years ago | on: Spreadsheets are all you need

Strictly necessary? Maybe not. I wrote that before URIAL [1][2]. I actually haven't tried URIAL in GPT2 small but I need to give it a whirl. Might be too small a model to work?

Even if URIAL works with GPT2 small, the really small context length in the Excel file as currently implemented will make it hard to leverage. I've considered a more flexible implementation to support a longer context length (e.g. using Macros to build the layout of the sheet) but have prioritized the teaching videos first.

[1] https://allenai.github.io/re-align/index.html [2] Summary https://twitter.com/intuitmachine/status/1732089266883141856

ianand | 2 years ago | on: Spreadsheets are all you need

> Brilliant idea/URL.

Thanks! There's a truth to the name beyond just the play on the transformers paper. Definitely have thought about how many SaaS apps could be a spreadsheet and vice versa and often use them to create mini-apps (often via apps script).

ianand | 2 years ago | on: Spreadsheets are all you need

Creator here. Thanks for posting this! Happy to answer questions or take suggestions and hope it helps folks better understand LLMs.

Next video will be on embeddings and hopefully done soon-ish.

ianand | 2 years ago | on: Spreadsheets are all you need

Author here. Yes, Jeremy Howard and fast.ai was one of the inspirations for this! I'd actually be curious what he thinks of the project if he ever sees it.

ianand | 2 years ago | on: GPT in 500 Lines of SQL

Thanks! Each one takes a surprisingly long time to make. Even figuring out how make the explanation accessible and compelling yet still accurate takes awhile and then there’s still the actual video to do.
page 2
newer more