frogamel's comments

The research in this space is very conflicting about what methods actually work. In the graph on the page, the ETS model (basically just a weighted moving average) outperforms multiple, recent deep learning models. But the papers for those models claim they outperform ETS and other basic methods by quite a bit.

You can find recent papers from researchers about how their new transformers model is the best and SOTA, papers which claim transformers is garbage for time series and claim their own MLP variant is SOTA, other papers which claim deep learning in general underperforms compared to xgboost/lightgbm, etc.

Realistically I think time series is incredibly diverse, and results are going to be highly dependent on which dataset was cherry-picked for benchmarking. IMO this is why the idea of a time series foundation model is fundamentally flawed - transfer learning is the reason why foundation models work in language models, but most time series are overwhelmingly noise and don't provide enough context to figure out what information is actually transferrable between different time series.

I've worked in many research scientist/MLE roles over the years and I haven't met any IC who has this much of a fixation on AI being a moral evil/good. The ones who do are inevitably nontechnical hucksters, usually just trying to get money or self-promote.

I might be miunderstanding something here, but what complexity here is resolved by making this a framework? Isnt this just:

1. Train model like normal

2. Evaluate model using self

3. Use eval results for DPO finetune

IMO if you're going to profit off open research, you should at least make your own work available for other researchers. The white paper has 10 pages of performance benchmarks but 5 sentences on methodology.

Exactly one state has! https://en.wikipedia.org/wiki/Bank_of_North_Dakota

It's state-owned but functions as a typical retail bank that takes deposits, makes mortgages, business loans, etc.

Dplyr actually supports some really cool join functionalities that I wish were in SQL implementations, including:

- Ability to specify whether your join should be one-to-one, many-to-one, etc. So that R will throw an error instead of quietly returning 100x as many rows as expected (which I've seen a lot in SQL pipelines).

- A direct anti_join function. Much cleaner than using LEFT JOIN... WHERE b IS NULL to replicate an anti join.

- Support for rolling joins. E.g. for each user, get the price of their last transaction. Super common but can be a pain in SQL since it requires nested subqueries or CTEs.

I know many people who've gotten big raises over the last 2-3 years, including individuals who have more than doubled their pay. And they always attribute their raises to hard work, job hopping, or climbing the promotion ladder. At the same time, they'll also act like the 20% or whatever inflation we've had over the past few years is equivalent to the government stealing their hard earned money.

From a macro perspective, the same factors that drove inflation is what drove their big pay hikes. But most people only see that "I earned more money and now I'm being screwed by inflation". These are often the people who spend like crazy while simultaneously complaining about how bad the economy is!

Every tech company minus the few doing core research have been doing this for at least half a year. Generate training data with GPT4 or sometimes even 3.5 -> use it to do a QLoRA finetune on a llama or mistral base -> roll it out as a "proprietary" AI model -> management claims a big win and talks about how they're leaders in "[industry name] AI".

It is remarkably easy - it takes practically zero knowledge of ML and can usually be done with less than <$1k of cloud compute costs. The issue is that for most realistic tasks you can expect to end up with something roughly on the level of GPT-3.5, and its actually really hard to compute with GPT-3.5 on a cost level, at least if you use cloud GPUs.

I've noticed a huge surge in negativity and pessimism on English-language social media within the last year or so, roughly corresponding with to the spread of LLM tech. I do wonder whether these people are mostly just bots.

I dont think the first part of this article is quite right.

1. I doubt many people would think of 9/11 as a "1 in 50 years" event if it didn't actually happen. If you had a year-by-country level dataset of every developed country post ww2, you'd have thousands of observations but none would have as many fatalies from terrorism as the US in 2001.

2. If a genuinely super rare event occurs (like one in thousands), its often more reasonable to think that theres been some fundamental shift in the world that you failed to recognize, rather than that you just got super lucky or unlucky to have lived through it.

In my experience, you can get extremely good performance in this style of sports prediction with extremely basic methods. E.g. doing some sort of basic Elo rating system + choosing the higher rated competitor to win every game. This is simply because almost all human competitors will frequently try to predict upsets (as its more fun), but it will harm their performance overall.

I would be cool to see if something like this can do can outperform a benchmark Elo system but even that is an extremely high bar that most sophisticated models can't beat.

Honestly shocking that they would lay off a guy whose had this much of an influence on data science tooling generally. Every company I've worked at has had a sizable proportion of the analytics team using knitr+rmd.

I hope this does not mean there are financial troubles within Rstudio/Posit.

I must be missing something here. I used to work on a voice assistant product and everything demoed is pretty bog standard stuff: play music, shop, order food, get Uber rides, ask random queries, recognize images.

This also runs into the same core issue as voice assistants, which is that it's way easier to transmit complex thoughts/actions on a nice big visual UI than it is to convert those thoughts/actions into verbal instructions.

For example, when people order food from an online menu, most people are not fine with "get me the closest pizza available". They're usually considering complex tradeoffs between lots of factors: alternative restaurants nearby, how visually appealing is pictures of the food, health content, other options on the menu including their prices and content and how well they complement other dishes, etc. Figuring out how to express these preferences via a back-and-forth with an AI is more tedious than just looking at a visual interface and letting your subconscious process all this information into a choice.

A few months ago I tried migrating a large pandas codebase to polars. I'm not much of a fan of doing analytics/data pipelining in Python - a complex transformation takes me 2-5x as long in pandas compared to Julia or R (using dataframes.jl & dplyr).

Unfortunately polars was not it. Too many bugs on standard operations, unreliable interoperability with pandas (which is an issue since so many libraries require pandas dataframes as inputs), the API is also very verbose for a modern dataframe library, though it's still better than pandas.

Hopefully these will get resolved out over time but for now I had the best luck using duckdb on top of pandas, it is as fast as polars but more stable/better interoperability.

Eventually I hope the Python dataframe ecosystem gets to the same point as R's, where you have a analytics-oriented dataframe library with an intuitive API (dplyr) that can be easily used alongside a high-performance dataframe library (data.table).

I think companies in this "data analytics platform but with LLM integration" space are going to struggle in the next few years. The pain points of analytics are completely different than traditional software engineering, and things that LLMs help the most with (writing boilerplate code, bouncing between different frameworks, handling tests and docs) aren't serious issues in analytics work. I use copilot religiously for webdev work but find it completely useless for analytics.

Many of the recent articles in the media about productivity are inaccurate, there is a big difference between plain English "productivity" and the economic concept of productivity. The quoted productivity #s are aggregates measured by the BLS, the only thing it measures is aggregate US output (~= to real GDP) divided by total US hours worked (# employees * avg hours/employee).

The reason productivity is up so much lately is that for the past two years:

- Employment (https://fred.stlouisfed.org/series/PAYEMS) has been increasing at 1-2% annualized rate (because unemployment rate/labor force participation rate hasn't been changing, so employment growth has come from population growth, which was 1-2% when current 18-30 year-olds were born)

- Real GDP (https://fred.stlouisfed.org/series/GDPC1) has been increasing at 3-5% annualized rate

The reality is there is a giant consumption boom the past few years, but minimal increase in employment since we were already at full employment, so by definition there will just be increase in aggregate productivity. It is unrelated to concepts like how hard are people working.

This is a really interesting perspective, and made me consider what pushed me to inflate my own prices (my salary) especially aggressively the last few years.

In 2020-2022 my input costs (housing, food) went up ~20%, so I also needed to increase the price of my labor. This means negotiating a promotion or switching jobs, which is a lot of effort. And since I'm putting in that effort anyways, I might as well optimize it and try to get every extra dime, which ended being more than a 20% increase.

In non-inflationary periods, it's not even worth the effort and stress to optimize my salary like this.

I imagine the same was true for a lot of businesses in the supply chain. If you're some SAAS company, raising prices 20% requires renegotiating contracts, dealing with angry clients, etc. So if you're going to do that anyways you might as well try to raise it even more.