Making ActiveRecord 2x faster

[cheald]

It's good to see members of the Rails core team taking performance seriously. I've been on a personal performance quest for the last couple of years, and what I've discovered is that the Rails community as a whole (not necessarily the core developers, but just people producing Rails-related software as a whole) really seems to consider performance to be something of an afterthought. There is a lot of low-hanging fruit laying around that could substantially improve quality of life for the entire Rails ecosystem if people would put a little time into profiling their stuff and eliminating hotspots.

It's been very discouraging to me to see responses to performance issues that range from lukewarm to plain "I don't care". The Rails community could benefit a lot from a visible push by the core team to make performance a priority.

[davidw]

> seems to consider performance to be something of an afterthought

Not an afterthought, but I'm more interested in providing features for my customers. The site is fast enough, and the time to develop new features is more important than making the thing scale for zillions of users when I only have 100's hitting the web site.

I don't ignore performance, but if you're charging enough, it's certainly not the primary consideration.

[fat0wl]

yea this attitude is a BIG part of the reason i switched to Java. If you learn some common Java design patterns (ahem... program to interfaces) then all of a sudden the magic of Ruby doesn't seem to add that much (in fact the data-binding in Java usually simplifies form validation). So then with a huge boost in performance for free, all of a sudden hosting costs are far lower and less billable hours are put to testing cache expiration.

That said, I will still probably end up migrating toward Clojure/cljs and js-focused apps...

EDIT: It's a waste to bleed a client if all your blood-money is going to Heroku! XD

[moe]

something of an afterthought

Cf. the comical performance of the entire rubygems ecosystem (no least bundler).

A programming language community can hardly make a stronger statement about their general mindset wrt performance than tolerating such an unmitigated trainwreck at the very core of their ecosystem and _every single developers_ daily workflow.

[atomical]

Is caching taking performance seriously? The cache everything approach keeps people from asking "What is it doing? Why is it taking so long when I'm only doing X, Y, and Z?"

[thibaut_barrere]

Quoting @tenderlove: "I think we can make the next release of Rails (the release after 4.1) the fastest version ever."

I'm glad to see the gradual progression since 3.2 stable!

[midas007]

Props for Aaron's continuous work on mature optimization of Ruby & Rails. I recall his yeoman's work on Ruby dtrace probes and Rails initialization time.

[mrcwinn]

I applaud anyone trying to improve performance. It is discouraging, though, that it only helps with some of the fastest and most efficient queries: direct SELECTs by ID without any joins or LIKE statements.

For those interested in good caching without bypassing parts of ActiveRecord, Memcache + Dalli (gem) is a great solution. Caching is not always the right solution, though. Sometimes the best choice is reversing how you get to that data or optimizing your query directly using SQL.

[jrochkind1]

these particular optimizations aren't caching any data from the database.

They are only caching calculations internal to AR. So you don't need to worry about using cached data when the actual external data has changed, is one thing.

The caching here is only of things that can't possibly change, essentially the output of a function for a given input.

Your memcache suggestions are a different sort of thing entirely, which certainly may be helpful for some apps in some circumstances.

[jrochkind1]

I am skeptical as to whether the calls speeded up by this optimization form a sufficient portion of the actual call time of any actual real world apps actions, such that this optimization will have any non-trivial effect on a real world app.

It might. But I definitely wouldn't assume it does. Micro-optimization.

Of course, it doesn't hurt either way, unless it does in code complexity, or opportunity cost.

[ikawe]

This seems to be but a first step - it's currently limited only to calls of the form `Article.find(id)` and `Article.find_by_xxx(value)`.

e.g. `Article.where(xxx: value)` or `Article.where(xxx: value).last(3)` will not benefit from this improvement yet.

[tenderlove]

It's possible to do that in AdequateRecord, but I've purposely left the mechanism undocumented because the API is not stable. You can do something like this:

  statement = undocumented_method_call { |params|
    Article.where(xxx: params[:xxx])
  }
  statement.execute xxx: "some value"

https://github.com/rails/rails/blob/e5e440f477a0b5e06b008ee7...

The trick is that the cache only supports caching ActiveRecord::Relation objects, and not everyone knows which calls will return a Relation object. For now, I want to hide this API and make it as "automatic" as possible so that people don't have to change app code, but still get performance boosts.

[adamkittelson]

Isn't the find_by_XXX style (soon to be?) deprecated?

[chancancode]

There has been a lot of confusion about this, but this is not true[1]. `find_by_xxx` and `find_by_xxx!` are not part of the deprecation.

1. http://edgeguides.rubyonrails.org/upgrading_ruby_on_rails.ht...

[ikawe]

Nope: http://edgeguides.rubyonrails.org/4_0_release_notes.html#act...

That one is staying, most of the others are going (e.g. find_all_by_xxx, find_last_by_xxx)

[sheff]

Something else to look forward to in 2014 for Rails performance, along with the planned arrival of JRuby 9k !

[troyk]

It appears the 2x speedup is from by-passing ActiveRecord::Relation

[mikkelewis]

Will this work for Rails 3.2?