Solr can store data inside its index so it won't hit the database when you want to display the data. I recommend you the sunspot gem that will you to achieve this. Checkout its wiki, especially the Setting up classes for search and indexing section where it is described.
Be aware that this way leads to data doubling - you will have the same data stored in the database and in the solr index.
Instead of storing data in solr index I advise you to use eager loading. You can preload colour percentages and also the color objects in the controller.
def show
# Loads fabrics, the join table and colours
@fabrics = Fabric.includes(:fabric_colours => [:colour])... # .all or .where or whatever
end
def search
# Searches for fabrics with eager loading
search = Fabric.search(:include => { :fabric_colours => :colour }) do
# ... search criteria
end
@fabrics = search.results
end
If this is still slow for you, use caching.