I'm using Thinking Sphinx to index a single table with 43,000 records. When I try to index it, the process runs very slowly and freezes after 36,000.
I assumed that there was some problem with the data and tried to discover which index caused the problem. After removing all but 11 indexes, I still consistently see the error. But here's the rub: no single index appears to be the culprit. If I remove any one of the last 11, the process finishes without freezing. If I add it back, the problem comes back. (Though I have noticed that when I remove one of these indexes, the process still slows down a lot at 39,000.)
So now I'm stumped. I have plenty of ram and disk space, so that's not the issue. I set sql_range_step to 15000 and mem_limit to 2047M. Nothing helps.
Here is my pared-down list of indexes:
define_index do
indexes :email
has :id, :as => :code
has :premium_school_id
has :current_employer_account_id
has :years_experience_id
has :user_status_id
has practice_areas(:id), :as => :practice_area_ids
has languages(:id), :as => :language_ids
has preferred_employer_types(:id), :as => :preferred_employer_type_ids
has geographical_regions(:id), :as => :geographical_region_ids
has workflow_events(:id), :as => :workflow_event_ids
end
Any ideas are welcome.