csdl - Frequently Updated CSDL query in DataSift

Question

Our DataSift CSDL query looks this way:

List<string> keywords=dbAccess.GetAllKeywords(); // there are 100K+ of them

string csKwList="\""+String.Join(",", keywords)+"\"";
string csdl = "facebook.message contains_any "+csKwList;

DataSiftManager.Resubscribe(csdl); //this involves deleting current subscritpion, recompiling a new csdl, and subscribing anew.

This works but each time a couple of new keywords are added to the list, I have to pull the entire list from the DB. This is unacceptable.

My question is, if there is a way to slightly modify a currently active subscription if I know exactly what keywords are being added and removed from the csdl query ?

Answer 1

At present, when you need to 'modify' your CSDL, you are required to recompile the definition. This will mean grabbing your full list of keywords, and adding them to your CSDL definition. DataSift is working to improve this process by allowing smarter management of large lists of keywords, though this feature is still in development.