nlp - Is there a way to load the wiki-fasttext model faster with load_word2vec_format

Question

Loading the wiki-fasttext model with the gensim library takes six minutes.

I'm aware of ways to cache the model but I'm looking for ways to speedup the initial model loading. The specific api is below:

en_model = KeyedVectors.load_word2vec_format(os.path.join(root_dir, model_file))

Granted, wiki-fasttext a very large model, however I have load the same model in many languages.

Answer 1

您可以尝试使用limit=vector_num参数从文件中加载vector_num词向量。您不会加载所有矢量，但可以加快加载过程。