python - How to improve Allen NLP question answering performance

Question

I am trying out Allen NLP pre-trained models for Q&A.

The online demo is here : https://demo.allennlp.org/reading-comprehension

I have created a python script to try out various models.

• python script
• script output

Here is the benchmark summary on my laptop

• Macbook Pro (2017)
• 2.9 Ghz Intel i7 quad-core
• 16 G memory

Benchmark	transformer-qa	bidaf-model	bidaf-elmo-model
loading time	31.6 seconds	1.6 seconds	13.8 seconds
questions
Who stars in The Matrix?	794 ms	62 ms	1,798 ms
where does polar bear live	2,211 ms	96 ms	7,125 ms
how much does a polar bear weigh	2,435 ms	98 ms	7,082 ms
what is lightning	1,361 ms	69 ms	3,173 ms
How many lightning bolts strike earth	1,019 ms	47 ms	2,885 ms

Looking at the output I can see all 3 models are providing good answers. I like the transformer-qa model but it takes a while (in the order of seconds) to predict.

Is there a way to speed up prediction times?

thanks!

Answer 1

该transformer-qa模型包含更多参数，因此预计需要更长的时间。加快推理时间的一种方法是使用 GPU。predict如果您一次在一个实例上运行，则加速可能并不显着，批量运行应该会有所帮助。