0

我有一个无限循环，它获取一些数据并将其写入 csv。有时由于随机原因，很可能是由于内存不足，脚本卡住了。因此，如果发生这种情况，我尝试重新启动脚本，借助正在编写的检查文件和逐字使用的重新启动功能，我得到了以下代码

from urllib2 import Request, urlopen
import pandas as pd

from datetime import datetime, timedelta
import time
import json
import os.path

import os
import sys
import psutil
import logging

currtime = time.localtime()
dct = time.strftime('%d%b%Y', currtime).upper()
datafile = 'Idx' + dct + '.csv'
csvfile = 'Index/' + datafile

ptime = datetime.now()
idx = pd.DataFrame()

last_size = os.stat(csvfile).st_size
last_time = ptime + timedelta(0,5*60)

while True:
    if (ptime.hour*60*60 + ptime.minute*60 + ptime.second) > stop:
        break
    else:
        try:
            ptime = datetime.now()
            idx = getIndexData()
            idx.to_csv(csvfile, index=False, mode='a', header=False)


            if not nosleep:
                end = datetime.now()
                tdiff = end-ptime
                time.sleep(scrapeGap - (tdiff.seconds + tdiff.microseconds*1.0/10**6))

        except Exception as e:
            with open("idx.log", "a") as lf:
                lf.write("Error at {0} \t {1}\n".format(str(ptime), str(e)))
            continue

        if ptime > last_time:
            if os.stat(csvfile).st_size - last_size > 0:
                continue
            else:
                restart_program()

            last_size = os.stat(csvfile).st_size
            last_time = ptime + timedelta(0,5*60)

这是构建整个事物的正确方法吗？我没有时间测试脚本的重启，因为我不能肯定地触发脚本卡住。

有关获取有关导致脚本卡住的原因的信息的任何方法的指针也将有很大帮助。

编辑：

正如@iAmTryingOK 在这里询问的那样，正在编写的 csv 文件

OI.call,changeOI.call,volume.call,IV.call,LTP.call,netChange.call,bidQ.call,bidP.call,askP.call,askQ.call,strike,bidQ.put,bidP.put,askP.put,askQ.put,netChange.put,LTP.put,IV.put,volume.put,changeOI.put,OI.put,timestamp
18300.0,-450.0,8.0,0.0,2474.85,43.1,75.0,2459.6,2471.6,600.0,9300.0,150.0,3.35,3.5,675.0,0.15,3.5,36.06,139.0,-6300.0,56625.0,2019-05-01 09:00:29.624
0.0,0.0,0.0,0.0,0.0,0.0,375.0,2261.9,2695.95,300.0,9350.0,300.0,0.1,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2019-05-01 09:00:29.624
1125.0,0.0,0.0,0.0,2384.95,-0.55,75.0,2358.1,2383.65,75.0,9400.0,75.0,3.5,5.4,75.0,0.05,4.2,35.47,2.0,75.0,600.0,2019-05-01 09:00:29.624
0.0,0.0,0.0,0.0,0.0,0.0,375.0,2141.7,2555.9,4950.0,9450.0,150.0,0.55,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2019-05-01 09:00:29.624
870225.0,4275.0,653.0,0.0,2264.95,-18.95,300.0,2264.1,2271.95,375.0,9500.0,2175.0,3.95,4.05,450.0,-0.55,3.95,33.75,1131.0,29100.0,293550.0,2019-05-01 09:00:29.624
4

1 回答 1

0

错误是，当scrapeGap - (tdiff.seconds + tdiff.microseconds*1.0/10**6)结果为负数时，它会将执行推迟一个负数的时间，这意味着它永远不会突破time.sleep. 这导致代码卡住了。只在睡眠前检查time.sleep一个正数解决了这个问题。

越野车部分：

time.sleep(scrapeGap - (tdiff.seconds + tdiff.microseconds*1.0/10**6))

更正部分：

tslip = scrapeGap - (tdiff.seconds + tdiff.microseconds*1.0/10**6)
if tslip > 0:
   time.sleep(tslip)

上面编辑的代码片段解决了这个问题，添加这个以供将来参考。

于 2019-11-30T11:19:58.160 回答