0

I have a measurement program that pushes measurements into a web server, which is written in django and postgres. Currently I'm doing this update through a django form. What I've found, though, is that the form interface is pretty slow. If I set up a script, uploading 100 records through the form takes 7 or 8 seconds and pegs a cpu in my VM. Potentially I may end up with 20 or 30 measurement systems updating into the server, each running at 15 measures per second, so that update speed is not going to work.

I originally wrote it using the django ORM, then I rewrote it building the SQL string myself. Both equally slow. Then I commented out the database update entirely and its still pretty slow. So I guess the form interface is the bottleneck.

Questions:

  • this is still in development so I'm still using "python manage.py runserver". will wsgi (or whatever) speed this up a bunch?
  • what's the best practice for a programmatic single-record update into django?

here's the model for the form in question:

class MeasurementForm(forms.Form):
  RunKey = forms.CharField()
  MeasureTime = forms.CharField()
  CaliperValue = forms.CharField()
  CaliperToleranceStatus = forms.CharField()
  IOFieldName = forms.CharField(required=False)
  IOFieldValue = forms.CharField(required=False)
  ToleranceStatus = forms.CharField()
  Count = forms.CharField(required=False)

And here's the code for actually doing the update. The actual updating is commented out though.

def entermeasurement(request):
  if request.method == 'POST': # If the form has been submitted...
    form = MeasurementForm(request.POST) # A form bound to the POST data
    response = HttpResponse(mimetype='text/plain')

    if form.is_valid(): # All validation rules pass

      raws = form.cleaned_data['CaliperValue'].split(",")
      vals = []
      for v in raws:
        if "null" in unicode.lower(v):
          vals.append(None)
        else:
          vals.append(v)

      # insert a measurement record.
      cursor = connection.cursor()

      # print ("presql")

      # get number of records removed given the date.

      sqlstring = """insert into bwmeasures_measurements
      ("RunKey_id", 
      "MeasureTime", 
      "CaliperValue", 
      "CaliperToleranceStatus",
      "IOFieldName",
      "IOFieldValue", 
      "ToleranceStatus")
      values
      ( """

      """
      sqlstring = sqlstring + \
        str(form.cleaned_data['RunKey']) + \
      ", timestamp'" + \
      str(dateutil.parser.parse(form.cleaned_data['MeasureTime'])) + \
      "', '" + \
      postgresarray(vals) + \
      "', '" + \
      postgresarray(form.cleaned_data['CaliperToleranceStatus'].split(",")) + \
      "', '" + \
      postgresarray(quodstrings(form.cleaned_data['IOFieldName'])) + \
      "', '" + \
      postgresarray(quodstrings(form.cleaned_data['IOFieldValue'])) + \
      "', '" + \
      str(form.cleaned_data['ToleranceStatus']) + \
      "')"
      """

      # print(sqlstring)

      #print("would update here!")
      #cursor.execute(sqlstring)
      #transaction.commit_unless_managed()


      '''
      # Django ORM way of doing the insert.

      measurement = Measurements()

      measurement.RunKey = ProductionRuns.objects.get(id=int(form.cleaned_data['RunKey']))
      measurement.MeasureTime = dateutil.parser.parse(form.cleaned_data['MeasureTime'])

      measurement.CaliperValue = vals  

      measurement.CaliperToleranceStatus = form.cleaned_data['CaliperToleranceStatus'].split(",")
      measurement.IOFieldName = quodstrings(form.cleaned_data['IOFieldName'])
      measurement.IOFieldValue = quodstrings(form.cleaned_data['IOFieldValue'])
      measurement.ToleranceStatus = int(form.cleaned_data['ToleranceStatus'])

      measurement.save()
      '''

      # indicate success
      response.write("saved\n")
      response.write(measurement.id)

      # Process the data in form.cleaned_data
      return response 
    else:
      response.write("failed")
      response.write(str(form.errors))
      return response 

  else:
      form = MeasurementForm() # An unbound form

  return render(request, 'entermeasurement.html', {
      'form': form,
  })
4

0 回答 0