mysql - Django admin MySQL slow INNER JOIN

Question

I have a simple model with 3 ForeignKey fields.

class Car(models.Model):
    wheel = models.ForeignKey('Wheel', related_name='wheels')
    created = models.DateTimeField(auto_now_add=True)
    max_speed = models.PositiveSmallIntegerField(null=True)
    dealer = models.ForeignKey('Dealer')
    category = models.ForeignKey('Category')

For the list view in the django admin i get 4 queries. One of them is a SELECT with 3 INNER JOINS. That one query is way to slow. Replacing the INNER JOINs with STRAIGHT_JOIN would fix the issue. Is there a way to patch the admin generated query just before it is evaluated?

score 7 · Accepted Answer

我已经为 Django ORM 实现了 INNER JOIN 的修复，如果使用 INNER JOIN 进行排序，它将使用 STRAIGHT_JOIN。我与 Django core-devs 进行了交谈，我们决定暂时将其作为一个单独的后端来做。所以你可以在这里查看：https ://pypi.python.org/pypi/django-mysql-fix

但是，还有另一种解决方法。使用 James 回答的片段，但将 select_related 替换为：

qs = qs.select_related('').prefetch_related('wheel', 'dealer', 'category')

它将取消 INNER JOIN 并使用 4 个单独的查询：1 个用于获取汽车，另外 3 个使用car_idIN (...)。

更新： 我发现了另一种解决方法。在 ForeignKey 字段中指定 null=True 后，Django 将使用 LEFT OUTER JOIN 而不是 INNER JOIN。在这种情况下，LEFT OUTER JOIN 没有性能问题，但您可能会遇到我还不知道的其他问题。

score 4 · Accepted Answer

您可以只指定list_select_related = ()阻止 django 使用内部连接：

class CarAdmin(admin.ModelAdmin):
    list_select_related = ()

score 2 · Accepted Answer

你可以覆盖

  def changelist_view(self, request, extra_context=None):

ModelAdmin从类继承的管理类中的方法

像这样的东西（但这个问题相当老了）： Django Admin：Getting a QuerySet filtered based on GET string，与更改列表中看到的完全一样？

score 1 · Accepted Answer

I came across the same issue in the Django admin (version 1.4.9) where fairly simple admin listing pages were very slow when backed by MySQL.

In my case it was caused by the ChangeList.get_query_set() method adding an overly-broad global select_related() to the query set if any fields in list_display were many-to-one relationships. For a proper database (cough PostgreSQL cough) this wouldn't be a problem, but it was for MySQL once more than a few joins were triggered this way.

The cleanest solution I found was to replace the global select_related() directive with a more targeted one that only joined tables that were really necessary. This was easy enough to do by calling select_related() with explicit relationship names.

This approach likely ends up swapping in-database joins for multiple follow-up queries, but if MySQL is choking on the large query many small ones may be faster for you.

Here's what I did, more or less:

from django.contrib.admin.views.main import ChangeList


class CarChangeList(ChangeList):

    def get_query_set(self, request):
        """
        Replace a global select_related() directive added by Django in 
        ChangeList.get_query_set() with a more limited one.
        """
        qs = super(CarChangeList, self).get_query_set(request)
        qs = qs.select_related('wheel')  # Don't join on dealer or category
        return qs


class CarAdmin(admin.ModelAdmin):

        def get_changelist(self, request, **kwargs):
            return CarChangeList

score 1 · Accepted Answer

好的，我找到了一种方法来修补管理员生成的查询。它很丑陋，但似乎有效：

class CarChangeList(ChangeList):

    def get_results(self, request):
        """Override to patch ORM generated SQL"""
        super(CarChangeList, self).get_results(request)
        original_qs = self.result_list
        sql = str(original_qs.query)
        new_qs = Car.objects.raw(sql.replace('INNER JOIN', 'STRAIGHT_JOIN'))

        def patch_len(self):
           return original_qs.count()
        new_qs.__class__.__len__ = patch_len

        self.result_list = new_qs


class CarAdmin(admin.ModelAdmin):

    list_display = ('wheel', 'max_speed', 'dealer', 'category', 'created')

    def get_changelist(self, request, **kwargs):
        """Return custom Changelist"""
        return CarChangeList

admin.site.register(Rank, RankAdmin)

score 1 · Accepted Answer

我在 MySQL 上进行了缓慢的管理查询，发现最简单的解决方案是将 STRAIGHT_JOIN 添加到查询中。我想出了一种将其添加到 aQuerySet而不是被迫转到的方法.raw()，这不适用于管理员，并将其作为django-mysql. 然后，您可以：

def get_queryset(self, request):
    qs = super(MyAdmin, self).get_queryset(request)
    return qs.straight_join()

score 0 · Accepted Answer

即使在版本 8 中 MySQL 仍然存在这个问题，并且 Django 仍然不允许您在查询集中添加 STRAIGHT_JOIN。我找到了一个添加 STRAIGHT_JOIN 的 hackish 解决方案...：

这是用 Django 2.1 和 MySQL 5.7 / 8.0 测试的

def fixQuerySet(querySet):
    # complete the SQL with params encapsulated in quotes
    sql, params = querySet.query.sql_with_params()
    newParams = ()
    for param in params:
        if not str(param).startswith("'"):
            if isinstance(param, str):
                param = re.sub("'", "\\'", param)
            newParams = newParams + ("'{}'".format(param),)
        else:
            newParams = newParams + (param,)
    rawQuery = sql % newParams

    # escape the percent used in SQL LIKE statements
    rawQuery = re.sub('%', '%%', rawQuery)

    # replace SELECT with SELECT STRAIGHT_JOIN
    rawQuery = rawQuery.replace('SELECT', 'SELECT STRAIGHT_JOIN')

    return querySet.model.objects.raw(rawQuery)

重要提示：此方法返回原始查询集，因此应在使用查询集之前调用

mysql - Django admin MySQL slow INNER JOIN

7 回答 7

Related

Reference