django - testcase.me

To remind, the N+1 problem means a code like this:

for comment in Comment.objects.filter(post_id=1).all():
    print(comment.author.avatar_url)

makes N+1 requests to the database (hence the name):

SELECT author_id FROM appname_comment WHERE post_id = 1;
SELECT avatar_url FROM appname_author WHERE id = 1;
...
SEELCT avatar_url FROM appname_author WHERE id = n;

despite only 2 requests were really needed here (not considering a single-request JOIN or nested SELECT for reasons that will be clear later):

SELECT author_id FROM appname_comment WHERE post_id = 1;
SELECT avatar_url FROM appname_author WHERE id IN (1, ..., n);

Indeed, it’s a well-known problem and Django offers a simple solution to it:

for comment in Comment.objects.prefetch_related("author").filter(post_id=1).all():
    print(comment.author.avatar_url)

But what if the “related” data isn’t stored in a DB table? For instance, if the user data such as avatars belongs to a separate service which is available over an HTTP API only.