Subquery in a JOIN not getting restricted?

When I run the following query:

select questions.id
from questions
join (
     select u.id as user_id
     from users as u
     left join scores as s
     on s.user_id = u.id
) as subquery
on subquery.user_id = questions.user_id;

the subquery is scanning my entire user table, even though it's restricted
by the outer query.  (My real subquery is much more complicated, of course,
but this is the minimal fail case.)

Is this just not a thing the optimizer can do?  Are there ways to rewrite
this, still as a subquery, that will be smart enough to only produce the one
row of subquery that matches questions.user_id?

Jay Levitt