range_agg extremely slow compared to naive implementation in obscure circumstances

In general range_agg is faster than the naive version

CREATE AGGREGATE public.naive_range_agg(anymultirange) (
     SFUNC = multirange_union,
     STYPE = anymultirange
);

however here is an example where using it is over 6000 times slower.  I'm not 
sure exactly what feature of the example triggers this - I failed to create a 
synthetic testcase using generate_series, thus the attached table data.

How to reproduce (Ubuntu 22.10, x86_64, postgresql 15.1-1.pgdg22.10+1):

$ cp data.txt.gz /tmp/
$ gunzip /tmp/data.txt.gz
$ psql
Pager usage is off.
psql (15.1 (Ubuntu 15.1-1.pgdg22.10+1))
Type "help" for help.

duncan=> CREATE TEMP TABLE wacky(priority bigint, valid tstzrange);
CREATE TABLE
duncan=> \COPY wacky FROM /tmp/data.txt
COPY 98094
duncan=> CREATE AGGREGATE public.naive_range_agg(anymultirange) (
     SFUNC = multirange_union,
     STYPE = anymultirange
);
CREATE AGGREGATE
duncan=> \timing on
Timing is on.
duncan=> EXPLAIN (ANALYZE) SELECT FROM (SELECT valid, 
naive_range_agg(valid::tstzmultirange) OVER (ORDER BY priority DESC GROUPS 
BETWEEN UNBOUNDED PRECEDING AND 1 PRECEDING) AS more_recent FROM wacky) foo 
WHERE valid <@ more_recent;
                                                          QUERY PLAN 


-----------------------------------------------------------------------------------------------------------------------------
  Subquery Scan on foo  (cost=11063.57..13879.37 rows=433 width=0) (actual 
time=88.086..88.087 rows=0 loops=1)
    Filter: (foo.valid <@ foo.more_recent)
    Rows Removed by Filter: 98094
    ->  WindowAgg  (cost=11063.57..12796.37 rows=86640 width=72) (actual 
time=16.102..84.242 rows=98094 loops=1)
          ->  Sort  (cost=11063.57..11280.17 rows=86640 width=40) (actual 
time=16.096..20.205 rows=98094 loops=1)
                Sort Key: wacky.priority DESC
                Sort Method: external merge  Disk: 3848kB
                ->  Seq Scan on wacky  (cost=0.00..1588.40 rows=86640 width=40) 
(actual time=0.021..5.479 rows=98094 loops=1)
  Planning Time: 0.277 ms
  Execution Time: 88.631 ms
(10 rows)

Time: 98.261 ms
duncan=> EXPLAIN (ANALYZE) SELECT FROM (SELECT valid, 
range_agg(valid::tstzmultirange) OVER (ORDER BY priority DESC GROUPS BETWEEN 
UNBOUNDED PRECEDING AND 1 PRECEDING) AS more_recent FROM wacky) foo WHERE valid 
<@ more_recent;
                                                          QUERY PLAN 


-----------------------------------------------------------------------------------------------------------------------------
  Subquery Scan on foo  (cost=11063.57..13879.37 rows=433 width=0) (actual 
time=566009.972..566009.973 rows=0 loops=1)
    Filter: (foo.valid <@ foo.more_recent)
    Rows Removed by Filter: 98094
    ->  WindowAgg  (cost=11063.57..12796.37 rows=86640 width=72) (actual 
time=21.996..565998.800 rows=98094 loops=1)
          ->  Sort  (cost=11063.57..11280.17 rows=86640 width=40) (actual 
time=21.988..26.154 rows=98094 loops=1)
                Sort Key: wacky.priority DESC
                Sort Method: external merge  Disk: 3848kB
                ->  Seq Scan on wacky  (cost=0.00..1588.40 rows=86640 width=40) 
(actual time=0.014..6.868 rows=98094 loops=1)
  Planning Time: 0.178 ms
  Execution Time: 566010.770 ms
(10 rows)

Time: 566018.613 ms (09:26.019)