Skip to content

Fix missing .unsqueeze(1) in attention mask for HunyuanVideo VAE#13133

Open
Minisal wants to merge 1 commit intohuggingface:mainfrom
Minisal:fix-hy1.5-attnmask
Open

Fix missing .unsqueeze(1) in attention mask for HunyuanVideo VAE#13133
Minisal wants to merge 1 commit intohuggingface:mainfrom
Minisal:fix-hy1.5-attnmask

Conversation

@Minisal
Copy link

@Minisal Minisal commented Feb 13, 2026

What does this PR do?

This PR fixes a dimension mismatch in the attention mask handling for the HunyuanVideo1.5 VAE implementation in autoencoder_kl_hunyuanvideo15.py.

Problem

In the current implementation (line 148 of autoencoder_kl_hunyuanvideo15.py), the attention_mask is passed directly to scaled_dot_product_attention without reshaping:

x = nn.functional.scaled_dot_product_attention(query, key, value, attn_mask=attention_mask)

However, the original HuanyuanVideo implementation (line 211 of huanyuanvideo_15_vae.py) correctly applies .unsqueeze(1) to the mask:

h_ = nn.functional.scaled_dot_product_attention(q, k, v, attn_mask=attention_mask.unsqueeze(1))

This ensures the attention mask has the correct shape for batched attention computation. Without this, the mask dimensions may not align with the query/key/value tensors, leading to potential runtime errors or incorrect attention behavior.

Fix

Add .unsqueeze(1) to the attention_mask argument in the scaled_dot_product_attention call to match the original implementation.


Changes

  • File: src/diffusers/models/autoencoder_kl_hunyuanvideo15.py
  • Line: 148
  • Before:
    x = nn.functional.scaled_dot_product_attention(query, key, value, attn_mask=attention_mask)
  • After:
    x = nn.functional.scaled_dot_product_attention(query, key, value, attn_mask=attention_mask.unsqueeze(1))

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@yiyixuxu
@asomoza
@sayakpaul
@DN6

@sayakpaul
Copy link
Member

Can you provide a reproducer?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants