Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scrollbar Styling tests don't work with overlay scrollbars #898

Open
gsnedders opened this issue Oct 18, 2024 · 1 comment
Open

Scrollbar Styling tests don't work with overlay scrollbars #898

gsnedders opened this issue Oct 18, 2024 · 1 comment
Labels
test-change-proposal Proposal to add or remove tests for an interop area

Comments

@gsnedders
Copy link
Member

gsnedders commented Oct 18, 2024

Test List

/css/css-overflow/scrollbar-gutter-001.html
/css/css-overflow/scrollbar-gutter-dynamic-001.html
/css/css-overflow/scrollbar-gutter-propagation-001.html
/css/css-overflow/scrollbar-gutter-propagation-002.html
/css/css-overflow/scrollbar-gutter-propagation-003.html
/css/css-overflow/scrollbar-gutter-propagation-007.html
/css/css-overflow/scrollbar-gutter-rtl-001.html
/css/css-overflow/scrollbar-gutter-vertical-lr-001.html
/css/css-overflow/scrollbar-gutter-vertical-rl-001.html
/css/css-scrollbars/scrollbar-width-001.html
/css/css-scrollbars/scrollbar-width-002.html
/css/css-scrollbars/scrollbar-width-003.html
/css/css-scrollbars/scrollbar-width-004.html
/css/css-scrollbars/scrollbar-width-005.html
/css/css-scrollbars/scrollbar-width-006.html
/css/css-scrollbars/scrollbar-width-008.html
/css/css-scrollbars/scrollbar-width-009.html
/css/css-scrollbars/scrollbar-width-011.html
/css/css-scrollbars/scrollbar-width-013.html
/css/css-scrollbars/scrollbar-width-015.html
/css/css-scrollbars/scrollbar-width-016.html

Rationale

WPT does not have any way to force classic scrollbars (web-platform-tests/wpt#36422) nor force some mocked scrollbars to be used (web-platform-tests/wpt#42697) to use classic scrollbars where the underlying system does not support them.

web-platform-tests/wpt#44196 / web-platform-tests/wpt#44195 reported a number of problems with scrollbars, if overlay scrollbars are used, including in the tests in our focus area.

This has caused a significant regression in the focus area score for Safari when we moved to Apple-managed infrastructure: https://wpt.fyi/results/css?q=label%3Ainterop-2024-scrollbar&diff&filter=ADC&run_id=5116283788197888&run_id=5200055011377152.

We currently plan on changing the configuration of macOS on the WPT bots to always use classic scrollbars, but the problem here is more fundamental insofar as we have tests relying on a scrollbar type that WPT does not guarantee nor can WPT tests opt in to.

There's no immediate actual test change being proposed here, as the only possible change with current WPT semantics would be to exclude large numbers of tests from the Scrollbar Styling focus area (and it's very hard to test scrollbars when they're overlay scrollbars which are only showed while scrolling!), however the Interop group should be aware of the problems here.

@gsnedders gsnedders added the test-change-proposal Proposal to add or remove tests for an interop area label Oct 18, 2024
@smfr
Copy link

smfr commented Oct 18, 2024

cc @lukewarlow

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
test-change-proposal Proposal to add or remove tests for an interop area
Projects
None yet
Development

No branches or pull requests

2 participants