Last week, I asked people to participate in WebGL 1.0.1 conformance testing. The response has been amazing and I want to thank everyone for that.
The results can be seen in the archives of the dedicated Google Group. Thanks to these, we have identified and fixed a couple of bugs in Gecko, we have landed a couple of work-arounds for widespread driver issues, and we have even found and corrected an issue in a WebGL conformance test that was non-passable on color-managed systems!
So now is a great time to start a second round of mass testing.
Please follow the instructions on this page!
Firefox users: please use a Nightly build from today (2012-04-21) or newer. Results should be very good. Also, at this point, any test failure is very likely to be a driver bug worth reporting to your driver vendor.
While I’m only speaking with my Mozilla hat here, we are interested in results from all WebGL-capable browsers on desktop platforms. These include Chrome, Firefox, Opera and Safari, on Windows, Mac OSX and desktop Linux.
full results sent. Here’s the summary;
Win 7 x64, Firefox 2012-04-21, ATI Radeon Mobility HD 4530
Previous failures (April 18):
conformance/textures/tex-image-and-sub-image-2d-with-image.html: 8 tests failed
conformance/textures/texture-size-cube-maps.html: 24 tests failed
conformance/more/functions/copyTexImage2D.html: 1 tests failed
Current failures:
conformance/textures/texture-size-cube-maps.html: 24 tests failed
conformance/more/functions/copyTexImage2D.html: 1 tests failed
Is there some reasoning behind the fact that the results page report different numbers of tests run for Firefox, Chrome and opera (on the same machine and configuration, I get 8915 tests run on Firefox, 8916 on Chrome, and 8621 on Opera)?
And now the fun facts:
1) conformance/textures/texture-mips.html: 2 tests failed (Firefox and Chrome, 3 Fails for Opera)
All browsers fail on this one, (with Opera failing slightly harder). Is that a good sign that it’s the driver’s bug?
2)conformance/uniforms/uniform-location.html: 1 tests failed
Both Firefox and Chrome fail on “contextA.getUniform(programS, locationSx) should be 3. Was 0″.
Opera however doesn’t fail on it.
How do we file a good bug on that?
(Unfortunately Opera gets too unstable and forces me to restart Xserver so I can not send the test results for it)
Should be also test native WebGL on Windows Firefox?
Sure, testing on OpenGL drivers on Windows is also useful. In Firefox, go to about:config and set webgl.prefer-native-gl=true. This will require force-enabling on certain drivers: webgl.force-enabled=true.
@Sonickydon: the reason for the different total numbers of tests is that different browsers/drivers allow for different sets of WebGL extensions. If your browser/driver supports more extensions, more WebGL tests are accordingly run.
texture-mips.html is indeed a driver bug.
uniform-location.html is also a driver bug. I guess Opera has a work-around for it. I wouldn’t bother about it.
Hey, why not use the 1.0.2 Beta WebGL conformance test?
https://cvs.khronos.org/svn/repos/registry/trunk/public/webgl/sdk/tests/webgl-conformance-tests.html
Note that 1.0.2 beta is really the development branch, not a ‘beta’. It’s completely in flux, and browser projects like Mozilla have not really started yet doing the work needed to pass certain of these new tests.
https://cvs.khronos.org/svn/repos/registry/trunk/public/webgl/conformance-suites/1.0.1/conformance/glsl/functions/glsl-function-normalize.html crashes on click with this configuration:
Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:14.0) Gecko/20120423 Firefox/14.0a1 ID:20120423034053
Adapter Description nouveau — Gallium 0.4 on NVA5
Vendor ID nouveau
Device ID Gallium 0.4 on NVA5
Driver Version 2.1 Mesa 8.0.2
WebGL Renderer nouveau — Gallium 0.4 on NVA5 — 2.1 Mesa 8.0.2
GPU Accelerated Windows 0
AzureBackend skia
Where should the bug be filled?
alex_mayorga: Please report this to https://bugs.freedesktop.org/enter_bug.cgi?product=Mesa
Thanks! Filed https://bugs.freedesktop.org/show_bug.cgi?id=49086 FWIW.
The rest of reds I see are listed below.
Anything I need to do on these?
conformance/canvas/buffer-offscreen-test.html (8 of 9 passed)
failed: remainder of buffer should be cleared
conformance/canvas/buffer-preserve-test.html (4 of 5 passed)
failed: remainder of buffer should be cleared
conformance/programs/program-test.html (60 of 61 passed)
failed: getError expected: NO_ERROR. Was INVALID_OPERATION : drawing with a valid program shouldn’t generate a GL error
conformance/textures/texture-mips.html (17 of 19 passed)
failed: texture that is only using the smallest 2 mips should draw with green
failed: texture that is only using smallest mips should draw with cyan
alex_mayorga: Thanks for filing that bug. The buffer-offscreen-test.html and buffer-preserve-test.html failures typically happen when the canvas is not visible: make sure you don’t scroll, don’t switch tabs and don’t use a too small window size while running webgl conformance tests. The program-test.html failure is currently being discussed on mailing lists. The texture-mips.html failures seems worth reporting on the same bug tracker.
Firefox 14.0a1 (2012-04-23), Windows 7 64bit, NVIDIA 301.24 Beta, GeForce GTX 460
Perfect pass rate without native gl, with native gl I get 12 failures
when i sent the email, I forgot to mention native gl was set on. I can’t see it in the graphics menu of about:support, shouldn’t it be there? (actually just noticed one of the lines changes, so no worries.)
fwiw, here’s the failing test:
conformance/textures/tex-image-and-sub-image-2d-with-array-buffer-view.html (182 of 194 passed)
failed: shouldBe 255,0,0,255
failed: shouldBe 0,255,0,255
failed: shouldBe 255,0,0,255
failed: shouldBe 0,0,0,255
failed: shouldBe 255,0,0,255
failed: shouldBe 0,255,0,255
failed: shouldBe 255,0,0,255
failed: shouldBe 0,0,0,255
failed: shouldBe 255,0,0,255
failed: shouldBe 0,255,0,255
failed: shouldBe 255,0,0,255
failed: shouldBe 0,0,0,255
“Native OpenGL” is implied when the “WebGL renderer” line in about:support does not mention ANGLE.
sent also for gma 3150
would you like fennec tests? arm mali 400 on galaxy s 2
I don’t know if related but it doesn’t have the OES_texture_float extension, many demos doesn’t run and I don’t know if required for the conformance.
AV: we’re only interested in desktop for now, fennec is not ready for this kind of testing. OES_texture_float is an extension, it’s not required for conformance, but yes, some advanced demos require it.
OS Description Microsoft Windows 7 SP1 64bit
CPU Description QuadCore Intel Core I7 860 @ 2.80 Ghz
GPU Description AMD Radeon HD 6870
Driver Version 8.960.0.0
Driver Date 3-15-2012
WebGL Renderer Google Inc. — ANGLE (AMD Radeon HD 6800 Series) — OpenGL ES 2.0 (ANGLE 1.0.0.1041)
Nightly 15.0a1 64bit (HW on(Direct2D/Direct3D10), WebGL on)
http://pastebin.mozilla.org/1600589
Opera 12 RC 64bit ((HW on (Direct3D10), WebGL on))
http://pastebin.mozilla.org/1600606
>Opera 12 RC 64bit ((HW on (Direct3D10), WebGL on))
>http://pastebin.mozilla.org/1600606
Sorry, incorrect data here/
Correct is:
http://pastebin.mozilla.org/1600627
I only got 1 failure when testing with:
Firefox 15.0a1 Nightly (2012-04-28)
NVIDIA 296.10, Geforce 6200
Windows XP 32-bit
However, I got a crash fairly quickly when testing with “webgl.prefer-native-gl” set to true: https://crash-stats.mozilla.com/report/index/bp-c9f9d4f8-df99-4ce2-9113-5d93a2120429
Hello, I hope my reports are helpful:
https://groups.google.com/group/webgl-driver-testing/t/ca10d77e129d2ab7 (crash)
https://groups.google.com/group/webgl-driver-testing/t/a9b8e42a4ff0d1ee
https://groups.google.com/group/webgl-driver-testing/browse_thread/thread/8484d6ab922acb7c
https://groups.google.com/group/webgl-driver-testing/browse_thread/thread/ac0d733470762ce2
https://groups.google.com/group/webgl-driver-testing/browse_thread/thread/e5d43a68ebad7477
https://groups.google.com/group/webgl-driver-testing/browse_thread/thread/61c8bb20956bd953