WebGL 1.0.1 conformance testing, part 2

Last week, I asked people to participate in WebGL 1.0.1 conformance testing. The response has been amazing and I want to thank everyone for that.

The results can be seen in the archives of the dedicated Google Group. Thanks to these, we have identified and fixed a couple of bugs in Gecko, we have landed a couple of work-arounds for widespread driver issues, and we have even found and corrected an issue in a WebGL conformance test that was non-passable on color-managed systems!

So now is a great time to start a second round of mass testing.

Please follow the instructions on this page!

Firefox users: please use a Nightly build from today (2012-04-21) or newer. Results should be very good. Also, at this point, any test failure is very likely to be a driver bug worth reporting to your driver vendor.

While I’m only speaking with my Mozilla hat here, we are interested in results from all WebGL-capable browsers on desktop platforms. These include Chrome, Firefox, Opera and Safari, on Windows, Mac OSX and desktop Linux.

20 Responses to “WebGL 1.0.1 conformance testing, part 2”

  1. Lozzy says:

    full results sent. Here’s the summary;

    Win 7 x64, Firefox 2012-04-21, ATI Radeon Mobility HD 4530

    Previous failures (April 18):

    conformance/textures/tex-image-and-sub-image-2d-with-image.html: 8 tests failed
    conformance/textures/texture-size-cube-maps.html: 24 tests failed
    conformance/more/functions/copyTexImage2D.html: 1 tests failed

    Current failures:

    conformance/textures/texture-size-cube-maps.html: 24 tests failed
    conformance/more/functions/copyTexImage2D.html: 1 tests failed

  2. Sonickydon says:

    Is there some reasoning behind the fact that the results page report different numbers of tests run for Firefox, Chrome and opera (on the same machine and configuration, I get 8915 tests run on Firefox, 8916 on Chrome, and 8621 on Opera)?

    And now the fun facts:

    1) conformance/textures/texture-mips.html: 2 tests failed (Firefox and Chrome, 3 Fails for Opera)
    All browsers fail on this one, (with Opera failing slightly harder). Is that a good sign that it’s the driver’s bug?

    2)conformance/uniforms/uniform-location.html: 1 tests failed
    Both Firefox and Chrome fail on “contextA.getUniform(programS, locationSx) should be 3. Was 0″.
    Opera however doesn’t fail on it.
    How do we file a good bug on that?

    (Unfortunately Opera gets too unstable and forces me to restart Xserver so I can not send the test results for it)

  3. Brenno says:

    Should be also test native WebGL on Windows Firefox?

  4. bjacob says:

    Sure, testing on OpenGL drivers on Windows is also useful. In Firefox, go to about:config and set webgl.prefer-native-gl=true. This will require force-enabling on certain drivers: webgl.force-enabled=true.

  5. bjacob says:

    @Sonickydon: the reason for the different total numbers of tests is that different browsers/drivers allow for different sets of WebGL extensions. If your browser/driver supports more extensions, more WebGL tests are accordingly run.

    texture-mips.html is indeed a driver bug.

    uniform-location.html is also a driver bug. I guess Opera has a work-around for it. I wouldn’t bother about it.

  6. bjacob says:

    Note that 1.0.2 beta is really the development branch, not a ‘beta’. It’s completely in flux, and browser projects like Mozilla have not really started yet doing the work needed to pass certain of these new tests.

  7. alex_mayorga says:

    https://cvs.khronos.org/svn/repos/registry/trunk/public/webgl/conformance-suites/1.0.1/conformance/glsl/functions/glsl-function-normalize.html crashes on click with this configuration:

    Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:14.0) Gecko/20120423 Firefox/14.0a1 ID:20120423034053

    Adapter Description nouveau — Gallium 0.4 on NVA5
    Vendor ID nouveau
    Device ID Gallium 0.4 on NVA5
    Driver Version 2.1 Mesa 8.0.2
    WebGL Renderer nouveau — Gallium 0.4 on NVA5 — 2.1 Mesa 8.0.2
    GPU Accelerated Windows 0
    AzureBackend skia

    Where should the bug be filled?

  8. alex_mayorga says:

    Thanks! Filed https://bugs.freedesktop.org/show_bug.cgi?id=49086 FWIW.

    The rest of reds I see are listed below.
    Anything I need to do on these?

    conformance/canvas/buffer-offscreen-test.html (8 of 9 passed)
    failed: remainder of buffer should be cleared

    conformance/canvas/buffer-preserve-test.html (4 of 5 passed)
    failed: remainder of buffer should be cleared

    conformance/programs/program-test.html (60 of 61 passed)
    failed: getError expected: NO_ERROR. Was INVALID_OPERATION : drawing with a valid program shouldn’t generate a GL error

    conformance/textures/texture-mips.html (17 of 19 passed)
    failed: texture that is only using the smallest 2 mips should draw with green
    failed: texture that is only using smallest mips should draw with cyan

  9. bjacob says:

    alex_mayorga: Thanks for filing that bug. The buffer-offscreen-test.html and buffer-preserve-test.html failures typically happen when the canvas is not visible: make sure you don’t scroll, don’t switch tabs and don’t use a too small window size while running webgl conformance tests. The program-test.html failure is currently being discussed on mailing lists. The texture-mips.html failures seems worth reporting on the same bug tracker.

  10. Finn says:

    Firefox 14.0a1 (2012-04-23), Windows 7 64bit, NVIDIA 301.24 Beta, GeForce GTX 460

    Perfect pass rate without native gl, with native gl I get 12 failures

    when i sent the email, I forgot to mention native gl was set on. I can’t see it in the graphics menu of about:support, shouldn’t it be there? (actually just noticed one of the lines changes, so no worries.)

    fwiw, here’s the failing test:

    conformance/textures/tex-image-and-sub-image-2d-with-array-buffer-view.html (182 of 194 passed)

    failed: shouldBe 255,0,0,255
    failed: shouldBe 0,255,0,255
    failed: shouldBe 255,0,0,255
    failed: shouldBe 0,0,0,255
    failed: shouldBe 255,0,0,255
    failed: shouldBe 0,255,0,255
    failed: shouldBe 255,0,0,255
    failed: shouldBe 0,0,0,255
    failed: shouldBe 255,0,0,255
    failed: shouldBe 0,255,0,255
    failed: shouldBe 255,0,0,255
    failed: shouldBe 0,0,0,255

  11. bjacob says:

    “Native OpenGL” is implied when the “WebGL renderer” line in about:support does not mention ANGLE.

  12. AV says:

    sent also for gma 3150
    would you like fennec tests? arm mali 400 on galaxy s 2

  13. AV says:

    I don’t know if related but it doesn’t have the OES_texture_float extension, many demos doesn’t run and I don’t know if required for the conformance.

  14. bjacob says:

    AV: we’re only interested in desktop for now, fennec is not ready for this kind of testing. OES_texture_float is an extension, it’s not required for conformance, but yes, some advanced demos require it.

  15. Chiana says:

    OS Description Microsoft Windows 7 SP1 64bit
    CPU Description QuadCore Intel Core I7 860 @ 2.80 Ghz
    GPU Description AMD Radeon HD 6870
    Driver Version 8.960.0.0
    Driver Date 3-15-2012
    WebGL Renderer Google Inc. — ANGLE (AMD Radeon HD 6800 Series) — OpenGL ES 2.0 (ANGLE 1.0.0.1041)

    Nightly 15.0a1 64bit (HW on(Direct2D/Direct3D10), WebGL on)
    http://pastebin.mozilla.org/1600589

    Opera 12 RC 64bit ((HW on (Direct3D10), WebGL on))
    http://pastebin.mozilla.org/1600606

  16. Chiana says:

    >Opera 12 RC 64bit ((HW on (Direct3D10), WebGL on))
    >http://pastebin.mozilla.org/1600606
    Sorry, incorrect data here/
    Correct is:
    http://pastebin.mozilla.org/1600627

  17. Dan says:

    I only got 1 failure when testing with:

    Firefox 15.0a1 Nightly (2012-04-28)
    NVIDIA 296.10, Geforce 6200
    Windows XP 32-bit

    However, I got a crash fairly quickly when testing with “webgl.prefer-native-gl” set to true: https://crash-stats.mozilla.com/report/index/bp-c9f9d4f8-df99-4ce2-9113-5d93a2120429

Leave a Reply