The Red O opera preservation

Developers › A Journal from Behind the Trenches

Is Internet Explorer 9 the most HTML5 compliant browser? Not quite.

November 2, 2010 · by Haavard

It's all over the web. IE9 is being labeled the HTML5 conformance champion, based on a set of official tests at the W3C.

PC World, for example, writes:

Internet Explorer 9 has topped all other browsers in conforming with the HTML5 specification, including Google Chrome and Firefox, according to tests carried out by the World Wide Web Consortium.

This would be quite amazing if it was true. But it isn't. You can't draw such conclusions based on those tests.

Webmonkey gets it right:

The reason IE9 tops the W3C’s list is that the test looks at only a fraction of the HTML5 spec. In other words, the test is very limited. Even better, it’s limited to things IE9 is good at.

The article is a recommended read for anyone who wants to understand what's actually going on here. Unfortunately, they are referring to the flawed html5test.com site (which doesn't test just HTML5, and which uses a strange way to assign scores), but that's a different story... Do read the article if you want to learn more about the test that is claimed to declare IE9 as the HTML5 leader.

There are also some comments from various browser vendors on the HTML test suite mailing list on the subject. For example, there seem to be bugs that affect the pass percentages, and the results for Opera are reportedly "partly bogus".

Needless to say, "IE9 wins at HTML5" is now all over the web. Unfortunately, I doubt that most of the articles that declare IE9 the "winner" are going to be updated to reflect the reality of the situation.

Yet another major PR win for Microsoft, then.

Readers' comments 19

Left under the original article, as archived.

  • lucideerTuesday, November 2, 2010 9:53:09 PM

    Worth pointing out also - Opera gets 0% in the SVG tests, despite having the highest SVG support of any browser. The reason is because the SVG tests don't use XHTML5 - there are at least 7 other tests in the suite that do use XHTML5 (all of which IE9 happens to get 100% in), but none of the SVG ones do, causing Opera's 0% score.

  • Charles SchlossTuesday, November 2, 2010 10:18:04 PM

    Microsoft likes to play the bull$hit card a lot with IE now

    I wonder how much they paid :wink:

  • Rafael LuikTuesday, November 2, 2010 10:20:32 PM

    :up:
    Now a post about Skyfire??
    (
    http://twitter.com/opvard/status/21475364630
    http://twitter.com/opvard/status/21537960372
    http://twitter.com/opvard/status/21537224294
    http://twitter.com/opvard/status/22736373000
    http://twitter.com/opvard/status/29480312603
    )

    :bigsmile:

  • SeldaekTuesday, November 2, 2010 10:22:10 PM

    There is one thing that you have to give msft credit for though. As you say, most of the tests are theirs, but at least they submitted tests. Also the tests were up for review..

    So it's not like you (Opera) and the other browser vendors didn't see it coming. The test suite is a plus for interoperability in the grand scale of things, if you look beyond the PR noise, and I hope every vendor will contribute/keep contributing.

  • HaavardTuesday, November 2, 2010 10:37:27 PM

    Originally posted by rafaelluik:

    Now a post about Skyfire??

    I have tried to reason with them (including their CEO). There is no getting through to them. I think attention is exactly what they are after.

    Originally posted by Seldaek:

    There is one thing that you have to give msft credit for though. As you say, most of the tests are theirs, but at least they submitted tests. Also the tests were up for review..

    Making a more comprehensive set of tests takes a bit longer than, say, making a handful of tests just for the sake of PR.

  • lucideerTuesday, November 2, 2010 10:47:10 PM

    Originally posted by Haavard:

    Making a more comprehensive set of tests takes a bit longer than, say, making a handful of tests just for the sake of PR.


    Well I for one would welcome Opera producing a set of superficial tests that do well only in Opera purely for the sake of PR. 'Twould make a change...

  • mrdTuesday, November 2, 2010 11:58:34 PM

    lucideer. That would make Opera every bit as bad as the other dishonest attention hound browser vendors.

    I'm glad - as long as it does not cause actual harm to the company and product - that Opera plays it straight down the line.

  • kailapisWednesday, November 3, 2010 12:13:51 AM

    Well, I see it that since Microsoft was the one that made most of the tests, it fits that the tests showcase IE in the best way possible. Other browser vendors should submit more tests to the W3C for other parts of the HTML5 spec before complaining about the validity of the tests.

    I believe the tests and their results are valid for the most part, although the relevancy of them and the conclusion many articles made are questionable.

  • d4rkn1ghtWednesday, November 3, 2010 2:35:01 AM

    Isn't this the same thing Apple did with Safari?

  • Artur „Jurgi” JurgawkaWednesday, November 3, 2010 10:10:22 AM

    It looked unvelievable at the very beginning for me.

  • FransWednesday, November 3, 2010 10:14:19 AM

    Originally posted by d4rkn1ght:

    Isn't this the same thing Apple did with Safari?


    In a sense. But that was on Apple's marketing page, while this one is an upcoming official W3C test suite. The title, however, only said "Official HTML5 Test Suite Conformance Results." Luckily they've amended it now, and it has a big fat warning notice on there.

    Originally posted by http://test.w3.org/html/tests/reporting/report.htm:

    The HTML5 Test suite is still being developed. The number of tests and the results on these tests will change.

    The results in this document may be updated, replaced or obsoleted by others documents at any time.

    It is inappropriate to cite those results as other than work in progress and unstable.

  • HaavardWednesday, November 3, 2010 11:47:22 AM

    Originally posted by kailapis:

    Well, I see it that since Microsoft was the one that made most of the tests, it fits that the tests showcase IE in the best way possible. Other browser vendors should submit more tests to the W3C for other parts of the HTML5 spec before complaining about the validity of the tests.

    I addressed this in a previous comment:

    "Making a more comprehensive set of tests takes a bit longer than, say, making a handful of tests just for the sake of PR."

    The problem is not that there are tests, but that people are jumping to conclusions based on that limited set of tests.

  • SeldaekWednesday, November 3, 2010 12:09:38 PM

    @Haavard: Obviously it's a problem that people are misinformed, but rather than complaining about a PR stunt I wish that more energy was put into actually enhancing the test suite. Because sure at the moment it's incomplete, but you gotta start somewhere. As you said it's a huge piece of work and it's not going to happen overnight.

    Also, for once that IE gets better press than it deserves rather than people just bashing them no matter what kind of effort they put into their work to catch up, I think it's only fair to let them have this one.

  • kailapisWednesday, November 3, 2010 4:44:46 PM

    Originally posted by haavard:

    I addressed this in a previous comment:

    "Making a more comprehensive set of tests takes a bit longer than, say, making a handful of tests just for the sake of PR."


    The tests that Microsoft provided were used for PR, but it's not just for PR. It's also to have a test so that people can argue about the specifics of implementing the standards. Microsoft invested a lot of man-hours creating the tests for W3C, and I don't really see any other browser vendor doing the same. Also, as Seldaek said, it has to start somewhere.


    Originally posted by haavard:

    The problem is not that there are tests, but that people are jumping to conclusions based on that limited set of tests.


    Originally posted by kailapis:

    I believe the tests and their results are valid for the most part, although the relevancy of them and the conclusion many articles made are questionable.


  • Rafael LuikWednesday, November 3, 2010 6:58:13 PM

    Originally posted by haavard:

    I think attention is exactly what they are after.

    Oops! :p:

  • HaavardWednesday, November 3, 2010 9:07:09 PM

    Originally posted by Seldaek:

    rather than complaining about a PR stunt I wish that more energy was put into actually enhancing the test suite

    Originally posted by kailapis:

    Microsoft invested a lot of man-hours creating the tests for W3C, and I don't really see any other browser vendor doing the same.

    As mentioned, I already addressed this:

    "Making a more comprehensive set of tests takes a bit longer than, say, making a handful of tests just for the sake of PR."

    In other words, other browser vendors are investing far more in more comprehensive tests, rather than rushing a handful of tests out the door. The problem is that the test suite was made available to the public far too early. It was a mistake on W3C's part.

  • YangombiUmpakatiWednesday, November 3, 2010 9:51:25 PM

    thx for clarification!

  • Blaz(ž) PristavThursday, November 4, 2010 9:47:48 AM

    IE6 all over again?

    Then some sites will work best only with IE9, how far did they get with this crap?

  • Cutting SpoonThursday, November 4, 2010 10:02:03 PM

    Everyone has been submitting tests; I would not be greatly surprised to see a shakedown within the W3C's greater body after this one, though. A big problem is that, just like the initial launch of the IE Test Drive site, many tests RELY on IE-specific timing or even outright bugs!

    At least on Test Drive, some examples have been fixed to support more browsers, and some actually broke in IE9 when they were repaired. The page on W3C includes a number of tests submitted (by Microsoft especially), and many of them have not been altered or audited since their submission. The test page is fundamentally broken at a low level.

About this article

Published inA Journal from Behind the Trenches
DateNovember 2, 2010
AuthorHaavard
Tagshtml5, press, misleading, w3c, open standards, marketing, microsoft
OriginalInternet Archive

The Red O » Developers » Haavard » Is Internet Explorer 9 the most HTML5 compliant browser? Not quite.

Browsers

Add-ons

Services

Developers

Help

Company

About · Contribute · Legal theredo.org
The Red O is an independent, non-commercial preservation project, not affiliated with Opera Norway AS. Opera is a trademark of its owner.