The Red O opera preservation

Developers › A Journal from Behind the Trenches

Some clarifications regarding Tom's Hardware's browser tests

July 13, 2010 · by Haavard

Tom's Hardware has posted a rather thorough test of the top five browsers.

Opera is declared the speed winner, which is great, but there are also some questions popping up regarding a couple of the tests.

First, the slow loading of local pages is a cache bug which only affects local pages directly from disk (Opera always re-decodes images when loaded from disk). I'm sure it's something we can fix fairly easily, but in the meantime it's possible to try loading pages from a local web server instead as a workaround.

The second issue is Mozilla's Dromaeo JavaScript test, where Opera is disqualified because it's so far ahead of the rest. From what I can gather, we are not doing anything wrong there. The test apparently uses empty loops that don't produce any data, and Carakan simply optimizes that away.

Finally, the memory tests may not tell the full story. Opera will run fine on systems that are low on memory, but will also use more memory when available in order to increase performance. Since their test system has 4GB RAM, one would expect applications to take advantage of that. Opera runs on a wide range of devices, from mobile phones to high-end PCs, so it has been designed to adapt its memory usage to the environment.

Perhaps their next test could take a look at how the different browsers handle a larger selection of environments with different amounts of memory?

Readers' comments 43

Left under the original article, as archived.

  • Pallab DeTuesday, July 13, 2010 4:50:39 PM

    Thanks for the clarification. I was wondering why Opera performed so miserably in the page load tests.

  • Daniel HendrycksTuesday, July 13, 2010 4:51:11 PM

    For the standards test they should have included SVG.

  • SavagemanTuesday, July 13, 2010 4:53:46 PM

    Thanks, that anwser my question from Twitter.

    Good to see Opera always on top! :cool:

  • CyroTuesday, July 13, 2010 5:10:22 PM

    Hey Haavard, nice arguments.
    I have something to add concerning memory usage.
    There is a company I visit on Saturdays (I give them a hand) that always faced this issue with Opera. They have an Ubuntu 9.10 with LTSP. What really happens is that, if all users share the same server and consequentially share the same RAM with their Opera instances, this interferes with their server performance.
    They dislike Opera because of it, their feeling is like one user compromises the others so they have to stick with Firefox or Chromium. This is definitely a browser deal, it doesn't happen with other software like the OpenOffice suite, for example.
    My only intention with this comment is to say the position Opera Software has taken concerning memory usage is not applicable to all cases. And sometimes, it can be harmful and cause a bad impression.

    I imagine there are ways to change it via opera:config but they are confusing (disabling automatic ram cache or setting memory size).

  • HaavardTuesday, July 13, 2010 5:19:28 PM

    I can't really comment on specific cases with special setups, and this isn't the right place for bug reports. If you are having problems under specific circumstances, I would recommend that you report them through the proper channels.

    I was commenting on testing of memory usage in general, for example like the one in the tests by Tom's Hardware.

    It's important to keep in mind that the memory usage strategies used by different browsers may have different goals and purposes. Opera is designed to dynamically adapt the memory usage to the system, for example. Other browsers may have more crude methods and simply impose hard limits, which could also lead to a performance penalty.

  • ouzowtfTuesday, July 13, 2010 5:27:56 PM

    I'm looking forward to seeing the cache bug affecting local pages fixed :wink:

  • .edTuesday, July 13, 2010 5:36:53 PM

    So, basically, Opera won, and could have reported even better.

  • Remco LantingTuesday, July 13, 2010 6:26:53 PM

    Instead of just looking at how much memory each browser used, we also focused on how well that memory was managed.

    What's missing there is (at least) two points:

    1) What do browsers actually keep in memory (and do with it).
    2) Do browsers give back the memory when another application needs it

    Testing might be hard, but the current test is skewed in favour of the browsers that don't do any memory management and just give it back immediately. And that was not the point of the test.

  • Charles SchlossTuesday, July 13, 2010 7:33:30 PM

    Great to know,

    I have noticed Opera does better on the browser test when the computer that Opera runs on has faster cpu and more ram so this makes since

    Edit:

    http://www.tomshardware.com/reviews/opera-10.60-internet-explorer-9-safari-5,2680-4.html

    They can't really use facebook.com for page loading speeds as it is full of over 1000 validation errors, so it depends on how browsers handle the bad coding, I do admit Opera is great with the browser js and other browsers have some fixes also for broken pages

  • PengeTuesday, July 13, 2010 8:36:29 PM

    Originally posted by haavard:

    Other browsers may have more crude methods and simply impose hard limits, which could also lead to a performance penalty.


    Opera also have. I must increase the limit in opera:config#Cache|Document, opera:config#Cache|ECMAScript and opera:config#Cache|Figure, because the Fast History Navigation Mode didn't work properly after I reach the automatic RAM cache limit, but the memory usage percent (opera:memdebug) continued to increase, after I reach of the 100% of the limit.

    Despite that I've increased the RAM cache limits and turned off the Automatic RAM Cache, I didn't noticed nothing negative impact on the memory usage.

  • Cutting SpoonTuesday, July 13, 2010 10:12:43 PM

    Yeah, something about minimizing or setting-aside the browser and working in another program would be good. Or artificially limiting system RAM to as little as 1GB (like the average netbook).

    For the person with a Linux question, it's possible to roll a profile fix for all current users, and to set a fix that will run when a new user account is created. Same as policy-setting tools for Windows. If the guy maintaining their box just took a few minutes online, he could whip up a fix with no trouble at all.

  • DillonWednesday, July 14, 2010 12:17:20 AM

    Holy cow I just checked that opera:memdebug page and my Disk Cache is at 1166% of the max (which is 20 MB)! I just made a bug report through the reporter, though I'd be glad to hear if anyone has a workaround.

  • Artur „Jurgi” JurgawkaWednesday, July 14, 2010 12:55:02 AM

    Opera will run fine on systems that are low on memory, but will also use more memory when available in order to increase performance.



    But why Opera does not give taken memory back, when pages are closed? Each opened page takes some memory, after 2-3 hours Opera take 0,5 GB, even with all tabs closed. I have to restart it several times a day.
    Old Operas, like 8, were running for months without restart. This memory leakage in 10 really sucks.

  • AvolaWednesday, July 14, 2010 1:02:37 AM

    Originally posted by Astrophizz:

    Holy cow I just checked that opera:memdebug page and my Disk Cache is at 1166% of the max (which is 20 MB)!


    Yeah I've noticed that Opera uses a lot of hard drive space when left open for a while (cache is set to 50 MB). I have less than 1 GB of space left on my Windows/programs partition, so after a day or so of browsing I have to clear the cache (restarting Opera also clears the cache even though I don't have that option checked off :confused: ) Pretty annoying. I guess I could make my partition larger, but I'm too lazy and mildy afraid I'll mess something up.

  • prd3Wednesday, July 14, 2010 1:39:41 AM

    Originally posted by mgillespie:

    As usual with Opera, everything turns into a PR Disaster with them. They could have fixed this simple bug and Opera would have trounced everything in those benchmarks (on a very high profile American site).


    As usuall, you are trolling and being an ass. Opera dominated the performance test. The message everyone is taking home is that Opera is the fastest.

    Stop whining about "PR disaster" every time some site writes good things about Opera.

    Opera: Fantastic browser, lousy marketing.


    Nice contradiction there. The unfixed bug is related to the development of the browser, not the marketing.

    It's completely idiotic to claim that this is about PR. It's about a bug which is extremely unimportant overall. It's only getting highlighted because of a browser speed test where Opera devastates everything else.

    Now go away and stop trolling this blog every single time a post is made.

  • prd3Wednesday, July 14, 2010 1:41:48 AM

    Originally posted by Jurgi:

    But why Opera does not give taken memory back, when pages are closed?


    Because you can reopen tabs and they will open instantly and without reloading. That's because Opera caches them in memory. They will be removed eventually, but you are basically asking Opera to use less memory and be slower.

    Fail.

    Each opened page takes some memory, after 2-3 hours Opera take 0,5 GB, even with all tabs closed. I have to restart it several times a day.


    No such problems here.

    Old Operas, like 8, were running for months without restart.


    Nope. Some people reported the same problems there. Depends on your system. Stop making sweeping remarks based on your own experiences.

  • PengeWednesday, July 14, 2010 1:59:56 AM

    Originally posted by Jurgi:

    after 2-3 hours Opera take 0,5 GB, even with all tabs closed. I have to restart it several times a day.


    You don't have to. You just should close to the taskbar for a while.

    Originally posted by Jurgi:

    Old Operas, like 8, were running for months without restart.


    The websites became more complex since 2005, the Opera became more faster since 8.0 and the Working Set was the default setting in the Windows XP taskmanager, however in the Vista and above the Private Bytes is the default.

    ps: The plugins (mostly the Flash) maybe cause memory leak, but this independent of the browser.

  • DillonWednesday, July 14, 2010 2:33:57 AM

    Originally posted by prd3:

    Because you can reopen tabs and they will open instantly and without reloading. That's because Opera caches them in memory. They will be removed eventually, but you are basically asking Opera to use less memory and be slower.


    Well, except that that memory isn't released even when the "trash" is emptied.

    Sigh, Windows just popped up a window informing me that it's running low on virtual memory. Opera is using 1.4 GB, more than 10x anything else I'm running. :sad:

  • Cutting SpoonWednesday, July 14, 2010 3:51:04 AM

    I have found that minimizing Opera, putting the PC on standby or hibernate can fix the issue without reopening many tabs. I first noticed when Opera ate up lots of space when torrenting linux iso's.

  • Daniel HendrycksWednesday, July 14, 2010 4:02:51 AM

    Originally posted by Cutting Spoon:

    I first noticed when Opera ate up lots of space when torrenting linux iso's.


    Confirmed, everytime I do that.

  • Sérgio CapelaWednesday, July 14, 2010 8:12:12 AM

    Didn't know opera:memdebug. There are two values below that make me worry:

    used max percent
    Documents 523700 kB 50000 kB 1047
    Images 49982 kB 50000 kB 99
    EcmaScript 268299 kB 2000 kB 13414
    Disk Cache 20940 kB 20000 kB 104
    RAM Cache 1746 kB 1024 kB 170


    13414% for EcmaScript???
    1047% for Documents

  • Artur „Jurgi” JurgawkaWednesday, July 14, 2010 8:19:11 AM

    Originally posted by prd3:

    Because you can reopen tabs and they will open instantly and without reloading. That's because Opera caches them in memory. They will be removed eventually, but you are basically asking Opera to use less memory and be slower.


    Nonsense, as Dillon said.
    Moreover, „trash” isn't new feature. Why storying hundreds of closed tabs in old Operas didn't take hundreds of MB?

    Originally posted by prd3:

    but you are basically asking Opera to use less memory and be slower.


    No. I'm basically asking Opera to be _really_ faster in real life, not only in benchmarks. Do you really think, that taking 100% of memory speeds up the work? Even with half of it Opera „thinks” several seconds before opening new, empty tab.

    Originally posted by Penge4:

    You don't have to. You just should close to the taskbar for a while.


    It doesn't help.

    Originally posted by Penge4:

    The plugins (mostly the Flash) maybe cause memory leak, but this independent of the browser.


    But in this case the issue would be 1) generally with all Operas, 2) also with other browsers.

    Originally posted by prd3:

    Depends on your system. Stop making sweeping remarks based on your own experiences.


    Not only my own, but also many other people raport this issue. Stop denying the facts.

  • prd3Wednesday, July 14, 2010 8:22:19 AM

    Originally posted by Jurgi:

    Nonsense, as Dillon said.


    No, you are talking crap as always.

    Moreover, „trash” isn't new feature. Why storying hundreds of closed tabs in old Operas didn't take hundreds of MB?


    It did.

    Originally posted by Jurgi:

    No. I'm basically asking Opera to be _really_ faster in real life, not only in benchmarks.


    It already is.

    Originally posted by Jurgi:

    Not only my own, but also many other people raport this issue.


    Nope. Very few do, actually, compared to Opera's total user base. People with problems are much more likely to comment.

    But stop hijacking the comments. This is about the Toms Hardware article, not your inability to stop whining everywhere about nonsense.

  • SavagemanWednesday, July 14, 2010 10:07:30 AM

    @prd3 you're feeding it... And it's also annoying to read your comments of "I know everything". Also the memory discussion is related to Toms Hardware article and even more to this blog article.

    We all love Opera here, but I think we can safely assume Opera could manage his memory better sometimes.

  • HaavardWednesday, July 14, 2010 11:18:40 AM

    Please stop the off-topic discussions now. If you want to discuss specific bugs, take it to the forums.

    Originally posted by mgillespie:

    As usual with Opera, everything turns into a PR Disaster with them. They could have fixed this simple bug and Opera would have trounced everything in those benchmarks (on a very high profile American site).

    Not being able to predict the future leads to yet another PR disaster for poor Opera. We'll be sure to adjust our crystal balls, and hire more clairvoyant people! We should have foreseen that someone would do this particular test. We should probably also set PR people to not do PR, and instead manage bug priorities internally. Because that's a job for PR.

    On a more serious note, this article is a major PR win for Opera. We're listed as the fastest browser, and that's the message people are taking home.

  • Keiv M.Wednesday, July 14, 2010 12:59:45 PM

    Memory issue is interesting, i will open 70+ tabs from a bookmark folder all at once (using folder nickname) and it will all open and the memory usage while heavy isnt that bad.

    but open 10 of those pages in chrome, especially the flash based pages, and memory gets eaten up more than opera and the cpu usage hits 100% making the pc unusable. usually at this time the flash plugin becomes unresponsive and chrome kills it. Buuut, flash wont work on any other page until i close chrome and start it again.

    So if opera doesn't release the memory it uses right away even though it may be a lot, i can live with it as it doesnt bring my system to its knees.

  • Cutting SpoonWednesday, July 14, 2010 5:28:06 PM

    Ironically, Memdebug tells me that both 10.10 and 10.60 are running great! And I haven't closed either of them in several days.

    As for the testing methods, perhaps whoever spoke with that Betanews guy should get in contact with the reviewer. The issue of manpower for testing is definitely a problem, yet I really hope that Tom's Hardware (or someone out there) will have a roundup for netbook users. (of which there are millions of us)

  • AvolaWednesday, July 14, 2010 8:27:22 PM

    Originally posted by mgillespie:

    I truely hope it is, but look at the comments, all the Chrome fanboys are now saying benchmarks don't matter anymore, their new spin is real-world browsing that matters


    Much like the Opera fanboys did before 10.50 came out..

  • endless loveWednesday, July 14, 2010 10:49:15 PM

    :up:

  • HaavardThursday, July 15, 2010 8:31:50 AM

    Originally posted by mgillespie:

    All because of a stupid cache bug

    The cache bug doesn't really matter in the end, because the message people are taking home is that Opera is the fastest. You seem to be worrying too much about things that aren't really relevant, and you are blaming PR for something they have no control over.

  • praetor87Thursday, July 15, 2010 10:46:10 AM

    "Much like the Opera fanboys did before 10.50 came out.."

    It's true. I use 10.10 and 9.64 and they aren't too slow compared to the 10.60 on an average (not gmail or on some exploding javascript design site madness) sites. I can say, the benchmarks nowaday isn't measure the real life situations, but in the future the javascript engine will be more important so i'm happy because the Opera have faster javascript engine than the toy browsers.

  • HaavardThursday, July 15, 2010 3:32:54 PM

    If you think this article is a disaster for Opera, your priorities are a bit off, I think. Apart from the bug being a technical issue rather than a PR issue, the test is a great win for Opera, and confirms Opera's position as the fastest browser.

    Android is completely off-topic. Kindly refrain from hijacking my blog posts.

    Anyway, the PR discussion ends here. This is not about PR, and I'll let our PR people deal with that part. I feel like I'm being trolled...

  • LuchioThursday, July 15, 2010 3:42:57 PM

    Can anyone tell me if these numbers are normal?

    Documents  386132 kB  20000 kB  1930%
Images     11946 kB   20000 kB  59%
EcmaScript 177783 kB  2000 kB   8889%
Disk Cache 20884 kB   20000 kB  104%
RAM Cache  1514 kB    1024 kB   147%

    386 MB of documents seems a lot! Why aren't the limits being respected?

  • HaavardThursday, July 15, 2010 3:43:37 PM

    Luchio, please take such discussions elsewhere.

  • LuchioThursday, July 15, 2010 5:33:47 PM

    Gee... I do think I was being on-topic... The original post discusses memory usage of tom's hardware test, I'm just trying to understand HOW Opera handles memory internally, and if I should consider these numbers as something a normal user would experience.

    Sorry...

  • HaavardThursday, July 15, 2010 8:37:15 PM

    Unfortunately, this is not the right place for detailed technical discussions.

  • Cutting SpoonFriday, July 16, 2010 12:27:00 AM

    I am most pleased with the 8-page live loading number for Opera. I often leave many tabs open, and make much use of session management. These numbers confirm my feeling that Opera can resume a large session much more smoothly than other browsers, and confirms Haavard's assertion that localhost issues are unrelated to actual performance.

  • Constantine VesnaFriday, July 16, 2010 4:25:48 AM

    offtopic, thanx for "opera:memdebug", it has lots of interesting info.

    As for memory management - Opera is the ONLY browser, who comes close to my needs (100-150 tabs per browser), and while not releasing memory is an annoying, but other browsers are not capable of doing that altogether.

    As for "real-life" speed, which should be better named "perceived speed" - you can read google's own findings, how people measure "slow" and "fast", and how loosely its connected to actual application speed.

  • Stephen TurrellSaturday, July 17, 2010 2:28:50 PM

    Originally posted by prd3:

    Stop making sweeping remarks based on your own experiences.


    Had to laugh when i read this from the master of sweeping remarks based on his limited experience.

    It was very odd that Opera was slowest on the page loading test when it was so fast Starting with 8 tabs.
    And the memory test with 40 tabs Chrome & Safari had problems loading Pages.
    "Side Note: Proper Page Loads

    During the 40-tab test, we noticed how each of the browsers performed when completely loading and properly rendering Web pages. Chrome had trouble opening all 40 tabs correctly. A handful of the last few tabs did not load at all. And about half of those that loaded needed refreshing for missing or broken elements. Firefox, IE, and Opera did not have nearly as many broken pages, just a few here and there between the three iterations. Obviously, we had a heck of a time with Safari."

    Thanks for the explanation Haavard :smile:

  • Carlos Manuel Juaristi ManriqueThursday, August 26, 2010 4:33:13 AM

    you know, about the browsers around and the performance. I have a very interesting anecdote about it. I was a common windows user, running IE as most of the people out there. I change to firefox almost 2 years ago and y use to love it it was pretty much similar to IE but the add-ons give sort of a plus. Yesterday i received my (sorry im not a native english speaker so some thing may been miss understood i apologize for that) google chrome fan laptop sticker. and yes y was a great fan of chrome one of the biggest ones, completely in love. But guess what i download and install opera from the download.com and i was surprised. Opera was just miles better than chrome, i refuse to admit it at first but i had used those 4 browser for a good time and i dont care about benchmarks in real life opera is just better in performance that i can tell. i received the sticker by mail yesterday, too late, i knew opera before it arrived. But hey opera is not only the fastest its also the most complete and featured, and the services like unite, link, mail, link. Hard to beat that

  • Carlos Manuel Juaristi ManriqueThursday, August 26, 2010 4:42:56 AM

    The reason i downloaded opera was just curiosity, but i didn't imagine the software to be that good, never expected that. Google and windows are software and web monsters and they promote their software and invest lots of money in them. I sowed opera to my friends in the clinic (i'm a physician) all of them become fans opera just need some more promotion thats all. And the speed is one of the great thing of opera but not the only one. Chrome is fast and pretty much just fast. Why not to compare the browsers in many other aspects, like privacy, safety, and services.

  • Daniel HendrycksFriday, August 27, 2010 10:17:08 PM

    @Carlos Manuel Juaristi Manrique
    :up:

  • Cutting SpoonThursday, September 16, 2010 2:01:41 AM

    +1

About this article

Published inA Journal from Behind the Trenches
DateJuly 13, 2010
AuthorHaavard
Tagsopera, tom's hardware, dromaeo, performance, carakan, browsers
OriginalInternet Archive

The Red O » Developers » Haavard » Some clarifications regarding Tom's Hardware's browser tests

Browsers

Add-ons

Services

Developers

Help

Company

About · Contribute · Legal theredo.org
The Red O is an independent, non-commercial preservation project, not affiliated with Opera Norway AS. Opera is a trademark of its owner.