Mozilla Security Blog
Categories: CA Program Firefox Security

Comodo Certificate Issue – Follow Up

Johnathan Nightingale

30 responses

This is a follow-up to the previous Mozilla report about the fraudulent certificates issued by Comodo last week.

On 15th March 2011, a RA partner of the Comodo CA suffered an internal security breach (Comodo incident report). The attacker used the RA’s account with Comodo to cause 9 fraudulent certificates to be issued.

The domain names of the certificates were as follows:

  • addons.mozilla.org
  • login.live.com
  • mail.google.com
  • www.google.com
  • login.yahoo.com (x3)
  • login.skype.com
  • global trustee

These certificates were issued between 6 and 8pm on 15th March from the UTN-UserFirst-Hardware root, without the use of an intermediate certificate. The attack was discovered almost immediately by an internal Comodo check, and the certificates were revoked (via both CRL and OCSP). The compromised RA account has been suspended.

So far, according to Comodo, the only OCSP activity seen relating to these certificates has been two hits, one from the attacker’s IP and one from another very close by, plus hits from testing by Comodo and the notified companies. This suggests that the certificates have not been deployed in an attack, though it is possible that the attackers would block OCSP requests as well.

The IP address 212.95.136.18 appeared to be associated with the attack. According to Comodo, this IP address is an ADSL connection in Iran. From the OCSP traffic mentioned above, we think the attacker is aware that the certificates have been revoked.

Risk Analysis

Given the above, we do not believe that there has been a root key compromise. Nevertheless, an attacker armed with these fraudulent certificates and an ability to control their victim’s network could impersonate the sites in a way that would be undetectable to most users.

With particular regard to Mozilla, the certificate for ‘addons.mozilla.org’ would allow the attacker to imitate the addons.mozilla.org website, and potentially deceive users into downloading malicious software. However, they would not be able to interfere with automatic updates, either to Firefox or to addons. These mechanisms use different domain names, and updates to Firefox also do additional checks to match the certificate issuer to the expected value (which is not UTN-UserFirst-Hardware).

Mitigating Risk

On being informed of this issue by Comodo at 9.47pm GMT on 16th March, Mozilla considered a number of technical avenues. Although Comodo’s revocation is a significant mitigating step, we thought that additional measures made sense and eventually decided to hard-code a blacklist of the certificate serial numbers into Firefox. We therefore produced RC2 of Firefox 4 (released as Firefox 4 final on 22nd March), with two additional code patches (1, 2). These patches disable these specific certificates, plus one additional certificate issued to us by Comodo for testing, making a total of 10. These fixes were also included in updates to Firefox 3.5 and 3.6, also released on 22nd March. As soon as all the patched versions were released, we made a release announcement with some details of the problem.

Mozilla did not publish the information we received prior to shipping a patch. In early discussions, we were concerned that any indication that we knew about the attack would lead to attackers blocking our security updates as well. We also recognized that the obvious mitigation advice we might offer (to change Firefox’s security preferences to require a valid OCSP response in all cases, or to remove trust from Comodo’s certificates, or both) risked causing a significant portion of the legitimate web to break as well.

Additionally, neither we nor Comodo have found any evidence of access to their OCSP responder being blocked, either in Iran or anywhere else. We have also found no evidence of any other sort of attack.

In hindsight, while it was made in good faith, this was the wrong decision. We should have informed web users more quickly about the threat and the potential mitigations as well as their side-effects.

Mozilla has requested that Comodo do the following:

  1. Publish a full account of exactly what happened. (So far: they have published an incident report and a blog post.)
  2. Monitor their OCSP logs for evidence of use of these certificates, or evidence that access to their OCSP responders is being blocked from any geographical locations. (So far: no sign of use or blocking.)
  3. Cancel all relationship with the RA concerned. (So far: the RA is suspended.)
  4. Change their practices to use intermediate certs rather than issuing directly off the root, and use a different one for each RA.

Ongoing Discussion

Unfortunately, the practice of issuing certs directly from the root eliminated some possible steps we could have taken to mitigate the problem. We are concerned about the amount of trust Comodo seems to have placed in RAs whose network security they did not oversee.

Comodo appropriately revoked the certificates immediately and notified the major browser vendors so that more proactive actions could be taken. We are still working with Comodo to find more information on how the breach occurred and what measures they plan to put in place to prevent future recurrences.

This issue raises many questions about the systems surrounding authentication and security on the web. We intend to have a vigorous discussion about what technical and policy changes we can make to significantly improve the situation. You can join the discussion in the mozilla.dev.security.policy forum.

30 comments on “Comodo Certificate Issue – Follow Up”

  1. Robert wrote on

    I am not sure they would not be able to interfere with automatic updates, if they can impersonate a server they can block Mozilla update servers entirely without needed to impersonate them (and other browser vendors), leaving a lot of browsers without the update with the embedded CRLs

  2. jfkastner wrote on

    PLEASE fix the PKI – DNSSEC etc is almost useless without it

  3. Ross wrote on

    Since this is the second incident with a Comodo agent being caught issuing valid certificates in error, the first being in 2008, I am curious to know what it would take for a CA to get blacklisted from Mozilla entirely? Here we had a certificate issued to someone who didn’t own the domains in question, and one of the domains for which a fraudulent certificate was issued was owned by Mozilla. I figured issuing a fraudulent certificate to domain owned by a major browser vendor and then sending said fraudulent certificate to an IP address in Iran would be enough to lose all trust in a CA, apparently not.

  4. Taylor Hedberg wrote on

    @Ross: Blacklisting an entire CA, especially a fairly large one like Comodo, is not really practical. All of the legitimate websites whose certificates are signed by Comodo (and there are a lot of them) would then be untrusted by Mozilla users. This would result in major problems for users of those sites, as well as for the sites themselves, which would then have to purchase certificates from some other CA in order to maintain trust with anyone who uses Firefox.

  5. Dan wrote on

    I have seen many (legitimate) certificates issued by Comodo RAs. All of them, however, were signed by intermediate CA certificates (e.g. UTN-UserFirst-Hardware -> XXX CA -> http://www.site.com). From what we read here, it appears that in this case the attacker was able to get stuff signed by the root CA.

    What exactly is the role of “UTN-UserFirst-Hardware” in the Comodo PKI infrastructure and who is in possession of the private key?

    I hope we will get an answer from Comodo to these questions.

  6. Luiz Toledo wrote on

    So Comodo seems to be “too big to fail”, like certain commercial banks.
    This situation is not really confortable.
    Rgds

  7. finack wrote on

    How much trust can you put in a site with a certificate signed by a CA without effective signing controls? What are the chances that they’ve caught every bogus cert issued by an RA over the years?

  8. Malor wrote on

    Which strikes me as meaning, essentially, that once you trust a given CA, you can never un-trust them again, no matter what mistakes they make.

    If you’re not willing to revoke trust because it’s painful, then you need to be way, WAY more careful about extending trust in the first place. I’d suggest exploring the separation of CAs into bundles for specific regions. You have us all trusting a bunch of tiny CAs in countries that seem quite corrupt and authoritarian. I, for one, find that slightly terrifying, and I would rather see useful bundles of CA certificates for specific zones, rather than defaulting to granting global trust for everyone you deal with.

  9. Horst H. von Brand wrote on

    The incident report is strange… How does a compromise of a user account get them to create a new account, and use that one to create certificates?

  10. Kyle Huey wrote on

    @Robert

    Mozilla’s internal update code (including that used in Firefox) performs a number of additional checks beyond just establishing an HTTPS connection to the update server. These checks include verifying the cert is one that we expect, and not one issued by some trusted but unexpected party.

  11. Chris Evans wrote on

    Quoting:
    “In hindsight, while it was made in good faith, this was the wrong decision. We should have informed web users more quickly about the threat and the potential mitigations as well as their side-effects.”

    I think the details were only pre-shared with browser manufacturers on the condition that disclosure be limited to cert blacklists, no? You have to respect that otherwise you come off as a tool, and you don’t get the heads-up (nor any ability to protect your users) next time!

  12. Stephan Adig wrote on

    I agree here with Ross.

    Comodo did issue a cert to someone who was not entrusted to do so.
    I expect a Trust Company to actually verify the requester (in this case to check if Yahoo, Mozilla and others had issues a new CSR and requested the to be signed certificate in question.

    This wasn’t the case…

    I do think that dropping Comodo or any other trust companies which are not doing a human check (with all the papers and time involved) would be a good sign that cheap is not always good.

    I already expressed my concerns and opinions here: http://www.shermann.name/2011/03/wrongly-issued-ssl-certificates-of.html

    Regards,

    \sh

  13. Nichol wrote on

    Is it sensible to set up something comparable to a fire drill? I mean: make it possible for e.g. (major) browsers to create fake security attacks, for testing, after which the CA’s should report e.g. to those browsers, in a well-defined way, exactly as if it were a real attack?

  14. anonymous wrote on

    Where is the corresponding update for Thunderbird?

  15. Skello wrote on

    @Chris Evans

    So, what you’re saying is that Comodo is in a position to keep everyone silent about a big screw-up like this and that’s OK?!

    I’m sorry, but I’m siding with ioerror on this one. Public disclosure in this case would not have benefited any other attackers. The risks would not have increased, but a lot of people could have protected themselves by being aware of the issue.

    In the worst case scenario this could have cost some Iranian activists their live and you’re saying: “Watch it Mozilla or next time you won’t get the heads-up”?

    I say that’s BS and I highly doubt Comodo would be able to keep everyone else informed about something like this, but keep Mozilla out of the loop because “last time they spilled the beans.” Imagine the scandal and the hit to their reputation if they would even attempt to do something like that— endanger millions of users.

  16. Mohammad Jamali wrote on

    I’m from Iran and has recently (about a week) faced problems reaching HTTPS sites mentioned above including addons.mozilla.org, login.yahoo.com and most of the time, mail.google.com which are unreachable. The error which is shown is “The requested URL could not be retrieved” “The DNS server returned Time out” which is shown in both Firefox and Opera.
    I wondered if this might have any relation to the forementioned certificates issue?
    I hope that both Comodo and Mozilla look for any activity from hackers more precisely.

  17. Richard Z. wrote on

    The whole concept of PKI is utterly useless, just have a look at the long list of root authorities that mozilla trusts indifferently. There are several agencies from countries with governments well known to be capable of torture and you can take it for granted that these agencies will issue faked certificates whenever it fits their government. Most likely it is happening routinely.

    At the very least the issuing Root-CA must be shown prominently for every SSL page.

  18. stuartb wrote on

    Quoting: “These fixes were also included in updates to Firefox 3.5 and 3.6, also released on 22nd March.”

    Please tell us what the designations of these releases where?

    I see releases of “Firefox Setup 3.6.16.exe” dated March 19, 2011 and “Firefox Setup 4.0.exe” dated March 18, 2011.

    The announcement was made on March 22. With these types of issues you need to be very precise about which versions are vulnerable and which versions fix the problem.

  19. ichsun wrote on

    Have you ever seen Hacker’s response?
    http://pastebin.com/74KXCaEZ

  20. Kob wrote on

    I wonder if I sign a piece of code with a code-signing cert, and have it time-stamped by a 3rd party server, and later the CA issuer gets its authority revoked altogether, will my code still provide “Sig is OK” by the OS?

  21. Google reklama wrote on

    So, what you’re saying is that Comodo is in a position to keep everyone silent about a big screw-up like this and that’s OK?!

  22. Paul van Brouwershaven wrote on

    @Taylor Hedberg
    I think this would not be practical for any kind of product but if there is a safty problem with your car you would be happy if there was a product recall. Just like with any faulty commodity.

    COMODO customers must be offered refunds for any certificates purchased through this CA and Mozilla / Microsoft should force a product recall!

    All affected parties (Google, Mozilla, Microsoft, Skype, Yahoo etc) should seek the full warranty to be paid on this mis-issued COMODO certificates.

  23. toko online wrote on

    thank you for it

  24. uU wrote on

    “I’m single hacker with experience of 1000 hackers” Is that Persian for script kiddy?

  25. Richard Whitehouse wrote on

    The only forseeable way of fixing PKI, and this type of attack is to use DNSSEC and which can be used to verify the certificate is the correct one (for how this works, see Dan Kaminsky’s blog on the subject – http://dankaminsky.com/2010/12/19/dnssec-ch4/ ).

  26. justmoi wrote on

    What about CAs not issuing certificates if they already exist? That plus blacklisting top100 domains except for the checked owner in case of renewal should make these attacks a bit more difficult.

  27. drs m sembiring wrote on

    ya, saya cukup kagum atas karya ini

  28. mhn wrote on

    check this:
    mozilla addon’s certificate
    http://pastebin.com/X8znzPWH

  29. Daniel Veditz wrote on

    justmoi: there’s a proposed specification for that, but it’s not going to help if a CA gets hacked. Similarly Comodo should have had processes in place to automatically check domain ownership but those got bypassed in this attack.

    There are similar proposal that would have _browsers_ check that the CA in the presented certificate matches the expected CA. These show more promise because it’s a check independent of the potentially-compromised certificate authority. One proposal requires DNSSEC support first, the other is to remember which CA you saw last time.

    mhn: that addons certificate is the one fraudulently issued in the Comodo hack. It has been revoked.

  30. Sonic Games wrote on

    That’s what I like about this blog. Full release of any security issues, I wonder if Comodo can be blacklisted?