{"id":2534,"date":"2020-01-09T13:00:11","date_gmt":"2020-01-09T21:00:11","guid":{"rendered":"https:\/\/blog.mozilla.org\/security\/?p=2534"},"modified":"2020-01-08T15:48:36","modified_gmt":"2020-01-08T23:48:36","slug":"crlite-part-2-end-to-end-design","status":"publish","type":"post","link":"https:\/\/blog.mozilla.org\/security\/2020\/01\/09\/crlite-part-2-end-to-end-design\/","title":{"rendered":"The End-to-End Design of CRLite"},"content":{"rendered":"

CRLite is a technology to efficiently compress revocation information for the whole Web PKI into a format easily delivered to Web users. It addresses the performance and privacy pitfalls of the Online Certificate Status Protocol (OCSP) while avoiding a need for some administrative decisions on the relative value of one revocation versus another. For details on the background of CRLite, see our first post, Introducing CRLite: All of the Web PKI\u2019s revocations, compressed<\/a>.<\/p>\n

To discuss CRLite\u2019s design, let\u2019s first discuss the input data, and from that we can discuss how the system is made reliable.<\/p>\n

Designing CRLite<\/h3>\n

When Firefox securely connects to a website, the browser validates that the website\u2019s certificate has a chain of trust<\/a> back to a Certificate Authority (CA) in the Mozilla Root CA Program<\/a>, including whether any of the CAs in the chain of trust are themselves revoked. At this time Firefox knows the issuing certificate\u2019s identity and public key, as well as the website\u2019s certificate\u2019s identity and public key.<\/p>\n

To determine whether the website\u2019s certificate is trusted, Firefox verifies that the chain of trust is unbroken, and then determines whether the website\u2019s certificate is revoked. Normally that\u2019s done via OCSP, but with CRLite Firefox simply has to answer the following questions:<\/p>\n

    \n
  1. Is this website\u2019s certificate older than my local CRLite Filter, e.g., is my filter fresh enough?<\/li>\n
  2. Is the CA that issued this website\u2019s certificate included in my local CRLite Filter, e.g. is that CA participating?<\/li>\n
  3. If \u201cyes\u201d to the above, and Firefox queries the local CRLite Filter, does it indicate the website\u2019s certificate is revoked?<\/li>\n<\/ol>\n

    That\u2019s a lot of moving parts, but let\u2019s inspect them one by one.<\/p>\n

    Freshness of CRLite Filter Data<\/h3>\n

    Mozilla\u2019s infrastructure continually monitors all of the known Certificate Transparency logs<\/a> for new certificates using our CRLite tooling; the details of how that works will be in a later blog post about the infrastructure. Since multiple browsers now require that all website certificates are disclosed to Certificate Transparency logs to be trusted, in effect the tooling has total knowledge of the certificates in the public Web PKI.<\/p>\n

    \"CRLite<\/a>

    Figure 1: CRLite Information Flow. More details on the infrastructure will be in Part 4 of this blog post series.<\/p><\/div>\n

    Four times per day, all website certificates that haven\u2019t reached their expiration date are processed, drawing out lists of their Certificate Authorities, their serial numbers, and the web URLs where they might be mentioned in a Certificate Revocation List (CRL).<\/p>\n

    All of the referenced CRLs are downloaded, verified, processed, and correlated against the lists of unexpired website certificates.<\/p>\n

    \"The<\/a>

    Figure 2: CRLite Filter Generation Process<\/p><\/div>\n

    At the end, we have a set of all known issuers that publish CRLs we could use, the identification numbers of every certificate they issued that is still unexpired, and the identification numbers of every certificate they issued that hasn\u2019t expired but was revoked.<\/p>\n

    With this knowledge, we can build a CRLite Filter.<\/p>\n

    Structure of A CRLite Filter<\/h4>\n

    CRLite data comes in the form of a series of cascading Bloom filters<\/a>, with each filter layer adding data to the one before it. Individual Bloom filters have a certain chance of false-positives, but using Certificate Transparency as an oracle, the whole Web PKI\u2019s certificate corpus is verified through the filter. When a false-positive is discovered, the algorithm adds it to another filter layer to resolve the false positive.<\/p>\n

    \"The<\/a>

    Figure 3: CRLite Filter Structure<\/p><\/div>\n

    The certificate\u2019s identifier is defined as shown in Figure 4:<\/p>\n

    \"The<\/a>

    Figure 4: CRLite Certificate Identifier<\/p><\/div>\n

    For complete details of this construction see Section III.B of the CRLite paper<\/a>.<\/p>\n

    After construction, the included Web PKI\u2019s certificate corpus is again verified through the filter, ensuring accuracy at that point-in-time.<\/p>\n

    Ensuring Filter Accuracy<\/h4>\n

    A CRLite filter is accurate at a given point-in-time, and should only be used for the certificates that were both known to the filter generator, and for which there is revocation information.<\/p>\n

    We can know whether a certificate could be included in the filter if that certificate has delivered with it a Signed Certificate Timestamp from a participating Certificate Transparency log that is at least one Maximum Merge Delay older than our CRLite filter date.<\/p>\n

    If that is true, we also determine whether the certificate\u2019s issuer is included in the CRLite filter, by referencing our preloaded Intermediate data for a boolean flag reporting whether CRLite includes its data. Specifically, the CA must be publishing accessible, fresh, verified CRL files at a URL included within their certificates\u2019 Authority Information Access data. This flag is updated with the same cadence as CRLite itself, and generally remains constant.<\/p>\n

    Firefox\u2019s Revocation Checking Algorithm Today<\/h4>\n

    Today, Firefox Nightly is using CRLite in telemetry-only mode, meaning that Firefox will continue to rely on OCSP to determine whether a website\u2019s certificate is valid. If an OCSP response is provided by the webserver itself — via OCSP Stapling<\/a> — that is used. However, at the same time, CRLite is evaluated, and that result is reported via Firefox Telemetry but not used for revocation.<\/p>\n

    At a future date, we will prefer to use CRLite for revocation checks, and only if the website cannot be validated via CRLite would we use OCSP, either live or stapled.<\/p>\n

    Firefox Nightly has a preference security.pki.crlite_mode which controls CRLite; set to 1 it gathers telemetry as stated above. Set to 2, CRLite will enforce revocations in the CRLite filter, but still use OCSP if the CRLite filter does not indicate a revocation.\u00a0 A future mode will permit CRLite-eligible certificates to bypass OCSP entirely, which is our ultimate goal.<\/p>\n

    Participating Certificate Authorities<\/h3>\n

    Only public CAs within the Mozilla Root Program are eligible to be included, and CAs are automatically enrolled when they publish CRLs. If a CA stops publishing CRLs, or problems arise with their CRLs, they will be automatically excluded from CRLite filters until the situation is resolved.<\/p>\n

    As mentioned earlier, if a CA chooses not to log a certificate to a known Certificate Transparency log, then CRLite will not be used to perform revocation checking for that certificate.<\/p>\n

    Ultimately, we expect CAs to be very interested in participating in CRLite, as it could significantly reduce the cost of operating their OCSP infrastructure.<\/p>\n

    Listing Enrolled Certificate Authorities<\/h4>\n

    The list of CAs currently enrolled is in our Intermediate Preloading<\/a> data served via Firefox Remote Settings<\/a>. In the FAQ for CRLite on Github<\/a>, there\u2019s information on how to download and process that data yourself to see what CAs revocations are included in the CRLite state.<\/p>\n

    Notably, Let\u2019s Encrypt<\/a> currently does not publish CRLs, and as such their revocations are not included in CRLite. The CRLite filters will increase in size as more CAs become enrolled, but the size increase is modeled to be modest.<\/p>\n

    Portion of the Web PKI Enrolled<\/h4>\n

    Currently CRLite covers only a portion of the Web PKI as a whole, though a sizable portion: As-generated through roughly a period covering December 2019, CRLite covered approximately 100M certificates in the WebPKI, of which about 750k were revoked.<\/p>\n

    \"100M<\/a>

    Figure 5: Number of Enrolled Revoked vs Enrolled But Not Revoked Certificates<\/p><\/div>\n

    The whole size of the WebPKI trusted by Mozilla with any CRL distribution point listed is 152M certificates, so CRLite today includes 66% of the potentially-compatible WebPKI\u00a0 [Censys.io<\/a>]. The missing portion is mostly due to CRL downloading or processing errors which are being addressed. That said, approximately 300M additional trusted certificates do not include CRL revocation information, and are not currently eligible to be included in CRLite.<\/p>\n

    Data Sizes, Update Frequency, and the Future<\/h3>\n

    CRLite promises substantial compression of the dataset; the binary form of all unexpired certificate serial numbers comprises about 16 GB of memory in Redis<\/a>; the hexadecimal form of all enrolled and unexpired certificate serial numbers comprises about 6.7 GB on disk, while the resulting binary Bloom filter compresses to approximately 1.3 MB.<\/p>\n

    \"Size<\/a>

    Figure 6: CRLite Filter Sizes over the month of December 2019 (in kilobytes)<\/p><\/div>\n

    To ensure freshness, our initial target was to produce new filters four times per day, with Firefox users generally downloading small delta difference files to catch-up to the current filter. At present, we are not shipping delta files, as we\u2019re still working toward an efficient delta-expression format.<\/p>\n

    Filter generation is a reasonably fast process even on modest hardware, with the majority of time being spent aggregating together all unexpired certificate serial numbers, all revoked serial numbers, and producing a final set of known-revoked and known-not-revoked certificate issuer-serial numbers (mean of 35 minutes). These aggregated lists are then fed into the CRLite bloom filter generator, which follows the process in Figure 2 (mean of 20 minutes).<\/p>\n

     <\/p>\n

    \"Distribution<\/a>

    Figure 7: Filter Generation Time [source<\/a>]<\/p><\/div>For the most part, faster disks and more efficient (but not human-readable) file formats would speed this process up, but the current speeds are more than sufficient to meet our initial goals, particularly while we continue improving other aspects of the system.<\/p>\n

    Our next blog post in this series, Part 3, will discuss the telemetry results<\/a> that our current users of Firefox Nightly are seeing, while Part 4 will discuss the design of the infrastructure.<\/p>\n","protected":false},"excerpt":{"rendered":"

    CRLite is a technology to efficiently compress revocation information for the whole Web PKI into a format easily delivered to Web users. It addresses the performance and privacy pitfalls of … Read more<\/a><\/p>\n","protected":false},"author":1349,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[320796,69,45499],"tags":[327155,320796,907,45499],"coauthors":[45540],"yoast_head":"\nThe End-to-End Design of CRLite - Mozilla Security Blog<\/title>\n<meta name=\"description\" content=\"The end-to-end design of CRLite gives us a new way to handle revocations in the Web PKI\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/blog.mozilla.org\/security\/2020\/01\/09\/crlite-part-2-end-to-end-design\/\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"J.C. Jones\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"8 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/blog.mozilla.org\/security\/2020\/01\/09\/crlite-part-2-end-to-end-design\/\",\"url\":\"https:\/\/blog.mozilla.org\/security\/2020\/01\/09\/crlite-part-2-end-to-end-design\/\",\"name\":\"The End-to-End Design of CRLite - Mozilla Security Blog\",\"isPartOf\":{\"@id\":\"https:\/\/blog.mozilla.org\/security\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/blog.mozilla.org\/security\/2020\/01\/09\/crlite-part-2-end-to-end-design\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/blog.mozilla.org\/security\/2020\/01\/09\/crlite-part-2-end-to-end-design\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/blog.mozilla.org\/security\/files\/2020\/01\/figure1-information-flow-600x572.png\",\"datePublished\":\"2020-01-09T21:00:11+00:00\",\"dateModified\":\"2020-01-08T23:48:36+00:00\",\"author\":{\"@id\":\"https:\/\/blog.mozilla.org\/security\/#\/schema\/person\/f2bfcea9a0c404ce2431925922bedbde\"},\"description\":\"The end-to-end design of CRLite gives us a new way to handle revocations in the Web PKI\",\"breadcrumb\":{\"@id\":\"https:\/\/blog.mozilla.org\/security\/2020\/01\/09\/crlite-part-2-end-to-end-design\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/blog.mozilla.org\/security\/2020\/01\/09\/crlite-part-2-end-to-end-design\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/blog.mozilla.org\/security\/2020\/01\/09\/crlite-part-2-end-to-end-design\/#primaryimage\",\"url\":\"https:\/\/blog.mozilla.org\/security\/files\/2020\/01\/figure1-information-flow.png\",\"contentUrl\":\"https:\/\/blog.mozilla.org\/security\/files\/2020\/01\/figure1-information-flow.png\",\"width\":946,\"height\":902,\"caption\":\"Figure 1: CRLite Information Flow. More details on the infrastructure will be in Part 4 of this blog post series.\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/blog.mozilla.org\/security\/2020\/01\/09\/crlite-part-2-end-to-end-design\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/blog.mozilla.org\/security\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"The End-to-End Design of CRLite\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/blog.mozilla.org\/security\/#website\",\"url\":\"https:\/\/blog.mozilla.org\/security\/\",\"name\":\"Mozilla Security Blog\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/blog.mozilla.org\/security\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/blog.mozilla.org\/security\/#\/schema\/person\/f2bfcea9a0c404ce2431925922bedbde\",\"name\":\"J.C. Jones\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/blog.mozilla.org\/security\/#\/schema\/person\/image\/d063fc46e7671301c178b2781210dff7\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/64eb1412c9354cf356df31936368cdac?s=96&d=identicon&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/64eb1412c9354cf356df31936368cdac?s=96&d=identicon&r=g\",\"caption\":\"J.C. Jones\"},\"description\":\"Keeping people safe on the 'net. Cryptography Engineering lead for Firefox.\",\"sameAs\":[\"https:\/\/tacticalsecret.com\/\",\"https:\/\/x.com\/JamesPugJones\"]}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"The End-to-End Design of CRLite - Mozilla Security Blog","description":"The end-to-end design of CRLite gives us a new way to handle revocations in the Web PKI","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/blog.mozilla.org\/security\/2020\/01\/09\/crlite-part-2-end-to-end-design\/","twitter_misc":{"Written by":"J.C. Jones","Est. reading time":"8 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/blog.mozilla.org\/security\/2020\/01\/09\/crlite-part-2-end-to-end-design\/","url":"https:\/\/blog.mozilla.org\/security\/2020\/01\/09\/crlite-part-2-end-to-end-design\/","name":"The End-to-End Design of CRLite - Mozilla Security Blog","isPartOf":{"@id":"https:\/\/blog.mozilla.org\/security\/#website"},"primaryImageOfPage":{"@id":"https:\/\/blog.mozilla.org\/security\/2020\/01\/09\/crlite-part-2-end-to-end-design\/#primaryimage"},"image":{"@id":"https:\/\/blog.mozilla.org\/security\/2020\/01\/09\/crlite-part-2-end-to-end-design\/#primaryimage"},"thumbnailUrl":"https:\/\/blog.mozilla.org\/security\/files\/2020\/01\/figure1-information-flow-600x572.png","datePublished":"2020-01-09T21:00:11+00:00","dateModified":"2020-01-08T23:48:36+00:00","author":{"@id":"https:\/\/blog.mozilla.org\/security\/#\/schema\/person\/f2bfcea9a0c404ce2431925922bedbde"},"description":"The end-to-end design of CRLite gives us a new way to handle revocations in the Web PKI","breadcrumb":{"@id":"https:\/\/blog.mozilla.org\/security\/2020\/01\/09\/crlite-part-2-end-to-end-design\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/blog.mozilla.org\/security\/2020\/01\/09\/crlite-part-2-end-to-end-design\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/blog.mozilla.org\/security\/2020\/01\/09\/crlite-part-2-end-to-end-design\/#primaryimage","url":"https:\/\/blog.mozilla.org\/security\/files\/2020\/01\/figure1-information-flow.png","contentUrl":"https:\/\/blog.mozilla.org\/security\/files\/2020\/01\/figure1-information-flow.png","width":946,"height":902,"caption":"Figure 1: CRLite Information Flow. More details on the infrastructure will be in Part 4 of this blog post series."},{"@type":"BreadcrumbList","@id":"https:\/\/blog.mozilla.org\/security\/2020\/01\/09\/crlite-part-2-end-to-end-design\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/blog.mozilla.org\/security\/"},{"@type":"ListItem","position":2,"name":"The End-to-End Design of CRLite"}]},{"@type":"WebSite","@id":"https:\/\/blog.mozilla.org\/security\/#website","url":"https:\/\/blog.mozilla.org\/security\/","name":"Mozilla Security Blog","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/blog.mozilla.org\/security\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/blog.mozilla.org\/security\/#\/schema\/person\/f2bfcea9a0c404ce2431925922bedbde","name":"J.C. Jones","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/blog.mozilla.org\/security\/#\/schema\/person\/image\/d063fc46e7671301c178b2781210dff7","url":"https:\/\/secure.gravatar.com\/avatar\/64eb1412c9354cf356df31936368cdac?s=96&d=identicon&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/64eb1412c9354cf356df31936368cdac?s=96&d=identicon&r=g","caption":"J.C. Jones"},"description":"Keeping people safe on the 'net. Cryptography Engineering lead for Firefox.","sameAs":["https:\/\/tacticalsecret.com\/","https:\/\/x.com\/JamesPugJones"]}]}},"_links":{"self":[{"href":"https:\/\/blog.mozilla.org\/security\/wp-json\/wp\/v2\/posts\/2534"}],"collection":[{"href":"https:\/\/blog.mozilla.org\/security\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.mozilla.org\/security\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/security\/wp-json\/wp\/v2\/users\/1349"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/security\/wp-json\/wp\/v2\/comments?post=2534"}],"version-history":[{"count":0,"href":"https:\/\/blog.mozilla.org\/security\/wp-json\/wp\/v2\/posts\/2534\/revisions"}],"wp:attachment":[{"href":"https:\/\/blog.mozilla.org\/security\/wp-json\/wp\/v2\/media?parent=2534"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.mozilla.org\/security\/wp-json\/wp\/v2\/categories?post=2534"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.mozilla.org\/security\/wp-json\/wp\/v2\/tags?post=2534"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/blog.mozilla.org\/security\/wp-json\/wp\/v2\/coauthors?post=2534"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}