{"id":72621,"date":"2023-07-06T07:41:20","date_gmt":"2023-07-06T14:41:20","guid":{"rendered":"https:\/\/blog.mozilla.org\/?p=72621"},"modified":"2023-07-06T09:00:40","modified_gmt":"2023-07-06T16:00:40","slug":"responsibly-empowering-developers-with-ai-on-mdn","status":"publish","type":"post","link":"https:\/\/blog.mozilla.org\/en\/mozilla\/responsibly-empowering-developers-with-ai-on-mdn\/","title":{"rendered":"Responsibly empowering developers with AI on MDN"},"content":{"rendered":"\n

Generative AI technologies powered by Large Language Models (LLMs), such as OpenAI\u2019s ChatGPT<\/a>, have shown themselves to be both a big boon to productivity as well as a concerningly confident purveyor of incorrect information. At Mozilla, we\u2019re excited about the potential role generative AI can play in creating new value for people while demonstrating leadership in responsible and ethical implementation approaches.<\/p>\n\n\n\n

One domain where we see high value is in training LLMs on reference documentation, enabling developers to more quickly find solutions or get answers about the purpose or behavior of code snippets. MDN’s mission is to provide a blueprint for a better internet and empower a new generation of developers and content creators to build it<\/em>. We see a path forward to \u201coverlay\u201d useful AI-driven helpers on top of MDN\u2019s canonical web dev documentation to aid developers in new ways. Of course, human-authored canonical documentation will always be available and clearly called out as such. To this end, last week, MDN launched AI integrations<\/a> with its body of web developer reference documentation, manifesting in two features that have been in development over the last few months: AI Help and AI Explain, both powered by GPT-3.5. <\/p>\n\n\n\n

AI Help<\/strong> enables MDN readers to ask questions with a conversational interface, and the tool offers concise answers with MDN articles related to their questions for contextual help. AI Help is limited to offering information only based on MDN content, and is now in beta and available to logged-in MDN readers.<\/p>\n\n\n\n

AI Explain<\/strong> enables readers to explore and understand code blocks and parts of code examples embedded in MDN documentation pages, describing the underlying purpose and behavior of the code. AI Explain is still an experimental tool.<\/p>\n\n\n\n

Our internal work on these tools has us hugely optimistic about their potential to save developers tons of time as they seek answers and learning resources. Early-stage developers appear to benefit most, as they are least likely to know exactly where to go or which keywords to search for to find the answers they seek. As we\u2019re still in the early stages, we\u2019re also seeing instances where these AI tools provide incorrect information in response to a query. The MDN team is working to identify these cases and develop fixes so that we can continuously raise the quality and utility of the answers provided by AI Help and AI Explain. We are also planning to make it easier for people to flag bad answers, creating issues sent to the team to investigate. In the end, our goal is to make MDN more accessible and useful to a wider variety of developers, without diminishing MDN\u2019s role as the canonical reference source for high-quality information about developing for the web.<\/p>\n\n\n\n

Feedback Matters<\/strong><\/h2>\n\n\n\n

With the launches of AI Help and AI Explain last week, we received a wide range of feedback from readers, from delight to constructive criticisms to concerns about the technical accuracy of the responses. We\u2019re only a handful of days into the journey, but the data so far seems to indicate a sense of skepticism towards AI and LLMs in general, while those who have tried the features to find answers tend to be happy with the results.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

For AI Help in particular, the feedback indicates that the majority of people who used this feature and voted consider the answers to be helpful. Please do try AI Help<\/a> and give us your feedback so that we can enhance this service based on how it works for you!<\/p>\n\n\n\n

In the case of AI Explain, the pattern of feedback we received was similar, but readers also pointed out a handful of concrete cases where an incorrect answer was rendered. This feedback is enormously helpful, and the MDN team is now investigating these bug reports. We’ve elected to be cautious in our approach and have temporarily removed the AI Explain tool from MDN until we\u2019ve completed our investigation and have high-quality remediations in place for the issues that have been observed.<\/p>\n\n\n\n

Here is an example of an incorrect answer: the AI indicates that the code defines a grid with two rows and two columns, where it actually only defines two rows, giving an incorrect answer.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

AI Assistants: A Helpful Complement<\/h2>\n\n\n\n

One of the user experience challenges LLMs present is how to set customer expectations. As useful and efficient as it is to use LLMs to interact with reference documentation, even extraordinarily well-trained LLMs \u2014 like humans \u2014 will sometimes be wrong. We see this as an emerging challenge in the field of human-computer interaction that goes well beyond MDN\u2019s limited use case: What should people do when chat-based systems render answers in good faith that are merely likely<\/em> to be correct? One approach to responsible systems design could be to provide people with better ways to check answers and build confidence in their veracity. This is a field we\u2019re excited to participate in.<\/p>\n\n\n\n

Speaking of the human element, LLMs also have the useful attributes of having zero ego and infinite patience. LLMs don\u2019t mind answering the same question over and over again, and they feel no compulsion to gate-keep for online communities. It\u2019s sadly not uncommon today for learners to face the discouraging experience of asking technical questions in online forums only to be dismissed or shamed for their \u201cdumb question.\u201d When done well, there is a clear opportunity to employ AI-based tools to improve the pace of learning as well as inclusivity for learners.<\/p>\n\n\n\n

The important question, and the core of this venture, is whether these AI assistants can improve our work and user experience. Do they help our users find information faster? Do they simplify complex concepts for them? Do they support Mozilla\u2019s values and mission? If the answer is ‘yes’ to these questions, then they are fulfilling their purpose. We will work to convey this perspective clearly to everyone who interacts with our AI features.<\/p>\n\n\n\n

The MDN Community<\/h2>\n\n\n\n

MDN has a long history as a human-authored and curated source of knowledge, so we know that AI integration in MDN will be a sensitive topic for some. While many are excited about the possibilities of generative AI, others might prefer that MDN stay how it is. That’s fine. We are a community, and differences of opinion are normal and healthy. MDN’s strength lies in our community. We do see a path forward that preserves the human-authored goodness while also providing tools that offer additional value over that amazing body of content. We need your input, criticism, kudos, and experience to ensure we’re employing AI in the most useful and responsible ways. Your feedback is critical to this process, and we will continue to take the feedback and adjust our plans in response to it. LLM technology remains relatively immature, so there will certainly be speed bumps along the way. This journey has just begun, and together, we’ll shape MDN’s future.<\/p>\n\n\n\n

In the continued spirit of community and transparency, the MDN team will publish a postmortem blog in the coming days, that will include a breakdown of the feedback we\u2019ve received and dive into some of the details of developing, launching, and pausing AI Explain.<\/p>\n","protected":false},"excerpt":{"rendered":"

Generative AI technologies powered by Large Language Models (LLMs), such as OpenAI\u2019s ChatGPT, have shown themselves to be both a big boon to productivity as well as a concerningly confident purveyor of incorrect information. At Mozilla, we\u2019re excited about the potential role generative AI can play in creating new value for people while demonstrating leadership […]<\/p>\n","protected":false},"author":1918,"featured_media":69139,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[463312,5],"tags":[317823,4708,464177],"coauthors":[464171],"acf":[],"yoast_head":"\nResponsibly empowering developers with AI on MDN<\/title>\n<meta name=\"description\" content=\"With the launches of AI Help and AI Explain last week, we received a wide range of feedback from readers, from delight to constructive criticisms to concerns about the technical accuracy of the responses.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/blog.mozilla.org\/en\/mozilla\/responsibly-empowering-developers-with-ai-on-mdn\/\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/blog.mozilla.org\/en\/mozilla\/responsibly-empowering-developers-with-ai-on-mdn\/\",\"url\":\"https:\/\/blog.mozilla.org\/en\/mozilla\/responsibly-empowering-developers-with-ai-on-mdn\/\",\"name\":\"Responsibly empowering developers with AI on MDN\",\"isPartOf\":{\"@id\":\"https:\/\/blog.mozilla.org\/en\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/blog.mozilla.org\/en\/mozilla\/responsibly-empowering-developers-with-ai-on-mdn\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/blog.mozilla.org\/en\/mozilla\/responsibly-empowering-developers-with-ai-on-mdn\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/blog.mozilla.org\/wp-content\/blogs.dir\/278\/files\/2022\/06\/moz_blog_header_sustainability_2020-11-11.png\",\"datePublished\":\"2023-07-06T14:41:20+00:00\",\"dateModified\":\"2023-07-06T16:00:40+00:00\",\"author\":{\"@id\":\"https:\/\/blog.mozilla.org\/en\/#\/schema\/person\/2cf6620089b2a8cb6a3746ec50d08f07\"},\"description\":\"With the launches of AI Help and AI Explain last week, we received a wide range of feedback from readers, from delight to constructive criticisms to concerns about the technical accuracy of the responses.\",\"breadcrumb\":{\"@id\":\"https:\/\/blog.mozilla.org\/en\/mozilla\/responsibly-empowering-developers-with-ai-on-mdn\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/blog.mozilla.org\/en\/mozilla\/responsibly-empowering-developers-with-ai-on-mdn\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/blog.mozilla.org\/en\/mozilla\/responsibly-empowering-developers-with-ai-on-mdn\/#primaryimage\",\"url\":\"https:\/\/blog.mozilla.org\/wp-content\/blogs.dir\/278\/files\/2022\/06\/moz_blog_header_sustainability_2020-11-11.png\",\"contentUrl\":\"https:\/\/blog.mozilla.org\/wp-content\/blogs.dir\/278\/files\/2022\/06\/moz_blog_header_sustainability_2020-11-11.png\",\"width\":1920,\"height\":1080},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/blog.mozilla.org\/en\/mozilla\/responsibly-empowering-developers-with-ai-on-mdn\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/blog.mozilla.org\/en\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Responsibly empowering developers with AI on MDN\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/blog.mozilla.org\/en\/#website\",\"url\":\"https:\/\/blog.mozilla.org\/en\/\",\"name\":\"The Mozilla Blog\",\"description\":\"News and Updates about Mozilla\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/blog.mozilla.org\/en\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/blog.mozilla.org\/en\/#\/schema\/person\/2cf6620089b2a8cb6a3746ec50d08f07\",\"name\":\"Steve Teixeira\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/blog.mozilla.org\/en\/#\/schema\/person\/image\/fc5ac6c22ec5a03bc943273c47e0e2ba\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/2eedd4d55fca6bd8dc5166348f672ea0?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/2eedd4d55fca6bd8dc5166348f672ea0?s=96&d=mm&r=g\",\"caption\":\"Steve Teixeira\"},\"url\":\"https:\/\/blog.mozilla.org\/en\/author\/stevetex\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Responsibly empowering developers with AI on MDN","description":"With the launches of AI Help and AI Explain last week, we received a wide range of feedback from readers, from delight to constructive criticisms to concerns about the technical accuracy of the responses.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/blog.mozilla.org\/en\/mozilla\/responsibly-empowering-developers-with-ai-on-mdn\/","schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/blog.mozilla.org\/en\/mozilla\/responsibly-empowering-developers-with-ai-on-mdn\/","url":"https:\/\/blog.mozilla.org\/en\/mozilla\/responsibly-empowering-developers-with-ai-on-mdn\/","name":"Responsibly empowering developers with AI on MDN","isPartOf":{"@id":"https:\/\/blog.mozilla.org\/en\/#website"},"primaryImageOfPage":{"@id":"https:\/\/blog.mozilla.org\/en\/mozilla\/responsibly-empowering-developers-with-ai-on-mdn\/#primaryimage"},"image":{"@id":"https:\/\/blog.mozilla.org\/en\/mozilla\/responsibly-empowering-developers-with-ai-on-mdn\/#primaryimage"},"thumbnailUrl":"https:\/\/blog.mozilla.org\/wp-content\/blogs.dir\/278\/files\/2022\/06\/moz_blog_header_sustainability_2020-11-11.png","datePublished":"2023-07-06T14:41:20+00:00","dateModified":"2023-07-06T16:00:40+00:00","author":{"@id":"https:\/\/blog.mozilla.org\/en\/#\/schema\/person\/2cf6620089b2a8cb6a3746ec50d08f07"},"description":"With the launches of AI Help and AI Explain last week, we received a wide range of feedback from readers, from delight to constructive criticisms to concerns about the technical accuracy of the responses.","breadcrumb":{"@id":"https:\/\/blog.mozilla.org\/en\/mozilla\/responsibly-empowering-developers-with-ai-on-mdn\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/blog.mozilla.org\/en\/mozilla\/responsibly-empowering-developers-with-ai-on-mdn\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/blog.mozilla.org\/en\/mozilla\/responsibly-empowering-developers-with-ai-on-mdn\/#primaryimage","url":"https:\/\/blog.mozilla.org\/wp-content\/blogs.dir\/278\/files\/2022\/06\/moz_blog_header_sustainability_2020-11-11.png","contentUrl":"https:\/\/blog.mozilla.org\/wp-content\/blogs.dir\/278\/files\/2022\/06\/moz_blog_header_sustainability_2020-11-11.png","width":1920,"height":1080},{"@type":"BreadcrumbList","@id":"https:\/\/blog.mozilla.org\/en\/mozilla\/responsibly-empowering-developers-with-ai-on-mdn\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/blog.mozilla.org\/en\/"},{"@type":"ListItem","position":2,"name":"Responsibly empowering developers with AI on MDN"}]},{"@type":"WebSite","@id":"https:\/\/blog.mozilla.org\/en\/#website","url":"https:\/\/blog.mozilla.org\/en\/","name":"The Mozilla Blog","description":"News and Updates about Mozilla","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/blog.mozilla.org\/en\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/blog.mozilla.org\/en\/#\/schema\/person\/2cf6620089b2a8cb6a3746ec50d08f07","name":"Steve Teixeira","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/blog.mozilla.org\/en\/#\/schema\/person\/image\/fc5ac6c22ec5a03bc943273c47e0e2ba","url":"https:\/\/secure.gravatar.com\/avatar\/2eedd4d55fca6bd8dc5166348f672ea0?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/2eedd4d55fca6bd8dc5166348f672ea0?s=96&d=mm&r=g","caption":"Steve Teixeira"},"url":"https:\/\/blog.mozilla.org\/en\/author\/stevetex\/"}]}},"_links":{"self":[{"href":"https:\/\/blog.mozilla.org\/en\/wp-json\/wp\/v2\/posts\/72621"}],"collection":[{"href":"https:\/\/blog.mozilla.org\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.mozilla.org\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/en\/wp-json\/wp\/v2\/users\/1918"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/en\/wp-json\/wp\/v2\/comments?post=72621"}],"version-history":[{"count":0,"href":"https:\/\/blog.mozilla.org\/en\/wp-json\/wp\/v2\/posts\/72621\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/en\/wp-json\/wp\/v2\/media\/69139"}],"wp:attachment":[{"href":"https:\/\/blog.mozilla.org\/en\/wp-json\/wp\/v2\/media?parent=72621"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.mozilla.org\/en\/wp-json\/wp\/v2\/categories?post=72621"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.mozilla.org\/en\/wp-json\/wp\/v2\/tags?post=72621"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/blog.mozilla.org\/en\/wp-json\/wp\/v2\/coauthors?post=72621"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}