{"id":75140,"date":"2024-06-10T06:00:00","date_gmt":"2024-06-10T13:00:00","guid":{"rendered":"https:\/\/blog.mozilla.org\/?p=75140"},"modified":"2024-06-09T19:53:47","modified_gmt":"2024-06-10T02:53:47","slug":"genai-trends","status":"publish","type":"post","link":"https:\/\/blog.mozilla.org\/en\/mozilla\/ai\/genai-trends\/","title":{"rendered":"Uncovering GenAI trends: Using local language models to explore 35 organizations\u00a0"},"content":{"rendered":"\n

(To read the complete analysis as well as the results of each language model, visit the mozilla.ai blog here<\/a>.)<\/p>\n\n\n\n

Over the past few months, Mozilla.ai has engaged with several organizations to learn how they are using language models in practice. We spoke with 35 organizations across various sectors, including finance, government, startups, and large enterprises. Our interviewees ranged from machine learning engineers to CTOs, capturing a diverse range of perspectives. <\/p>\n\n\n\n

To analyze these interviews, we used open-source local language models running on our laptops. The analysis confirmed the trends we had anticipated during our interviews and shed light on the differences in each model\u2019s presentation of said trends<\/a>.<\/p>\n\n\n\n

Objective: Help shape our product vision<\/strong><\/h2>\n\n\n\n

Our primary aim was to identify patterns and trends that could inform our product development strategy. Despite the unique nature of each discussion, we usually focused on four critical areas:<\/p>\n\n\n\n

    \n
  1. LLM use cases being explored<\/li>\n\n\n\n
  2. Technology, methodologies, and approaches employed<\/li>\n\n\n\n
  3. Challenges in developing and delivering LLM solutions<\/li>\n\n\n\n
  4. Desired capabilities and features<\/li>\n<\/ol>\n\n\n\n

    Data collection & model selection<\/strong><\/h2>\n\n\n\n

    After each conversation, we wrote up summary notes. In total, these notes for the 35 conversations amounted to 18,481 words (approximately 24,600 tokens<\/a>), almost the length of a novella. To avoid confirmation bias and subjective interpretation, we decided to leverage language models for a more objective analysis of the data. By providing the models with the complete set of notes, we aimed to uncover patterns and trends without our pre-existing notions and biases.<\/p>\n\n\n\n

    Given privacy concerns, we decided to keep the information local. Therefore, I selected a set of models that I could run on my MacBook Pro M3 (36GB) locally. Here’s an overview of the models and configurations used:<\/p>\n\n\n\n

    Model<\/strong><\/td>Parameters<\/strong><\/td>Quantization<\/strong><\/td>Size<\/strong><\/td><\/tr>
    Llama-3-8B-Instruct-Gradient-1048k<\/a>\u00a0<\/td>8B<\/td>Q5_0<\/td>5.6GB<\/td><\/tr>
    Phi-3-medium-128k-instruct<\/a><\/td>14B<\/td>IQ3_M<\/td>6.47GB<\/td><\/tr>
    Qwen1.5-7B-Chat<\/a><\/td>7B<\/td>1_5<\/td>5.53GB<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n

    There are a number of options to run LLMs locally, such as ollama<\/a>, lm-studio<\/a>, and llamafile<\/a>. I used both lm-studio and llamafile (an in-house solution by the Mozilla Innovation Team<\/a>).<\/p>\n\n\n\n

    Summarizing with local language models<\/strong><\/h2>\n\n\n\n

    The prompt used to generate model outputs was: \u201cSummarize the following information to get the key takeaways about developing LLM solutions in 10 bullet points. Take the full information from start to finish into account. Never use company names or an individual’s name. [Full notes]\u201d<\/em><\/p>\n\n\n\n

    To read the complete analysis as well as the results of each language model, visit the <\/em>mozilla.ai blog here<\/em><\/a>.<\/em><\/p>\n\n\n\n

    Key takeaways<\/strong><\/h2>\n\n\n\n

    I was impressed by the quality of the responses from these models, which were all capable of running locally on my laptop. They identified the majority of trends and patterns among the 35 organizations we studied. Each model also highlighted unique insights and communicated in different styles:<\/p>\n\n\n\n