Choosing an LLM for Work: Top Considerations & Mistakes to Avoid

At a recent conference, Hacks & Hops, hosted by FR Secure, I had the opportunity to witness the diverse opinions surrounding language learning models (LLMs) in the professional landscape. On one end of the spectrum, a respected cybersecurity professional demonstrated elementary mistakes made by ChatGPT, humorously highlighting that AI is far from replacing human jobs. Conversely, the next speaker, another industry expert, detailed how their company successfully leverages a council of LLMs to prioritize cybersecurity threats and allocate resources efficiently. This stark contrast raised an important question: 

Why is there such a wide gap in opinions and experiences with LLMs in the workplace?

Why is there such a wide gap in opinions and experiences with LLMs in the workplace?

The wide gap in opinions and experiences with LLMs in the workplace, as highlighted by a May 2023 Pew Research article, stems from a multitude of factors. The study found that only 2% of adults find ChatGPT to be “extremely useful”, a statistic that resonates with my personal experiences of interacting with hundreds of people regarding the use of LLMs like ChatGPT. One significant hurdle is the lack of necessary skills to effectively communicate with ChatGPT, as utilizing LLMs is a unique skillset that requires practice and study to master.

Moreover, the rapidly evolving landscape of AI and LLM technology adds an additional layer of complexity, posing a challenge for businesses to stay abreast of the latest advancements. Companies are grappling with various questions, such as:

• Data Privacy and Security: Understanding the measures in place to protect sensitive company data.
• Customization vs. General Model: Deciding between a general LLM and a custom-trained model.
• Technical Expertise Required: Assessing the technical expertise required for integration and maintenance.
• Cost: Considering the costs involved, including setup, maintenance, and potential hidden fees.
• Regulatory Compliance: Ensuring compliance with industry regulations and standards.

Each of these factors contributes to the diverse range of opinions and experiences surrounding the use of LLMs in the workplace.

When choosing an LLM for work, here are some of the top considerations

1. Hosting: Decide between a public server for broader accessibility or a private server for enhanced data control and security.

• Choice: Public or Private

• • Public: Using a public server for hosting.
  • Private: Using a private server for hosting with enhanced data privacy.

• Pros & Cons:

• • Public: Cost-effective and easy setup but limited control and potential data privacy concerns.
  • Private: Full control and data privacy but may require more technical expertise and be costlier.

• Examples:

• • Public: An organization introduces their employees to ChatGPT 4.0, providing them with strict guidelines about the types of information that can be shared or prompted. This allows for quick access to the LLM but with clear boundaries to ensure data protection.
  • Private: An enterprise deploys Llama 2.0 on a private cloud server, enabling employees to engage the LLM with client-sensitive data. By keeping the LLM within the organization’s private infrastructure, the risk of sensitive information leaving the company’s environment is mitigated.

• Estimated Price Range:

• • Public: Low ($0 – $100/month)
  • Private: High ($500 – $5,000/month)

2. Data Training: Opt for a general model for versatile applications or a custom-trained model for specific organizational needs and precision.

• Choice: Pre-trained or Custom-trained

• • Pre-trained: Using a model trained on vast amounts of data, suitable for general tasks.
  • Custom-trained: Tailoring a model specifically to the data and tasks of your organization.

• Pros & Cons:

• • Pre-trained: Quick to deploy and cost-effective, but might not be specialized for unique business needs.
  • Custom-trained: Offers a precise fit for specific tasks or data but requires more time, resources, and data for training.

• Examples:

• • Pre-trained: A retail company, valuing the privacy advantages of an LLM, sets up Llama 2.0 on a private cloud server. Given their general LLM usage within the organization, they opt for the basic version of Llama 2.0 7b without needing to add any proprietary data to the training set.
  • Custom-trained: Similarly prioritizing privacy, a pharmaceutical firm hosts Llama 2.0 on a private cloud server. They go a step further by training it with their proprietary research data, enabling the model to assist researchers in generating hypotheses or predicting experimental outcomes specific to their niche field.

• Estimated Price Range:

• • Pre-trained: Low to Medium ($100 – $1,000/month)
  • Custom-trained: Medium to High ($1,000 – $10,000/month)

3. Technical Expertise: Evaluate if basic tech skills suffice for your goals or if advanced expertise is essential for full customization and integration.

• Choice: Low, Medium, High

• • Low: Using an LLM for standard tasks with minimal setup and maintenance.
  • Medium: Employing an LLM that requires some level of configuration, periodic updates, and moderate integrations.
  • High: Deploying and maintaining an LLM with intricate configurations, custom integrations, and ongoing adjustments.

• Pros & Cons:

• • Low: Easy to use and set up, ideal for straightforward applications without a specialized team.
  • Medium: Offers a balance between ease of use and customization. Requires some technical knowledge, but not a full-fledged team.
  • High: Offers the greatest flexibility and customization but demands a dedicated team with in-depth technical expertise.

• Examples:

• • Low: A local bookstore integrates a basic version of ChatGPT 3.5 on their website to recommend books to online shoppers. This setup requires minimal technical adjustments and uses the LLM’s standard capabilities to suggest readings based on popular genres.
  • Medium: An e-commerce platform introduces Llama 2.0 to handle customer inquiries and improve their search functionality. The LLM is configured to understand specific product categories and seasonal promotions, necessitating occasional updates and moderate integrations.
  • High: A multinational finance firm integrates Llama 2.0 into their advanced analytical software. The model processes vast amounts of financial data, predicts market trends, and aligns with their existing infrastructure. A team of IT experts ensures seamless integration and maintains the system regularly.

• Estimated Price Range:

• • Low: $100 – $500/month
  • Medium: $700 – $3,000/month
  • High: $5,000 – $20,000/month

4. Regulatory Compliance: Determine the level of regulatory adherence required, balancing between simplicity and rigorous industry standards.

• Choice: Low, Medium, High

• • Low: Using an LLM in a domain with minimal to no specific regulations.
  • Medium: Employing an LLM in sectors with some industry-specific standards and best practices to adhere to.
  • High: Leveraging an LLM in domains that are highly regulated with stringent standards and potential legal consequences for non-compliance.

• Pros & Cons:

• • Low: Fewer constraints on LLM deployment and use, simpler management without the need for regular compliance checks.
  • Medium: Requires monitoring to ensure adherence to some industry standards, with occasional updates and checks.
  • High: Mandates regular audits, rigorous documentation, and continuous monitoring to ensure full compliance, potentially demanding a dedicated compliance team.

• Examples:

• • Low: An independent blog uses ChatGPT 4.0 to draft articles and recommend content. Given the minimal regulatory landscape for blogging, there’s limited concern about compliance.
  • Medium: An online health and wellness platform integrates ChatGPT 3.5 to provide dietary recommendations. They ensure that their advice aligns with established nutritional standards and best practices, requiring periodic reviews.
  • High: A healthcare provider uses Llama 2.0 to manage patient data and assist in medical diagnostics. Due to the stringent regulations surrounding patient data privacy (like HIPAA in the U.S.), they undertake rigorous audits, maintain extensive documentation, and adhere strictly to all compliance mandates.

• Estimated Price Range:

• • Low: Minimal additional costs
  • Medium: Some added costs for periodic reviews and adjustments
  • High: Significant budgeting for regular audits, documentation, and potential penalties for non-compliance.

5. Costs: Factor in both immediate and ongoing expenses, ensuring the chosen LLM aligns with your budgetary framework.

• Choice: Low, Medium, High

• • Low: Minimal financial investment in LLMs, ideal for startups or businesses with tight budgets.
  • Medium: A balanced budget allocation for LLMs, catering to established businesses aiming for more sophisticated deployments.
  • High: A substantial financial commitment to LLMs, suitable for enterprises looking for cutting-edge implementations and custom solutions.

• Pros & Cons:

• • Low: Cost-effective and suitable for basic requirements, but might lack advanced features or scalability options.
  • Medium: Strikes a balance between cost and features, providing a broader range of functionalities while staying within a moderate budget.
  • High: Offers the most comprehensive set of features, capabilities, and customizability, but demands a significant financial outlay.

• Examples:

• • Low: A startup integrates ChatGPT 3.5 for customer service inquiries at an initial cost of $2,500, followed by an ongoing fee of $250/month, providing a cost-effective solution to handle basic customer interactions.
  • Medium: An e-commerce platform invests $10,000 to set up a privately hosted Llama 2.0, trained on proprietary product and customer data. They then budget $750/month to handle the higher LLM usage and occasional updates.
  • High: A multinational corporation invests $75,000 into a highly tailored LLM that aligns seamlessly with their existing software. They allocate $5,000/month for continuous updates, maintenance, and further proprietary data training, treating their LLM as a central business asset.

• Estimated Price Range:

• • Low: Initial cost with minor ongoing expenses, primarily for basic support and periodic updates.
  • Medium: Moderate upfront investment with subsequent costs for enhanced features, customizations, and regular support.
  • High: Substantial initial investment with significant monthly allocations for custom development, top-tier support, and regular enhancements.

Costs of LLMs: A Deeper Dive

One of the most prominent considerations for businesses when adopting any new technology, including LLMs, is the cost. However, it’s essential to understand that the expenses tied to LLMs aren’t just about the initial investment but span across multiple areas:

Setup and Integration Costs

The initial cost of integrating an LLM into an existing infrastructure can vary significantly. For instance, integrating a general model like ChatGPT might require minimal initial setup if it’s a standalone solution. Still, custom solutions tailored to specific business needs could entail extensive initial investment. For instance, a small business might be able to set up a basic chatbot service for as little as $50/month, while a large corporation could spend upwards of $50,000 on a custom solution.

Maintenance Costs

Once an LLM is up and running, there will be ongoing expenses associated with keeping it operational. This might include server costs, software updates, or even costs related to re-training the model as your business needs evolve. For example, maintaining a cloud-based LLM can range from $100/month for basic hosting to thousands of dollars for high-end, dedicated servers and support.

Training Costs

If a company opts for a custom-trained model to meet its specific needs, there are costs associated with the training data, computational power, and the expertise required. Training a sophisticated LLM model could cost anywhere from $5,000 to over $200,000, depending on the depth and breadth of the training required.

Hidden Fees

As with many services, there might be hidden fees that businesses need to be aware of. These could be in the form of overage charges (if an LLM processes more queries than a subscription plan allows), additional costs for premium features, or fees for technical support. For instance, while a basic subscription to an LLM platform might cost $200/month for a set number of queries, going over that limit could incur additional charges at a rate of, say, $0.10 per query.

Potential Cost Savings

It’s also essential to consider the potential savings LLMs can bring. Automated customer service chatbots, for example, can significantly reduce the need for human agents, leading to considerable long-term savings. Similarly, an LLM assisting in data analysis could potentially save hundreds of hours of manual work, translating to substantial cost savings.

Mistakes to Avoid: Common LLM Implementation Mistakes

Ignoring Data Privacy

• Not ensuring the LLM follows best practices in data protection.
• Lack of encryption for data both in transit and at rest.
• Not regularly reviewing and updating data protection policies.

Overlooking Maintenance Needs

• Neglecting regular software updates.
• Not periodically training the model or monitoring for glitches.
• Failure to allocate budget and manpower for LLM upkeep.

Underestimating Costs

• Overlooking setup, integration, and training costs.
• Not accounting for ongoing maintenance and potential hidden fees.
• Failing to conduct a thorough cost analysis before integration.

Not Considering Scalability

• Choosing an LLM without considering future business growth.
• Overlooking system capacity and functionality as the business expands.
• Not evaluating the LLM’s scalability features.

Choosing the Wrong Model

• Opting for a general LLM when specialized understanding is required.
• Selecting a custom solution when a general model would suffice.
• Not aligning the LLM choice with specific business needs and tasks.

Ready to unlock the potential of Large Language Models for your organization? Don’t navigate the intricate landscape of LLM integration alone. Let our team of experts guide you every step of the way. Reach out today and ensure your AI journey is optimized for success.

Schedule a free consultation today!