The advent of large language models (LLM) has created uncertainty for businesses regarding their effective utilisation. These models transform high-dimensional inputs into high-dimensional outputs. The type of transformation depends on the model’s training and can commonly be applied to any pairwise input/output combination of text, images or audio.

Training custom large language models for small businesses is usually not viable due to resource requirements. Consequently, most businesses utilise pre-trained models with a general knowledge base. This has proven extremely useful for tasks such as:

• proof reading (grammar and spell checking) and rewriting
• image and text summarisation
• transcription
• categorisation (moderation and sentiment analysis)
• translation (text-to-speech, image-to-text, text-to-text, speech-to-speech, …)

Adaptations

Due to the hierarchical nature of the neural network, which processes more general features in the early layers and more specific features in the later ones, it is generally possible to adapt a general knowledge model to obtain domain-specific knowledge through fine-tuning. This involves training the final or additional layers. This significantly increases the likelihood of training success. For example, your proprietory documents could be used to fine-tune an LLM allowing interactive prompts and more precise retrieval and better recall.

Note of caution

LLMs are ultimately designed for the transformation of one sequence into another. As large models their training and query processing can be resource-intensive. The former is reflected in the initial investment cost whereas the latter results in processing latencies which can negatively impact customer experience. The significant initial investment and ongoing processing costs are often overlooked in project costing. LLMs are frequently applied to problems where alternative machine learning techniques may ultimately be more appropriate and cost-effective.

Pricing is usually per million tokens and varies depending on the model. A token represents a unit of text including words punctuation and parts of words. These tokens are used to create a numerical embedding for input into the transformer. As of now the Mistral Large model charges $2 and $6 per million input and output tokens respectively. For context, Leo Tolstoy's War and Peace, approximately 580,000 words, translates to around 750,000 input tokens using standard AI tokenisers.

Practicalities

Weirholme Software is utilising LLMs from Mistral AI only.

Mistral AI is a French company that closely adheres to European data protection regulations (GDRP and EU AI Act). This is often a critical consideration for the application of AI when targeting European or UK customers.

We provide the necessary integration to enable a seamless user experience between your system and Mistral AI. Please make an initial contact to see if we have a potential solution for your idea.