Understanding AI 101: Large Language Model (LLM)

What is a LLM?

A LLM, or large language model, is a type of language model (LM) that has more parameters compared to traditional language models. LMs are trained to analyse bodies of text and then perform text prediction using statistics and probabilities. Traditional LMs can only perform one task each (e.g. Google Translate) while LLMs are able to perform a variety of tasks (e.g. ChatGPT).

How are LLMs trained?

LLMs are pre-trained on a larger amount of datasets or corpus than LMs hence the word ‘large’. They then undergo fine-tuning or are re-trained for a specific purpose. Whether that’s to simply ensure the LLM knows not to echo prejudiced content, to hone its capability to perform a specific task, or to feed it custom datasets. Fine-tuning improves accuracy, relevance and performance as well as reduces size and complexity thereby making LLMs faster, cheaper, and easier to maintain.

What can LLMs do?

Content generation, question and answering, summarization, translation, speech recognition, simple coding, conversation, sentiment analysis, text suggestion, and parsing are examples of tasks that LLMs are capable of performing. The proficiency of LLMs is dictated by the datasets or corpus that they are pre-trained with which can be as expansive as the internet, as dedicated as academic papers, as specific as one organization’s data bank, or as well-rounded as all of the above.

With such a wide range of capabilities, LLMs can be helpful in various aspects of business.

For example, it can improve the relevance of search results with its understanding of context; this is useful for customers browsing a company’s website and for healthcare professionals to use as a quick reference. It can assist with content moderation by efficiently filtering offensive language, hate speech, spam and more. It can bolster security by detecting and classifying malware. Or it can support developers with the use of simple coding to complete, synthesize, explain and secure code.

What can’t LLMs do?

Although LLMs are extremely powerful, they – as all technology do – have limitations and challenges. Present limitations generally revolve around the fact that LLMs can only mimic human intelligence and not replicate the consciousness and intelligence unique to human beings. Meaning, they don’t have reasoning abilities, can’t truly understand what they’re saying, can’t plan and can’t think systematically. LLMs might also reflect harmful biases or present dated information as recent.

Present challenges are more in the practical sense and cover the LLM’s journey from creation to operation. To start, LLMs take a very long time to train and require a ton of data as well as computational power before launch. After launch, they still require a lot of computational power and the issue of their access to sensitive personal information through its corpus also arises as personal data risk. Moving forward, their corpus would need to be updated which repeats the cycle.

There are also a few challenges to practical usage outside of stale training data should LLMs not be updated and these come about due to its size. Because LLMs are pre-trained with large corpuses, they are difficult to completely fine-tune so there are instances in which “hallucinations” or false responses might occur. And, depending on whether they were properly re-trained on custom datasets, LLMs might not be able to give accurate answers to business or industry-specific queries.

Future of LLM

It is important to note that the sciences behind LLMs are constantly evolving and various solutions as well as alternatives are being tested to make LLMs more efficient, practical and economical. The focus of ongoing efforts revolve around optimization and quantization, scaling and complexity, multi-modality, accuracy and efficiency, and transparency. While approaches being considered are: enabling LLMs to perform fact-verification; using synthetic training data as a faster, cheaper and more diverse alternative; and reducing computational requirements with fewer parameters or layers.