What is an LLM?
LLMs, or Large Language Models, are an exciting type of machine learning technology model that perform language processing tasks and allow us to interact with large data models in ways that are far more familiar to us. From the point of view of a computer or mobile phone user, we can interact directly with LLMs through “prompts”, often in the form of chat bot frontends (whether typed or spoken), to ask questions of and receive answers back in natural language or other forums, such as spoken words or pictures. These can also be used indirectly, with our writing tools, web browsers, software development environments, office software, databases, CRMs and ERPs, and countless other information processing systems where LLMs can find patterns in our work, and return results that fulfil common needs or repetitive tasks.
LLMs are the natural progression of search engines. The Internet was once a tedious and manually curated collection of links. The explosion of search tools from Altavista to Yahoo to Google saw the web become a powerful tool for people looking for a wider variety of information more quickly. LLMs can now help us build better contextual links and reduce the need to perform repetitive tasks across not only data on the web, but even the data we create, store and interact with daily.
While it’s true that neural networks, tensor transformations and machine learning have been specialised fields of computer science for decades, what is new and exciting in recent years is the explosion of processing power and storage volume we have access to today to utilise this technology at scales that makes it extremely valuable to an enormous range of people. Whether for ourselves, our co-workers or our customers, the methods in which LLMs can minimise rote and repetitive tasks and increase efficiency has exciting potential.
Who makes LLMs?
Like any software, LLMs are released under various licenses. Some are proprietary and closed, some are on offer as-a-service only, and some are entirely free and open source.
Can I run these open source LLMs for free myself?
Absolutely. Tools like Ollama, Open-WebUI and others make privately hosting a wide range of free and open source LLMs quite easy. However there is a catch – especially for larger models, powerful hardware is required to make the most of this experience. Why you can absolutely load up smaller models on your laptop computer, or even some medium sized models on a powerful workstation or gaming PC, the largest models require powerful specialised processing hardware to enable answers to come back in seconds rather than minutes. GPUs are especially well suited to the task, with their ability to rapidly process the matrix functions required, and GPU-as-a-service vendors, such as Sharon AI, allow you to build completely private and fully customised environments for your own needs.
Wouldn’t it be easier to just use an LLM service provider?
LLMs are not unlike any other software. There can be great benefit to having a third party manage all aspects of building, owning and hosting the model, its processing, and of course a complete history of everything loaded in to or asked of that model. But that also presents some serious privacy and security risks. The questions it has been asked, the documents that were uploaded to it, the complete history of every interaction you, your co-workers, your employers or employees have with the LLM, and perhaps even customer data – all if this is transmitted into the LLM service provider’s tools and stored there with no guarantee of their safety or security. And often these providers exist in jurisdictions outside of your country of business.
This might be fine for the casual user, but for individuals wanting to summarise personal health data, businesses wanting to analyse customer data, people or businesses wanting help with financial information, researchers wanting to speed up work on sensitive data – all of these scenarios present considerable risk.
Host your own private LLM – the benefits
Hosting your own LLM on a platform such as Sharon AI brings considerable benefits:
Privacy first
Build your tools with only the software you want to install, give access to the data to only the people and systems you authorise. Apply the security controls that meet your needs – whether personal, professional, or legislative. Under a “zero trust” model, ensure that data, whether it belongs to you or your customer, meets all of the security and privacy requirements placed on it without the need to trust third party vendors or service providers.
Data sovereignty
Sharon AI servers are hosted on Australia soil in tier 4 datacenters in Sydney and Melbourne. You are in total control of where that data goes, and who has access. Servers can be access controlled by industry standard network controls and data encryption (both at rest and in transit).
Prevent ransom attacks
With full control over the running instance and storage, when the data is no longer required, it can be securely destroyed. Threats of ransomware or private information breach vanish.
Visibility and auditing
Apply full application, network and access level logging to prove exactly where your data is in real-time, and access audit logs to see past events. Give immediate access to security teams, auditors or customer queries where information security is required by policy or required by law.
Trust, but verify
With self-hosted, open source tools and large language models, you can inspect the software all the way to source code level if required. Network data safety can be inspected, and information storage, access and destruction can be reliably demonstrated.
What industries could benefit from private LLMs?
LLMs present enormous opportunity for task and workflow automation, information summarisation, research and more. When LLMs are hosted by foreign companies with limited privacy policies, these are immediately out of reach for many organisations with strict compliance or privacy guidelines. The following industries are excellent examples of those that could benefit from private LLMs:
Education
Allow students to experiment safely with LLMs. Safely manage their interactions and digital footprint
Allow researchers to summarise vast amounts of material quickly, or have access to complex machine learning algorithms to speed up experiment results
Allow faculty to utilise the time-saving benefits of LLMs without diminishing student privacy
Finance
Use LLMs to assist finding patterns in confidential finance data while still applying necessary access controls, encryption and auditing
Place friendly chat-bot interfaces on customer web portals without breaching legislative requirements
Apply private data lifecycle management policies and have certainty that they are adhered to
Government
Utilise LLM driven results on council or government websites to help your communities find the information they need quickly and securely
Summarise lengthy or historical documents quickly without risking confidential data leakage
Medical
Summarise patient records or lengthy email chains without risk of privacy breaches
Enable medical research access to powerful LLM and ML tools while meeting privacy and compliance requirements and legislation
Engineering, mining and manufacturing
Enable the mass scale data processing benefits of LLMs while keeping proprietary data safe
Minimise ransomware attack and data breach opportunities with guaranteed data lifecycle management
Information and Cyber Security
Quickly find patterns in logs from firewalls, malware scanners or SIEM software while guaranteeing privacy
Software development
Enable rapid auto-completion of repetitive development tasks without risking proprietary code
Scan existing code bases for security vulnerabilities, accidental private key exposure, or other common security risks
