Local Large Language Model

Starting from version 4.0, the Xliff Tool for Xcode has integrated a local large language model, enabling a host of new capabilities. Here's a guide to installing this local large language model. For detailed usage instructions, consult the "Best Practices " documentation.

Local Large Language Model Installation

The Xliff Tool employs Ollama as its local large language model platform, capable of hosting a variety of large language models. Installation can be done via Homebrew or by downloading directly from the official website.

Homebrew Installation

brew install ollama

Note: Homebrew offers two versions of Ollama, with one available under the Cask category. Both versions provide the core functionality of Ollama. However, the Cask version includes an additional standalone application that operates in the background and features a persistent icon in the system's menu bar. The non-Cask version is recommended for its minimal resource usage.

Operating Ollama

Post-installation, you must launch Ollama before proceeding with the installation of specific models. The process is straightforward; simply execute the following command:

ollama serve

This method of operation is advised as it consumes the fewest resources and can be halted at any moment with a ctrl+c command.

Once Ollama is running, the current terminal session will be dedicated to it. To open a new terminal, use the cmd+n shortcut.

For future invocations of Ollama, remember to initiate it in this manner to allow applications to interact with it.

Model Installation

After setting up Ollama, you'll need to install the appropriate model. Here are some considerations when selecting a model:

1. Larger models consume more memory and may operate more slowly upon invocation. However, they generally offer higher accuracy.
2. Models up to 7 billion parameters (7B) require at least 8GB of RAM, while those of 13 billion parameters (13B) necessitate 16GB of RAM.
3. Not every model is compatible with the functionalities of the Xliff Tool. It's suggested to start with the recommended models and explore other options during your downtime.
4. The installation process is designed to support resumable downloads. If the download speed is unsatisfactory, you can interrupt it with ctrl+c and then use the up arrow key to reissue the installation command, potentially improving the download speed.

Using neural-chat as an example, the installation command is:

ollama run neural-chat

Upon completion, you'll be presented with a command-line interface for interaction. To exit, simply type "/bye".

When utilizing Ollama through the Xliff Tool, there's no need to manually run the model. Instead, initiate the Ollama service with "ollama serve".