Nextcloud Assistant — Nextcloud latest Administration Manual latest documentation

# Nextcloud Assistant

Nextcloud assistant is the primary graphical user interface for interacting with artificial intelligence features in Nextcloud.

It offers the graphical user interface for the unified AI Task processing API offering features like summarizing text, generating headlines, asking arbitrary questions, transcription of media files, image generation and it integrates with the context_chat app to offer in-context answers about your own data stored in Nextcloud. The assistant app also offers a chat interface to interact with the chosen language model. Nextcloud can provide customer support upon request, please talk to your account manager for the possibilities.

Find the user documentation here: https://github.com/nextcloud/assistant/tree/main/docs/user

## Installation

You can install the assistant app via the “Apps” page in Nextcloud, or by executing

    sudo -E -u www-data php occ app:enable assistant

## App store

You can also find the app in our app store, where you can write a review: https://apps.nextcloud.com/apps/assistant

## Repository

You can find the app’s code repository on GitHub where you can report bugs and contribute fixes and features: https://github.com/nextcloud/assistant

Nextcloud customers should file bugs directly with our Customer Support.

## Related apps

Artificial intelligence at Nextcloud is built in a modular way, allowing you to choose from a variety of solutions for your needs. In order to make use of the various features of the Assistant you will need additional apps that act as backends to provide the actual implementation of the AI functionality. In the Nextcloud administration settings under “Artificial intelligence” you can select which AI backend app to use for which tasks. Note that some of the backend apps are only community maintained, while others are available for Customer support upon request.

The AI admin settings will show all types of Assistant Tasks that are implemented by all your installed apps. Task types can be disabled in the AI admin settings so they are not available for the Assistant or other apps even if they are implemented. All implemented Task types are enabled by default.

Note: At Nextcloud we focus on creating on-premise AI apps that run fully self-hosted on your own servers in order to preserve your privacy and data sovereignty. However, you can also offload these resource-heavy tasks to an “AI as a Service” provider.

Note: When using our on-premise AI apps, make sure you have a GPU with enough VRAM that fits all the features you need. For each app documented here you will find its hardware requirements.

### Machine translation

In order to make use of machine translation features in the assistant, you will need an app that provides a translation backend: * translate2 (ExApp) - Runs open source AI translation models locally on your own server hardware (Customer support available upon request) * integration_deepl - Integrates with the deepl API to provide translation functionality from Deepl.com servers (Only community supported)

### Speech-To-Text

In order to make use of Speech-to-Text, you will need an app that provides a Speech-To-Text backend:

- stt_whisper2 - Runs open source AI Speech-To-Text models on your own server hardware (Customer support available upon request)

- OpenAI and LocalAI integration (via OpenAI API) - Integrates with the OpenAI API to provide AI functionality from OpenAI servers (Customer support available upon request; see AI as a Service)

### Text processing

In order to make use of text processing features in the assistant, you will need an app that provides a Text processing backend:

- llm2 - Runs open source AI language models locally on your own server hardware (Customer support available upon request)

- OpenAI and LocalAI integration (via OpenAI API) - Integrates with the OpenAI API to provide AI functionality from OpenAI servers (Customer support available upon request; see AI as a Service)

- integration_watsonx - Integrates with the IBM watsonx.ai API to provide AI functionality from IBM Cloud servers (Customer support available upon request; see AI as a Service)

These apps currently implement the following Assistant Tasks:

- Generate text (Tested with OpenAI GPT-3.5 and Llama 3.1 8B)

- Summarize (Tested with OpenAI GPT-3.5 and Llama 3.1 8B)

- Generate headline (Tested with OpenAI GPT-3.5 and Llama 3.1 8B)

- Extract topics (Tested with OpenAI GPT-3.5 and Llama 3.1 8B)

- Context write (Tested with OpenAI GPT-3.5 and Llama 3.1 8B)

- Reformulate text (Tested with OpenAI GPT-3.5 and Llama 3.1 8B)

These tasks may work with other models, but we can give no guarantees.

### Text-To-Image

In order to make use of Text-To-Image features, you will need an app that provides an image generation backend:

- tex2image_stablediffusion2 (Customer support available upon request)

- OpenAI and LocalAI integration (via OpenAI API) - Integrates with the OpenAI API to provide AI functionality from OpenAI servers (Customer support available upon request; see AI as a Service)

- integration_replicate - Integrates with the replicate API to provide AI functionality from replicate servers (see AI as a Service)

### Context Chat

In order to make use of our special Context Chat feature, offering insights about your own documents and data stored in Nextcloud, you will need the following apps:

- context_chat + context_chat_backend - (Customer support available upon request)

You will also need a text processing provider as specified above (ie. llm2, integration_openai or integration_watsonx).

### Chat

In order to make use of our “Chat with AI” feature you will need any one of the following apps:

- llm2 - Runs open source AI language models locally on your own server hardware (Customer support available upon request)

- OpenAI and LocalAI integration (via OpenAI API) - Integrates with the OpenAI API to provide AI functionality from OpenAI servers (Customer support available upon request; see AI as a Service)

### Voice Chat

In order to make use of our “Audio chat” feature which allows you to interact with the assistant Chat via your Voice and Ears as if in a real conversation, you will need any of the following set of apps:

- Either

    - OpenAI and LocalAI integration (via OpenAI API) - Integrates with the OpenAI API to provide AI functionality from OpenAI servers (Customer support available upon request; see AI as a Service)

- Or

    - llm2 - Runs open source AI language models locally on your own server hardware (Customer support available upon request)

    - stt_whisper2 - Runs open source AI Speech-To-Text models on your own server hardware (Customer support available upon request)

    - text2speech_kokoro - Runs open source AI Text-To-Speech models on your own server hardware (Customer support available upon request)

### Context Agent

In order to make use of our AI agent feature, offering the execution of actions on behalf of the user based on the AI chat, you will need the following apps:

- context_agent - (Customer support available upon request)

You will also need a text processing provider as specified above (ie. llm2 or integration_openai).

### Text-To-Speech

In order to make use of Text-To-Speech, you will need an app that provides a Text-To-Speech backend, which is one of the following:

- OpenAI and LocalAI integration (via OpenAI API) - Integrates with the OpenAI API to provide AI functionality from OpenAI servers (Customer support available upon request; see AI as a Service)

- text2speech_kokoro - Runs open source AI Text-To-Speech models on your own server hardware (Customer support available upon request)

## Configuration

The Assistant admin settings can be found under the “Artificial intelligence” section. You can disable the assistant top menu entry there. You can also disable the AI-related smart pickers. The occ commands to change the options are listed below.

### Assistant configuration

1. Top-right Assistant

    occ config:app:set assistant assistant_enabled --value=1 --type=string

To enable/disable the assistant button from the top-right corner for all the users.

2. AI text generation smart picker

    occ config:app:set assistant free_prompt_picker_enabled --value=1 --type=string

To enable/disable the AI text generation smart picker for all the users.

3. Text-to-image smart picker

    occ config:app:set assistant text_to_image_picker_enabled --value=1 --type=string

To enable/disable the text-to-image smart picker for all the users.

4. Speech-to-text smart picker

    occ config:app:set assistant speech_to_text_picker_enabled --value=1 --type=string

To enable/disable the speech-to-text smart picker for all the users.

### Task processing

1. List Tasks

    occ taskprocessing:task:list

lists all task processing tasks.

2. Get Task

    occ taskprocessing:task:get $TASK_ID

shows all information for a specific task.

3. Enable or disable a Task type

    occ taskprocessing:task-type:set-enabled $TASK_TYPE_ID 1

Set 1 to enable and 0 to disable an implemented task type.

4. Get Task statistics

    occ taskprocessing:task:stats

shows statistics for all task processing Tasks.

5. Clean-up old tasks

    occ taskprocessing:task:cleanup

delete tasks that are older than this number of seconds, defaults to 4 months.

### Image storage

Days until generated images are deleted if they are not viewed.

    occ config:app:set assistant max_image_generation_idle_time --value=90 --type=integer

### Chat with AI

1. Chat User Instructions for Chat Completions

    occ config:app:set assistant chat_user_instructions --value="hello world"

The user instructions that are prepended before the chat messages for the AI model to understand the context of the block of text. This is a good place not only to instruct the AI model to be polite and kind but also to for example answer all the queries in a particular language or better yet, follow the user’s language. The sky is the limit.

Note: The default instructions are optimized to work well across a variety of language models, but may not be optimal for the specific model you choose. Specifically, the model may be tempted to mention the user’s name a bit too often and may mention the user’s language in an unusual manner.

2. Chat User Instructions for Title Generation

    occ config:app:set assistant chat_user_instructions_title --value="hello title"

This field is appended to the block of chat messages, i.e. attached after the messages. It is done this way to allow it to be used even with text completion models which could have the instructions as “The title for the above conversation could be "”.

3. Last N messages to consider for chat completions

    occ config:app:set assistant chat_last_n_messages --value=10

The number of latest messages to consider for generating the next message. This does not include the user instructions, which is always considered in addition to this. This value should be adjusted in case you are hitting the token limit in your conversations too often. The AI text generation provider should ideally handle the max token limit case.

### Improve AI task pickup speed

See the relevant section in AI Overview for more information.

☁️ latest

☁️ Versions
  latest
  stable
  32
  31