科学文献可视化

Install & run

The Raven constellation consists traditional desktop apps. It needs to be installed.

Currently, this takes the form of installing the app and dependencies into a venv (virtual environment). At least at this stage of development, app packaging into a single executable is not a priority.

Raven is developed and tested on Linux Mint. It should work in any environment that has bash and pdm.

It has been reported to work on Mac OS X, as well as on Windows (with Miniconda to provide Python).

Install PDM in your Python environment

Raven uses PDM to manage its dependencies. This allows easy installation of the app and its dependencies into a venv (virtual environment) that is local to this one app, so that installing Raven will not break your other apps that use machine-learning libraries (which tend to be very version-sensitive).

Note that in contrast to many AI/ML apps, which use conda to manage the venv for the app, Raven instead uses PDM. The venv creation and management for the app is automatic, but you need a Python environment to run PDM in. That Python environment is used for running PDM only. Raven itself will run in the venv created automatically by PDM, which may even have a Python version different from that of the environment where PDM runs.

If your Python environment does not have PDM, you will need to install it first:

python -m pip install pdm

Don't worry; it won't break pip, poetry, uv, or other similar tools.

Install Raven via PDM

Then, to install Raven, in a terminal that sees your Python environment, navigate to the Raven folder.

We will next initialize the new venv, installing the required Python version into it. This Python will be available for PDM venvs, and is independent of Python that PDM itself runs on.

Raven is currently developed against the minimum supported Python version, so we recommend to install that version, like this:

pdm python install --min

The venv will be installed in the .venv hidden subfolder of the Raven folder.

Then, install Raven's dependencies as follows. (If you are a seasoned pythonista, note that there is no requirements.txt; the dependency list lives in pyproject.toml.)

Basic install without GPU compute support

pdm install

This may take a while (several minutes).

Now the installation should be complete.

Install with GPU compute support

:exclamation: Currently GPU compute support requires an NVIDIA GPU and CUDA. :exclamation:

:exclamation: Using CUDA requires the proprietary NVIDIA drivers, also on Linux. :exclamation:

:exclamation: Currently Raven uses CUDA 12.x. Make sure your NVIDIA drivers support this version. :exclamation:

pdm install --prod -G cuda

If you want to add GPU compute support later, you can run this install command on top of an already installed Raven.

Installing dependencies may take a long time (up to 15-30 minutes, depending on your internet connection), because torch and the NVIDIA packages are rather large (my .venv shows 11.1 GB in total).

Now the installation should be complete.

Install on an Intel Mac with MacOSX 10.x

Installing Raven may fail, if Torch cannot be installed.

On MacOSX, installing torch 2.3.0 or later requires an ARM64 processor and MacOSX 11.0 or later.

If you have an Intel Mac (x86_64) with MacOSX 10.x, to work around this, you can use Torch 2.2.x.

To do this, modify Raven's pyproject.toml in a text editor, so that the lines

    "torch>=2.4.0",
    "torchvision>=0.22.0",

become

    "torch>=2.2.0,<2.3.0",
    "torchvision>=0.17.2",

Also, ChromaDB requires onnxruntime, which doesn't seem to be installable on this version of OS X. This means Raven-librarian and Raven-server won't work (as the RAG backend and the server's embeddings module require ChromaDB), but you can still get Raven-visualizer to work, by removing ChromaDB. Run this command in the terminal:

pdm remove chromadb

Then run pdm install again.

:exclamation: In general, if a package fails to install, but is not explicitly listed in the dependencies, you can try to find out which package pulls it in, by issuing the command pdm list --tree. This shows a tree-structured summary of the dependencies. :exclamation:

Install on Windows (if Windows Defender gets angry)

Installing Raven does not need admin rights.

• Raven can be installed as a regular user.
• We recommend Miniconda as the Python environment.
• The only exception, that does need admin rights, is installing espeak-ng, so the TTS (speech synthesizer) can use that as its fallback phonemizer.
• Raven only ever calls espeak-ng from Raven-server's tts module, and only for those inputs for which the TTS's built-in Misaki phonemizer fails.
• In practice, that is for out-of-dictionary words in English, as well as for some non-English languages.

Using Raven does not need admin rights.

• All the apps are regular userspace apps that you can run as a regular user.

If you get a permission error when trying to run pdm, try replacing "pdm" with "python -m pdm".

For example, instead of:

pdm install

run the command:

python -m pdm install

This works because PDM is just a Python module. This will be allowed to run if python is allowed to run.

Similarly, Raven apps are just Python modules, and can be run via Python, as follows. Full list as of Raven v0.2.5:

Command                                Replacement

raven-visualizer                  →    python -m raven.visualizer.app
raven-importer                    →    python -m raven.visualizer.importer
raven-librarian                   →    python -m raven.librarian.app
raven-xdot-viewer                 →    python -m raven.xdot_viewer.app
raven-conference-timer            →    python -m raven.conference_timer.app
raven-arxiv2id                    →    python -m raven.papers.identifiers
raven-arxiv-download              →    python -m raven.papers.download
raven-arxiv-search                →    python -m raven.papers.search
raven-burstbib                    →    python -m raven.papers.burstbib
raven-dehyphenate                 →    python -m raven.tools.dehyphenate
raven-csv2bib                     →    python -m raven.papers.csv2bib
raven-wos2bib                     →    python -m raven.papers.wos2bib
raven-pdf2bib                     →    python -m raven.papers.pdf2bib
raven-server                      →    python -m raven.server.app
raven-avatar-settings-editor      →    python -m raven.avatar.settings_editor.app
raven-avatar-pose-editor          →    python -m raven.avatar.pose_editor.app
raven-check-cuda                  →    python -m raven.tools.check_cuda
raven-check-audio-devices         →    python -m raven.tools.check_audio_devices
raven-minichat                    →    python -m raven.librarian.minichat

Pin vsync to the right display on multi-monitor setups (NVIDIA + X11)

If you use multiple displays at different refresh rates (e.g. a 60 Hz external monitor alongside a 144 Hz laptop panel), Raven's GUI apps may run at the wrong refresh rate even though vsync is enabled. The symptom: raven-librarian (or any other Raven app) reports e.g. 144 FPS (in its Ctrl+Shift+M metrics debug window) while its window is actually on a 60 Hz display, wasting large amounts of CPU and GPU.

This is a quirk of the NVIDIA proprietary driver under X11: it picks one display to vsync to (by default the X11 primary), regardless of which display the window is actually on. Wayland handles per-output refresh rates correctly.

To pin vsync to the right display on X11, set __GL_SYNC_DISPLAY_DEVICE to the output name (as reported by xrandr --query) before launching the app:

__GL_SYNC_DISPLAY_DEVICE=DP-1 raven-librarian

Alternatively, make the desired display your X11 primary:

xrandr --output DP-1 --primary

The 4K external desktop monitor is a common case where this matters; on a single-display setup, no action needed.

Uninstall

python -m pip uninstall raven-visualizer

Or just delete the venv, located in the .venv subfolder of the Raven folder.

AI models auto-install themselves elsewhere:

• The THA3 AI animator (of Raven-avatar) is auto-installed in the raven/vendor/tha3/models/ subdirectory of your top-level raven directory.

• The dehyphenator AI model (of Raven-server's sanitize module) is auto-installed in ~/.flair/embeddings/.

• All other AI models are auto-installed from HuggingFace Hub.
• These live at the default models cache location of the huggingface_hub Python package, which is usually ~/.cache/huggingface/hub.
• Note that this models cache is shared between many different Python-based AI apps, so removing everything is not recommended.

Quickstart

To start the server, the basic command sequence is:

$(pdm venv activate)  # activate Raven venv
source env.sh  # set up paths for CUDA libraries
raven-server

Other scripts that can be sourced before starting raven-server:

source run-on-internal-gpu.sh  # set GPU that is visible to Torch as cuda:0

source no-hammer-hf.sh  # use the installed AI models without checking for new versions

Full sequence:

$(pdm venv activate)
source env.sh
source run-on-internal-gpu.sh
source no-hammer-hf.sh
raven-server

Check that CUDA works (optional)

Once you have installed Raven with GPU compute support, you can check if Raven detects your CUDA installation:

raven-check-cuda

This command will print some system info into the terminal, saying whether it found CUDA, and if it did, which device CUDA is running on.

It will also check whether the cupy library loads successfully. This library is needed by the spaCy natural language analyzer (so that too can run on GPU).

Example output:

INFO:raven.tools.check_cuda:Raven-check-cuda version 0.2.3
Checking dependencies...
1. PyTorch availability check [SUCCESS] ✅
2. CUDA device availability check [SUCCESS] ✅ (Using NVIDIA GeForce RTX 3070 Ti Laptop GPU)
3. CuPy & CuPyX (for spaCy NLP) [SUCCESS] ✅

System information:
   Python version: 3.10.12
   OS: Linux 6.8.0-109049-tuxedo
   PyTorch version: 2.7.0+cu126

Activate the Raven venv (to run Raven commands such as `raven-visualizer` or `raven-server`)

In a terminal that sees your Python environment, navigate to the Raven folder.

Then, activate Raven's venv with the command:

$(pdm venv activate)

Note the Bash exec syntax $(...); the command pdm venv activate just prints the actual internal activation command.

:exclamation: Windows users note: The command $(pdm venv activate) needs the bash shell, and will not work in most Windows command prompts. :exclamation:

Alternatively, you can run the venv activation script directly. You can find the script in .venv/bin/.

:exclamation: For Linux and Mac OS X, the script is typically named .venv/bin/activate; for Windows, typically .venv/bin/activate.ps1 or ./venv/bin/activate.bat. :exclamation:

Whenever Raven's venv is active, you can use Raven commands, such as raven-visualizer.

Activate GPU compute support (optional)

If CUDA support is installed but not working, you can try enabling CUDA (for the current command prompt session) as follows.

With the venv activated, and the terminal in the Raven folder, run the following bash command:

source env.sh

This sets up the library paths and $PATH so that Raven finds the CUDA libraries. This script is coded to look for them in Raven's .venv subfolder.

Choose which GPU to use (optional)

If your machine has multiple GPUs, there are two ways to tell Raven which GPU to use.

If your system permanently has several GPUs connected, and you want to use a different GPU permanently, you can adjust the device settings in raven.server.config, raven.visualizer.config, and raven.librarian.config.

If you switch GPUs only occasionally (e.g. a laptop that sometimes has an eGPU connected and sometimes doesn't), you can use the CUDA_VISIBLE_DEVICES environment variable to choose the GPU temporarily, for the duration of a command prompt session.

We provide an example script run-on-internal-gpu.sh, meant for a laptop with a Thunderbolt eGPU (external GPU), which forces Raven to run on the internal GPU when the external is connected (which is useful e.g. if your eGPU is dedicated for a self-hosted LLM). On the machine where the script was tested, PyTorch sees the eGPU as GPU 0 when available, pushing the internal GPU to become GPU 1. When the eGPU is not connected, the internal is GPU 0.

With the venv activated, and the terminal in the Raven folder, run the following bash command:

source run-on-internal-gpu.sh

Then for the rest of the command prompt session, any Raven commands (such as raven-visualizer) will only see the internal GPU, and "cuda:0" in the device settings will point to the only visible GPU.

Exit from the Raven venv (optional, to end the session)

:exclamation: There is usually no need to do this. You can just close the terminal window. :exclamation:

If you want to exit from the Raven venv without exiting your terminal session, you can deactivate the venv like this:

deactivate

After this command completes, python again points to the Python in your Python environment (where e.g. PDM runs), not to Raven's app-local Python.

If you want to also exit your terminal session, you can just close the terminal window as usual; there is no need to deactivate the venv unless you want to continue working in the same terminal session.

Configuration

Raven is currently mostly configured via text files - more specifically, Python modules (.py) that exist specifically as configuration files.

We believe that .py files are as good a plaintext configuration format as any, but in the long term, we aim to have a GUI to configure at least the most important parts.

In the meantime: each part of the Raven constellation has its own configuration file. Each configuration file is named config.py.

In the documentation as well as in the source code docstrings and comments, we refer to these files by their dotted module names. The most important ones are:

• raven.visualizer.config → raven/visualizer/config.py
• Raven-visualizer settings, including plotter and word cloud colors, and word cloud image size.
• Local AI model loading settings. Used if Visualizer is started when Server is not running.
• raven.librarian.config → raven/librarian/config.py
• Raven-librarian settings, including the AI avatar.
• LLM configuration for the whole Raven constellation: server URL, system prompt, AI personality settings, text generation sampler settings.
• The AI avatar has some more separate configuration:
• Avatar video postprocessor settings are configured separately, in raven/avatar/assets/settings/animator.json.
• Since finding nice-looking settings for the postprocessor requires interactive experimentation, we provide a GUI app for this. Use raven-avatar-settings-editor to edit animator.json.
• Avatar emotion templates are shared between all characters and configured separately, in raven/avatar/assets/emotions/*.json.
• There is usually no need to edit the emotion templates. But if you really want to, you can use the GUI app raven-avatar-pose-editor.
• Avatar image assets are loaded from raven/avatar/assets/characters/.
• The default character (Aria, main image aria1.png), contains an example of the additional cels needed to support all optional features of the animator, as well as the optional chat icon for Raven-librarian.
• The backdrop image is loaded from raven/avatar/assets/backdrops/.
• raven.server.config → raven/server/config.py
• AI model settings, except LLM.
• A low-VRAM variant is also available, for systems with 8 GB or less VRAM.
• raven.server.config_lowvram → raven/server/config_lowvram.py
• To use it, start Raven-server as raven-server --config raven.server.config_lowvram
• raven.client.config → raven/client/config.py
• Raven-server URL, shared between all client apps.
• Audio device selection for voice mode (TTS/STT, i.e. speech synthesizer and speech recognition).

The paths are relative to the top level of the raven repository (i.e. to the directory this README is in).

For more, see the documentation for the individual constellation components (Visualizer, Librarian, Server).

Raven-server: Web API server

<a href="raven/server/README.md"><img src="img/screenshot-server.png" alt="Screenshot of Raven-server" height="200"/></a>

• Documentation: Server user manual
• Goal: Run all GPU processing on the server, anywhere on the local network.
• Status: :white_check_mark: Fully operational.
• Features:
• AI components for natural language processing (NLP).
• Speech synthesizer (TTS), using Kokoro-82M.
• Speech recognition (STT), using whisper-large-v3-turbo.
• Server side of Raven-avatar.
• Partially compatible with SillyTavern. Originally developed as a continuation of SillyTavern-extras.
• Python bindings (client for web API) provided.
• JS bindings possible, but not implemented yet. See #2.