Frontends and software
- Oobabooga’s Text Generation Web-UI (includes several backends)
- SillyTavern (a frontend with lots of features) (You need to provide your own backend or online service. Also all the settings can be a bit overwhelming for a beginner.)
- KoboldCPP (easy to install, works without a high-end GPU) (Download models in GGUF format. Q4_K_M or Q5_K_M quantization is recommended.)
- KoboldAI
- Agnaistic (Github)
(There are lots more.)
Free of charge services
- Kobold Lite
- Google Colab offers Google users some free computing resources including GPU runtime.
Models (I recommend you start with one of these)
- Mistral-Nemo-Instruct-2409
- (Cydonia-22b)
- MythoMax-L2-13b
- Fimbulvetr-11B
- Velara 11B
- Psyfighter v2 13B
- Dolphin 2.1 Mistral 7B
- Toppy-M-7B
(This list is outdated.)
More models
- an up to date ranking is here: Ayumi LLM evaluation
Most models available in quantized GGUF format, pay attention to what you’re downloading. And choose a proper quantization level that fit’s onto your hardware. Usually that’d be 4-6 or 8bits.
Characters for roleplay
Character editors
- Character card editor (v1 format)
- Character card editor (v2 format)
- Guide (Don’t pay too much attention to the old guides telling you to use a specific format. That has become obsolete. Listing properties or plain normal text is fine nowadays. But make sure to make it clear, concise and without contradictions. LLMs aren’t too smart. And have a look at the prompt format of your model and recommended settings. That really matters.)
Guides
More info
- A model benchmark for ERP: https://rentry.org/ayumi_erp_rating and the LLM leaderboard (a general ‘intelligence’ benchmark)
- 4chan’s /g/ board, with /lmg/ and /aicg/
- Comparison between different (paid) services and frontends: https://rentry.org/aicg_meta
- Local Models Links
[I invite you to share and reuse my content. This text is licensed CC-BY 4.0]
deleted by creator