Tagged local-llm

Add a new model architecture to llama.cpp with a guided step-by-step workflow

Review llama.cpp diffs and PRs against project conventions and common pitfalls

Format chat prompts for local model templates and route queries to matching patterns

Compare cost per million tokens for local model servers against hosted APIs

Configure local LLM inference with MicroLoRA and SONA adapters for air-gapped use

Run local Ollama models, chat and create embeddings from the command line

Measure local model task latency, token use and errors with Ollama telemetry

Quantize models to GGUF from 2 to 8 bits for llama.cpp on consumer hardware

Run LLM inference on CPUs, Apple Silicon and consumer GPUs with llama.cpp

Run MiniCPM5 GGUF models on CPU or consumer GPUs with llama.cpp

Chat with MiniCPM5 in LM Studio using GGUF or Apple MLX and a local server

Run MiniCPM5 natively on Apple Silicon Macs with the MLX framework

Run MiniCPM5 on a laptop with Ollama using the released GGUF build

Train a LoRA adapter for MiniCPM5-1B and convert it to GGUF for llama.cpp

Run GGUF models locally with llama.cpp and pick quantizations for your hardware

Benchmark LLM and VLM serving on Jetson across vLLM, llama.cpp and Ollama

Configure TanStack AI provider adapters like OpenAI, Anthropic, Gemini and Ollama

Transcribe local audio, video and Voice Memos quickly with cached MLX Whisper

Run llama.cpp inference, convert GGUF models and quantize them for local deployment

Delegate a coding task to Aider and review its diff before landing it

Pick the right hosted Ollama cloud model for a prompt and call it

Test Jupyter, MLflow, Ray, Gradio and Ollama servers for missing authentication

Burn word-level karaoke captions into video with MLX Whisper, ASS subtitles and FFmpeg

Use MLX Swift arrays, lazy evaluation and autodiff for Apple Silicon machine learning

Run LLMs and vision models on Apple Silicon with MLX Swift, including LoRA tuning

Build on-device AI features on Apple devices with Foundation Models or Core ML

Turn a Mac mini into an always-on private local LLM server with remote access

Run local LLMs with Ollama and Open WebUI for private dev environments