Featured Open Source Advanced Verified

llama.cpp

LLM inference in plain C/C++, fast enough to run on a laptop.

llama.cpp is the inference engine underneath a large share of the local-AI ecosystem. It implements transformer inference with no heavyweight runtime, adds aggressive quantisation via GGUF, and ships backends for CUDA, Metal, Vulkan and pure CPU. Reading it is one of the better ways to understand what actually happens during token generation.

C++ CUDA
0Views
0Likes
0Downloads
0Bookmarks
Source Code
invitationbuddy.com/ai-projects/llama-cpp
l
0GitHub Stars

About This Project

llama.cpp is the inference engine underneath a large share of the local-AI ecosystem. It implements transformer inference with no heavyweight runtime, adds aggressive quantisation via GGUF, and ships backends for CUDA, Metal, Vulkan and pure CPU. Reading it is one of the better ways to understand what actually happens during token generation.

Technology Stack (2)

C++ Open-source AI projects built with C++. CUDA Open-source AI projects built with CUDA.

Source Code

ggml-org/llama.cpp Open Source
C++ 0 Stars 0 Forks 0 Contributors MIT
View on GitHub Download ZIP

Project Metrics

Total Views0
Downloads0
GitHub Stars0
Avg Rating0.0/5

Project Reviews

No reviews yet — be the first to review this project.

Sign in to write a review.

Sign In