Cultivate your career with expert-led programs, job-ready certificates, and 10,000 ways to grow. All for $25/month, billed annually. Save now
Instructors: Noah Gift
Included with 
Recommended experience
Beginner level
To get the most out of this course, learners should have:
Experience with command line interfaces and running programs from the terminal
Recommended experience
Beginner level
To get the most out of this course, learners should have:
Experience with command line interfaces and running programs from the terminal
Learn how to serve large language models as production-ready web APIs using the llama.cpp framework
Understand the architecture and capabilities of the llama.cpp example server for text generation, tokenization, and embedding extraction
Gain hands-on experience in configuring and customizing the server using command line options and API parameters
Add to your LinkedIn profile
4 assignments
Add this credential to your LinkedIn profile, resume, or CV
Share it on social media and in your performance review
Learners will gain the skills to serve powerful language models as practical and scalable web APIs. They will learn how to use the llama.cpp example server to expose a large language model through a set of REST API endpoints for tasks like text generation, tokenization, and embedding extraction.
The course dives into the technical details of running the llama.cpp server, configuring various options to customize model behavior, and efficiently handling requests. Learners will understand how to interact with the API using tools like curl and Python, allowing them to integrate language model capabilities into their own applications. Throughout the course, hands-on exercises and code examples reinforce the concepts and provide learners with practical experience in setting up and using the llama.cpp server. By the end, participants will be equipped to deploy robust language model APIs for a variety of natural language processing tasks. The course stands out by focusing on the practical aspects of serving large language models in production environments using the efficient and flexible llama.cpp framework. It empowers learners to harness the power of state-of-the-art NLP models in their projects through a convenient and performant API interface.
This week, you run language models locally. Keep data private. Avoid latency and fees. Use Mixtral model and llamafile.
8 videos17 readings4 assignments1 discussion prompt4 ungraded labs
Duke University has about 13,000 undergraduate and graduate students and a world-class faculty helping to expand the frontiers of knowledge. The university has a strong commitment to applying knowledge in service to society, both near its North Carolina campus and around the world.
Duke University
Course
University of Michigan
Course
Duke University
Specialization
Unlimited access to 10,000+ world-class courses, hands-on projects, and job-ready certificate programs - all included in your subscription
Earn a degree from world-class universities - 100% online
Upskill your employees to excel in the digital economy
Access to lectures and assignments depends on your type of enrollment. If you take a course in audit mode, you will be able to see most course materials for free. To access graded assignments and to earn a Certificate, you will need to purchase the Certificate experience, during or after your audit. If you don't see the audit option:
The course may not offer an audit option. You can try a Free Trial instead, or apply for Financial Aid.
The course may offer 'Full Course, No Certificate' instead. This option lets you see all course materials, submit required assessments, and get a final grade. This also means that you will not be able to purchase a Certificate experience.
When you purchase a Certificate you get access to all course materials, including graded assignments. Upon completing the course, your electronic Certificate will be added to your Accomplishments page - from there, you can print your Certificate or add it to your LinkedIn profile. If you only want to read and view the course content, you can audit the course for free.
You will be eligible for a full refund until two weeks after your payment date, or (for courses that have just launched) until two weeks after the first session of the course begins, whichever is later. You cannot receive a refund once you’ve earned a Course Certificate, even if you complete the course within the two-week refund period. See our full refund policy.
Yes. In select learning programs, you can apply for financial aid or a scholarship if you can’t afford the enrollment fee. If fin aid or scholarship is available for your learning program selection, you’ll find a link to apply on the description page.
Financial aid available,
New to Coursera?
Having trouble logging in? Learner help center
This site is protected by reCAPTCHA Enterprise and the Google Privacy Policy and Terms of Service apply.
