What Is Fine-Tuning?

Fine-tuning in machine learning and artificial intelligence (AI) is the process of taking an already trained, or pre-trained, model and training it on other datasets so it can perform specialized tasks. It allows for large language models (LLMs) and other generative AI tools to perform specific tasks, like image processing. Fine-tuning allows developers to save time and money by allowing them to use existing models rather than having to create new ones.

Explore fine-tuning, the ways you can fine-tune a model, what fine-tuning does, who uses it, its challenges and limitations, and how you can start fine-tuning your AI models. 

Types of fine-tuning

Fine-tuning LLMs and other machine learning models typically come in two types: feature extraction and full fine-tuning. Which of these is used depends on the needs of the practitioner. Each type uses various methods to achieve your final model’s goal. Learn more about these two types of fine-tuning and the methods associated with them below:

Feature extraction fine-tuning

Feature extraction fine-tuning, sometimes referred to as parameter-efficient fine-tuning, “freezes” the top layers of the model while the final layers train on the specified dataset. This keeps the majority of the existing model intact, making it a more computationally efficient process. Three kinds of feature extraction methods exist:

• Partial fine-tuning: This method follows the basic principles of feature extraction by only training the outer layers of the network. By only updating the parameters of the outer layers, the core of the model remains the same as it analyzes general features. However, the prediction becomes more accurate as the outer layers learn how to analyze the finer details. 

• Additive fine-tuning: In this method, the pre-trained parameters of the model are all frozen or kept the same, and new parameters are added to the model. All of the original model weights remain the same. The new functions added to the model save processing power.

• Reparameterization: This method, known as low rank adaptation (LoRA), uses different matrices to update the model weights separately without directly affecting the existing model. Instead, the new matrices are lower-rank matrices, which represent weight updates broken into smaller chunks. This allows for swapping various iterations of fine-tuned updates in and out of the model as needed. 

Full model fine-tuning

Full model fine-tuning is similar to the pre-training process in that the entire model trains on a dataset and updates the weights accordingly. This process updates the whole model, which could potentially lead to stability issues. To stabilize the model, you can tune hyperparameters like the learning rate to encourage more balanced growth. This fine-tuning method is more computationally intensive than feature extraction fine-tuning. It’s best suited to situations where the new dataset is both sizable and distinct from the original training dataset.

Reinforcement learning from human feedback 

Reinforcement learning from human feedback (RLHF) is a fine-tuning method that helps models learn about complex human behaviors like comedy or humor, which are not easily translated into straightforward mathematical data to train the model. This approach involves humans stepping in to correct the model at various stages in its development and is best suited to situations in which the model will eventually be interacting with users directly, such as a chatbot. 

What does fine-tuning do?

Fine-tuning a model requires the use of a specialized dataset smaller than the original pre-training dataset. Using one of the methods outlined above, the model develops new capabilities depending on which aspects the developer decides are important. While fine-tuning specializes the model’s abilities, developers have to be careful not to drastically change the pre-trained model's weights and can reduce this by choosing a slower learning rate.

Many neural networks initially train on a specific task, such as image processing. Fine-tuning gives these models specialized abilities within that field. For example, a model that already specializes in image processing may be adjusted to be able to identify bees more accurately in an image. Fine-tuning also makes the implementation of LLMs more useful. As another example, a pre-trained model might have a general grasp of English, but a law firm may fine-tune the model to specifically understand legal jargon. 

Who uses fine-tuning?

Organizations and AI developers use fine-tuning to adapt a model to their use case. For example a business may want to develop an application that identifies plants. That organization can work with developers to fine-tune an image identification model that specializes in identifying plants. Below are some other industries that could use fine-tuning:

• Customer service: Many LLM chatbots like ChatGPT have conversational abilities, but a customer service company may want to fine-tune a model’s ability to help customers with their company’s products and questions specific to their business. 

• Recommendations: A company in retail, e-commerce, or media may have a recommendation algorithm already in place. They could then utilize fine-tuning to tailor that model to their customers using browsing data, purchase history, and liked content. 

• Health care: Health care professionals may want an image processing model to learn about certain disease markers in medical scans. They could use fine-tuning to create a model that specializes in identifying those disease markers on the scans. 

Some other jobs that may implement fine-tuning include:

• AI engineers

• Software developers

• Data scientists

Learn more about each job, such as salary and job outlook. 

AI engineer

Median annual base pay (Glassdoor): $132,219 [1]

Job outlook (projected growth from 2023 to 2033): 26 percent [2]

AI engineers develop models using machine learning and deep learning. They may work with businesses, developers, data scientists, and others to build new AI models and manage existing models. They develop the entire infrastructure and environment that goes into the training of the model. 

Software developers

Median annual base pay (Glassdoor): $102,207 [3]

Job outlook (projected growth from 2023 to 2033): 17 percent [4]

Software developers create, design, and implement new computer programs to perform specific computing tasks. They design how each piece of the software works, then work with the team to implement the new software or upgrade older software. 

Data scientists

Median annual base pay (Glassdoor): $115,667 [5]

Job outlook (project growth from 2023 to 2033): 36 percent [6]

Data scientists develop algorithms, which aids in the implementation of machine learning models that make predictions. They test these algorithms and use or develop additional datasets to create more accurate models. 

Benefits and challenges of fine-tuning

Fine-tuning can be a beneficial process when creating specialized AI models because it can save you time and money, but the process comes with some challenges. Explore more about the benefits and challenges of fine-tuning. 

Benefits

Fine-tuning is an effective process that optimizes pre-trained models to perform better in specific use cases depending on your needs. Some other benefits of fine-tuning include:

• Fine-tuning costs less than creating an entire model for a specific task, saving a business time, computing power, and having to buy new hardware. 

• It creates improved accuracy in niche uses, combining the pre-trained model and refining its capabilities using a smaller dataset.

• It makes machine learning more accessible: Smaller organizations can take a pre-trained model and use their own specific dataset to customize the model for their use. 

Challenges

While it has many benefits, fine-tuning comes with some challenges, especially when it comes to keeping the learning of the pre-trained model intact. 

• Due to the small datasets used in fine-tuning, you can risk “overfitting” your model. An overfitted model is so attuned to the training data that it doesn’t perform well with new data.

• Fine-tuning a model can risk having the pre-trained model “forget” some of its original training. This happens when you target specific layers too heavily or use a dataset too different from the training set. 

• If the original pre-trained model has certain biases, flaws, or security issues, then your fine-tuned model will also have those issues.

How to get started with fine-tuning

Fine-tuning a model is an advanced machine learning technique that requires skills in deep learning and AI. Before you learn about fine-tuning, it's important to have an understanding of the machine learning process. Learn the basics before you start tuning:

• Explore the fundamentals of machine learning in the Supervised Machine Learning: Regression and Classification course from Stanford and DeepLearning.AI on Coursera. 

• Learn about some of the popular pre-trained models like GPT-4, BERT, RoBERTa, and PaLM, and the kinds of fine-tuning you can do on those models [7].

• Discover communities around machine learning and fine-tuning like TensorFlow and HuggingFace, which have extensive libraries of pre-trained models you can tune. 

• Try a guided project like the Transfer Learning for NLP with TensorFlow Hub on Coursera when you’re ready to start fine-tuning your first pre-trained model. 

Best practices for fine-tuning

Fine-tuning has some best practices associated with it:

1. Data preparation: When preparing your dataset, you must find quality data that fits your desired task. Then you clean the data, standardize and format it for your model, and properly label it if you’re using supervised learning. 

2. Model selection: Selecting your pre-trained model is key to creating your fine-tuned model. Select the model most closely associated with your task. For example, an LLM wouldn't be a good choice for an image processing task. Research the task’s probable data and computational requirements. 

3. Parameter selection: Explore the different parameters you need to tune as well as the layers you want to freeze. Identify which methods will most likely work best for your desired outcome. 

4. Fine-tuning: Run your fine-tuning by optimizing your hyperparameters, choosing a gradient descent algorithm, watching for biases, and identifying metrics you want to use to evaluate your model performance. 

5. Iteration: Continue to refine your model as you evaluate your results based on the metrics you set. You can adjust hyperparameters such as batch size, learning rate, and regularization until the model performs at an optimum level.

6. Deployment: Once the model performs to a standard you’ve set, deploy the model into the environment in which it’s meant to work. Take into account its computational requirements, security, and scalability as you continually monitor its real-world performance. 

Getting started with fine-tuning on Coursera

Fine-tuning is a core component of making deep learning applications more accessible for specialized use cases. If you’re looking to learn more about fine-tuning and deep learning, try the Finetuning Large Language Models project from DeepLearning.AI on Coursera. Gain fine-tuning and deep learning skills in the Generative AI Advance Fine-Tuning for LLMs from IBM, or the Deep Learning Specialization from DeepLearning.AI, both on Coursera.