What Are Neural Network Parameters?

Network parameters are the internal configurations of a neural network that determine how the neural network will manipulate and make predictions from data. Variables that determine the accuracy of neural networks are hyperparameters, which are external configurations. You set hyperparameters during the creation of the artificial neural network. Determining the optimum hyperparameters is essential to the machine learning process and can help you create a more accurate neural network. You can use regularization and hyperparameter tuning techniques to help you find those optimum values. 

Learn more about neural network parameters, including the difference between parameters and hyperparameters and why each matters to the final performance of your neural network model. 

specialization

Deep Learning

Become a Machine Learning expert. Master the fundamentals of deep learning and break into AI. Recently updated with cutting-edge techniques!

4.9

(135,738 ratings)

911,099 already enrolled

Intermediate level

Learn More

Average time: 3 month(s)

Learn at your own pace

Skills you'll build:

Recurrent Neural Network, Tensorflow, Convolutional Neural Network, Artificial Neural Network, Transformers, Backpropagation, Python Programming, Deep Learning, Neural Network Architecture, Facial Recognition System, Object Detection and Segmentation, hyperparameter tuning, Mathematical Optimization, Decision-Making, Machine Learning, Inductive Transfer, Multi-Task Learning, Gated Recurrent Unit (GRU), Natural Language Processing, Long Short Term Memory (LSTM), Attention Models

What is a neural network?

A neural network, sometimes called an artificial neural network (ANN), falls under the umbrella of artificial intelligence. It is a type of machine learning that mimics how humans think. These models have layers of nodes hidden between input and output layers. Each node in each layer connects to nodes in the previous and subsequent layers, allowing the network to pass information through as a web of nodes that interacts with the data at each point. As a neural network grows to consist of more than three layers, it becomes known as a deep learning network. Adding these additional layers allows the AI model to manipulate data in increasingly complex ways. 

What are parameters and hyperparameters in neural networks?

Parameters are the internal variables that a machine learning model configures as it moves through the training process. The configuration of your parameters has an impact on your output. For example, one type of neural network parameter is weight, which is the numerical value that measures the strength of the connections between nodes. The weight influences how the model manipulates or changes the data as it passes through the node. Depending on how many nodes and layers are within your neural network, a slight weight change can make an exponential difference in your output. 

While parameters explain the internal configuration of an AI model, hyperparameters are the external variables. These can include the number of layers and nodes within a neural network and can help determine the value of features like learning rate and model architecture. 

Parameters in neural networks: Weights and biases

Neural network parameters are defined by the concepts of weight and bias, two values that determine how the model will interact with data at each node. If a neural network mimics how a human brain works, and the nodes in each hidden layer are analogous to neurons, then weights are comparable to the synapses between neurons that fire to send data through the biological network. Biases are similar to biases in the human brain in that they provide a constant that affects how all of the nodes will interact with data. The bias will ultimately shape every conclusion the AI model makes. 

For example, if you want to decide where to eat, you will likely consider a number of variables—some very important and others less so— before making your decision. You might consider what time of day it is, what kind of meal you want, what type of food you’re craving, how hungry you are, and how much money you have. You might have a food allergy or a dietary preference that determines which restaurants you consider in the first place. If you place high importance on sticking to your budget, that will play a more prominent role in determining where you eat than a factor you prioritize as less important, such as the type of food you are actually hungry for, for instance.

In a neural network, weights and biases are the mechanisms for similarly weighing decisions. A strong connection between nodes will weigh that data as more important, while weaker connections signal that data is less critical to the output. As you train your neural network, the model will fine-tune these parameters to help you get a more accurate output based on feedback on the errors the model made in each attempt. 

Hyperparameters in neural networks

The model automatically sets neural network parameters during its training process by adjusting weights and biases, whereas hyperparameters are the variables that you, as a data scientist for example, configure when you create the model. Optimizing your hyperparameters can help you achieve a more accurate output from your AI model. Finding the optimum settings can take time and may require experimenting while using a process called hyperparameter tuning. 

Hyperparameters include: 

• Number of hidden layers and nodes within layers: The number of hidden layers in a neural network and the number of nodes in each layer influences what kind of problems your network can solve and how complicated the network’s analysis of your data can be. 

• Learning rate: The rate by which a neural network learns refers to how much the model changes its weights during each iteration of training. Adjusting its weights for accuracy is a balance that requires finding the optimal learning rate. 

• Convergence rate: When a model reaches optimum weights and can predict outputs accurately, the model is “converged.” The time it takes for the neural network to reach convergence is the convergence rate. 

• Epochs: One epoch has been completed once the entire training data set has passed through the neural network. You will determine how many epochs your model will complete (how many times it will work through the training materials) to create the most accurate result. The correct number of epochs helps you avoid common problems such as underfitting and overfitting—when the model becomes too general or too specific to predict outputs on unfamiliar training data accurately.

• Activation function: A neural network’s activation function is a mechanism that tells the model which neurons to activate based on their relevance to the problem it’s solving. The activation function is important because it allows you to build models that work with linear and nonlinear data or data containing dynamic relationships between variables. 

Optimizing and tuning neural network parameters

Optimizing or tuning your neural network patterns to find the correct balance can allow you to create a neural network that accurately predicts the correct output to train data and new information it’s never seen before. Two techniques you may use are regularization and hyperparameter tuning. 

• Regularization: Regularization is a technique that helps you create a model that can generalize to new information. You train a model with a data set, but your goal is to create a model that can accurately generalize what it learned in training and apply those principles to new data. You can use regularization techniques like lasso regression, ridge regression, and elastic net regularization to help you avoid problems like underfitting and overfitting. 

• Hyperparameter tuning: Hyperparameter tuning is the process of determining the optimal hyperparameters under which your neural network returns accurate information. For example, you can influence how your neural network updates its weights with factors such as weight initialization (how you select the values for your starting weights) and learning algorithms, such as backpropagation, which explain how the model will get feedback to adjust its weights. You can tune your hyperparameters manually or use techniques like Bayesian optimization, grid search, and random search to automate hyperparameter tuning. 

Continue exploring neural network parameters on Coursera.

Neural network parameters are the configurations the model sets during training to determine how a neural network model will interact with data. Hyperparameter tuning allows you to choose the optimum variables within your control. If you want to learn more about neural network parameters or hyper-tuning, you can learn the skills you need on Coursera. For example, consider the Deep Learning Specialization foundational program offered by DeepLearning.AI or the Machine Learning Specialization offered by DeepLearning.AI in partnership with Stanford Online. Both of these Specializations can help you learn more about working with neural networks and machine learning. 

specialization

Deep Learning

Become a Machine Learning expert. Master the fundamentals of deep learning and break into AI. Recently updated with cutting-edge techniques!

4.9

(135,738 ratings)

911,099 already enrolled

Intermediate level

Learn More

Average time: 3 month(s)

Learn at your own pace

Skills you'll build:

Recurrent Neural Network, Tensorflow, Convolutional Neural Network, Artificial Neural Network, Transformers, Backpropagation, Python Programming, Deep Learning, Neural Network Architecture, Facial Recognition System, Object Detection and Segmentation, hyperparameter tuning, Mathematical Optimization, Decision-Making, Machine Learning, Inductive Transfer, Multi-Task Learning, Gated Recurrent Unit (GRU), Natural Language Processing, Long Short Term Memory (LSTM), Attention Models

specialization

Machine Learning

#BreakIntoAI with Machine Learning Specialization. Master fundamental AI concepts and develop practical machine learning skills in the beginner-friendly, 3-course program by AI visionary Andrew Ng

4.9

(32,112 ratings)

598,672 already enrolled

Beginner level

Learn More

Average time: 2 month(s)

Learn at your own pace

Skills you'll build:

Logistic Regression, Artificial Neural Network, Linear Regression, Decision Trees, Recommender Systems, Tensorflow, Advice for Model Development, Xgboost, Tree Ensembles, Regularization to Avoid Overfitting, Logistic Regression for Classification, Gradient Descent, Supervised Learning, Anomaly Detection, Unsupervised Learning, Reinforcement Learning, Collaborative Filtering