Chevron Left
Back to Improving Deep Neural Networks: Hyperparameter Tuning, Regularization and Optimization

Learner Reviews & Feedback for Improving Deep Neural Networks: Hyperparameter Tuning, Regularization and Optimization by DeepLearning.AI

4.9
stars
63,175 ratings

About the Course

In the second course of the Deep Learning Specialization, you will open the deep learning black box to understand the processes that drive performance and generate good results systematically. By the end, you will learn the best practices to train and develop test sets and analyze bias/variance for building deep learning applications; be able to use standard neural network techniques such as initialization, L2 and dropout regularization, hyperparameter tuning, batch normalization, and gradient checking; implement and apply a variety of optimization algorithms, such as mini-batch gradient descent, Momentum, RMSprop and Adam, and check for their convergence; and implement a neural network in TensorFlow. The Deep Learning Specialization is our foundational program that will help you understand the capabilities, challenges, and consequences of deep learning and prepare you to participate in the development of leading-edge AI technology. It provides a pathway for you to gain the knowledge and skills to apply machine learning to your work, level up your technical career, and take the definitive step in the world of AI....

Top reviews

AM

Oct 8, 2019

I really enjoyed this course. Many details are given here that are crucial to gain experience and tips on things that looks easy at first sight but are important for a faster ML project implementation

XG

Oct 30, 2017

Thank you Andrew!! I know start to use Tensorflow, however, this tool is not well for a research goal. Maybe, pytorch could be considered in the future!! And let us know how to use pytorch in Windows.

Filter by:

6076 - 6100 of 7,253 Reviews for Improving Deep Neural Networks: Hyperparameter Tuning, Regularization and Optimization

By Ravi R

•

Oct 10, 2020

Very good course for those who wants to start their career in Deep learning. One thing which disappointed me was programming assignments. Some functions and libraries of python and Tensorflow used in assignments are not well explained for students. How can a student understand if he wants to write code from scratch to develop an algorithm. Only small function or some logics are allowed to write by students in assignments which is not enough to build confidence. Many libraries imported are not explained. Theory part of course was excellent.

By Lucas S

•

Oct 14, 2018

Theoretical part amazing, explanations very comprehensive. I think is the best course on deep learning right now. The one thing i would like to see improved is the structural choice for the practical part: this concept of functions that already built in and we come in and fill in the code is ok, and it helps, but i think there should be some kind of path where we can build everything from scratch and you guys could provide the answers as something we can check to see if we got it right...i don't maybe could help, but overall amzing course.

By Markus B

•

Sep 7, 2017

Pro: The course content is well explained and the examples are usually understandable. There are some well explained programming exercises that allow you to get in touch with the "machine room".

Con: Not enough programming exercises to explain all concepts and also the programming exercises sometimes boil down to copy&pasting some code from the instructions. Furthermore, I would expect that this couse with "intermediate" difficulty would allow you to really write code from scratch at some point instead of filling in "Jupiter" notebooks.

By Mijael M

•

Mar 26, 2018

The content is great as usual, but I have two minor complaints: the quality of prof. Ng's handwriting went down, which sometimes makes it more difficult to follow the explanation, or taking notes; the second one is that the error reporting for the grader on the programming exercises is not very user-friendly. I had to try a couple times on weeks 2 and 3 not because my code was wrong at all, but because the grader was finicky. Due to the honor code, I believe I won't be able to post more details here, but check on the forums.

By Aaron L

•

Nov 25, 2017

The content is great, let me say that first and foremost. I feel like some small improvements would be that there were some typos in the notebooks, but not big deal. Helps one learn about debugging and think independently.

About the TensorFlow section of week 3. It was a pretty deep dive into TensorFlow, and I ended up going to the documentation a lot. Maybe some background on the framework would help. I will probably next go through the TensorFlow "getting started" tutorial, to better understand it.

By Eduardo M

•

Oct 29, 2018

The course is pretty good. I just feel notation used by Andrew is quite confusing for people with matrix algebra and matrix calculus. I understand the course is intended for people with different knowledge leveles but generalizing notation with matrix algebra could be save time for students.

There are some little bugs on the last Jupyter implementation with TensorFlow. Nothing too serious but reading the forums I noticed bugs were reported one year ago and nothing has been done to fix them.

By sudhir k

•

Nov 20, 2019

This was a great course with a great assignment. The assignments were moderately hard to complete. I think if students were challenged to improve accuracy of the model by a X%(10%) for extra credit. It this would have triggered independent thinking. I think Students can do it without extra credit also. I think extra credit from Instructor triggers different incentive to complete it. This was done to some extent in the 1st course. I think doing it in this course also would have been ideal.

By Douglas C

•

Oct 28, 2021

The course presented a number of practical techniques for implementing DNNs. The presentation was clear and sufficiently detailed to give a grounding in the techniques. Dr. Ng makes the material accessible while still offering technical details and insights that make the course both interesting and useful. The programming assignment forced me to dig into the documentation for tensorflow, which was at first frustrating, but in then gave me a much better understanding of what was going on.

By Hamidreza C

•

Mar 13, 2019

Many many thanks for putting this great deep learning specialization together!!!

For course 2, it took long long to get to the meat of the course, i.e. hyper parameter tuning, and yet there were no exercises to grasp how we can tune (more than one) hyper parameters through programming exercise. Perhaps we will learn that in course 3. I haven't done it yet.

The first course exercises were more effective.

Other than this comment, everything else for this specialization course looks awesome.

By Amit W

•

Oct 6, 2018

Hyperparameter tuning to improve performance of model is one of the most important part of lifecycle of development of any machine learning model. I would say with confidence now that I have at least got intuition of how different hyper parameters affect the performance of model and how to obtain the optimal value of them. I have got some imagination around hyper-parameters. Thank you Andrew and all team for taking diligent efforts to make this course easy to understand.

By Shiraz R

•

Feb 22, 2018

Course content was complex, yet progressive, helping to grasp key concepts easily. I think the assignment material can be improved. For instance, I got a full grade on the Tensorflow assignment, but my compute_cost function was wrong (hadn't passed the right arguments to the tf cross-entropy cost function). Some of the assignment instructions are also unclear at times.

Overall, this course helps build some invaluable skills for practical machine learning applications.

By Luisa F A S

•

Aug 3, 2022

Theoretical foundation on algorithms and tuning and also insights on these topics are amazing. However, I would have loved to see a more detailed intro to TensorFlow, as W3's assignment is quite challenging for someone who's never worked with the framework before. I know there's an option for taking a course prior to the assignment, but at least in my case it wasn't possible for time constraints. Maybe mentioning that prior TF knowledge is required would also help.

By André M

•

Oct 24, 2019

4* only because the TensorFlow lectures and assignment were too much in too little time. Also from what I see, TF has massively changed syntax to 2.0 so it felt a bit pointless to learn TF1 syntax (which is ***horrible***) at this point. To me it detracted a lot from the learning experience.

The remaining lectures and modules were excellent as usual though. I'd still recommend this highly, and Andrew's insights into what tends to work and why are brilliant as always.

By Roudy E

•

Nov 8, 2020

Another great course, the amount of information per week is right on point (not too packed and not too poor). Also, it was interesting to go behind the scenes and learn what batch normalization and regularization actually does and how it can actually help a neural network perform better. And, to top it off, it gives a brief introduction on TensorFlow and how to use it, although it would have been better if the course thought the material on TF 2.0 instead of 1.0.

By Konstantinos K

•

Aug 6, 2020

The course is great!

It really helps in understanding how the algorithms work, under the hood and the implementation tips

are very helpful! (This is visible in both the Optimization and Batch Normalization algorithms sections)

It is awesome that a programming framework is also introduced in the course, Tensorflow. But to be honest PyTorch could be also introduced, in order to select the framework in which the student could implement the last programming assignment.

By Joshua H

•

May 17, 2020

The content covers a wide variety of useful topics in deep learning. Andrew's explanations are sufficient, as are his use of both examples and analogies. I was only slightly disappointed to see that he has left the derivation of the equations governing back propagation along a batch normalized neural network as an exercise to his audience. The quizzes were sufficiently challenging, and the programming exercises were either informative or insightful, or both.

By dheeraj i

•

Sep 25, 2017

I felt this interesting but bit easier compared to the first course. Please don't provide the parameters of a method directly in the description above. I want to learn how this method can be executed by thinking and understanding the parameters I have to pass to this method. So, I felt the tensorflow assignment little straight forward. But overall a very good course. I need to practice a lot to actually understand and write the code from scratch. Thank you.

By Jian L (

•

Sep 4, 2018

I wish to give 4.5 instead. The only pitfall is the whole video series have a high frequent sound which keep distracting me when try to concentrate to the content.

content vise is very good. Coding practice is very helpful in understand the process. However there is only a basic level. With giving too much help on the background, it's very easy forget afterwards. May be a suggestion more practice would be better.

After all, it's the best course in DL for me.

By John C

•

Nov 6, 2020

The number of mathematical symbols grows quickly, and I started getting a little lost trying to remember which Latin or Greek symbol meant what and in which context. Still, I think that I've learned enough about overfitting (bias), underfitting (variance), regularization, adaptive learning rates, and normalization that I'll at least have the concepts in mind moving forward, even if I didn't memorize the equations and code necessary to do it from scratch.

By Bruce W

•

Jun 23, 2020

This was a good course to deal with some of the inner working of the machine learning and neural network models. It was good to see one of the existing frameworks (TensorFlow); although, I find it to be more difficult to configure than Torch (PyTorch). And it was unclear from the lab whether or not this framework was using GPU acceleration; although, this could probably be determined with a little research and experimentation in the lab environment.

By Steve I

•

Sep 26, 2019

This is a great overview for those wanting their neural networks to run more effectively and efficiently. Lots of ideas to improve your networks. The documentation and description of Tensorflow for the exercises is inadequate to be able to diagnose errors in the "expected" code without expert assistance. When debugging Tensorflow for these exercises, its almost a Trial and Error exercise instead of using first principles taught in the presentations.

By Mats K

•

Jan 29, 2021

The material is very interesting, but a little light on the mathematics, which I personally would enjoy seeing more of. I would like to see more elaborations and proofs, but they can be optional. A little too much hand-holding in the assignments. Learning to find relevant information is part of the training and as a programmer I find that the assignments consists of a lot of cutting and pasting snippets of code from the instructions in the notebooks.

By Robert S

•

Jun 15, 2021

A great follow-up to the first course in the specialization. Answers a lot of questions that might have occurred to you while taking the first course.

Compared to university courses I have taken, this one feels to me as if it is taught at about a second year level. As such, keep in mind that to fully absorb the material you will need to do more than just follow along, you will need to practice on your own by finding or creating your own problems.

By Marcello

•

Nov 29, 2023

The content of the course is very good and well explained, only this would be 5 stars. I gave only 4 stars because all the lessons are video lessons. Even though there is a transcript, the text is not well formatted and it does not underline core concepts, math formulas, code snippets, etc. So if you prefer a text lesson, it is hard to read, and you still rely on the video because the teacher writes many explanations on the whiteboard.

By Calvin K

•

Mar 4, 2018

Love the orthogonalization part and the explanation on why training deep neural networks is possible (local minimum is rare in hyperspace; for the most part there are saddle points). Tho I was hoping there would be some advice on how to design a neural network. Overall I think it's a bit too easy for those who have already known deep learning or taken Ng's Machine Learning course. It'd be great if the homework would be more challenging.