Do we need to build standard networks from scratch? Absolutely not, However doing this for colv net will give us some good hands on python programming and help us to understand the complex conv model. So please give it try.
https://beckernick.github.io/convolutions/
https://beckernick.github.io/convolutions/
botfactory 