DevDays Vilnius 2019


May 21-23, 2019


Wilder Rodrigues

The Netherlands


Wilder is a Machine Learning Engineer, a City.AI Ambassador, an IBM Watson AI XPRIZE Contestant and Committer and PMC member of the Apache Software Foundation. He was a guest attendee at AI for Good Global Summit at the United Nations.


SineReLU: An Alternative to the ReLU Activation Function

During my journey on the way to become a Machine Learning Engineer after 20 years working with traditional Software Development, I came up with an idea to reduce the impact caused by the Dying Rectified Linear Unit (a.k.a Dying ReLU) issue on Neural Networks and improve accuracy, the new Rectified Linear Unit with Sinusoidal Curve (a.k.a. SineReLU) does not flattens out negative weights to zero, but instead, it combines them with a two extra functions and an extra hyper-parameter, then enabling differentiability at any point of the function.

This approach is also superior to the Leaky-ReLU implementation due to the oscillations that it adds to negative weights. So, instead of linearly moving down the bottom-left quadrant based on the Leaky-ReLU alpha parameter, it uses the difference between the sine and cosine functions to create a wave. The amplitude of this wave depends on the hyper-parameter ε.

With this approach, problems like vanishing or exploring gradients, introduced by sigmoid and tanh functions, and the dying ReLU, are mitigated.

This talk will cover the Path; the Inspiration; and the Accomplishments after one hard year of studies.


Machine Learning for Software Engineers

In this workshop, I would like to explore the realm of Artificial Intelligence and deep dive into one of its sub areas: Deep Learning. We will go through the theory behind Neural Networks, exploring shallow and deep nets. We will also engage at the world of Convolutional Neural Networks, how they work and why they are the state of the art implementation for object recognition to date. Last, but not least, we will approach practical aspects of those subjects, training models using Keras and TensorFlow for object recognition and sentiment analysis.