Optimization Algorithms ( Gradient Descent and Gradient Ascent )

 

What do you mean by Optimization Algorithms?

Optimization algorithms are algorithms that are designed to find the optimal solution to a problem. The term “optimal” can mean different things depending on the context, but in general it means finding the best solution among a set of possible solutions. Optimization algorithms can be used to solve problems in a wide range of fields, including engineering, economics, and machine learning. Some common types of optimization algorithms include linear programming algorithms, gradient descent algorithms, and evolutionary algorithms.

What do you mean by Optimization Algorithms in Machine Learning?

In the context of machine learning, optimization algorithms are used to find the set of parameters for a model that minimize some measure of error. For example, in supervised learning, we might have a set of labeled training data and we want to find the model parameters that result in the smallest error when we apply the model to the training data.

There are many different optimization algorithms that can be used for this purpose, and the choice of which algorithm to use can depend on the specific characteristics of the problem at hand.

Types of Optimization Algorithms

What is Gradient Descent Algorithm?

Gradient descent is an optimization algorithm used to minimize some function by iteratively moving in the direction of steepest descent as defined by the negative of the gradient. In machine learning, we often use gradient descent to update the parameters of a model in order to minimize some measure of error.

There are several variations of gradient descent, including batch gradient descent, mini-batch gradient descent, and stochastic gradient descent. Batch gradient descent computes the gradient of the cost function with respect to the parameters for the entire training dataset before making a parameter update, while mini-batch gradient descent updates the parameters after computing the gradient for only a small subset of the training data. Stochastic gradient descent performs a parameter update for each training example, and is generally faster than mini-batch gradient descent, but may be less stable.

The general form of the gradient descent algorithm is as follows:

1. Initialize the parameters of the model.
2. Compute the gradient of the cost function with respect to the parameters.
3. Update the parameters in the opposite direction of the gradient.
4. Repeat steps 2 and 3 until the parameters have converged to a local minimum of the cost function.

What is Gradient Ascent Algorithm?

Gradient ascent is an optimization algorithm used to maximize some function by iteratively moving in the direction of the gradient. It is the opposite of gradient descent, which is used to minimize a function.

Like gradient descent, there are several variations of gradient ascent, including batch gradient ascent and stochastic gradient ascent. The general form of the gradient ascent algorithm is similar to gradient descent, but with the direction of the parameter updates reversed:

1. Initialize the parameters of the model.
2. Compute the gradient of the cost function with respect to the parameters.
3. Update the parameters in the direction of the gradient.
4. Repeat steps 2 and 3 until the parameters have converged to a local maximum of the cost function.

Gradient ascent is not as commonly used as gradient descent in machine learning, but it can be useful in certain situations, such as when trying to maximize the likelihood of a model given some data.

Take a Small dataset and solve the gradient descent Algorithm

Certainly. Let’s consider a small dataset with two input features and a continuous target variable. We will use this dataset to train a simple linear regression model using gradient descent.

First, let’s define our dataset:

To train a linear regression model using gradient descent, we need to define the following:

1. The model: In this case, our model will be a simple linear regression model with two coefficients, w1 and w2, and an intercept term, b. The prediction made by our model will be given by the equation: y = w1 * x1 + w2 * x2 + b
2. The cost function: We will use mean squared error as our cost function, which is given by the equation: MSE = (1/n) * sum((y — y_pred)²)
3. The learning rate: The learning rate determines the step size at which the model parameters are updated during each iteration of gradient descent. A larger learning rate can lead to faster convergence, but may also result in the model oscillating around the minimum.

With these defined, we can implement the gradient descent algorithm as follows:

1. Initialize the model parameters w1, w2, and b to some initial values.
2. Compute the predictions y_pred for the current model parameters.
3. Compute the gradient of the cost function with respect to each model parameter.
4. Update the model parameters in the opposite direction of the gradient, using the learning rate as a scaling factor.
5. Repeat steps 2 through 4 until the model parameters have converged to a minimum of the cost function.

Here is some example code in Python that demonstrates how to implement gradient descent for our simple linear regression model:

This code will perform gradient descent for 1000 iterations, updating the model parameters at each iteration to minimize the cost function. The final model parameters should be printed to the console.

Take a Small dataset and solve the Gradient Ascent Algorithm

Certainly. Let’s consider a small dataset with two input features and a continuous target variable. We will use this dataset to maximize the likelihood of a simple linear regression model using gradient ascent.

Lets Use Same DataSet of Previous Example.

To maximize the likelihood of a linear regression model using gradient ascent, we need to define the following:

1. The model: In this case, our model will be a simple linear regression model with two coefficients, w1 and w2, and an intercept term, b. The prediction made by our model will be given by the equation: y = w1 * x1 + w2 * x2 + b
2. The likelihood function: The likelihood function for a linear regression model is given by the equation : L = (1 /sqrt(2pisigma²))^n * exp(-(1/(2*sigma²)) * sum((y — y_pred)²))
3. The learning rate: The learning rate determines the step size at which the model parameters are updated during each iteration of gradient ascent. A larger learning rate can lead to faster convergence, but may also result in the model oscillating around the maximum.

With these defined, we can implement the gradient ascent algorithm as follows:

1. Initialize the model parameters w1, w2, and b to some initial values.
2. Compute the predictions y_pred for the current model parameters.
3. Compute the gradient of the likelihood function with respect to each model parameter.
4. Update the model parameters in the direction of the gradient, using the learning rate as a scaling factor.
5. Repeat steps 2 through 4 until the model parameters have converged to a maximum of the likelihood function.

Here is some example code in Python that demonstrates how to implement gradient ascent for our simple linear regression model:

This code will perform gradient ascent for 1000 iterations, updating the model parameters at each iteration to maximize the likelihood of the model given the data. The final model parameters should be printed to the console.

Conclusion short:

In summary, we discussed optimization algorithms, with a focus on gradient descent and gradient ascent. These algorithms are used to find the optimal solution to a problem by iteratively moving in the direction that either minimizes or maximizes some measure of performance. We also demonstrated how to implement these algorithms in Python using a simple linear regression model as an example.

Understanding Word Embeddings: Mathematical Representations of Meaningful Words

Introduction

In the field of natural language processing, understanding the meaning and context of words is crucial for tasks such as sentiment analysis, language translation, and text generation. One powerful technique for representing words in a way that captures their meaning is through word embeddings.

What are Word Embeddings?

Word embeddings are mathematical representations of words in a high-dimensional space. These embeddings are learned from large amounts of text data and can be used to perform various NLP tasks with great accuracy. The most popular method for learning word embeddings is through the use of neural network models like Word2Vec and GloVe.

The Benefits of Word Embeddings

One of the key benefits of word embeddings is that they allow us to perform mathematical operations on words. For example, we can find the cosine similarity between two words, which tells us how similar the meanings of those words are. This can be incredibly useful for tasks like text classification, where we want to determine the topic of a given piece of text.

Similarly, There are several benefits of using word embeddings in natural language processing and machine learning applications:

1. Improved accuracy: Word embeddings capture the meaning and context of words, which can improve the accuracy of language processing tasks such as sentiment analysis, named entity recognition, and machine translation.

2. Reduced dimensionality: Traditional language processing techniques require large amounts of memory and processing power to represent and manipulate language data. Word embeddings reduce the dimensionality of language data by representing words as dense vectors, which can lead to more efficient and faster processing.

3. Transfer learning: Word embeddings can be pre-trained on large datasets and then used as input to other language processing tasks. This allows for transfer learning, where models can learn from pre-existing knowledge and then apply that knowledge to new tasks.

4. Semantic relationships: Word embeddings capture the semantic relationships between words, such as synonyms, antonyms, and analogies. This can be useful for tasks such as word sense disambiguation, where the meaning of a word must be determined based on context.

5. Multilingual support: Word embeddings can be trained on multilingual data, allowing for language processing tasks across multiple languages. This can be useful for applications such as machine translation or sentiment analysis on social media data from multiple countries.

Example of Word Embedding Applications

Another important aspect of word embeddings is that they can be used to understand the relationships between words. For example, using embeddings, we can find the words that are most similar to a given word, or we can find the analogy between words. For instance, if we know that “king” is to “queen” as “man” is to “woman”, we can find the embedding of “king” — “man” + “woman” will be close to the embedding of “queen”.

There are many applications of word embeddings in natural language processing and machine learning. Here are some examples:

1. Sentiment Analysis: Word embeddings can be used to analyze the sentiment of text data, such as product reviews or social media posts. By representing words as vectors, machine learning models can identify words with positive or negative connotations and use that information to predict the overall sentiment of the text.
2. Named Entity Recognition: Word embeddings can be used to identify named entities, such as people, places, and organizations, in text data. By training a machine learning model on annotated data, the model can learn to recognize patterns in the text and identify named entities more accurately.
3. Machine Translation: Word embeddings can be used to improve the accuracy of machine translation systems. By representing words as vectors, the model can better capture the meaning and context of words in the source language and use that information to generate more accurate translations in the target language.
4. Information Retrieval: Word embeddings can be used to improve the performance of search engines and information retrieval systems. By representing queries and documents as vectors, the model can more accurately match queries with relevant documents, improving the relevance of search results.
5. Chatbots: Word embeddings can be used to improve the performance of chatbots by enabling them to understand the meaning and context of user input. By representing user input and the chatbot's responses as vectors, the model can learn to generate more accurate and relevant responses to user queries.

Conclusion

In conclusion, word embeddings are a powerful tool in the field of natural language processing, and they have been used to achieve state-of-the-art results in various NLP tasks. They allow us to understand the meaning and context of words in a mathematical way, and they can be used to perform various operations on words and understand the relationships between them. As the field of NLP continues to evolve, we can expect to see even more exciting applications of word embeddings in the future.

Impact of Social Media On Postgraduate Students and Young Working Personals

 This Survey was conducted on students and personals from Pakistan only.

INTRODUCTION

This study aimed at finding the trend of Average Screen Time Users spend on social media and how much it affects their overall well being and time management capability. For this purpose, a sample of Postgraduate students and Young Working personals aged between 20 -37 was taken. A questionnaire survey was administered to get the response from them. The questionnaire survey contained both open and close-ended questions. The responses show the statistics of screen time of people.

Screen Time usually refers to the time a person spends using digital screens like monitors, laptops, mobile phones and multimedia etc. Whereas, Social Media is a term used to define applications and sites that allow users to share their thoughts , ideas and information including all through virtual networks and communities. Social media is internet-based and gives users quick electronic communication of content, such as personal information, documents, videos, and photos.

BACKGROUND

Being in the age of technology everybody is accustomed to using digital devices to get their usual work done. Covid-19 has given rise to the use of digital devices and average screen time of users, especially young students and working personnel.

In a reidhealth.org Blog about ‘healthy amount of screen time for adults’, It is mentioned that adults should limit screen time outside of work to less than two hours per day. Any time beyond that which you would typically spend on screens should instead be spent participating in physical activity.

Those spending six hours or more per day watching screens had a higher risk for depression. Screen time more than 8.5–10 hours causes Insomnia and Poor Sleep, Eye Strain and Headaches, Neck, Shoulder and Back Pain, Changes in Cognition etc.

https://www.reidhealth.org › blog › screen-time-for-adults

OBJECTIVES OF THE STUDY

Keeping the gap in assessing the use of social media among young students/professionals in view, this study aims at:

• To find out the screen time and usage of social media among target audience.
• To explore their views about impacts that social media usage and screen time creates on their well-being and everyday time management.

MATERIALS

1. Google Forms : The study is based on a questionnaire survey. We used Google Forms to make questionnaires containing both open and close-ended questions to distribute among the students.
2. Ms Excel : It was used to clean the data and perform multiple regression test
3. Minitab : Gathered responses were analyzed with the help of Minitab Software. Mini-tab was used to perform normality , proportion and mean tests on data where required.

QUESTIONNAIRE

Following are the relevant set of questions we asked from our target audience to conduct our study.

1 — Name

2 — Age

3 — Gender

4 — What is your average screen time per day (including study and work screen time)?

5 — How much free time do you get each day?

6 — How much time do you spend on social media per day?

7 — Average time on social media

8 — Which device do you usually use to access social media sites?

9 — Which social media app consumes most of your time?

10 — Do you think social media sites bring a bad impact on your overall well-being and everyday time management?

11 — How much do you think social media invades your privacy? Rate 1–5.

DATASET

Survey was distributed to 1000 people and 600 responses were received back from users aged between 21–37. Responses data set images are attached below.

Responses show that 300 persons which made up to 50% of the data set are not enrolled in masters whereas the remaining persons are enrolled in different postgrad disciplines.

Dataset also shows that 50% of the responses received were from Female Postgrad Students and Working Personals and 50% were from Male Postgrad Students and Working Personals.

AVERAGE DAILY SCREEN TIME OF USERS

Dataset shows that 46.6% of users have an average screen time of more than 10 hours per day. Whereas 31.7% of users have daily screen time between 5–10 hours and only 21% of users have less than 5 hours of average daily Screen time.

To check the significance of our null hypothesis ‘Screen Time > 10 hours for 46% users’, we performed a one-proportion test. The test shows a p-value > 0.05 which means we can not reject the hypothesis that almost 46% of users have more than 10 hours of average daily screen time. Whereas, the hypothesis that less than 46% of users have Screen time >10 hours will be rejected.

AVERAGE DAILY FREE TIME OF USERS

Dataset shows that 72% users have average free time of less than 5 hours per day. Whereas 25% users have daily free time between 5–10 hours and only 3% users have more than 10 hours of average free time per day.

To check the significance of our null hypothesis ‘Free Time < 5 hours for 70% users’, we performed a one-proportion test. The test shows p-value > 0.05 which means we can not reject the hypothesis that almost 70% of users have less than 5 hours of average daily free time.

AVERAGE TIME USERS SPEND ON SOCIAL MEDIA PER DAY

Users Responses show that 56.7% users spend less than 2 hours on Social Media per day. Whereas 26.7% users daily spend 2–5 hours on Social Media and only 6.7% users spend more than 5 hours on social media per day.

Normality Test

Since in the Normality test the P-Value is much less than 0.05, that indicates that our data is not distributed normally for responses received for Average time spent on social media.

To check the significance of our null hypothesis ‘Time Spent on Social Media > 2 hours for 35% users’, we performed a one-proportion test. The test shows p-value > 0.05 which means we can not reject the hypothesis that almost 35% of users have less than 5 hours of average daily free time.

MOSTLY USED DEVICE TO ACCESS SOCIAL MEDIA

Observing received responses we conclude that more than 85% of users mostly use Mobile phones to access social media.

MOSTLY USED SOCIAL MEDIA APPS

The study shows that the mostly used social media apps are Facebook, Whatsapp, Instagram. More than 90% of users responded that one of these 3 apps consumes the most of the time they spend using social media.

IMPACT OF SOCIAL MEDIA ON WELL BEING

Received responses shows that 56.7% users have experienced the bad impact of social media on their overall well being. Whereas 13.3% users disagree with the statement that ‘Social Media has a bad Impact on Users Overall Well Being’ and 30% users have neutral response.

Normality Test : Since the P-Value is much less than 0.005 that indicates that, our data is not distributed normally for responses received for Effects of social media on Overall Well-being of users.

Mean Test

To check the significance of our null hypothesis ‘Social Media has no Effect on Users overall well being’ vs alternative hypothesis ‘Social media has bad Impact on user overall well-being’, we performed a mean test. The test shows p-value = 0.00 which means our null hypothesis is rejected and we can not reject the hypothesis that the rate of effect of social media on users’ well-being is greater than 3 which means we can agree that Social media has the bad impact of users ‘ overall well being.

ANOVA Table

Using excel we applied Regression analysis on our data where we calculated the dependency of users overall well being on users total screen time, time they spend on social media and their age.

Received Model has R square = 0.898 and standard error of 0.7333 and significant f = 1.6 e-25 and all parameters have p-value of greater than 0.05.

CONCLUSION

The study concluded that more than 46% of users have screen time of more than 10 hours and users who spend more than 2 hours on social media are 35% . And Tests Shows That all users who spend more time on social media have higher risks of having a bad impact on their well- being and time management capabilities.