LinkedIn, the world’s largest professional network on the internet has launched an open-source framework for Natural Language Processing (NLP) tasks related to ranking, language generation works, and classification called DeText.
DeText supports semantic matching using neural networks to understand the texts. LinkedIn says that DeText can be used for a wide range of tasks such as search and recommendation ranking, text understanding, multi-class classification.
Weiwei Guo, senior engineering manager at LinkedIn says that DeText is designed in such a way to meet all the requirements from different services. It is built by “State-of-the-are” algorithms integrated into an end-to-end model. The variables in the model are updated together by balancing the over effectiveness with high efficiency.
The new framework allows users to use models and embeddings efficiently. It has been applied to LinkedIn across search and recommendation ranking, text classification (query)with appropriate improvements in ranking for users searching jobs and people.
To run DeText, it requires a new environment with necessary dependencies like Python. After the installation, the model can be trained on the sample data set.
Deep learning-based NLP has the ability to understand how the search and recommendations systems understand users. Depending on the use case, DeText allows users to easily swap and enhance natural language processing (NLP) models.
DeText consists of different components which can be tailored using certain templates such as:
- An embedding layer that changes words into a matrix
- Text encoding models
- MLP layer merges wide and deep features
- An interactive layer generates features depending on the embedding layer
In October 2019, Microsoft’s platform launched a model, which has the capability to generate text descriptions for images that have been uploaded to LinkedIn. It is achieved through Microsoft’s Cognitive Services and through the LinkedIn- derived data set. The Recommended Candidates feature of LinkedIn understands the hiring criteria for a particular role and automatically selects appropriate candidates. It’s an AI-driven model. This search engine includes data such as different types of posts user posts, and the searches have given by the users to know about the best-fit jobs and job seekers. LinkedIn’s AI-driven tool automatically finds and freezes unwanted user accounts.
