pdfcoffee

Chapter 7
The idea of neural embeddings is very similar; that is, entities that occur in similar
contexts tend to be strongly related to each other. Ways in which these contexts
are constructed is usually situation-dependent. We will describe two techniques
here that are foundational and general enough to be applied easily to a variety of
use cases.
Item2Vec
The Item2Vec embedding model was originally proposed by Barkan and
Koenigstein [14] for the collaborative filtering use case, that is, recommending
items to users based on purchases by other users that have similar purchase histories
to this user. It uses items in a web-store as the "words" and the itemset (the sequence
of items purchased by a user over time) as the "sentence" from which the "word
context" is derived.
For example, consider the problem of recommending items to shoppers in
a supermarket. Assume that our supermarket sells 5000 items, so each item
can be represented as a sparse one-hot encoded vector of size 5000. Each user is
represented by their shopping cart, which is a sequence of such vectors. Applying
a context window similar to the one we saw in the Word2Vec section, we can train
a skip-gram model to predict likely item pairs. The learned embedding model maps
the items to a dense low-dimensional space where similar items are close together,
which can be used to make similar item recommendations.
node2vec
The node2vec embedding model was proposed by Grover and Leskovec [15], as
a scalable way to learn features for nodes in a graph. It learns an embedding of the
structure of the graph by executing a large number of fixed length random walks
on the graph. The nodes are the "words" and the random walks are the "sentences"
from which the "word context" is derived in node2vec.
The Something2Vec page [41] provides a comprehensive list of ways in which
researchers have tried to apply the distributional hypothesis to entities other
than words. Hopefully this list will spark ideas for your own "something2vec"
representation.
To illustrate how easy it is to create your own neural embedding, we will generate
a node2vec-like model or, more accurately, a predecessor graph based embedding
called DeepWalk, proposed by Perozzi, et al. [42] for papers presented at the
NeurIPS conference from 1987-2015, by leveraging word co-occurrence relationships
between them.
[ 253 ]