LinkSage: GNN-based Pinterest Off-site Content material Understanding | by Pinterest Engineering | Pinterest Engineering Weblog | Mar, 2024

Pinterest Engineering
Pinterest Engineering Blog

Adopted by Pinterest a number of consumer going through surfaces, Advertisements, and Board.

Jianjin Dong | Employees Machine Studying Engineer, Content material High quality; Michal Giemza| Machine Studying Engineer, Content material High quality; Qinglong Zeng | Senior Engineering Supervisor, Content material High quality; Andrey Gusev | Director, Content material High quality; Yangyi Lu | Machine Studying Engineer, Residence Feed; Han Solar | Employees Machine Studying Engineer, Advertisements Conversion Modeling; William Zhao | Software program Engineer, Boards Basis, Jay Ma | Machine Studying Engineer, Advertisements Light-weight Rating

LinkSage: Graph Neural Community primarily based mannequin for Pinterest off-site content material semantic embeddings

Pinterest is the visible inspiration platform the place Pinners come to look, save, and store one of the best concepts on the earth for all of life’s moments. A lot of the Pins are linked to off-site content material to offer Pinners with inspiration and actionability. It’s crucial to know off-site content material (photographs, textual content, construction), as a result of understanding their semantics is a vital consider assessing how secure (e.g. community guidelines), purposeful, related, and actionable (e.g. Advertisements and Buying) the off-site content material is. Extra importantly, Pinterest can have a greater understanding of Pinterest customers via customers’ click on via occasions. Each of the above can enhance total engagement and monetization of Pinterest contents. To attain it, we developed LinkSage, which is a Graph Neural Network (GNN) primarily based mannequin that learns the semantics of touchdown web page contents.

Determine 1: Off-site content material understanding and its functions

To make full use of Pinterest off-site content material to enhance Pinners’ engagement and purchasing expertise, we established the next objectives:

  • Unified semantics embedding: Present a unified semantic embedding of all of the Pinterest off-site content material. All of the touchdown pages associated to downstream fashions can leverage LinkSage embedding as a key enter.
  • Graph primarily based mannequin: Leverage the Pinner’s curation knowledge to construct a heterogeneous graph that helps several types of entities. The GNN can study from close by touchdown pages/nodes to enhance accuracy.
  • XSage ecosystem: Make the LinkSage embedding appropriate with all of the XSage embedding area.
  • Multi-dimensional illustration: Present a multi-dimensional illustration of the LinkSage embedding so shoppers would have a flexibility of selecting efficiency vs price.
  • Impression on engagement and monetization: Enhance each engagement (e.g. lengthy clicks) and purchasing/advertisements expertise (e.g. CVR) via a greater understanding of Pinterest content material and Pinner profile.

On this weblog, we contact on:

  • Technical design
  • Key improvements
  • Offline outcomes
  • On-line outcomes


Most Pins are related to a touchdown web page. We deal with “(Pin, touchdown web page):” as a optimistic pair if the Pin and its related touchdown web page have related semantics, and we leverage Pinterest Cohesion ML sign to guage the semantic similarity between a Pin and its touchdown web page. We additionally label a “(Pin, touchdown web page)” pair as optimistic if the Cohesion rating is larger than a sure threshold.

For unfavourable pairs, we embody each batch and random negatives. Within the case of batch negatives, we use Pins which can be paired with different touchdown pages in the identical batch. Within the case of random negatives, we use random Pins throughout Pinterest, which will not be seen within the optimistic pairs. This helps to coach a mannequin generic to new contents.

Within the latter model of LinkSage, we’d leverage Pinner onsite engagement knowledge and Pinner off-site conversion knowledge to complement our coaching targets.


We leverage Pinner’s curated knowledge to construct the graph. Graph compilation and random stroll is carried out utilizing Pinterest XPixie, which helps heterogeneous graphs of several types of entities. In our case, a heterogeneous graph is constructed through the use of “(Pin, touchdown web page)” pairs. We leverage Pinterest Cohesion ML sign to filter out non-cohesive pairs, much like coaching knowledge technology. Thus, all of the “(Pin, touchdown web page)” pairs used within the graph have related semantics. To extend the graph density, we leverage Pinterest Neardup ML sign to cluster related Pin photographs to a picture cluster. Graph pruning is completed on each graph nodes and edges to make sure graph connections aren’t skewed on sure fashionable touchdown pages or Pins. On this graph, touchdown pages with related semantics are related with Pins which can be cohesive to the touchdown pages.

After the random stroll, for every touchdown web page, we get a listing of its neighbor touchdown pages and their go to counts. Random stroll is configurable primarily based on the node entity sort.

In our latter model, we totally make the most of the heterogeneous graph function of XPixie that we add extra several types of entities, together with Pinterest Boards and hyperlink clusters.


There are three sorts of options: self touchdown web page options, neighbor touchdown web page options, and graph construction options.

For each self touchdown pages and neighbor touchdown pages, we use two sorts of content material options: touchdown web page textual content embedding (which summarize the semantics of title, description, major physique textual content), and visible embedding of every crawled picture. We carry out a weighted aggregation of all of the crawled photographs by their dimension to cut back the calculation whereas conserving the principle crawled photographs’ data of the touchdown pages.

For graph construction options, we use graph node go to counts and self diploma to signify the topological construction of the graph. Graph node go to counts signify the significance of the neighbor touchdown pages, whereas self diploma represents the recognition of the self touchdown web page within the graph.


The mannequin leverages a Transformer encoder to study the cross consideration of self touchdown web page options, neighbor touchdown web page options, and graph construction options.

The textual content and crawled picture options are cut up within the transformer encoder to let the mannequin study the cross consideration of them. The neighbors are reverse sorted by the visited counts so the highest neighbors could be extra essential than the underside ones. Along with place embeddings, our mannequin can study the significance of various neighbors. The variety of neighbors is chosen to stability computational price and mannequin efficiency.

Within the latter model, we cut up crawled photographs and deal with them as separate tokens within the transformer encoder, which would offer the mannequin with extra correct visible data of the touchdown pages.

Determine 2: Mannequin schematics of LinkSage

Multi-dimensional illustration

Downstream groups would eat completely different dims of embedding primarily based on their desire between efficiency and computational price. As a substitute of coaching 5 completely different fashions individually, we leverage the analysis of Matryoshka Representation Learning to offer 5 dims of LinkSage in place by coaching one mannequin. Shorter dims would seize a rough illustration of the touchdown pages, and extra particulars are embedded within the longer ones.

Determine 3: Schematic of the loss perform of multi-dimensional illustration

Compatibility of XSage

The compatibility of the embedding area between LinkSage and XSage (e.g. PinSage) would make the downstream utilization simpler. Downstream groups may even use proximity in embedding area to match the similarity of various contents throughout Pinterest, like Pins and their touchdown pages. To attain this, we leverage PinSage because the illustration of the Pins in our coaching goal.

Incremental serving

Pinterest has tens of billions of touchdown pages related to Pins. To serve all of the touchdown pages, it will take an enormous quantity of computational price and time. To unravel it, we apply incremental serving that we solely run serving of each day crawled touchdown pages. After each day inference, we merge at present’s inference outcomes with the earlier ones. Our incremental serving not solely saves a considerable amount of pointless computations but additionally retains the identical accuracy and protection as the complete corpus serving.


Recall is probably the most generally used metric for rating duties. When given a question touchdown web page, it evaluates how good the mannequin can retrieve the optimistic candidate Pins amongst all of the negatives. Greater recall means a greater mannequin.

Desk 1: Recall of LinkSage throughout completely different serving dimensions.

From the desk above, through the use of 256 dims of LinkSage, the chance of fetching the optimistic candidate Pins is 72.9% from the highest 100 rating outcomes. By utilizing 64 dims of it, it saves 75% of the price and the efficiency solely drops by 8.3%.

Rating distribution

Rating distribution is plotted to indicate the distribution of cosine similarity scores between (1) question touchdown web page and optimistic candidate Pins, and (2) question touchdown web page and unfavourable candidate Pins

Determine 4: Rating distribution of LinkSage optimistic and unfavourable pairs

From the histogram beneath, nearly all of the unfavourable pairs have a rating < 0.25 and the imply worth is near 0. Alternatively, greater than 50% of the optimistic pairs have a rating > 0.25.


Kurtosis is used to guage the flexibility of the embedding to differentiate between completely different touchdown pages.

For embedding pairwise cosine similarity distribution, a smaller kurtosis is preferable as a result of a wide-spread distribution tends to have higher “decision” to differentiate between queries (aka touchdown pages) of various relevance.

The Kurtosis of LinkSage is 1.66.

Determine 5: Kurtosis evaluation of LinkSage


Given a touchdown web page, the highest ok ranked Pins will be fetched and visualized to test whether or not the touchdown web page and Pins have related semantics.

We launched A/B experiments in a number of consumer going through surfaces, Advertisements, and Boards.

Consumer going through surfaces

A number of consumer going through floor groups have adopted LinkSage into their rating mannequin to enhance the understanding of each candidate Pins and consumer profiles (via Consumer Sequence).

On Pinterest, “repin, lengthy click on, engaged classes” are the important thing indicators of optimistic consumer engagement. Alternatively, “disguise” is the important thing indicator of unfavourable consumer engagements on the platform. We noticed vital positive factors on all of the metrics.

Desk 2: LinkSage positive factors on consumer going through floor rating mannequin: from candidate Pins (prime) and consumer sequence (backside)


Advertisements has adopted LinkSage into their Conversion rating mannequin and Engagement rating mannequin.

On Pinterest Advertisements, conversion rate per impression (iCVR), conversion quantity, lengthy click through rate (GCTR30), and cost per click (CPC) are the important thing indicators of consumer conversion and engagement. We noticed vital positive factors on all of the metrics.

Desk 3: Mixed positive factors with LinkSage on Advertisements conversion (prime) and engagement rating mannequin (backside)


LinkSage use within the Boarding rating mannequin (or known as Board Picker) has improved the understanding of exterior hyperlinks. Important positive factors have been noticed:

Desk 4: LinkSage positive factors on Board rating mannequin

We developed LinkSage, a Graph Neural Community-based mannequin, which is educated utilizing a heterogeneous graph that helps several types of entities (e.g. Pins and touchdown pages). It leverages Pinner curated knowledge to construct the graph and coaching targets. It makes use of Pinterest ML indicators (e.g. Cohesion and Neardup) to prune the graph/goal and enhance the graph density. It incorporates Pinterest ML indicators (e.g. PinSage) into coaching to make its embedding area appropriate with XSage. It applies leading edge analysis of Matryoshka Illustration Studying to offer multi-dimensional illustration. It applies incremental serving to serve all of the Pinterest touchdown pages corpus with a low computational price and time.

We comprehensively evaluated the standard of LinkSage embeddings with offline metrics and on-line A/B experiments on floor rating fashions. We now have seen substantial on-line positive factors throughout a number of consumer going through surfaces, Advertisements, and Board, which covers all the important thing surfaces of Pinterest.

This work fills the clean of all of the Pinterest off-site content material understanding. It supercharges the backend of all the opposite touchdown pages indicators’ improvement (e.g. Hyperlink High quality). It enriches Pinterest’s understanding of Pins, Pinterest customers, and powers the way forward for advertisements and purchasing at Pinterest.

In case you are involved in any such work we do, be a part of Pinterest!

Within the latter model of LinkSage, we’d enhance the graph technology, function engineering, and mannequin structure. We’d incorporate extra Pinterest entities within the heterogeneous graph to extend graph density. We’d cut up crawled photographs as separate enter to the transformer’s encoder to cut back data dilution. We’d discover FastTransformer to avoid wasting computation time and price.

Along with batch serving, we’d set up a Close to Actual Time (NRT) infrastructure to serve LinkSage in actual time. Pinterest has leveraged Apache Flink for NRT serving; for instance, NRT Neardup efficiently reduces the latency to sub-seconds as a substitute of hours. We’d set up an analogous streaming pipeline to extend the protection of contemporary contents with out compromising accuracy.

Contributors to LinkSage improvement and adoption:

  • ATG (GraphSage framework)
  • Search Infrastructure (XPixie)
  • Residence Feed
  • Advertisements Conversion
  • Content material Curation
  • Notification
  • Search
  • Associated Pins
  • Advertisements Sign
  • Advertisements Engagement
  • Advertisements Relevance

To study extra about engineering at Pinterest, take a look at the remainder of our Engineering Weblog and go to our Pinterest Labs web site. To discover and apply to open roles, go to our Careers web page.