Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Where on Earth Do Users Say They Are?: Geo-Entity Linking for Noisy Multilingual User Input

Tessa Masis, Brendan O'Connor

TL;DR

This work tackles geo-entity linking for noisy multilingual social media input by representing each real-world location as the average of embeddings from associated user-input mentions, enabling selective prediction via a cosine-similarity threshold. The method, named UserGeo, compares against baselines including NameGeo, Carmen 2.0, and GeoLM, using a modified GeoNames database and a large multilingual Twitter-Global dataset. Results show state-of-the-art performance at country- and administrative-level granularity, with city-level performance hindered by ground-truth and input-noise challenges; upper-bound analysis suggests page performance is near feasible limits at broader granularities. The authors discuss ethical considerations, data limitations, and potential future directions, including releasing embeddings, applying the approach to geoparsing and other noisy-domain datasets, and benchmarking against LLM-based systems despite cost considerations. Key formulas include Z_d = \frac{1}{|\{t: y = d\}|} \sum_{t: y = d} e(x_t) and m = \max_{d \in D} c(Z_d, e(i)) with predictions \hat{d}_i = \arg\max_{d \in D} c(Z_d, e(i)) if m \ge t, Null otherwise.

Abstract

Geo-entity linking is the task of linking a location mention to the real-world geographic location. In this paper we explore the challenging task of geo-entity linking for noisy, multilingual social media data. There are few open-source multilingual geo-entity linking tools available and existing ones are often rule-based, which break easily in social media settings, or LLM-based, which are too expensive for large-scale datasets. We present a method which represents real-world locations as averaged embeddings from labeled user-input location names and allows for selective prediction via an interpretable confidence score. We show that our approach improves geo-entity linking on a global and multilingual social media dataset, and discuss progress and problems with evaluating at different geographic granularities.

Where on Earth Do Users Say They Are?: Geo-Entity Linking for Noisy Multilingual User Input

TL;DR

This work tackles geo-entity linking for noisy multilingual social media input by representing each real-world location as the average of embeddings from associated user-input mentions, enabling selective prediction via a cosine-similarity threshold. The method, named UserGeo, compares against baselines including NameGeo, Carmen 2.0, and GeoLM, using a modified GeoNames database and a large multilingual Twitter-Global dataset. Results show state-of-the-art performance at country- and administrative-level granularity, with city-level performance hindered by ground-truth and input-noise challenges; upper-bound analysis suggests page performance is near feasible limits at broader granularities. The authors discuss ethical considerations, data limitations, and potential future directions, including releasing embeddings, applying the approach to geoparsing and other noisy-domain datasets, and benchmarking against LLM-based systems despite cost considerations. Key formulas include Z_d = \frac{1}{|\{t: y = d\}|} \sum_{t: y = d} e(x_t) and m = \max_{d \in D} c(Z_d, e(i)) with predictions \hat{d}_i = \arg\max_{d \in D} c(Z_d, e(i)) if m \ge t, Null otherwise.

Abstract

Geo-entity linking is the task of linking a location mention to the real-world geographic location. In this paper we explore the challenging task of geo-entity linking for noisy, multilingual social media data. There are few open-source multilingual geo-entity linking tools available and existing ones are often rule-based, which break easily in social media settings, or LLM-based, which are too expensive for large-scale datasets. We present a method which represents real-world locations as averaged embeddings from labeled user-input location names and allows for selective prediction via an interpretable confidence score. We show that our approach improves geo-entity linking on a global and multilingual social media dataset, and discuss progress and problems with evaluating at different geographic granularities.
Paper Structure (15 sections, 5 equations, 4 figures, 6 tables)

This paper contains 15 sections, 5 equations, 4 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Task and Data
  4. Geo-entity linking task
  5. Data
  6. Methods
  7. Proposed Method
  8. Baselines
  9. Experiments
  10. Experimental setup
  11. Results
  12. Accuracy Upper Bounds
  13. Conclusion and Future Work
  14. Ethical Considerations
  15. Appendix

Figures (4)

  • Figure 1: Tweet frequency for top 15 countries in the geocoordinate-tagged subset of Twitter-Global.
  • Figure 2: Precision-coverage curves at the country (a) and administrative (b) levels. Red points are Carmen 2.0, purple are NameGeo, and cyan are UserGeo. NameGeo and UserGeo are shown with four different embedding models, where each point represents the precision and coverage at a threshold $t \in \{0, .1, .2, .3, .4, .5, .6, .7, .8, .9\}$. These curves demonstrate how thresholds can be used to tradeoff between precision and coverage with NameGeo and UserGeo.
  • Figure 3: Average accuracy for a given number of mentions in training data (bucketed by $\left \lfloor{\log_2 \text{\#mentions}}\right \rfloor$), for NameGeo and UserGeo with and without adding location name variants. Location entities at all 3 geographic granularities are present in the plot.
  • Figure 4: Precision-coverage curve at the city level. The red point is Carmen 2.0, purple points are NameGeo, and cyan are UserGeo.