← AI HoodmanNo. 003 · Embeddings

Location Embeddings

How do you feed a location into a model?

01of 07
Use the arrows, swipe or the arrow keys to turn the slides. Click a slide to read it page by page.
  1. Slide 1 of 7: Two places, 300 meters apart, on opposite sides of a river. River B A 300 m 37.7752, -122.4161 37.7749, -122.4194 How do you feed these locations into a model?
  2. Slide 2 of 7: Not with raw coordinates. To the model, they're nearly the same input. Place Latitude Longitude A 37.7749 -122.4194 B 37.7752 -122.4161 Difference 0.0003 0.0033 The river between them is invisible in two numbers.
  3. Slide 3 of 7: The fix: Give each map cell its own embedding. A B A cell 8928308280fffff → [0.12, -0.48, 0.91, …] B cell 89283082873ffff → [-0.63, 0.27, 0.05, …] Just like words in a vocabulary.
  4. Slide 4 of 7: Three common grids: Same idea, different cell shapes. H3 by Uber Hexagons. Every neighbor is equally far away. S2 by Google Squares. Each one splits neatly into 4. Geohash 9q8yy Rectangles named by short strings. Sizes vary with latitude. For ML features, H3 is usually the best default.
  5. Slide 5 of 7: But most small cells have little data. Add embeddings from three cell sizes. City ~250 km² [0.31, -0.12, …] + Neighborhood ~5 km² [-0.08, 0.44, …] + Block ~0.1 km² [0.67, 0.05, …] = Location embedding [0.90, 0.37, …] A block with little data still gets a good embedding from its neighborhood and city. Every place inside one block (about 350 m across) gets the same embedding.
  6. Slide 6 of 7: Putting it together: How location flows into training. 01 Raw location 37.7749, -122.4194 Map to cell IDs 02 85283083fffffff 872830828ffffff 8928308280fffff Look up embeddings and add 03 [0.31, -0.12, …] 04 + [-0.08, 0.44, …] + [0.67, 0.05, …] Location embedding [0.90, 0.37, …] Concatenate with other features 05 location user item context model The embeddings train with the model.
  7. Slide 7 of 7: Takeaway: Turn locations into cells. Learn embeddings like words.

Keep reading

Location Embeddings · 1 / 7
  1. Slide 1 of 7: Two places, 300 meters apart, on opposite sides of a river. River B A 300 m 37.7752, -122.4161 37.7749, -122.4194 How do you feed these locations into a model?
  2. Slide 2 of 7: Not with raw coordinates. To the model, they're nearly the same input. Place Latitude Longitude A 37.7749 -122.4194 B 37.7752 -122.4161 Difference 0.0003 0.0033 The river between them is invisible in two numbers.
  3. Slide 3 of 7: The fix: Give each map cell its own embedding. A B A cell 8928308280fffff → [0.12, -0.48, 0.91, …] B cell 89283082873ffff → [-0.63, 0.27, 0.05, …] Just like words in a vocabulary.
  4. Slide 4 of 7: Three common grids: Same idea, different cell shapes. H3 by Uber Hexagons. Every neighbor is equally far away. S2 by Google Squares. Each one splits neatly into 4. Geohash 9q8yy Rectangles named by short strings. Sizes vary with latitude. For ML features, H3 is usually the best default.
  5. Slide 5 of 7: But most small cells have little data. Add embeddings from three cell sizes. City ~250 km² [0.31, -0.12, …] + Neighborhood ~5 km² [-0.08, 0.44, …] + Block ~0.1 km² [0.67, 0.05, …] = Location embedding [0.90, 0.37, …] A block with little data still gets a good embedding from its neighborhood and city. Every place inside one block (about 350 m across) gets the same embedding.
  6. Slide 6 of 7: Putting it together: How location flows into training. 01 Raw location 37.7749, -122.4194 Map to cell IDs 02 85283083fffffff 872830828ffffff 8928308280fffff Look up embeddings and add 03 [0.31, -0.12, …] 04 + [-0.08, 0.44, …] + [0.67, 0.05, …] Location embedding [0.90, 0.37, …] Concatenate with other features 05 location user item context model The embeddings train with the model.
  7. Slide 7 of 7: Takeaway: Turn locations into cells. Learn embeddings like words.